I'd like to optimize my code and get a better of understanding of how I can perform the task I am doing better. I've only used Perl threads about 3 or 4 times now.

The purpose of my script / code block is to scrape the Pastebin API for every new public paste that appears at regular intervals using a crontab. Then search inside those new pastes for user defined regular expressions such as "google.com." The paste is then saved to a sqlite DB for later viewing and a notification is sent to a listening nodejs bot.

That being said here is my code block I wish to have improved upon:

sub threadCheckKey {
    my ($url, $key, @regexs) = @_;
    my $fullURL = $url.$key;
    my @flaggedRegex = ();
    my $date = strftime "%D", localtime;
    my @data = ();

    my $thread = threads->create(sub {
        my $dbConnection = openDB();
        open(GET_DATA, "-|", "curl -s " . $fullURL . " -k") or die("$!");
        open(WRITE_FILE, ">", $key . ".txt") or die("$!");
        while(my $line = <GET_DATA>) {
            print WRITE_FILE $line;
            foreach my $regex(@regexs) {
                if($line =~ m/$regex/) {
                    if(!(grep(/$regex/, @flaggedRegex))) {
                        push(@flaggedRegex, $regex);

        my $flaggedSize = @flaggedRegex;
        if($flaggedSize == 0) {
            #Uncomment below for debugging.
            #print "No user defined regex found\n";

        open(READ_FILE, "<", $key . ".txt") or die("$!");
        while(my $line = <READ_FILE>) {
            push(@data, $line);

        my $inputData = join("\r", @data);
        my $flaggedReg = join(" | ", @flaggedRegex);
        my $updateRow = qq(UPDATE $tables[0] set data = ?, date = ?, regex = ? where pastekey = ?);
        my $updateRowPrepare = $dbConnection->prepare($updateRow);
        my $executeRowUpdate = $updateRowPrepare->execute($inputData, $date, $flaggedReg, $key);

        if($executeRowUpdate < 0) {
            print $DBI::errstr;

        my $msg = qq(**Alert:** New paste found! The paste can be viewed via https://pastebin.com/$key - Regex Trigger: $flaggedReg);

       open(SEND_ALERT, "-|", "curl https://localhost:8051/bots/bot_service_id/alert -k --data '{\"text\":\"$msg\"}' -H \"Content-Type: application/json\" -X POST") or die("$!");


    if($thread->is_running()) {

    if($thread->is_joinable()) {

    unlink $key . ".txt";
  • \$\begingroup\$ Any specific reason you don't use scalars for file operations? \$\endgroup\$ – yuri Mar 14 '19 at 16:14
  • \$\begingroup\$ @yuri No specific reason. I think it's just I was never taught best practices for Perl. I am going by old programming methods I self taught. The code above wasn't even how I originally coded it. I had to do some research after others made comments about insecure methods I was performing for the SQL calls and bad file handles. \$\endgroup\$ – falconspy Mar 14 '19 at 18:41
  • \$\begingroup\$ To rephrase yuri's comment: You should avoid using global variables as file handles, an instead of use lexical variables. It's a simple drop-in replacement (open(my $FH, ...), <$FH>, etc) \$\endgroup\$ – ikegami Nov 6 '19 at 9:04
  • \$\begingroup\$ if($thread->is_running()) { sleep(2); } if($thread->is_joinable()) { $thread->join(); } should simply be $thread->join; though I don't see the point of creating a thread then simply waiting for it to end. \$\endgroup\$ – ikegami Nov 6 '19 at 9:19
  • \$\begingroup\$ My main concerns are the improper code injections. Which begs the question of why shell out at all. I'll come back to that. /// The secondary concern would be the completely useless thread. There's no point to start a thread only to wait for it to finish. Furthermore, repeated thread creation is discouraged for performance reasons. It's generally recommended to reuse a pool of worker threads, as shown here. (I believe modules exist that can reduce this.) \$\endgroup\$ – ikegami Nov 6 '19 at 9:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.