4
\$\begingroup\$

I'd like to optimize my code and get a better of understanding of how I can perform the task I am doing better. I've only used Perl threads about 3 or 4 times now.

The purpose of my script / code block is to scrape the Pastebin API for every new public paste that appears at regular intervals using a crontab. Then search inside those new pastes for user defined regular expressions such as "google.com." The paste is then saved to a sqlite DB for later viewing and a notification is sent to a listening nodejs bot.

That being said here is my code block I wish to have improved upon:

sub threadCheckKey {
    my ($url, $key, @regexs) = @_;
    my $fullURL = $url.$key;
    my @flaggedRegex = ();
    my $date = strftime "%D", localtime;
    my @data = ();

    my $thread = threads->create(sub {
        my $dbConnection = openDB();
        open(GET_DATA, "-|", "curl -s " . $fullURL . " -k") or die("$!");
        open(WRITE_FILE, ">", $key . ".txt") or die("$!");
        while(my $line = <GET_DATA>) {
            print WRITE_FILE $line;
            foreach my $regex(@regexs) {
                if($line =~ m/$regex/) {
                    if(!(grep(/$regex/, @flaggedRegex))) {
                        push(@flaggedRegex, $regex);
                    }
                }
            }
        }
        close(WRITE_FILE);
        close(GET_DATA);

        my $flaggedSize = @flaggedRegex;
        if($flaggedSize == 0) {
            #Uncomment below for debugging.
            #print "No user defined regex found\n";
            return;
        }

        open(READ_FILE, "<", $key . ".txt") or die("$!");
        while(my $line = <READ_FILE>) {
            push(@data, $line);
        }
        close(READ_FILE);

        my $inputData = join("\r", @data);
        my $flaggedReg = join(" | ", @flaggedRegex);
        my $updateRow = qq(UPDATE $tables[0] set data = ?, date = ?, regex = ? where pastekey = ?);
        my $updateRowPrepare = $dbConnection->prepare($updateRow);
        my $executeRowUpdate = $updateRowPrepare->execute($inputData, $date, $flaggedReg, $key);

        if($executeRowUpdate < 0) {
            print $DBI::errstr;
        }

        my $msg = qq(**Alert:** New paste found! The paste can be viewed via https://pastebin.com/$key - Regex Trigger: $flaggedReg);

       open(SEND_ALERT, "-|", "curl https://localhost:8051/bots/bot_service_id/alert -k --data '{\"text\":\"$msg\"}' -H \"Content-Type: application/json\" -X POST") or die("$!");
        close(SEND_ALERT);

        $dbConnection->disconnect();
    });

    if($thread->is_running()) {
        sleep(2);
    }

    if($thread->is_joinable()) {
        $thread->join();
    }

    unlink $key . ".txt";
    return;
}
\$\endgroup\$
  • \$\begingroup\$ Any specific reason you don't use scalars for file operations? \$\endgroup\$ – yuri Mar 14 at 16:14
  • \$\begingroup\$ @yuri No specific reason. I think it's just I was never taught best practices for Perl. I am going by old programming methods I self taught. The code above wasn't even how I originally coded it. I had to do some research after others made comments about insecure methods I was performing for the SQL calls and bad file handles. \$\endgroup\$ – falconspy Mar 14 at 18:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.