Here are all the threads write the log file. I want to know if the below script is fine, if it needs any modifications, and whether I am utilising threads properly to execute this code quickly.
open(LOG,"+>","log.txt")or die "cant open the file";
my $file_total= scalar @listOfFiles;
my $div = $file_total/$thread_count;
$div = ceil($div);
my $start = 0;
my $end = $div;
for ($count = 1; $count <=$thread_count ; $count++)
{
my $thread = threads->new(\®ister, $start, $end );
push(@threads,$thread);
$start = $end;
$end = $end + $div;
if ($end > $file_total)
{
$end = $file_total;
}
}
foreach (@threads)
{
$_->join;
}
close(LOG);
sub register
{
my $lstart = shift;
my $lend = shift;
for (my $index = $lstart; $index < $lend; ++$index)
{
print LOG $listOfFiles[$index];
process......
..........
.......
}
}
process...
placeholder: I have a huge number of documents, say, 1 million. I am collecting all the 1 million filenames, storing them in the listOfFiles
array, and dividing the array index based on the number of threads the user wants. I am passing every thread and the start and end indices to fetch the document name, process the document, and store the status of the output to a log file.