Optimizing Perl script which looks for world-writable files

Question

A script which scans a local filesystem and writes to a file a list of world-writable files found on the filesystem.

I ran it on a server with a couple terabytes of storage; not all of it used. In fact, only 195GB have been used. I'm not sure how many files there are, though.

The script takes 23 minutes to run. Is there something I can do to make it run faster?

#!/usr/bin/perl

use warnings;
use strict;
use Fcntl ':mode';
use File::Find;
no warnings 'File::Find';
no warnings 'uninitialized';

my $dir = "/var/log/tivoli/";
my $mtab = "/etc/mtab";
my $permFile = "world_writable_w_files.txt";
my $tmpFile = "world_writable_files.tmp";
my $exclude = "/usr/local/etc/world_writable_excludes.txt";
my $root = "/";
my (%excludes, %devNums);
my ($regExcld, $errHeader);

# Create an array of the file stats for "/"
my @rootStats = stat($root);

# Compile a list of mountpoints that need to be scanned
my @mounts;

open MT, "<${mtab}" or die "Cannot open ${mtab}, $!";

# We only want the local mountpoints
while (<MT>) {
  if ($_ =~ /ext[34]/) {
    my @line = split;
    push(@mounts, $line[1]);
  }
}

close MT;

# Build a hash of each mountpoint's device number for future comparison
foreach (@mounts) {
  my @stats = stat($_);
  $devNums{$stats[0]} = $_;
}

# Build a hash from /usr/local/etc/world_writables_excludes.txt
if ((! -e $exclude) || (-z $exclude)) {
  $errHeader = <<HEADER;
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!                                                  !!
!! /usr/local/etc/world_writable_excludes.txt is    !!
!! is missing or empty. This report includes        !!
!! every world-writable file including those which  !!
!! are expected and should be excluded.             !!
!!                                                  !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


HEADER

} else {
  open XCLD, "<${exclude}" or die "Cannot open ${exclude}, $!\n";
  while (<XCLD>) {
    chomp;
    $excludes{$_} = 1;
}

sub wanted {
  # Is it excluded from the report...
  return if (exists $excludes{$File::Find::name});

  # ...in a basic directory, ...
  return if $File::Find::dir =~ /sys|proc|dev/;

  # ...a regular file, ...
  return unless -f;

  # ...local, ...
  my @dirStats = stat($File::Find::name);
  return if (exists $devNums{$dirStats[0]});

  # ...and world writable?
  return unless (((stat)[2] & S_IWUSR) && ((stat)[2] & S_IWGRP) && ((stat)[2] & S_IWOTH));

  # If so, add the file to the list of world writable files
  print(WWFILE "$File::Find::name\n");

}

# Create the output file path if it doesn't already exist.
mkdir($dir or die "Cannot execute mkdir on ${dir}, $!") unless (-d $dir);

# Create our filehandle for writing our findings
open WWFILE, ">${dir}${tmpFile}" or die "Cannot open ${dir}${tmpFile}, $!";
print(WWFILE "${errHeader}") if ($errHeader);

find(\&wanted, @mounts);

close WWFILE;

# If no world-writable files have been found ${tmpFile} should be zero-size;
# Delete it so Tivoli won't alert
if (-z "${dir}${tmpFile}") {
  unlink "${dir}${tmpFile}";

} else {
  rename("${dir}${tmpFile}","${dir}${permFile}") or die "Cannot rename file ${dir}${tmpFile}, $!";

}

Answer 1

here are my suggestions:

Your hash %excludes would use less memory, if you would set the value to undef instead of 1.

You could set $File::Find::prune=1 in those cases where the whole directory is to be excluded. This shortcuts the search for this directory.

You could also use the cached data of stat() afterwards like this -f _.

The regex should use anchors (faster and more robust).

Here is the revised wanted subroutine (untested).

sub wanted {
  my @dirStats = stat($File::Find::name);

  # Is it excluded from the report...
  if (exists $excludes{$File::Find::name}) {
    $File::Find::prune=1 if (-d _);
    return;
  }

  # ...in a basic directory, ...
  if ($File::Find::name =~ /^\bsys\b|\bproc\b|\bdev\b$/) {
    $File::Find::prune=1 if (-d _);
    return;
  }

  # ... not a regular file, ...
  return unless -f _;

  # ...local, ...
  return if (exists $devNums{$dirStats[0]});

  # ...and world writable?
  my $protection = $dirStats[2];
  my $writemask = (S_IWUSR | S_IWGRP | S_IWOTH);
  return unless $writemask == $protection & $writemask;

  # If so, add the file to the list of world writable files
  print(WWFILE "$File::Find::name\n");

}