2
\$\begingroup\$

I've just spent a good hour or so developing a python script that hunts down and removes .php files depending on the regex expression the situation was a client has got several sites and got hacked recently and as a result a large portion of 8 character random php files have created, so in order to remove them all I made a python script to hunt and find the files whilst avvoiding the files needed also which are 8 characters.

Target:

n1qv0iop.php
e7wo0sb8.php
sdt24g9o.php
9ktrhstu.php
2cvgc7qa.php
nlrgnwzt.php
pyj67osr.php

Script:

import os
import re

rootdir = "/ROOTDIRECTORY"
regex = re.compile('^((((?!absolute|abstract|accounts|activate|activity|advanced|archives|autoload|autoplay|basiccss|bookmark|calendar|captions|category|checkbox|comments|compiled|conntest|controls|counters|courierb|courieri|creative|cronview|database|defaults|disabled|document|dropcaps|elements|embedded|error404|external|facebook|features|fieldset|flamingo|flipcard|freedoms|frontend|getcombo|getnodes|groupped|gzdecode|homepage|icongrid|iconlist|igniteup|importer|includer|launcher|lazyload|licenses|lightbox|location|magazine|mailpoet|manifest|mappings|myticket|optimize|packager|password|phpthumb|platform|pointers|products|progress|promobox|radiotab|redirect|register|renderer|repeater|required|response|revision|richtext|rollback|rollover|security|services|settings|sidebars|sitemaps|switcher|taxonomy|template|testmail|textarea|thankyou|timeline|tracking|upgrader|upgrades|variable|videobox|whatsnew|payments|donation|checkout|readmore|overview|tooltips|purchase|endpoint|firewood|pagepost|jwplayer|analytic|builders|bulkedit|carousel|delivery|featured|feedback|fontello|fontpack|headline|honeypot|messages|metadata|position|registry|sendaway|sections|singular|siblings|subpages|tabgroup|thickbox|turfmail|webfonts|networks|htaccess))[a-z0-9]{8})).php$')

for root, dirs, files in os.walk(rootdir):
  for file in files:
    if regex.match(file):
   print(os.path.join(root,file))
   #os.remove(os.path.join(root,file))

   print "END";

My question is can this be improved? is there a better way to write the regex pattern?just looking for advice :)

\$\endgroup\$
5
  • 1
    \$\begingroup\$ any reason for regex? glob searching for **.php would be easier imo \$\endgroup\$
    – hjpotter92
    Oct 7, 2020 at 6:54
  • \$\begingroup\$ Which Python version are you using? \$\endgroup\$
    – AMC
    Oct 9, 2020 at 21:16
  • \$\begingroup\$ @AMC I am using version 2 \$\endgroup\$ Oct 11, 2020 at 0:10
  • \$\begingroup\$ @hjpotter92 no reason just thought it might be easier \$\endgroup\$ Oct 11, 2020 at 0:11
  • \$\begingroup\$ I am using version 2 I'm guessing that's a hard requirement? \$\endgroup\$
    – AMC
    Oct 12, 2020 at 22:46

2 Answers 2

3
\$\begingroup\$

I think a regex is overkill here. There comes a point where evaluating all these or'ed exceptions becomes just too slow. I would instead use a simple glob and a blacklist:

from pathlib import Path

IGNORE = {"absolute", "abstract", ...}

root = Path("/ROOTDIRECTORY")
for file in root.glob("**/*.php"):
    if len(file.stem) == 8 and file.stem not in IGNORE:
        print(file)

Note that I used a set for a fast check with in.

You could also use the glob "**/????????.php", as suggested by @RootTwo in the comments, then you wouldn't have to do the length check, although that is quite fast.

Indeed, what you should ideally do is not take my or anyone else's word for it, but measure. Measure how fast your approach is, how fast an optimized regex is, how fast mine is and how fast using a different glob is. Maybe it matters, or maybe all of this is premature optimization and it doesn't actually matter that one takes a millisecond and the other a second (or whatever the actual results are). That depends entirely on your use case.

\$\endgroup\$
4
  • 1
    \$\begingroup\$ Perhaps use "**/????????.py" for the pattern. \$\endgroup\$
    – RootTwo
    Oct 7, 2020 at 18:44
  • \$\begingroup\$ @RootTwo That's definitely also a possibility, but for that I would run tests to see which is faster in that particular use case. Actually, I would run tests either way, so let me add that to the answer. \$\endgroup\$
    – Graipher
    Oct 7, 2020 at 18:49
  • \$\begingroup\$ @RootTwo will "**/????????.py" avoid words tho like comment, autoload etc. \$\endgroup\$ Oct 11, 2020 at 0:12
  • \$\begingroup\$ @MrJoshFisher No, you will still need the ignore list. It just selects filenames with eight characters without using a regex. \$\endgroup\$
    – Graipher
    Oct 11, 2020 at 1:56
1
\$\begingroup\$

Looks OK!

  • I guess it's just missing a \ near .php. Currently, these strings are being accepted:

    n1qv0iopAphp
    e7wo0sb8Bphp
    sdt24g9oCphp
    9ktrhstu1php
    2cvgc7qa&php
    

    Current Expression
    Updated Expression 1

  • We can likely get ride of some of those capturing groups. Updated Expression 2

    ^(?!absolute|abstract|accounts|activate|activity|advanced|archives|autoload|autoplay|basiccss|bookmark|calendar|captions|category|checkbox|comments|compiled|conntest|controls|counters|courierb|courieri|creative|cronview|database|defaults|disabled|document|dropcaps|elements|embedded|error404|external|facebook|features|fieldset|flamingo|flipcard|freedoms|frontend|getcombo|getnodes|groupped|gzdecode|homepage|icongrid|iconlist|igniteup|importer|includer|launcher|lazyload|licenses|lightbox|location|magazine|mailpoet|manifest|mappings|myticket|optimize|packager|password|phpthumb|platform|pointers|products|progress|promobox|radiotab|redirect|register|renderer|repeater|required|response|revision|richtext|rollback|rollover|security|services|settings|sidebars|sitemaps|switcher|taxonomy|template|testmail|textarea|thankyou|timeline|tracking|upgrader|upgrades|variable|videobox|whatsnew|payments|donation|checkout|readmore|overview|tooltips|purchase|endpoint|firewood|pagepost|jwplayer|analytic|builders|bulkedit|carousel|delivery|featured|feedback|fontello|fontpack|headline|honeypot|messages|metadata|position|registry|sendaway|sections|singular|siblings|subpages|tabgroup|thickbox|turfmail|webfonts|networks|htaccess)[a-z0-9]{8}\.php$
    
  • We can also make the stopwords shorter, but I would leave it as is, because it would be more readable and it does not improve the performance. For instance autoload|autoplay could be one character shorter: auto(?:load|play). Updated Expression 3

    ^(?!absolute|abstract|accounts|activate|activity|advanced|archives|auto(?:load|play)|basiccss|bookmark|calendar|captions|category|checkbox|comments|compiled|conntest|controls|counters|courierb|courieri|creative|cronview|database|defaults|disabled|document|dropcaps|elements|embedded|error404|external|facebook|features|fieldset|flamingo|flipcard|freedoms|frontend|getcombo|getnodes|groupped|gzdecode|homepage|icongrid|iconlist|igniteup|importer|includer|launcher|lazyload|licenses|lightbox|location|magazine|mailpoet|manifest|mappings|myticket|optimize|packager|password|phpthumb|platform|pointers|products|progress|promobox|radiotab|redirect|register|renderer|repeater|required|response|revision|richtext|rollback|rollover|security|services|settings|sidebars|sitemaps|switcher|taxonomy|template|testmail|textarea|thankyou|timeline|tracking|upgrader|upgrades|variable|videobox|whatsnew|payments|donation|checkout|readmore|overview|tooltips|purchase|endpoint|firewood|pagepost|jwplayer|analytic|builders|bulkedit|carousel|delivery|featured|feedback|fontello|fontpack|headline|honeypot|messages|metadata|position|registry|sendaway|sections|singular|siblings|subpages|tabgroup|thickbox|turfmail|webfonts|networks|htaccess)[a-z0-9]{8}\.php$
    

Overall it'd best to stay as far away from the regular expression as possible. ( ˆ_ˆ )


If you wish to simplify/update/explore the expression, it's been explained on the top right panel of regex101.com. You can watch the matching steps or modify them in this debugger link, if you'd be interested. The debugger demonstrates that how a RegEx engine might step by step consume some sample input strings and would perform the matching process.


RegEx Circuit

jex.im visualizes regular expressions:

enter image description here

\$\endgroup\$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.