7
\$\begingroup\$

I have some old RAR files and I know the general format of passwords I used to use, so I have generated a dictionary file combining those words in a variety of ways. The dictionary file is 500mb and I've wrote a python program to use the dictionary.

The issue is that the program has been running all night and is only 1% of the way in. I've now divided the dictionary file up and I am now running multiple instances of the program, but each instance is running pretty slow.

I would love any advice on how to improve the speed. I'm very new to Python, which will be obvious by my code (but I'm really enjoying Python).

import itertools
import sys, traceback
import os

with open('dictionary.txt') as infile:
    words = [line.strip() for line in infile]
    for word1 in itertools.product(words, repeat=1):
        fword = word1[0]
        print "Attempting: " + fword

        # open unrar and test password
        output = os.popen("unrar.exe x protected.rar -p" + fword)
        outstring = output.read()
        slng = len(outstring)

        # unrar.exe returns "All OK" for correct password
        validate = outstring.find("All OK", 0, slng) 
        if validate != -1:
            print "Found password: " + fword
            with open('correct_password.txt', 'w') as outfile:
                outfile.write(fword)
            sys.exit(0)
        else:
            # continue searching
            print "invalid password"

raw_input("Password was not in this dictionary!")

Sample dictionary.txt

FableLand
FableWay
FableTree
FableSpirit
FableApple
\$\endgroup\$
4
\$\begingroup\$

The line that jumps out at me as being a performance concern is this one:

words = [line.strip() for line in infile]

You’re generating a list that contains every word in dictionary.txt. This means a 500 MB dictionary file will create a similarly large list (which is going to be a huge structure). It also means that if the password is early on in the list, you still have the whole file as a list in memory.

This gets followed by the line

for word1 in itertools.product(words, repeat=1):
    fword = word1[0]

and I’m not sure quite why you’re getting the words out in this way. As you go through itertools.product, you get tuples, and you need to get the first element out. Using an iterable is generally good for performance, but the problem is creating the list in the first place, more than iterating over it.

The advice in How to read large file, line by line in Python is helpful here. Consider reading your file as

with open('dictionary.txt') as f:
    for line in f:
        # try that line as your password

That’s probably going to give a noticeable benefit.


Other comments:

  • Right now, all the functionality is tied up together. I’d separate the code which tests a password, and the code which gets an iterable of passwords from dictionary.txt. For example:

    def try_password(fword, rar_file):
        print "Attempting: %s" % fword
    
        # open unrar and test password
        output = os.popen("unrar.exe x %s -p %s" % (fword, rar_file))
        outstring = output.read()
    
        # unrar.exe returns "All OK" for correct password
        validate = outstring.find("All OK", 0, len(outstring)) 
        if validate != -1:
            print "Found password: %s" % fword
            return 0
        else:
            # continue searching
            return 1
    
    with open('dictionary.txt') as f:
         for line in f:
             attempt = try_password(line.strip(), 'protected.rar')
             if attempt == 0:
                 break
    

    Now you could also use the same code which gets your passwords to tackle, say, a bunch of encrypted ZIP files.

    I added the return codes 0 and 1 to denote success and failure when trying a password.

    What did the slng variable stand for? Anyway, it was only used once so I removed it.

    There are a few other tweaks I made, but nothing very major.

  • Your import sys, traceback statement should really be split onto two separate lines. Quoting from PEP 8, the Python style guide:

    Imports should usually be on separate lines

  • Why is the final line a raw_input instead of a print statement?

    • Where did dictionary.txt actually come from? You say you “generated” it, so could you just bypass the step where you write the passwords out to a file and just try them directly within the generator?
\$\endgroup\$
1
  • \$\begingroup\$ So much great advice. It is going to take me a little while to digest it all. I really appreciate your time and effort. \$\endgroup\$ – sarasimple Apr 20 '14 at 0:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.