I wrote a program to retrieve the key/messages from 10 different ciphers which were all encrypted with the same key with an xor one-time-pad method via crib dragging.

To do this, I wrote a python script which turns a string to test to hex code. Then it xors the 10 cipher strings (200 hex characters each), two at a time. After that it, xors the test string with the 5 xor hashes on each position and then looks via a website whether this is a reasonably good string to go further and try again.

I already tested it and got positive results.

Now when I do it with the real ciphers I got nowhere so far, and I am not sure whether this is because of the script or me not being able to expand on retrieved valid string fragments.

Here is the code, I know that the website lookup is very bad, but I found no other way this fast/easy (to implement, not to run).

EDIT: Since I do not think overwriting the code is good, I put it here in case somebody wants to see it: Gist link

# -*- coding: UTF-8 -*-
import codecs
from urllib.error import URLError
import urllib.request

forbiddenSigns = ['{', '}', '[', ']', '(', ')', '?', '=', '%', '_', ';', '-', '^', ':', '<', '>', '|', '!', '`', '´', "'", '\\']

with open("testCiphers.txt") as f:
    ciphers = f.readlines()

ciphers = [s.strip() for s in ciphers]

crib = "the"
cribList = []

for c in crib:

cribHash = ''.join(cribList)
cribLength = len(cribHash)

cipherHashList = []
for i in range(0, len(ciphers) - 1):
    cipherHashList.append(int(ciphers[i], 16) ^ int(ciphers[i + 1], 16))

hashResultList = [str(hex(hr))[2:] for hr in cipherHashList]

hexList = []

for hashResultString in hashResultList:
    for i in range(0, len(hashResultString) - cribLength + 1, 1):
        hexList.append(hashResultString[i:i + cribLength])

for ha in hexList:
    cribAttempt = int(cribHash, 16) ^ int(ha, 16)
    cribAttempt = (hex(cribAttempt))[2:].zfill(cribLength)
    clearText = codecs.decode(cribAttempt, "hex")
    if any(s not in str(clearText) for s in forbiddenSigns) and "\\x" not in str(clearText):
        url = 'http://www.morewords.com/contains/' + str(clearText, "ASCII")
            siteRequest = str(urllib.request.urlopen(url).read())
            if "words found.</strong></p>" in siteRequest:
                print(str(clearText, "ASCII"), end="\n")
        except URLError :
            "error getting webpage"
  • \$\begingroup\$ Can you give us some examples of the "real ciphers" you failed on? (Please say where you got them from too.) \$\endgroup\$ Oct 29, 2015 at 9:03

2 Answers 2


List comprehensions

You use list comprehensions sometimes, and sometimes you use for ...: someList.append(...). This is not consistent. Use list comprehensions as much as you can since you are already building lists (and list comprehensions are slightly faster than for loops):

cribList = [hex(ord(letter)[2:] for letter in crib]

cipherHashList = [int(ciphers[i],16)^int(ciphers[i+1],16) for i in range(len(ciphers)-1)]

hexList = [hashResultString[i:i+cribLength] for hashResultString in hashResultList for i in range(len(hashResultString) - cribLength + 1)]

You can speed things up a little more (maybe at the cost of readability) by using more complex expressions in the list comprehension to avoid creating intermediate lists:

hashResultList = [hex(int(ciphers[i],16)^int(ciphers[i+1],16))[2:] for i in range(len(ciphers)-1)]


Check PEP 8 for the rationale but you’d better use UPPERCASE names for constants such as FORBIDDEN_SIGNS or CRIB and lower_snake_case for variables such as hex_list or crib_length.

Some of the variable are named thingList and might be improved using things.


Make functions to ease testing into the interpreter and/or reusability:

def get_crib_hash():
    return ''.join(hex(ord(letter))[2:] for letter in CRIB)

change the value of CRIB in the interpreter and voilà, every time you (or any function) use get_crib_hash() the value is easily updated without having to recompute everything.

For efficiency reasons I recommend using a generator function for file reading.


On my computer, instead of returning b'test':

>>> codecs.decode("74657374","hex")
Traceback (most recent call last):
  File "<pyshell#39>", line 1, in <module>
LookupError: unknown encoding: hex

You can use a more robust approach in the name of binascii.unhexlify which is doing exactly what you want.

For your issue with “real ciphers”, you might want to print the clear text first instead of trying to lookup words through a web API. It’ll help understand what is happening.

Small improvements

  • forbiddenSigns = ['{', '}', '[', ']', '(', ')', '?', '=', '%', '_', ';', '-', '^', ':', '<', '>', '|', '!', '`', '´', "'", '\\']

    is better written (more resource friendly) as:

    FORBIDDEN_SIGNS = '{}[]()?=%_;-^:<>|!`´\'\\'

    and you will still be able to iterate over each symbol.

  • range() does not need a 0 as first argument (unless you want to specify a step); similarly a step of 1 is the default.
  • compute once if you can; e.g: str(clearText, "ASCII").
  • consider using an if __name__ == '__main__': construct to emphasize on reusability and improve testing in the interactive interpreter.
  • format is often more efficient for building strings; it can even replace hex for your use case.

Proposed improvements

import binascii

FORBIDDEN_SIGNS = '{}[]()?=%_;-^:<>|!`´\'\\'
CRIB = 'the'

def get_crib_hash():
    return ''.join(format(ord(letter), '02x') for letter in CRIB)

def hash_file(filename):
    with open(filename) as f:
        previous = None
        for line in f:
            line = line.strip()
                yield format(int(previous, 16) ^ int(line, 16), 'x')
            except TypeError: # for int(None, 16) at first pass
            previous = line # Yeah, I’ll compute int(this_thing, 16) a second time…
            # Avoiding it is left as an exercise for the reader.

def decipher_file(filename):
    crib_hash = get_crib_hash()
    crib_length = len(crib_hash)
    crib_hash = int(crib_hash, 16)

    for hash_result in hash_file(filename):
        for i in range(len(hash_result) - crib_length + 1):
            attempt = crib_hash ^ int(hash_result[i:i+crib_length], 16)
            attempt = '{0:0{fill}x}'.format(attempt, fill=crib_length)
            yield str(binascii.unhexlify(attempt), 'ASCII')

if __name__ == '__main__':
    from urllib.error import URLError
    from urllib.request import urlopen

    for clear_text in decipher_file('testCiphers.txt'):
        # or just print(clear_text)…
        if '\\x' not in clear_text and any(s not in clear_text
                                           for s in FORBIDDEN_SIGNS):
            url = 'http://www.morewords.com/contains/{}'.format(clear_text)
                response = str(urlopen(url).read())
                if 'words found.</strong></p>' in response:
            except URLError :
                'error getting webpage'
  • \$\begingroup\$ @GarethRees A for loop that's appending instead of just using a list comprehension does reduce efficiency. \$\endgroup\$ Oct 29, 2015 at 12:56
  • \$\begingroup\$ @GarethRees Somehow, yes. list.append for the sake of building a list out of another one is better written as a list comprehension whenever possible. \$\endgroup\$ Oct 29, 2015 at 12:56
  • \$\begingroup\$ @GarethRees I'm actually not sure of the actualy reason, but tests with timeit often show that it's faster to use a list comprehension. Approximately 17% faster in the case of the first loop Mathias cited. (This is in Python 2.7, which may affect results) \$\endgroup\$ Oct 29, 2015 at 14:18
  • \$\begingroup\$ Ah, so when you say "efficiency", you mean "speed". (Normally "efficiency" refers to algorithmic efficiency.) \$\endgroup\$ Oct 29, 2015 at 14:20
  • \$\begingroup\$ I revised the opening paragraph accordingly. \$\endgroup\$ Oct 29, 2015 at 14:30

You could read the ciphers in a slightly more efficiently. You can use map to run a function over each element of a list, which is even faster than a list comprehension. You could directly create ciphers here by running strip on every element in f.readlines().

with open("testCiphers.txt") as f:
    ciphers = map(strip, f.readlines())

You call str on clearText every time it's used, just make it a string from the start.

clearText = str(codecs.decode(cribAttempt, "hex"))

This saves time and aids readability.

Also Python supports short circuiting. This means that when evaluating a set of conditions (like x and y, x or y etc.) it will only evaluate as much as it needs to. So in this line:

if any(s not in str(clearText) for s in forbiddenSigns) and "\\x" not in str(clearText):

it will only check clearText for \\x if the any call returns True. If any returns False it already knows the result of False and any_value so it will move on. You could save time by putting the faster to evaluate condition first, in this case your \\x in clearText is faster since it only checks one thing, while any runs over each iteration of forbiddenSigns.

if "\\x" not in str(clearText) and any(s not in str(clearText) for s in forbiddenSigns):

Now that they're flipped you only ever need to call any if you haven't found '\\x'. Speaking of which, you can use a raw string to avoid that extra backslash. Raw strings basically don't treat a backslash as a special escape character, so instead of needing '\\x' you can just use r'\x'. The r indicates that it's a raw string.

(Note the one exception to this is that you still can't end a string with a backslash, as that still escapes the closing quotation mark)

And lastly, you don't treat errors properly. Look at your exception:

    except URLError:
        "error getting webpage"

except URLError is good! Lots of people make the mistake of using generic errors. But then, you don't do anything. You're just putting in a string but not printing it or raising it as an error. If you want to raise a URLError with your custom message, you need to do this:

    except URLError:
        URLError("error getting webpage")
  • \$\begingroup\$ i was able to work most of the changes into my code, only for some multiple times str calls I was not able to find a workaround. If you want to see it, here is the code: gist.github.com/Acru2015/22ffb2514fae1e62af55 \$\endgroup\$
    – Anton
    Oct 29, 2015 at 17:15
  • \$\begingroup\$ I'm not sure what you mean about the multiple str calls, can you explain? Also if you'd like you could post a new question on here with your updated code. If you do that, make sure to link back here and explain that it's a follow up question. You could probably wait a bit though as you might get more answers still. \$\endgroup\$ Oct 29, 2015 at 17:38
  • \$\begingroup\$ With str I meant that only declaring str(codecs.decode(cribAttempt, "hex")) avariable once dis not wirk for me at every position in the code. \$\endgroup\$
    – Anton
    Oct 30, 2015 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.