# Tokenizing and sorting the strings of a text file with the results of a public competition

I made this script to carry out a specific real-life task, which was to sort a list of candidates in a public competition by grade. The file with the informations consists of a text file copied from part of a pdf with the ID, name, grade in the exam and grade in the composition. There are a little more than 1,000 candidates. I needed it sorted by the final grade, which is obtained by the formula:

Final grade = Exam + (Composition * 2)/3

1) The information about each candidate was separated by '/' and not by new lines. And each information about the candidate (ID, name, exam note, composition note) was separated by ','

2) new line characters were spread randomly throughout the text (I think that was caused by the Notepad or by the PDF program auto-break line)

3) The page numbers of the original PDF remained in the text file, and could be found beside the ID number, one of the grades or between parts of a name, surrounded by new line characters like "\n26\n". The page numbers ranged from 15 to 26.

This is a fragment of the original text as read by Python:

\xa0 9.00,\xa0 8.41\xa0/\xa0\n26\n10069207,\xa0Wilix\xa0Gabriel\xa0Pastor\xa0Silva,\xa08.19,\xa06.35\xa0/\xa010095535,\xa0Willes\xa0Vitorio\xa0do\xa0Carmo,\xa08.25,\xa08.83\xa0/\xa010000327,\xa0\nWilliam\xa0Maia\

(You may note that there's a page number between the last grade of one of the candidates and the ID number of the next one right in the beginning of the fragment)

So I had to find a way of detecting and removing the new line characters and the page numbers, and put each part of the string in the correct place taking into account that the names of the candidates could be of any size (Like "Anderson Soares Freixo" or "Maria Auxiliadora da Silva Santos Xavier".

I came up with this code:

from operator import itemgetter

def remove_page_number(origin_list):
"""Finds if there is a page number with 2
digits anywhere inside the strings of a given list
and removes it"""
new_list = []
for element in origin_list:
subelements = element.split()
no_page = ''
for el in subelements:
if el.isdigit():
if len(el) > 2:
no_page+=el
else:
no_page+=el+' '
new_list.append(no_page)
return new_list

return round((exam+(composition*2))/3, 2)

def categorize(raw):
"""Gets the raw material and put each piece
of information in the right category"""
for cand in raw:
items = remove_page_number(cand.split(','))
candidate = {'ID': items[0],
'Name': items[1],
'Exam':float(items[2]),
'Composition': float(items[3]),
}

def print_list(candidates):
"""Prints the list of candidates"""
rank = 0
for c in candidates:
rank+=1
print("{:4}|{:10}|{:50}|{:4}|{:4}|{:4}|".format(rank, c['ID'],
c['Name'],
c['Exam'],
c['Composition'],
c['Final']))

if __name__ == '__main__':