This is roughly what my data file looks like:
# Monid U B V R I u g r i J Jerr H Herr K Kerr IRAC1 I1err IRAC2 I2err IRAC3 I3err IRAC4 I4err MIPS24 M24err SpT HaEW mem comp
Mon-000001 99.999 99.999 21.427 99.999 18.844 99.999 99.999 99.999 99.999 16.144 99.999 15.809 0.137 16.249 99.999 15.274 0.033 15.286 0.038 99.999 99.999 99.999 99.999 99.999 99.999 null 55.000 1 N
Mon-000002 99.999 99.999 20.905 19.410 17.517 99.999 99.999 99.999 99.999 15.601 0.080 15.312 0.100 14.810 0.110 14.467 0.013 14.328 0.019 14.276 0.103 99.999 0.048 99.999 99.999 null -99.999 2 N
...and it's a total of 31mb in size. Here's my python script that pulls the Mon-###### IDs (found at the beginning of each of the lines).
import re
def pullIDs(file_input):
'''Pulls Mon-IDs from input file.'''
arrayID = []
with open(file_input,'rU') as user_file:
for line in user_file:
arrayID.append(re.findall('Mon\-\d{6}',line))
return arrayID
print pullIDs(raw_input("Enter your first file: "))
The script works but for this particular file it ran for well into 5 minutes and I eventually just killed the process due to impatience. Is this just something I'll have to deal with in python? i.e. Should this be written with a compiled language considering the size of my data file?
Further info: This script is being run within Emacs. This, by the checked answer, explains why it was running so slow.