While working with someone else's code I've isolated a time-killer in the (very large) script. Basically this program runs through a list of thousands and thousands of objects to find matches to an input (or many inputs) object. It works well when the number is below 50 but any input higher than that and it begins to take a looooong time. Is this just an issue with the database I'm running it against (30,000 things)? Or is this code written inefficiently?
for i in range(BestX): Tin=model[BestFitsModel[i]] Tau2=np.append(Tau2,model[BestFitsModel[i]]) Dir=model[BestFitsModel[i]] Dir=Dir.replace('inp','out',1) ##open up correct .out file T=open(Dir,"rb") hold='' Tau= for line in T: if not line.strip(): continue else: hold=line.split() Tau.append(hold) Tauline=0 Taunumber=0 print 'Loop' for j in range(5,len(Tau)-50): if 'RESULTS:' in Tau[j-5]: for k in range(j,j+50): if (Tau[k] == model[BestFitsModel[i]]): Tauline=i #line number of the tau Taunumber=k-(j) #Tau 1, 2, 3, etc. Dir=Dir.replace('out','stb',1)#.stb file Dir2=np.append(Dir2,Dir) F=open(Dir, "rb") hold='' Spectrum= for line in F: hold=line.split() Spectrum.append(hold)
BestX is a list of 30,000 objects (stars).
I'm not looking for an answer to this problem, I'm looking for code-critique...there has to be a way to write this so that it performs faster, right?
EDIT: Reading through the Python Performance Tips I've noticed one little snag (so far). Starting with the line
Tau= and continuing for the next six lines. I can shorten it up by
Tau=[line.split() for line in T]
I'm gonna keep looking. Hopefully I'm on the right track.