1
\$\begingroup\$
with open(filepath) as fp:
   #line = fp.readline()
   cnt = 1
   while line:
       line = fp.readline()
       print("Line {}%: {}".format(cnt, line.strip()))

       line = line.replace('\n', '')
       #print('line->',line)
       rst = read_ct[read_ct['user_id']==line]
       #print(rst)
       if rst.shape[0] != 0:
           AI = rst.iloc[0,1]
       else:
           AI ='32'
       index=df2[df2['id']==AI].index[0]

       df['distances'] = cosine_similarity(df, df.iloc[0:index])
       n = 1000 # or however many you want
       n_largest = df['distances'].nlargest(n + 1) # this contains the parent itself as the most similar entry, hence n+1 to get n children

       file.write(line)
       #file.writelines([df2.at[i,'article_id'] for i in list(n_largest.index)])

       for i in list(n_largest.index):
            file.write(" "+df2.at[i,'article_id'])




       file.write('\n') 
       #break
       cnt += 1

I'm trying to write some preprocessed data to the file but it takes too long so want to gather the idea to make it faster

its writing 1000 results for each ID want to know how to make this code "faster"

\$\endgroup\$
  • \$\begingroup\$ Use the python profiler to see where your script is spending the most time. \$\endgroup\$ – RootTwo Jul 25 at 6:23
  • \$\begingroup\$ This code is doing a whole lot more than just writing to a file! Please explain what you are calculating. See How to Ask. \$\endgroup\$ – 200_success Jul 25 at 14:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.