-2
\$\begingroup\$

hey everyone I have a project to do and I will be glad to receive your help the dataframe I am working on is relatively large from Kaggel, it has 250 columns and 22000 rows, the data are from financial statements of companies traded on the stock exchange and in some of the columns / rows I have zero values, the problem is that I can not remove all the zero values and replace them with another value because I guess some of the zeros of the data are correct, I mean there are financial companies that make sense they have zero values in their reports,so how can I go over the columns in a smart way to know which zeros to replace and which not?

Thanks for the help

this is what I tried to do but it remove all zeros if there are over 40% zero values from the data but its not the best way handle zero values

num=0
for column_name in df_cp.columns:
    column = df_cp[column_name]
    # Get the count of Zeros in column 
    count = (column == 0).sum()
    print('Count of zeros in column ', column_name, ' is : ', count,', percent : ',round(count/df_cp.shape[0]*100),'%')
    if (round(count/df_cp.shape[0]*100) >= 40) & (column_name !='Class') :
      df_cp.drop(column_name,axis=1,inplace=True)
      print('________________drop before_______________')
      num+=1
print('number of columns that we drop: ',num) 
New contributor
eve is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
\$\endgroup\$
3
  • \$\begingroup\$ Welcome to Code Review! It sounds like the code is not working as intended - can you please confirm this? \$\endgroup\$ 2 days ago
  • \$\begingroup\$ The code is ok but its not the best way to handel with zero values in dataframe, Im trying to figure out how to leave logical zero values in a table without moving column by column manually \$\endgroup\$
    – eve
    2 days ago
  • 1
    \$\begingroup\$ If you don't know which are "good zeros" and which are "bad zeros", we certainly won't be able to do any better. \$\endgroup\$
    – Reinderien
    2 days ago

Browse other questions tagged or ask your own question.