9
\$\begingroup\$

I am dropping rows from a PANDAS dataframe when some of its columns have 0 value. I got the output by using the below code, but I hope we can do the same with less code — perhaps in a single line.

df:

    A   B   C
 0  1   2   5
 1  4   4   0
 2  6   8   4
 3  0   4   2

My code:

 drop_A=df.index[df["A"] == 0].tolist()
 drop_B=df.index[df["C"] == 0].tolist()
 c=drop_A+drop_B
 df=df.drop(df.index[c])

[out]

    A   B   C
 0  1   2   5
 2  6   8   4
\$\endgroup\$
  • \$\begingroup\$ Do you want to know a better way to do what your code is doing, or do you want us to code golf it? \$\endgroup\$ – Peilonrayz Jan 18 '18 at 11:27
  • \$\begingroup\$ I need a better way \$\endgroup\$ – pyd Jan 18 '18 at 11:27
12
\$\begingroup\$

I think you need create boolean DataFrame by compare all filtered columns values by scalar for not equality and then check all Trues per rows by all:

df = df[(df[['A','C']] != 0).all(axis=1)]
print (df)
   A  B  C
0  1  2  5
2  6  8  4

Details:

print (df[['A','C']] != 0)
       A      C
0   True   True
1   True  False
2   True   True
3  False   True

print ((df[['A','C']] != 0).all(axis=1))

0     True
1    False
2     True
3    False
dtype: bool

I think you need create boolean DataFrame by compare all values by scalar and then check any Trues per rows by any and last invert mask by ~:

df = df[~(df[['A','C']] == 0).any(axis=1)]

Details:

print (df[['A','C']])
   A  C
0  1  5
1  4  0
2  6  4
3  0  2

print (df[['A','C']] == 0)
       A      C
0  False  False
1  False   True
2  False  False
3   True  False

print ((df[['A','C']] == 0).any(axis=1))
0    False
1     True
2    False
3     True
dtype: bool

print (~(df[['A','C']] == 0).any(axis=1))
0     True
1    False
2     True
3    False
dtype: bool
\$\endgroup\$
  • \$\begingroup\$ Jezrael , I want to consider only column A and C , pls check my question once \$\endgroup\$ – pyd Jan 18 '18 at 11:31
  • \$\begingroup\$ @pyd Clarify this in your question. \$\endgroup\$ – Mast Jan 18 '18 at 11:39
  • \$\begingroup\$ You have both "all not equal to 0" and "not any equal to zero". Did you intend these to be two options, or did you accidentally post two solutions? \$\endgroup\$ – Acccumulation Jan 18 '18 at 17:51
  • \$\begingroup\$ @Accumulation No, it was no accident. I post first the best solution and second very nice, the best 2. :) \$\endgroup\$ – jezrael Jan 18 '18 at 18:03
4
\$\begingroup\$

One line hack using .dropna()

import pandas as pd

df = pd.DataFrame({'A':[1,4,6,0],'B':[2,4,8,4],'C':[5,0,4,2]})
print df
   A  B  C
0  1  2  5
1  4  4  0
2  6  8  4
3  0  4  2

columns = ['A', 'C']
df = df.replace(0, pd.np.nan).dropna(axis=0, how='any', subset=columns).fillna(0).astype(int)

print df
   A  B  C
0  1  2  5
2  6  8  4

So, what's happening is:

  1. Replace 0 by NaN with .replace()
  2. Use .dropna() to drop NaN considering only columns A and C
  3. Replace NaN back to 0 with .fillna() (not needed if you use all columns instead of only a subset)
  4. Correct the data type from float to int with .astype()
\$\endgroup\$
  • \$\begingroup\$ Nice! I was hoping there was .dropna() hack to be had... good one paulo! \$\endgroup\$ – killian95 Apr 5 at 21:59

protected by Jamal Feb 8 at 4:24

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.