pandas - Python Dataframe: How to check specific columns for elements

Question

I want to check whether all elements from a certain column contain the number 0?

I have a dataset that I read with df=pd.read_table('ad-data')
From this I felt a data frame with elements

[0] [1.] [2] [3] [4] [5] [6] [7] ....1559

[1.]  3   2   3   0   0   0   0

[2]  2   3   2   0   0   0   0

[3]  3   2   2   0   0   0   0

[4]  6   7   3   0   0   0   0

[5]  3   2   1   0   0   0   0

...
3220

I would like to check whether the data set from column 4 to 1559 contains only 0 or also other values.

Answer 1

You can check for equality with 0 element-wise and use all for rows:

df['all_zeros'] = (df.iloc[:, 4:1560] == 0).all(axis=1)

Small example to demonstrate it (based on columns 1 to 3 here):

N = 5
df = pd.DataFrame(np.random.binomial(1, 0.4, size=(N, N)))
df['all_zeros'] = (df.iloc[:, 1:4] == 0).all(axis=1)
df

Output:

   0  1  2  3  4  all_zeros
0  0  1  1  0  0      False
1  0  0  1  1  1      False
2  0  1  1  0  0      False
3  0  0  0  0  0       True
4  1  0  0  0  0       True

Update: Filtering non-zero values:

df[~df['all_zeros']]

Output:

   0  1  2  3  4  all_zeros
0  0  1  1  0  0      False
1  0  0  1  1  1      False
2  0  1  1  0  0      False

Update 2: To show only non-zero values:

pd.melt(
    df_filtered.iloc[:, 1:4].reset_index(),
    id_vars='index', var_name='column'
).query('value != 0').sort_values('index')

Output:

   index column  value
0      0      1      1
3      0      2      1
4      1      2      1
7      1      3      1
2      2      1      1
5      2      2      1