split - Convert dataframe from wide to long - pandas

Question

I have a dataframe like this:

index    S_1 S_2 S_3 S_4
 0        1    0   0   1
 1        1    1  Nan Nan

I am trying to change it from long to wide. Eg.

index num S
 0     1  1
 0     2  0
 0     3  0
 0     4  1
 1     1  1
 1     2  1
 1     3  Nan
 1     4  Nan

I have tried the following code, based on this answer, but I get the following error:

matches_df.columns = matches_df.columns.str.split('_', expand=True)

TypeError: object of type 'float' has no len()

Why am I unable to split on the "_"? There is other information in the columns which I would like to preserve.

Answer 1

There is pandas.wide_to_long, which is nice when the columns have stubs like that.

import pandas as pd

df.reset_index(inplace=True,drop=True)
df['id'] = df.index
df = pd.wide_to_long(df, stubnames='S_', i='id', j='num').reset_index().rename(columns={'S_':'S'})

#  id num  index    S
#0   0   1      0  1.0
#1   1   1      1  1.0
#2   0   2      0  0.0
#3   1   2      1  1.0
#4   0   3      0  0.0
#5   1   3      1  NaN
#6   0   4      0  1.0
#7   1   4      1  NaN