Pandas split column of lists into multiple columns

Question

I have a pandas DataFrame with one column   import pandas as pd  df   pd DataFrame      data            teams                   SF    NYG                  SF    NYG                  SF    NYG                  SF    NYG                  SF    NYG                  SF    NYG                  SF    NYG                       print df    Output          teams 0   SF  NYG  1   SF  NYG  2   SF  NYG  3   SF  NYG  4   SF  NYG  5   SF  NYG  6   SF  NYG    How can split this column of lists into 2 columns

User · Answer

Much simpler solution   pd DataFrame df2  teams   to list    columns   team1    team2      Yields     team1 team2 ------------- 0    SF   NYG 1    SF   NYG 2    SF   NYG 3    SF   NYG 4    SF   NYG 5    SF   NYG 6    SF   NYG 7    SF   NYG   If you wanted to split a column of delimited strings rather than lists  you could similarly do   pd DataFrame df  teams   str split   lt delim gt    expand True  values               columns   team1    team2

User · Answer

Based on the previous answers  here is another solution which returns the same result as df2 teams apply pd Series  with a much faster run time   pd DataFrame   x  y for x  y in enumerate item   for item in df2  teams   values tolist     index df2 index    Timings   In  1   import pandas as pd d1     teams      SF    NYG     SF    NYG     SF    NYG                      SF    NYG     SF    NYG     SF    NYG     SF    NYG     df2   pd DataFrame d1  df2   pd concat  df2  1000  reset index drop True   In  2    timeit df2  teams   apply pd Series   8 27 s    2 73 s per loop  mean    std  dev  of 7 runs  1 loop each   In  3    timeit pd DataFrame   x  y for x  y in enumerate item   for item in df2  teams   values tolist     index df2 index   35 4 ms    5 22 ms per loop  mean    std  dev  of 7 runs  10 loops each

User · Answer

There seems to be a syntactically simpler way  and therefore easier to remember  as opposed to the proposed solutions  I m assuming that the column is called  meta  in a dataframe df    df2   pd DataFrame df  meta   str split   values tolist

User · Answer

list comprehension  simple implementation with list comprehension   my favorite   df   pd DataFrame  pd Series x  for x in df teams   df columns     team     format x 1  for x in df columns    timing on output   CPU times  user 0 ns  sys  0 ns  total  0 ns Wall time  2 71 ms    output   team 1  team 2 0   SF  NYG 1   SF  NYG 2   SF  NYG 3   SF  NYG 4   SF  NYG 5   SF  NYG 6   SF  NYG

User · Answer

This solution preserves the index of the df2 DataFrame  unlike any solution that uses tolist     df3   df2 teams apply pd Series  df3 columns     team1    team2     Here s the result     team1 team2 0    SF   NYG 1    SF   NYG 2    SF   NYG 3    SF   NYG 4    SF   NYG 5    SF   NYG 6    SF   NYG

User · Answer

Here s another solution using df transform and df set index   gt  gt  gt   df  teams           transform  lambda x x 0   lambda x x 1            set axis   team1   team2                      axis 1                    inplace False           team1 team2 0    SF   NYG 1    SF   NYG 2    SF   NYG 3    SF   NYG 4    SF   NYG 5    SF   NYG 6    SF   NYG

User · Answer

You can use DataFrame constructor with lists created by to list   import pandas as pd  d1     teams      SF    NYG     SF    NYG     SF    NYG                      SF    NYG     SF    NYG     SF    NYG     SF    NYG     df2   pd DataFrame d1  print  df2         teams 0   SF  NYG  1   SF  NYG  2   SF  NYG  3   SF  NYG  4   SF  NYG  5   SF  NYG  6   SF  NYG      df2   team1   team2      pd DataFrame df2 teams tolist    index  df2 index  print  df2         teams team1 team2 0   SF  NYG     SF   NYG 1   SF  NYG     SF   NYG 2   SF  NYG     SF   NYG 3   SF  NYG     SF   NYG 4   SF  NYG     SF   NYG 5   SF  NYG     SF   NYG 6   SF  NYG     SF   NYG   And for new DataFrame   df3   pd DataFrame df2  teams   to list    columns   team1   team2    print  df3    team1 team2 0    SF   NYG 1    SF   NYG 2    SF   NYG 3    SF   NYG 4    SF   NYG 5    SF   NYG 6    SF   NYG   Solution with apply pd Series  is very slow    7k rows df2   pd concat  df2  1000  reset index drop True   In  121    timeit df2  teams   apply pd Series  1 79 s    52 5 ms per loop  mean    std  dev  of 7 runs  1 loop each   In  122    timeit pd DataFrame df2  teams   to list    columns   team1   team2    1 63 ms    54 3   s per loop  mean    std  dev  of 7 runs  1000 loops each

User · Answer

The above solutions didn t work for me since I have nan observations in my dataframe  In my case df2   team1   team2      pd DataFrame df2 teams values tolist    index  df2 index  yields   object of type  float  has no len     I solve this using list comprehension  Here the replicable example   import pandas as pd import numpy as np d1     teams      SF    NYG     SF    NYG     SF    NYG                  SF    NYG     SF    NYG     SF    NYG     SF    NYG     df2   pd DataFrame d1  df2 loc 2  teams     np nan df2 loc 4  teams     np nan df2   output           teams 0    SF  NYG  1    SF  NYG  2   NaN 3    SF  NYG  4   NaN 5    SF  NYG  6    SF  NYG   df2  team1   np nan df2  team2   np nan   solving with list comprehension   for i in  0 1       df2  team    format str i 1     k i  if isinstance k list  else k for k in df2  teams     df2   yields       teams   team1   team2 0    SF  NYG    SF  NYG 1    SF  NYG    SF  NYG 2   NaN        NaN  NaN 3    SF  NYG    SF  NYG 4   NaN        NaN  NaN 5    SF  NYG    SF  NYG 6    SF  NYG    SF  NYG

[python] Pandas split column of lists into multiple columns

Examples related to python

Examples related to pandas