How to add multiple columns to pandas dataframe in one assignment

Question

I m new to pandas and trying to figure out how to add multiple columns to pandas simultaneously   Any help here is appreciated   Ideally I would like to do this in one step rather than multiple repeated steps     import pandas as pd  df     col 1    0  1  2  3            col 2    4  5  6  7   df   pd DataFrame df   df    column new 1    column new 2   column new 3       np nan   dogs  3    thought this would work here

User · Answer

You could instantiate the values from a dictionary if you wanted different values for each column & you don't mind making a dictionary on the line before.

>>> import pandas as pd
>>> import numpy as np
>>> df = pd.DataFrame({
  'col_1': [0, 1, 2, 3], 
  'col_2': [4, 5, 6, 7]
})
>>> df
   col_1  col_2
0      0      4
1      1      5
2      2      6
3      3      7
>>> cols = {
  'column_new_1':np.nan,
  'column_new_2':'dogs',
  'column_new_3': 3
}
>>> df[list(cols)] = pd.DataFrame(data={k:[v]*len(df) for k,v in cols.items()})
>>> df
   col_1  col_2  column_new_1 column_new_2  column_new_3
0      0      4           NaN         dogs             3
1      1      5           NaN         dogs             3
2      2      6           NaN         dogs             3
3      3      7           NaN         dogs             3

Not necessarily better than the accepted answer, but it's another approach not yet listed.

User · Answer

You could use assign with a dict of column names and values   In  1069   df assign     col new 1   np nan   col2 new 2    dogs    col3 new 3   3   Out 1069      col 1  col 2 col2 new 2  col3 new 3  col new 1 0      0      4       dogs           3        NaN 1      1      5       dogs           3        NaN 2      2      6       dogs           3        NaN 3      3      7       dogs           3        NaN

User · Answer

I would have expected your syntax to work too  The problem arises because when you create new columns with the column-list syntax  df  new1  new2           pandas requires that the right hand side be a DataFrame  note that it doesn t actually matter if the columns of the DataFrame have the same names as the columns you are creating     Your syntax works fine for assigning scalar values to existing columns  and pandas is also happy to assign scalar values to a new column using the single-column syntax  df new1          So the solution is either to convert this into several single-column assignments  or create a suitable DataFrame for the right-hand side   Here are several approaches that will work   import pandas as pd import numpy as np  df   pd DataFrame        col 1    0  1  2  3        col 2    4  5  6  7       Then one of the following   1  Three assignments in one  using list unpacking   df  column new 1    df  column new 2    df  column new 3      np nan   dogs   3    2  DataFrame conveniently expands a single row to match the index  so you can do this   df   column new 1    column new 2    column new 3      pd DataFrame   np nan   dogs   3    index df index    3  Make a temporary data frame with new columns  then combine with the original data frame later   df   pd concat                df          pd DataFrame                np nan   dogs   3                 index df index               columns   column new 1    column new 2    column new 3                    axis 1     4  Similar to the previous  but using join instead of concat  may be less efficient    df   df join pd DataFrame        np nan   dogs   3         index df index       columns   column new 1    column new 2    column new 3        5  Using a dict is a more  natural  way to create the new data frame than the previous two  but the new columns will be sorted alphabetically  at least before Python 3 6 or 3 7    df   df join pd DataFrame                 column new 1   np nan           column new 2    dogs            column new 3   3        index df index      6  Use  assign   with multiple column arguments   I like this variant on  zero s answer a lot  but like the previous one  the new columns will always be sorted alphabetically  at least with early versions of Python   df   df assign column new 1 np nan  column new 2  dogs   column new 3 3    7  This is interesting  based on https   stackoverflow com a 44951376 3830997   but I don t know when it would be worth the trouble   new cols     column new 1    column new 2    column new 3   new vals    np nan   dogs   3  df   df reindex columns df columns tolist     new cols      add empty cols df new cols    new vals    multi-column assignment works for existing cols   8  In the end it s hard to beat three separate assignments   df  column new 1     np nan df  column new 2      dogs  df  column new 3     3   Note  many of these options have already been covered in other answers  Add multiple columns to DataFrame and set them equal to an existing column  Is it possible to add several columns at once to a pandas DataFrame   Add multiple empty columns to pandas DataFrame

User · Answer

Just want to point out that option2 in  Matthias Fripp s answer   2  I wouldn t necessarily expect DataFrame to work this way  but it does df   column new 1    column new 2    column new 3      pd DataFrame   np nan   dogs   3    index df index   is already documented in pandas  own documentation http   pandas pydata org pandas-docs stable indexing html basics  You can pass a list of columns to    to select columns in that order  If a column is not contained in the DataFrame  an exception will be raised  Multiple columns can also be set in this manner  You may find this useful for applying a transform  in-place  to a subset of the columns

User · Answer

If you just want to add empty new columns  reindex will do the job  df    col 1  col 2 0      0      4 1      1      5 2      2      6 3      3      7  df reindex list df    column new 1    column new 2   column new 3    axis 1     col 1  col 2  column new 1  column new 2  column new 3 0      0      4           NaN           NaN           NaN 1      1      5           NaN           NaN           NaN 2      2      6           NaN           NaN           NaN 3      3      7           NaN           NaN           NaN   full code example  import numpy as np import pandas as pd  df     col 1    0  1  2  3            col 2    4  5  6  7   df   pd DataFrame df  print  df  df  sep   n   print   df df reindex list df    column new 1    column new 2   column new 3    axis 1  print    df reindex list df    column new 1    column new 2   column new 3    axis 1     df  sep   n     otherwise go for zeros answer with assign

User · Answer

I am not comfortable using  Index  and so on   could come up as below  df columns Index   A123    B123    dtype  object    df pd concat  df pd DataFrame columns list  CDE       df rename columns        C   C123        D   D123        E   E123    inplace True    df columns Index   A123    B123    C123    D123    E123    dtype  object

User · Answer

if adding a lot of missing columns  a  b  c        with the same value  here 0  i did this       new cols     a    b    c         df new cols    pd DataFrame   0    len new cols    index df index    It s based on the second variant of the accepted answer

User · Answer

With the use of concat   In  128   df Out 128       col 1  col 2 0      0      4 1      1      5 2      2      6 3      3      7  In  129   pd concat  df  pd DataFrame columns      column new 1    column new 2   column new 3      Out 129       col 1  col 2 column new 1 column new 2 column new 3 0    0 0    4 0          NaN          NaN          NaN 1    1 0    5 0          NaN          NaN          NaN 2    2 0    6 0          NaN          NaN          NaN 3    3 0    7 0          NaN          NaN          NaN   Not very sure of what you wanted to do with  np nan   dogs  3   Maybe now set them as default values   In  142   df1   pd concat  df  pd DataFrame columns      column new 1    column new 2   column new 3      In  143   df1    column new 1    column new 2   column new 3       np nan   dogs   3   In  144   df1 Out 144       col 1  col 2  column new 1 column new 2  column new 3 0    0 0    4 0           NaN         dogs             3 1    1 0    5 0           NaN         dogs             3 2    2 0    6 0           NaN         dogs             3 3    3 0    7 0           NaN         dogs             3

User · Answer

use of list comprehension  pd DataFrame and pd concat  pd concat                df          pd DataFrame                np nan   dogs   3  for   in range df shape 0                 df index    column new 1    column new 2   column new 3                    axis 1

[python] How to add multiple columns to pandas dataframe in one assignment?

Examples related to python

Examples related to pandas

Examples related to dataframe