How to shift a column in Pandas DataFrame

Question

I would like to shift a column in a Pandas DataFrame  but I haven t been able to find a method to do it from the documentation without rewriting the whole DF  Does anyone know how to do it   DataFrame         x1   x2   0  206  214   1  226  234   2  245  253   3  265  272   4  283  291   Desired output         x1   x2   0  206  nan   1  226  214   2  245  234   3  265  253   4  283  272   5  nan  291

User · Answer

Lets define the dataframe from your example by

>>> df = pd.DataFrame([[206, 214], [226, 234], [245, 253], [265, 272], [283, 291]], 
    columns=[1, 2])
>>> df
     1    2
0  206  214
1  226  234
2  245  253
3  265  272
4  283  291

Then you could manipulate the index of the second column by

>>> df[2].index = df[2].index+1

and finally re-combine the single columns

>>> pd.concat([df[1], df[2]], axis=1)
       1      2
0  206.0    NaN
1  226.0  214.0
2  245.0  234.0
3  265.0  253.0
4  283.0  272.0
5    NaN  291.0

Perhaps not fast but simple to read. Consider setting variables for the column names and the actual shift required.

Edit: Generally shifting is possible by df[2].shift(1) as already posted however would that cut-off the carryover.

User · Answer

In  18   a Out 18       x1  x2 0   0   5 1   1   6 2   2   7 3   3   8 4   4   9  In  19   a  x2     a x2 shift 1   In  20   a Out 20       x1  x2 0   0 NaN 1   1   5 2   2   6 3   3   7 4   4   8

User · Answer

This is how I do it   df ext   pd DataFrame index pd date range df index -1   periods 8  closed  right    df2   pd concat  df  df ext   axis 0  sort True  df2  forecast     df2  some column   shift 7    Basically I am generating an empty dataframe with the desired index and then just concatenate them together  But I would really like to see this as a standard feature in pandas so I have proposed an enhancement to pandas

User · Answer

I suppose imports  import pandas as pd import numpy as np   First append new row with NaN  NaN     at the end of DataFrame  df     s1   df iloc 0       copy 1st row to a new Series s1 s1      np NaN       set all values to NaN df2   df append s1  ignore index True     add s1 to the end of df   It will create new DF df2  Maybe there is more elegant way but this works   Now you can shift it   df2 x2   df2 x2 shift 1     shift what you want

User · Answer

You need to use df shift here  df shift i  shifts the entire dataframe by i units down   So  for i   1   Input       x1   x2   0  206  214   1  226  234   2  245  253   3  265  272     4  283  291   Output        x1   x2 0  Nan  Nan    1  206  214   2  226  234   3  245  253   4  265  272    So  run this script to get the expected output   import pandas as pd  df   pd DataFrame   x1     206    226    245    265    283                        x2     214    234    253    272    291      print df  df  x2     df  x2   shift 1  print df

User · Answer

Trying to answer a personal problem and similar to yours I found on Pandas Doc what I think would answer this question   DataFrame shift periods 1  freq None  axis 0  Shift index by desired number of periods with an optional time freq Notes If freq is specified then the index values are shifted but the data is not realigned  That is  use freq if you would like to extend the index when shifting and preserve the original data   Hope to help future questions in this matter

User · Answer

If you don t want to lose the columns you shift past the end of your dataframe  simply append the required number first       offset   5     DF   DF append  np nan for x in range offset        DF   DF shift periods offset      DF   DF reset index    Only works if sequential index

[python] How to shift a column in Pandas DataFrame

Examples related to python

Examples related to pandas

Examples related to dataframe