How do I combine two dataframes

Question

I m using Pandas data frames  I have a initial data frame  say D  I extract two data frames from it like this  A   D D label    k  B   D D label    k   I want to combine A and B so I can have them as one DataFrame  something like a union operation  The order of the data is not important  However  when we sample A and B from D  they retain their indexes from D

User · Answer

1st dataFrame  train shape   result -   31962  3    2nd dataFrame  test shape   result -   17197  2    Combine  new data train append test ignore index True    Check  new data shape   result -   49159  3

User · Answer

There s another solution for the case that you are working with big data and need to concatenate multiple datasets  concat can get performance-intensive  so if you don t want to create a new df each time  you can instead use a list comprehension   frames     process file f  for f in dataset files   result   pd append frames     as pointed out here in the docs at the bottom of the section       Note  It is worth noting however  that concat  and therefore append    makes a full copy of the data  and that constantly reusing this   function can create a significant performance hit  If you need to use   the operation over several datasets  use a list comprehension

User · Answer

If you want to update replace the values of first dataframe df1 with the values of second dataframe df2  you can do it by following steps      Step 1  Set index of the first dataframe  df1   df1 set index  id     Step 2  Set index of the second dataframe  df2   df2 set index  id     and finally update the dataframe using the following snippet      df1 update df2

User · Answer

You can also use pd concat  which is particularly helpful when you are joining more than two dataframes   bigdata   pd concat  data1  data2   ignore index True  sort False

User · Answer

I believe you can use the append method  bigdata   data1 append data2  ignore index True    to keep their indexes just dont use the ignore index keyword

User · Answer

Thought to add this here in case someone finds it useful   ostrokach already mentioned how you can merge the data frames across rows which is  df row merged   pd concat  df a  df b   ignore index True    To merge across columns  you can use the following syntax   df col merged   pd concat  df a  df b   axis 1

[python] How do I combine two dataframes?

Examples related to python

Examples related to pandas