[python] Is it possible to append Series to rows of DataFrame without making a list first?

I have some data I'm trying to organize into a DataFrame in Pandas. I was trying to make each row a Series and append it to the DataFrame. I found a way to do it by appending the Series to an empty list and then converting the list of Series to a DataFrame

e.g. DF = DataFrame([series1,series2],columns=series1.index)

This list to DataFrame step seems to be excessive. I've checked out a few examples on here but none of the Series preserved the Index labels from the Series to use them as column labels.

My long way where columns are id_names and rows are type_names: enter image description here

Is it possible to append Series to rows of DataFrame without making a list first?

#!/usr/bin/python

DF = DataFrame()
for sample,data in D_sample_data.items():
    SR_row = pd.Series(data.D_key_value)
    DF.append(SR_row)
DF.head()

TypeError: Can only append a Series if ignore_index=True or if the Series has a name

Then I tried

DF = DataFrame()
for sample,data in D_sample_data.items():
    SR_row = pd.Series(data.D_key_value,name=sample)
    DF.append(SR_row)
DF.head()

Empty DataFrame

Tried Insert a row to pandas dataframe Still getting an empty dataframe :/

I am trying to get the Series to be the rows, where the index of the Series becomes the column labels of the DataFrame

This question is related to python pandas machine-learning dataframe series

The answer is


Maybe an easier way would be to add the pandas.Series into the pandas.DataFrame with ignore_index=True argument to DataFrame.append(). Example -

DF = DataFrame()
for sample,data in D_sample_data.items():
    SR_row = pd.Series(data.D_key_value)
    DF = DF.append(SR_row,ignore_index=True)

Demo -

In [1]: import pandas as pd

In [2]: df = pd.DataFrame([[1,2],[3,4]],columns=['A','B'])

In [3]: df
Out[3]:
   A  B
0  1  2
1  3  4

In [5]: s = pd.Series([5,6],index=['A','B'])

In [6]: s
Out[6]:
A    5
B    6
dtype: int64

In [36]: df.append(s,ignore_index=True)
Out[36]:
   A  B
0  1  2
1  3  4
2  5  6

Another issue in your code is that DataFrame.append() is not in-place, it returns the appended dataframe, you would need to assign it back to your original dataframe for it to work. Example -

DF = DF.append(SR_row,ignore_index=True)

To preserve the labels, you can use your solution to include name for the series along with assigning the appended DataFrame back to DF. Example -

DF = DataFrame()
for sample,data in D_sample_data.items():
    SR_row = pd.Series(data.D_key_value,name=sample)
    DF = DF.append(SR_row)
DF.head()

Something like this could work...

mydf.loc['newindex'] = myseries

Here is an example where I used it...

stats = df[['bp_prob', 'ICD9_prob', 'meds_prob', 'regex_prob']].describe()

stats
Out[32]: 
          bp_prob   ICD9_prob   meds_prob  regex_prob
count  171.000000  171.000000  171.000000  171.000000
mean     0.179946    0.059071    0.067020    0.126812
std      0.271546    0.142681    0.152560    0.207014
min      0.000000    0.000000    0.000000    0.000000
25%      0.000000    0.000000    0.000000    0.000000
50%      0.000000    0.000000    0.000000    0.013116
75%      0.309019    0.065248    0.066667    0.192954
max      1.000000    1.000000    1.000000    1.000000

medians = df[['bp_prob', 'ICD9_prob', 'meds_prob', 'regex_prob']].median()

stats.loc['median'] = medians

stats
Out[36]: 
           bp_prob   ICD9_prob   meds_prob  regex_prob
count   171.000000  171.000000  171.000000  171.000000
mean      0.179946    0.059071    0.067020    0.126812
std       0.271546    0.142681    0.152560    0.207014
min       0.000000    0.000000    0.000000    0.000000
25%       0.000000    0.000000    0.000000    0.000000
50%       0.000000    0.000000    0.000000    0.013116
75%       0.309019    0.065248    0.066667    0.192954
max       1.000000    1.000000    1.000000    1.000000
median    0.000000    0.000000    0.000000    0.013116

Try using this command. See the example given below:

Before image

df.loc[len(df)] = ['Product 9',99,9.99,8.88,1.11]

df

After Image


Convert the series to a dataframe and transpose it, then append normally.

srs = srs.to_frame().T
df = df.append(srs)

DataFrame.append does not modify the DataFrame in place. You need to do df = df.append(...) if you want to reassign it back to the original variable.


This would work as well:

df = pd.DataFrame()
new_line = pd.Series({'A2M': 4.059, 'A2ML1': 4.28}, name='HCC1419')
df = df.append(new_line, ignore_index=False)

The name in the Series will be the index in the dataframe. ignore_index=False is the important flag in this case.


Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to pandas

xlrd.biffh.XLRDError: Excel xlsx file; not supported Pandas Merging 101 How to increase image size of pandas.DataFrame.plot in jupyter notebook? Trying to merge 2 dataframes but get ValueError Python Pandas User Warning: Sorting because non-concatenation axis is not aligned How to show all of columns name on pandas dataframe? Pandas/Python: Set value of one column based on value in another column Python Pandas - Find difference between two data frames Pandas get the most frequent values of a column Python convert object to float

Examples related to machine-learning

Error in Python script "Expected 2D array, got 1D array instead:"? How to predict input image using trained model in Keras? What is the role of "Flatten" in Keras? How to concatenate two layers in keras? How to save final model using keras? scikit-learn random state in splitting dataset Why binary_crossentropy and categorical_crossentropy give different performances for the same problem? What is the meaning of the word logits in TensorFlow? Can anyone explain me StandardScaler? Can Keras with Tensorflow backend be forced to use CPU or GPU at will?

Examples related to dataframe

Trying to merge 2 dataframes but get ValueError How to show all of columns name on pandas dataframe? Python Pandas - Find difference between two data frames Pandas get the most frequent values of a column Display all dataframe columns in a Jupyter Python Notebook How to convert column with string type to int form in pyspark data frame? Display/Print one column from a DataFrame of Series in Pandas Binning column with python pandas Selection with .loc in python Set value to an entire column of a pandas dataframe

Examples related to series

Display/Print one column from a DataFrame of Series in Pandas Python Pandas iterate over rows and access column names Extract values in Pandas value_counts() Convert pandas data frame to series Is it possible to append Series to rows of DataFrame without making a list first? assigning column names to a pandas series Convert pandas Series to DataFrame Get first element of Series without knowing the index Pandas: change data type of Series to String Conditional Replace Pandas