[python] Pandas: Setting no. of max rows

I have a problem viewing the following DataFrame:

n = 100
foo = DataFrame(index=range(n))
foo['floats'] = np.random.randn(n)
foo

The problem is that it does not print all rows per default in ipython notebook, but I have to slice to view the resulting rows. Even the following option does not change the output:

pd.set_option('display.max_rows', 500)

Does anyone know how to display the whole array?

This question is related to python formatting pandas ipython-notebook

The answer is


Personally, I like setting the options directly with an assignment statement as it is easy to find via tab completion thanks to iPython. I find it hard to remember what the exact option names are, so this method works for me.

For instance, all I have to remember is that it begins with pd.options

pd.options.<TAB>

enter image description here

Most of the options are available under display

pd.options.display.<TAB>

enter image description here

From here, I usually output what the current value is like this:

pd.options.display.max_rows
60

I then set it to what I want it to be:

pd.options.display.max_rows = 100

Also, you should be aware of the context manager for options, which temporarily sets the options inside of a block of code. Pass in the option name as a string followed by the value you want it to be. You may pass in any number of options in the same line:

with pd.option_context('display.max_rows', 100, 'display.max_columns', 10):
    some pandas stuff

You can also reset an option back to its default value like this:

pd.reset_option('display.max_rows')

And reset all of them back:

pd.reset_option('all')

It is still perfectly good to set options via pd.set_option. I just find using the attributes directly is easier and there is less need for get_option and set_option.


As in this answer to a similar question, there is no need to hack settings. It is much simpler to write:

print(foo.to_string())

to set unlimited number of rows use

None

i.e.,

pd.set_option('display.max_cols', None)

now the notebook will display all the rows in all datasets within the notebook ;)

Similarly you can set to show all columns as

pd.set_option('display.max_rows', None)

now if you use run the cell with only dataframe with out any head or tail tags as

df

then it will show all the rows and columns in the dataframe df


As @hanleyhansen noted in a comment, as of version 0.18.1, the display.height option is deprecated, and says "use display.max_rows instead". So you just have to configure it like this:

pd.set_option('display.max_rows', 500)

See the Release Notes — pandas 0.18.1 documentation:

Deprecated display.height, display.width is now only a formatting option does not control triggering of summary, similar to < 0.11.0.


pd.set_option('display.max_rows', 500)
df

Does not work in Jupyter!
Instead use:

pd.set_option('display.max_rows', 500)
df.head(500)

It was already pointed in this comment and in this answer, but I'll try to give a more direct answer to the question:

from IPython.display import display
import numpy as np
import pandas as pd

n = 100
foo = pd.DataFrame(index=range(n))
foo['floats'] = np.random.randn(n)

with pd.option_context("display.max_rows", foo.shape[0]):
    display(foo)

pandas.option_context is available since pandas 0.13.1 (pandas 0.13.1 release notes). According to this,

[it] allow[s] you to execute a codeblock with a set of options that revert to prior settings when you exit the with block.


Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to formatting

How to add empty spaces into MD markdown readme on GitHub? VBA: Convert Text to Number How to change indentation in Visual Studio Code? How do you change the formatting options in Visual Studio Code? (Excel) Conditional Formatting based on Adjacent Cell Value 80-characters / right margin line in Sublime Text 3 Format certain floating dataframe columns into percentage in pandas Format JavaScript date as yyyy-mm-dd AngularJS format JSON string output converting multiple columns from character to numeric format in r

Examples related to pandas

xlrd.biffh.XLRDError: Excel xlsx file; not supported Pandas Merging 101 How to increase image size of pandas.DataFrame.plot in jupyter notebook? Trying to merge 2 dataframes but get ValueError Python Pandas User Warning: Sorting because non-concatenation axis is not aligned How to show all of columns name on pandas dataframe? Pandas/Python: Set value of one column based on value in another column Python Pandas - Find difference between two data frames Pandas get the most frequent values of a column Python convert object to float

Examples related to ipython-notebook

collapse cell in jupyter notebook Simple way to measure cell execution time in ipython notebook Using both Python 2.x and Python 3.x in IPython Notebook How do I add python3 kernel to jupyter (IPython) How to hide code from cells in ipython notebook visualized with nbviewer? Show DataFrame as table in iPython Notebook How to show PIL Image in ipython notebook ipython notebook clear cell output in code Format certain floating dataframe columns into percentage in pandas Pandas - Plotting a stacked Bar Chart