Is there a way to widen the display of output in either interactive or script-execution mode?
Specifically, I am using the describe()
function on a pandas DataFrame
. When the DataFrame
is 5 columns (labels) wide, I get the descriptive statistics that I want. However, if the DataFrame
has any more columns, the statistics are suppressed and something like this is returned:
>> Index: 8 entries, count to max
>> Data columns:
>> x1 8 non-null values
>> x2 8 non-null values
>> x3 8 non-null values
>> x4 8 non-null values
>> x5 8 non-null values
>> x6 8 non-null values
>> x7 8 non-null values
The "8" value is given whether there are 6 or 7 columns. What does the "8" refer to?
I have already tried dragging the IDLE window larger, as well as increasing the "Configure IDLE" width options, to no avail.
My purpose in using pandas and describe()
is to avoid using a second program like Stata to do basic data manipulation and investigation.
This question is related to
python
pandas
printing
column-width
It seems like all above answers solve the problem. One more point: instead of pd.set_option('option_name')
, you can use the (auto-complete-able)
pd.options.display.width = None
See Pandas doc: Options and Settings:
Options have a full “dotted-style”, case-insensitive name (e.g.
display.max_rows
). You can get/set options directly as attributes of the top-leveloptions
attribute:In [1]: import pandas as pd In [2]: pd.options.display.max_rows Out[2]: 15 In [3]: pd.options.display.max_rows = 999 In [4]: pd.options.display.max_rows Out[4]: 999
[...]
for the max_...
params:
max_rows
andmax_columns
are used in__repr__()
methods to decide ifto_string()
orinfo()
is used to render an object to a string. In case python/IPython is running in a terminal this can be set to 0 and pandas will correctly auto-detect the width the terminal and swap to a smaller format in case all columns would not fit vertically. The IPython notebook, IPython qtconsole, or IDLE do not run in a terminal and hence it is not possible to do correct auto-detection. ‘None
’ value means unlimited. [emphasis not in original]
for the width
param:
Width of the display in characters. In case python/IPython is running in a terminal this can be set to
None
and pandas will correctly auto-detect the width. Note that the IPython notebook, IPython qtconsole, or IDLE do not run in a terminal and hence it is not possible to correctly detect the width.
You can adjust pandas print options with set_printoptions
.
In [3]: df.describe()
Out[3]:
<class 'pandas.core.frame.DataFrame'>
Index: 8 entries, count to max
Data columns:
x1 8 non-null values
x2 8 non-null values
x3 8 non-null values
x4 8 non-null values
x5 8 non-null values
x6 8 non-null values
x7 8 non-null values
dtypes: float64(7)
In [4]: pd.set_printoptions(precision=2)
In [5]: df.describe()
Out[5]:
x1 x2 x3 x4 x5 x6 x7
count 8.0 8.0 8.0 8.0 8.0 8.0 8.0
mean 69024.5 69025.5 69026.5 69027.5 69028.5 69029.5 69030.5
std 17.1 17.1 17.1 17.1 17.1 17.1 17.1
min 69000.0 69001.0 69002.0 69003.0 69004.0 69005.0 69006.0
25% 69012.2 69013.2 69014.2 69015.2 69016.2 69017.2 69018.2
50% 69024.5 69025.5 69026.5 69027.5 69028.5 69029.5 69030.5
75% 69036.8 69037.8 69038.8 69039.8 69040.8 69041.8 69042.8
max 69049.0 69050.0 69051.0 69052.0 69053.0 69054.0 69055.0
However this will not work in all cases as pandas detects your console width and it will only use to_string
if the output fits in the console (see the docstring of set_printoptions
).
In this case you can explicitly call to_string
as answered by BrenBarn.
Update
With version 0.10 the way wide dataframes are printed changed:
In [3]: df.describe()
Out[3]:
x1 x2 x3 x4 x5 \
count 8.000000 8.000000 8.000000 8.000000 8.000000
mean 59832.361578 27356.711336 49317.281222 51214.837838 51254.839690
std 22600.723536 26867.192716 28071.737509 21012.422793 33831.515761
min 31906.695474 1648.359160 56.378115 16278.322271 43.745574
25% 45264.625201 12799.540572 41429.628749 40374.273582 29789.643875
50% 56340.214856 18666.456293 51995.661512 54894.562656 47667.684422
75% 75587.003417 31375.610322 61069.190523 67811.893435 76014.884048
max 98136.474782 84544.484627 91743.983895 75154.587156 99012.695717
x6 x7
count 8.000000 8.000000
mean 41863.000717 33950.235126
std 38709.468281 29075.745673
min 3590.990740 1833.464154
25% 15145.759625 6879.523949
50% 22139.243042 33706.029946
75% 72038.983496 51449.893980
max 98601.190488 83309.051963
Further more the API for setting pandas options changed:
In [4]: pd.set_option('display.precision', 2)
In [5]: df.describe()
Out[5]:
x1 x2 x3 x4 x5 x6 x7
count 8.0 8.0 8.0 8.0 8.0 8.0 8.0
mean 59832.4 27356.7 49317.3 51214.8 51254.8 41863.0 33950.2
std 22600.7 26867.2 28071.7 21012.4 33831.5 38709.5 29075.7
min 31906.7 1648.4 56.4 16278.3 43.7 3591.0 1833.5
25% 45264.6 12799.5 41429.6 40374.3 29789.6 15145.8 6879.5
50% 56340.2 18666.5 51995.7 54894.6 47667.7 22139.2 33706.0
75% 75587.0 31375.6 61069.2 67811.9 76014.9 72039.0 51449.9
max 98136.5 84544.5 91744.0 75154.6 99012.7 98601.2 83309.1
If you want to set options temporarily to display one large DataFrame, you can use option_context:
with pd.option_context('display.max_rows', None, 'display.max_columns', None):
print (df)
Option values are restored automatically when you exit the with
block.
I used these settings when scale of data is high.
# environment settings:
pd.set_option('display.max_column',None)
pd.set_option('display.max_rows',None)
pd.set_option('display.max_seq_items',None)
pd.set_option('display.max_colwidth', 500)
pd.set_option('expand_frame_repr', True)
You can refer to the documentation here
You can simply do the following steps,
You can change the options for pandas max_columns feature as follows
import pandas as pd
pd.options.display.max_columns = 10
(this allows 10 columns to display, you can change this as you need)
Like that you can change the number of rows as you need to display as follows (if you need to change maximum rows as well)
pd.options.display.max_rows = 999
(this allows to print 999 rows at a time)
Please kindly refer the doc to change different options/settings for pandas
The below line is enough to display all columns from dataframe.
pd.set_option('display.max_columns', None)
According to the docs for v0.18.0, if you're running on a terminal (ie not iPython notebook, qtconsole or IDLE), it's a 2-liner to have Pandas auto-detect your screen width and adapt on the fly with how many columns it shows:
pd.set_option('display.large_repr', 'truncate')
pd.set_option('display.max_columns', 0)
If you don't want to mess with your display options and you just want to see this one particular list of columns without expanding out every dataframe you view, you could try:
df.columns.values
Only using these 3 lines worked for me:
pd.set_option('display.max_columns', None)
pd.set_option('display.expand_frame_repr', False)
pd.set_option('max_colwidth', -1)
Anaconda / Python 3.6.5 / pandas: 0.23.0 / Visual Studio Code 1.26
You can set the output display to match your current terminal width:
pd.set_option('display.width', pd.util.terminal.get_terminal_size()[0])
You can also try in a loop:
for col in df.columns:
print(col)
pd.options.display.max_columns = 100
You can specify the numbers of columns as per your requirement in max_columns.
Try this:
pd.set_option('display.expand_frame_repr', False)
From the documentation:
display.expand_frame_repr : boolean
Whether to print out the full DataFrame repr for wide DataFrames across multiple lines, max_columns is still respected, but the output will wrap-around across multiple “pages” if it’s width exceeds display.width. [default: True] [currently: True]
See: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.set_option.html
You can use print df.describe().to_string()
to force it to show the whole table. (You can use to_string()
like this for any DataFrame. The result of describe
is just a DataFrame itself.)
The 8 is the number of rows in the DataFrame holding the "description" (because describe
computes 8 statistics, min, max, mean, etc.).
None of these answers were working for me. A couple of them would indeed print all the columns, but it would look sloppy. As in all the information was there, but it wasn't formatted correctly. I'm using a terminal inside of Neovim so I suspect that to be the reason.
This mini function does exactly what I need, just change df_data
in the two places it is for your dataframe name (col_range
is set to what pandas naturally shows, for me it is 5 but it could be bigger or smaller for you).
import math
col_range = 5
for _ in range(int(math.ceil(len(df_data.columns)/col_range))):
idx1 = _*col_range
idx2 = idx1+col_range
print(df_data.iloc[:, idx1:idx2].describe())
You can use this custom function for displaying things for pandas Dataframe
.
def display_all(df): # for any Dataframe df
with pd.option_context('display.max_rows',1000): # change number of rows accordingly
with pd.option_context('display.max_columns',1000): # change number of columns accordingly
display(df)
display_all(df.head()) # pass this function to your dataframe and Voila!
You don't have use pd.set_option
for whole notebook just use for single cell.
Set column max width using:
pd.set_option('max_colwidth', 800)
This particular statement sets max width to 800px, per column.
import pandas as pd
pd.set_option('display.max_columns', 100)
pd.set_option('display.width', 1000)
SentenceA = "William likes Piano and Piano likes William"
SentenceB = "Sara likes Guitar"
SentenceC = "Mamoosh likes Piano"
SentenceD = "William is a CS Student"
SentenceE = "Sara is kind"
SentenceF = "Mamoosh is kind"
bowA = SentenceA.split(" ")
bowB = SentenceB.split(" ")
bowC = SentenceC.split(" ")
bowD = SentenceD.split(" ")
bowE = SentenceE.split(" ")
bowF = SentenceF.split(" ")
# Creating a set consisted of all words
wordSet = set(bowA).union(set(bowB)).union(set(bowC)).union(set(bowD)).union(set(bowE)).union(set(bowF))
print("Set of all words is: ", wordSet)
# Initiating dictionary with 0 value for all BOWs
wordDictA = dict.fromkeys(wordSet, 0)
wordDictB = dict.fromkeys(wordSet, 0)
wordDictC = dict.fromkeys(wordSet, 0)
wordDictD = dict.fromkeys(wordSet, 0)
wordDictE = dict.fromkeys(wordSet, 0)
wordDictF = dict.fromkeys(wordSet, 0)
for word in bowA:
wordDictA[word] += 1
for word in bowB:
wordDictB[word] += 1
for word in bowC:
wordDictC[word] += 1
for word in bowD:
wordDictD[word] += 1
for word in bowE:
wordDictE[word] += 1
for word in bowF:
wordDictF[word] += 1
# Printing Term frequency
print("SentenceA TF: ", wordDictA)
print("SentenceB TF: ", wordDictB)
print("SentenceC TF: ", wordDictC)
print("SentenceD TF: ", wordDictD)
print("SentenceE TF: ", wordDictE)
print("SentenceF TF: ", wordDictF)
print(pd.DataFrame([wordDictA, wordDictB, wordDictB, wordDictC, wordDictD, wordDictE, wordDictF]))
OutPut:
CS Guitar Mamoosh Piano Sara Student William a and is kind likes
0 0 0 0 2 0 0 2 0 1 0 0 2
1 0 1 0 0 1 0 0 0 0 0 0 1
2 0 1 0 0 1 0 0 0 0 0 0 1
3 0 0 1 1 0 0 0 0 0 0 0 1
4 1 0 0 0 0 1 1 1 0 1 0 0
5 0 0 0 0 1 0 0 0 0 1 1 0
6 0 0 1 0 0 0 0 0 0 1 1 0
Source: Stackoverflow.com