pandas read csv index col None not working with delimiters at the end of each line

Question

I am going through the  Python for Data Analysis  book and having trouble in the  Example  2012 Federal Election Commision Database  section reading the data to a DataFrame  The trouble is that one of the columns of data is always being set as the index column  even when the index col argument is set to None    Here is the link to the data   http   www fec gov disclosurep PDownload do   Here is the loading code  to save time in the checking  I set the nrows 10    import pandas as pd fec   pd read csv  P00000001-ALL csv  nrows 10 index col None    To keep it short I am excluding the data column outputs  but here is my output  please not the Index values    In  20   fec  Out 20    lt class  pandas core frame DataFrame  gt  Index  10 entries  C00410118 to C00410118 Data columns      dtypes  float64 4   int64 3   object 11    And here is the book s output  again with data columns excluded    In  13   fec   read csv  P00000001-ALL csv   In  14   fec Out 14    lt class  pandas core frame DataFrame  gt  Int64Index  1001731 entries  0 to 1001730     dtypes  float64 1   int64 1   object 14    The Index values in my output are actually the first column of data in the file  which is then moving all the rest of the data to the left by one  Would anyone know how to prevent this column of data to be listed as an index  I would like to have the index just  1 increasing integers   I am fairly new to python and pandas  so I apologize for any inconvenience  Thanks

User · Accepted Answer

Quick Answer Use index col False instead of index col None when you have delimiters at the end of each line to turn off index column inference and discard the last column  More Detail After looking at the data  there is a comma at the end of each line  And this  quote  the documentation has been edited since the time this post was created    index col  column number  column name  or list of column numbers names  to use as the index  row labels  of the resulting DataFrame  By default  it will number the rows without using any column  unless there is one more data column than there are headers  in which case the first column is taken as the index   from the documentation shows that pandas believes you have n headers and n 1 data columns and is treating the first column as the index   EDIT 10 20 2014 - More information I found another valuable entry that is specifically about trailing limiters and how to simply ignore them   If a file has one more column of data than the number of column names  the first column will be used as the DataFrame   s row names      Ordinarily  you can achieve this behavior using the index col option  There are some exception cases when a file has been prepared with delimiters at the end of each data line  confusing the parser  To explicitly disable the index column inference and discard the last column  pass index col False

User · Answer

Re  craigts s response  for anyone having trouble with using either False or None parameters for index col  such as in cases where you re trying to get rid of a range index  you can instead use an integer to specify the column you want to use as the index  For example   df   pd read csv  file csv   index col 0    The above will set the first column as the index  and not add a range index in my  common case     Update  Given the popularity of this answer  I thought i d add some context  a demo     Setting up the dummy data In  1   df   pd DataFrame   A   1  2  3    B   4  5  6     In  2   df Out 2      A  B 0  1  4 1  2  5 2  3  6  In  3   df to csv  file csv   index None  File 3   A  B 1  4 2  5 3  6   Reading without index col or with None False will all result in a range index   In  4   pd read csv  file csv   Out 4      A  B 0  1  4 1  2  5 2  3  6    Note that this is the default behavior  so the same as In  4  In  5   pd read csv  file csv   index col None  Out 5      A  B 0  1  4 1  2  5 2  3  6  In  6   pd read csv  file csv   index col False  Out 6      A  B 0  1  4 1  2  5 2  3  6   However  if we specify that  A   the 0th column  is actually the index  we can avoid the range index   In  7   pd read csv  file csv   index col 0  Out 7      B A 1  4 2  5 3  6

[python] pandas read_csv index_col=None not working with delimiters at the end of each line

Quick Answer

More Detail

Examples related to python

Examples related to pandas