Read a zipped file as a pandas DataFrame

Question

I m trying to unzip a csv file and pass it into pandas so I can work on the file  The code I have tried so far is    import requests  zipfile  StringIO r   requests get  http   data octo dc gov feeds crime incidents archive crime incidents 2013 CSV zip   z   zipfile ZipFile StringIO StringIO r content   crime2013   pandas read csv z read  crime incidents 2013 CSV csv      After the last line  although python is able to get the file  I get a  does not exist  at the end of the error   Can someone tell me what I m doing incorrectly

User · Answer

If you want to read a zipped or a tar gz file into pandas dataframe  the read csv methods includes this particular implementation   df   pd read csv  filename zip     Or the long form       df   pd read csv  filename zip   compression  zip   header 0  sep      quotechar        Description of the compression argument from the docs      compression       infer        gzip        bz2        zip        xz     None   default    infer      For on-the-fly decompression of on-disk data  If    infer    and filepath or buffer is path-like  then detect compression from the following extensions      gz         bz2         zip     or     xz     otherwise no decompression   If using    zip     the ZIP file must contain only one data file to be read in  Set to None for no decompression       New in version 0 18 1  support for    zip    and    xz    compression

User · Answer

It seems you don t even have to specify the compression any more  The following snippet loads the data from filename zip into df   import pandas as pd df   pd read csv  filename zip      Of course you will need to specify separator  header  etc  if they are different from the defaults

User · Answer

For  zip  files  you can use import zipfile and your code will be working simply with these lines   import zipfile import pandas as pd with zipfile ZipFile  Crime Incidents in 2013 zip   as z     with z open  Crime Incidents in 2013 csv   as f        train   pd read csv f  header 0  delimiter   t         print train head         print the first 5 rows   And the result will be   X Y CCN REPORT DAT SHIFT METHOD OFFENSE BLOCK XBLOCK YBLOCK WARD ANC DISTRICT PSA NEIGHBORHOOD CLUSTER BLOCK GROUP CENSUS TRACT VOTING PRECINCT XCOORD YCOORD LATITUDE LONGITUDE BID START DATE END DATE OBJECTID 0  -77 054968548763071 38 899775938598317 0925135                                                                                                                                                                   1  -76 967309569035052 38 872119553647011 1003352                                                                                                                                                                   2  -76 996184958456539 38 927921847721443 1101010                                                                                                                                                                   3  -76 943077541353617 38 883686046653935 1104551                                                                                                                                                                   4  -76 939209158039446 38 892278093281632 1125028

User · Answer

I think you want to open the ZipFile  which returns a file-like object   rather than read   In  11   crime2013   pd read csv z open  crime incidents 2013 CSV csv     In  12   crime2013 Out 12    lt class  pandas core frame DataFrame  gt  Int64Index  24567 entries  0 to 24566 Data columns  total 15 columns   CCN                            24567  non-null values REPORTDATETIME                 24567  non-null values SHIFT                          24567  non-null values OFFENSE                        24567  non-null values METHOD                         24567  non-null values LASTMODIFIEDDATE               24567  non-null values BLOCKSITEADDRESS               24567  non-null values BLOCKXCOORD                    24567  non-null values BLOCKYCOORD                    24567  non-null values WARD                           24563  non-null values ANC                            24567  non-null values DISTRICT                       24567  non-null values PSA                            24567  non-null values NEIGHBORHOODCLUSTER            24263  non-null values BUSINESSIMPROVEMENTDISTRICT    3613  non-null values dtypes  float64 4   int64 1   object 10

User · Answer

https   www kaggle com jboysen quick-gz-pandas-tutorial Please follow this link  import pandas as pd traffic station df   pd read csv  C   Folders  Jupiter Feed txt gz   compression  gzip                                    header 1  sep   t   quotechar   quot      traffic station df  Address      address    traffic station df append traffic station df  print traffic station df

[python] Read a zipped file as a pandas DataFrame

Examples related to python

Examples related to zip

Examples related to pandas