Pandas read csv from url

Question

I m trying to read a csv-file from given URL  using Python 3 x  import pandas as pd import requests  url    quot https   github com cs109 2014 data blob master countries csv quot  s   requests get url  content c   pd read csv s   I have the following error   quot Expected file path name or file-like object  got  lt class  bytes  gt  type quot   How can I fix this  I m using Python 3 4

User · Answer

In the latest version of pandas  0 19 2  you can directly pass the url  import pandas as pd  url  https   raw githubusercontent com cs109 2014 data master countries csv  c pd read csv url

User · Answer

The problem you re having is that the output you get into the variable  s  is not a csv  but a html file   In order to get the raw csv  you have to modify the url to     https   raw githubusercontent com cs109 2014 data master countries csv   Your second problem is that read csv expects a file name  we can solve this by using StringIO from io module   Third problem is that request get url  content delivers a byte stream  we can solve this using the request get url  text instead    End result is this code    from io import StringIO  import pandas as pd import requests url  https   raw githubusercontent com cs109 2014 data master countries csv  s requests get url  text  c pd read csv StringIO s     output     gt  gt  gt  c head       Country  Region 0   Algeria  AFRICA 1    Angola  AFRICA 2     Benin  AFRICA 3  Botswana  AFRICA 4   Burkina  AFRICA

User · Answer

As I commented you need to use a StringIO  object and decode i e c pd read csv io StringIO s decode  utf-8     if using requests  you need to decode as  content returns bytes if you used  text you would just need to pass s as is s   requests get url  text c   pd read csv StringIO s      A simpler approach is to pass the correct url of the raw data directly to read csv  you don t have to pass a file like object  you can pass a url so you don t need requests at all   c   pd read csv  https   raw githubusercontent com cs109 2014 data master countries csv    print c    Output                                 Country         Region 0                             Algeria         AFRICA 1                              Angola         AFRICA 2                               Benin         AFRICA 3                            Botswana         AFRICA 4                             Burkina         AFRICA 5                             Burundi         AFRICA 6                            Cameroon         AFRICA                                      From the  docs   filepath or buffer       string or file handle   StringIO   The string could be a URL  Valid URL schemes include http  ftp  s3  and file  For file URLs  a host is expected  For instance  a local file could be file    localhost path to table csv

User · Answer

url    https   github com cs109 2014 data blob master countries csv  c   pd read csv url  sep     t

User · Answer

To Import Data through URL in pandas just apply the simple below code it works actually better    import pandas as pd train   pd read table  https   urlandfile com dataset csv   train head        If you are having issues with a raw data then just put  r  before URL   import pandas as pd train   pd read table r https   urlandfile com dataset csv   train head

User · Answer

UPDATE  From pandas 0 19 2 you can now just pass read csv   the url directly  although that will fail if it requires authentication   For older pandas versions  or if you need authentication  or for any other HTTP-fault-tolerant reason  Use pandas read csv with a file-like object as the first argument   If you want to read the csv from a string  you can use io StringIO   For the URL https   github com cs109 2014 data blob master countries csv  you get html response  not raw csv  you should use the url given by the Raw link in the github page for getting raw csv response   which is https   raw githubusercontent com cs109 2014 data master countries csv   Example  import pandas as pd import io import requests url  quot https   raw githubusercontent com cs109 2014 data master countries csv quot  s requests get url  content c pd read csv io StringIO s decode  utf-8      Notes  in Python 2 x  the string-buffer object was StringIO StringIO

[python] Pandas read_csv from url

Examples related to python

Examples related to csv

Examples related to pandas

Examples related to request