Python POST binary data

Question

I am writing some code to interface with redmine and I need to upload some files as part of the process  but I am not sure how to do a POST request from python containing a binary file   I am trying to mimic the commands here   curl --data-binary   image png  -H  Content-Type  application octet-stream  -X POST -u login password http   redmine uploads xml   In python  below   but it does not seem to work  I am not sure if the problem is somehow related to encoding the file or if something is wrong with the headers   import urllib2  os  FilePath    C  somefolder somefile 7z  FileData   open FilePath   rb   length   os path getsize FilePath   password manager   urllib2 HTTPPasswordMgrWithDefaultRealm   password manager add password None   http   redmine     admin    admin   auth handler   urllib2 HTTPBasicAuthHandler password manager  opener   urllib2 build opener auth handler  urllib2 install opener opener  request   urllib2 Request  r http   redmine uploads xml   FileData  request add header  Content-Length     d    length  request add header  Content-Type    application octet-stream   try      response   urllib2 urlopen  request      print response read   except urllib2 HTTPError as e      error message   e read       print error message   I have access to the server and it looks like a encoding error       invalid byte sequence in UTF-8 Line  1 Position  624 Last 80 unconsumed characters  7z             2       4g  R lt s      6k             jcdjSP    -       AtD  H7        j     further down   Started POST   uploads xml  for 192 168 0 117 at 2013-01-16 09 57 49 -0800 Processing by AttachmentsController upload as XML WARNING  Can t verify CSRF token authenticity   Current user  anonymous Filter chain halted as  authorize global rendered or redirected Completed 401 Unauthorized in 13ms  ActiveRecord  3 1ms

User · Accepted Answer

Basically what you do is correct  Looking at redmine docs you linked to  it seems that suffix after the dot in the url denotes type of posted data   json for JSON   xml for XML   which agrees with the response you get - Processing by AttachmentsController upload as XML  I guess maybe there s a bug in docs and to post binary data you should try using http   redmine uploads url instead of http   redmine uploads xml  Btw  I highly recommend very good and very popular Requests library for http in Python  It s much better than what s in the standard lib  urllib2   It supports authentication as well but I skipped it for brevity here  import requests with open    x png    rb   as f      data   f read   res   requests post url  http   httpbin org post                       data data                      headers   Content-Type    application octet-stream       let s check if what we sent is what we intended to send    import json import base64  assert base64 b64decode res json    data   len  data application octet-stream base64          data  UPDATE To find out why this works with Requests but not with urllib2 we have to examine the difference in what s being sent  To see this I m sending traffic to http proxy  Fiddler  running on port 8888  Using Requests import requests  data    test data  res   requests post url  http   localhost 8888                       data data                      headers   Content-Type    application octet-stream     we see POST http   localhost 8888  HTTP 1 1 Host  localhost 8888 Content-Length  9 Content-Type  application octet-stream Accept-Encoding  gzip  deflate  compress Accept      User-Agent  python-requests 1 0 4 CPython 2 7 3 Windows Vista  test data  and using urllib2 import urllib2  data    test data      req   urllib2 Request  http   localhost 8888   data  req add header  Content-Length     d    len data   req add header  Content-Type    application octet-stream   res   urllib2 urlopen req   we get POST http   localhost 8888  HTTP 1 1 Accept-Encoding  identity Content-Length  9 Host  localhost 8888 Content-Type  application octet-stream Connection  close User-Agent  Python-urllib 2 7  test data  I don t see any differences which would warrant different behavior you observe  Having said that it s not uncommon for http servers to inspect User-Agent header and vary behavior based on its value  Try to change headers sent by Requests one by one making them the same as those being sent by urllib2 and see when it stops working

User · Answer

This has nothing to do with a malformed upload  The HTTP error clearly specifies 401 unauthorized  and tells you the CSRF token is invalid  Try sending a valid CSRF token with the upload   More about csrf tokens here       What is a CSRF token   What is its importance and how does it work

User · Answer

You can use unirest  It provides easy method to post request    import unirest   def callback response    print  quot code  quot   str response code   print  quot                    quot   print  quot headers  quot   str response headers   print  quot                    quot   print  quot body  quot   str response body   print  quot                    quot   print  quot raw body  quot   str response raw body      consume async post request def consumePOSTRequestASync     params     test1   param1   test2   param2        we need to pass a dummy variable which is open method    actually unirest does not provide variable to shift between    application-x-www-form-urlencoded and    multipart form-data     params  dummy     open  dummy txt    r    url    http   httpbin org post   headers     quot Accept quot    quot application json quot      call get service with headers and params  unirest post url  headers   headers params   params  callback   callback        post async request multipart form-data consumePOSTRequestASync

User · Answer

you need to add Content-Disposition header  smth like this  although I used mod-python here  but principle should be the same    request headers out  Content-Disposition      attachment  filename  s    myfname

[python] Python POST binary data

Examples related to python

Examples related to rest

Examples related to post

Examples related to urllib2

Examples related to redmine