[python] Making a POST call instead of GET using urllib2

There's a lot of stuff out there on urllib2 and POST calls, but I'm stuck on a problem.

I'm trying to do a simple POST call to a service:

url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe',
                         'age'  : '10'})
content = urllib2.urlopen(url=url, data=data).read()
print content

I can see the server logs and it says that I'm doing GET calls, when I'm sending the data argument to urlopen.

The library is raising an 404 error (not found), which is correct for a GET call, POST calls are processed well (I'm also trying with a POST within a HTML form).

This question is related to python urllib2 urllib

The answer is

it should be sending a POST if you provide a data parameter (like you are doing):

from the docs: "the HTTP request will be a POST instead of a GET when the data parameter is provided"

so.. add some debug output to see what's up from the client side.

you can modify your code to this and try again:

import urllib
import urllib2

url = 'http://myserver/post_service'
opener = urllib2.build_opener(urllib2.HTTPHandler(debuglevel=1))
data = urllib.urlencode({'name' : 'joe',
                         'age'  : '10'})
content = opener.open(url, data=data).read()

Try this instead:

url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe',
                         'age'  : '10'})
req = urllib2.Request(url=url,data=data)
content = urllib2.urlopen(req).read()
print content

Do it in stages, and modify the object, like this:

# make a string with the request type in it:
method = "POST"
# create a handler. you can specify different handlers here (file uploads etc)
# but we go for the default
handler = urllib2.HTTPHandler()
# create an openerdirector instance
opener = urllib2.build_opener(handler)
# build a request
data = urllib.urlencode(dictionary_of_POST_fields_or_None)
request = urllib2.Request(url, data=data)
# add any other information you want
# overload the get method function with a small anonymous function...
request.get_method = lambda: method
# try it; don't forget to catch the result
    connection = opener.open(request)
except urllib2.HTTPError,e:
    connection = e

# check. Substitute with appropriate HTTP code.
if connection.code == 200:
    data = connection.read()
    # handle the error case. connection.read() will still contain data
    # if any was returned, but it probably won't be of any use

This way allows you to extend to making PUT, DELETE, HEAD and OPTIONS requests too, simply by substituting the value of method or even wrapping it up in a function. Depending on what you're trying to do, you may also need a different HTTP handler, e.g. for multi file upload.

Have a read of the urllib Missing Manual. Pulled from there is the following simple example of a POST request.

url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe', 'age'  : '10'})
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
print response.read()

As suggested by @Michael Kent do consider requests, it's great.

EDIT: This said, I do not know why passing data to urlopen() does not result in a POST request; It should. I suspect your server is redirecting, or misbehaving.

The requests module may ease your pain.

url = 'http://myserver/post_service'
data = dict(name='joe', age='10')

r = requests.post(url, data=data, allow_redirects=True)
print r.content

data["name"] = "joe"
data["age"] = "20"
data_encoded = urllib2.urlencode(data)
print urllib2.urlopen(url + "?" + data_encoded).read()

May be this can help

Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to urllib2

Need to install urllib2 for Python 3.5.1 Python handling socket.error: [Errno 104] Connection reset by peer python ignore certificate validation urllib2 Python: URLError: <urlopen error [Errno 10060] Python POST binary data Python urllib2: Receive JSON response from url How to download image using requests AttributeError("'str' object has no attribute 'read'") Python-Requests close http connection Making a POST call instead of GET using urllib2

Examples related to urllib

installing urllib in Python3.6 SSL: CERTIFICATE_VERIFY_FAILED with Python3 Python: Importing urllib.quote Python 2.7.10 error "from urllib.request import urlopen" no module named request python save image from url no module named urllib.parse (How should I install it?) 'module' has no attribute 'urlencode' urllib and "SSL: CERTIFICATE_VERIFY_FAILED" Error UnicodeEncodeError: 'charmap' codec can't encode characters replace special characters in a string python