I was wondering, how do you close a connection with Requests (python-requests.org)?
With httplib
it's HTTPConnection.close()
, but how do I do the same with Requests?
Code:
r = requests.post("https://stream.twitter.com/1/statuses/filter.json", data={'track':toTrack}, auth=('username', 'passwd'))
for line in r.iter_lines():
if line:
self.mongo['db'].tweets.insert(json.loads(line))
This question is related to
python
http
urllib2
httplib
python-requests
please use response.close()
to close to avoid "too many open files" error
for example:
r = requests.post("https://stream.twitter.com/1/statuses/filter.json", data={'track':toTrack}, auth=('username', 'passwd'))
....
r.close()
this works for me:
res = requests.get(<url>, timeout=10).content
requests.session().close()
To remove the "keep-alive" header in requests, I just created it from the Request object and then send it with Session
headers = {
'Host' : '1.2.3.4',
'User-Agent' : 'Test client (x86_64-pc-linux-gnu 7.16.3)',
'Accept' : '*/*',
'Accept-Encoding' : 'deflate, gzip',
'Accept-Language' : 'it_IT'
}
url = "https://stream.twitter.com/1/statuses/filter.json"
#r = requests.get(url, headers = headers) #this triggers keep-alive: True
s = requests.Session()
r = requests.Request('GET', url, headers)
I came to this question looking to solve the "too many open files" error
, but I am using requests.session()
in my code. A few searches later and I came up with an answer on the Python Requests Documentation which suggests to use the with
block so that the session is closed even if there are unhandled exceptions:
with requests.Session() as s:
s.get('http://google.com')
If you're not using Session you can actually do the same thing: https://2.python-requests.org/en/master/user/advanced/#session-objects
with requests.get('http://httpbin.org/get', stream=True) as r:
# Do something
I think a more reliable way of closing a connection is to tell the sever explicitly to close it in a way compliant with HTTP specification:
HTTP/1.1 defines the "close" connection option for the sender to signal that the connection will be closed after completion of the response. For example,
Connection: close
in either the request or the response header fields indicates that the connection SHOULD NOT be considered `persistent' (section 8.1) after the current request/response is complete.
The Connection: close
header is added to the actual request:
r = requests.post(url=url, data=body, headers={'Connection':'close'})
On Requests 1.X, the connection is available on the response object:
r = requests.post("https://stream.twitter.com/1/statuses/filter.json",
data={'track': toTrack}, auth=('username', 'passwd'))
r.connection.close()
Source: Stackoverflow.com