[python] Python-Requests close http connection

I was wondering, how do you close a connection with Requests (python-requests.org)?

With httplib it's HTTPConnection.close(), but how do I do the same with Requests?

Code:

r = requests.post("https://stream.twitter.com/1/statuses/filter.json", data={'track':toTrack}, auth=('username', 'passwd'))
for line in r.iter_lines():
    if line:
        self.mongo['db'].tweets.insert(json.loads(line))

This question is related to python http urllib2 httplib python-requests

The answer is


please use response.close() to close to avoid "too many open files" error

for example:

r = requests.post("https://stream.twitter.com/1/statuses/filter.json", data={'track':toTrack}, auth=('username', 'passwd'))
....
r.close()

this works for me:

res = requests.get(<url>, timeout=10).content
requests.session().close()

To remove the "keep-alive" header in requests, I just created it from the Request object and then send it with Session

headers = {
'Host' : '1.2.3.4',
'User-Agent' : 'Test client (x86_64-pc-linux-gnu 7.16.3)',
'Accept' : '*/*',
'Accept-Encoding' : 'deflate, gzip',
'Accept-Language' : 'it_IT'
}

url = "https://stream.twitter.com/1/statuses/filter.json"
#r = requests.get(url, headers = headers) #this triggers keep-alive: True
s = requests.Session()
r = requests.Request('GET', url, headers)

I came to this question looking to solve the "too many open files" error, but I am using requests.session() in my code. A few searches later and I came up with an answer on the Python Requests Documentation which suggests to use the with block so that the session is closed even if there are unhandled exceptions:

with requests.Session() as s:
    s.get('http://google.com')

If you're not using Session you can actually do the same thing: https://2.python-requests.org/en/master/user/advanced/#session-objects

with requests.get('http://httpbin.org/get', stream=True) as r:
    # Do something

I think a more reliable way of closing a connection is to tell the sever explicitly to close it in a way compliant with HTTP specification:

HTTP/1.1 defines the "close" connection option for the sender to signal that the connection will be closed after completion of the response. For example,

   Connection: close

in either the request or the response header fields indicates that the connection SHOULD NOT be considered `persistent' (section 8.1) after the current request/response is complete.

The Connection: close header is added to the actual request:

r = requests.post(url=url, data=body, headers={'Connection':'close'})

On Requests 1.X, the connection is available on the response object:

r = requests.post("https://stream.twitter.com/1/statuses/filter.json",
                  data={'track': toTrack}, auth=('username', 'passwd'))

r.connection.close()

Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to http

Access blocked by CORS policy: Response to preflight request doesn't pass access control check Axios Delete request with body and headers? Read response headers from API response - Angular 5 + TypeScript Android 8: Cleartext HTTP traffic not permitted Angular 4 HttpClient Query Parameters Load json from local file with http.get() in angular 2 Angular 2: How to access an HTTP response body? What is HTTP "Host" header? Golang read request body Angular 2 - Checking for server errors from subscribe

Examples related to urllib2

Need to install urllib2 for Python 3.5.1 Python handling socket.error: [Errno 104] Connection reset by peer python ignore certificate validation urllib2 Python: URLError: <urlopen error [Errno 10060] Python POST binary data Python urllib2: Receive JSON response from url How to download image using requests AttributeError("'str' object has no attribute 'read'") Python-Requests close http connection Making a POST call instead of GET using urllib2

Examples related to httplib

How to send POST request? Python-Requests close http connection HTTPS connection Python

Examples related to python-requests

Requests (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.") Error in PyCharm requesting website Use python requests to download CSV Download and save PDF file with Python requests module ImportError: No module named 'Queue' How to use cookies in Python Requests Saving response from Requests to file How to get Python requests to trust a self signed SSL certificate? How to install requests module in Python 3.4, instead of 2.7 Upload Image using POST form data in Python-requests SSL InsecurePlatform error when using Requests package