[python] python: [Errno 10054] An existing connection was forcibly closed by the remote host

I am writing python to crawl Twitter space using Twitter-py. I have set the crawler to sleep for a while (2 seconds) between each request to api.twitter.com. However, after some times of running (around 1), when the Twitter's rate limit not exceeded yet, I got this error.

[Errno 10054] An existing connection was forcibly closed by the remote host.

What are possible causes of this problem and how to solve this?

I have searched through and found that the Twitter server itself may force to close the connection due to many requests.

Thank you very much in advance.

This question is related to python twitter web-crawler

The answer is


For me this problem arised while trying to connect to the SAP Hana database. When I got this error,

OperationalError: Lost connection to HANA server (ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

I tried to run the code for connection(mentioned below), which created that error, again and it worked.


    import pyhdb
    connection = pyhdb.connect(host="example.com",port=30015,user="user",password="secret")
    cursor = connection.cursor()
    cursor.execute("SELECT 'Hello Python World' FROM DUMMY")
    cursor.fetchone()
    connection.close()

It was because the server refused to connect. It might require you to wait for a while and try again. Try closing the Hana Studio by logging off and then logging in again. Keep running the code for a number of times.


there are many causes such as

  • The network link between server and client may be temporarily going down.
  • running out of system resources.
  • sending malformed data.

To examine the problem in detail, you can use Wireshark.

or you can just re-request or re-connect again.


I know this is a very old question but it may be that you need to set the request headers. This solved it for me.

For example 'user-agent', 'accept' etc. here is an example with user-agent:

url = 'your-url-here'
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36'}
r = requests.get(url, headers=headers)

This can be caused by the two sides of the connection disagreeing over whether the connection timed out or not during a keepalive. (Your code tries to reused the connection just as the server is closing it because it has been idle for too long.) You should basically just retry the operation over a new connection. (I'm surprised your library doesn't do this automatically.)


Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to twitter

Twitter - How to embed native video from someone else's tweet into a New Tweet or a DM How to get user's high resolution profile picture on Twitter? How to extract hours and minutes from a datetime.datetime object? Incorrect string value: '\xF0\x9F\x8E\xB6\xF0\x9F...' MySQL Twitter - share button, but with image How to access elements of a JArray (or iterate over them) Simplest PHP example for retrieving user_timeline with Twitter API version 1.1 Twitter API returns error 215, Bad Authentication Data bower command not found How do I fix twitter-bootstrap on IE?

Examples related to web-crawler

TypeError: can't use a string pattern on a bytes-like object in re.findall() Finding the layers and layer sizes for each Docker image Sending "User-agent" using Requests library in Python How to find sitemap.xml path on websites? How to request Google to re-crawl my website? python: [Errno 10054] An existing connection was forcibly closed by the remote host How to get a web page's source code from Java Python: maximum recursion depth exceeded while calling a Python object Python Web Crawlers and "getting" html source code How do I make a simple crawler in PHP?