I tried some of the suggested answers but it didn't resolve the issue. Finally I found out that the default connection timeout (not command timeout) is 15 seconds and once I increased it to 60 it almost never happened again.
simply add this to you connection string:
;Connection Timeout=60
I chose 60 seconds but you can put any value you think would fit the best to your needs.