Adjusting HttpWebRequest Connection Timeout in C

Question

I believe after lengthy research and searching  I have discovered that what I want to do is probably better served by setting up an asynchronous connection and terminating it after the desired timeout    But I will go ahead and ask anyway   Quick snippet of code   HttpWebRequest webReq    HttpWebRequest HttpWebRequest Create url   webReq Timeout   5000  HttpWebResponse response    HttpWebResponse webReq GetResponse        this takes  20  sec on servers that aren t on the proper port  etc    I have an HttpWebRequest method that is in a multi-threaded application  in which I am connecting to a large number of company web servers  In cases where the server is not responding  the HttpWebRequest GetResponse   is taking about 20 seconds to time out  even though I have specified a timeout of only 5 seconds  In the interest of getting through the servers on a regular interval  I want to skip those taking longer than 5 seconds to connect to   So the question is   Is there a simple way to specify decrease a connection timeout for a WebRequest or HttpWebRequest

User · Answer

Sorry for tacking on to an old thread  but I think something that was said above may be incorrect misleading   From what I can tell  Timeout is NOT the connection time  it is the TOTAL time allowed for the entire life of the HttpWebRequest and response   Proof   I Set    Timeout 5000  ReadWriteTimeout 32000   The connect and post time for the HttpWebRequest took 26ms  but the subsequent call HttpWebRequest GetResponse   timed out in 4974ms thus proving that the 5000ms was the time limit for the whole send request get response set of calls   I didn t verify if the DNS name resolution was measured as part of the time as this is irrelevant to me since none of this works the way I really need it to work--my intention was to time out quicker when connecting to systems that weren t accepting connections as shown by them failing during the connect phase of the request   For example  I m willing to wait 30 seconds on a connection request that has a chance of returning a result  but I only want to burn 10 seconds waiting to send a request to a host that is misbehaving

User · Answer

No matter what we tried we couldn t manage to get the timeout below 21 seconds when the server we were checking was down    To work around this we combined a TcpClient check to see if the domain was alive followed by a separate check to see if the URL was active  public static bool IsUrlAlive string aUrl  int aTimeoutSeconds        try                 check the domain first         if  IsDomainAlive new Uri aUrl  Host  aTimeoutSeconds                           only now check the url itself             var request   System Net WebRequest Create aUrl               request Method    HEAD               request Timeout   aTimeoutSeconds   1000              var response    HttpWebResponse request GetResponse                return response StatusCode    HttpStatusCode OK                      catch                 return false      private static bool IsDomainAlive string aDomain  int aTimeoutSeconds        try               using  TcpClient client   new TcpClient                          var result   client BeginConnect aDomain  80  null  null                var success   result AsyncWaitHandle WaitOne TimeSpan FromSeconds aTimeoutSeconds                 if   success                                return false                                we have connected             client EndConnect result               return true                      catch                 return false

User · Answer

From the documentation of the HttpWebRequest Timeout property      A Domain Name System  DNS  query may   take up to 15 seconds to return or   time out  If your request contains a   host name that requires resolution and   you set Timeout to a value less than   15 seconds  it may take 15 seconds or   more before a WebException is thrown   to indicate a timeout on your request    Is it possible that your DNS query is the cause of the timeout

User · Answer

Something I found later which helped  is the  ReadWriteTimeout property  This  in addition to the  Timeout property seemed to finally cut down on the time threads would spend trying to download from a problematic server  The default time for  ReadWriteTimeout is 5 minutes  which for my application was far too long   So  it seems to me    Timeout   time spent trying to establish a connection  not including lookup time   ReadWriteTimeout   time spent trying to read or write data after connection established  More info  HttpWebRequest ReadWriteTimeout Property  Edit   Per  KyleM s comment  the Timeout property is for the entire connection attempt  and reading up on it at MSDN shows      Timeout is the number of milliseconds that a subsequent synchronous request made with the GetResponse method waits for a response  and the GetRequestStream method waits for a stream  The Timeout applies to the entire request and response  not individually to the GetRequestStream and GetResponse method calls  If the resource is not returned within the time-out period  the request throws a WebException with the Status property set to WebExceptionStatus Timeout     Emphasis mine

User · Answer

I believe that the problem is that the WebRequest measures the time only after the request is actually made  If you submit multiple requests to the same address then the ServicePointManager will throttle your requests and only actually submit as many concurrent connections as the value of the corresponding ServicePoint ConnectionLimit which by default gets the value from ServicePointManager DefaultConnectionLimit  Application CLR host sets this to 2  ASP host to 10  So if you have a multithreaded application that submits multiple requests to the same host only two are actually placed on the wire  the rest are queued up   I have not researched this to a conclusive evidence whether this is what really happens  but on a similar project I had things were horrible until I removed the ServicePoint limitation   Another factor to consider is the DNS lookup time  Again  is my belief not backed by hard evidence  but I think the WebRequest does not count the DNS lookup time against the request timeout  DNS lookup time can show up as very big time factor on some deployments   And yes  you must code your app around the WebRequest BeginGetRequestStream  for POSTs with content  and WebRequest BeginGetResponse  for GETs and POSTSs   Synchronous calls will not scale  I won t enter into details why  but that I do have hard evidence for   Anyway  the ServicePoint issue is orthogonal to this  the queueing behavior happens with async calls too

[c#] Adjusting HttpWebRequest Connection Timeout in C#

Examples related to c#

Examples related to httpwebrequest

Examples related to timeout