I am trying to download a client's data to my local machine (programatically) and their webserver is very, very slow which is causing a timeout in my WebClient object.
Here is my code:
WebClient webClient = new WebClient();
webClient.Encoding = Encoding.UTF8;
webClient.DownloadFile(downloadUrl, downloadFile);
Is there a way to set an infinite timeout on this object? Or if not can anyone help me with an example on an alternate way to do this?
The URL works fine in a browser - it just takes about 3 minutes to show.
In some cases it is necessary to add user agent to headers:
WebClient myWebClient = new WebClient();
myWebClient.DownloadFile(myStringWebResource, fileName);
myWebClient.Headers["User-Agent"] = "Mozilla/4.0 (Compatible; Windows NT 5.1; MSIE 6.0) (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
This was the solution to my case.
Credit:
http://genjurosdojo.blogspot.com/2012/10/the-remote-server-returned-error-504.html
Usage:
using (var client = new TimeoutWebClient(TimeSpan.FromSeconds(10)))
{
return await client.DownloadStringTaskAsync(url).ConfigureAwait(false);
}
Class:
using System;
using System.Net;
namespace Utilities
{
public class TimeoutWebClient : WebClient
{
public TimeSpan Timeout { get; set; }
public TimeoutWebClient(TimeSpan timeout)
{
Timeout = timeout;
}
protected override WebRequest GetWebRequest(Uri uri)
{
var request = base.GetWebRequest(uri);
if (request == null)
{
return null;
}
var timeoutInMilliseconds = (int) Timeout.TotalMilliseconds;
request.Timeout = timeoutInMilliseconds;
if (request is HttpWebRequest httpWebRequest)
{
httpWebRequest.ReadWriteTimeout = timeoutInMilliseconds;
}
return request;
}
}
}
But I recommend a more modern solution:
using System;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
public static async Task<string> ReadGetRequestDataAsync(Uri uri, TimeSpan? timeout = null, CancellationToken cancellationToken = default)
{
using var source = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
if (timeout != null)
{
source.CancelAfter(timeout.Value);
}
using var client = new HttpClient();
using var response = await client.GetAsync(uri, source.Token).ConfigureAwait(false);
return await response.Content.ReadAsStringAsync().ConfigureAwait(false);
}
It will throw an OperationCanceledException after a timeout.
You need to use HttpWebRequest rather than WebClient as you can't set the timeout on WebClient without extending it (even though it uses the HttpWebRequest). Using the HttpWebRequest instead will allow you to set the timeout.
As Sohnee says, using System.Net.HttpWebRequest and set the Timeout property instead of using System.Net.WebClient.
You can't however set an infinite timeout value (it's not supported and attempting to do so will throw an ArgumentOutOfRangeException).
I'd recommend first performing a HEAD HTTP request and examining the Content-Length header value returned to determine the number of bytes in the file you're downloading and then setting the timeout value accordingly for subsequent GET request or simply specifying a very long timeout value that you would never expect to exceed.
Couldn't get the w.Timeout code to work when pulled out the network cable, it just wasn't timing out, moved to using HttpWebRequest and does the job now.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(downloadUrl);
request.Timeout = 10000;
request.ReadWriteTimeout = 10000;
var wresp = (HttpWebResponse)request.GetResponse();
using (Stream file = File.OpenWrite(downloadFile))
{
wresp.GetResponseStream().CopyTo(file);
}
I had to fight with this issue yesterday and I've also ended up to write my custom extension class.
As you can see by looking at the code below and comparing it with the accepted answer, I tried to tweak the suggestion a little bit more in order to have a more versatile class: this way you can set a precise timeout either upon instancing the object or right before using a method that uses the internal WebRequest handler.
using System;
using System.Net;
namespace Ryadel.Components.Web
{
/// <summary>
/// An extension of the standard System.Net.WebClient
/// featuring a customizable constructor and [Timeout] property.
/// </summary>
public class RyadelWebClient : WebClient
{
/// <summary>
/// Default constructor (30000 ms timeout)
/// NOTE: timeout can be changed later on using the [Timeout] property.
/// </summary>
public RyadelWebClient() : this(30000) { }
/// <summary>
/// Constructor with customizable timeout
/// </summary>
/// <param name="timeout">
/// Web request timeout (in milliseconds)
/// </param>
public RyadelWebClient(int timeout)
{
Timeout = timeout;
}
#region Methods
protected override WebRequest GetWebRequest(Uri uri)
{
WebRequest w = base.GetWebRequest(uri);
w.Timeout = Timeout;
((HttpWebRequest)w).ReadWriteTimeout = Timeout;
return w;
}
#endregion
/// <summary>
/// Web request timeout (in milliseconds)
/// </summary>
public int Timeout { get; set; }
}
}
While I was there, I also took the chance to lower the default Timeout value to 30 seconds, as 100 seemed way too much for me.
In case you need additional info regarding this class or how to use it, check out this post I wrote on my blog.
'CORRECTED VERSION OF LAST FUNCTION IN VISUAL BASIC BY GLENNG
Protected Overrides Function GetWebRequest(ByVal address As System.Uri) As System.Net.WebRequest
Dim w As System.Net.WebRequest = MyBase.GetWebRequest(address)
If _TimeoutMS <> 0 Then
w.Timeout = _TimeoutMS
End If
Return w '<<< NOTICE: MyBase.GetWebRequest(address) DOES NOT WORK >>>
End Function
For anyone who needs a WebClient with a timeout that works for async/task methods, the suggested solutions won't work. Here's what does work:
public class WebClientWithTimeout : WebClient
{
//10 secs default
public int Timeout { get; set; } = 10000;
//for sync requests
protected override WebRequest GetWebRequest(Uri uri)
{
var w = base.GetWebRequest(uri);
w.Timeout = Timeout; //10 seconds timeout
return w;
}
//the above will not work for async requests :(
//let's create a workaround by hiding the method
//and creating our own version of DownloadStringTaskAsync
public new async Task<string> DownloadStringTaskAsync(Uri address)
{
var t = base.DownloadStringTaskAsync(address);
if(await Task.WhenAny(t, Task.Delay(Timeout)) != t) //time out!
{
CancelAsync();
}
return await t;
}
}
I blogged about the full workaround here
The first solution did not work for me but here is some code that did work for me.
private class WebClient : System.Net.WebClient
{
public int Timeout { get; set; }
protected override WebRequest GetWebRequest(Uri uri)
{
WebRequest lWebRequest = base.GetWebRequest(uri);
lWebRequest.Timeout = Timeout;
((HttpWebRequest)lWebRequest).ReadWriteTimeout = Timeout;
return lWebRequest;
}
}
private string GetRequest(string aURL)
{
using (var lWebClient = new WebClient())
{
lWebClient.Timeout = 600 * 60 * 1000;
return lWebClient.DownloadString(aURL);
}
}
According to kisp solution this is my edited version working async:
Class WebConnection.cs
internal class WebConnection : WebClient
{
internal int Timeout { get; set; }
protected override WebRequest GetWebRequest(Uri Address)
{
WebRequest WebReq = base.GetWebRequest(Address);
WebReq.Timeout = Timeout * 1000 // Seconds
return WebReq;
}
}
The async Task
private async Task GetDataAsyncWithTimeout()
{
await Task.Run(() =>
{
using (WebConnection webClient = new WebConnection())
{
webClient.Timeout = 5; // Five seconds
webClient.DownloadData("https://www.yourwebsite.com");
}
});
} // await GetDataAsyncWithTimeout()
Else, if you don't want to use async:
private void GetDataSyncWithTimeout()
{
using (WebConnection webClient = new WebConnection())
{
webClient.Timeout = 5; // Five seconds
webClient.DownloadData("https://www.yourwebsite.com");
}
} // GetDataSyncWithTimeout()
For completeness, here's kisp's solution ported to VB (can't add code to a comment)
Namespace Utils
''' <summary>
''' Subclass of WebClient to provide access to the timeout property
''' </summary>
Public Class WebClient
Inherits System.Net.WebClient
Private _TimeoutMS As Integer = 0
Public Sub New()
MyBase.New()
End Sub
Public Sub New(ByVal TimeoutMS As Integer)
MyBase.New()
_TimeoutMS = TimeoutMS
End Sub
''' <summary>
''' Set the web call timeout in Milliseconds
''' </summary>
''' <value></value>
Public WriteOnly Property setTimeout() As Integer
Set(ByVal value As Integer)
_TimeoutMS = value
End Set
End Property
Protected Overrides Function GetWebRequest(ByVal address As System.Uri) As System.Net.WebRequest
Dim w As System.Net.WebRequest = MyBase.GetWebRequest(address)
If _TimeoutMS <> 0 Then
w.Timeout = _TimeoutMS
End If
Return w
End Function
End Class
End Namespace
Source: Stackoverflow.com