[c#] How to use HttpWebRequest (.NET) asynchronously?

How can I use HttpWebRequest (.NET, C#) asynchronously?

By far the easiest way is by using TaskFactory.FromAsync from the TPL. It's literally a couple of lines of code when used in conjunction with the new async/await keywords:

var request = WebRequest.Create("http://www.stackoverflow.com");
var response = (HttpWebResponse) await Task.Factory
    .FromAsync<WebResponse>(request.BeginGetResponse,
                            request.EndGetResponse,
                            null);
Debug.Assert(response.StatusCode == HttpStatusCode.OK);

If you can't use the C#5 compiler then the above can be accomplished using the Task.ContinueWith method:

Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse,
                                    request.EndGetResponse,
                                    null)
    .ContinueWith(task =>
    {
        var response = (HttpWebResponse) task.Result;
        Debug.Assert(response.StatusCode == HttpStatusCode.OK);
    });

Everyone so far has been wrong, because BeginGetResponse() does some work on the current thread. From the documentation:

The BeginGetResponse method requires some synchronous setup tasks to complete (DNS resolution, proxy detection, and TCP socket connection, for example) before this method becomes asynchronous. As a result, this method should never be called on a user interface (UI) thread because it might take considerable time (up to several minutes depending on network settings) to complete the initial synchronous setup tasks before an exception for an error is thrown or the method succeeds.

So to do this right:

void DoWithResponse(HttpWebRequest request, Action<HttpWebResponse> responseAction)
{
    Action wrapperAction = () =>
    {
        request.BeginGetResponse(new AsyncCallback((iar) =>
        {
            var response = (HttpWebResponse)((HttpWebRequest)iar.AsyncState).EndGetResponse(iar);
            responseAction(response);
        }), request);
    };
    wrapperAction.BeginInvoke(new AsyncCallback((iar) =>
    {
        var action = (Action)iar.AsyncState;
        action.EndInvoke(iar);
    }), wrapperAction);
}

You can then do what you need to with the response. For example:

HttpWebRequest request;
// init your request...then:
DoWithResponse(request, (response) => {
    var body = new StreamReader(response.GetResponseStream()).ReadToEnd();
    Console.Write(body);
});

public void GetResponseAsync (HttpWebRequest request, Action<HttpWebResponse> gotResponse)
    {
        if (request != null) { 
            request.BeginGetRequestStream ((r) => {
                try { // there's a try/catch here because execution path is different from invokation one, exception here may cause a crash
                    HttpWebResponse response = request.EndGetResponse (r);
                    if (gotResponse != null) 
                        gotResponse (response);
                } catch (Exception x) {
                    Console.WriteLine ("Unable to get response for '" + request.RequestUri + "' Err: " + x);
                }
            }, null);
        } 
    }

By far the easiest way is by using TaskFactory.FromAsync from the TPL. It's literally a couple of lines of code when used in conjunction with the new async/await keywords:

var request = WebRequest.Create("http://www.stackoverflow.com");
var response = (HttpWebResponse) await Task.Factory
    .FromAsync<WebResponse>(request.BeginGetResponse,
                            request.EndGetResponse,
                            null);
Debug.Assert(response.StatusCode == HttpStatusCode.OK);

If you can't use the C#5 compiler then the above can be accomplished using the Task.ContinueWith method:

Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse,
                                    request.EndGetResponse,
                                    null)
    .ContinueWith(task =>
    {
        var response = (HttpWebResponse) task.Result;
        Debug.Assert(response.StatusCode == HttpStatusCode.OK);
    });

.NET has changed since many of these answers were posted, and I'd like to provide a more up-to-date answer. Use an async method to start a Task that will run on a background thread:

private async Task<String> MakeRequestAsync(String url)
{    
    String responseText = await Task.Run(() =>
    {
        try
        {
            HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
            WebResponse response = request.GetResponse();            
            Stream responseStream = response.GetResponseStream();
            return new StreamReader(responseStream).ReadToEnd();            
        }
        catch (Exception e)
        {
            Console.WriteLine("Error: " + e.Message);
        }
        return null;
    });

    return responseText;
}

To use the async method:

String response = await MakeRequestAsync("http://example.com/");

Update:

This solution does not work for UWP apps which use WebRequest.GetResponseAsync() instead of WebRequest.GetResponse(), and it does not call the Dispose() methods where appropriate. @dragansr has a good alternative solution that addresses these issues.

I ended up using BackgroundWorker, it is definitely asynchronous unlike some of the above solutions, it handles returning to the GUI thread for you, and it is very easy to understand.

It is also very easy to handle exceptions, as they end up in the RunWorkerCompleted method, but make sure you read this: Unhandled exceptions in BackgroundWorker

I used WebClient but obviously you could use HttpWebRequest.GetResponse if you wanted.

var worker = new BackgroundWorker();

worker.DoWork += (sender, args) => {
    args.Result = new WebClient().DownloadString(settings.test_url);
};

worker.RunWorkerCompleted += (sender, e) => {
    if (e.Error != null) {
        connectivityLabel.Text = "Error: " + e.Error.Message;
    } else {
        connectivityLabel.Text = "Connectivity OK";
        Log.d("result:" + e.Result);
    }
};

connectivityLabel.Text = "Testing Connectivity";
worker.RunWorkerAsync();

Everyone so far has been wrong, because BeginGetResponse() does some work on the current thread. From the documentation:

The BeginGetResponse method requires some synchronous setup tasks to complete (DNS resolution, proxy detection, and TCP socket connection, for example) before this method becomes asynchronous. As a result, this method should never be called on a user interface (UI) thread because it might take considerable time (up to several minutes depending on network settings) to complete the initial synchronous setup tasks before an exception for an error is thrown or the method succeeds.

So to do this right:

void DoWithResponse(HttpWebRequest request, Action<HttpWebResponse> responseAction)
{
    Action wrapperAction = () =>
    {
        request.BeginGetResponse(new AsyncCallback((iar) =>
        {
            var response = (HttpWebResponse)((HttpWebRequest)iar.AsyncState).EndGetResponse(iar);
            responseAction(response);
        }), request);
    };
    wrapperAction.BeginInvoke(new AsyncCallback((iar) =>
    {
        var action = (Action)iar.AsyncState;
        action.EndInvoke(iar);
    }), wrapperAction);
}

You can then do what you need to with the response. For example:

HttpWebRequest request;
// init your request...then:
DoWithResponse(request, (response) => {
    var body = new StreamReader(response.GetResponseStream()).ReadToEnd();
    Console.Write(body);
});

.NET has changed since many of these answers were posted, and I'd like to provide a more up-to-date answer. Use an async method to start a Task that will run on a background thread:

private async Task<String> MakeRequestAsync(String url)
{    
    String responseText = await Task.Run(() =>
    {
        try
        {
            HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
            WebResponse response = request.GetResponse();            
            Stream responseStream = response.GetResponseStream();
            return new StreamReader(responseStream).ReadToEnd();            
        }
        catch (Exception e)
        {
            Console.WriteLine("Error: " + e.Message);
        }
        return null;
    });

    return responseText;
}

To use the async method:

String response = await MakeRequestAsync("http://example.com/");

Update:

This solution does not work for UWP apps which use WebRequest.GetResponseAsync() instead of WebRequest.GetResponse(), and it does not call the Dispose() methods where appropriate. @dragansr has a good alternative solution that addresses these issues.

Considering the answer:

HttpWebRequest webRequest;

void StartWebRequest()
{
    webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), null);
}

void FinishWebRequest(IAsyncResult result)
{
    webRequest.EndGetResponse(result);
}

You could send the request pointer or any other object like this:

void StartWebRequest()
{
    HttpWebRequest webRequest = ...;
    webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), webRequest);
}

void FinishWebRequest(IAsyncResult result)
{
    HttpWebResponse response = (result.AsyncState as HttpWebRequest).EndGetResponse(result) as HttpWebResponse;
}

Greetings

I ended up using BackgroundWorker, it is definitely asynchronous unlike some of the above solutions, it handles returning to the GUI thread for you, and it is very easy to understand.

It is also very easy to handle exceptions, as they end up in the RunWorkerCompleted method, but make sure you read this: Unhandled exceptions in BackgroundWorker

I used WebClient but obviously you could use HttpWebRequest.GetResponse if you wanted.

var worker = new BackgroundWorker();

worker.DoWork += (sender, args) => {
    args.Result = new WebClient().DownloadString(settings.test_url);
};

worker.RunWorkerCompleted += (sender, e) => {
    if (e.Error != null) {
        connectivityLabel.Text = "Error: " + e.Error.Message;
    } else {
        connectivityLabel.Text = "Connectivity OK";
        Log.d("result:" + e.Result);
    }
};

connectivityLabel.Text = "Testing Connectivity";
worker.RunWorkerAsync();

Considering the answer:

HttpWebRequest webRequest;

void StartWebRequest()
{
    webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), null);
}

void FinishWebRequest(IAsyncResult result)
{
    webRequest.EndGetResponse(result);
}

You could send the request pointer or any other object like this:

void StartWebRequest()
{
    HttpWebRequest webRequest = ...;
    webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), webRequest);
}

void FinishWebRequest(IAsyncResult result)
{
    HttpWebResponse response = (result.AsyncState as HttpWebRequest).EndGetResponse(result) as HttpWebResponse;
}

Greetings

public void GetResponseAsync (HttpWebRequest request, Action<HttpWebResponse> gotResponse)
    {
        if (request != null) { 
            request.BeginGetRequestStream ((r) => {
                try { // there's a try/catch here because execution path is different from invokation one, exception here may cause a crash
                    HttpWebResponse response = request.EndGetResponse (r);
                    if (gotResponse != null) 
                        gotResponse (response);
                } catch (Exception x) {
                    Console.WriteLine ("Unable to get response for '" + request.RequestUri + "' Err: " + x);
                }
            }, null);
        } 
    }

public static async Task<byte[]> GetBytesAsync(string url) {
    var request = (HttpWebRequest)WebRequest.Create(url);
    using (var response = await request.GetResponseAsync())
    using (var content = new MemoryStream())
    using (var responseStream = response.GetResponseStream()) {
        await responseStream.CopyToAsync(content);
        return content.ToArray();
    }
}

public static async Task<string> GetStringAsync(string url) {
    var bytes = await GetBytesAsync(url);
    return Encoding.UTF8.GetString(bytes, 0, bytes.Length);
}