[c#] Running multiple async tasks and waiting for them all to complete

I need to run multiple async tasks in a console application, and wait for them all to complete before further processing.

There's many articles out there, but I seem to get more confused the more I read. I've read and understand the basic principles of the Task library, but I'm clearly missing a link somewhere.

I understand that it's possible to chain tasks so that they start after another completes (which is pretty much the scenario for all the articles I've read), but I want all my Tasks running at the same time, and I want to know once they're all completed.

What's the simplest implementation for a scenario like this?

This question is related to c# .net asynchronous task-parallel-library async-await

The answer is


Yet another answer...but I usually find myself in a case, when I need to load data simultaneously and put it into variables, like:

var cats = new List<Cat>();
var dog = new Dog();

var loadDataTasks = new Task[]
{
    Task.Run(async () => cats = await LoadCatsAsync()),
    Task.Run(async () => dog = await LoadDogAsync())
};

try
{
    await Task.WhenAll(loadDataTasks);
}
catch (Exception ex)
{
    // handle exception
}

You could create many tasks like:

List<Task> TaskList = new List<Task>();
foreach(...)
{
   var LastTask = new Task(SomeFunction);
   LastTask.Start();
   TaskList.Add(LastTask);
}

Task.WaitAll(TaskList.ToArray());

Do you want to chain the Tasks, or can they be invoked in a parallel manner?

For chaining
Just do something like

Task.Run(...).ContinueWith(...).ContinueWith(...).ContinueWith(...);
Task.Factory.StartNew(...).ContinueWith(...).ContinueWith(...).ContinueWith(...);

and don't forget to check the previous Task instance in each ContinueWith as it might be faulted.

For the parallel manner
The most simple method I came across: Parallel.Invoke Otherwise there's Task.WaitAll or you can even use WaitHandles for doing a countdown to zero actions left (wait, there's a new class: CountdownEvent), or ...


There should be a more succinct solution than the accepted answer. It shouldn't take three steps to run multiple tasks simultaneously and get their results.

  1. Create tasks
  2. await Task.WhenAll(tasks)
  3. Get task results (e.g., task1.Result)

Here's a method that cuts this down to two steps:

    public async Task<Tuple<T1, T2>> WhenAllGeneric<T1, T2>(Task<T1> task1, Task<T2> task2)
    {
        await Task.WhenAll(task1, task2);
        return Tuple.Create(task1.Result, task2.Result);
    }

You can use it like this:

var taskResults = await Task.WhenAll(DoWorkAsync(), DoMoreWorkAsync());
var DoWorkResult = taskResults.Result.Item1;
var DoMoreWorkResult = taskResults.Result.Item2;

This removes the need for the temporary task variables. The problem with using this is that while it works for two tasks, you'd need to update it for three tasks, or any other number of tasks. Also it doesn't work well if one of the tasks doesn't return anything. Really, the .Net library should provide something that can do this


You can use WhenAll which will return an awaitable Task or WaitAll which has no return type and will block further code execution simular to Thread.Sleep until all tasks are completed, canceled or faulted.

enter image description here

Example

var tasks = new Task[] {
    TaskOperationOne(),
    TaskOperationTwo()
};

Task.WaitAll(tasks);
// or
await Task.WhenAll(tasks);

If you want to run the tasks in a praticular order you can get inspiration form this anwser.


This is how I do it with an array Func<>:

var tasks = new Func<Task>[]
{
   () => myAsyncWork1(),
   () => myAsyncWork2(),
   () => myAsyncWork3()
};

await Task.WhenAll(tasks.Select(task => task()).ToArray()); //Async    
Task.WaitAll(tasks.Select(task => task()).ToArray()); //Or use WaitAll for Sync

I prepared a piece of code to show you how to use the task for some of these scenarios.

    // method to run tasks in a parallel 
    public async Task RunMultipleTaskParallel(Task[] tasks) {

        await Task.WhenAll(tasks);
    }
    // methode to run task one by one 
    public async Task RunMultipleTaskOneByOne(Task[] tasks)
    {
        for (int i = 0; i < tasks.Length - 1; i++)
            await tasks[i];
    }
    // method to run i task in parallel 
    public async Task RunMultipleTaskParallel(Task[] tasks, int i)
    {
        var countTask = tasks.Length;
        var remainTasks = 0;
        do
        {
            int toTake = (countTask < i) ? countTask : i;
            var limitedTasks = tasks.Skip(remainTasks)
                                    .Take(toTake);
            remainTasks += toTake;
            await RunMultipleTaskParallel(limitedTasks.ToArray());
        } while (remainTasks < countTask);
    }

The best option I've seen is the following extension method:

public static Task ForEachAsync<T>(this IEnumerable<T> sequence, Func<T, Task> action) {
    return Task.WhenAll(sequence.Select(action));
}

Call it like this:

await sequence.ForEachAsync(item => item.SomethingAsync(blah));

Or with an async lambda:

await sequence.ForEachAsync(async item => {
    var more = await GetMoreAsync(item);
    await more.FrobbleAsync();
});

Examples related to c#

How can I convert this one line of ActionScript to C#? Microsoft Advertising SDK doesn't deliverer ads How to use a global array in C#? How to correctly write async method? C# - insert values from file into two arrays Uploading into folder in FTP? Are these methods thread safe? dotnet ef not found in .NET Core 3 HTTP Error 500.30 - ANCM In-Process Start Failure Best way to "push" into C# array

Examples related to .net

You must add a reference to assembly 'netstandard, Version=2.0.0.0 How to use Bootstrap 4 in ASP.NET Core No authenticationScheme was specified, and there was no DefaultChallengeScheme found with default authentification and custom authorization .net Core 2.0 - Package was restored using .NetFramework 4.6.1 instead of target framework .netCore 2.0. The package may not be fully compatible Update .NET web service to use TLS 1.2 EF Core add-migration Build Failed What is the difference between .NET Core and .NET Standard Class Library project types? Visual Studio 2017 - Could not load file or assembly 'System.Runtime, Version=4.1.0.0' or one of its dependencies Nuget connection attempt failed "Unable to load the service index for source" Token based authentication in Web API without any user interface

Examples related to asynchronous

How to read file with async/await properly? Use Async/Await with Axios in React.js Waiting until the task finishes How to reject in async/await syntax? React - Display loading screen while DOM is rendering? angular 2 how to return data from subscribe How do I access store state in React Redux? SyntaxError: Unexpected token function - Async Await Nodejs Why does .json() return a promise? Why is setState in reactjs Async instead of Sync?

Examples related to task-parallel-library

What is the difference between Task.Run() and Task.Factory.StartNew() Task.Run with Parameter(s)? HttpClient - A task was cancelled? Running multiple async tasks and waiting for them all to complete Deserialize JSON to Array or List with HTTPClient .ReadAsAsync using .NET 4.0 Task pattern Calling async method synchronously How can I tell Moq to return a Task? When to use Task.Delay, when to use Thread.Sleep? Awaiting multiple Tasks with different results How to safely call an async method in C# without await

Examples related to async-await

How to correctly write async method? How can I use async/await at the top level? Any difference between await Promise.all() and multiple await? Async/Await Class Constructor Syntax for async arrow function try/catch blocks with async/await Using filesystem in node.js with async / await Use async await with Array.map Using await outside of an async function SyntaxError: Unexpected token function - Async Await Nodejs