How can I use Async with ForEach

Question

Is it possible to use Async when using ForEach  Below is the code I am trying   using  DataContext db   new DataLayer DataContext          db Groups ToList   ForEach i   gt  async           await GetAdminsFromGroup i Gid               I am getting the error      The name  Async  does not exist in the current context   The method the using statement is enclosed in is set to async

User · Answer

Starting with C  8 0  you can create and consume streams asynchronously        private async void button1 Click object sender  EventArgs e                IAsyncEnumerable lt int gt  enumerable   GenerateSequence             await foreach  var i in enumerable                        Debug WriteLine i                        public static async IAsyncEnumerable lt int gt  GenerateSequence                 for  int i   0  i  lt  20  i                          await Task Delay 100               yield return i                    More

User · Answer

This is method I created to handle async scenarios with ForEach   If one of tasks fails then other tasks will continue their execution  You have ability to add function that will be executed on every exception  Exceptions are being collected as aggregateException at the end and are available for you  Can handle CancellationToken   public static class ParallelExecutor                    lt summary gt              Executes asynchronously given function on all elements of given enumerable with task count restriction              Executor will continue starting new tasks even if one of the tasks throws  If at least one of the tasks throwed exception then  lt see cref  quot AggregateException quot   gt  is throwed at the end of the method run               lt  summary gt               lt typeparam name  quot T quot  gt Type of elements in enumerable lt  typeparam gt               lt param name  quot maxTaskCount quot  gt The maximum task count  lt  param gt               lt param name  quot enumerable quot  gt The enumerable  lt  param gt               lt param name  quot asyncFunc quot  gt asynchronous function that will be executed on every element of the enumerable  MUST be thread safe  lt  param gt               lt param name  quot onException quot  gt Acton that will be executed on every exception that would be thrown by asyncFunc  CAN be thread unsafe  lt  param gt               lt param name  quot cancellationToken quot  gt The cancellation token  lt  param gt          public static async Task ForEachAsync lt T gt  int maxTaskCount  IEnumerable lt T gt  enumerable  Func lt T  Task gt  asyncFunc  Action lt Exception gt  onException   null  CancellationToken cancellationToken   default                        using var semaphore   new SemaphoreSlim initialCount  maxTaskCount  maxCount  maxTaskCount                   This  lockObject  is used only in  catch      block              object lockObject   new object                var exceptions   new List lt Exception gt                 var tasks   new Task enumerable Count                 int i   0               try                               foreach  var t in enumerable                                        await semaphore WaitAsync cancellationToken                       tasks i      Task Run                          async      gt                                                        try                                                               await asyncFunc t                                                             catch  Exception e                                                                if  onException    null                                                                        lock  lockObject                                                                                onException Invoke e                                                                                                               This exception will be swallowed here but it will be collected at the end of ForEachAsync method in order to generate AggregateException                                  throw                                                            finally                                                               semaphore Release                                                             cancellationToken                        if  cancellationToken IsCancellationRequested                                                break                                                                    catch  OperationCanceledException e                                exceptions Add e                              foreach  var t in tasks                                if  cancellationToken IsCancellationRequested                                        break                                        Exception handling in this case is actually pretty fast                     https   gist github com shoter d943500eda37c7d99461ce3dace42141                 try                                       await t                     pragma warning disable CA1031    Do not catch general exception types - we want to throw that exception later as aggregate exception  Nothing wrong here                  catch  Exception e   pragma warning restore CA1031    Do not catch general exception types                                       exceptions Add e                                                if  exceptions Any                                  throw new AggregateException exceptions

User · Answer

Here is an actual working version of the above async foreach variants with sequential processing   public static async Task ForEachAsync lt T gt  this List lt T gt  enumerable  Action lt T gt  action        foreach  var item in enumerable          await Task Run      gt    action item      ConfigureAwait false       Here is the implementation   public async void SequentialAsync         var list   new List lt Action gt          Action action1        gt              do stuff 1             Action action2        gt              do stuff 2             list Add action1       list Add action2        await list ForEachAsync        What s the key difference   ConfigureAwait false   which keeps the context of main thread while async sequential processing of each task

User · Answer

Add this extension method  public static class ForEachAsyncExtension       public static Task ForEachAsync lt T gt  this IEnumerable lt T gt  source  int dop  Func lt T  Task gt  body                return Task WhenAll from partition in Partitioner Create source  GetPartitions dop               select Task Run async delegate                               using  partition                      while  partition MoveNext                            await body partition Current  ConfigureAwait false                              And then use like so   Task Run async      gt        var s3   new AmazonS3Client Config Instance Aws Credentials  Config Instance Aws RegionEndpoint       var buckets   await s3 ListBucketsAsync         foreach  var s3Bucket in buckets Buckets                if  s3Bucket BucketName StartsWith  mybucket-                          log Information  Bucket   gt   BucketName    s3Bucket BucketName                ListObjectsResponse objects              try                               objects   await s3 ListObjectsAsync s3Bucket BucketName                             catch                               log Error  Error getting objects  Bucket   gt   BucketName    s3Bucket BucketName                   continue                                ForEachAsync  4 is how many tasks you want to run in parallel              await objects S3Objects ForEachAsync 4  async s3Object   gt                                try                                       log Information  Bucket   gt   BucketName    gt   Key    s3Bucket BucketName  s3Object Key                       await s3 DeleteObjectAsync s3Bucket BucketName  s3Object Key                                     catch                                       log Error  Error deleting bucket  BucketName  object  Key    s3Bucket BucketName  s3Object Key                                                  try                               await s3 DeleteBucketAsync s3Bucket BucketName                             catch                               log Error  Error deleting bucket  BucketName    s3Bucket BucketName                                    Wait

User · Answer

List lt T gt  ForEach doesn t play particularly well with async  neither does LINQ-to-objects  for the same reasons     In this case  I recommend projecting each element into an asynchronous operation  and you can then  asynchronously  wait for them all to complete   using  DataContext db   new DataLayer DataContext          var tasks   db Groups ToList   Select i   gt  GetAdminsFromGroupAsync i Gid        var results   await Task WhenAll tasks       The benefits of this approach over giving an async delegate to ForEach are    Error handling is more proper  Exceptions from async void cannot be caught with catch  this approach will propagate exceptions at the await Task WhenAll line  allowing natural exception handling  You know that the tasks are complete at the end of this method  since it does an await Task WhenAll  If you use async void  you cannot easily tell when the operations have completed  This approach has a natural syntax for retrieving the results  GetAdminsFromGroupAsync sounds like it s an operation that produces a result  the admins   and such code is more natural if such operations can return their results rather than setting a value as a side effect

User · Answer

The problem was that the async keyword needs to appear before the lambda  not before the body   db Groups ToList   ForEach async  i    gt        await GetAdminsFromGroup i Gid

User · Answer

The simple answer is to use the foreach keyword instead of the ForEach   method of List     using  DataContext db   new DataLayer DataContext          foreach var i in db Groups                await GetAdminsFromGroup i Gid

User · Answer

This little extension method should give you exception-safe async iteration   public static async Task ForEachAsync lt T gt  this List lt T gt  list  Func lt T  Task gt  func        foreach  var value in list                await func value             Since we re changing the return type of the lambda from void to Task  exceptions will propagate up correctly   This will allow you to write something like this in practice   await db Groups ToList   ForEachAsync async i   gt        await GetAdminsFromGroup i Gid

User · Answer

I would like to add that there is a Parallel class with ForEach function built in that can be used for this purpose

[c#] How can I use Async with ForEach?

Examples related to c#

Examples related to async-await