Map and Reduce in NET

Question

What scenarios would warrant the use of the  Map and Reduce  algorithm    Is there a  NET implementation of this algorithm

User · Accepted Answer

Linq equivalents of Map and Reduce   If you   re lucky enough to have linq then you don   t need to write your own map and reduce functions  C  3 5 and Linq already has it albeit under different names    Map is Select   Enumerable Range 1  10  Select x   gt  x   2    Reduce is Aggregate   Enumerable Range 1  10  Aggregate 0   acc  x    gt  acc   x    Filter is Where   Enumerable Range 1  10  Where x   gt  x   2    0      https   www justinshield com 2011 06 mapreduce-in-c

User · Answer

The classes of problem that are well suited for a mapreduce style solution are problems of aggregation   Of extracting data from a dataset   In C   one could take advantage of LINQ to program in this style   From the following article  http   codecube net 2009 02 mapreduce-in-c-using-linq      the GroupBy method is acting as the map  while the Select method does the job of reducing the intermediate results into the final list of results    var wordOccurrences   words                  GroupBy w   gt  w                   Select intermediate   gt  new                                       Word   intermediate Key                      Frequency   intermediate Sum w   gt  1                                      Where w   gt  w Frequency  gt  10                   OrderBy w   gt  w Frequency     For the distributed portion  you could check out DryadLINQ  http   research microsoft com en-us projects dryadlinq default aspx

User · Answer

The classes of problem that are well suited for a mapreduce style solution are problems of aggregation   Of extracting data from a dataset   In C   one could take advantage of LINQ to program in this style   From the following article  http   codecube net 2009 02 mapreduce-in-c-using-linq      the GroupBy method is acting as the map  while the Select method does the job of reducing the intermediate results into the final list of results    var wordOccurrences   words                  GroupBy w   gt  w                   Select intermediate   gt  new                                       Word   intermediate Key                      Frequency   intermediate Sum w   gt  1                                      Where w   gt  w Frequency  gt  10                   OrderBy w   gt  w Frequency     For the distributed portion  you could check out DryadLINQ  http   research microsoft com en-us projects dryadlinq default aspx

User · Answer

Since I never can remember that LINQ calls it Where  Select and Aggregate instead of Filter  Map and Reduce so I created a few extension methods you can use   IEnumerable lt string gt  myStrings   new List lt string gt       1    2    3    4    5     IEnumerable lt int gt  convertedToInts   myStrings Map s   gt  int Parse s    IEnumerable lt int gt  filteredInts   convertedToInts Filter i   gt  i  lt   3      Keep 1 2 3 int sumOfAllInts   filteredInts Reduce  sum  i    gt  sum   i      Sum up all ints Assert Equal 6  sumOfAllInts      1 2 3 is 6   Here are the 3 methods  from https   github com cs-util-com cscore blob master CsCore PlainNetClassLib src Plugins CsCore com csutil collections IEnumerableExtensions cs     public static IEnumerable lt R gt  Map lt T  R gt  this IEnumerable lt T gt  self  Func lt T  R gt  selector        return self Select selector      public static T Reduce lt T gt  this IEnumerable lt T gt  self  Func lt T  T  T gt  func        return self Aggregate func      public static IEnumerable lt T gt  Filter lt T gt  this IEnumerable lt T gt  self  Func lt T  bool gt  predicate        return self Where predicate       Some more details from https   github com cs-util-com cscore ienumerable-extensions

User · Answer

Since I never can remember that LINQ calls it Where  Select and Aggregate instead of Filter  Map and Reduce so I created a few extension methods you can use   IEnumerable lt string gt  myStrings   new List lt string gt       1    2    3    4    5     IEnumerable lt int gt  convertedToInts   myStrings Map s   gt  int Parse s    IEnumerable lt int gt  filteredInts   convertedToInts Filter i   gt  i  lt   3      Keep 1 2 3 int sumOfAllInts   filteredInts Reduce  sum  i    gt  sum   i      Sum up all ints Assert Equal 6  sumOfAllInts      1 2 3 is 6   Here are the 3 methods  from https   github com cs-util-com cscore blob master CsCore PlainNetClassLib src Plugins CsCore com csutil collections IEnumerableExtensions cs     public static IEnumerable lt R gt  Map lt T  R gt  this IEnumerable lt T gt  self  Func lt T  R gt  selector        return self Select selector      public static T Reduce lt T gt  this IEnumerable lt T gt  self  Func lt T  T  T gt  func        return self Aggregate func      public static IEnumerable lt T gt  Filter lt T gt  this IEnumerable lt T gt  self  Func lt T  bool gt  predicate        return self Where predicate       Some more details from https   github com cs-util-com cscore ienumerable-extensions

[c#] Map and Reduce in .NET

Examples related to c#

Examples related to mapreduce