C LINQ find duplicates in List

Question

Using LINQ  from a List lt int gt   how can I retrieve a list that contains entries repeated more than once and their values

User · Answer

To find the duplicate values only   var duplicates   list GroupBy x   gt  x Key  Any g   gt  g Count    gt  1    E g  var list   new    1 2 3 1 4 2    GroupBy will group the numbers by their keys and will maintain the count  number of times it repeated  with it  After that  we are just checking the values who have repeated more than once  To find the unique values only   var unique   list GroupBy x   gt  x Key  All g   gt  g Count      1    E g  var list   new    1 2 3 1 4 2    GroupBy will group the numbers by their keys and will maintain the count  number of times it repeated  with it  After that  we are just checking the values who have repeated only once means are unique

User · Answer

I created a extention to response to this you could includ it in your projects  I think this return the most case when you search for duplicates in List or Linq   Example     Dummy class to compare in list public class Person       public int Id   get  set        public string Name   get  set        public string Surname   get  set        public Person int id  string name  string surname                this Id   id          this Name   name          this Surname   surname              The extention static class public static class Extention       public static IEnumerable lt T gt  getMoreThanOnceRepeated lt T gt  this IEnumerable lt T gt  extList  Func lt T  object gt  groupProps  where T   class         Return only the second and next reptition         return extList              GroupBy groupProps               SelectMany z   gt  z Skip 1      Skip the first occur and return all the others that repeats           public static IEnumerable lt T gt  getAllRepeated lt T gt  this IEnumerable lt T gt  extList  Func lt T  object gt  groupProps  where T   class                 Get All the lines that has repeating         return extList              GroupBy groupProps               Where z   gt  z Count    gt  1    Filter only the distinct one              SelectMany z   gt  z    All in where has to be retuned            how to use it  void DuplicateExample           Populate List     List lt Person gt  PersonsLst   new List lt Person gt         new Person 1  Ricardo   Figueiredo      fist Duplicate to the example     new Person 2  Ana   Figueiredo        new Person 3  Ricardo   Figueiredo     second Duplicate to the example     new Person 4  Margarida   Figueiredo        new Person 5  Ricardo   Figueiredo    third Duplicate to the example             Console WriteLine  All         PersonsLst ForEach z   gt  Console WriteLine   0  - gt   1   2    z Id  z Name  z Surname           OUTPUT          All          1 - gt  Ricardo Figueiredo         2 - gt  Ana Figueiredo         3 - gt  Ricardo Figueiredo         4 - gt  Margarida Figueiredo         5 - gt  Ricardo Figueiredo                 Console WriteLine  All lines with repeated data        PersonsLst getAllRepeated z   gt  new   z Name  z Surname             ToList            ForEach z   gt  Console WriteLine   0  - gt   1   2    z Id  z Name  z Surname           OUTPUT          All lines with repeated data         1 - gt  Ricardo Figueiredo         3 - gt  Ricardo Figueiredo         5 - gt  Ricardo Figueiredo                Console WriteLine  Only Repeated more than once        PersonsLst getMoreThanOnceRepeated z   gt  new   z Name  z Surname             ToList            ForEach z   gt  Console WriteLine   0  - gt   1   2    z Id  z Name  z Surname           OUTPUT          Only Repeated more than once         3 - gt  Ricardo Figueiredo         5 - gt  Ricardo Figueiredo

User · Answer

The easiest way to solve the problem is to group the elements based on their value  and then pick a representative of the group if there are more than one element in the group  In LINQ  this translates to   var query   lst GroupBy x   gt  x                 Where g   gt  g Count    gt  1                 Select y   gt  y Key                 ToList      If you want to know how many times the elements are repeated  you can use   var query   lst GroupBy x   gt  x                 Where g   gt  g Count    gt  1                 Select y   gt  new   Element   y Key  Counter   y Count                     ToList      This will return a List of an anonymous type  and each element will have the properties Element and Counter  to retrieve the information you need   And lastly  if it s a dictionary you are looking for  you can use  var query   lst GroupBy x   gt  x                 Where g   gt  g Count    gt  1                 ToDictionary x   gt  x Key  y   gt  y Count       This will return a dictionary  with your element as key  and the number of times it s repeated as value

User · Answer

You can do this   var list   new    1 2 3 1 4 2   var duplicateItems   list Duplicates      With these extension methods   public static class Extensions       public static IEnumerable lt TSource gt  Duplicates lt TSource  TKey gt  this IEnumerable lt TSource gt  source  Func lt TSource  TKey gt  selector                var grouped   source GroupBy selector           var moreThan1   grouped Where i   gt  i IsMultiple             return moreThan1 SelectMany i   gt  i              public static IEnumerable lt TSource gt  Duplicates lt TSource  TKey gt  this IEnumerable lt TSource gt  source                return source Duplicates i   gt  i              public static bool IsMultiple lt T gt  this IEnumerable lt T gt  source                var enumerator   source GetEnumerator            return enumerator MoveNext    amp  amp  enumerator MoveNext              Using IsMultiple   in the Duplicates method is faster than Count   because this does not iterate the whole collection

User · Answer

Find out if an enumerable contains any duplicate    var anyDuplicate   enumerable GroupBy x   gt  x Key  Any g   gt  g Count    gt  1     Find out if all values in an enumerable are unique    var allUnique   enumerable GroupBy x   gt  x Key  All g   gt  g Count      1

User · Answer

Complete set of Linq to SQL extensions of Duplicates functions checked in MS SQL Server  Without using  ToList   or IEnumerable  These queries executing in SQL Server rather than in memory   The results only return at memory   public static class Linq2SqlExtensions        public class CountOfT lt T gt            public T Key   get  set            public int Count   get  set               public static IQueryable lt TKey gt  Duplicates lt TSource  TKey gt  this IQueryable lt TSource gt  source  Expression lt Func lt TSource  TKey gt  gt  groupBy            gt  source GroupBy groupBy  Where w   gt  w Count    gt  1  Select s   gt  s Key        public static IQueryable lt TSource gt  GetDuplicates lt TSource  TKey gt  this IQueryable lt TSource gt  source  Expression lt Func lt TSource  TKey gt  gt  groupBy            gt  source GroupBy groupBy  Where w   gt  w Count    gt  1  SelectMany s   gt  s        public static IQueryable lt CountOfT lt TKey gt  gt  DuplicatesCounts lt TSource  TKey gt  this IQueryable lt TSource gt  source  Expression lt Func lt TSource  TKey gt  gt  groupBy            gt  source GroupBy groupBy  Where w   gt  w Count    gt  1  Select y   gt  new CountOfT lt TKey gt    Key   y Key  Count   y Count            public static IQueryable lt Tuple lt TKey  int gt  gt  DuplicatesCountsAsTuble lt TSource  TKey gt  this IQueryable lt TSource gt  source  Expression lt Func lt TSource  TKey gt  gt  groupBy            gt  source GroupBy groupBy  Where w   gt  w Count    gt  1  Select s   gt  Tuple Create s Key  s Count

User · Answer

there is an answer but i did not understand why is not working   var anyDuplicate   enumerable GroupBy x   gt  x Key  Any g   gt  g Count    gt  1     my solution is like that in this situation   var duplicates   model list                      GroupBy s   gt  s SAME ID                       Where g   gt  g Count    gt  1  Count    gt  0  if duplicates        doSomething

User · Answer

Another way is using HashSet   var hash   new HashSet lt int gt     var duplicates   list Where i   gt   hash Add i      If you want unique values in your duplicates list   var myhash   new HashSet lt int gt     var mylist   new List lt int gt    1 1 2 2 3 3 3 4 4 4   var duplicates   mylist Where item   gt   myhash Add item   Distinct   ToList      Here is the same solution as a generic extension method   public static class Extensions     public static IEnumerable lt TSource gt  GetDuplicates lt TSource  TKey gt  this IEnumerable lt TSource gt  source  Func lt TSource  TKey gt  selector  IEqualityComparer lt TKey gt  comparer          var hash   new HashSet lt TKey gt  comparer       return source Where item   gt   hash Add selector item    ToList           public static IEnumerable lt TSource gt  GetDuplicates lt TSource gt  this IEnumerable lt TSource gt  source  IEqualityComparer lt TSource gt  comparer          return source GetDuplicates x   gt  x  comparer                public static IEnumerable lt TSource gt  GetDuplicates lt TSource  TKey gt  this IEnumerable lt TSource gt  source  Func lt TSource  TKey gt  selector          return source GetDuplicates selector  null          public static IEnumerable lt TSource gt  GetDuplicates lt TSource gt  this IEnumerable lt TSource gt  source          return source GetDuplicates x   gt  x  null

User · Answer

Linq query  var query   from s2 in  from s in someList group s by new   s Column1  s Column2   into sg select sg  where s2 Count    gt  1 select s2

[linq] C# LINQ find duplicates in List

Examples related to linq

Examples related to list

Examples related to duplicate-removal