How to remove duplicates from a list

Question

I want to remove duplicates from a list but what I am doing is not working   List lt Customer gt  listCustomer   new ArrayList lt Customer gt         for  Customer customer  tmpListCustomer      if   listCustomer contains customer            listCustomer add customer

User · Answer

List   Set   List  distinct   Just add all your elements to a Set  it does not allow it s elements to be repeated  If you need a list afterwards  use new ArrayList theSet  constructor afterwards  where theSet is your resulting set

User · Answer

Does Customer implement the equals   contract    If it doesn t implement equals   and hashCode    then listCustomer contains customer  will check to see if the exact same instance already exists in the list  By instance I mean the exact same object--memory address  etc   If what you are looking for is to test whether or not the same Customer  perhaps it s the same customer if they have the same customer name  or customer number  is in the list already  then you would need to override equals   to ensure that it checks whether or not the relevant fields e g  customer names  match    Note  Don t forget to override hashCode   if you are going to override equals    Otherwise  you might get trouble with your HashMaps and other data structures  For a good coverage of why this is and what pitfalls to avoid  consider having a look at Josh Bloch s Effective Java chapters on equals   and hashCode    The link only contains iformation about why you must implement hashCode   when you implement equals    but there is good coverage about how to override equals   too    By the way  is there an ordering restriction on your set  If there isn t  a slightly easier way to solve this problem is use a Set lt Customer gt  like so   Set lt Customer gt  noDups   new HashSet lt Customer gt     noDups addAll tmpListCustomer   return new ArrayList lt Customer gt  noDups     Which will nicely remove duplicates for you  since Sets don t allow duplicates  However  this will lose any ordering that was applied to tmpListCustomer  since HashSet has no explicit ordering  You can get around that by using a TreeSet  but that s not exactly related to your question   This can simplify your code a little bit

User · Answer

The cleanest way is   List lt XXX gt  lstConsultada   dao findByPropertyList YYY   List lt XXX gt  lstFinal   new ArrayList lt XXX gt  new LinkedHashSet lt GrupoOrigen gt  XXX      and override hascode and equals over the Id s properties of each entity

User · Answer

Nearly all of the above answers are right but what I suggest is to use a Map or Set while creating the related list  not after to gain performance  Because converting a list to a Set or Map and then reconverting it to a List again is a trivial work   Sample Code   Set lt String gt  stringsSet   new LinkedHashSet lt String gt      A Linked hash set    prevents the adding order of the elements for  String string  stringsList        stringsSet add string     return new ArrayList lt String gt  stringsSet

User · Answer

Using java 8 stream api       List lt String gt  list   new ArrayList lt  gt         list add  one        list add  one        list add  two        System out println list       Collection lt String gt  c   list stream   collect Collectors toSet         System out println c     Output   Before values    one  one  two   After Values    one  two

User · Answer

Two suggestions    Use a HashSet instead of an ArrayList  This will speed up the contains   checks considerably if you have a long list Make sure Customer equals   and Customer hashCode   are implemented properly  i e  they should be based on the combined values of the underlying fields in the customer object

User · Answer

If the code in your question doesn t work  you probably have not implemented equals Object  on the Customer class appropriately  Presumably there is some key  let us call it customerId  that uniquely identifies a customer  e g  class Customer       private String customerId           An appropriate definition of equals Object  would look like this      public boolean equals Object obj            if  obj    this                return true                    if    obj instanceof Customer                 return false                    Customer other    Customer  obj          return this customerId equals other customerId          For completeness  you should also implement hashCode so that two Customer objects that are equal will return the same hash value   A matching hashCode for the above definition of equals would be      public int hashCode             return customerId hashCode           It is also worth noting that this is not an efficient way to remove duplicates if the list is large    For a list with N customers  you will need to perform N  N-1  2 comparisons in the worst case  i e  when there are no duplicates    For a more efficient solution you should use something like a HashSet to do the duplicate checking

User · Answer

The correct answer for Java is use a Set  If you already have a List lt Customer gt  and want to de duplicate it  Set lt Customer gt  s   new HashSet lt Customer gt  listCustomer     Otherise just use a Set implemenation HashSet  TreeSet directly and skip the List construction phase   You will need to override hashCode   and equals   on your domain classes that are put in the Set as well to make sure that the behavior you want actually what you get  equals   can be as simple as comparing unique ids of the objects to as complex as comparing every field  hashCode   can be as simple as returning the hashCode   of the unique id  String representation or the hashCode

User · Answer

Assuming you want to keep the current order and don t want a Set  perhaps the easiest is   List lt Customer gt  depdupeCustomers       new ArrayList lt  gt  new LinkedHashSet lt  gt  customers      If you want to change the original list   Set lt Customer gt  depdupeCustomers   new LinkedHashSet lt  gt  customers   customers clear    customers addAll dedupeCustomers

User · Answer

private void removeTheDuplicates List lt Customer gt myList        for ListIterator lt Customer gt iterator   myList listIterator    iterator hasNext               Customer customer   iterator next            if Collections frequency myList  customer   gt  1                iterator remove                        System out println myList toString

User · Answer

I suspect you might not have Customer equals   implemented properly  or at all    List contains   uses equals   to verify whether any of its elements is identical to the object passed as parameter  However  the default implementation of equals tests for physical identity  not value identity  So if you have not overwritten it in Customer  it will return false for two distinct Customer objects having identical state   Here are the nitty-gritty details of how to implement equals  and hashCode  which is its pair - you must practically always implement both if you need to implement either of them   Since you haven t shown us the Customer class  it is difficult to give more concrete advice   As others have noted  you are better off using a Set rather than doing the job by hand  but even for that  you still need to implement those methods

User · Answer

IMHO best way how to do it these days   Suppose you have a Collection  dups  and you want to create another Collection containing the same elements but with all duplicates eliminated  The following one-liner does the trick   Collection lt collectionType gt  noDups   new HashSet lt collectionType gt  dups     It works by creating a Set which  by definition  cannot contain duplicates   Based on oracle doc

User · Answer

As others have mentioned  you are probably not implementing equals   correctly   However  you should also note that this code is considered quite inefficient  since the runtime could be the number of elements squared   You might want to consider using a Set structure instead of a List instead  or building a Set first and then turning it into a list

User · Answer

java 8 update you can use stream of array as below     Arrays stream yourArray  distinct                        collect Collectors toList

User · Answer

The  contains  method searched for whether the list contains an entry that returns true from Customer equals Object o   If you have not overridden equals Object  in Customer or one of its parents then it will only search for an existing occurrence of the same object  It may be this was what you wanted  in which case your code should work  But if you were looking for not having two objects both representing the same customer  then you need to override equals Object  to return true when that is the case    It is also true that using one of the implementations of Set instead of List would give you duplicate removal automatically  and faster  for anything other than very small Lists   You will still need to provide code for equals   You should also override hashCode   when you override equals

[java] How to remove duplicates from a list?

List ? Set ? List (distinct)

Examples related to java

Examples related to list

Examples related to collections

Examples related to duplicates