[java] Removing Duplicate Values from ArrayList

I have one Arraylist of String and I have added Some Duplicate Value in that. and i just wanna remove that Duplicate value So how to remove it.

Here Example I got one Idea.

List<String> list = new ArrayList<String>();
        list.add("Krishna");
        list.add("Krishna");
        list.add("Kishan");
        list.add("Krishn");
        list.add("Aryan");
        list.add("Harm");

        System.out.println("List"+list);

        for (int i = 1; i < list.size(); i++) {
            String a1 = list.get(i);
            String a2 = list.get(i-1);
            if (a1.equals(a2)) {
                list.remove(a1);
            }
        }

        System.out.println("List after short"+list);

But is there any Sufficient way remove that Duplicate form list. with out using For loop ? And ya i can do it by using HashSet or some other way but using array list only. would like to have your suggestion for that. thank you for your answer in advance.

This question is related to java list sorting arraylist

The answer is


 public void removeDuplicates() {
    ArrayList<Object> al = new ArrayList<Object>();
    al.add("java");
    al.add('a');
    al.add('b');
    al.add('a');
    al.add("java");
    al.add(10.3);
    al.add('c');
    al.add(14);
    al.add("java");
    al.add(12);

    System.out.println("Before Remove Duplicate elements:" + al);
    for (int i = 0; i < al.size(); i++) {
        for (int j = i + 1; j < al.size(); j++) {
            if (al.get(i).equals(al.get(j))) {
                al.remove(j);
                j--;
            }
        }
    }
    System.out.println("After Removing duplicate elements:" + al);
}

Before Remove Duplicate elements:

[java, a, b, a, java, 10.3, c, 14, java, 12]

After Removing duplicate elements:

[java, a, b, 10.3, c, 14, 12]

List<String> list = new ArrayList<String>();
        list.add("Krishna");
        list.add("Krishna");
        list.add("Kishan");
        list.add("Krishn");
        list.add("Aryan");
        list.add("Harm");

HashSet<String> hs=new HashSet<>(list);

System.out.println("=========With Duplicate Element========");
System.out.println(list);
System.out.println("=========Removed Duplicate Element========");
System.out.println(hs);

Simple function for removing duplicates from list

private void removeDuplicates(List<?> list)
{
    int count = list.size();

    for (int i = 0; i < count; i++) 
    {
        for (int j = i + 1; j < count; j++) 
        {
            if (list.get(i).equals(list.get(j)))
            {
                list.remove(j--);
                count--;
            }
        }
    }
}

Example:
Input: [1, 2, 2, 3, 1, 3, 3, 2, 3, 1, 2, 3, 3, 4, 4, 4, 1]
Output: [1, 2, 3, 4]


public static List<String> removeDuplicateElements(List<String> array){
    List<String> temp = new ArrayList<String>();
    List<Integer> count = new ArrayList<Integer>();
    for (int i=0; i<array.size()-2; i++){
        for (int j=i+1;j<array.size()-1;j++)
            {
                if (array.get(i).compareTo(array.get(j))==0) {
                    count.add(i);
                    int kk = i;
                }
            }
        }
        for (int i = count.size()+1;i>0;i--) {
            array.remove(i);
        }
        return array;
    }
}

This will be the best way

    List<String> list = new ArrayList<String>();
    list.add("Krishna");
    list.add("Krishna");
    list.add("Kishan");
    list.add("Krishn");
    list.add("Aryan");
    list.add("Harm");

    Set<String> set=new HashSet<>(list);

list = list.stream().distinct().collect(Collectors.toList());
This could be one of the solutions using Java8 Stream API. Hope this helps.


     public List<Contact> removeDuplicates(List<Contact> list) {
    // Set set1 = new LinkedHashSet(list);
    Set set = new TreeSet(new Comparator() {
        @Override
        public int compare(Object o1, Object o2) {
                 if(((Contact)o1).getId().equalsIgnoreCase(((Contact)2).getId()) ) {
                return 0;
            }
            return 1;
        }
    });
    set.addAll(list);
    final List newList = new ArrayList(set);
    return newList;
}

In case you just need to remove the duplicates using only ArrayList, no other Collection classes, then:-

//list is the original arraylist containing the duplicates as well
List<String> uniqueList = new ArrayList<String>();
    for(int i=0;i<list.size();i++) {
        if(!uniqueList.contains(list.get(i)))
            uniqueList.add(list.get(i));
    }

Hope this helps!


if you want to use only arraylist then I am worried there is no better way which will create a huge performance benefit. But by only using arraylist i would check before adding into the list like following

void addToList(String s){
  if(!yourList.contains(s))
       yourList.add(s);
}

In this cases using a Set is suitable.


I don't think the list = new ArrayList<String>(new LinkedHashSet<String>(list)) is not the best way , since we are using the LinkedHashset(We could use directly LinkedHashset instead of ArrayList),

Solution:

import java.util.ArrayList;
public class Arrays extends ArrayList{

@Override
public boolean add(Object e) {
    if(!contains(e)){
        return super.add(e);
    }else{
        return false;
    }
}

public static void main(String[] args) {
    Arrays element=new Arrays();
    element.add(1);
    element.add(2);
    element.add(2);
    element.add(3);

    System.out.println(element);
}
}

Output: [1, 2, 3]

Here I am extending the ArrayList , as I am using the it with some changes by overriding the add method.


private static void removeDuplicates(List<Integer> list)
{
    Collections.sort(list);
    int count = list.size();
    for (int i = 0; i < count; i++) 
    {
        if(i+1<count && list.get(i)==list.get(i+1)){
            list.remove(i);
            i--;
            count--;
        }
    }
}

Using java 8:

public static <T> List<T> removeDuplicates(List<T> list) {
    return list.stream().collect(Collectors.toSet()).stream().collect(Collectors.toList());
}

It is better to use HastSet

1-a) A HashSet holds a set of objects, but in a way that it allows you to easily and quickly determine whether an object is already in the set or not. It does so by internally managing an array and storing the object using an index which is calculated from the hashcode of the object. Take a look here

1-b) HashSet is an unordered collection containing unique elements. It has the standard collection operations Add, Remove, Contains, but since it uses a hash-based implementation, these operation are O(1). (As opposed to List for example, which is O(n) for Contains and Remove.) HashSet also provides standard set operations such as union, intersection, and symmetric difference.Take a look here

2) There are different implementations of Sets. Some make insertion and lookup operations super fast by hashing elements. However that means that the order in which the elements were added is lost. Other implementations preserve the added order at the cost of slower running times.

The HashSet class in C# goes for the first approach, thus not preserving the order of elements. It is much faster than a regular List. Some basic benchmarks showed that HashSet is decently faster when dealing with primary types (int, double, bool, etc.). It is a lot faster when working with class objects. So that point is that HashSet is fast.

The only catch of HashSet is that there is no access by indices. To access elements you can either use an enumerator or use the built-in function to convert the HashSet into a List and iterate through that.Take a look here


You can make use of Google Guava utilities, as shown below

 list = ImmutableSet.copyOf(list).asList(); 

This is probably the most efficient way of eliminating the duplicates from the list and interestingly, it preserves the iteration order as well.

UPDATE

But, in case, you don't want to involve Guava then duplicates can be removed as shown below.

ArrayList<String> list = new ArrayList<String>();
    list.add("Krishna");
    list.add("Krishna");
    list.add("Kishan");
    list.add("Krishn");
    list.add("Aryan");
    list.add("Harm");

System.out.println("List"+list);
HashSet hs = new HashSet();
hs.addAll(list);
list.clear();
list.addAll(hs);

But, of course, this will destroys the iteration order of the elements in the ArrayList.

Shishir


Java 8 stream function

You could use the distinct function like above to get the distinct elements of the list,

stringList.stream().distinct();

From the documentation,

Returns a stream consisting of the distinct elements (according to Object.equals(Object)) of this stream.


Another way, if you do not wish to use the equals method is by using the collect function like this,

stringList.stream()  
    .collect(Collectors.toCollection(() -> 
        new TreeSet<String>((p1, p2) -> p1.compareTo(p2)) 
));  

From the documentation,

Performs a mutable reduction operation on the elements of this stream using a Collector.

Hope that helps.


Without a loop, No! Since ArrayList is indexed by order rather than by key, you can not found the target element without iterate the whole list.

A good practice of programming is to choose proper data structure to suit your scenario. So if Set suits your scenario the most, the discussion of implementing it with List and trying to find the fastest way of using an improper data structure makes no sense.


public static void main(String[] args) {
    @SuppressWarnings("serial")
    List<Object> lst = new ArrayList<Object>() {
        @Override
        public boolean add(Object e) {
            if(!contains(e))
            return super.add(e);
            else
            return false;
        }
    };
    lst.add("ABC");
    lst.add("ABC");
    lst.add("ABCD");
    lst.add("ABCD");
    lst.add("ABCE");
    System.out.println(lst);

}

This is the better way


Examples related to java

Under what circumstances can I call findViewById with an Options Menu / Action Bar item? How much should a function trust another function How to implement a simple scenario the OO way Two constructors How do I get some variable from another class in Java? this in equals method How to split a string in two and store it in a field How to do perspective fixing? String index out of range: 4 My eclipse won't open, i download the bundle pack it keeps saying error log

Examples related to list

Convert List to Pandas Dataframe Column Python find elements in one list that are not in the other Sorting a list with stream.sorted() in Java Python Loop: List Index Out of Range How to combine two lists in R How do I multiply each element in a list by a number? Save a list to a .txt file The most efficient way to remove first N elements in a list? TypeError: list indices must be integers or slices, not str Parse JSON String into List<string>

Examples related to sorting

Sort Array of object by object field in Angular 6 Sorting a list with stream.sorted() in Java How to sort dates from Oldest to Newest in Excel? how to sort pandas dataframe from one column Reverse a comparator in Java 8 Find the unique values in a column and then sort them pandas groupby sort within groups pandas groupby sort descending order Efficiently sorting a numpy array in descending order? Swift: Sort array of objects alphabetically

Examples related to arraylist

Adding null values to arraylist How to iterate through an ArrayList of Objects of ArrayList of Objects? Dynamically adding elements to ArrayList in Groovy How to replace existing value of ArrayList element in Java How to remove the last element added into the List? How to append elements at the end of ArrayList in Java? Removing Duplicate Values from ArrayList How to declare an ArrayList with values? In Java, can you modify a List while iterating through it? Load arrayList data into JTable