I have an ArrayList<String>, and I want to remove repeated strings from it. How can I do this?
This question is related to
java
list
collections
arraylist
duplicates
If you don't want duplicates in a Collection, you should consider why you're using a Collection that allows duplicates. The easiest way to remove repeated elements is to add the contents to a Set (which will not allow duplicates) and then add the Set back to the ArrayList:
Set<String> set = new HashSet<>(yourList);
yourList.clear();
yourList.addAll(set);
Of course, this destroys the ordering of the elements in the ArrayList.
Although converting the ArrayList to a HashSet effectively removes duplicates, if you need to preserve insertion order, I'd rather suggest you to use this variant
// list is some List of Strings
Set<String> s = new LinkedHashSet<>(list);
Then, if you need to get back a List reference, you can use again the conversion constructor.
Here is my answer without using any other data structure like set or hashmap etc.
public static <T> ArrayList<T> uniquefy(ArrayList<T> myList) {
ArrayList <T> uniqueArrayList = new ArrayList<T>();
for (int i = 0; i < myList.size(); i++){
if (!uniqueArrayList.contains(myList.get(i))){
uniqueArrayList.add(myList.get(i));
}
}
return uniqueArrayList;
}
Set<String> strSet = strList.stream().collect(Collectors.toSet());
Is the easiest way to remove your duplicates.
As said before, you should use a class implementing the Set interface instead of List to be sure of the unicity of elements. If you have to keep the order of elements, the SortedSet interface can then be used; the TreeSet class implements that interface.
This is used for your Custom Objects list
public List<Contact> removeDuplicates(List<Contact> list) {
// Set set1 = new LinkedHashSet(list);
Set set = new TreeSet(new Comparator() {
@Override
public int compare(Object o1, Object o2) {
if (((Contact) o1).getId().equalsIgnoreCase(((Contact) o2).getId()) /*&&
((Contact)o1).getName().equalsIgnoreCase(((Contact)o2).getName())*/) {
return 0;
}
return 1;
}
});
set.addAll(list);
final List newList = new ArrayList(set);
return newList;
}
LinkedHashSet will do the trick.
String[] arr2 = {"5","1","2","3","3","4","1","2"};
Set<String> set = new LinkedHashSet<String>(Arrays.asList(arr2));
for(String s1 : set)
System.out.println(s1);
System.out.println( "------------------------" );
String[] arr3 = set.toArray(new String[0]);
for(int i = 0; i < arr3.length; i++)
System.out.println(arr3[i].toString());
//output: 5,1,2,3,4
Probably a bit overkill, but I enjoy this kind of isolated problem. :)
This code uses a temporary Set (for the uniqueness check) but removes elements directly inside the original list. Since element removal inside an ArrayList can induce a huge amount of array copying, the remove(int)-method is avoided.
public static <T> void removeDuplicates(ArrayList<T> list) {
int size = list.size();
int out = 0;
{
final Set<T> encountered = new HashSet<T>();
for (int in = 0; in < size; in++) {
final T t = list.get(in);
final boolean first = encountered.add(t);
if (first) {
list.set(out++, t);
}
}
}
while (out < size) {
list.remove(--size);
}
}
While we're at it, here's a version for LinkedList (a lot nicer!):
public static <T> void removeDuplicates(LinkedList<T> list) {
final Set<T> encountered = new HashSet<T>();
for (Iterator<T> iter = list.iterator(); iter.hasNext(); ) {
final T t = iter.next();
final boolean first = encountered.add(t);
if (!first) {
iter.remove();
}
}
}
Use the marker interface to present a unified solution for List:
public static <T> void removeDuplicates(List<T> list) {
if (list instanceof RandomAccess) {
// use first version here
} else {
// use other version here
}
}
EDIT: I guess the generics-stuff doesn't really add any value here.. Oh well. :)
If you're willing to use a third-party library, you can use the method distinct() in Eclipse Collections (formerly GS Collections).
ListIterable<Integer> integers = FastList.newListWith(1, 3, 1, 2, 2, 1);
Assert.assertEquals(
FastList.newListWith(1, 3, 2),
integers.distinct());
The advantage of using distinct() instead of converting to a Set and then back to a List is that distinct() preserves the order of the original List, retaining the first occurrence of each element. It's implemented by using both a Set and a List.
MutableSet<T> seenSoFar = UnifiedSet.newSet();
int size = list.size();
for (int i = 0; i < size; i++)
{
T item = list.get(i);
if (seenSoFar.add(item))
{
targetCollection.add(item);
}
}
return targetCollection;
If you cannot convert your original List into an Eclipse Collections type, you can use ListAdapter to get the same API.
MutableList<Integer> distinct = ListAdapter.adapt(integers).distinct();
Note: I am a committer for Eclipse Collections.
you can use nested loop in follow :
ArrayList<Class1> l1 = new ArrayList<Class1>();
ArrayList<Class1> l2 = new ArrayList<Class1>();
Iterator iterator1 = l1.iterator();
boolean repeated = false;
while (iterator1.hasNext())
{
Class1 c1 = (Class1) iterator1.next();
for (Class1 _c: l2) {
if(_c.getId() == c1.getId())
repeated = true;
}
if(!repeated)
l2.add(c1);
}
If you want to remove duplicates from ArrayList means find the below logic,
public static Object[] removeDuplicate(Object[] inputArray)
{
long startTime = System.nanoTime();
int totalSize = inputArray.length;
Object[] resultArray = new Object[totalSize];
int newSize = 0;
for(int i=0; i<totalSize; i++)
{
Object value = inputArray[i];
if(value == null)
{
continue;
}
for(int j=i+1; j<totalSize; j++)
{
if(value.equals(inputArray[j]))
{
inputArray[j] = null;
}
}
resultArray[newSize++] = value;
}
long endTime = System.nanoTime()-startTime;
System.out.println("Total Time-B:"+endTime);
return resultArray;
}
If you want to preserve your Order then it is best to use LinkedHashSet. Because if you want to pass this List to an Insert Query by Iterating it, the order would be preserved.
Try this
LinkedHashSet link=new LinkedHashSet();
List listOfValues=new ArrayList();
listOfValues.add(link);
This conversion will be very helpful when you want to return a List but not a Set.
Here's a way that doesn't affect your list ordering:
ArrayList l1 = new ArrayList();
ArrayList l2 = new ArrayList();
Iterator iterator = l1.iterator();
while (iterator.hasNext()) {
YourClass o = (YourClass) iterator.next();
if(!l2.contains(o)) l2.add(o);
}
l1 is the original list, and l2 is the list without repeated items (Make sure YourClass has the equals method according to what you want to stand for equality)
There is also ImmutableSet from Guava as an option (here is the documentation):
ImmutableSet.copyOf(list);
It is possible to remove duplicates from arraylist without using HashSet or one more arraylist.
Try this code..
ArrayList<String> lst = new ArrayList<String>();
lst.add("ABC");
lst.add("ABC");
lst.add("ABCD");
lst.add("ABCD");
lst.add("ABCE");
System.out.println("Duplicates List "+lst);
Object[] st = lst.toArray();
for (Object s : st) {
if (lst.indexOf(s) != lst.lastIndexOf(s)) {
lst.remove(lst.lastIndexOf(s));
}
}
System.out.println("Distinct List "+lst);
Output is
Duplicates List [ABC, ABC, ABCD, ABCD, ABCE]
Distinct List [ABC, ABCD, ABCE]
public static void main(String[] args){
ArrayList<Object> al = new ArrayList<Object>();
al.add("abc");
al.add('a');
al.add('b');
al.add('a');
al.add("abc");
al.add(10.3);
al.add('c');
al.add(10);
al.add("abc");
al.add(10);
System.out.println("Before Duplicate Remove:"+al);
for(int i=0;i<al.size();i++){
for(int j=i+1;j<al.size();j++){
if(al.get(i).equals(al.get(j))){
al.remove(j);
j--;
}
}
}
System.out.println("After Removing duplicate:"+al);
}
The @jonathan-stafford solution is OK. But this don't preserve the list order.
If you want preserve the list order you have to use this:
public static <T> void removeDuplicate(List <T> list) {
Set <T> set = new HashSet <T>();
List <T> newList = new ArrayList <T>();
for (Iterator <T>iter = list.iterator(); iter.hasNext(); ) {
Object element = iter.next();
if (set.add((T) element))
newList.add((T) element);
}
list.clear();
list.addAll(newList);
}
It's only to complete the answer. Very good!
This is the right one (if you are concerned about the overhead of HashSet.
public static ArrayList<String> removeDuplicates (ArrayList<String> arrayList){
if (arrayList.isEmpty()) return null; //return what makes sense for your app
Collections.sort(arrayList, String.CASE_INSENSITIVE_ORDER);
//remove duplicates
ArrayList <String> arrayList_mod = new ArrayList<>();
arrayList_mod.add(arrayList.get(0));
for (int i=1; i<arrayList.size(); i++){
if (!arrayList.get(i).equals(arrayList.get(i-1))) arrayList_mod.add(arrayList.get(i));
}
return arrayList_mod;
}
When you are filling the ArrayList, use a condition for each element. For example:
ArrayList< Integer > al = new ArrayList< Integer >();
// fill 1
for ( int i = 0; i <= 5; i++ )
if ( !al.contains( i ) )
al.add( i );
// fill 2
for (int i = 0; i <= 10; i++ )
if ( !al.contains( i ) )
al.add( i );
for( Integer i: al )
{
System.out.print( i + " ");
}
We will get an array {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
Suppose we have a list of String like:
List<String> strList = new ArrayList<>(5);
// insert up to five items to list.
Then we can remove duplicate elements in multiple ways.
List<String> deDupStringList = new ArrayList<>(new HashSet<>(strList));
Note: If we want to maintain the insertion order then we need to use LinkedHashSet in place of HashSet
List<String> deDupStringList2 = Lists.newArrayList(Sets.newHashSet(strList));
List<String> deDupStringList3 = strList.stream().distinct().collect(Collectors.toList());
Note: In case we want to collect the result in a specific list implementation e.g. LinkedList then we can modify the above example as:
List<String> deDupStringList3 = strList.stream().distinct()
.collect(Collectors.toCollection(LinkedList::new));
We can use parallelStream also in the above code but it may not give expected performace benefits. Check this question for more.
If you want your list to automatically ignore duplicates and preserve its order, you could create a HashList(a HashMap embedded List).
public static class HashList<T> extends ArrayList<T>{
private HashMap <T,T> hashMap;
public HashList(){
hashMap=new HashMap<>();
}
@Override
public boolean add(T t){
if(hashMap.get(t)==null){
hashMap.put(t,t);
return super.add(t);
}else return false;
}
@Override
public boolean addAll(Collection<? extends T> c){
HashList<T> addup=(HashList<T>)c;
for(int i=0;i<addup.size();i++){
add(addup.get(i));
}return true;
}
}
Usage Example:
HashList<String> hashlist=new HashList<>();
hashList.add("hello");
hashList.add("hello");
System.out.println(" HashList: "+hashlist);
public Set<Object> findDuplicates(List<Object> list) {
Set<Object> items = new HashSet<Object>();
Set<Object> duplicates = new HashSet<Object>();
for (Object item : list) {
if (items.contains(item)) {
duplicates.add(item);
} else {
items.add(item);
}
}
return duplicates;
}
Java 8 streams provide a very simple way to remove duplicate elements from a list. Using the distinct method. If we have a list of cities and we want to remove duplicates from that list it can be done in a single line -
List<String> cityList = new ArrayList<>();
cityList.add("Delhi");
cityList.add("Mumbai");
cityList.add("Bangalore");
cityList.add("Chennai");
cityList.add("Kolkata");
cityList.add("Mumbai");
cityList = cityList.stream().distinct().collect(Collectors.toList());
public Set<Object> findDuplicates(List<Object> list) {
Set<Object> items = new HashSet<Object>();
Set<Object> duplicates = new HashSet<Object>();
for (Object item : list) {
if (items.contains(item)) {
duplicates.add(item);
} else {
items.add(item);
}
}
return duplicates;
}
If you want to preserve your Order then it is best to use LinkedHashSet. Because if you want to pass this List to an Insert Query by Iterating it, the order would be preserved.
Try this
LinkedHashSet link=new LinkedHashSet();
List listOfValues=new ArrayList();
listOfValues.add(link);
This conversion will be very helpful when you want to return a List but not a Set.
In Java, List permits ordered access of their elements. They can have duplicates because their lookup key is the position not some hash code, every element can be modified while they remain in the list where as Set represents a collection of unique elements and while elements are in set, they must not be modified.While there is no restriction preventing you from modifying elements in a set, if an element is modified, then it could become forever lost in the set.
public static void main(String[] args) {
List<String> l = new ArrayList<String>();
l.add("A");
l.add("B");
l.add("C");
l.add("A");
System.out.println("Before removing duplicates: ");
for (String s : l) {
System.out.println(s);
}
Set<String> set = new HashSet<String>(l);
List<String> newlist = new ArrayList<String>(set);
System.out.println("after removing duplicates: ");
for (String s : newlist) {
System.out.println(s);
}
}
for reference, refer this link How to remove duplicates from ArrayList
public static void main(String[] args){
ArrayList<Object> al = new ArrayList<Object>();
al.add("abc");
al.add('a');
al.add('b');
al.add('a');
al.add("abc");
al.add(10.3);
al.add('c');
al.add(10);
al.add("abc");
al.add(10);
System.out.println("Before Duplicate Remove:"+al);
for(int i=0;i<al.size();i++){
for(int j=i+1;j<al.size();j++){
if(al.get(i).equals(al.get(j))){
al.remove(j);
j--;
}
}
}
System.out.println("After Removing duplicate:"+al);
}
This is the right one (if you are concerned about the overhead of HashSet.
public static ArrayList<String> removeDuplicates (ArrayList<String> arrayList){
if (arrayList.isEmpty()) return null; //return what makes sense for your app
Collections.sort(arrayList, String.CASE_INSENSITIVE_ORDER);
//remove duplicates
ArrayList <String> arrayList_mod = new ArrayList<>();
arrayList_mod.add(arrayList.get(0));
for (int i=1; i<arrayList.size(); i++){
if (!arrayList.get(i).equals(arrayList.get(i-1))) arrayList_mod.add(arrayList.get(i));
}
return arrayList_mod;
}
There is also ImmutableSet from Guava as an option (here is the documentation):
ImmutableSet.copyOf(list);
for(int a=0;a<myArray.size();a++){
for(int b=a+1;b<myArray.size();b++){
if(myArray.get(a).equalsIgnoreCase(myArray.get(b))){
myArray.remove(b);
dups++;
b--;
}
}
}
Time Complexity : O(n) : Without Set
private static void removeDup(ArrayList<String> listWithDuplicateElements) {
System.out.println(" Original Duplicate List :" + listWithDuplicateElements);
List<String> listWithoutDuplicateElements = new ArrayList<>(listWithDuplicateElements.size());
listWithDuplicateElements.stream().forEach(str -> {
if (listWithoutDuplicateElements.indexOf(str) == -1) {
listWithoutDuplicateElements.add(str);
}
});
System.out.println(" Without Duplicate List :" + listWithoutDuplicateElements);
}
Suppose we have a list of String like:
List<String> strList = new ArrayList<>(5);
// insert up to five items to list.
Then we can remove duplicate elements in multiple ways.
List<String> deDupStringList = new ArrayList<>(new HashSet<>(strList));
Note: If we want to maintain the insertion order then we need to use LinkedHashSet in place of HashSet
List<String> deDupStringList2 = Lists.newArrayList(Sets.newHashSet(strList));
List<String> deDupStringList3 = strList.stream().distinct().collect(Collectors.toList());
Note: In case we want to collect the result in a specific list implementation e.g. LinkedList then we can modify the above example as:
List<String> deDupStringList3 = strList.stream().distinct()
.collect(Collectors.toCollection(LinkedList::new));
We can use parallelStream also in the above code but it may not give expected performace benefits. Check this question for more.
Probably a bit overkill, but I enjoy this kind of isolated problem. :)
This code uses a temporary Set (for the uniqueness check) but removes elements directly inside the original list. Since element removal inside an ArrayList can induce a huge amount of array copying, the remove(int)-method is avoided.
public static <T> void removeDuplicates(ArrayList<T> list) {
int size = list.size();
int out = 0;
{
final Set<T> encountered = new HashSet<T>();
for (int in = 0; in < size; in++) {
final T t = list.get(in);
final boolean first = encountered.add(t);
if (first) {
list.set(out++, t);
}
}
}
while (out < size) {
list.remove(--size);
}
}
While we're at it, here's a version for LinkedList (a lot nicer!):
public static <T> void removeDuplicates(LinkedList<T> list) {
final Set<T> encountered = new HashSet<T>();
for (Iterator<T> iter = list.iterator(); iter.hasNext(); ) {
final T t = iter.next();
final boolean first = encountered.add(t);
if (!first) {
iter.remove();
}
}
}
Use the marker interface to present a unified solution for List:
public static <T> void removeDuplicates(List<T> list) {
if (list instanceof RandomAccess) {
// use first version here
} else {
// use other version here
}
}
EDIT: I guess the generics-stuff doesn't really add any value here.. Oh well. :)
This three lines of code can remove the duplicated element from ArrayList or any collection.
List<Entity> entities = repository.findByUserId(userId);
Set<Entity> s = new LinkedHashSet<Entity>(entities);
entities.clear();
entities.addAll(s);
Code:
List<String> duplicatList = new ArrayList<String>();
duplicatList = Arrays.asList("AA","BB","CC","DD","DD","EE","AA","FF");
//above AA and DD are duplicate
Set<String> uniqueList = new HashSet<String>(duplicatList);
duplicatList = new ArrayList<String>(uniqueList); //let GC will doing free memory
System.out.println("Removed Duplicate : "+duplicatList);
Note: Definitely, there will be memory overhead.
If you want to remove duplicates from ArrayList means find the below logic,
public static Object[] removeDuplicate(Object[] inputArray)
{
long startTime = System.nanoTime();
int totalSize = inputArray.length;
Object[] resultArray = new Object[totalSize];
int newSize = 0;
for(int i=0; i<totalSize; i++)
{
Object value = inputArray[i];
if(value == null)
{
continue;
}
for(int j=i+1; j<totalSize; j++)
{
if(value.equals(inputArray[j]))
{
inputArray[j] = null;
}
}
resultArray[newSize++] = value;
}
long endTime = System.nanoTime()-startTime;
System.out.println("Total Time-B:"+endTime);
return resultArray;
}
import java.util.*;
class RemoveDupFrmString
{
public static void main(String[] args)
{
String s="appsc";
Set<Character> unique = new LinkedHashSet<Character> ();
for(char c : s.toCharArray()) {
System.out.println(unique.add(c));
}
for(char dis:unique){
System.out.println(dis);
}
}
}
Here is my answer without using any other data structure like set or hashmap etc.
public static <T> ArrayList<T> uniquefy(ArrayList<T> myList) {
ArrayList <T> uniqueArrayList = new ArrayList<T>();
for (int i = 0; i < myList.size(); i++){
if (!uniqueArrayList.contains(myList.get(i))){
uniqueArrayList.add(myList.get(i));
}
}
return uniqueArrayList;
}
Would something like this work better ?
public static void removeDuplicates(ArrayList<String> list) {
Arraylist<Object> ar = new Arraylist<Object>();
Arraylist<Object> tempAR = new Arraylist<Object>();
while (list.size()>0){
ar.add(list(0));
list.removeall(Collections.singleton(list(0)));
}
list.addAll(ar);
}
That should maintain the order and also not be quadratic in run time.
If you don't want duplicates, use a Set instead of a List. To convert a List to a Set you can use the following code:
// list is some List of Strings
Set<String> s = new HashSet<String>(list);
If really necessary you can use the same construction to convert a Set back into a List.
you can use nested loop in follow :
ArrayList<Class1> l1 = new ArrayList<Class1>();
ArrayList<Class1> l2 = new ArrayList<Class1>();
Iterator iterator1 = l1.iterator();
boolean repeated = false;
while (iterator1.hasNext())
{
Class1 c1 = (Class1) iterator1.next();
for (Class1 _c: l2) {
if(_c.getId() == c1.getId())
repeated = true;
}
if(!repeated)
l2.add(c1);
}
If you don't want duplicates, use a Set instead of a List. To convert a List to a Set you can use the following code:
// list is some List of Strings
Set<String> s = new HashSet<String>(list);
If really necessary you can use the same construction to convert a Set back into a List.
Probably a bit overkill, but I enjoy this kind of isolated problem. :)
This code uses a temporary Set (for the uniqueness check) but removes elements directly inside the original list. Since element removal inside an ArrayList can induce a huge amount of array copying, the remove(int)-method is avoided.
public static <T> void removeDuplicates(ArrayList<T> list) {
int size = list.size();
int out = 0;
{
final Set<T> encountered = new HashSet<T>();
for (int in = 0; in < size; in++) {
final T t = list.get(in);
final boolean first = encountered.add(t);
if (first) {
list.set(out++, t);
}
}
}
while (out < size) {
list.remove(--size);
}
}
While we're at it, here's a version for LinkedList (a lot nicer!):
public static <T> void removeDuplicates(LinkedList<T> list) {
final Set<T> encountered = new HashSet<T>();
for (Iterator<T> iter = list.iterator(); iter.hasNext(); ) {
final T t = iter.next();
final boolean first = encountered.add(t);
if (!first) {
iter.remove();
}
}
}
Use the marker interface to present a unified solution for List:
public static <T> void removeDuplicates(List<T> list) {
if (list instanceof RandomAccess) {
// use first version here
} else {
// use other version here
}
}
EDIT: I guess the generics-stuff doesn't really add any value here.. Oh well. :)
this can solve the problem:
private List<SomeClass> clearListFromDuplicateFirstName(List<SomeClass> list1) {
Map<String, SomeClass> cleanMap = new LinkedHashMap<String, SomeClass>();
for (int i = 0; i < list1.size(); i++) {
cleanMap.put(list1.get(i).getFirstName(), list1.get(i));
}
List<SomeClass> list = new ArrayList<SomeClass>(cleanMap.values());
return list;
}
As said before, you should use a class implementing the Set interface instead of List to be sure of the unicity of elements. If you have to keep the order of elements, the SortedSet interface can then be used; the TreeSet class implements that interface.
this can solve the problem:
private List<SomeClass> clearListFromDuplicateFirstName(List<SomeClass> list1) {
Map<String, SomeClass> cleanMap = new LinkedHashMap<String, SomeClass>();
for (int i = 0; i < list1.size(); i++) {
cleanMap.put(list1.get(i).getFirstName(), list1.get(i));
}
List<SomeClass> list = new ArrayList<SomeClass>(cleanMap.values());
return list;
}
Set<String> strSet = strList.stream().collect(Collectors.toSet());
Is the easiest way to remove your duplicates.
Code:
List<String> duplicatList = new ArrayList<String>();
duplicatList = Arrays.asList("AA","BB","CC","DD","DD","EE","AA","FF");
//above AA and DD are duplicate
Set<String> uniqueList = new HashSet<String>(duplicatList);
duplicatList = new ArrayList<String>(uniqueList); //let GC will doing free memory
System.out.println("Removed Duplicate : "+duplicatList);
Note: Definitely, there will be memory overhead.
If you want your list to automatically ignore duplicates and preserve its order, you could create a HashList(a HashMap embedded List).
public static class HashList<T> extends ArrayList<T>{
private HashMap <T,T> hashMap;
public HashList(){
hashMap=new HashMap<>();
}
@Override
public boolean add(T t){
if(hashMap.get(t)==null){
hashMap.put(t,t);
return super.add(t);
}else return false;
}
@Override
public boolean addAll(Collection<? extends T> c){
HashList<T> addup=(HashList<T>)c;
for(int i=0;i<addup.size();i++){
add(addup.get(i));
}return true;
}
}
Usage Example:
HashList<String> hashlist=new HashList<>();
hashList.add("hello");
hashList.add("hello");
System.out.println(" HashList: "+hashlist);
As said before, you should use a class implementing the Set interface instead of List to be sure of the unicity of elements. If you have to keep the order of elements, the SortedSet interface can then be used; the TreeSet class implements that interface.
Would something like this work better ?
public static void removeDuplicates(ArrayList<String> list) {
Arraylist<Object> ar = new Arraylist<Object>();
Arraylist<Object> tempAR = new Arraylist<Object>();
while (list.size()>0){
ar.add(list(0));
list.removeall(Collections.singleton(list(0)));
}
list.addAll(ar);
}
That should maintain the order and also not be quadratic in run time.
If you don't want duplicates, use a Set instead of a List. To convert a List to a Set you can use the following code:
// list is some List of Strings
Set<String> s = new HashSet<String>(list);
If really necessary you can use the same construction to convert a Set back into a List.
In Java 8:
List<String> deduped = list.stream().distinct().collect(Collectors.toList());
Please note that the hashCode-equals contract for list members should be respected for the filtering to work properly.
ArrayList<String> list = new ArrayList<String>();
HashSet<String> unique = new LinkedHashSet<String>();
HashSet<String> dup = new LinkedHashSet<String>();
boolean b = false;
list.add("Hello");
list.add("Hello");
list.add("how");
list.add("are");
list.add("u");
list.add("u");
for(Iterator iterator= list.iterator();iterator.hasNext();)
{
String value = (String)iterator.next();
System.out.println(value);
if(b==unique.add(value))
dup.add(value);
else
unique.add(value);
}
System.out.println(unique);
System.out.println(dup);
When you are filling the ArrayList, use a condition for each element. For example:
ArrayList< Integer > al = new ArrayList< Integer >();
// fill 1
for ( int i = 0; i <= 5; i++ )
if ( !al.contains( i ) )
al.add( i );
// fill 2
for (int i = 0; i <= 10; i++ )
if ( !al.contains( i ) )
al.add( i );
for( Integer i: al )
{
System.out.print( i + " ");
}
We will get an array {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
You can also do it this way, and preserve order:
// delete duplicates (if any) from 'myArrayList'
myArrayList = new ArrayList<String>(new LinkedHashSet<String>(myArrayList));
Although converting the ArrayList to a HashSet effectively removes duplicates, if you need to preserve insertion order, I'd rather suggest you to use this variant
// list is some List of Strings
Set<String> s = new LinkedHashSet<>(list);
Then, if you need to get back a List reference, you can use again the conversion constructor.
for(int a=0;a<myArray.size();a++){
for(int b=a+1;b<myArray.size();b++){
if(myArray.get(a).equalsIgnoreCase(myArray.get(b))){
myArray.remove(b);
dups++;
b--;
}
}
}
This three lines of code can remove the duplicated element from ArrayList or any collection.
List<Entity> entities = repository.findByUserId(userId);
Set<Entity> s = new LinkedHashSet<Entity>(entities);
entities.clear();
entities.addAll(s);
Here is my code without using any other data structure like set or hashmap
for (int i = 0; i < Models.size(); i++){
for (int j = i + 1; j < Models.size(); j++) {
if (Models.get(i).getName().equals(Models.get(j).getName())) {
Models.remove(j);
j--;
}
}
}
LinkedHashSet will do the trick.
String[] arr2 = {"5","1","2","3","3","4","1","2"};
Set<String> set = new LinkedHashSet<String>(Arrays.asList(arr2));
for(String s1 : set)
System.out.println(s1);
System.out.println( "------------------------" );
String[] arr3 = set.toArray(new String[0]);
for(int i = 0; i < arr3.length; i++)
System.out.println(arr3[i].toString());
//output: 5,1,2,3,4
ArrayList<String> list = new ArrayList<String>();
HashSet<String> unique = new LinkedHashSet<String>();
HashSet<String> dup = new LinkedHashSet<String>();
boolean b = false;
list.add("Hello");
list.add("Hello");
list.add("how");
list.add("are");
list.add("u");
list.add("u");
for(Iterator iterator= list.iterator();iterator.hasNext();)
{
String value = (String)iterator.next();
System.out.println(value);
if(b==unique.add(value))
dup.add(value);
else
unique.add(value);
}
System.out.println(unique);
System.out.println(dup);
Although converting the ArrayList to a HashSet effectively removes duplicates, if you need to preserve insertion order, I'd rather suggest you to use this variant
// list is some List of Strings
Set<String> s = new LinkedHashSet<>(list);
Then, if you need to get back a List reference, you can use again the conversion constructor.
This is used for your Custom Objects list
public List<Contact> removeDuplicates(List<Contact> list) {
// Set set1 = new LinkedHashSet(list);
Set set = new TreeSet(new Comparator() {
@Override
public int compare(Object o1, Object o2) {
if (((Contact) o1).getId().equalsIgnoreCase(((Contact) o2).getId()) /*&&
((Contact)o1).getName().equalsIgnoreCase(((Contact)o2).getName())*/) {
return 0;
}
return 1;
}
});
set.addAll(list);
final List newList = new ArrayList(set);
return newList;
}
You can also do it this way, and preserve order:
// delete duplicates (if any) from 'myArrayList'
myArrayList = new ArrayList<String>(new LinkedHashSet<String>(myArrayList));
If you don't want duplicates, use a Set instead of a List. To convert a List to a Set you can use the following code:
// list is some List of Strings
Set<String> s = new HashSet<String>(list);
If really necessary you can use the same construction to convert a Set back into a List.
It is possible to remove duplicates from arraylist without using HashSet or one more arraylist.
Try this code..
ArrayList<String> lst = new ArrayList<String>();
lst.add("ABC");
lst.add("ABC");
lst.add("ABCD");
lst.add("ABCD");
lst.add("ABCE");
System.out.println("Duplicates List "+lst);
Object[] st = lst.toArray();
for (Object s : st) {
if (lst.indexOf(s) != lst.lastIndexOf(s)) {
lst.remove(lst.lastIndexOf(s));
}
}
System.out.println("Distinct List "+lst);
Output is
Duplicates List [ABC, ABC, ABCD, ABCD, ABCE]
Distinct List [ABC, ABCD, ABCE]
In Java 8:
List<String> deduped = list.stream().distinct().collect(Collectors.toList());
Please note that the hashCode-equals contract for list members should be respected for the filtering to work properly.
In Java, List permits ordered access of their elements. They can have duplicates because their lookup key is the position not some hash code, every element can be modified while they remain in the list where as Set represents a collection of unique elements and while elements are in set, they must not be modified.While there is no restriction preventing you from modifying elements in a set, if an element is modified, then it could become forever lost in the set.
public static void main(String[] args) {
List<String> l = new ArrayList<String>();
l.add("A");
l.add("B");
l.add("C");
l.add("A");
System.out.println("Before removing duplicates: ");
for (String s : l) {
System.out.println(s);
}
Set<String> set = new HashSet<String>(l);
List<String> newlist = new ArrayList<String>(set);
System.out.println("after removing duplicates: ");
for (String s : newlist) {
System.out.println(s);
}
}
for reference, refer this link How to remove duplicates from ArrayList
List<String> result = new ArrayList<String>();
Set<String> set = new LinkedHashSet<String>();
String s = "ravi is a good!boy. But ravi is very nasty fellow.";
StringTokenizer st = new StringTokenizer(s, " ,. ,!");
while (st.hasMoreTokens()) {
result.add(st.nextToken());
}
System.out.println(result);
set.addAll(result);
result.clear();
result.addAll(set);
System.out.println(result);
output:
[ravi, is, a, good, boy, But, ravi, is, very, nasty, fellow]
[ravi, is, a, good, boy, But, very, nasty, fellow]
ArrayList<String> city=new ArrayList<String>();
city.add("rajkot");
city.add("gondal");
city.add("rajkot");
city.add("gova");
city.add("baroda");
city.add("morbi");
city.add("gova");
HashSet<String> hashSet = new HashSet<String>();
hashSet.addAll(city);
city.clear();
city.addAll(hashSet);
Toast.makeText(getActivity(),"" + city.toString(),Toast.LENGTH_SHORT).show();
Here is my code without using any other data structure like set or hashmap
for (int i = 0; i < Models.size(); i++){
for (int j = i + 1; j < Models.size(); j++) {
if (Models.get(i).getName().equals(Models.get(j).getName())) {
Models.remove(j);
j--;
}
}
}
import java.util.*;
class RemoveDupFrmString
{
public static void main(String[] args)
{
String s="appsc";
Set<Character> unique = new LinkedHashSet<Character> ();
for(char c : s.toCharArray()) {
System.out.println(unique.add(c));
}
for(char dis:unique){
System.out.println(dis);
}
}
}
Probably a bit overkill, but I enjoy this kind of isolated problem. :)
This code uses a temporary Set (for the uniqueness check) but removes elements directly inside the original list. Since element removal inside an ArrayList can induce a huge amount of array copying, the remove(int)-method is avoided.
public static <T> void removeDuplicates(ArrayList<T> list) {
int size = list.size();
int out = 0;
{
final Set<T> encountered = new HashSet<T>();
for (int in = 0; in < size; in++) {
final T t = list.get(in);
final boolean first = encountered.add(t);
if (first) {
list.set(out++, t);
}
}
}
while (out < size) {
list.remove(--size);
}
}
While we're at it, here's a version for LinkedList (a lot nicer!):
public static <T> void removeDuplicates(LinkedList<T> list) {
final Set<T> encountered = new HashSet<T>();
for (Iterator<T> iter = list.iterator(); iter.hasNext(); ) {
final T t = iter.next();
final boolean first = encountered.add(t);
if (!first) {
iter.remove();
}
}
}
Use the marker interface to present a unified solution for List:
public static <T> void removeDuplicates(List<T> list) {
if (list instanceof RandomAccess) {
// use first version here
} else {
// use other version here
}
}
EDIT: I guess the generics-stuff doesn't really add any value here.. Oh well. :)
If you're willing to use a third-party library, you can use the method distinct() in Eclipse Collections (formerly GS Collections).
ListIterable<Integer> integers = FastList.newListWith(1, 3, 1, 2, 2, 1);
Assert.assertEquals(
FastList.newListWith(1, 3, 2),
integers.distinct());
The advantage of using distinct() instead of converting to a Set and then back to a List is that distinct() preserves the order of the original List, retaining the first occurrence of each element. It's implemented by using both a Set and a List.
MutableSet<T> seenSoFar = UnifiedSet.newSet();
int size = list.size();
for (int i = 0; i < size; i++)
{
T item = list.get(i);
if (seenSoFar.add(item))
{
targetCollection.add(item);
}
}
return targetCollection;
If you cannot convert your original List into an Eclipse Collections type, you can use ListAdapter to get the same API.
MutableList<Integer> distinct = ListAdapter.adapt(integers).distinct();
Note: I am a committer for Eclipse Collections.
Here's a way that doesn't affect your list ordering:
ArrayList l1 = new ArrayList();
ArrayList l2 = new ArrayList();
Iterator iterator = l1.iterator();
while (iterator.hasNext()) {
YourClass o = (YourClass) iterator.next();
if(!l2.contains(o)) l2.add(o);
}
l1 is the original list, and l2 is the list without repeated items (Make sure YourClass has the equals method according to what you want to stand for equality)
The @jonathan-stafford solution is OK. But this don't preserve the list order.
If you want preserve the list order you have to use this:
public static <T> void removeDuplicate(List <T> list) {
Set <T> set = new HashSet <T>();
List <T> newList = new ArrayList <T>();
for (Iterator <T>iter = list.iterator(); iter.hasNext(); ) {
Object element = iter.next();
if (set.add((T) element))
newList.add((T) element);
}
list.clear();
list.addAll(newList);
}
It's only to complete the answer. Very good!
Although converting the ArrayList to a HashSet effectively removes duplicates, if you need to preserve insertion order, I'd rather suggest you to use this variant
// list is some List of Strings
Set<String> s = new LinkedHashSet<>(list);
Then, if you need to get back a List reference, you can use again the conversion constructor.
List<String> result = new ArrayList<String>();
Set<String> set = new LinkedHashSet<String>();
String s = "ravi is a good!boy. But ravi is very nasty fellow.";
StringTokenizer st = new StringTokenizer(s, " ,. ,!");
while (st.hasMoreTokens()) {
result.add(st.nextToken());
}
System.out.println(result);
set.addAll(result);
result.clear();
result.addAll(set);
System.out.println(result);
output:
[ravi, is, a, good, boy, But, ravi, is, very, nasty, fellow]
[ravi, is, a, good, boy, But, very, nasty, fellow]
Time Complexity : O(n) : Without Set
private static void removeDup(ArrayList<String> listWithDuplicateElements) {
System.out.println(" Original Duplicate List :" + listWithDuplicateElements);
List<String> listWithoutDuplicateElements = new ArrayList<>(listWithDuplicateElements.size());
listWithDuplicateElements.stream().forEach(str -> {
if (listWithoutDuplicateElements.indexOf(str) == -1) {
listWithoutDuplicateElements.add(str);
}
});
System.out.println(" Without Duplicate List :" + listWithoutDuplicateElements);
}
Java 8 streams provide a very simple way to remove duplicate elements from a list. Using the distinct method. If we have a list of cities and we want to remove duplicates from that list it can be done in a single line -
List<String> cityList = new ArrayList<>();
cityList.add("Delhi");
cityList.add("Mumbai");
cityList.add("Bangalore");
cityList.add("Chennai");
cityList.add("Kolkata");
cityList.add("Mumbai");
cityList = cityList.stream().distinct().collect(Collectors.toList());
As said before, you should use a class implementing the Set interface instead of List to be sure of the unicity of elements. If you have to keep the order of elements, the SortedSet interface can then be used; the TreeSet class implements that interface.
ArrayList<String> city=new ArrayList<String>();
city.add("rajkot");
city.add("gondal");
city.add("rajkot");
city.add("gova");
city.add("baroda");
city.add("morbi");
city.add("gova");
HashSet<String> hashSet = new HashSet<String>();
hashSet.addAll(city);
city.clear();
city.addAll(hashSet);
Toast.makeText(getActivity(),"" + city.toString(),Toast.LENGTH_SHORT).show();
Source: Stackoverflow.com