[java] How to add elements of a Java8 stream into an existing List

NOTE: nosid's answer shows how to add to an existing collection using forEachOrdered(). This is a useful and effective technique for mutating existing collections. My answer addresses why you shouldn't use a Collector to mutate an existing collection.

The short answer is no, at least, not in general, you shouldn't use a Collector to modify an existing collection.

The reason is that collectors are designed to support parallelism, even over collections that aren't thread-safe. The way they do this is to have each thread operate independently on its own collection of intermediate results. The way each thread gets its own collection is to call the Collector.supplier() which is required to return a new collection each time.

These collections of intermediate results are then merged, again in a thread-confined fashion, until there is a single result collection. This is the final result of the collect() operation.

A couple answers from Balder and assylias have suggested using Collectors.toCollection() and then passing a supplier that returns an existing list instead of a new list. This violates the requirement on the supplier, which is that it return a new, empty collection each time.

This will work for simple cases, as the examples in their answers demonstrate. However, it will fail, particularly if the stream is run in parallel. (A future version of the library might change in some unforeseen way that will cause it to fail, even in the sequential case.)

Let's take a simple example:

List<String> destList = new ArrayList<>(Arrays.asList("foo"));
List<String> newList = Arrays.asList("0", "1", "2", "3", "4", "5");
newList.parallelStream()
       .collect(Collectors.toCollection(() -> destList));
System.out.println(destList);

When I run this program, I often get an ArrayIndexOutOfBoundsException. This is because multiple threads are operating on ArrayList, a thread-unsafe data structure. OK, let's make it synchronized:

List<String> destList =
    Collections.synchronizedList(new ArrayList<>(Arrays.asList("foo")));

This will no longer fail with an exception. But instead of the expected result:

[foo, 0, 1, 2, 3]

it gives weird results like this:

[foo, 2, 3, foo, 2, 3, 1, 0, foo, 2, 3, foo, 2, 3, 1, 0, foo, 2, 3, foo, 2, 3, 1, 0, foo, 2, 3, foo, 2, 3, 1, 0]

This is the result of the thread-confined accumulation/merging operations I described above. With a parallel stream, each thread calls the supplier to get its own collection for intermediate accumulation. If you pass a supplier that returns the same collection, each thread appends its results to that collection. Since there is no ordering among the threads, results will be appended in some arbitrary order.

Then, when these intermediate collections are merged, this basically merges the list with itself. Lists are merged using List.addAll(), which says that the results are undefined if the source collection is modified during the operation. In this case, ArrayList.addAll() does an array-copy operation, so it ends up duplicating itself, which is sort-of what one would expect, I guess. (Note that other List implementations might have completely different behavior.) Anyway, this explains the weird results and duplicated elements in the destination.

You might say, "I'll just make sure to run my stream sequentially" and go ahead and write code like this

stream.collect(Collectors.toCollection(() -> existingList))

anyway. I'd recommend against doing this. If you control the stream, sure, you can guarantee that it won't run in parallel. I expect that a style of programming will emerge where streams get handed around instead of collections. If somebody hands you a stream and you use this code, it'll fail if the stream happens to be parallel. Worse, somebody might hand you a sequential stream and this code will work fine for a while, pass all tests, etc. Then, some arbitrary amount of time later, code elsewhere in the system might change to use parallel streams which will cause your code to break.

OK, then just make sure to remember to call sequential() on any stream before you use this code:

stream.sequential().collect(Collectors.toCollection(() -> existingList))

Of course, you'll remember to do this every time, right? :-) Let's say you do. Then, the performance team will be wondering why all their carefully crafted parallel implementations aren't providing any speedup. And once again they'll trace it down to your code which is forcing the entire stream to run sequentially.

Don't do it.

Examples related to java

Under what circumstances can I call findViewById with an Options Menu / Action Bar item? How much should a function trust another function How to implement a simple scenario the OO way Two constructors How do I get some variable from another class in Java? this in equals method How to split a string in two and store it in a field How to do perspective fixing? String index out of range: 4 My eclipse won't open, i download the bundle pack it keeps saying error log

Examples related to java-8

Default interface methods are only supported starting with Android N Class has been compiled by a more recent version of the Java Environment Why is ZoneOffset.UTC != ZoneId.of("UTC")? Modify property value of the objects in list using Java 8 streams How to use if-else logic in Java 8 stream forEach Android Studio Error: Error:CreateProcess error=216, This version of %1 is not compatible with the version of Windows you're running Error:could not create the Java Virtual Machine Error:A fatal exception has occured.Program will exit What are functional interfaces used for in Java 8? java.time.format.DateTimeParseException: Text could not be parsed at index 21 Java 8 lambda get and remove element from list

Examples related to java-stream

Sorting a list with stream.sorted() in Java Modify property value of the objects in list using Java 8 streams How to use if-else logic in Java 8 stream forEach Java 8 lambda get and remove element from list Create list of object from another using Java 8 Streams Java 8 Stream API to find Unique Object matching a property value Reverse a comparator in Java 8 Ignore duplicates when producing map using streams Modifying Objects within stream in Java8 while iterating How can I get a List from some class properties with Java 8 Stream?

Examples related to collectors

Hashmap with Streams in Java 8 Streams to collect value of Map Java8: HashMap<X, Y> to HashMap<X, Z> using Stream / Map-Reduce / Collector Java 8 NullPointerException in Collectors.toMap How to add elements of a Java8 stream into an existing List