[java] Adding two Java 8 streams, or an extra element to a stream

I can add streams or extra elements, like this:

Stream stream = Stream.concat(stream1, Stream.concat(stream2, Stream.of(element));

And I can add new stuff as I go, like this:

Stream stream = Stream.concat(
                       Stream.concat(
                              stream1.filter(x -> x!=0), stream2)
                              .filter(x -> x!=1),
                                  Stream.of(element))
                                  .filter(x -> x!=2);

But this is ugly, because concat is static. If concat were an instance method, the above examples would be much easier to read:

 Stream stream = stream1.concat(stream2).concat(element);

And

 Stream stream = stream1
                 .filter(x -> x!=0)
                 .concat(stream2)
                 .filter(x -> x!=1)
                 .concat(element)
                 .filter(x -> x!=2);

My question is:

1) Is there any good reason why concat is static? Or is there some equivalent instance method I'm missing?

2) In any case, is there a better way of doing this?

This question is related to java concat java-8 java-stream

The answer is


Just do:

Stream.of(stream1, stream2, Stream.of(element)).flatMap(identity());

where identity() is a static import of Function.identity().

Concatenating multiple streams into one stream is the same as flattening a stream.

However, unfortunately, for some reason there is no flatten() method on Stream, so you have to use flatMap() with the identity function.


How about writing your own concat method?

public static Stream<T> concat(Stream<? extends T> a, 
                               Stream<? extends T> b, 
                               Stream<? extends T> args)
{
    Stream<T> concatenated = Stream.concat(a, b);
    for (Stream<T> stream : args)
    {
        concatenated = Stream.concat(concatenated, stream);
    }
    return concatenated;
}

This at least makes your first example a lot more readable.


If you don't mind using 3rd Party Libraries cyclops-react has an extended Stream type that will allow you to do just that via the append / prepend operators.

Individual values, arrays, iterables, Streams or reactive-streams Publishers can be appended and prepended as instance methods.

Stream stream = ReactiveSeq.of(1,2)
                           .filter(x -> x!=0)
                           .append(ReactiveSeq.of(3,4))
                           .filter(x -> x!=1)
                           .append(5)
                           .filter(x -> x!=2);

[Disclosure I am the lead developer of cyclops-react]


My StreamEx library extends the functionality of Stream API. In particular it offers methods like append and prepend which solve this issue (internally they use concat). These methods can accept either another stream or collection or varargs array. Using my library your problem can be solved this way (note that x != 0 look strange for non-primitive stream):

Stream<Integer> stream = StreamEx.of(stream1)
             .filter(x -> !x.equals(0))
             .append(stream2)
             .filter(x -> !x.equals(1))
             .append(element)
             .filter(x -> !x.equals(2));

By the way there's also a shortcut for your filter operation:

Stream<Integer> stream = StreamEx.of(stream1).without(0)
                                 .append(stream2).without(1)
                                 .append(element).without(2);

You can use Guava's Streams.concat(Stream<? extends T>... streams) method, which will be very short with static imports:

Stream stream = concat(stream1, stream2, of(element));

At the end of the day I'm not interested in combining streams, but in obtaining the combined result of processing each element of all those streams.

While combining streams might prove to be cumbersome (thus this thread), combining their processing results is fairly easy.

The key to solve is to create your own collector and ensure that the supplier function for the new collector returns the same collection every time (not a new one), the code below illustrates this approach.

package scratchpad;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collector;
import java.util.stream.Stream;

public class CombineStreams {
    public CombineStreams() {
        super();
    }

    public static void main(String[] args) {
        List<String> resultList = new ArrayList<>();
        Collector<String, List<String>, List<String>> collector = Collector.of(
                () -> resultList,
                (list, item) -> {
                    list.add(item);
                },
                (llist, rlist) -> {
                    llist.addAll(rlist);
                    return llist;
                }
        );
        String searchString = "Wil";

        System.out.println("After processing first stream\n"
                + createFirstStream().filter(name -> name.contains(searchString)).collect(collector));
        System.out.println();

        System.out.println("After processing second stream\n"
                + createSecondStream().filter(name -> name.contains(searchString)).collect(collector));
        System.out.println();

        System.out.println("After processing third stream\n"
                + createThirdStream().filter(name -> name.contains(searchString)).collect(collector));
        System.out.println();

    }

    private static Stream<String> createFirstStream() {
        return Arrays.asList(
                "William Shakespeare",
                "Emily Dickinson",
                "H. P. Lovecraft",
                "Arthur Conan Doyle",
                "Leo Tolstoy",
                "Edgar Allan Poe",
                "Robert Ervin Howard",
                "Rabindranath Tagore",
                "Rudyard Kipling",
                "Seneca",
                "John Donne",
                "Sarah Williams",
                "Oscar Wilde",
                "Catullus",
                "Alfred Tennyson",
                "William Blake",
                "Charles Dickens",
                "John Keats",
                "Theodor Herzl"
        ).stream();
    }

    private static Stream<String> createSecondStream() {
        return Arrays.asList(
                "Percy Bysshe Shelley",
                "Ernest Hemingway",
                "Barack Obama",
                "Anton Chekhov",
                "Henry Wadsworth Longfellow",
                "Arthur Schopenhauer",
                "Jacob De Haas",
                "George Gordon Byron",
                "Jack London",
                "Robert Frost",
                "Abraham Lincoln",
                "O. Henry",
                "Ovid",
                "Robert Louis Stevenson",
                "John Masefield",
                "James Joyce",
                "Clark Ashton Smith",
                "Aristotle",
                "William Wordsworth",
                "Jane Austen"
        ).stream();
    }

    private static Stream<String> createThirdStream() {
        return Arrays.asList(
                "Niccolò Machiavelli",
                "Lewis Carroll",
                "Robert Burns",
                "Edgar Rice Burroughs",
                "Plato",
                "John Milton",
                "Ralph Waldo Emerson",
                "Margaret Thatcher",
                "Sylvie d'Avigdor",
                "Marcus Tullius Cicero",
                "Banjo Paterson",
                "Woodrow Wilson",
                "Walt Whitman",
                "Theodore Roosevelt",
                "Agatha Christie",
                "Ambrose Bierce",
                "Nikola Tesla",
                "Franz Kafka"
        ).stream();
    }
}

Unfortunately this answer is probably of little or no help whatsoever, but I did a forensics analysis of the Java Lambda Mailing list to see if I could find the cause of this design. This is what I found out.

In the beginning there was an instance method for Stream.concat(Stream)

In the mailing list I can clearly see the method was originally implemented as an instance method, as you can read in this thread by Paul Sandoz, about the concat operation.

In it they discuss the issues that could arise from those cases in which the stream could be infinite and what concatenation would mean in those cases, but I do not think that was the reason for the modification.

You see in this other thread that some early users of the JDK 8 questioned about the behavior of the concat instance method when used with null arguments.

This other thread reveals, though, that the design of the concat method was under discussion.

Refactored to Streams.concat(Stream,Stream)

But without any explanation, suddenly, the methods were changed to static methods, as you can see in this thread about combining streams. This is perhaps the only mail thread that sheds a bit of light about this change, but it was not clear enough for me to determine the reason for the refactoring. But we can see they did a commit in which they suggested to move the concat method out of Stream and into the helper class Streams.

Refactored to Stream.concat(Stream,Stream)

Later, it was moved again from Streams to Stream, but yet again, no explanation for that.

So, bottom line, the reason for the design is not entirely clear for me and I could not find a good explanation. I guess you could still ask the question in the mailing list.

Some Alternatives for Stream Concatenation

This other thread by Michael Hixson discusses/asks about other ways to combine/concat streams

  1. To combine two streams, I should do this:

    Stream.concat(s1, s2)
    

    not this:

    Stream.of(s1, s2).flatMap(x -> x)
    

    ... right?

  2. To combine more than two streams, I should do this:

    Stream.of(s1, s2, s3, ...).flatMap(x -> x)
    

    not this:

    Stream.of(s1, s2, s3, ...).reduce(Stream.empty(), Stream::concat)
    

    ... right?


Examples related to java

Under what circumstances can I call findViewById with an Options Menu / Action Bar item? How much should a function trust another function How to implement a simple scenario the OO way Two constructors How do I get some variable from another class in Java? this in equals method How to split a string in two and store it in a field How to do perspective fixing? String index out of range: 4 My eclipse won't open, i download the bundle pack it keeps saying error log

Examples related to concat

Merge two dataframes by index Concatenate a list of pandas dataframes together How do I concatenate strings in Swift? Adding two Java 8 streams, or an extra element to a stream MySQL CONCAT returns NULL if any field contain NULL How to concat two ArrayLists? Concat a string to SELECT * MySql How to use GROUP_CONCAT in a CONCAT in MySQL Which is the preferred way to concatenate a string in Python? Prepend text to beginning of string

Examples related to java-8

Default interface methods are only supported starting with Android N Class has been compiled by a more recent version of the Java Environment Why is ZoneOffset.UTC != ZoneId.of("UTC")? Modify property value of the objects in list using Java 8 streams How to use if-else logic in Java 8 stream forEach Android Studio Error: Error:CreateProcess error=216, This version of %1 is not compatible with the version of Windows you're running Error:could not create the Java Virtual Machine Error:A fatal exception has occured.Program will exit What are functional interfaces used for in Java 8? java.time.format.DateTimeParseException: Text could not be parsed at index 21 Java 8 lambda get and remove element from list

Examples related to java-stream

Sorting a list with stream.sorted() in Java Modify property value of the objects in list using Java 8 streams How to use if-else logic in Java 8 stream forEach Java 8 lambda get and remove element from list Create list of object from another using Java 8 Streams Java 8 Stream API to find Unique Object matching a property value Reverse a comparator in Java 8 Ignore duplicates when producing map using streams Modifying Objects within stream in Java8 while iterating How can I get a List from some class properties with Java 8 Stream?