[java] Using Java 8's Optional with Stream::flatMap

The new Java 8 stream framework and friends make for some very concise java code, but I have come across a seemingly-simple situation that is tricky to do concisely.

Consider a List<Thing> things and method Optional<Other> resolve(Thing thing). I want to map the Things to Optional<Other>s and get the first Other. The obvious solution would be to use things.stream().flatMap(this::resolve).findFirst(), but flatMap requires that you return a stream, and Optional doesn't have a stream() method (or is it a Collection or provide a method to convert it to or view it as a Collection).

The best I can come up with is this:

things.stream()
    .map(this::resolve)
    .filter(Optional::isPresent)
    .map(Optional::get)
    .findFirst();

But that seems awfully long-winded for what seems like a very common case. Anyone have a better idea?

A slightly shorter version using reduce:

things.stream()
  .map(this::resolve)
  .reduce(Optional.empty(), (a, b) -> a.isPresent() ? a : b );

You could also move the reduce function to a static utility method and then it becomes:

  .reduce(Optional.empty(), Util::firstPresent );

Late to the party, but what about

things.stream()
    .map(this::resolve)
    .filter(Optional::isPresent)
    .findFirst().get();

You can get rid of the last get() if you create a util method to convert optional to stream manually:

things.stream()
    .map(this::resolve)
    .flatMap(Util::optionalToStream)
    .findFirst();

If you return stream right away from your resolve function, you save one more line.

What about that?

private static List<String> extractString(List<Optional<String>> list) {
    List<String> result = new ArrayList<>();
    list.forEach(element -> element.ifPresent(result::add));
    return result;
}

https://stackoverflow.com/a/58281000/3477539

If you're stuck with Java 8 but have access to Guava 21.0 or newer, you can use Streams.stream to convert an optional into a stream.

Thus, given

import com.google.common.collect.Streams;

you can write

Optional<Other> result =
    things.stream()
        .map(this::resolve)
        .flatMap(Streams::stream)
        .findFirst();

I'm adding this second answer based on a proposed edit by user srborlongan to my other answer. I think the technique proposed was interesting, but it wasn't really suitable as an edit to my answer. Others agreed and the proposed edit was voted down. (I wasn't one of the voters.) The technique has merit, though. It would have been best if srborlongan had posted his/her own answer. This hasn't happened yet, and I didn't want the technique to be lost in the mists of the StackOverflow rejected edit history, so I decided to surface it as a separate answer myself.

Basically the technique is to use some of the Optional methods in a clever way to avoid having to use a ternary operator (? :) or an if/else statement.

My inline example would be rewritten this way:

Optional<Other> result =
    things.stream()
          .map(this::resolve)
          .flatMap(o -> o.map(Stream::of).orElseGet(Stream::empty))
          .findFirst();

An my example that uses a helper method would be rewritten this way:

/**
 * Turns an Optional<T> into a Stream<T> of length zero or one depending upon
 * whether a value is present.
 */
static <T> Stream<T> streamopt(Optional<T> opt) {
    return opt.map(Stream::of)
              .orElseGet(Stream::empty);
}

Optional<Other> result =
    things.stream()
          .flatMap(t -> streamopt(resolve(t)))
          .findFirst();

COMMENTARY

Let's compare the original vs modified versions directly:

// original
.flatMap(o -> o.isPresent() ? Stream.of(o.get()) : Stream.empty())

// modified
.flatMap(o -> o.map(Stream::of).orElseGet(Stream::empty))

The original is a straightforward if workmanlike approach: we get an Optional<Other>; if it has a value, we return a stream containing that value, and if it has no value, we return an empty stream. Pretty simple and easy to explain.

The modification is clever and has the advantage that it avoids conditionals. (I know that some people dislike the ternary operator. If misused it can indeed make code hard to understand.) However, sometimes things can be too clever. The modified code also starts off with an Optional<Other>. Then it calls Optional.map which is defined as follows:

If a value is present, apply the provided mapping function to it, and if the result is non-null, return an Optional describing the result. Otherwise return an empty Optional.

The map(Stream::of) call returns an Optional<Stream<Other>>. If a value was present in the input Optional, the returned Optional contains a Stream that contains the single Other result. But if the value was not present, the result is an empty Optional.

Next, the call to orElseGet(Stream::empty) returns a value of type Stream<Other>. If its input value is present, it gets the value, which is the single-element Stream<Other>. Otherwise (if the input value is absent) it returns an empty Stream<Other>. So the result is correct, the same as the original conditional code.

In the comments discussing on my answer, regarding the rejected edit, I had described this technique as "more concise but also more obscure". I stand by this. It took me a while to figure out what it was doing, and it also took me a while to write up the above description of what it was doing. The key subtlety is the transformation from Optional<Other> to Optional<Stream<Other>>. Once you grok this it makes sense, but it wasn't obvious to me.

I'll acknowledge, though, that things that are initially obscure can become idiomatic over time. It might be that this technique ends up being the best way in practice, at least until Optional.stream gets added (if it ever does).

UPDATE: Optional.stream has been added to JDK 9.

As my previous answer appeared not to be very popular, I will give this another go.

A short answer:

You are mostly on a right track. The shortest code to get to your desired output I could come up with is this:

things.stream()
      .map(this::resolve)
      .filter(Optional::isPresent)
      .findFirst()
      .flatMap( Function.identity() );

This will fit all your requirements:

1. It will find first response that resolves to a nonempty Optional<Result>
2. It calls this::resolve lazily as needed
3. this::resolve will not be called after first non-empty result
4. It will return Optional<Result>

Longer answer

The only modification compared to OP initial version was that I removed .map(Optional::get) before call to .findFirst() and added .flatMap(o -> o) as the last call in the chain.

This has a nice effect of getting rid of the double-Optional, whenever stream finds an actual result.

You can't really go any shorter than this in Java.

The alternative snippet of code using the more conventional for loop technique is going to be about same number of lines of code and have more or less same order and number of operations you need to perform:

1. Calling this.resolve,
2. filtering based on Optional.isPresent
3. returning the result and
4. some way of dealing with negative result (when nothing was found)

Just to prove that my solution works as advertised, I wrote a small test program:

public class StackOverflow {

    public static void main( String... args ) {
        try {
            final int integer = Stream.of( args )
                    .peek( s -> System.out.println( "Looking at " + s ) )
                    .map( StackOverflow::resolve )
                    .filter( Optional::isPresent )
                    .findFirst()
                    .flatMap( o -> o )
                    .orElseThrow( NoSuchElementException::new )
                    .intValue();

            System.out.println( "First integer found is " + integer );
        }
        catch ( NoSuchElementException e ) {
            System.out.println( "No integers provided!" );
        }
    }

    private static Optional<Integer> resolve( String string ) {
        try {
            return Optional.of( Integer.valueOf( string ) );
        }
        catch ( NumberFormatException e )
        {
            System.out.println( '"' + string + '"' + " is not an integer");
            return Optional.empty();
        }
    }

}

(It does have few extra lines for debugging and verifying that only as many calls to resolve as needed...)

Executing this on a command line, I got the following results:

$ java StackOferflow a b 3 c 4
Looking at a
"a" is not an integer
Looking at b
"b" is not an integer
Looking at 3
First integer found is 3

I'd like to promote factory methods for creating helpers for functional APIs:

Optional<R> result = things.stream()
        .flatMap(streamopt(this::resolve))
        .findFirst();

The factory method:

<T, R> Function<T, Stream<R>> streamopt(Function<T, Optional<R>> f) {
    return f.andThen(Optional::stream); // or the J8 alternative:
    // return t -> f.apply(t).map(Stream::of).orElseGet(Stream::empty);
}

Reasoning:

• As with method references in general, compared to lambda expressions, you can't accidentaly capture a variable from the accessible scope, like:

  t -> streamopt(resolve(o))

• It's composable, you can e.g. call Function::andThen on the factory method result:

  streamopt(this::resolve).andThen(...)

  Whereas in the case of a lambda, you'd need to cast it first:

  ((Function<T, Stream<R>>) t -> streamopt(resolve(t))).andThen(...)

You cannot do it more concise as you are already doing.

You claim that you do not want .filter(Optional::isPresent) and .map(Optional::get).

This has been resolved by the method @StuartMarks describes, however as a result you now map it to an Optional<T>, so now you need to use .flatMap(this::streamopt) and a get() in the end.

So it still consists of two statements and you can now get exceptions with the new method! Because, what if every optional is empty? Then the findFirst() will return an empty optional and your get() will fail!

So what you have:

things.stream()
    .map(this::resolve)
    .filter(Optional::isPresent)
    .map(Optional::get)
    .findFirst();

is actually the best way to accomplish what you want, and that is you want to save the result as a T, not as an Optional<T>.

I took the liberty of creating a CustomOptional<T> class that wraps the Optional<T> and provides an extra method, flatStream(). Note that you cannot extend Optional<T>:

class CustomOptional<T> {
    private final Optional<T> optional;

    private CustomOptional() {
        this.optional = Optional.empty();
    }

    private CustomOptional(final T value) {
        this.optional = Optional.of(value);
    }

    private CustomOptional(final Optional<T> optional) {
        this.optional = optional;
    }

    public Optional<T> getOptional() {
        return optional;
    }

    public static <T> CustomOptional<T> empty() {
        return new CustomOptional<>();
    }

    public static <T> CustomOptional<T> of(final T value) {
        return new CustomOptional<>(value);
    }

    public static <T> CustomOptional<T> ofNullable(final T value) {
        return (value == null) ? empty() : of(value);
    }

    public T get() {
        return optional.get();
    }

    public boolean isPresent() {
        return optional.isPresent();
    }

    public void ifPresent(final Consumer<? super T> consumer) {
        optional.ifPresent(consumer);
    }

    public CustomOptional<T> filter(final Predicate<? super T> predicate) {
        return new CustomOptional<>(optional.filter(predicate));
    }

    public <U> CustomOptional<U> map(final Function<? super T, ? extends U> mapper) {
        return new CustomOptional<>(optional.map(mapper));
    }

    public <U> CustomOptional<U> flatMap(final Function<? super T, ? extends CustomOptional<U>> mapper) {
        return new CustomOptional<>(optional.flatMap(mapper.andThen(cu -> cu.getOptional())));
    }

    public T orElse(final T other) {
        return optional.orElse(other);
    }

    public T orElseGet(final Supplier<? extends T> other) {
        return optional.orElseGet(other);
    }

    public <X extends Throwable> T orElseThrow(final Supplier<? extends X> exceptionSuppier) throws X {
        return optional.orElseThrow(exceptionSuppier);
    }

    public Stream<T> flatStream() {
        if (!optional.isPresent()) {
            return Stream.empty();
        }
        return Stream.of(get());
    }

    public T getTOrNull() {
        if (!optional.isPresent()) {
            return null;
        }
        return get();
    }

    @Override
    public boolean equals(final Object obj) {
        return optional.equals(obj);
    }

    @Override
    public int hashCode() {
        return optional.hashCode();
    }

    @Override
    public String toString() {
        return optional.toString();
    }
}

You will see that I added flatStream(), as here:

public Stream<T> flatStream() {
    if (!optional.isPresent()) {
        return Stream.empty();
    }
    return Stream.of(get());
}

Used as:

String result = Stream.of("a", "b", "c", "de", "fg", "hij")
        .map(this::resolve)
        .flatMap(CustomOptional::flatStream)
        .findFirst()
        .get();

You still will need to return a Stream<T> here, as you cannot return T, because if !optional.isPresent(), then T == null if you declare it such, but then your .flatMap(CustomOptional::flatStream) would attempt to add null to a stream and that is not possible.

As example:

public T getTOrNull() {
    if (!optional.isPresent()) {
        return null;
    }
    return get();
}

Used as:

String result = Stream.of("a", "b", "c", "de", "fg", "hij")
        .map(this::resolve)
        .map(CustomOptional::getTOrNull)
        .findFirst()
        .get();

Will now throw a NullPointerException inside the stream operations.

Conclusion

The method you used, is actually the best method.

Most likely You are doing it wrong.

Java 8 Optional is not meant to be used in this manner. It is usually only reserved for terminal stream operations that may or may not return a value, like find for example.

In your case it might be better to first try to find a cheap way to filter out those items that are resolvable and then get the first item as an optional and resolve it as a last operation. Better yet - instead of filtering, find the first resolvable item and resolve it.

things.filter(Thing::isResolvable)
      .findFirst()
      .flatMap(this::resolve)
      .get();

Rule of thumb is that you should strive to reduce number of items in the stream before you transform them to something else. YMMV of course.

Null is supported by the Stream provided My library AbacusUtil. Here is code:

Stream.of(things).map(e -> resolve(e).orNull()).skipNull().first();

If you don't mind to use a third party library you may use Javaslang. It is like Scala, but implemented in Java.

It comes with a complete immutable collection library that is very similar to that known from Scala. These collections replace Java's collections and Java 8's Stream. It also has its own implementation of Option.

import javaslang.collection.Stream;
import javaslang.control.Option;

Stream<Option<String>> options = Stream.of(Option.some("foo"), Option.none(), Option.some("bar"));

// = Stream("foo", "bar")
Stream<String> strings = options.flatMap(o -> o);

Here is a solution for the example of the initial question:

import javaslang.collection.Stream;
import javaslang.control.Option;

public class Test {

    void run() {

        // = Stream(Thing(1), Thing(2), Thing(3))
        Stream<Thing> things = Stream.of(new Thing(1), new Thing(2), new Thing(3));

        // = Some(Other(2))
        Option<Other> others = things.flatMap(this::resolve).headOption();
    }

    Option<Other> resolve(Thing thing) {
        Other other = (thing.i % 2 == 0) ? new Other(i + "") : null;
        return Option.of(other);
    }

}

class Thing {
    final int i;
    Thing(int i) { this.i = i; }
    public String toString() { return "Thing(" + i + ")"; }
}

class Other {
    final String s;
    Other(String s) { this.s = s; }
    public String toString() { return "Other(" + s + ")"; }
}

Disclaimer: I'm the creator of Javaslang.