In my case it was a proxy issue (requests proxied from nginx to a varnish cache) that caused the issue. I needed to add the following to my proxy definition
proxy_set_header Connection keep-alive;
I found the answer here: https://stackoverflow.com/a/55341260/1062129