upstream sent too big header while reading response header from upstream

Question

I am getting these kind of errors      2014 05 24 11 49 06  error  8376 0   54031 upstream sent too big header while reading response header from upstream  client  107 21 193 210  server  aamjanata com  request   GET  the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https   aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20https  aamjanata com the-brainwash-chronicles-sponsored-by-gujarat-government   20ht   Always it is the same  A url repeated over and over with comma separating  Can t figure out what is causing this  Anyone have an idea   Update  Another error   http request count is zero while sending response to client   Here is the config  There are other irrelevant things  but this part was added edited   fastcgi cache path  var nginx-cache levels 1 2 keys zone WORDPRESS 100m inactive 60m  fastcgi cache key   scheme request method host request uri   fastcgi cache use stale error timeout invalid header http 500  fastcgi ignore headers Cache-Control Expires Set-Cookie  proxy buffer size   128k  proxy buffers   4 256k  proxy busy buffers size   256k        Upstream to abstract backend connection s  for PHP      upstream php                this should match value of  listen  directive in php-fpm pool             server unix  var run php5-fpm sock          And then in the server block          set  skip cache 0         POST requests and urls with a query string should always go to PHP     if   request method   POST                set  skip cache 1            if   query string                      set  skip cache 1               Don t cache uris containing the following segments     if   request uri      wp-admin   xmlrpc php wp-   php  feed  index php sitemap  index   xml                 set  skip cache 1               Don t use the cache for logged in users or recent commenters     if   http cookie     comment author wordpress  a-f0-9   wp-postpass wordpress no cache wordpress logged in                 set  skip cache 1             location                   This is cool because no php is touched for static content                include the    args  part so non-default permalinks doesn t break when using query string             try files  uri  uri   index php  args              location     php                try files  uri  index php              include fastcgi params              fastcgi pass php              fastcgi read timeout 3000               fastcgi cache bypass  skip cache              fastcgi no cache  skip cache               fastcgi cache WORDPRESS              fastcgi cache valid  60m             location    purge                fastcgi cache purge WORDPRESS   scheme request method host 1

User · Answer

If nginx is running as a proxy   reverse proxy  that is  for users of ngx http proxy module   In addition to fastcgi  the proxy module also saves the request header in a temporary buffer   So you may need also to increase the proxy buffer size and the proxy buffers  or disable it totally  Please read the nginx documentation    Example of proxy buffering configuration  http     proxy buffer size   128k    proxy buffers   4 256k    proxy busy buffers size   256k      Example of disabling your proxy buffer  recommended for long polling servers   http     proxy buffering off      For more information  Nginx proxy module documentation

User · Answer

Add  fastcgi buffers 16 16k  fastcgi buffer size 32k  proxy buffer size   128k  proxy buffers   4 256k  proxy busy buffers size   256k   To server   in nginx conf Works for me

User · Answer

I am not sure that the issue is related to what header php is sending  Make sure that the buffering is enabled  The simple way is to create a proxy conf file   proxy redirect          off  proxy set header        Host             host  proxy set header        X-Real-IP        remote addr  proxy set header        X-Forwarded-For  proxy add x forwarded for  client max body size    100m  client body buffer size 128k  proxy connect timeout   90  proxy send timeout      90  proxy read timeout      90  proxy buffering         on  proxy buffer size       128k  proxy buffers           4 256k  proxy busy buffers size 256k    And a fascgi conf file   fastcgi param  SCRIPT FILENAME     document root fastcgi script name  fastcgi param  QUERY STRING        query string  fastcgi param  REQUEST METHOD      request method  fastcgi param  CONTENT TYPE        content type  fastcgi param  CONTENT LENGTH      content length  fastcgi param  SCRIPT NAME         fastcgi script name  fastcgi param  REQUEST URI         request uri  fastcgi param  DOCUMENT URI        document uri  fastcgi param  DOCUMENT ROOT       document root  fastcgi param  SERVER PROTOCOL     server protocol  fastcgi param  GATEWAY INTERFACE  CGI 1 1  fastcgi param  SERVER SOFTWARE    nginx  nginx version  fastcgi param  REMOTE ADDR         remote addr  fastcgi param  REMOTE PORT         remote port  fastcgi param  SERVER ADDR         server addr  fastcgi param  SERVER PORT         server port  fastcgi param  SERVER NAME         server name  fastcgi buffers 128 4096k  fastcgi buffer size 4096k  fastcgi index  index php  fastcgi param  REDIRECT STATUS    200    Next you need to call them in your default config server this way   http     include     etc nginx mime types    include     etc nginx proxy conf    include     etc nginx fastcgi conf    index    index html index htm index php    log format   main   remote addr -  remote user   time local    status          request   body bytes sent   http referer           http user agent    http x forwarded for       access log    logs access log  main    sendfile     on    tcp nopush   on

User · Answer

Add the following to your conf file  fastcgi buffers 16 16k   fastcgi buffer size 32k

User · Answer

If you re using Symfony framework  Before messing with Nginx config  try to disable ChromePHP first   1 - Open app config config dev yml  2 - Comment these lines    chromephp       type    chromephp      level   info   ChromePHP pack the debug info json-encoded in the X-ChromePhp-Data header  which is too big for the default config of nginx with fastcgi   Source  https   github com symfony symfony issues 8413 issuecomment-20412848

User · Answer

This is still the highest SO-question on Google when searching for this error  so let s bump it   When getting this error and not wanting to deep-dive into the NGINX settings immediately  you might want to check your outputs to the debug console  In my case I was outputting loads of text to the FirePHP   Chromelogger console  and since this is all sent as a header  it was causing the overflow   It might not be needed to change the webserver settings if this error is caused by just sending insane amounts of log messages

User · Answer

We ended up realising that our one server that was experiencing this had busted fpm config resulting in php errors warnings notices that d normally be logged to disk were being sent over the FCGI socket  It looks like there s a parsing bug when part of the header gets split across the buffer chunks   So setting php admin value error log  to something actually writeable and restarting php-fpm was enough to fix the problem   We could reproduce the problem with a smaller script    lt  php for   i   0   i lt   GET  iterations     i        error log str pad  a     GET  size     a     echo  got here n     Raising the buffers made the 502s harder to hit but not impossible  e g native   bash-4 1  for it in  30  200  3   do for size in  100  250  3   do echo  size  size iterations  it   curl -sv  http   localhost debug php size  size amp iterations  it  2 gt  amp 1   egrep    lt  HTTP     done  done   grep 502   head size 121 iterations 30  lt  HTTP 1 1 502 Bad Gateway size 109 iterations 33  lt  HTTP 1 1 502 Bad Gateway size 232 iterations 33  lt  HTTP 1 1 502 Bad Gateway size 241 iterations 48  lt  HTTP 1 1 502 Bad Gateway size 145 iterations 51  lt  HTTP 1 1 502 Bad Gateway size 226 iterations 51  lt  HTTP 1 1 502 Bad Gateway size 190 iterations 60  lt  HTTP 1 1 502 Bad Gateway size 115 iterations 63  lt  HTTP 1 1 502 Bad Gateway size 109 iterations 66  lt  HTTP 1 1 502 Bad Gateway size 163 iterations 69  lt  HTTP 1 1 502 Bad Gateway      there would be more here  but I piped through head        fastcgi buffers 16 16k  fastcgi buffer size 32k    bash-4 1  for it in  30  200  3   do for size in  100  250  3   do echo  size  size iterations  it   curl -sv  http   localhost debug php size  size amp iterations  it  2 gt  amp 1   egrep    lt  HTTP     done  done   grep 502   head size 223 iterations 69  lt  HTTP 1 1 502 Bad Gateway size 184 iterations 165  lt  HTTP 1 1 502 Bad Gateway size 151 iterations 198  lt  HTTP 1 1 502 Bad Gateway   So I believe the correct answer is  fix your fpm config so it logs errors to disk

User · Answer

upstream sent too big header while reading response header from upstream is nginx s generic way of saying  I don t like what I m seeing    Your upstream server thread crashed The upstream server sent an invalid header back The Notice Warnings sent back from STDERR overflowed their buffer and both it and STDOUT were closed   3  Look at the error logs above the message  is it streaming with logged lines preceding the message   PHP message  PHP Notice   Undefined index  Example snippet from a loop my log file   2015 11 23 10 30 02  error  32451 0   580927 FastCGI sent in stderr   PHP message  PHP Notice   Undefined index  Firstname in  srv www classes data convert php on line 1090 PHP message  PHP Notice   Undefined index  Lastname in  srv www classes data convert php on line 1090        20 lines of same PHP message  PHP Notice   Undefined index  Firstname in  srv www classes data convert php on line 1090 PHP message  PHP Notice   Undefined index  Lastname in  srv www classes data convert php on line 1090 PHP message  PHP Notice   Undef 2015 11 23 10 30 02  error  32451 0   580927 FastCGI sent in stderr   ta convert php on line 1090 PHP message  PHP Notice   Undefined index  Firstname   you can see in the 3rd line from the bottom that the buffer limit was hit  broke  and the next thread wrote in over it   Nginx then closed the connection and returned 502 to the client   2  log all the headers sent per request  review them and make sure they conform to standards  nginx does not permit anything older than 24 hours to delete expire a cookie  sending invalid content length because error messages were buffered before the content counted      getallheaders function call can usually help out in abstracted code situations php get all headers  examples include    lt  php   expire cookie setcookie    bookmark       strtotime  2012-01-01 00 00 00         nginx will refuse this header response  too far past to accept        gt    and this    lt  php header  Content-type  image jpg      gt    lt  php     a space was injected into the output above this line header  Content-length      filesize  image jpg      echo file get contents  image jpg       error  the response is now 1-byte longer than header     gt    1  verify  or make a script log  to ensure your thread is reaching the correct end point and not exiting before completion

User · Answer

Plesk instructions I combined the top two answers here In Plesk 12  I had nginx running as a reverse proxy  which I think is the default   So the current top answer doesn t work as nginx is also being run as a proxy  I went to Subscriptions    subscription domain    Websites  amp  Domains  tab     Virtual Host domain    Web Server Settings  Then at the bottom of that page you can set the Additional nginx directives which I set to be a combination of the top two answers here  fastcgi buffers         16  16k  fastcgi buffer size         32k  proxy buffer size          128k  proxy buffers            4 256k  proxy busy buffers size    256k

[php] upstream sent too big header while reading response header from upstream

If nginx is running as a proxy / reverse proxy

Example of proxy buffering configuration

Example of disabling your proxy buffer (recommended for long polling servers)

Examples related to php

Examples related to wordpress

Examples related to nginx

Examples related to fastcgi

Examples related to nginx-reverse-proxy