Logging POST data from request body

Question

I have my config setup to handle a bunch of GET requests which render pixels that work fine to handle analytics and parse query strings for logging  With an additional third party data stream  I need to handle a POST request to a given url that has JSON in an expected loggable format inside of it s request body  I don t want to use a secondary server with proxy pass and just want to log the whole response into an associated log file like what it does with GET requests  A snippet of some code that I m using looks like the following   GET request  which works great    location     rl gif     set  rl lcid  arg lcid    if   http cookie     lcid     S            set  rl lcid  cookie lcid        empty gif    log format my tracking     guid      rl lcid    data      arg  rlcdnsegs        access log   mnt logs nginx my access log my tracking    rewrite        http   my url id  cookie lcid  redirect      Here is kinda what I am trying to do  POST request  which does not work    location  bk     log format bk tracking  request body    access log   mnt logs nginx bk access log bk tracking      Curling curl http   myurl bk -d name example gives me a 404 page not found   Then I tried   location  bk gif     empty gif    log format bk tracking  request body    access log   mnt logs nginx bk access log bk tracking      Curling curl http   myurl bk gif -d name example gives me a 405 Not Allowed   My current version is nginx 0 7 62  Any help in the right direction is very much appreciated  Thanks   UPDATE So now my post looks like this   location  bk     if   request method    POST        return 405        proxy pass  scheme   127 0 0 1  server port dummy    log format my tracking  request body    access log   mnt logs nginx my access log my tracking    location  dummy   set  test 0      It is logging the post data correctly  but returns a 404 on the requesters end  If I change the above code to return a 200 like so   location  bk     if   request method    POST        return 405        proxy pass  scheme   127 0 0 1  server port dummy    log format my tracking  request body    access log   mnt logs nginx my access log my tracking    return 200    location  dummy   set  test 0      Then it return the 200 correctly  but no longer records the post data   ANOTHER UPDATE Kinda found a working solution  Hopefully this can help other on their way

User · Answer

The solution below was the best format I found.

log_format postdata escape=json '$remote_addr - $remote_user [$time_local] '
                       '"$request" $status $bytes_sent '
                       '"$http_referer" "$http_user_agent" "$request_body"';
server {
        listen 80;

        server_name api.some.com;

        location / {
         access_log  /var/log/nginx/postdata.log  postdata;
         proxy_pass      http://127.0.0.1:8080;
        }

}

For this input

curl -d '{"key1":"value1", "key2":"value2"}' -H "Content-Type: application/json" -X POST http://api.deprod.com/postEndpoint

Generate that great result

201.23.89.149 -  [22/Aug/2019:15:58:40 +0000] "POST /postEndpoint HTTP/1.1" 200 265 "" "curl/7.64.0" "{\"key1\":\"value1\", \"key2\":\"value2\"}"

User · Answer

Try echo read request body    echo read request body     Explicitly reads request body so that the  request body variable will always have non-empty values  unless the body is so big that it has been saved by Nginx to a local temporary file     location  log     log format postdata  request body    access log  mnt logs nginx my tracking access log postdata    echo read request body

User · Answer

Ok  So finally I was able to log the post data and return a 200  It s kind of a hacky solution that I m not too proud of which basically overrides the natural behavior for error page  but my inexperience of nginx plus timelines lead me to this solution   location  bk     if   request method    POST        return 405        proxy set header  X-Forwarded-For  proxy add x forwarded for    proxy set header Host  host    proxy set header X-Real-IP  remote addr    proxy redirect off    proxy pass  scheme   127 0 0 1  server port success    log format my tracking  request body    access log   mnt logs nginx my tracking access log my tracking    location  success     return 200    error page   500 502 503 504   50x html  location    50x html     root    var www nginx-default    log format my tracking  request body    access log   mnt logs nginx my tracking access log my tracking 2      Now according to that config  it would seem that the proxy pass would return a 200 all the time  Occasionally I would get 500 but when I threw in an error log to see what was going on  all of my request body data was in there and I couldn t see a problem  So I caught that and wrote to the same log  Since nginx doesn t like the same name for the tracking variable  I just used my tracking 2 and wrote to the same log as when it returns a 200  Definitely not the most elegant solution and I welcome any better solution  I ve seen the post module  but in my scenario  I couldn t recompile from source

User · Answer

I had a similar problem   GET requests worked and their  empty  request bodies got written to the the log file   POST requests failed with a 404   Experimenting a bit  I found that all POST requests were failing   I found a forum posting asking about POST requests and the solution there worked for me   That solution   Add a proxy header line right before the proxy pass line  exactly like the one in the example below   server       listen       192 168 0 1 45080      server name  foo example org       access log   path to log nginx post bodies log post bodies      location               add the following proxy header line to get POSTs to work       proxy set header Host  http host        proxy pass   http   10 1 2 3             This is with nginx 1 2 1 for what it is worth

User · Answer

FWIW  this config worked for me   location    logpush html     if   request method   POST        access log  var log nginx push log push requests      proxy pass  scheme   127 0 0 1 logsink      break           return 200  scheme    host serviceup html         location  logsink     return 200

User · Answer

This solution works like a charm  updated in 2017 to honor that log format needs to be in the http part of the nginx config    log format postdata  request body   server                    location    post php          access log   var log nginx postdata log  postdata         fastcgi pass php cgi            I think the trick is making nginx believe that you will call a cgi script

User · Answer

nginx log format taken from here  http   nginx org en docs http ngx http log module html  no need to install anything extra  worked for me for GET and POST requests   upstream my upstream      server upstream ip upstream port     location         log format postdata   remote addr -  remote user   time local                              request   status  bytes sent                             http referer    http user agent    request body        access log  path to nginx access log postdata      proxy set header Host  http host      proxy pass http   my upstream            just change upstream ip and upstream port

[logging] Logging POST data from $request_body

Examples related to logging

Examples related to nginx

Examples related to http-post