Full request/response body logging in nginx

nginx.conf

http {
  log_format bodylog '$remote_addr - $remote_user [$time_local] '
      '"$request" $status $body_bytes_sent '
      '"$http_referer" "$http_user_agent" $request_time '
      '<"$request_body" >"$resp_body"';

  lua_need_request_body on;

  set $resp_body "";
  body_filter_by_lua '
    local resp_body = string.sub(ngx.arg[1], 1, 1000)
    ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body
    if ngx.arg[2] then
      ngx.var.resp_body = ngx.ctx.buffered
    end
  ';

}
 
server {
  listen 1.2.3.4;
 
  location / {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://127.0.0.1:3000;
    access_log  /var/log/nginx/server.log bodylog;
  }
}

This will log full body of all requests and responses into nginx log. Depends on lua, so nginx needs to be built with lua module as well.

the request body is ok, but the reponse body is null ,

I can't get the response body in the access log but , can get it by
ngx.log(ngx.ERR, ngx.var.resp_body, ngx.var.request_body )

just wondering why you did local resp_body = string.sub(ngx.arg[1], 1, 1000) instead of local resp_body = string.sub(ngx.arg[1], 1, -1) or perhaps local resp_body = ngx.arg[1] for that matter?

The last two approaches makes sure that you do not lose the response body. Which happens in your case where it gets truncated.

@shoeb751 this is to avoid dumbing big binary files into the logs. I mostly use it to monitor API requests/responses, and 1KB covers all my need - it would make sense for you adjust it per your own requirements, of course.

I'm getting a bunch of escaped characters in using this, especially since my payloads tend to be json objects. For example " gets escaped to \x22. Any solution for that?

@aehernandez If you are using nginx > 1.11.8, you can set escape=json in the log_format directive. http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format

@morhekil I understand. Thanks

@morhekil
getting the exception: "set" directive is not allowed here in ~ /nginx/conf/nginx.conf
any solution?

@mallikharjunrao just move it on server { ... } block

I'm using nginx < 1.11.8 and escape=json is disable. Is there any other solution to decode json? Maybe something with lua

Looks like this isn't working anymore? Just gave it a try with a current distribution of OpenResty (it already has the lua module baked in) but even moving the set directive into the server block will just have the parser complain [emerg] unknown "resp_body" variable.

Update: I managed to get it to work after all. The problem seemed to originate from the missing nesting.
Once I moved the server block into the http block, it started to work.

I'm trying to get the response body but it always returns empty, I have tried both ways below:

  set $resp_body "";
#    body_filter_by_lua '
#                       local resp_body = string.sub(ngx.arg[1], 1, 1000)
#                       ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body
#                       if ngx.arg[2] then
#                               ngx.var.resp_body = ngx.ctx.buffered
#                       end
#                       ';
   body_filter_by_lua 'ngx.log(ngx.ERR, ngx.var.resp_body, ngx.var.request_body )';

Any ideas how to capture the response body?

@jimmyss04

A complete example and works as of 7/25/2018 on Ubuntu 16.04. Some APIs have changed, and the concept of contexts and phases is confusing working with nginx/openresty.

user www-data;
worker_processes auto;
pid /run/nginx.pid;

events {
    worker_connections 768;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    include /etc/nginx/mime.types;
    default_type application/octet-stream;
    gzip on;
    gzip_disable "msie6";

    log_format bodylog '$remote_addr - $remote_user [$time_local] '
    '"$request" $status $body_bytes_sent '
    '"$http_referer" "$http_user_agent" $request_time '
    '\n\n"$req_headers" \n"$req_body" \n>"$resp_body"';


    server {
        listen 80 default_server;
        access_log  /var/log/nginx/server.log bodylog;
     
        location / {
        }

        lua_need_request_body on;

        set $resp_body "";
        set $req_body "";
        set $req_headers "";

        client_body_buffer_size 16k;
        client_max_body_size 16k;

        rewrite_by_lua_block {
            local req_headers = "Headers: ";
            ngx.var.req_body = ngx.req.get_body_data();
            local h, err = ngx.req.get_headers()
            for k, v in pairs(h) do
                req_headers = req_headers .. k .. ": " .. v .. "\n";
            end

            ngx.var.req_headers = req_headers;
        }

        body_filter_by_lua '
        local resp_body = string.sub(ngx.arg[1], 1, 1000)
        ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body
        if ngx.arg[2] then
          ngx.var.resp_body = ngx.ctx.buffered
        end
        ';
     }
}

Does this require lua-nginx-module?

@simonecogno yes, it requires.

$req_headers

nginx: [emerg] unknown "req_headers" variable
the version of my Nginx on Ubuntu 16.04 is 1.10.3, installed with 'apt-get install nginx-extras'

Thanks Eugene, your working example helped me figure out what my issue was. Basically, I had a leftover content_by_lua_block { …} which was overwriting the response from the remote server

@morhekil I am running Red Hat Enterprise Linux release 8.8 (Ootpa)

`# nginx -t
nginx: [emerg] unknown directive "log_by_lua_block" in /etc/nginx/conf.d/microservice.conf:8
nginx: configuration file /etc/nginx/nginx.conf test failed

nginx -v

nginx version: nginx/1.24.0

nginx -V

nginx version: nginx/1.24.0
built by gcc 8.5.0 20210514 (Red Hat 8.5.0-4) (GCC)
built with OpenSSL 1.1.1k FIPS 25 Mar 2021
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'
#`

Hi,
First of all thanks for the code, it helped me a lot. Second, how would we tackle logging gzipped response bodies? I tried lua-zlib, but for some reason I can't make it work. Any suggestions? Thanks!