Kong Docker header_filter_by_lua_block custom configuration

[rudijs]

Hi,

I'm using the offical Kong docker image and trying to remove the Via and X-Kong* headers.

I've gained some insight from this closed issue: #1315

I'm using a custom configuration with the method described in the docs: https://getkong.org/docs/0.9.x/configuration/#custom-nginx-configuration

The custom configuration is working well, I have custom log format and logging to stdout.

I think I'm adding the header_filter_by_lua_block block in the wrong place as it's having no effect.

I've tried a few different places for the header_filter_by_lua_block block without joy.

Can you see from my configuration below where I'm going wrong?

Any tips or suggestions here very much appreciated.

Thanks!

# ---------------------
# custom_nginx.template
# ---------------------

worker_processes ${{NGINX_WORKER_PROCESSES}}; # can be set by kong.conf
daemon ${{NGINX_DAEMON}};                     # can be set by kong.conf

pid pids/nginx.pid;                      # this setting is mandatory
# error_log logs/error.log ${{LOG_LEVEL}}; # can be set by kong.conf
# Log errors to stdout for Openshift centralized logging
error_log /proc/1/fd/1 ${{LOG_LEVEL}}; # can be set by kong.conf

events {
  use epoll; # custom setting
  multi_accept on;
}

http {
  # Add additional response headers
  header_filter_by_lua_block {
      kong.header_filter()
      ngx.header["Server"] = nil
      ngx.header["Via"] = nil
      ngx.header["X-Kong-Proxy-Latency"] = nil
      ngx.header["X-Kong-Upstream-Latency"] = nil
  }

  # Custom JSON log format
  log_format logstash_json '{ "@timestamp": "$time_iso8601", '
                             '"@fields": { '
                               '"remote_addr": "$remote_addr", '
                               '"remote_user": "$remote_user", '
                               '"body_bytes_sent": "$body_bytes_sent", '
                               '"request_time": "$request_time", '
                               '"status": "$status", '
                               '"request": "$request", '
                               '"request_method": "$request_method", '
                               '"http_referrer": "$http_referer", '
                               '"http_user_agent": "$http_user_agent" } }';  

  # Log errors to stdout for Openshift centralized logging
  access_log /proc/1/fd/1 logstash_json;

  # include default Kong Nginx config
  include 'nginx-kong.conf';
}

Additional Details & Logs

• Kong version 0.9.5

[Tieske]

The issue you refer to is based on an older Kong version using a YAML configuration format.

What they did was customizing the Kong handler. But you placed it in the generic nginx config file template (the new format uses two files).

At the bottom of the file there is;

  # include default Kong Nginx config
  include 'nginx-kong.conf';
}

The Kong configuration is in that file nginx-kong.conf, and there you need to update the existing header_filer_by_lua_block.

see also https://getkong.org/docs/0.9.x/configuration/#custom-nginx-configuration-amp-embedding-kong

[rudijs]

Ah OK, I see - thanks.

So I'll have to inline the nginx-kong.conf and not include it - so that I can add to the header_filer_by_lua_block

I can make that work no worries, I'll have to alter my docker build process but that's fine.

I'll have a shot at it and post my results here.

Cheers.

[rudijs]

@Tieske That works all good - thanks!

What I did was start the Kong docker image, then from the container made a copy of nginx-kong.conf and use it as a template in a custom docker build.

The complete configuration looks like:

# ---------------------
# custom_nginx.template
# ---------------------

worker_processes ${{NGINX_WORKER_PROCESSES}}; # can be set by kong.conf
daemon ${{NGINX_DAEMON}};                     # can be set by kong.conf

pid pids/nginx.pid;                      # this setting is mandatory
# error_log logs/error.log ${{LOG_LEVEL}}; # can be set by kong.conf
# Log errors to stdout for Openshift centralized logging
error_log /proc/1/fd/1 ${{LOG_LEVEL}}; # can be set by kong.conf

events {
  use epoll; # custom setting
  multi_accept on;
}

http {

  # Custom JSON log format
  log_format logstash_json '{ "@timestamp": "$time_iso8601", '
                             '"@fields": { '
                               '"remote_addr": "$remote_addr", '
                               '"remote_user": "$remote_user", '
                               '"body_bytes_sent": "$body_bytes_sent", '
                               '"request_time": "$request_time", '
                               '"status": "$status", '
                               '"request": "$request", '
                               '"request_method": "$request_method", '
                               '"http_referrer": "$http_referer", '
                               '"http_user_agent": "$http_user_agent" } }';  

  # Log errors to stdout for Openshift centralized logging
  access_log /proc/1/fd/1 logstash_json;

  # include default Kong Nginx config
  #include 'nginx-kong.conf';
  # include default Kong Nginx config with headers removed in header_filter_by_lua_block
  include '/opt/kong/kong_nginx.template';
}

and the /opt/kong/kong_nginx.template is:

resolver 127.0.0.1:8053 ipv6=off;
charset UTF-8;

error_log logs/error.log notice;
access_log logs/access.log;

error_log syslog:server=kong-hf.mashape.com:61828 error;


client_max_body_size 0;
proxy_ssl_server_name on;
underscores_in_headers on;

real_ip_header X-Forwarded-For;
set_real_ip_from 0.0.0.0/0;
real_ip_recursive on;

lua_package_path '?/init.lua;./kong/?.lua;;';
lua_package_cpath ';;';
lua_code_cache on;
lua_max_running_timers 4096;
lua_max_pending_timers 16384;
lua_shared_dict kong 4m;
lua_shared_dict cache 128m;
lua_shared_dict cache_locks 100k;
lua_shared_dict cassandra 1m;
lua_shared_dict cassandra_prepared 5m;
lua_socket_log_errors off;

init_by_lua_block {
    require 'resty.core'
    kong = require 'kong'
    kong.init()
}

init_worker_by_lua_block {
    kong.init_worker()
}

server {
    server_name kong;
    listen 0.0.0.0:8000;
    error_page 404 408 411 412 413 414 417 /kong_error_handler;
    error_page 500 502 503 504 /kong_error_handler;

    listen 0.0.0.0:8443 ssl;
    ssl_certificate /usr/local/kong/ssl/kong-default.crt;
    ssl_certificate_key /usr/local/kong/ssl/kong-default.key;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_certificate_by_lua_block {
        kong.ssl_certificate()
    }

    location / {
        set $upstream_host nil;
        set $upstream_url nil;

        access_by_lua_block {
            kong.access()
        }

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $upstream_host;
        proxy_pass_header Server;
        proxy_pass $upstream_url;

        header_filter_by_lua_block {
            kong.header_filter()
            ngx.header["Server"] = nil
            ngx.header["Via"] = nil
            ngx.header["X-Kong-Proxy-Latency"] = nil
            ngx.header["X-Kong-Upstream-Latency"] = nil
        }

        body_filter_by_lua_block {
            kong.body_filter()
        }

        log_by_lua_block {
            kong.log()
        }
    }

    location = /kong_error_handler {
        internal;
        content_by_lua_block {
            require('kong.core.error_handlers')(ngx)
        }
    }
}

server {
    server_name kong_admin;
    listen 0.0.0.0:8001;

    client_max_body_size 10m;
    client_body_buffer_size 10m;

    location / {
        default_type application/json;
        content_by_lua_block {
            ngx.header['Access-Control-Allow-Origin'] = '*'
            if ngx.req.get_method() == 'OPTIONS' then
                ngx.header['Access-Control-Allow-Methods'] = 'GET,HEAD,PUT,PATCH,POST,DELETE'
                ngx.header['Access-Control-Allow-Headers'] = 'Content-Type'
                ngx.exit(204)
            end

            require('lapis').serve('kong.api')
        }
    }

    location /nginx_status {
        internal;
        access_log off;
        stub_status;
    }

    location /robots.txt {
        return 200 'User-agent: *\nDisallow: /';
    }
}

I'll just have to be aware moving forward with newer versions of Kong to keep the nginx-kong.conf file up to date.

[Tieske]

glad it works. Maybe @thibaultcha knows an easier way of achieving this?

[thibaultcha]

@rudijs You can inline the Kong part of the Nginx configuration inside the main part (the template, not the "compiled" version), and use that whole Nginx config file when starting Kong: kong start -v kong.conf --nginx-conf nginx.conf.template. That will allow you to only rely on kong.conf for your entire configuration, as well as track the future Kong's nginx config changes more easily (since it will be untouched, but rather just inlined in your main Nginx config.

That is currently the only way (still more flexible than previous versions of Kong) although we could improve this system.