Support "none" keyword in log-format escape configuration

[knbnnate]

I would like the ingress-nginx ingress controller to support the native nginx functionality of disabling escaping characters in access logs. The specific requirement that is driving me here is that I'd like to log all of the request and response headers going through nginx in a json format. I've got set_by_lua_block and log_by_lua_block code to do that, and I've got my log-format-upstream set so that I can create the exact json log message that I want. The problem is, the request headers and response headers in the message are a string containing escaped json rather than json. Example:

Configmap entries:

  location-snippet: |
    set $request_headers '';
    set_by_lua_block $request_headers{
      local cjson = require("cjson")
      local headers = ngx.req.get_headers()
      return cjson.encode(headers)
    }
  log-format-escape-json: true
  log-format-upstream: '{"host":"$host","request":"$request","request_headers":"$request_headers"}'

Desired configmap entry instead of log-format-escape-json:

  log-format-escape-none: true

Generated nginx.conf line inside the ingress controller pod:

# kubectl -n ingress exec ingress-nginx-controller-597785cb88-xr7gr -- cat nginx.conf | grep log.format
        log_format upstreaminfo escape=json '{"host":"$host","request":"$request","request_headers":"$request_headers"}';

Desired nginx.conf line inside the ingress controller pod:

# kubectl -n ingress exec ingress-nginx-controller-597785cb88-xr7gr -- cat nginx.conf | grep log.format
        log_format upstreaminfo escape=none '{"host":"$host","request":"$request","request_headers":"$request_headers"}';

Request:

# curl -H "request-foo: bar" https://endpoint.local

Logs:

{"host":"endpoint.local","request":"GET / HTTP/2.0","request_headers":"{\"request-foo\": \"bar\"}"}

Desired logs:

{"host":"endpoint.local","request":"GET / HTTP/2.0","request_headers":{"request-foo": "bar"}}

Nginx supports the "none" keyword in the log-format escape option specifically for this use case. The ingress controller simply hasn't exposed the option.

There is currently another issue associated with this: #8105
This issue failed to get traction; I've taken a more incremental approach here that should be easier to swallow.

Implementation of the more incremental approach is at #8692

This does not require a particular kubernetes version.

The documentation should be updated to reflect that log-format-escape-none can be used to disable escaping entirely.

/kind documentation

[longwuyuan]

/kind feature
/triage accepted
/remove-kind documentation

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[knbnnate]

@longwuyuan is there a current estimate on how much longer the stability work will take priority? I'm guessing three more months or so based on the original assessment. Just checking in, I don't have access to the kubernetes slack so I can't really keep an eye on the conversation. Thanks!

[longwuyuan]

More details are available here https://youtu.be/UBt4N82ymOE and here https://kubernetes.slack.com/archives/CANQGM8BA/p1656020331133589.
The announcement in the dev mailing list is here https://groups.google.com/a/kubernetes.io/g/dev/c/rxtrKvT_Q8E

[knbnnate]

More details are available here https://youtu.be/UBt4N82ymOE and here https://kubernetes.slack.com/archives/CANQGM8BA/p1656020331133589. The announcement in the dev mailing list is here https://groups.google.com/a/kubernetes.io/g/dev/c/rxtrKvT_Q8E

Thanks!