Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[reopened] feat(request-id): introduce Request ID #11663

Merged
merged 6 commits into from
Sep 28, 2023
Merged

Conversation

dndx
Copy link
Member

@dndx dndx commented Sep 27, 2023

Summary

Introduce Request ID that:

  • uniquely identifies one client request
  • is populated in all error log lines that refer to a request
  • is populated in the access log and error templates
  • is included in the output of the PDK’s log serializer Along with Trace IDs, Correlation ID
  • is injected in a new X-Kong-Request-Id header for both upstream and downstream
  • is included in all traces via the new kong.request.id span attribute

Checklist

Full changelog

  • Introduce the immutable Request ID
  • Bump lua-kong-nginx-module to 0.7.1
  • Add request id to: error log, access log, (default) error templates, traces (span attribute), log serializer
  • New header X-Kong-Request-Id

Issue reference

KAG-2034
FTI-4837

kong/runloop/handler.lua Outdated Show resolved Hide resolved
kong/runloop/handler.lua Outdated Show resolved Hide resolved
kong/templates/nginx_kong.lua Outdated Show resolved Hide resolved
@samugi samugi force-pushed the feat/request-id-phase-1 branch from 3e5ac5d to 9d6115b Compare September 27, 2023 07:36
@samugi samugi requested review from ADD-SP and chobits September 27, 2023 07:38
kong/tracing/request_id.lua Outdated Show resolved Hide resolved
kong/tracing/request_id.lua Outdated Show resolved Hide resolved
kong/runloop/handler.lua Outdated Show resolved Hide resolved
@samugi samugi force-pushed the feat/request-id-phase-1 branch from 9d6115b to 11b0962 Compare September 27, 2023 11:26
@samugi samugi requested a review from ADD-SP September 27, 2023 11:27
@samugi samugi force-pushed the feat/request-id-phase-1 branch from 11b0962 to 2d3d0f1 Compare September 27, 2023 12:30
kong.conf.default Outdated Show resolved Hide resolved
kong/runloop/handler.lua Show resolved Hide resolved
kong/tracing/propagation.lua Outdated Show resolved Hide resolved
kong/tracing/request_id.lua Show resolved Hide resolved
@samugi samugi force-pushed the feat/request-id-phase-1 branch 2 times, most recently from 65405a4 to fa61dad Compare September 27, 2023 14:54
@samugi samugi requested a review from chobits September 27, 2023 14:56
@samugi samugi added this to the 3.5.0 milestone Sep 27, 2023
ADD-SP
ADD-SP previously requested changes Sep 28, 2023
kong/pdk/log.lua Outdated Show resolved Hide resolved
kong/tracing/instrumentation.lua Outdated Show resolved Hide resolved
* Add an immutable request ID
* Include request ID + trace and correlation IDs to the log serializer
* update Access log and Error log to append request id
* update the error templates to include the request id
* Bump lua-kong-nginx-module to version 0.7.1
  * Use the new directive `lua_kong_error_log_request_id`
    introduced in 0.7.0 which adds the request id to the error log output

Includes:

* unit tests for the new `request_id` module
* integration tests to check:
  * request id, correlation id, trace ids are added to log serializer
samugi added a commit that referenced this pull request Nov 9, 2023
* feat(request-id): add Request ID
* Add an immutable request ID
* Include request ID + trace and correlation IDs to the log serializer
* update Access log and Error log to append request id
* update the error templates to include the request id
* Bump lua-kong-nginx-module to version 0.7.1
  * Use the new directive `lua_kong_error_log_request_id`
    introduced in 0.7.0 which adds the request id to the error log output

Includes:

* unit tests for the new `request_id` module
* integration tests to check:
  * request id, correlation id, trace ids are added to log serializer

* feat(request-id): add request-id to error templates

* feat(request-id): request ID header + span attribute

* add the x-kong-request-id downstream header which contains the value
  of the request_id, and can be controlled via the `headers` config
  option
* add the x-kong-request-id upstream header which contains the value
  of the request_id, and can be controlled via the `headers_upstream`
  config option
* add the `kong.request.id` span attribute which contains the value of
  the request_id
* tests for all the above

* docs(conf): request ID

Co-authored-by: Enrique García Cota <[email protected]>

* feat(request-id): address PR feedback

* rephrase log messages
* remove unneeded conditional

* better changelog
* use upvalues to cache headers access
* use request id instead of kong_request_id (no longer needed as we
  don't need write access)
* cache locals in hot path
* improved performance of add_trace_id_formats function
* refactored docs in kong.conf.default

* perf: cache `request_id.get()` at the module level

KAG-2034
FTI-4837

---------

Co-authored-by: samugi <[email protected]>
Co-authored-by: Enrique García Cota <[email protected]>
Co-authored-by: Qi <[email protected]>
samugi added a commit that referenced this pull request Nov 15, 2023
* feat(request-id): add Request ID
* Add an immutable request ID
* Include request ID + trace and correlation IDs to the log serializer
* update Access log and Error log to append request id
* update the error templates to include the request id
* Bump lua-kong-nginx-module to version 0.7.1
  * Use the new directive `lua_kong_error_log_request_id`
    introduced in 0.7.0 which adds the request id to the error log output

Includes:

* unit tests for the new `request_id` module
* integration tests to check:
  * request id, correlation id, trace ids are added to log serializer

* feat(request-id): add request-id to error templates

* feat(request-id): request ID header + span attribute

* add the x-kong-request-id downstream header which contains the value
  of the request_id, and can be controlled via the `headers` config
  option
* add the x-kong-request-id upstream header which contains the value
  of the request_id, and can be controlled via the `headers_upstream`
  config option
* add the `kong.request.id` span attribute which contains the value of
  the request_id
* tests for all the above

* docs(conf): request ID

Co-authored-by: Enrique García Cota <[email protected]>

* feat(request-id): address PR feedback

* rephrase log messages
* remove unneeded conditional

* better changelog
* use upvalues to cache headers access
* use request id instead of kong_request_id (no longer needed as we
  don't need write access)
* cache locals in hot path
* improved performance of add_trace_id_formats function
* refactored docs in kong.conf.default

* perf: cache `request_id.get()` at the module level

KAG-2034
FTI-4837

---------

Co-authored-by: Enrique García Cota <[email protected]>
Co-authored-by: Qi <[email protected]>
samugi added a commit that referenced this pull request Nov 15, 2023
* feat(request-id): add Request ID
* Add an immutable request ID
* Include request ID + trace and correlation IDs to the log serializer
* update Access log and Error log to append request id
* update the error templates to include the request id
* Bump lua-kong-nginx-module to version 0.7.1
  * Use the new directive `lua_kong_error_log_request_id`
    introduced in 0.7.0 which adds the request id to the error log output

Includes:

* unit tests for the new `request_id` module
* integration tests to check:
  * request id, correlation id, trace ids are added to log serializer

* feat(request-id): add request-id to error templates

* feat(request-id): request ID header + span attribute

* add the x-kong-request-id downstream header which contains the value
  of the request_id, and can be controlled via the `headers` config
  option
* add the x-kong-request-id upstream header which contains the value
  of the request_id, and can be controlled via the `headers_upstream`
  config option
* add the `kong.request.id` span attribute which contains the value of
  the request_id
* tests for all the above

* docs(conf): request ID

Co-authored-by: Enrique García Cota <[email protected]>

* feat(request-id): address PR feedback

* rephrase log messages
* remove unneeded conditional

* better changelog
* use upvalues to cache headers access
* use request id instead of kong_request_id (no longer needed as we
  don't need write access)
* cache locals in hot path
* improved performance of add_trace_id_formats function
* refactored docs in kong.conf.default

* perf: cache `request_id.get()` at the module level

KAG-2034
FTI-4837

---------

Co-authored-by: samugi <[email protected]>
Co-authored-by: Enrique García Cota <[email protected]>
Co-authored-by: Qi <[email protected]>
samugi added a commit that referenced this pull request Nov 15, 2023
* feat(request-id): add Request ID
* Add an immutable request ID
* Include request ID + trace and correlation IDs to the log serializer
* update Access log and Error log to append request id
* update the error templates to include the request id
* Bump lua-kong-nginx-module to version 0.7.1
  * Use the new directive `lua_kong_error_log_request_id`
    introduced in 0.7.0 which adds the request id to the error log output

Includes:

* unit tests for the new `request_id` module
* integration tests to check:
  * request id, correlation id, trace ids are added to log serializer

* feat(request-id): add request-id to error templates

* feat(request-id): request ID header + span attribute

* add the x-kong-request-id downstream header which contains the value
  of the request_id, and can be controlled via the `headers` config
  option
* add the x-kong-request-id upstream header which contains the value
  of the request_id, and can be controlled via the `headers_upstream`
  config option
* add the `kong.request.id` span attribute which contains the value of
  the request_id
* tests for all the above

* docs(conf): request ID

Co-authored-by: Enrique García Cota <[email protected]>

* feat(request-id): address PR feedback

* rephrase log messages
* remove unneeded conditional

* better changelog
* use upvalues to cache headers access
* use request id instead of kong_request_id (no longer needed as we
  don't need write access)
* cache locals in hot path
* improved performance of add_trace_id_formats function
* refactored docs in kong.conf.default

* perf: cache `request_id.get()` at the module level

KAG-2034
FTI-4837

---------

Co-authored-by: Enrique García Cota <[email protected]>
Co-authored-by: Qi <[email protected]>
samugi added a commit that referenced this pull request Nov 20, 2023
* feat(request-id): add Request ID
* Add an immutable request ID
* Include request ID + trace and correlation IDs to the log serializer
* update Access log and Error log to append request id
* update the error templates to include the request id
* Bump lua-kong-nginx-module to version 0.7.1
  * Use the new directive `lua_kong_error_log_request_id`
    introduced in 0.7.0 which adds the request id to the error log output

Includes:

* unit tests for the new `request_id` module
* integration tests to check:
  * request id, correlation id, trace ids are added to log serializer

* feat(request-id): add request-id to error templates

* feat(request-id): request ID header + span attribute

* add the x-kong-request-id downstream header which contains the value
  of the request_id, and can be controlled via the `headers` config
  option
* add the x-kong-request-id upstream header which contains the value
  of the request_id, and can be controlled via the `headers_upstream`
  config option
* add the `kong.request.id` span attribute which contains the value of
  the request_id
* tests for all the above

* docs(conf): request ID

Co-authored-by: Enrique García Cota <[email protected]>

* feat(request-id): address PR feedback

* rephrase log messages
* remove unneeded conditional

* better changelog
* use upvalues to cache headers access
* use request id instead of kong_request_id (no longer needed as we
  don't need write access)
* cache locals in hot path
* improved performance of add_trace_id_formats function
* refactored docs in kong.conf.default

* perf: cache `request_id.get()` at the module level

KAG-2034
FTI-4837

---------

Co-authored-by: samugi <[email protected]>
Co-authored-by: Enrique García Cota <[email protected]>
Co-authored-by: Qi <[email protected]>
outsinre added a commit that referenced this pull request Nov 22, 2023
Kong now has a fixed access log format `kong_log_format`
that prevents customization and error on `kong start`.
Related to #11663.

If the `proxy_access_log` is not a valid pathname, then
replace `kong_log_format` with the custom value.
samugi pushed a commit that referenced this pull request Nov 28, 2023
* fix(core): respect custom proxy_access_log

Kong now has a fixed access log format `kong_log_format`
that prevents customization and error on `kong start`.
Related to #11663.

If the `proxy_access_log` is not a valid pathname, then
replace `kong_log_format` with the custom value.

* fix(config): cover log_format name with hyphen

* fix(config): early error when access log format is not defined

* fix(config): discard warning or return nil

* chore(config): style and comments

* chore(*): comments
team-gateway-bot pushed a commit that referenced this pull request Nov 28, 2023
* fix(core): respect custom proxy_access_log

Kong now has a fixed access log format `kong_log_format`
that prevents customization and error on `kong start`.
Related to #11663.

If the `proxy_access_log` is not a valid pathname, then
replace `kong_log_format` with the custom value.

* fix(config): cover log_format name with hyphen

* fix(config): early error when access log format is not defined

* fix(config): discard warning or return nil

* chore(config): style and comments

* chore(*): comments

(cherry picked from commit 9ec3494)
outsinre added a commit that referenced this pull request Nov 28, 2023
* fix(core): respect custom proxy_access_log

Kong now has a fixed access log format `kong_log_format`
that prevents customization and error on `kong start`.
Related to #11663.

If the `proxy_access_log` is not a valid pathname, then
replace `kong_log_format` with the custom value.

* fix(config): cover log_format name with hyphen

* fix(config): early error when access log format is not defined

* fix(config): discard warning or return nil

* chore(config): style and comments

* chore(*): comments
fffonion pushed a commit that referenced this pull request Nov 29, 2023
* fix(core): respect custom proxy_access_log

Kong now has a fixed access log format `kong_log_format`
that prevents customization and error on `kong start`.
Related to #11663.

If the `proxy_access_log` is not a valid pathname, then
replace `kong_log_format` with the custom value.

* fix(config): cover log_format name with hyphen

* fix(config): early error when access log format is not defined

* fix(config): discard warning or return nil

* chore(config): style and comments

* chore(*): comments

(cherry picked from commit 9ec3494)
fffonion pushed a commit that referenced this pull request Nov 29, 2023
* fix(core): respect custom proxy_access_log

Kong now has a fixed access log format `kong_log_format`
that prevents customization and error on `kong start`.
Related to #11663.

If the `proxy_access_log` is not a valid pathname, then
replace `kong_log_format` with the custom value.

* fix(config): cover log_format name with hyphen

* fix(config): early error when access log format is not defined

* fix(config): discard warning or return nil

* chore(config): style and comments

* chore(*): comments
AndyZhang0707 added a commit that referenced this pull request Jul 18, 2024
AndyZhang0707 added a commit that referenced this pull request Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants