[core.logging] Uses host timezone as default #90368
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TinaHeiligers
added
Team:Core
TinaHeiligers
changed the title
TinaHeiligers
changed the title
TinaHeiligers
commented
Feb 5, 2021
docs/setup/settings.asciidoc
Outdated
| @@ -306,10 +306,10 @@ the `polling` method could be used enabling that option. *Default: `false`* | |||
| suppress all logging output. *Default: `false`* | |||
|
|
|||
| | `logging.timezone` | |||
| | Set to the canonical time zone ID | |||
| | Optional. Set to the canonical time zone ID | |||
There was a problem hiding this comment.
The legacy logging timezone default was changed from UTC to the host timezone in #22696. It appears that the documentation wasn't updated to reflect the change. As it stands right now, logging.timezone is actually optional (internally falling back to the local timezone of the host) and we needed to update the docs to reflect that.
There was a problem hiding this comment.
As it stands right now, logging.timezone is actually optional (internally falling back to the local timezone of the host)
Are we certain that this is the current behavior?
I see that the changes made in #22696 are no longer present in the legacy logging config schema: https://github.com/elastic/kibana/blob/master/packages/kbn-legacy-logging/src/schema.ts#L61
It seems to me that the current setup docs on master are accurate based on how the system works, but that's a little confusing because this update to the migration doc which was added in #22696 is no longer correct: https://github.com/elastic/kibana/blob/master/docs/migration/migrate_8_0.asciidoc
There was a problem hiding this comment.
@lukeelmers The global logging.timezone is only applied to the legacy logging system. In the KP, this setting is ignored. This PR doesn't make any changes to the legacy logging behavior, it changes the default timezone used in the KP logging system to match the behavior of the legacy logger.
There was a problem hiding this comment.
Update: I removed Optional and replaced *Default: 'UTC'* with "When not set, log events use the host timezone"
TinaHeiligers
commented
Feb 5, 2021
| @@ -87,7 +87,7 @@ describe('logging service', () => { | |||
| const loggedString = getPlatformLogsFromMock(mockConsoleLog); | |||
| expect(loggedString).toMatchInlineSnapshot(` | |||
| Array [ | |||
| "[xxxx-xx-xxTxx:xx:xx.xxxZ][INFO ][test-file] handled by NP", | |||
| "[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][INFO ][test-file] handled by NP", | |||
There was a problem hiding this comment.
Default format changed from ISO8601 (UTC) to ISO8601_TZ (retains timezone offset) in the Kibana Platform.
This legacy logging test had to be updated for the setup, where we have both the legacy logging and KP logging systems configured.
lukeelmers
reviewed
Feb 5, 2021
There was a problem hiding this comment.
Overall makes sense, added a few notes and questions.
A few other thoughts:
- Should we still consider adding a core deprecation notice for
logging.timezonein this PR, as that setting shouldn't be needed in 8.0 any more since it can be configured with the pattern layout? - Should we be updating
JsonLayoutas well? Currently it's always using UTC. Not sure if we would want to make this configurable or not
docs/setup/settings.asciidoc
Outdated
| @@ -306,10 +306,10 @@ the `polling` method could be used enabling that option. *Default: `false`* | |||
| suppress all logging output. *Default: `false`* | |||
|
|
|||
| | `logging.timezone` | |||
| | Set to the canonical time zone ID | |||
| | Optional. Set to the canonical time zone ID | |||
There was a problem hiding this comment.
As it stands right now, logging.timezone is actually optional (internally falling back to the local timezone of the host)
Are we certain that this is the current behavior?
I see that the changes made in #22696 are no longer present in the legacy logging config schema: https://github.com/elastic/kibana/blob/master/packages/kbn-legacy-logging/src/schema.ts#L61
It seems to me that the current setup docs on master are accurate based on how the system works, but that's a little confusing because this update to the migration doc which was added in #22696 is no longer correct: https://github.com/elastic/kibana/blob/master/docs/migration/migrate_8_0.asciidoc
TinaHeiligers
commented
Feb 5, 2021
docs/setup/settings.asciidoc
Outdated
| @@ -306,10 +306,10 @@ the `polling` method could be used enabling that option. *Default: `false`* | |||
| suppress all logging output. *Default: `false`* | |||
|
|
|||
| | `logging.timezone` | |||
| | Set to the canonical time zone ID | |||
| | Optional. Set to the canonical time zone ID | |||
There was a problem hiding this comment.
@lukeelmers The global logging.timezone is only applied to the legacy logging system. In the KP, this setting is ignored. This PR doesn't make any changes to the legacy logging behavior, it changes the default timezone used in the KP logging system to match the behavior of the legacy logger.
@lukeelmers I added one and tried to give enough detail in the deprecation notice. My only concern is not being able to set the timezone on a more global log level. Each context that doesn't have a specified appender will inherit from the root logger, which has a default pattern of The root pattern isn't exposed and can't be modified. I guess one could define a custom appender and add that to the root logger, but we will end up with duplicate logs. |
@lukeelmers In the For example: With the following logging configuration: We get We could consider adding an optional modifier to the json layout for the timestamp field but at the moment, no conversions are supported. log4j also doesn't support as many modifiers to a If we do feel that supporting field modifiers for |
|
@elasticmachine merge upstream |
|
Pinging @elastic/kibana-core (Team:Core) |
TinaHeiligers
commented
Feb 8, 2021
lukeelmers
approved these changes
Feb 9, 2021
There was a problem hiding this comment.
In the json layout, @timestamp is formatted as YYYY-MM-DDTHH:mm:ss.SSSZ by default and is equivalent to the pattern layout for ISO8601_TZ.
Thanks for highlighting this - I had read the code too quickly and missed the Z 😉
If we do feel that supporting field modifiers for json layout is needed and part of the larger logging effort, I think we need a new issue for the enhancement.
++ Yeah once this PR lands, this is the only case I can find related to timezones where we don't have feature parity with legacy platform, so I agree we should add it to the list for posterity, even if we don't address it soon. If I run Kibana like this:
node scripts/kibana serve --logging.dest=./kibana.log --logging.json=true --logging.timezone=EST
Then my json logs will indeed have their timezones changed, otherwise they continue to default to the host TZ. But there isn't currently a way to do this with json layout; it's always just the host TZ.
Tested and seems to be working well! I did another pass on the code too... My only nit would be to also update the migration docs which mention logging.timezone. Since we are deprecating that setting we should probably remove mention of the setting there and instead explain how to do this with pattern layout (or link to our README or whatever).
TinaHeiligers
added
the
auto-backport
TinaHeiligers
changed the title
💛 Build succeeded, but was flaky
Test FailuresKibana Pipeline / general / Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/transform/feature_controls/transform_security·ts.transform feature controls security global all privileges (aka kibana_admin) should not render the "Stack" sectionStandard OutStack TraceMetrics [docs]
History
To update your PR or re-run it, just comment with: |
|
Backport result |
TinaHeiligers
added a commit
that referenced
this pull request
Feb 10, 2021
Co-authored-by: Kibana Machine <[email protected]> # Conflicts: # docs/migration/migrate_8_0.asciidoc # src/core/server/config/deprecation/core_deprecations.ts
gmmorris
added a commit
to gmmorris/kibana
that referenced
this pull request
Feb 10, 2021
* master: (99 commits) [Fleet] Use Fleet Server indices in the search bar (elastic#90835) [Search Sessions] added an info flyout to session management (elastic#90559) [ILM] Revisit searchable snapshot field after new redesign (elastic#90793) [Alerting] License Errors on Alert List View (elastic#89920) RFC Improve saved object migrations algorithm (elastic#84333) [Lens] (Accessibility) Fix focus on drag and drop actions (elastic#90561) Use new shortcut links to Fleet discuss forums. (elastic#90786) Do not generate an ephemeral encryption key in production. (elastic#81511) [Fleet] Use staging registry for snapshot builds (elastic#90327) Actually deleting x-pack/tsconfig.refs.json (elastic#90898) Add deprecation warning to all Beats CM pages. (elastic#90741) skip flaky suite (elastic#90136) Revert "Revert "[Metrics UI] Add Metrics Anomaly Alert Type (elastic#89244)"" (elastic#90889) remove ref to removed tsconfig file [core.logging] Uses host timezone as default (elastic#90368) [Maps] remove maps_file_upload plugin and fold public folder into file_upload plugin (elastic#90292) Revert "[Metrics UI] Add Metrics Anomaly Alert Type (elastic#89244)" [dev-utils/ci-stats] support disabling ship errors (elastic#90851) Prefix with / (elastic#90836) [Metrics UI] Add Metrics Anomaly Alert Type (elastic#89244) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Elasticsearch uses the host timezone as a logging default, whereas logging from the Kibana platform uses UTC as the default.
This PR aligns Kibana's logging settings in the Kibana platform with Elasticsearch, defaulting to the host timezone when the timezone for an appender is not explicitly set.
Addresses #89689
Checklist
Delete any items that are not applicable to this PR.
For maintainers