Extended docs with logging customisation at runtime #3968

ElenaKhaustova · 2024-06-26T14:23:47Z

Description

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

Read the contributing guidelines
Signed off each commit with a Developer Certificate of Origin (DCO)
Opened this PR as a 'Draft Pull Request' if it is work-in-progress
Updated the documentation to reflect the code changes
Added a description of this change in the RELEASE.md file
Added tests to cover my changes
Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

Signed-off-by: Elena Khaustova <[email protected]>

datajoely

Some stylistic soft suggestions - I also wonder if it's worth listing out the other other class paths we may want to tell the user they can quieten 🤔

datajoely · 2024-06-26T14:48:15Z

docs/source/logging/index.md

@@ -42,6 +42,17 @@ After setting the environment variable, any subsequent Kedro commands use the lo
 If the `KEDRO_LOGGING_CONFIG` environment variable is not set, Kedro will use the [default logging configuration](https://github.com/kedro-org/kedro/blob/main/kedro/framework/project/default_logging.yml).
 ```

+One can also customise logging at the runtime and redefine the logging configuration provided in the `logging.yml`.


Suggested change

One can also customise logging at the runtime and redefine the logging configuration provided in the `logging.yml`.

### Change the verbosity of specific parts of Kedro

One can also customise logging at the runtime and redefine the logging configuration provided in the `logging.yml`.

"You can" instead of "One can", maybe? Most of the docs are in second person I think

datajoely · 2024-06-26T14:49:10Z

docs/source/logging/index.md

@@ -42,6 +42,17 @@ After setting the environment variable, any subsequent Kedro commands use the lo
 If the `KEDRO_LOGGING_CONFIG` environment variable is not set, Kedro will use the [default logging configuration](https://github.com/kedro-org/kedro/blob/main/kedro/framework/project/default_logging.yml).
 ```

+One can also customise logging at the runtime and redefine the logging configuration provided in the `logging.yml`.
+In the example below we change a logging level to `WARNING` for the `kedro.io.data_catalog` component, so the setting is only applied to the `kedro.io.data_catalog` logger and for the rest of the components the default settings provided via `logging.yml` are remained unchanged.


Suggested change

In the example below we change a logging level to `WARNING` for the `kedro.io.data_catalog` component, so the setting is only applied to the `kedro.io.data_catalog` logger and for the rest of the components the default settings provided via `logging.yml` are remained unchanged.

In the example below we change a logging level to `WARNING` for the `kedro.io.data_catalog` component logger specifically, the logging for the rest of the components will remain unchanged.

astrojuanlu

Apart from the vale suggestions and @datajoely's comment, LGTM!

Signed-off-by: Elena Khaustova <[email protected]>

ElenaKhaustova · 2024-06-26T15:44:30Z

Some stylistic soft suggestions - I also wonder if it's worth listing out the other other class paths we may want to tell the user they can quieten 🤔

I have applied the suggestions, thank you!

As for the other class paths, do you have any in mind that are worth adding or about which we have complaints? Overall, adding another path should be straightforward now even with this example.

ankatiyar · 2024-06-26T16:23:39Z

docs/source/logging/index.md

@@ -42,6 +42,17 @@ After setting the environment variable, any subsequent Kedro commands use the lo
 If the `KEDRO_LOGGING_CONFIG` environment variable is not set, Kedro will use the [default logging configuration](https://github.com/kedro-org/kedro/blob/main/kedro/framework/project/default_logging.yml).
 ```

+One can also customise logging at the runtime and redefine the logging configuration provided in the `logging.yml`.


"You can" instead of "One can", maybe? Most of the docs are in second person I think

ankatiyar · 2024-06-26T16:25:31Z

docs/source/logging/index.md

+In the example below we change a logging level to `WARNING` for the `kedro.io.data_catalog` component logger specifically, the logging for the rest of the components will remain unchanged.
+The same can be done for higher/lower-level components without affecting the top-level.
+
+```python


This would go in the settings.py, right? Worth mentioning where to put this code.

That's a good catch, I've added a clarification that this is relevant for the notebook environment. We cannot make it in settings.py since logging configuration from logging.yml is applied after settings.py configuration

kedro/kedro/framework/project/__init__.py

Line 273 in 413dbca

def configure_project(package_name: str) -> None:

, so it will be redefined by what is provided in logging.yml

So, the alternative is to customise logging.yml described in the neighbouring section.

Ah I see, makes sense!

Signed-off-by: Elena Khaustova <[email protected]>

merelcht

Some small suggestions, but otherwise looks good to me! 👍 Great way to resolve the original issue.

merelcht · 2024-06-27T10:11:30Z

docs/source/logging/index.md

@@ -42,6 +42,21 @@ After setting the environment variable, any subsequent Kedro commands use the lo
 If the `KEDRO_LOGGING_CONFIG` environment variable is not set, Kedro will use the [default logging configuration](https://github.com/kedro-org/kedro/blob/main/kedro/framework/project/default_logging.yml).
 ```

+### Change the verbosity of specific parts of Kedro
+
+You can also customise logging at the runtime and redefine the logging configuration provided in the `logging.yml` when using jupyter notebook.


Suggested change

You can also customise logging at the runtime and redefine the logging configuration provided in the `logging.yml` when using jupyter notebook.

You can also customise logging at runtime and redefine the logging configuration provided in the `logging.yml` when using jupyter notebook.

merelcht · 2024-06-27T10:12:31Z

docs/source/logging/index.md

+### Change the verbosity of specific parts of Kedro
+
+You can also customise logging at the runtime and redefine the logging configuration provided in the `logging.yml` when using jupyter notebook.
+In the example below we change a logging level to `WARNING` for the `kedro.io.data_catalog` component logger specifically, the logging for the rest of the components will remain unchanged.


Suggested change

In the example below we change a logging level to `WARNING` for the `kedro.io.data_catalog` component logger specifically, the logging for the rest of the components will remain unchanged.

The example below demonstrates how you can change the logging level from `INFO` to `WARNING` for the `kedro.io.data_catalog` component logger specifically, the logging for the rest of the components will remain unchanged.

Signed-off-by: Elena Khaustova <[email protected]>

ElenaKhaustova added 2 commits June 26, 2024 15:21

Added a note on logging customisation at runtime

ad4d3a3

Signed-off-by: Elena Khaustova <[email protected]>

Updated RELEASE.md

88b137d

Signed-off-by: Elena Khaustova <[email protected]>

ElenaKhaustova marked this pull request as ready for review June 26, 2024 14:39

ElenaKhaustova requested review from yetudada, astrojuanlu and merelcht as code owners June 26, 2024 14:39

ElenaKhaustova requested review from datajoely and ankatiyar June 26, 2024 14:39

datajoely reviewed Jun 26, 2024

View reviewed changes

astrojuanlu approved these changes Jun 26, 2024

View reviewed changes

Applied siggesions

141c278

Signed-off-by: Elena Khaustova <[email protected]>

ankatiyar approved these changes Jun 26, 2024

View reviewed changes

ankatiyar reviewed Jun 26, 2024

View reviewed changes

Applied suggestions

b703d39

Signed-off-by: Elena Khaustova <[email protected]>

merelcht approved these changes Jun 27, 2024

View reviewed changes

Applied suggested changes

cc1b8ea

Signed-off-by: Elena Khaustova <[email protected]>

datajoely approved these changes Jun 27, 2024

View reviewed changes

ElenaKhaustova merged commit a683cd3 into main Jun 27, 2024
10 checks passed

ElenaKhaustova deleted the docs/3918-disable-logging branch June 27, 2024 13:21

merelcht linked an issue Jun 28, 2024 that may be closed by this pull request

[DataCatalog]: Disable catalog.load() logging #3918

Closed

merelcht mentioned this pull request Jun 28, 2024

[DataCatalog]: Disable catalog.load() logging #3918

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extended docs with logging customisation at runtime #3968

Extended docs with logging customisation at runtime #3968

ElenaKhaustova commented Jun 26, 2024 •

edited

Loading

datajoely left a comment

datajoely Jun 26, 2024

ankatiyar Jun 26, 2024

datajoely Jun 26, 2024

astrojuanlu left a comment

ElenaKhaustova commented Jun 26, 2024

ankatiyar Jun 26, 2024

ankatiyar Jun 26, 2024

ElenaKhaustova Jun 27, 2024

ElenaKhaustova Jun 27, 2024

ankatiyar Jun 27, 2024

merelcht left a comment

merelcht Jun 27, 2024

merelcht Jun 27, 2024

	In the example below we change a logging level to `WARNING` for the `kedro.io.data_catalog` component, so the setting is only applied to the `kedro.io.data_catalog` logger and for the rest of the components the default settings provided via `logging.yml` are remained unchanged.
	In the example below we change a logging level to `WARNING` for the `kedro.io.data_catalog` component logger specifically, the logging for the rest of the components will remain unchanged.

	You can also customise logging at the runtime and redefine the logging configuration provided in the `logging.yml` when using jupyter notebook.
	You can also customise logging at runtime and redefine the logging configuration provided in the `logging.yml` when using jupyter notebook.

	In the example below we change a logging level to `WARNING` for the `kedro.io.data_catalog` component logger specifically, the logging for the rest of the components will remain unchanged.
	The example below demonstrates how you can change the logging level from `INFO` to `WARNING` for the `kedro.io.data_catalog` component logger specifically, the logging for the rest of the components will remain unchanged.

Extended docs with logging customisation at runtime #3968

Extended docs with logging customisation at runtime #3968

Conversation

ElenaKhaustova commented Jun 26, 2024 • edited Loading

Description

Developer Certificate of Origin

Checklist

datajoely left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

astrojuanlu left a comment

Choose a reason for hiding this comment

ElenaKhaustova commented Jun 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

merelcht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ElenaKhaustova commented Jun 26, 2024 •

edited

Loading