Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(logger): document enriching logs with logrecord attributes #1271

Merged
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 25 additions & 4 deletions docs/core/logger.md
Original file line number Diff line number Diff line change
Expand Up @@ -585,7 +585,7 @@ POWERTOOLS_LOG_DEDUPLICATION_DISABLED="1" pytest -o log_cli=1

## FAQ

**How can I enable boto3 and botocore library logging?**
### How can I enable boto3 and botocore library logging?

You can enable the `botocore` and `boto3` logs by using the `set_stream_logger` method, this method will add a stream handler
for the given name and level to the logging module. By default, this logs all boto3 messages to stdout.
Expand All @@ -594,7 +594,7 @@ for the given name and level to the logging module. By default, this logs all bo
---8<-- "examples/logger/src/enabling_boto_logging.py"
```

**How can I enable powertools logging for imported libraries?**
### How can I enable Powertools logging for imported libraries?

You can copy the Logger setup to all or sub-sets of registered external loggers. Use the `copy_config_to_registered_logger` method to do this.

Expand All @@ -604,7 +604,28 @@ By default all registered loggers will be modified. You can change this behavior
---8<-- "examples/logger/src/cloning_logger_config.py"
```

**What's the difference between `append_keys` and `extra`?**
### How can I add standard library logging attributes to a log record?

The Python standard library log records contains a [large set of atttributes](https://docs.python.org/3/library/logging.html#logrecord-attributes){target="_blank"}, however only a few are included in Powertools Logger log record by default.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, but.... "Powertools Logger" instead of `Powertools Logger"?

Suggested change
The Python standard library log records contains a [large set of atttributes](https://docs.python.org/3/library/logging.html#logrecord-attributes){target="_blank"}, however only a few are included in Powertools Logger log record by default.
The Python standard library log records contains a [large set of atttributes](https://docs.python.org/3/library/logging.html#logrecord-attributes){target="_blank"}, however only a few are included in Powertools `Logger` log record by default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see why it'd make sense, but it'd be difficult to stay consistent over time - let's keep Powertools Logger. At least until we explore tools like Vale.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I think this is ready to merge then.


You can include any of these logging attributes as key value arguments (`kwargs`) when instantiating `Logger` or `LambdaPowertoolsFormatter`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, here Logger is more explicit in intent than Logger.


You can also add them later anywhere in your code with `append_keys`, or remove them with `remove_keys` methods.

=== "collect.py"

```python hl_lines="3 8 10"
---8<-- "examples/logger/src/append_and_remove_keys.py"
```
=== "Example CloudWatch Logs excerpt"

```json hl_lines="6 15-16"
---8<-- "examples/logger/src/append_and_remove_keys.json"
```

For log records originating from Powertools `Logger`, the `name` attribute will be the same as `service`, for log records coming from standard library logger, it will be the name of the logger (i.e. what was used as name argument to `logging.getLogger`).
kbakk marked this conversation as resolved.
Show resolved Hide resolved

### What's the difference between `append_keys` and `extra`?

Keys added with `append_keys` will persist across multiple log messages while keys added via `extra` will only be available in a given log message operation.

Expand All @@ -622,6 +643,6 @@ Here's an example where we persist `payment_id` not `request_id`. Note that `pay
---8<-- "examples/logger/src/append_keys_vs_extra_output.json"
```

**How do I aggregate and search Powertools logs across accounts?**
### How do I aggregate and search Powertools logs across accounts?

As of now, ElasticSearch (ELK) or 3rd party solutions are best suited to this task. Please refer to this [discussion for more details](https://github.com/awslabs/aws-lambda-powertools-python/issues/460)
20 changes: 20 additions & 0 deletions examples/logger/src/append_and_remove_keys.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[
{
"level": "INFO",
"location": "<module>:16",
"message": "Name should be equal service value",
"name": "payment",
"service": "payment",
"timestamp": "2022-07-01 07:09:46,330+0000"
},
{
"level": "INFO",
"location": "<module>:23",
"message": "This will include process ID and name",
"name": "payment",
"process": "9",
"processName": "MainProcess",
"service": "payment",
"timestamp": "2022-07-01 07:09:46,330+0000"
}
]
12 changes: 12 additions & 0 deletions examples/logger/src/append_and_remove_keys.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
from aws_lambda_powertools import Logger

logger = Logger(service="payment", name="%(name)s")

logger.info("Name should be equal service value")

additional_log_attributes = {"process": "%(process)d", "processName": "%(processName)s"}
logger.append_keys(**additional_log_attributes)
logger.info("This will include process ID and name")
logger.remove_keys(["processName"])

# further messages will not include processName
2 changes: 1 addition & 1 deletion examples/logger/src/cloning_logger_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

logger = Logger()

external_logger = logging.logger()
external_logger = logging.getLogger()
heitorlessa marked this conversation as resolved.
Show resolved Hide resolved

utils.copy_config_to_registered_loggers(source_logger=logger)
external_logger.info("test message")