-
Notifications
You must be signed in to change notification settings - Fork 402
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs(logger): document enriching logs with logrecord attributes #1271
Merged
heitorlessa
merged 6 commits into
aws-powertools:develop
from
kbakk:docs/improve-logger-keys-usage
Jul 1, 2022
Merged
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
7505fcb
docs(logger): document enriching logs with logrecord attributes
kbakk f2a24d2
docs(logger): write Powertools with Capitalization
kbakk 3175731
docs(logger): shorten sentences for readability
kbakk 3e5ba84
docs(logger): fix incorrect function name
kbakk fe34e7a
docs(logger): Use headers for FAQ section
kbakk abecfb5
docs(logger): be consistent in use of "Powertools Logger"
kbakk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -585,7 +585,7 @@ POWERTOOLS_LOG_DEDUPLICATION_DISABLED="1" pytest -o log_cli=1 | |
|
||
## FAQ | ||
|
||
**How can I enable boto3 and botocore library logging?** | ||
### How can I enable boto3 and botocore library logging? | ||
|
||
You can enable the `botocore` and `boto3` logs by using the `set_stream_logger` method, this method will add a stream handler | ||
for the given name and level to the logging module. By default, this logs all boto3 messages to stdout. | ||
|
@@ -594,7 +594,7 @@ for the given name and level to the logging module. By default, this logs all bo | |
---8<-- "examples/logger/src/enabling_boto_logging.py" | ||
``` | ||
|
||
**How can I enable powertools logging for imported libraries?** | ||
### How can I enable Powertools logging for imported libraries? | ||
|
||
You can copy the Logger setup to all or sub-sets of registered external loggers. Use the `copy_config_to_registered_logger` method to do this. | ||
|
||
|
@@ -604,7 +604,28 @@ By default all registered loggers will be modified. You can change this behavior | |
---8<-- "examples/logger/src/cloning_logger_config.py" | ||
``` | ||
|
||
**What's the difference between `append_keys` and `extra`?** | ||
### How can I add standard library logging attributes to a log record? | ||
|
||
The Python standard library log records contains a [large set of atttributes](https://docs.python.org/3/library/logging.html#logrecord-attributes){target="_blank"}, however only a few are included in Powertools Logger log record by default. | ||
|
||
You can include any of these logging attributes as key value arguments (`kwargs`) when instantiating `Logger` or `LambdaPowertoolsFormatter`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For example, here |
||
|
||
You can also add them later anywhere in your code with `append_keys`, or remove them with `remove_keys` methods. | ||
|
||
=== "collect.py" | ||
|
||
```python hl_lines="3 8 10" | ||
---8<-- "examples/logger/src/append_and_remove_keys.py" | ||
``` | ||
=== "Example CloudWatch Logs excerpt" | ||
|
||
```json hl_lines="6 15-16" | ||
---8<-- "examples/logger/src/append_and_remove_keys.json" | ||
``` | ||
|
||
For log records originating from Powertools `Logger`, the `name` attribute will be the same as `service`, for log records coming from standard library logger, it will be the name of the logger (i.e. what was used as name argument to `logging.getLogger`). | ||
kbakk marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### What's the difference between `append_keys` and `extra`? | ||
|
||
Keys added with `append_keys` will persist across multiple log messages while keys added via `extra` will only be available in a given log message operation. | ||
|
||
|
@@ -622,6 +643,6 @@ Here's an example where we persist `payment_id` not `request_id`. Note that `pay | |
---8<-- "examples/logger/src/append_keys_vs_extra_output.json" | ||
``` | ||
|
||
**How do I aggregate and search Powertools logs across accounts?** | ||
### How do I aggregate and search Powertools logs across accounts? | ||
|
||
As of now, ElasticSearch (ELK) or 3rd party solutions are best suited to this task. Please refer to this [discussion for more details](https://github.com/awslabs/aws-lambda-powertools-python/issues/460) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
[ | ||
{ | ||
"level": "INFO", | ||
"location": "<module>:16", | ||
"message": "Name should be equal service value", | ||
"name": "payment", | ||
"service": "payment", | ||
"timestamp": "2022-07-01 07:09:46,330+0000" | ||
}, | ||
{ | ||
"level": "INFO", | ||
"location": "<module>:23", | ||
"message": "This will include process ID and name", | ||
"name": "payment", | ||
"process": "9", | ||
"processName": "MainProcess", | ||
"service": "payment", | ||
"timestamp": "2022-07-01 07:09:46,330+0000" | ||
} | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
from aws_lambda_powertools import Logger | ||
|
||
logger = Logger(service="payment", name="%(name)s") | ||
|
||
logger.info("Name should be equal service value") | ||
|
||
additional_log_attributes = {"process": "%(process)d", "processName": "%(processName)s"} | ||
logger.append_keys(**additional_log_attributes) | ||
logger.info("This will include process ID and name") | ||
logger.remove_keys(["processName"]) | ||
|
||
# further messages will not include processName |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit, but.... "Powertools
Logger
" instead of `Powertools Logger"?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see why it'd make sense, but it'd be difficult to stay consistent over time - let's keep Powertools Logger. At least until we explore tools like Vale.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good. I think this is ready to merge then.