Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Powertools logging for imported libraries #40

Closed
FredrikMeyer opened this issue Nov 4, 2021 · 8 comments
Closed

Enable Powertools logging for imported libraries #40

FredrikMeyer opened this issue Nov 4, 2021 · 8 comments
Assignees
Labels
help wanted Extra attention is needed Logger Logger utility Proposed Community submited Python

Comments

@FredrikMeyer
Copy link

Is your feature request related to a problem? Please describe.

It would be nice to easily modify the logger of imported libraries to use JSON logging from Powertools as well.

Describe the solution you'd like

Something like discussed in this discussion: aws-powertools/powertools-lambda-python#799

Maybe something like this?

logger = Logger(...., level="INFO", deny_list=["dont-log-this-library", "and-not-this"])

Describe alternatives you've considered
See the linked discussion.

Thank you!

@heitorlessa heitorlessa self-assigned this Nov 12, 2021
@heitorlessa heitorlessa transferred this issue from aws-powertools/powertools-lambda-python Nov 13, 2021
@heitorlessa heitorlessa added Logger Logger utility Proposed Community submited Python labels Nov 13, 2021
@heitorlessa
Copy link
Contributor

hey @FredrikMeyer thanks for creating the feature request! I'm copying the high level items shared in the discussion on how to make this happen - We can tackle it after the upcoming release this week.


For this feature, it'd need to be slightly more involved. From the top of my head, it would need:

  1. Add a new method in Logger to propagate its configuration to any loggers available, except the ones in the deny list
  2. Add functional tests
  3. Add docs explaining how to use it (we can do that)

Unsure on the UX yet, needs some iteration, but something like this:

from aws_lambda_powertools import Logger

logger = Logger()

logger.clone_config_to_all_loggers(level=…, deny_list=[“logger_named_X”, “botocore”])

This would copy the existing logger logging level & handler/formatter to all registered loggers except root logger (log duplication), and for those in the deny list.

Another area I'm unsure is when using log sampling feature - dynamically set log level to DEBUG on a % of function invocation - do we want to also propagate a DEBUG level to other loggers? I think the answer is no as it could bring too much noise vs actual application per se, as it's intrusive (implicit vs explicit).

@houbie
Copy link

houbie commented Nov 19, 2021

looks good, but I would suggest a standalone function:
def configure_standard_loggers(source_logger: Logger, level=None, includes=None, excludes=None)

from aws_lambda_powertools import Logger

logger = Logger()
configure_standard_loggers(logger) # clone for all loggers with same level as logger
configure_standard_loggers(logger, level=…, includes=[“logger_named_X”]) # only logger_named_X
configure_standard_loggers(level=…, excludes=[“botocore”]) # all except botocore

@heitorlessa
Copy link
Contributor

heitorlessa commented Nov 30, 2021

Thanks @houbie that's what I'm leaning towards the most. One minor possible modification to make it less confusing is to support exclude only instead of both include and exclude. Naming is still the hardest :D.

Edit: Got an initial UX; will try the includes as I'm having second thoughts on how many loggers one might have registered due to dependencies one might be unaware of.

import boto3

from aws_lambda_powertools import Logger
from aws_lambda_powertools.logging.utils import copy_config_to_registered_loggers

logger = Logger()
logger.info("test")

# copies handler and log level info from `logger` to `boto3`, `botocore`, `urllib`
copy_config_to_registered_loggers(source_logger=logger)

session = boto3.Session()
s3 = session.client("s3")

s3.list_buckets()

@heitorlessa
Copy link
Contributor

Updating here to let you know that @mploski is taking this opportunity to get his first contribution - we'll let you know which release this will be part once he's ready... but you can also take a peek at the PR code for your own use

@FredrikMeyer
Copy link
Author

Thanks for the update @heitorlessa 👍

@mploski
Copy link

mploski commented Jan 18, 2022

@FredrikMeyer PR has been merged and this feature will be available in coming release. Here is the PR link if you would like to look at this: aws-powertools/powertools-lambda-python#927

@FredrikMeyer
Copy link
Author

Thank you very much for the update. Appreciated!

@heitorlessa
Copy link
Contributor

This is now released as part of 1.24.1 ;) https://github.com/awslabs/aws-lambda-powertools-python/releases/tag/v1.24.1

Lambda Layers will be published later today

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed Logger Logger utility Proposed Community submited Python
Projects
None yet
Development

No branches or pull requests

4 participants