feat(metrics): support to bring your own metrics provider #2194

roger-zhangg · 2023-05-02T21:38:33Z

Issue number: #2015

Summary

Changes

This PR refactors Metrics class to use Metrics Provider. Enabling customer to bring their own provider for outputting custom metrics format.The default CloudWatch EMF logic is refactored into a EMF provider.

User experience

For Customer using the default EMF metric format, this PR won't change any UX
For Customer willing to implement custom metrics format in Powertools Metrics
- Before:
  - The Metrics class only supports EMF format.
- After:
  - User can bring their own metrics provider to support custom metrics format

Metric provider UX

The default use case for metrics before is metrics=Metrics()
After we have this provider feature, Customers can still use original CloudWatch Metrics by metrics=Metrics() or metrics=CloudWatchEMF(). They can also use provider for third party provider by e.g.:Metrics=DataDogMetrics()

from aws_lambda_powertools.utilities.typing import LambdaContext
from aws_lambda_powertools.metrics.provider import DatadogMetrics
from aws_lambda_powertools.metrics.provider import DatadogMetricsProvider

# Use datadog-defined metrics provider
provider = DatadogMetricsProvider()
metrics = DatadogMetrics(provider=provider)

@metrics.log_metrics(capture_cold_start_metric=True, raise_on_empty_metrics=False)
def lambda_handler(event: dict, context: LambdaContext):
    metrics.add_metric(name="SuccessfulBooking", value=1, tags={"product":"ticket","ordier":"online"})
    
# JSON output to log
# {
#     "m": "SuccessfulBooking", 
#     "v": 1,
#     "e": 1678509106, 
#     "t": "['product:ticket', 'order:online']"
# }

self-defined metrics provider UX

If the customer would like to use another observability provider, or define their own metrics functions, the provider acts as an interface that the customer can implement and pass to the Metrics class using the provider parameter

from aws_lambda_powertools.metrics.provider import MetricsBase, MetricsProviderBase


class DataDogProvider(MetricsProviderBase):
    def __init__(self, namespace):

    def add_metric(self, name: str, value: float, timestamp: Optional[int] = None, tags: Optional[List] = None):
        self.metrics.append({"m": name, "v": int(value), "e": timestamp, "t": tags})

    def serialize(self) -> Dict:
        # logic here is to add dimension and metadata to each metric's tag with "key:value" format
        extra_tags: List = []
        output_list: List = []

        for single_metric in self.metrics:
            output_list.append(
                {
                    "m": f"{self.namespace}.{single_metric['m']}",
                    "v": single_metric["v"],
                    "e": single_metric["e"],
                    "t": single_metric["t"] + extra_tags,
                }
            )
        return {"List": output_list}

    def flush(self, metrics):
        # flush

    def clear(self):


class DataDogMetrics(MetricsBase):
    """Class for datadog metrics standalone class.

    Example
    -------
    dd_provider = DataDogProvider(namespace="default")
    metrics = DataDogMetrics(provider=dd_provider)

    @metrics.log_metrics(capture_cold_start_metric: bool = True, raise_on_empty_metrics: bool = False)
    def lambda_handler(event, context)
        metrics.add_metric(name="item_sold",value=1,tags)
    """

    # `log_metrics` and `_add_cold_start_metric` are directly inherited from `MetricsBase`
    def __init__(self, provider):
        self.provider = provider
        super().__init__()

    def add_metric(self, name: str, value: float, timestamp: Optional[int] = None, tags: Optional[List] = None):
        self.provider.add_metric(name=name, value=value, timestamp=timestamp, tags=tags)

    def flush_metrics(self, raise_on_empty_metrics: bool = False) -> None:
        metrics = self.provider.serialize()
        if not metrics and raise_on_empty_metrics:
            warnings.warn(
                "No application metrics to publish. The cold-start metric may be published if enabled. "
                "If application metrics should never be empty, consider using 'raise_on_empty_metrics'",
                stacklevel=2,
            )
        self.provider.flush(metrics)
        self.provider.clear()
         
    
metrics = DataDogMetrics(provider=DataDogProvider())

@metrics.log_metrics(capture_cold_start_metric: bool = True, raise_on_empty_metrics: bool = False)
def lambda_handler(event: dict, context: LambdaContext):
    metrics.add_metric(name="SuccessfulBooking", value=1, tags={"product":"ticket","ordier":"online"})
    # call functions in provider
    metrics.provider.add_tag({"test":True})       

# JSON output to log
# {
#     "m": "SuccessfulBooking", 
#     "v": 1,
#     "e": 1678509106, 
#     "t": "['product:ticket', 'order:online']"
# }

Tasks

Create documentation
Check the change that can generate breaking change
Use the simplified version of typing from Python 3.10
Create an external provider like DataDog to validate our logic

Not covered in this PR (Added - 12/07/2023)

Method to create standard operational metrics

Checklist

If your change doesn't seem to apply, please leave them unchecked.

Meet tenets criteria
I have performed a self-review of this change
Changes have been tested
Changes are documented -> Document after changes are reviewed and finalized
PR title follows conventional commit semantics

Is this a breaking change?

RFC issue number:

Checklist:

Migration process documented
Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

boring-cyborg · 2023-05-02T21:38:37Z

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our AWS Lambda Powertools Discord: Invite link

codecov-commenter · 2023-05-02T21:44:44Z

Codecov Report

Patch coverage: 93.96% and project coverage change: -1.14% ⚠️

Comparison is base (0916bc5) 97.57% compared to head (e8cfc81) 96.43%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2194      +/-   ##
===========================================
- Coverage    97.57%   96.43%   -1.14%     
===========================================
  Files          162      169       +7     
  Lines         7475     7666     +191     
  Branches      1416     1447      +31     
===========================================
+ Hits          7294     7393      +99     
- Misses         133      217      +84     
- Partials        48       56       +8

Files Changed	Coverage Δ
aws_lambda_powertools/metrics/metric.py	`0.00% <0.00%> (-100.00%)`	⬇️
aws_lambda_powertools/metrics/metrics.py	`93.75% <90.32%> (-3.75%)`	⬇️
...ools/metrics/provider/cloudwatch_emf/cloudwatch.py	`91.85% <91.85%> (ø)`
aws_lambda_powertools/metrics/provider/base.py	`97.29% <97.29%> (ø)`
aws_lambda_powertools/metrics/__init__.py	`100.00% <100.00%> (ø)`
aws_lambda_powertools/metrics/base.py	`71.23% <100.00%> (-28.77%)`	⬇️
aws_lambda_powertools/metrics/exceptions.py	`100.00% <100.00%> (ø)`
aws_lambda_powertools/metrics/provider/__init__.py	`100.00% <100.00%> (ø)`
...ools/metrics/provider/cloudwatch_emf/cold_start.py	`100.00% <100.00%> (ø)`
...tools/metrics/provider/cloudwatch_emf/constants.py	`100.00% <100.00%> (ø)`
... and 2 more

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

leandrodamascena

Hi @roger-zhangg, excellent work this PR! This is a feature that we really want to have because it is a need for some customers. I made some initial considerations that can help us to make the code simpler and optimize some things.
I still have more to suggest, but I need to look at the code in more detail.

I'll leave some notes to remember what we still have to do to get this ready:

Create documentation
Check the change that can generate breaking change
Use the simplified version of typing from Python 3.10

Thank you very much for this PR, let's work together to merge as soon as possible.

aws_lambda_powertools/metrics/__init__.py

aws_lambda_powertools/metrics/base.py

aws_lambda_powertools/metrics/provider.py

aws_lambda_powertools/metrics/types.py

tests/functional/test_metrics.py

aws_lambda_powertools/metrics/provider/base.py

aws_lambda_powertools/metrics/provider/cloudwatchemf_provider.py

github-actions · 2023-06-05T22:35:11Z