Unit tests of example do not pass

[Nxtra]

Current behaviour

I followed the instructions in the README and installed the dependencies in a virtual environment

pip install -r hello_world/requirements.txt && pip install -r requirements-dev.txt

Then I run the tests with : POWERTOOLS_TRACE_DISABLED=1 python -m pytest.

The test fails with output:
FAILED tests/unit/test_handler.py::test_lambda_handler - aws_lambda_powertools.metrics.exceptions.SchemaValidationError: Invalid format. Error: data._aws.CloudWatchMetrics[0].Namespace must be string, Invalid item: data._aws.CloudWatchMetrics[0...
( I can provide a full stacktrace if necessary)

Potential solution
I looked into base.py of the aws_lambda_powertools metrics package and found that it is looking for an env var POWERTOOLS_METRICS_NAMESPACE.
This env var is not there when running the tests.

First setting the env var and then running it solves the issue:
POWERTOOLS_TRACE_DISABLED=1 POWERTOOLS_METRICS_NAMESPACE=test python -m pytest tests -v

If my assessment is correct, I propose to update the README. If incorrect can you help me by determining what is causing this problem?

[heitorlessa]

Thanks for reporting it! Your assessment is spot on. I believe you were trying to run the tests for the example inside the project - Correct me I’m wrong, and you used a fresh new project instead. For tests to work correctly without a metric namespace, we have to either set the namespace explicitly, or to use pytest monkeypatch.setenv(..) as part of the test. I’ll get it fixed later today. Thanks again

…

On Sun, 26 Apr 2020 at 23:12, Nxtra ***@***.***> wrote: *Current behaviour* I followed the instructions in the README and installed the dependencies in a virtual environment pip install -r hello_world/requirements.txt && pip install -r requirements-dev.txt Then I run the tests with : POWERTOOLS_TRACE_DISABLED=1 python -m pytest. The test fails with output: FAILED tests/unit/test_handler.py::test_lambda_handler - aws_lambda_powertools.metrics.exceptions.SchemaValidationError: Invalid format. Error: data._aws.CloudWatchMetrics[0].Namespace must be string, Invalid item: data._aws.CloudWatchMetrics[0... ( I can provide a full stacktrace if necessary) *Potential solution* I looked into base.py of the aws_lambda_powertools metrics package and found that it is looking for an env var POWERTOOLS_METRICS_NAMESPACE. This env var is not there when running the tests. First setting the env var and then running it solves the issue: POWERTOOLS_TRACE_DISABLED=1 POWERTOOLS_METRICS_NAMESPACE=test python -m pytest tests -v If my assessment is correct, I propose to update the README. If incorrect can you help me by determining what is causing this problem? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <aws-powertools/powertools-lambda#24>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZPQBFZ3GIHSOEZZ3W7KO3ROSWWHANCNFSM4MRNWNFQ> .

[heitorlessa]

@Nxtra - Just pushed a fix to develop branch, and thank you again for flagging this and help us improve while in beta.

Updates to the README

Relevant and updated part in bold

• Unit Tests: We recommend proceeding with the following commands in a virtual environment
  • Install deps: pip install -r hello_world/requirements.txt && pip install -r requirements-dev.txt
  • Run tests with tracing disabled and namespace set
    • POWERTOOLS_METRICS_NAMESPACE="Example" POWERTOOLS_TRACE_DISABLED=1 python -m pytest
    • Both are necessary because app.py initializes them in the global scope, since both Tracer and Metrics will be initialized and configured during import time. For unit tests, we could always patch and explicitly config but env vars do just fine for this example.

Updates to the test

We now capture the stdout, and assert if our custom middleware and metrics were correct:

def test_lambda_handler(apigw_event, mocker, capsys):
    ...

    output = capsys.readouterr()
    output = output.out.split('\n')
    stdout_one_string = '\t'.join(output)
   ...
    
    # assess custom metric was flushed in stdout/logs
    assert "SuccessfulLocations" in stdout_one_string 
    assert "ColdStart" in stdout_one_string 
    assert "UniqueMetricDimension" in stdout_one_string

    # assess our custom middleware ran
    assert "Logging response after Handler is called" in stdout_one_string
    assert "Logging event before Handler is called" in stdout_one_string

[greenpau]

POWERTOOLS_METRICS_NAMESPACE="Example" POWERTOOLS_TRACE_DISABLED=1 python -m pytest

When running in tests, you could apply it to all tests inside a test case:

@patch.dict(os.environ, {
    "UNIT_TEST_ENV": "yes",
    "POWERTOOLS_METRICS_NAMESPACE": "Example",
    "POWERTOOLS_TRACE_DISABLED": "1",
})
class HandlerTestCase(BasicTestCase):
    def test_handler(self):