Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(metrics): log directly to stdout #1786

Merged
merged 4 commits into from
Nov 10, 2023
Merged

feat(metrics): log directly to stdout #1786

merged 4 commits into from
Nov 10, 2023

Conversation

dreamorosi
Copy link
Contributor

@dreamorosi dreamorosi commented Nov 8, 2023 •
edited
Loading

Description of your changes

This PR changes the console object used in the Metrics utility from the global one (aka console.log) to one instantiated by the utility itself. For most customers this change will have no impact given that the EMF object is going to continue to be parsed as usual, however those customers who send their logs to other destinations other than CloudWatch will now be able to parse the log as a well-formed JSON rather than a log that contains a timestamp, a request ID, and the EMF object.

The implementation follows the one in the Logger utility, which means that by default it will use a new instance of the Console object from the node:console module. When the POWERTOOLS_DEV environment variable is set to a truthy value, the both the Metrics & Logger utility will instead revert back to the global console object. This allows customers to easily set spies in their tests.

I have extracted the logic that checks this variable from the Logger to the Commons package, so that the Metrics utility can reuse it.

Related issues, RFCs

Issue number: #1766

Checklist

  • My changes meet the tenets criteria
  • I have performed a self-review of my own code
  • I have commented my code where necessary, particularly in areas that should be flagged with a TODO, or hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my change is effective and works
  • The PR title follows the conventional commit semantics

Breaking change checklist

Is it a breaking change?: NO

  • I have documented the migration process
  • I have added, implemented necessary warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

@dreamorosi dreamorosi self-assigned this Nov 8, 2023
@dreamorosi dreamorosi requested a review from a team as a code owner November 8, 2023 18:04
@boring-cyborg boring-cyborg bot added commons This item relates to the Commons Utility logger This item relates to the Logger Utility metrics This item relates to the Metrics Utility tests PRs that add or change tests labels Nov 8, 2023
@pull-request-size pull-request-size bot added the size/XL PRs between 500-999 LOC, often PRs that grown with feedback label Nov 8, 2023
@github-actions github-actions bot added the feature PRs that introduce new features or minor changes label Nov 8, 2023
@dreamorosi dreamorosi marked this pull request as draft November 8, 2023 19:02
@dreamorosi
Copy link
Contributor Author

Moving this back to draft as I have to fix integration tests in the Layers that are now failing.

Metrics integration tests are passing because we are asserting against the metrics retrieved from CloudWatch, this shows that the change is successful. Layers on the other hand are failing because we are doing an assertion on the shape of the logs, which has now changed.

Tomorrow I'll fix and mark it ready.

@sonarqubecloud SonarQubeCloud
Copy link

sonarqubecloud bot commented Nov 9, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
1.9% 1.9% Duplication

@dreamorosi
Copy link
Contributor Author

I have integration tests passing:
https://github.com/aws-powertools/powertools-lambda-typescript/actions/runs/6816958295

@dreamorosi dreamorosi marked this pull request as ready for review November 9, 2023 20:25
@dreamorosi dreamorosi removed the tests PRs that add or change tests label Nov 9, 2023
@dreamorosi dreamorosi linked an issue Nov 10, 2023 that may be closed by this pull request
Feature request: TypeScript: Log Metrics with JSON (not console.log) #1766
Closed
2 tasks
@am29d am29d merged commit 75dc5b1 into main Nov 10, 2023
47 checks passed
@am29d am29d deleted the feat/metrics_stdout branch November 10, 2023 12:44
dreamorosi added a commit that referenced this pull request Nov 20, 2023
@dreamorosi
feat(metrics): log directly to stdout (#1786)
9c47362 
* chore(commons): move isDevMode to commons

* chore(logger): move isDev config out of logger to commons

* feat(metrics): use own console object by default

* tests(layers): fix unit tests
dreamorosi added a commit that referenced this pull request Jan 27, 2024
@dreamorosi
feat(metrics): log directly to stdout (#1786)
bdb6d00 
* chore(commons): move isDevMode to commons

* chore(logger): move isDev config out of logger to commons

* feat(metrics): use own console object by default

* tests(layers): fix unit tests
dreamorosi added a commit that referenced this pull request Feb 5, 2024
@dreamorosi
feat(metrics): log directly to stdout (#1786)
60faa14 
* chore(commons): move isDevMode to commons

* chore(logger): move isDev config out of logger to commons

* feat(metrics): use own console object by default

* tests(layers): fix unit tests
dreamorosi added a commit that referenced this pull request Feb 5, 2024
@dreamorosi
feat(metrics): log directly to stdout (#1786)
b31803a 
* chore(commons): move isDevMode to commons

* chore(logger): move isDev config out of logger to commons

* feat(metrics): use own console object by default

* tests(layers): fix unit tests
dreamorosi added a commit that referenced this pull request Feb 20, 2024
@dreamorosi
feat(metrics): log directly to stdout (#1786)
f0dd4c0 
* chore(commons): move isDevMode to commons

* chore(logger): move isDev config out of logger to commons

* feat(metrics): use own console object by default

* tests(layers): fix unit tests
dreamorosi added a commit that referenced this pull request Feb 28, 2024
@dreamorosi @erikayao93
@antstanley @am29d @shdq @github-actions @dependabot
feat!: add ESM support & logger extensibility (#2117)
3b99263 
* feat(logger): Support for external observability providers (#1511)

* Updated formatAttributes for additional parameters and LogItem return type

* Updated the unit tests to pass with new formatter

* Updated Powertool named objects to Powertools

* Updated tests to match new naming consistency

* Updated for tests for new naming consistency

* Updated formatter for new design decisions

* Update Logger for ephemeral attributes

* Update bringYourOwnFormatter documentation to match new formatter

---------

Co-authored-by: erikayao93 <[email protected]>

* chore(logger): PowertoolsLogFormatter docstring and variable naming update (#1585)

* Updated formatAttributes for additional parameters and LogItem return type

* Updated the unit tests to pass with new formatter

* Updated Powertool named objects to Powertools

* Updated tests to match new naming consistency

* Updated for tests for new naming consistency

* Updated formatter for new design decisions

* Update Logger for ephemeral attributes

* Update bringYourOwnFormatter documentation to match new formatter

* Fixed incorrect return type, renamed variable for consistency

* feat(logger): Support for external observability providers (#1511)

* Updated formatAttributes for additional parameters and LogItem return type

* Updated the unit tests to pass with new formatter

* Updated Powertool named objects to Powertools

* Updated tests to match new naming consistency

* Updated for tests for new naming consistency

* Updated formatter for new design decisions

* Update Logger for ephemeral attributes

* Update bringYourOwnFormatter documentation to match new formatter

---------

Co-authored-by: erikayao93 <[email protected]>

* chore(logger): PowertoolsLogFormatter docstring and variable naming update (#1585)

* Updated formatAttributes for additional parameters and LogItem return type

* Updated the unit tests to pass with new formatter

* Updated Powertool named objects to Powertools

* Updated tests to match new naming consistency

* Updated for tests for new naming consistency

* Updated formatter for new design decisions

* Update Logger for ephemeral attributes

* Update bringYourOwnFormatter documentation to match new formatter

* Fixed incorrect return type, renamed variable for consistency

* chore(maintenance): bump dependencies & drop nodejs14x (#1687)

* chore: update release script to mark all utilities as alpha

* chore: restore version to ease conflicts

* chore: release version change

* chore: release version change

* chore(maintenance): remove `createLogger` and `createTracer` helpers (#1722)

* chore(maintenance): bump dependencies & drop nodejs14x (#1687)

* chore: add pre-release script

* chore: restore deps

* chore: added v2 shim

* chore(maintenance): remove logger and tracer helper function

* chore: remove imports

* chore: fix deps & versions

* tests: moved unit tests

* tests: move logger tests

* chore: added v2 shim

* chore: added v2 shim

* feat(logger): add esmodule support (#1734)

* feat(logger): add esm build output

* fix(Logger): Remove barrel files update references

* test(Logger): update jest/ts-jest to use ESM

* chore(Logger): remove unused lodash.merge

* fix(logger): reinstate lodash.merge

* chore(logger): revert TS assertion

* chore(logger): revert format changes

* chore(logger): update postbuild to remove incremental tsbuildinfo files

* fix(logger): correct reference to types output

* feat(logging): add middleware export

* chore(logger): replace postbuild script with echo statement

* feat(logger): add typesVersions property and barrel files to /types

* chore(logger): file not used, can be added back if needed

* chore(logger): add space back to README

* chore(logger): revert space in README

* feat(commons): add esmodule support (#1735)

* chore(logger): adapt logger to commons exports

* feat(commons): add esmodule support

* chore: address sonar findings

* chore(commons): exported version

* chore: fixed imports in examples

* chore(parameters): fixed imports

* chore(metrics): fixed imports

* chore(tracer): fixed imports

* chore(idempotency): fixed imports

* chore(commons): test coverage

* chore(batch): fix imports

* feat(parameters): add esmodule support (#1736)

* feat(batch): add esmodule support (#1737)

* feat(internal): add esmodule support (#1738)

* feat(testing): add esmodule support

* chore(all): update imports

* feat(metrics): add esmodule support (#1739)

* feat(tracer): add esmodule support (#1741)

* feat(tracer): add esmodule support

* chore(docs): update imports

* feat(idempotency): add esmodule support  (#1743)

* feat(idempotency): add esmodule support

* chore(metrics): fix import

* chore(ci): v2 release line

* chore(ci): fix alpha versioning pre-release

* docs(maintenance): add processes tab (#1747)

* docs(maintenance): update mkdocs to support tabs

* chore(ci): add parallel test npm script

* chore(ci): add jest command

* docs(maintenance): add testing page to navbar

* docs(maintenance): add contributing info

* chore: update roadmap

* chore: update release drafter workflow to allow for manual trigger

* fix formatting

* docs: maintainers handbook

* chore: link to new location

* fix links

* Update docs/maintainers.md

Co-authored-by: Alexander Schueren <[email protected]>

---------

Co-authored-by: Alexander Schueren <[email protected]>

* chore(docs): add invisible unicode char to decorator docs (#1755)

* chore: remove extra comma

* chore(docs): upgrade doc intro

* feat(logger): align sampling debug logs feature implementation with the other runtimes (#1744)

* test(logger): remove logsSampled field, add default sampleRateValue

* test(logger): add tests for sampling debug logs feature

* feat(logger): change implementation to make sampling decision at per-function level

* refactor(logger): remove redundant createLogger method

* refactor(logger): remove getSampleRateValue method

* test(logger): improve tests

* refactor(logger): return createLogger() back with the detailed comment of the method importance

* test(logger): add constructor/custom config/env var priority tests for sampling rate feature, improve description

* refactor(logger): address review comments

* feat(logger): add refreshSampleRateCalculation method and tests

* test(logger): adjust end-to-end tests

* chore(logger): refactor types and interfaces (#1758)

* chore(logger): refactor types and interfaces

* chore: grouped type files

* chore: fix code smell

* chore: fix ci

* chore: fix ci

* chore(maintenance): bump Middy v4 & run tests (#1760)

* chore(parameters): fix esm bundling

* chore(parameters): change declare client param

* chore(layers) widen version check in e2e

* chore(maintenance): enable `isolatedModules` and isolate cache (#1765)

* chore(layers) widen version check in e2e

* chore(maintenance): enable isolatedModules

* chore: remove redundant comments from tsconfig

* chore: changed path of tsbuild cache

* fix: idempotency types

* build(maintenance): bump aws sdk dev dependencies

* chore(parameters): add export types

* chore(logger): set default UTC timezone (#1775)

* chore(parameters): add export types

* chore(logger): set default utc timezone

* chore(logger): pass down envvarsservice to log formatter

* fix(metrics): deduplicate dimensions when serialising (#1780)

* fix: deduplicate dimensions when serializing

* fix tests

* remove tsbuildinfo

* remove whitespace

* fix gitignore again

* play some sonar games

* fix test

* chore(release): v1.14.2 [skip ci]

* feat(metrics): log directly to stdout  (#1786)

* chore(commons): move isDevMode to commons

* chore(logger): move isDev config out of logger to commons

* feat(metrics): use own console object by default

* tests(layers): fix unit tests

* feat(maintenance): add support for nodejs20.x runtime (#1790)

* feat(maintenance): support nodejs20.x runtime

* tests(metrics): fix object ordering in tests

* build(testing): bump cdk

* build(maintenance): revert aws-cdk-lib to support nodejs14

* tests(maintenance): set runtime with family

* chore(docs): patch runtime in cdk

* chore(docs): patch runtime in cdk

* chore(maintenance): increment version in commons ahead of release

* chore(commons): simplify config service interface

* chore(release): v1.15.0 [skip ci]

* feat(logger): add support for `AWS_LAMBDA_LOG_LEVEL` and `POWERTOOLS_LOG_LEVEL` (#1795)

* feat(logger): support advanced logging

* docs(logger): add alc info

* feat(logger): support alc

* docs: fix alc docs links

* tests(logger): add unit tests for the feature

* docs(logger): make POWERTOOLS_LOG_LEVEL default

* chore(release): v1.16.0 [skip ci]

* chore(maintenance): drop support for Node.js 14 (#1802)

* chore: package lock

* chore(docs): update docs url in comments & readme files (#1728)

* chore(ci): update v2 release workflow (#1745)

* docs(maintenance): add processes tab (#1747)

* docs(maintenance): update mkdocs to support tabs

* chore(ci): add parallel test npm script

* chore(ci): add jest command

* docs(maintenance): add testing page to navbar

* docs(maintenance): add contributing info

* chore: update roadmap

* chore: update release drafter workflow to allow for manual trigger

* fix formatting

* docs: maintainers handbook

* chore: link to new location

* fix links

* Update docs/maintainers.md

Co-authored-by: Alexander Schueren <[email protected]>

---------

Co-authored-by: Alexander Schueren <[email protected]>

* chore(maintenance): add --require-hashes flag to pip installs (#1827)

* bump version to 9.5.2, rerun pip-compile with correct deps (#1830)

* chore(ci): Dependabot fine tuning (#1862)

* ignore major updates for mike

* set versioning strategy for cdk, dependabot is running on auto upgrade strategy per default

* ignore middy major upgrades

* remove CodeQL, it's enough to have Sonar

* update package-lock

* chore(deps-dev): bump @aws-sdk/client-cloudwatch from 3.438.0 to 3.485.0 (#1857)

* chore(deps-dev): bump @types/node from 20.11.0 to 20.11.2 (#1912)

* chore(deps): fix dependencies and dependabot config (#1917)

* chore(deps-dev): bump @typescript-eslint/parser from 6.19.0 to 6.19.1 (#1946)

Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 6.19.0 to 6.19.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v6.19.1/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps-dev): bump @typescript-eslint/eslint-plugin (#1948)

Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 6.19.0 to 6.19.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v6.19.1/packages/eslint-plugin)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(ci): refactor workflows to scope permissions (#1978)

* chore(maintenance): rebase conflicts

* chore(maintenance): rebase conflicts

* chore: rebase conflicts

* chore: remove v2 specific release

* chore: release patch script

* docs: update docs & snippets

* chore: address SonarCloud findings

* chore: address SonarCloud findings

* chore: address SonarCloud findings

* tests: coverage

* chore: add layers v2

* chore: bump ua version

* docs: add install steps to batch

* chore: fix install command

* chore: remove unused dependency

* Update examples/cdk/package.json

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Erika Yao <[email protected]>
Co-authored-by: erikayao93 <[email protected]>
Co-authored-by: Ant Stanley <[email protected]>
Co-authored-by: Alexander Schueren <[email protected]>
Co-authored-by: Sergei Cherniaev <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@am29d am29d am29d approved these changes

Assignees

@dreamorosi dreamorosi

Labels
commons This item relates to the Commons Utility feature PRs that introduce new features or minor changes logger This item relates to the Logger Utility metrics This item relates to the Metrics Utility size/XL PRs between 500-999 LOC, often PRs that grown with feedback
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Feature request: TypeScript: Log Metrics with JSON (not console.log)
2 participants
@dreamorosi @am29d