Powertools for AWS Lambda (TypeScript) v3 - Ideas and notes

[dreamorosi]

I would like to open a discussion to gather and ideally discuss ideas and changes for our next v3 milestone.

Note that at the time of writing we don't have any date nor plans for our next major version, however I think it'd be good to start gathering notes & ideas for things we might encounter along the way that we wish we could address but can't due to major refactors or breaking changes.

[dreamorosi]

Relating to the Batch Processor utility, the components of the utility follow an inheritance pattern with some abstract classes and methods. During the initial implementation we weren't necessarily the most diligent and so almost all methods were marked as public.

Some of these methods (my feeling is that it's most of them) could potentially be converted to protected instead, since they are meant to be used only by classes extending the base class.

As part of v3 we could consider doing an exercise of minor refactoring and clean up the public API.

[HaaLeo]

Maybe consider #749 for v3 if that would cause any breaking changes.

[dreamorosi]

In aws-powertools/powertools-lambda-python#4907 the extractDataFromEnvelope() function part of the JMESPath utility was renamed to simply query(), we should do the same.

[dreamorosi]

As discussed in #3085, in v3 we will switch the experimentalDecorators setting in the tsconfig.json file to false.

In v5.0 of TypeScript, the default changed to Stage 3 decorators, which modify the signature of the decorator significantly as discussed more in depth here.

While doing this, we should also make our decorators compatible with asynchronous methods only. Today they support both but in reality they implicitly convert sync methods to async under the hood. This is not a huge issue when decorating the main function handler since AWS Lambda will handle the promise correctly, but when decorating arbitrary methods this can cause subtle bugs.

[dreamorosi]

Following the implementation of #3160 that adds async processing to SQS FIFO queues, to avoid breaking changes we had to invert the convention of XXProcessor being async and XXProcessorSync being the synchronous opt-in. In this case the existing SQS FIFO processor stayed sync and we added an asynchronous one.

At minimum, on v3 we should standardize this implementation to follow the rest of the Batch Processor classes.

Additionally, as @am29d pointed out, there might be an opportunity to streamline the implementation and API of this utility significantly while making minimal breaking changes.

For example, we could explore reducing the number of classes to have only one batch processor for event source, and then have both synchronous and asynchronous methods on it, so that customers don't have to think which one they need to import and use but just call BatchProcessor.process() (async) or BatchProcessor.processSync().

We are unsure if this is possible or the extent of breaking changes it'll require - for now we're adding this to the v3 list to explore further.

[dreamorosi]

Another item we're looking to remove or rework is the custom configuration services from the core utilities.

Today it's possible to pass a custom configuration service to the constructor of Tracer, Metrics, and Logger which will be used during initialization.

Currently this feature doesn't seem to have gotten any traction, most likely also because it's not documented anywhere, but also because it's too rigid.

Defining your custom configurations service requires you to reimplement the entire interface, which might not be desirable when you want to only override one behavior.

[dreamorosi]

In v3 we should stop including the service dimension to metrics emitted when the service name is not specified via POWERTOOLS_SERVICE_NAME env variable or serviceName constructor options.

Currently we still add the dimension but with value service_undefined, which adds no value and only increases the size of the log emitted.

Customers have asked us to change this but doing so requires a breaking change, so I'm adding this here.

More info in #3374.

[am29d]

Change POWERTOOLS_TRACE_ENABLED to POWERTOOLS_TRACE_DISABLED to make it consistent across all runtimes.