Maintenance: add mechanism to check compatibility with @types/aws-lambda package

[am29d]

Summary

When using our built-in schema with zod, developers can infer the types with type SqsEvent = z.infer<typeof SqsSchema> and use it in their Lambda handler. Another popular option is to use @types/aws-lambda package wich has 3M downloads per week. We need to make sure that both types are compatible, so there is no confusion or unexpected behaviour.

Why is this needed?

If either @types/aws-lambda or our schema is updated, we want to have a reliable test to verify that they are both compatible and if not to fix one them.

Which area does this relate to?

Parser

Solution

I have checked the first idea that worked:

import { z } from 'zod';
import { ALBEvent, APIGatewayProxyEvent } from 'aws-lambda';
import { AlbSchema } from '../../src/schemas/alb';
import { APIGatewayProxyEventSchema } from '../../src/schemas/apigw';

type AlbType = z.infer<typeof AlbSchema>;
type ApiGatewayType = z.infer<typeof APIGatewayProxyEventSchema>;
describe('Types compatibility', () => {
  it('ALBEvent types are compatible', () => {
    // These functions will fail to compile if A and B are not structurally identical
    // eslint-disable-next-line @typescript-eslint/no-unused-vars
    const A = (value: AlbType): ALBEvent => value;
    // eslint-disable-next-line @typescript-eslint/no-unused-vars
    const B = (value: ALBEvent): AlbType => value;
  });

  it('ApiGatewayEvent types are compatible', () => {
    // eslint-disable-next-line @typescript-eslint/no-unused-vars
    const A = (value: ApiGatewayType): APIGatewayProxyEvent => value;
    // eslint-disable-next-line @typescript-eslint/no-unused-vars
    const B = (value: APIGatewayProxyEvent): ApiGatewayType => value;
  });
});

The compiler will fail if the structure does not match and will tell exactly which properties are not compatible.

Alternatively research more about https://github.com/fabien0102/ts-to-zod. If we can convert @types/aws-lambda to schemas, we might directly compare the schemas to find the differences.

Acknowledgment

• This request meets Powertools for AWS Lambda (TypeScript) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[dreamorosi]

Copying this comment I have left under a PR since it's relevant to the conversation:

---

[...] I think we should agree & articulate our stance on this topic a bit better to avoid future confusion amongst us and towards customers.

The main point here, besides calling out potential discrepacies between us & the other library, is in my opinion to establish our strategy towards event types as a whole. This will then influence which actions we'll take now and in the future, as well as when discrepacies are detected.

Currently the @types/aws-lambda types are community maintained and manually authored. AWS events can change and the types can contain issues.

While I agree with the sentiment of maintaining these types and our schemas (and in the future JSON schemas as well) up to date and aligned, I don't think this ultimately guarantees that the types/schemas are actually correct.

As a think big idea, we should at least consider the option of instead investing efforts in creating automations to periodically test against real AWS events to get the actual shapes of the events sent to Lambda. Based of those, we could then derive the schemas as well as the TypeScript types.

Once we detect drift (aka changes in the events), we could then propagate them to the appropriate destinations be it within the scope of Powertools or @types/aws-lambda.

I am not expecting the whole industry to move over our own types (i.e. @aws-lambda-powertools/events or @aws-lambda-powertools/types) but I would still aim at becoming the source of truth for these events/schemas/types.

The DefinitelyTyped repo (aka the one that hosts @types/* packages) already has precedents of depending on external type definition, one of them was added in the past 6 months to the @types/node types to support the fetch module. This module was originally implemented under undici and later incorporated into Node.js. The source still resides in the undici repo, so to add support for these types they started publishing type definitions for under undici-types that are now a direct depenencey pf the @types/node package (see here).

While it'll be a lot of work, I believe this would bring a lot of value to customers and eventually would help with establishing a single and authoritative source of truth for AWS events that today does not exist. Doing this would benefit not only Powertools TS or the Node.js community but also Powertools Python and the Python community as a whole via Pydantic. Additionally, it would open the door to future use cases like testing factories (#445), integrations with SAM or CDK, and similar.