Skip to content
This repository has been archived by the owner on Jan 28, 2025. It is now read-only.

feat(lambda-at-edge): use new aws s3 client for faster require time #583

Merged
merged 7 commits into from
Sep 4, 2020

Conversation

dphang
Copy link
Collaborator

@dphang dphang commented Sep 4, 2020

Description

This closes #580.

  • Uses the client here in AWS SDK JS v3, or @aws-sdk/client-s3 (https://github.com/aws/aws-sdk-js-v3/tree/master/clients/client-s3) which is a more modularized client. It will allow us to import only the commands needed (GetObject, PutObject) instead of heavy S3 client in aws-sdk, and do it dynamically instead of in every function invocation (i.e before the handler). E.g this will only need it for getStaticPaths cases.
  • Tests in the linked Issue have shown it takes < 3 ms to load all of these. Unfortunately it does increase handler size from ~150 kB to ~500 kB because there are still a lot of other types for other S3 APIs (but the command themselves only import 2) (but code size has very little impact on cold start times compared to require size, i.e requiring aws-sdk/S3 will load all APIs. In another PR, we could use terser with Rollup to reduce this to ~180 kB (and we can provide both minified and normal handlers).

Note, I had to use rollup-plugin-typescript2 (a fork of original plugin) as I was getting these errors and could not resolve it:

[!] (plugin typescript) Error: Could not load /Users/danielphang/IdeaProjects/serverless-next.js/packages/libs/lambda-at-edge/node_modules/@aws-sdk/client-s3/S3Client.ts (imported by src/default-handler.ts): Debug Failure. False expression: Expected fileName to be present in command line

Though this is better in the sense that it prints more informative error messages. I also added json rollup plugin as it is needed to bundle the AWS client correctly.

Tests

Updated the unit tests in origin-response handler test.

Currently in progress of testing end-to-end.

@dphang dphang marked this pull request as ready for review September 4, 2020 05:47
@dphang dphang marked this pull request as draft September 4, 2020 05:49
@dphang
Copy link
Collaborator Author

dphang commented Sep 4, 2020

This has been tested on this test app:

https://d2fn5f8m5hw83u.cloudfront.net/tags/128128218 (try replacing the end number with a random number, it will load fallback until it gets the data)

@dphang dphang marked this pull request as ready for review September 4, 2020 19:15
import { performance } from "perf_hooks";
import { ServerResponse } from "http";
import jsonwebtoken from "jsonwebtoken";
import { Readable } from "stream";
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a type, should be fine to import here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
import { Readable } from "stream";
import type { Readable } from "stream";

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I thought Rollup would optimize this but I guess explicitly importing just type is good practice, too.

const { Body } = await s3.send(new GetObjectCommand(s3Params));

// Body is stream per: https://github.com/aws/aws-sdk-js-v3/issues/1096
const getStream = await import("get-stream");
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am using get-stream package: https://bundlephobia.com/[email protected] which is small

@@ -156,8 +156,10 @@ const router = (
export const handler = async (
event: OriginRequestEvent | OriginResponseEvent
): Promise<CloudFrontResultResponse | CloudFrontRequest> => {
// @ts-ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need these @ts-ignores ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I saw errors when building at one point due to this being from json file, maybe this one is not needed anymore, let me try to remove it

Copy link
Contributor

@danielcondemarin danielcondemarin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few minor things but overall lgtm 👍

@danielcondemarin danielcondemarin merged commit f9eef45 into serverless-nextjs:master Sep 4, 2020
@dphang
Copy link
Collaborator Author

dphang commented Sep 4, 2020

Thanks for merging!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

S3 import in default-handler likely causing higher cold start time
2 participants