Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update to aws-sdk 3.4.1 #7726

Merged
merged 12 commits into from
Feb 10, 2021
Merged

chore: update to aws-sdk 3.4.1 #7726

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
70f15aa
chore: upgrade aws-sdk to rc.9
amhinson Feb 9, 2021
2f4150c
integ testing
amhinson Feb 9, 2021
d9d6c45
chore: update aws-sdk to rc.10
amhinson Feb 9, 2021
86db32a
chore: bump aws-sdk to 3.1.0
amhinson Feb 9, 2021
5df6112
upgrade to aws-sdk 3.4.0
amhinson Feb 10, 2021
9cbf6b9
bump to 3.4.1
amhinson Feb 10, 2021
6e44f5a
add middleware
amhinson Feb 10, 2021
0d645fa
temp
amhinson Feb 10, 2021
ddf1f6d
add middleware
amhinson Feb 10, 2021
2a155a6
remove CCI testing
amhinson Feb 10, 2021
c4b4caf
update bundle sizes
amhinson Feb 10, 2021
abe342e
require multi-part integ tests for deploy
amhinson Feb 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -885,8 +885,7 @@ workflows:
- integ_react_predictions
- integ_react_datastore
- integ_react_storage
# Removing this until AWS SDK issue is resolved: https://github.com/aws/aws-sdk-js-v3/issues/2000
# - integ_react_storage_multipart_progress
- integ_react_storage_multipart_progress
- integ_react_storage_ui
- integ_react_interactions
- integ_angular_interactions
Expand All @@ -896,12 +895,10 @@ workflows:
- integ_angular_auth
- integ_vue_auth
- integ_rn_ios_storage
# Removing this until AWS SDK issue is resolved: https://github.com/aws/aws-sdk-js-v3/issues/2000
# - integ_rn_ios_storage_multipart_progress
- integ_rn_ios_storage_multipart_progress
- integ_rn_ios_push_notifications
- integ_rn_android_storage
# Removing this until AWS SDK issue is resolved: https://github.com/aws/aws-sdk-js-v3/issues/2000
# - integ_rn_android_storage_multipart_progress
- integ_rn_android_storage_multipart_progress
- integ_datastore_auth
- post_release:
filters:
Expand Down
8 changes: 4 additions & 4 deletions .github/actions/bundle-size-action/webpack/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,23 +31,23 @@
},
{
"path": "dist/Amplify+Auth.js.min.js",
"maxSize": "70kB"
"maxSize": "75kB"
},
{
"path": "dist/Amplify+Auth+Storage.js.min.js",
"maxSize": "110kB"
"maxSize": "115kB"
},
{
"path": "dist/Amplify+Storage.js.min.js",
"maxSize": "75kB"
},
{
"path": "dist/withSSRContext.js.min.js",
"maxSize": "140kB"
"maxSize": "145kB"
},
{
"path": "dist/withSSRContext+Storage.js.min.js",
"maxSize": "175kB"
"maxSize": "180kB"
}
]
},
Expand Down
2 changes: 1 addition & 1 deletion .github/actions/bundle-size-action/webpack4/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
},
{
"path": "dist/Amplify+Auth.js.min.js",
"maxSize": "200kB"
"maxSize": "215kB"
}
]
},
Expand Down
10 changes: 5 additions & 5 deletions packages/analytics/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,11 @@
"dependencies": {
"@aws-amplify/cache": "3.1.46",
"@aws-amplify/core": "3.8.13",
"@aws-sdk/client-firehose": "1.0.0-rc.4",
"@aws-sdk/client-kinesis": "1.0.0-rc.4",
"@aws-sdk/client-personalize-events": "1.0.0-rc.4",
"@aws-sdk/client-pinpoint": "1.0.0-rc.4",
"@aws-sdk/util-utf8-browser": "1.0.0-rc.3",
"@aws-sdk/client-firehose": "3.4.1",
"@aws-sdk/client-kinesis": "3.4.1",
"@aws-sdk/client-personalize-events": "3.4.1",
"@aws-sdk/client-pinpoint": "3.4.1",
"@aws-sdk/util-utf8-browser": "3.4.1",
"lodash": "^4.17.20",
"uuid": "^3.2.1"
},
Expand Down
2 changes: 1 addition & 1 deletion packages/analytics/src/Providers/AWSPinpointProvider.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ import {
getAmplifyUserAgent,
} from '@aws-amplify/core';
import {
EventsBatch,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to import from @aws-sdk/client-pinpoint/models since it is exported from the root: https://github.com/aws/aws-sdk-js-v3/blob/master/clients/client-pinpoint/index.ts#L115

PinpointClient,
PutEventsCommand,
PutEventsCommandInput,
UpdateEndpointCommand,
} from '@aws-sdk/client-pinpoint';
import { EventsBatch } from '@aws-sdk/client-pinpoint/models';
import Cache from '@aws-amplify/cache';

import {
Expand Down
8 changes: 4 additions & 4 deletions packages/core/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,10 @@
},
"dependencies": {
"@aws-crypto/sha256-js": "1.0.0-alpha.0",
"@aws-sdk/client-cognito-identity": "1.0.0-rc.4",
"@aws-sdk/credential-provider-cognito-identity": "1.0.0-rc.4",
"@aws-sdk/types": "1.0.0-rc.3",
"@aws-sdk/util-hex-encoding": "1.0.0-rc.3",
"@aws-sdk/client-cognito-identity": "3.4.1",
"@aws-sdk/credential-provider-cognito-identity": "3.4.1",
"@aws-sdk/types": "3.4.1",
"@aws-sdk/util-hex-encoding": "3.4.1",
"universal-cookie": "^4.0.4",
"zen-observable-ts": "0.8.19"
},
Expand Down
2 changes: 1 addition & 1 deletion packages/interactions/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
"homepage": "https://aws-amplify.github.io/",
"dependencies": {
"@aws-amplify/core": "3.8.13",
"@aws-sdk/client-lex-runtime-service": "1.0.0-rc.4"
"@aws-sdk/client-lex-runtime-service": "3.4.1"
},
"jest": {
"globals": {
Expand Down
14 changes: 7 additions & 7 deletions packages/predictions/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,13 @@
"dependencies": {
"@aws-amplify/core": "3.8.13",
"@aws-amplify/storage": "3.3.21",
"@aws-sdk/client-comprehend": "1.0.0-rc.4",
"@aws-sdk/client-polly": "1.0.0-rc.4",
"@aws-sdk/client-rekognition": "1.0.0-rc.4",
"@aws-sdk/client-textract": "1.0.0-rc.4",
"@aws-sdk/client-translate": "1.0.0-rc.4",
"@aws-sdk/eventstream-marshaller": "1.0.0-rc.3",
"@aws-sdk/util-utf8-node": "1.0.0-rc.3",
"@aws-sdk/client-comprehend": "3.4.1",
"@aws-sdk/client-polly": "3.4.1",
"@aws-sdk/client-rekognition": "3.4.1",
"@aws-sdk/client-textract": "3.4.1",
"@aws-sdk/client-translate": "3.4.1",
"@aws-sdk/eventstream-marshaller": "3.4.1",
"@aws-sdk/util-utf8-node": "3.4.1",
"uuid": "^3.2.1"
},
"jest": {
Expand Down
8 changes: 4 additions & 4 deletions packages/storage/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@
"homepage": "https://aws-amplify.github.io/",
"dependencies": {
"@aws-amplify/core": "3.8.13",
"@aws-sdk/client-s3": "1.0.0-rc.4",
"@aws-sdk/s3-request-presigner": "1.0.0-rc.4",
"@aws-sdk/util-create-request": "1.0.0-rc.4",
"@aws-sdk/util-format-url": "1.0.0-rc.4",
"@aws-sdk/client-s3": "3.4.1",
"@aws-sdk/s3-request-presigner": "3.4.1",
"@aws-sdk/util-create-request": "3.4.1",
"@aws-sdk/util-format-url": "3.4.1",
"axios": "0.21.1",
"events": "^3.1.0",
"sinon": "^7.5.0"
Expand Down
23 changes: 21 additions & 2 deletions packages/storage/src/providers/AWSS3ProviderManagedUpload.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ import {
} from '@aws-sdk/client-s3';
import { AxiosHttpHandler, SEND_PROGRESS_EVENT } from './axios-http-handler';
import * as events from 'events';
import { parseUrl } from '@aws-sdk/url-parser-node';
import { streamCollector } from '@aws-sdk/fetch-http-handler';

const logger = new Logger('AWSS3ProviderManagedUpload');
Expand Down Expand Up @@ -140,6 +139,27 @@ export class AWSS3ProviderManagedUpload {
this.params
);
const s3 = await this._createNewS3Client(this.opts);

// @aws-sdk/client-s3 seems to be ignoring the `ContentType` parameter, so we
// are explicitly adding it via middleware.
// https://github.com/aws/aws-sdk-js-v3/issues/2000
s3.middlewareStack.add(
next => (args: any) => {
if (
this.params.ContentType &&
args &&
args.request &&
args.request.headers
) {
args.request.headers['Content-Type'] = this.params.ContentType;
}
return next(args);
},
{
step: 'build',
}
);
Comment on lines +143 to +161
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment is pretty self explanatory, but there appears to be an issue with the SDK where it is removing ContentType when creating CreateMultipartUploadCommand, which causes multi-part uploads to fail. For now, we can get around this by using middleware, and we can revisit this in future SDK upgrades.

We now have multiple tests in CircleCI for React & React Native (iOS & Android) to validate multi-part uploads.


const response = await s3.send(createMultiPartUploadCommand);
logger.debug(response.UploadId);
return response.UploadId;
Expand Down Expand Up @@ -350,7 +370,6 @@ export class AWSS3ProviderManagedUpload {
...localTestingConfig,
requestHandler: new AxiosHttpHandler({}, emitter),
customUserAgent: getAmplifyUserAgent(),
urlParser: parseUrl,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per conversation with SDK team, there is no need to explicitly pass urlParser because S3Client already knows what parser needs to be used based on the environment. Based on initial testing, this change doesn't seem to have any impact on functionality.

This was removed because @aws-sdk/url-parser-node has a breaking change for React Native.

});
client.middlewareStack.remove(SET_CONTENT_LENGTH_HEADER);
return client;
Expand Down
4 changes: 2 additions & 2 deletions packages/storage/src/providers/axios-http-handler.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,14 @@ import { HttpHandler, HttpRequest, HttpResponse } from '@aws-sdk/protocol-http';
import { buildQueryString } from '@aws-sdk/querystring-builder';
import axios, { AxiosRequestConfig, Method } from 'axios';
import { ConsoleLogger as Logger } from '@aws-amplify/core';
import { BrowserHttpOptions } from '@aws-sdk/fetch-http-handler';
import { FetchHttpHandlerOptions } from '@aws-sdk/fetch-http-handler';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was renamed in rc.8: aws/aws-sdk-js-v3#1697


const logger = new Logger('axios-http-handler');
export const SEND_PROGRESS_EVENT = 'sendProgress';

export class AxiosHttpHandler implements HttpHandler {
constructor(
private readonly httpOptions: BrowserHttpOptions = {},
private readonly httpOptions: FetchHttpHandlerOptions = {},
private readonly emitter?: any
) {}

Expand Down