Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate API validation instead of using reflection #649

Merged
merged 5 commits into from
Apr 23, 2016

Conversation

jasdel
Copy link
Contributor

@jasdel jasdel commented Apr 22, 2016

Updates the SDK to generate its input parameter validation logic instead
of using reflection. This improves the performance (cpu/memory) of
building requests by about 15-20%.

This change should be transparent to users of the SDK, other than reduction of memory usage, and delay caused by param validation. This change does also refactor the SDK's internal API generator's templates to require go1.6. This will not impact usage of the SDK, just its code generation, which is covered by the private pkg pact.

Fix #537

Suggested to not review the diff as a whole at once since the generated services/protocol commit is pretty large.

This PR greatly improves the performance of validating API operation params. At the micro level the impact of validation now is extremely little. In the context of marshaling and signing a request this improves validation by about 15%-20%.

go test -bench . -benchmem -run NONE ./aws/corehandlers/

benchmark                  old ns/op     new ns/op     delta
BenchmarkValidateAny-4     203116        790           -99.61%

benchmark                  old allocs     new allocs     delta
BenchmarkValidateAny-4     1909           0              -100.00%

benchmark                  old bytes     new bytes     delta
BenchmarkValidateAny-4     19363         0             -100.00%

Benchmark results with a simpler operation, and additional isolated bench of validation by its self.

go test -bench . -run NONE -benchmem bench_test.go

benchmark                old ns/op     new ns/op     delta
BenchmarkOperation-4     77574         66292         -14.54%
BenchmarkValidate-4      12554         36.4          -99.71%

benchmark                old allocs     new allocs     delta
BenchmarkOperation-4     494            421            -14.78%
BenchmarkValidate-4      73             0              -100.00%

benchmark                old bytes     new bytes     delta
BenchmarkOperation-4     24571         24135         -1.77%
BenchmarkValidate-4      440           0             -100.00%

jasdel added 5 commits April 21, 2016 17:22
Removes the dependancies on the API generator from awstesting. This
ensures that the API generator does not leak into the SDK or the SDK
testing.
Updates the SDK to generate its input parameter validation logic instead
of using reflection. This improves the performance (cpu/memory) of
sending requests by about 20%.

Fix aws#537
@@ -60,5 +60,9 @@ func (a *API) Setup() {
a.removeUnusedShapes()
}

if !a.NoValidataShapeMethods {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is never set anywhere in the cli/gen-api folder. Should it be?

@xibz
Copy link
Contributor

xibz commented Apr 22, 2016

Awesome change, and it looks good. :shipit:

@jasdel jasdel merged commit be0a434 into aws:master Apr 23, 2016
@jasdel jasdel deleted the fix537GenValidation branch April 23, 2016 00:09
jasdel added a commit that referenced this pull request Apr 26, 2016
@jasdel jasdel mentioned this pull request Jun 17, 2016
skotambkar pushed a commit to skotambkar/aws-sdk-go that referenced this pull request May 20, 2021
Regenerates the SDK's protocol tests based on the fixes in smithy-go aws/smithy-go#119. To fix generation of protocol tests with streaming shapes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants