Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rfc43: fix identity cache partition #3566

Merged
merged 1 commit into from
Apr 10, 2024
Merged

rfc43: fix identity cache partition #3566

merged 1 commit into from
Apr 10, 2024

Conversation

aajtodd
Copy link
Contributor

@aajtodd aajtodd commented Apr 5, 2024

Motivation and Context

Original issue: #3427
RFC: #3544

Description

Fixes the SharedCredentialsProvider and SharedTokenProvider to re-use a consistent cache partition by default.
SdkConfig does not create an identity cache by default still, that will need to be a follow on PR that gates it behind a new behavior version. Ideally we do that with the stalled stream protection work in #3527

Testing

Added new unit and integration tests.

Checklist

  • I have updated CHANGELOG.next.toml if I made changes to the smithy-rs codegen or runtime crates

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@aajtodd aajtodd requested review from a team as code owners April 5, 2024 20:52
Copy link

github-actions bot commented Apr 5, 2024

A new generated diff is ready to view.

  • AWS SDK (ignoring whitespace)
  • No codegen difference in the Client Test
  • No codegen difference in the Server Test
  • No codegen difference in the Server Test Python
  • No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

Copy link
Contributor

@ysaito1001 ysaito1001 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work (including RFC)! The only feedback I have is to write tests as Kotlin integration tests.

use aws_smithy_runtime::client::http::test_util::infallible_client_fn;

#[tokio::test]
async fn test_identity_cache_reused_by_default() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an integration test is service agnostic, it's a good practice we write it in Kotlin. Since the test uses credentials_provider, you can use awsSdkIntegrationTest (example).

Copy link
Contributor Author

@aajtodd aajtodd Apr 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this end up on every generated client? If so is that what we want here?

It does use credentials_provider but with fake/test credentials. It really is just testing the cache behavior and interaction between SdkConfig and generated client config. It doesn't make a real request.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it does. awsSdkIntegrationTest is a test helper that tests codegen itself but does not run against generated AWS SDKs.

It really is just testing the cache behavior and interaction between SdkConfig and generated client config. It doesn't make a real request.

That does sound like service-agnostic, and that's what awsSdkIntegrationTest is for

Copy link
Contributor

@ysaito1001 ysaito1001 Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just throwing one last thing on the table before leaving tests in integration-tests/s3, can we directly construct SdkConfig in tests with necessary fields without going through aws_config::defaults? I'm assuming the tests won't need default configs in order to verify what they are supposed to test? If we don't use aws-config, then we may be able to avoid the problem we discussed offline.

But I am ok with what we have, and wanted to double-check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can which would allow us to test some of this but not how different behavior versions behave in loading (which still require a subsequent PR to fully implement) if I understand correctly how it's all wired.

Comment on lines 13 to 20
[[smithy-rs]]
message = """
Fixes the identity resolver types (`credentials_provider()` and `token_provider()`) from `SdkConfig` to have
a consistent identity cache partition when re-used across different clients.
"""
references = ["smithy-rs#3427"]
meta = { "breaking" = false, "tada" = false, "bug" = true }
authors = ["aajtodd"]
Copy link
Contributor

@ysaito1001 ysaito1001 Apr 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make this changelog entry [[aws-sdk-rust]] since credentials_provider and token_provider are AWS-related concepts, rather than general smithy concepts.

In addition, we can also create a changelog entry for [[smithy-rs]] mentioning that SharedIdentityResolver now respects a cache partition reserved by the passed-in IdentityResolver.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure I can update it as suggested but I want to take issue with this comment for a second:

since credentials_provider and token_provider are AWS-related concepts, rather than general smithy concepts.

From Smithy and SRA standpoint they are not AWS SDK specific. They are "AWS" concepts in that they originate from AWS but they are not "AWS SDK" only concepts. A customer could use sigv4 in their own model if they wanted (or internally within Amazon/AWS). The way Rust has this organized now is not entirely correct. Sigv4 and token provider support should be in the Smithy runtime (the specific providers like SsoTokenProvider or DefaultCredentialsChain are of course AWS SDK specific though).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From Smithy and SRA standpoint they are not AWS SDK specific

Good point. To be clear, what I meant was a tooling standpoint that's processing CHANGELOG.next.toml. When we write a changelog entry for [smithy-rs], that entry only appears in the release notes in the smithy-rs repository, but not in the aws-sdk-rust repository. So when those who only use Smithy SDK, but not AWS SDK, see a changelog entry mentioning credentials_provider and token_provider they don't find the methods because they are defined in the AWS runtime crates & generated by AWS codegen decorators. It can confuse customers.

The way Rust has this organized now is not entirely correct. Sigv4 and token provider support should be in the Smithy runtime

Interesting. Till we address it, I'd be consistent with the way it has been.

Copy link

github-actions bot commented Apr 8, 2024

A new generated diff is ready to view.

  • AWS SDK (ignoring whitespace)
  • No codegen difference in the Client Test
  • No codegen difference in the Server Test
  • No codegen difference in the Server Test Python
  • No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

@aajtodd aajtodd changed the title rfc41: fix identity cache partition rfc43: fix identity cache partition Apr 9, 2024
Copy link

github-actions bot commented Apr 9, 2024

A new generated diff is ready to view.

  • AWS SDK (ignoring whitespace)
  • No codegen difference in the Client Test
  • No codegen difference in the Server Test
  • No codegen difference in the Server Test Python
  • No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

@aajtodd aajtodd added this pull request to the merge queue Apr 10, 2024
Merged via the queue into main with commit 692cdfe Apr 10, 2024
44 checks passed
@aajtodd aajtodd deleted the atodd/fix-identity-cache branch April 10, 2024 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants