Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor secrets cache to sdk client #278

Merged
merged 2 commits into from
Aug 29, 2023

Conversation

lewijacn
Copy link
Collaborator

@lewijacn lewijacn commented Aug 21, 2023

Description

This reworks the SecretCache to be a SecretsManager async client that retrieves the secret. This will be helpful in the future as needed with integrating with Netty, for now it has a synchronous method for constructing an auth header from a secret.

Issues Resolved

None

Testing

Unit tests

Check List

  • New functionality includes testing
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@codecov
Copy link

codecov bot commented Aug 22, 2023

Codecov Report

Merging #278 (8c6694a) into main (b7c3da0) will increase coverage by 0.12%.
Report is 20 commits behind head on main.
The diff coverage is 69.71%.

@@             Coverage Diff              @@
##               main     #278      +/-   ##
============================================
+ Coverage     61.93%   62.05%   +0.12%     
  Complexity      617      617              
============================================
  Files            82       82              
  Lines          3129     3139      +10     
  Branches        292      292              
============================================
+ Hits           1938     1948      +10     
  Misses         1014     1014              
  Partials        177      177              
Flag Coverage Δ
unittests 62.05% <69.71%> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...n/java/org/opensearch/migrations/replay/Utils.java 40.00% <ø> (ø)
.../datahandlers/http/NettyJsonContentCompressor.java 88.37% <ø> (ø)
...ions/transform/RemovingAuthTransformerFactory.java 0.00% <0.00%> (ø)
.../opensearch/migrations/replay/TrafficReplayer.java 6.75% <13.51%> (-0.42%) ⬇️
...nsearch/migrations/transform/IAuthTransformer.java 28.57% <28.57%> (ø)
...g/opensearch/migrations/replay/AWSAuthService.java 66.66% <71.42%> (+3.03%) ⬆️
...replay/PacketToTransformingHttpHandlerFactory.java 63.63% <75.00%> (ø)
.../org/opensearch/migrations/replay/SigV4Signer.java 77.58% <77.58%> (ø)
.../datahandlers/http/NettyJsonContentAuthSigner.java 86.36% <86.36%> (ø)
...datahandlers/http/RequestPipelineOrchestrator.java 96.36% <92.30%> (ø)
... and 17 more

... and 2 files with indirect coverage changes

public String getBasicAuthHeaderFromSecret(String username, String secretId) {
String authHeaderString = username + ":" + getSecret(secretId);
public String getBasicAuthHeaderFromSecret(String username, String secretId) throws ExecutionException, InterruptedException {
String secretValue = getSecret(secretId).get().secretString();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like having the synchronous method alongside the async one and then being opinionated on exceptions that can come out of the synchronous method that would only happen when dealing with something asynchronous.

You could also return a CompletableFuture by applying the getSecret Output through a finalizer that does everything else. That let's the caller determine how to handle the exceptions and keeps these methods more consistent.

Comment on lines 258 to 259
private static IAuthTransformerFactory buildAuthTransformerFactory(Parameters params)
throws ExecutionException, InterruptedException {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't imagine a case where I'd need to make these factories asynchronously. I could see needing to create the AuthTransformers themselves asynchronously, or signing the requests - but this seems like it would always be part of bootstrapping at the top level.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I am inclined to agree with you. Initially when I set this up I thought this might be used in the Netty pipeline so having async would be useful, but for our current use case I really only see synchronous and supporting the off case where someone's secrets has changed in the middle of a migration doesn't seem immediate. I've refactored to use a synchronous client now

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also included the fix for the failing test case since I don't plan to merge this to the release branch

@lewijacn lewijacn merged commit 7eb5371 into opensearch-project:main Aug 29, 2023
gregschohn added a commit to gregschohn/opensearch-migrations that referenced this pull request Sep 5, 2023
* main:
  [Fetch Migration] Monitoring script for Data Prepper (opensearch-project#264)
  Refactor secrets cache to sdk client (opensearch-project#278)
  Update humanReadableLogs script
  Update primary->source and shadow->target and documentation
  Add shadowRequest and connectionId to tuple
  Update tests for CRLF
  Use CRLF in NettyJsonToByteBufHandler

Signed-off-by: Greg Schohn <[email protected]>

# Conflicts:
#	TrafficCapture/trafficCaptureProxyServer/src/test/java/org/opensearch/migrations/trafficcapture/proxyserver/netty/ExpiringSubstitutableItemPoolTest.java
#	TrafficCapture/trafficReplayer/build.gradle
#	TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/SourceTargetCaptureTuple.java
#	TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/TrafficReplayer.java
@lewijacn lewijacn deleted the refactor-secrets-cache branch March 29, 2024 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants