Allow IAM username suffix to be added to S3 scratch URL #69
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vinceatbluelabs
changed the title
vinceatbluelabs
commented
May 29, 2020
| @@ -1 +1 @@ | |||
| 981 | |||
| 982 | |||
There was a problem hiding this comment.
I added a comment to setup.py to explain a new version constraint.
vinceatbluelabs
commented
May 29, 2020
| @@ -1,7 +1,9 @@ | |||
| from db_facts.db_facts_types import DBFacts | |||
| from .database import db_facts_from_env | |||
| from typing import TYPE_CHECKING, Iterable, Union, Optional | |||
| from records_mover.mover_types import NotYetFetched | |||
| from records_mover.mover_types import PleaseInfer | |||
There was a problem hiding this comment.
I standardized on the latter here; the difference in semantics was so subtle as to be meaningless for the uses here.
vinceatbluelabs
added a commit
that referenced
this pull request
Jun 2, 2020
This will allow folks using custom cred types (like LastPass!) to configure this globally via config files. Combined with #69 this will allow enterprise uses to be configured while more generic settings are used by default.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR retires the scratch-s3-bucket script and introduces the ability to use an IAM username as part of the scratch bucket URL with appropriate config files in place. I'll work to get those set up so we can release this and have it work for our internal use.
I also moved the scratch bucket calculation over to BaseCreds, and renamed the functions to include the word 'default'. Your AWS IAM username depends on which AWS creds you are using. We always use the default AWS creds when loading/unloading on Redshift. That might not be true in the future, and once we provide more flexibility there, we'll want to extend this code to take in the appropriate credentials name for the AWS creds.