Use AWS ECS credentials #1283

johnjelinek · 2019-06-12T17:48:39Z

Is your feature request related to a problem? Please describe.

When running athens in AWS ECS (Fargate), the environment is provided credentials through a different avenue than the usual environment variables (i.e.: AWS_ACCESS_KEY_ID). I have a sidecar process go fetch the credentials and inject them into the config file:

for s in $(wget -qO- 169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI | jq -r \"to_entries|map(\\\"\\(.key)=\\(.value|tostring)\\\")|.[]\"); do
  export $s;
done && \
  sed -i'' -e 's|MY_AWS_ACCESS_KEY_ID|'\"$AccessKeyId\"'|' /config/config.toml && \
  sed -i'' -e 's|MY_AWS_SECRET_ACCESS_KEY|'\"$SecretAccessKey\"'|' /config/config.toml && \
  sed -i'' -e 's|MY_AWS_SESSION_TOKEN|'\"$Token\"'|' /config/config.toml

The problem is that these credentials expire and I don't have a way to have athens fetch the updated credentials in the running instance.

Describe the solution you'd like
I'd like athens to be able to fetch these credentials from the environment automatically.

Describe alternatives you've considered
Right now, I have a health check on the athens instances that fetch a package that is stored on S3. When that fails, the health check fails and the instance is destroyed and a new one is provisioned in its place with valid credentials.

The text was updated successfully, but these errors were encountered:

marpio · 2019-06-12T19:02:05Z

@johnjelinek
Just for my understanding - the problem is that currently Athens fetches the credentials only once (either from the config file or environment variables) while starting and there is no way to provide new credentials?

johnjelinek · 2019-06-12T19:28:51Z

Yes, I bootstrap it out of process and have no way to send updates although that URL is available during the lifetime of the container and is always available to get new values. The AWS SDK should have support to make fetching these credentials seamless if you wanted to provide another way for Athens to learn about its IAM task role.

…

On Wed, Jun 12, 2019, 2:02 PM marpio ***@***.***> wrote: @johnjelinek <https://github.com/johnjelinek> Just for my understanding - the problem is that currently Athens fetches the credentials only once (either from the config file or environment variables) while starting and there is no way to provide new credentials? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1283?email_source=notifications&email_token=AAGVJCX3RC6KOZHQOW2BXSTP2FB3DA5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXRPGOI#issuecomment-501412665>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGVJCUJOKCJC2VZGERT2E3P2FB3DANCNFSM4HXM6B5Q> .

marpio · 2019-06-12T19:57:56Z

Athens s3 config file contains the UseDefaultConfiguration option. If it's set to true then, instead of using credentials.NewStaticCredentials the defaults are being used.
Fom docs:

// The credentials object to use when signing requests. Defaults to a
// chain of credential providers to search for credentials in environment
// variables, shared credential file, and EC2 Instance Roles.

Would that be of any help to you?

johnjelinek · 2019-06-12T21:09:32Z

No, I first tried setting that to true and it didn't find the credentials. That's why I made that bootstrapping process to inject values into the config file.

…

On Wed, Jun 12, 2019, 2:58 PM marpio ***@***.***> wrote: Athens s3 config file contains the UseDefaultConfiguration option. If it's set to true then, instead of using credentials.NewStaticCredentials the defaults are being used. Fom docs: // The credentials object to use when signing requests. Defaults to a // chain of credential providers to search for credentials in environment // variables, shared credential file, and EC2 Instance Roles. Would that be of any help to you? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1283?email_source=notifications&email_token=AAGVJCTIMOA6S7PKOQ7N2ELP2FIMRA5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXRUAXQ#issuecomment-501432414>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGVJCVYIWATK3PC7CU46VLP2FIMRANCNFSM4HXM6B5Q> .

manugupt1 · 2019-06-12T21:29:23Z

Hi, What happens when you do not add credentials to the config file? If Athens does not start, I think we should fix this as we can get credentials from A) Profile (default / otherwise) B) Config C) Env (for security token) Cheers, Manu Gupta On Wed, Jun 12, 2019 at 2:09 PM John Jelinek IV <[email protected]> wrote:

…

No, I first tried setting that to true and it didn't find the credentials. That's why I made that bootstrapping process to inject values into the config file. On Wed, Jun 12, 2019, 2:58 PM marpio ***@***.***> wrote: > Athens s3 config file contains the UseDefaultConfiguration option. If > it's set to true then, instead of using credentials.NewStaticCredentials > the defaults are being used. > Fom docs: > > // The credentials object to use when signing requests. Defaults to a > // chain of credential providers to search for credentials in environment > // variables, shared credential file, and EC2 Instance Roles. > > Would that be of any help to you? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #1283?email_source=notifications&email_token=AAGVJCTIMOA6S7PKOQ7N2ELP2FIMRA5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXRUAXQ#issuecomment-501432414 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AAGVJCVYIWATK3PC7CU46VLP2FIMRANCNFSM4HXM6B5Q > > . > — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1283?email_source=notifications&email_token=AADAXDYECMP6YO2ILGF7AXDP2FQY3A5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXRZ4OA#issuecomment-501456440>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADAXD6WYFE7UMH645NX37TP2FQY3ANCNFSM4HXM6B5Q> .

johnjelinek · 2019-06-12T22:44:48Z

If I do not add credentials, athens starts and I can get the metrics endpoint, but all artifact requests respond with 500 because it can't get the credentials of the ECS task.

…

On Wed, Jun 12, 2019, 4:29 PM Manu Gupta ***@***.***> wrote: Hi, What happens when you do not add credentials to the config file? If Athens does not start, I think we should fix this as we can get credentials from A) Profile (default / otherwise) B) Config C) Env (for security token) Cheers, Manu Gupta On Wed, Jun 12, 2019 at 2:09 PM John Jelinek IV ***@***.***> wrote: > No, I first tried setting that to true and it didn't find the credentials. > That's why I made that bootstrapping process to inject values into the > config file. > > On Wed, Jun 12, 2019, 2:58 PM marpio ***@***.***> wrote: > > > Athens s3 config file contains the UseDefaultConfiguration option. If > > it's set to true then, instead of using credentials.NewStaticCredentials > > the defaults are being used. > > Fom docs: > > > > // The credentials object to use when signing requests. Defaults to a > > // chain of credential providers to search for credentials in environment > > // variables, shared credential file, and EC2 Instance Roles. > > > > Would that be of any help to you? > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > < > #1283?email_source=notifications&email_token=AAGVJCTIMOA6S7PKOQ7N2ELP2FIMRA5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXRUAXQ#issuecomment-501432414 > >, > > or mute the thread > > < > https://github.com/notifications/unsubscribe-auth/AAGVJCVYIWATK3PC7CU46VLP2FIMRANCNFSM4HXM6B5Q > > > > . > > > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > < #1283?email_source=notifications&email_token=AADAXDYECMP6YO2ILGF7AXDP2FQY3A5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXRZ4OA#issuecomment-501456440 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AADAXD6WYFE7UMH645NX37TP2FQY3ANCNFSM4HXM6B5Q > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1283?email_source=notifications&email_token=AAGVJCSVUQZQWJS45O4W6Y3P2FTDPA5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXR3KCQ#issuecomment-501462282>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGVJCTEBUCXRVE3OGQBMTTP2FTDPANCNFSM4HXM6B5Q> .

marpio · 2019-06-13T11:24:27Z

@johnjelinek since you're fetching the credentials from an endpoint currently - we could support https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/endpointcreds/

cc: @manugupt1

johnjelinek · 2019-06-13T11:41:17Z

That should work. The endpoint will always be the same in AWS ECS (Fargate).

…

On Thu, Jun 13, 2019, 6:24 AM marpio ***@***.***> wrote: @johnjelinek <https://github.com/johnjelinek> since you're fetching the credentials from an endpoint currently - we could support https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/endpointcreds/ cc: @manugupt1 <https://github.com/manugupt1> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1283?email_source=notifications&email_token=AAGVJCU2VL5BXM3U7AAIDDTP2IU7BA5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXTMFSA#issuecomment-501662408>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGVJCVROOVNX4WINMUDEVDP2IU7BANCNFSM4HXM6B5Q> .

manugupt1 · 2019-06-13T14:15:44Z

Yep! that should work. I would like to work on it. I should have a PR next week.

johnjelinek · 2019-06-13T23:18:53Z

I just had to add this kind of support to another project. This works:

if ECSCredentialsURI, exists := os.LookupEnv("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI"); exists {
	endpoint := fmt.Sprintf("169.254.170.2%s", ECSCredentialsURI)
	awsSession := session.New(aws.NewConfig().WithRegion("us-east-1").WithMaxRetries(3))
	awsCreds, err = creds.GetFromEndpoint(*awsSession.Config, awsSession.Handlers, endpoint)
	if err != nil {
		return nil, err
	}
}

johnjelinek · 2019-06-14T16:55:43Z

apparently session.New is deprecated ... maybe use session.NewSession() instead.

johnjelinek · 2019-06-24T15:17:03Z

@marpio: any updates?

marpio · 2019-06-24T18:52:02Z

@manugupt1 are you still on it?

manugupt1 · 2019-06-25T04:34:24Z

yep! sorry did not notice it got assigned to me. I looked into it a bit now.

It seems like we can end up using a combination of Static and Env credentials and if we get a 403, then we re-try by fetching new credentials rather than enforcing an HTTP handler.

This will enable users to inject new credentials in whatever way they see fit (HTTP / otherwise)

What do you think?

Ref: https://github.com/aws/aws-sdk-go/blob/master/aws/credentials/env_provider.go#L36
https://github.com/aws/aws-sdk-go/blob/master/aws/credentials/static_provider.go

marpio · 2019-06-25T05:29:43Z

@manugupt1
I was thinking about supporting the endpoint provider b/c it "knows" when the creds expire. That way we wouldn't need to wait for a 403.
Please see https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/endpointcreds/ and #1283 (comment) for an example.

What is your opinion @johnjelinek ?

johnjelinek · 2019-06-25T12:28:39Z

Precisely. It knows when it's expired and when you call Get it knows how to renew.

cudle · 2019-06-26T12:20:53Z

I am currently in the same situation as @johnjelinek. That feature would be highly appreciated.

manugupt1 · 2019-06-26T13:59:03Z

Yep! I have an idea on what to do. I am working on a PR soon and expect it in a couple of days.

manugupt1 · 2019-06-28T06:43:40Z

FYI: Ran into some issues. Here is a draft PR: #1291

Please start putting in comments so that I can address as I make changes to it.

marpio · 2019-06-28T12:12:25Z

@manugupt1 What kind of issues?

manugupt1 · 2019-06-30T11:10:58Z

@marpio I have added a PR that is ready for review. https://github.com/gomods/athens/pull/1291/files

Do you think we should also add a unit test that spins up a server and retrieves creds from that server and see if the test succeeds?

johnjelinek · 2019-07-16T18:38:27Z

any new updates on this @marpio?

manugupt1 · 2019-07-16T19:25:47Z

Hi I have a patch PR, I was travelling and am on-call, so I will try to get to it asap. Cheers, Manu Gupta

…

On Tue, Jul 16, 2019 at 11:38 AM John Jelinek IV ***@***.***> wrote: any new updates on this @marpio <https://github.com/marpio>? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1283?email_source=notifications&email_token=AADAXD43ZRVMOJEDJPZGYX3P7YISJA5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2BYNHA#issuecomment-511936156>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADAXD7L2H5MK3VZWRUN6D3P7YISJANCNFSM4HXM6B5Q> .

johnjelinek · 2019-07-22T22:36:59Z

@marpio / @manugupt1: I'm having difficulty getting this to work. Here's my config change:

CredentialsEndpoint = "http://169.254.170.2/v2/credentials/0357419a-4eea-4eef-8510-16c5c451ba33"

but I'm getting an exception:


  2019-07-22 22:33:05.413265 I | Starting application at port :3000
  panic: runtime error: invalid memory address or nil pointer dereference
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x68aa42]
  
  goroutine 20 [running]:
  net/http.(*Client).deadline(0x0, 0xc00000e3a0, 0x0, 0x40bad9)
    /usr/local/go/src/net/http/client.go:187 +0x22
  net/http.(*Client).do(0x0, 0xc0003c8f00, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/client.go:527 +0xab
  net/http.(*Client).Do(...)
    /usr/local/go/src/net/http/client.go:509
  github.com/aws/aws-sdk-go/aws/corehandlers.sendFollowRedirects(0xc000448c00, 0x154cc80, 0xc0003c8e00, 0xc000448c00)
    /go/pkg/mod/github.com/aws/[email protected]/aws/corehandlers/handlers.go:120 +0x3c
  github.com/aws/aws-sdk-go/aws/corehandlers.glob..func3(0xc000448c00)
    /go/pkg/mod/github.com/aws/[email protected]/aws/corehandlers/handlers.go:112 +0x7f
  github.com/aws/aws-sdk-go/aws/request.(*HandlerList).Run(0xc000448da8, 0xc000448c00)
    /go/pkg/mod/github.com/aws/[email protected]/aws/request/handlers.go:213 +0x98
  github.com/aws/aws-sdk-go/aws/request.(*Request).Send(0xc000448c00, 0x0, 0x0)
    /go/pkg/mod/github.com/aws/[email protected]/aws/request/request.go:493 +0x19e
  github.com/aws/aws-sdk-go/aws/credentials/endpointcreds.(*Provider).getCredentials(0xc0004c85c0, 0x1339a04, 0x6, 0x1886101)
    /go/pkg/mod/github.com/aws/[email protected]/aws/credentials/endpointcreds/provider.go:156 +0x2ef
  github.com/aws/aws-sdk-go/aws/credentials/endpointcreds.(*Provider).Retrieve(0xc0004c85c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7f56660b0008, ...)
    /go/pkg/mod/github.com/aws/[email protected]/aws/credentials/endpointcreds/provider.go:114 +0x63
  github.com/aws/aws-sdk-go/aws/credentials.(*ChainProvider).Retrieve(0xc0004ad020, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/pkg/mod/github.com/aws/[email protected]/aws/credentials/chain_provider.go:77 +0xd6
  github.com/aws/aws-sdk-go/aws/credentials.(*Credentials).Get(0xc0004b9b90, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/pkg/mod/github.com/aws/[email protected]/aws/credentials/credentials.go:221 +0x157
  github.com/aws/aws-sdk-go/aws/signer/v4.Signer.signWithBody(0xc0004b9b90, 0x0, 0x1836ee0, 0xc00039e6c0, 0x10100, 0x154ff38, 0x0, 0xc0003c8d00, 0x1847100, 0xc00047a2c0, ...)
    /go/pkg/mod/github.com/aws/[email protected]/aws/signer/v4/v4.go:339 +0x28e
  github.com/aws/aws-sdk-go/aws/signer/v4.signSDKRequestWithCurrTime(0xc000448800, 0x154ff38, 0x0, 0x0, 0x0)
    /go/pkg/mod/github.com/aws/[email protected]/aws/signer/v4/v4.go:478 +0x300
  github.com/aws/aws-sdk-go/aws/signer/v4.SignSDKRequest(0xc000448800)
    /go/pkg/mod/github.com/aws/[email protected]/aws/signer/v4/v4.go:424 +0x48
  github.com/aws/aws-sdk-go/aws/request.(*HandlerList).Run(0xc000448988, 0xc000448800)
    /go/pkg/mod/github.com/aws/[email protected]/aws/request/handlers.go:213 +0x98
  github.com/aws/aws-sdk-go/aws/request.(*Request).Sign(0xc000448800, 0x14154ddd7e, 0x23dcb20)
    /go/pkg/mod/github.com/aws/[email protected]/aws/request/request.go:384 +0xb0
  github.com/aws/aws-sdk-go/aws/request.(*Request).Send(0xc000448800, 0x0, 0x0)
    /go/pkg/mod/github.com/aws/[email protected]/aws/request/request.go:486 +0x152
  github.com/aws/aws-sdk-go/service/s3.(*S3).ListObjectsWithContext(0xc00039e6d0, 0x7f5663e4d840, 0xc0003557d0, 0xc0003a6380, 0x0, 0x0, 0x0, 0x185ed80, 0xc0003557d0, 0xc00007ecc0)
    /go/pkg/mod/github.com/aws/[email protected]/service/s3/api.go:4012 +0x199
  github.com/gomods/athens/pkg/storage/s3.(*Storage).Exists(0xc0004c8640, 0x185ed80, 0xc0003557a0, 0xc0003de9c0, 0x24, 0xc0003de9f0, 0x22, 0x185ed00, 0x0, 0x0)
    /go/src/github.com/gomods/athens/pkg/storage/s3/checker.go:29 +0x31e
  github.com/gomods/athens/pkg/storage/s3.(*Storage).Info(0xc0004c8640, 0x185ed80, 0xc000355770, 0xc0003de9c0, 0x24, 0xc0003de9f0, 0x22, 0x0, 0x0, 0x0, ...)
    /go/src/github.com/gomods/athens/pkg/storage/s3/getter.go:21 +0x128
  github.com/gomods/athens/pkg/download.(*protocol).Info(0xc0004c8880, 0x185ed80, 0xc0003556e0, 0xc0003de9c0, 0x24, 0xc0003de9f0, 0x22, 0x0, 0x0, 0x0, ...)
    /go/src/github.com/gomods/athens/pkg/download/protocol.go:159 +0x131
  github.com/gomods/athens/pkg/download/addons.(*withpool).Info.func1()
    /go/src/github.com/gomods/athens/pkg/download/addons/with_pool.go:73 +0x8a
  github.com/gomods/athens/pkg/download/addons.(*withpool).listen(0xc0004d2320)
    /go/src/github.com/gomods/athens/pkg/download/addons/with_pool.go:46 +0x49
  created by github.com/gomods/athens/pkg/download/addons.(*withpool).start

johnjelinek · 2019-07-22T22:48:53Z

One of the tricky things about running this in AWS ECS Fargate is that the URL changes every time. I have to do something like this in a sidecar sed -i'' -e 's|MY_AWS_CREDENTIALS_ENDPOINT|http://169.254.170.2'\"$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI\"'|' /config/config.toml to populate the config file before starting athens. I'm not sure if you know of a way to get env vars to interpolate when the value of an env var itself contains an env var, i.e.: AWS_CREDENTIALS_ENDPOINT=169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI.

marpio · 2019-08-05T20:43:28Z

@arschles #1291 didn't address this issue (we're getting a panic). I think we can leave it open until we fix the current implementation.

manugupt1 · 2019-08-06T03:28:49Z

@johnjelinek I have been able to get a 403 on this with endpoint creds. Can you verify if it works with real ones: #1325

johnjelinek · 2019-08-06T13:00:59Z

Cool! I think that looks good. Can you push this to the canary container so I can pull it in Fargate?

marpio · 2019-08-06T19:23:08Z

@johnjelinek canary should be pushed on every master build so you should be able to test it.

johnjelinek · 2019-08-08T18:11:46Z

@marpio: works great! The only other part that would make this nicer is if it could figure out the endpoint in process instead of me having to use a sidecar to generate the URL to pass into the config.

marpio · 2019-08-08T20:07:58Z

@johnjelinek glad to hear that!
Would using an environment variable be an option for you?

athens/pkg/config/s3.go

Line 11 in a336c55

CredentialsEndpoint string `envconfig:"AWS_CREDENTIALS_ENDPOINT"`

johnjelinek · 2019-08-08T20:10:53Z

@marpio: no, the env var would have to be set a deploy-time, but AWS generates it dynamically, so I generate the URL at run-time in a sidecar, modifying the config file in a shared volume, and then I start the athens container -- however, the URL is always discoverable at: http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI, but I can't pass in a reference to another env var to AWS_CREDENTIALS_ENDPOINT.

Do you think it makes sense to add support for resolving other env vars so I could set it like this:

AWS_CREDENTIALS_ENDPOINT=http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI

Or do you think it makes more sense to embed this endpoint as a default URL, since it's always available here when running in AWS Fargate (and then I don't have to pass in an endpoint at all)?

manugupt1 · 2019-08-11T15:54:02Z

@johnjelinek Can you be a bit more clear?

Is it a new URI that it generates or do the tokens expire after a while?

johnjelinek · 2019-08-12T17:35:02Z

A service, in Fargate, is an essential container and a collection of sidecars that run to provide functionality. Every time a service starts in Fargate, the environment variable AWS_CONTAINER_CREDENTIALS_RELATIVE_URI changes. The environment variable is the same for all containers of the service. So, right now, I have a sidecar that fetches the value of AWS_CONTAINER_CREDENTIALS_RELATIVE_URI and appends it to http://169.254.170.2 and then I take all that and modify the config file, so when the essential athens container starts, it's already configured with where to fetch credentials from. That provided endpoint already has everything needed to refresh credentials as long as the Fargate service does not end. The URL will never change for that instance of the service.

To simplify, if athens had a default to make AWS_CREDENTIALS_ENDPOINT point to http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI, then I could remove my sidecar container from boot and have the same functionality. Hopefully, this explanation makes more sense. Let me know if you have further questions @manugupt1.

manugupt1 · 2019-08-13T05:53:45Z

@johnjelinek to keep track I am reopening the issue.

Can you look at this PR and see if it makes sense: https://github.com/gomods/athens/pull/1337/files

johnjelinek · 2019-08-13T12:53:10Z

I'm not sure this makes sense. It doesn't look like it's linking to anything from the environment. I think what you're trying to implement is an environment variable alias. The $AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is already an environment variable, it's not part of the URL. It might make more sense to see if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI exists as an env var and if so, evaluate AWS_CREDENTIALS_ENDPOINT to whatever that evaluates to instead of requiring and config input.

manugupt1 · 2019-08-13T15:07:04Z

We use envconfig that should help override env vars. https://github.com/kelseyhightower/envconfig/blob/master/README.md That should work! Is there an easy way for you to test this branch? If not, let’s figure something out Cheers Manu

…

________________________________ From: John Jelinek IV <[email protected]> Sent: Tuesday, August 13, 2019 5:53 AM To: gomods/athens Cc: Manu Gupta; State change Subject: Re: [gomods/athens] Use AWS ECS credentials (#1283) I'm not sure this makes sense. It doesn't look like it's linking to anything from the environment. I think what you're trying to implement is an environment variable alias. The $AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is already an environment variable, it's not part of the URL. It might make more sense to see if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI exists as an env var and if so, evaluate AWS_CREDENTIALS_ENDPOINT to whatever that evaluates to instead of requiring and config input. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub<#1283?email_source=notifications&email_token=AADAXD6S6RQQTOSX52GIV4TQEKVDPA5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4FRZYA#issuecomment-520821984>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AADAXD7FZDTSJLHO7HVT4PTQEKVDPANCNFSM4HXM6B5Q>.

johnjelinek · 2019-08-13T15:46:08Z

@manugupt1: If you could push your latest changes to the canary container release, I can pull it in and try it out ... I suspect it won't work, but let's find out!

johnjelinek · 2019-08-13T15:53:44Z

@manugupt1: I left comments on your PR

manugupt1 · 2019-08-14T03:13:37Z

@johnjelinek It should work as I tested out printing out exported env vars. I will open the PR for review and see if everyone is okay to merge it in.

johnjelinek · 2019-08-14T16:51:25Z

cool, when it gets to the canary container, I'll try it out

manugupt1 · 2019-08-21T13:37:08Z

It should be on canary now can you test it John! Thanks Cheers Manu

…

________________________________ From: John Jelinek IV <[email protected]> Sent: Wednesday, August 14, 2019 9:51 AM To: gomods/athens Cc: Manu Gupta; Mention Subject: Re: [gomods/athens] Use AWS ECS credentials (#1283) cool, when it gets to the canary container, I'll try it out — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1283?email_source=notifications&email_token=AADAXDZPYK3EDGY7DLGGYS3QEQZY5A5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4JNGSI#issuecomment-521327433>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AADAXD64J43K44CQMZBU4N3QEQZY5ANCNFSM4HXM6B5Q>.

johnjelinek · 2019-08-21T16:46:37Z

Excellent, I should be able to test it tomorrow

johnjelinek · 2019-08-22T16:32:39Z

@manugupt1: canary works great. I removed all my sidecars and it still works with this env var:

  {
    "name": "AWS_CREDENTIALS_ENDPOINT",
    "value": "http://169.254.170.2"
  }

I think can be closed now. It might be nice to document somewhere that http://169.254.170.2 is the default Fargate endpoint for everyone.

manugupt1 · 2019-08-22T16:49:12Z

Is it a global endpoint or is it region specific? Cheers Manu

…

________________________________ From: John Jelinek IV <[email protected]> Sent: Thursday, August 22, 2019 9:32 AM To: gomods/athens Cc: Manu Gupta; Mention Subject: Re: [gomods/athens] Use AWS ECS credentials (#1283) @manugupt1<https://github.com/manugupt1>: canary works great. I removed all my sidecars and it still works with this env var: { "name": "AWS_CREDENTIALS_ENDPOINT", "value": "http://169.254.170.2" } I think can be closed now. It might be nice to document somewhere that http://169.254.170.2 is the default Fargate endpoint for everyone. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1283?email_source=notifications&email_token=AADAXD2TMQOEZOCQNTCNWHTQF25SRA5CNFSM4HXM6B52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD45U46I#issuecomment-523980409>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AADAXD35QFHU6SWC4JR6TZTQF25SRANCNFSM4HXM6B5Q>.

johnjelinek · 2019-08-22T17:52:50Z

global

manugupt1 · 2019-08-23T14:15:57Z

Makes sense! I looked at the docs. It does not get there specifically. However, I created two issues for these:

#1349
#1350

Let's track them there!

thanks!

venkz · 2020-07-31T02:57:41Z

@johnjelinek : I want to use the aws credentials in AWS Fargate. I am using a golang docker container to read some AWS SSM Params and need credentials. Can you please help me with this snippet..

sess := session.Must(session.NewSessionWithOptions(session.Options{
		SharedConfigState: session.SharedConfigEnable,
	}))
	svc := ssm.New(sess)

I am unable to access any ssm secrets.

marpio added the question Further information is requested label Jun 12, 2019

marpio assigned manugupt1 Jun 13, 2019

marpio added the enhancement New feature or request label Jun 13, 2019

manugupt1 mentioned this issue Jun 30, 2019

endpoint creds #1291

Merged

marpio closed this as completed in #1291 Jul 22, 2019

marpio reopened this Aug 5, 2019

manugupt1 mentioned this issue Aug 6, 2019

Fixes endpoint creds #1325

Merged

marwan-at-work closed this as completed in #1325 Aug 6, 2019

manugupt1 reopened this Aug 13, 2019

arschles mentioned this issue Aug 15, 2019

Add relative URI for AWS fargate. #1337

Merged

manugupt1 closed this as completed Aug 23, 2019

arschles mentioned this issue Aug 28, 2019

Add docs to launch an instance of Athens on AWS Fargate #1350

Open

Use AWS ECS credentials #1283

Use AWS ECS credentials #1283

Comments

johnjelinek commented Jun 12, 2019

marpio commented Jun 12, 2019

johnjelinek commented Jun 12, 2019 via email

marpio commented Jun 12, 2019

johnjelinek commented Jun 12, 2019 via email

manugupt1 commented Jun 12, 2019 via email

johnjelinek commented Jun 12, 2019 via email

marpio commented Jun 13, 2019

johnjelinek commented Jun 13, 2019 via email

manugupt1 commented Jun 13, 2019

johnjelinek commented Jun 13, 2019

johnjelinek commented Jun 14, 2019

johnjelinek commented Jun 24, 2019

marpio commented Jun 24, 2019

manugupt1 commented Jun 25, 2019

marpio commented Jun 25, 2019 • edited Loading

johnjelinek commented Jun 25, 2019

cudle commented Jun 26, 2019

manugupt1 commented Jun 26, 2019

manugupt1 commented Jun 28, 2019

marpio commented Jun 28, 2019

manugupt1 commented Jun 30, 2019

johnjelinek commented Jul 16, 2019

manugupt1 commented Jul 16, 2019 via email

johnjelinek commented Jul 22, 2019

johnjelinek commented Jul 22, 2019

marpio commented Aug 5, 2019

manugupt1 commented Aug 6, 2019

johnjelinek commented Aug 6, 2019

marpio commented Aug 6, 2019

johnjelinek commented Aug 8, 2019

marpio commented Aug 8, 2019

johnjelinek commented Aug 8, 2019 • edited Loading

manugupt1 commented Aug 11, 2019

johnjelinek commented Aug 12, 2019

manugupt1 commented Aug 13, 2019

johnjelinek commented Aug 13, 2019

manugupt1 commented Aug 13, 2019 via email

johnjelinek commented Aug 13, 2019

johnjelinek commented Aug 13, 2019

manugupt1 commented Aug 14, 2019

johnjelinek commented Aug 14, 2019

manugupt1 commented Aug 21, 2019 via email

johnjelinek commented Aug 21, 2019

johnjelinek commented Aug 22, 2019 • edited Loading

manugupt1 commented Aug 22, 2019 via email

johnjelinek commented Aug 22, 2019

manugupt1 commented Aug 23, 2019

venkz commented Jul 31, 2020 • edited Loading

marpio commented Jun 25, 2019 •

edited

Loading

johnjelinek commented Aug 8, 2019 •

edited

Loading

johnjelinek commented Aug 22, 2019 •

edited

Loading

venkz commented Jul 31, 2020 •

edited

Loading