Support AWS SSO #27

iainelder · 2021-02-25T14:08:56Z

I have this profile that uses AWS SSO (configured using the AWS CLI v2 aws configure sso command):

[profile example]
sso_start_url = https://d-1234567890.awsapps.com/start#/
sso_region = eu-west-1
sso_account_id = 123456789012
sso_role_name = AdministratorAccess
region = eu-west-1
output = json

I run any query using the profile:

$ ./bin/awsls --profiles example --regions eu-west-1 aws_network_interface

I get this error.

Error: failed to configure provider (name=aws, version=3.16.0): error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.

Please see https://registry.terraform.io/providers/hashicorp/aws
for more information about providing credentials.

Error: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

The Terraform aws provider version is 3.16.0. It doesn't support AWS SSO.

AWS SSO support was introduced in 3.26.0. See hashicorp/terraform-provider-aws#10851 (comment)

Preferrably use the latest version of the provider to avoid any initial bugs when AWS SSO support was introduced.

The text was updated successfully, but these errors were encountered:

jckuester · 2021-02-26T06:45:47Z

Hi @iainelder, I will upgrade the Terraform Provider and let you know once done. Thanks for filing the issue!

jckuester · 2021-03-06T15:33:08Z

Hi @iainelder again, I upgraded the Terraform AWS Provider in the new release: v0.9.0. Please let me know if SSO is working now for you. Thanks.

iainelder · 2021-03-06T17:10:06Z

I installed v0.9.0.

I see in the linked PR there a line that refers to v3.31.0 of the AWS provider.

awsls/gen/terraform/tags.go

Line 30 in c19a9da

[]aws.ClientKey{awsClientKey}, "3.31.0", "~/.awsls", 10*time.Second)

However, it appears that v3.16.0 is still being used. It still appears in the same error message.

$ awsls --version

version: 0.9.0
commit: 92b90fd
built at: 2021-03-06T14:44:44Z
using: go1.15.8

$ awsls --profiles example --regions eu-west-1 aws_network_interface


Error: failed to configure provider (name=aws, version=3.16.0): error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.

Please see https://registry.terraform.io/providers/hashicorp/aws
for more information about providing credentials.

Error: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

jckuester · 2021-03-06T18:05:37Z

sry for that, next try 🙂

The line you are referring to is part of the generator (based on which the list functions are generated - that's why there are +10 supported resources in the new release). I thought I had changed the provider version also in the main.go (https://github.com/jckuester/awsls/blob/master/main.go#L165), but somehow didn't. It should work now 🤞

v0.9.1 is on it's way... (https://github.com/jckuester/awsls/actions/runs/627770504)

iainelder · 2021-03-07T19:08:16Z

I installed v0.9.1 and tried again with AWS SSO.

This time it produces a weird error about failing to get the identity.

$ awsls --version

version: 0.9.1
commit: 6949109
built at: 2021-03-06T18:01:34Z
using: go1.15.8

$ awsls --profiles example --regions eu-west-1 aws_network_interface

Error aws_network_interface: failed to get caller identity: EC2RoleRequestError: no EC2 instance role found
caused by: request canceled, context deadline exceeded

For comparison here is what the AWS CLI v2 returns for the caller identity.

$ aws sts get-caller-identity --profile example
{
    "UserId": "AROA_EXAMPLE:iain",
    "Account": "0123456789012",
    "Arn": "arn:aws:sts::0123456789012:assumed-role/AWSReservedSSO_AdministratorAccess_d2c6e1a568becd2e/iain"
}

It looks the error originates from the client's SetAccountID function (just a guess from searching for the error message in the repo).

awsls/gen/aws/client.go

Lines 84 to 95 in 196ade0

    
           // SetAccountID populates the AccountID field of the client. 
        
           func (client *Client) SetAccountID() error { 
        
           	req := client.Stsconn.GetCallerIdentityRequest(&sts.GetCallerIdentityInput{}) 
        
           	resp, err := req.Send(context.Background()) 
        
           	if err != nil { 
        
           		return fmt.Errorf("failed to get caller identity: %s", err) 
        
           	} 
        
           	client.AccountID = *resp.Account 
        
           	return nil 
        
           }

jckuester · 2021-03-08T18:19:00Z

Ok, I think this makes sense. I assume that I also need to upgrade to a higher AWS SDK version (latest: https://github.com/aws/aws-sdk-go-v2/releases/tag/v1.2.0), which supports SSO. I will come back to you with a PR.

iainelder · 2021-03-13T09:15:47Z

According to go.mod, awsls is currently using aws-sdk-go v1.35.28.

awsls/go.mod

Line 7 in 6949109

github.com/aws/aws-sdk-go v1.35.28

It looks like you can stay on v1 and still use SSO.

From a comment in the provider issue, it looks like SSO support was introduced in v1.37.0.

hashicorp/terraform-provider-aws#10851 (comment)

aws/aws-sdk-go#3755 (AWS Single Sign-On (SSO) Provider Support)

Just mentioning it in case upgrading to v2 of the SDK would require other API changes to stay compatible.

jckuester · 2021-03-13T21:07:16Z

Unfortunately, I am using aws-sdk-go-v2 in version 0.24 for awsls (in line 8). The aws-sdk-go v1.35.28 dependency accidentally sneaked in by using in one place an unimportant helper method which exists also in v2 (will change this).

The change of using the v2 API in the first stable release is a bigger change and I have now a working PR: #29. Can you build the PR and check if it is working with SSO? If so, I will continue with some more refactoring and clean ups.

iainelder · 2021-03-14T04:56:23Z

Can you build the PR and check if it is working with SSO?

Sure, but I need your help!

How do you build it?

I don't see instructions in the README, so I've guessed.

I'm using my own script in a container running ubuntu-20.04 to install the prequisites and build awsls. It uses the make ci command I found in the Makefile.

https://github.com/iainelder/dotfiles/blob/fb694d0b0fea6963d48b5989e547f4a4cb0a685b/scripts/programs/awsls.source.bash

I've copied the complete output of make ci below.

There appear several errors about missing Terraform:

FatalError{Underlying: exec: "terraform": executable file not found in $PATH}

Is Terraform not compiled into the executable?

go generate
go build
go clean -testcache github.com/jckuester/awsls github.com/jckuester/awsls/aws github.com/jckuester/awsls/internal github.com/jckuester/awsls/resource github.com/jckuester/awsls/test
go test -v -p 1 -race -timeout 20m  github.com/jckuester/awsls github.com/jckuester/awsls/aws github.com/jckuester/awsls/internal github.com/jckuester/awsls/resource github.com/jckuester/awsls/test
?   	github.com/jckuester/awsls	[no test files]
?   	github.com/jckuester/awsls/aws	[no test files]
=== RUN   TestBuildVersionString
--- PASS: TestBuildVersionString (0.00s)
PASS
ok  	github.com/jckuester/awsls/internal	0.020s
=== RUN   TestIsType
=== RUN   TestIsType/existing_Terraform_resource_type
=== RUN   TestIsType/not_existing_Terraform_resource_type
--- PASS: TestIsType (0.00s)
    --- PASS: TestIsType/existing_Terraform_resource_type (0.00s)
    --- PASS: TestIsType/not_existing_Terraform_resource_type (0.00s)
=== RUN   TestIsSupportedType
=== RUN   TestIsSupportedType/supported_resource_type
=== RUN   TestIsSupportedType/not_supported_resource_type
--- PASS: TestIsSupportedType (0.00s)
    --- PASS: TestIsSupportedType/supported_resource_type (0.00s)
    --- PASS: TestIsSupportedType/not_supported_resource_type (0.00s)
=== RUN   TestMatchSupportedTypes
=== RUN   TestMatchSupportedTypes/no_match_found
=== RUN   TestMatchSupportedTypes/invalid_glob_pattern
=== RUN   TestMatchSupportedTypes/single_resource_matches,_no_wildcard
=== RUN   TestMatchSupportedTypes/glob_pattern_with_wildcard
--- PASS: TestMatchSupportedTypes (0.00s)
    --- PASS: TestMatchSupportedTypes/no_match_found (0.00s)
    --- PASS: TestMatchSupportedTypes/invalid_glob_pattern (0.00s)
    --- PASS: TestMatchSupportedTypes/single_resource_matches,_no_wildcard (0.00s)
    --- PASS: TestMatchSupportedTypes/glob_pattern_with_wildcard (0.00s)
=== RUN   TestSupportsTags
=== RUN   TestSupportsTags/resource_type_supports_tags
=== RUN   TestSupportsTags/resource_type_doesn't_support_tags
--- PASS: TestSupportsTags (0.00s)
    --- PASS: TestSupportsTags/resource_type_supports_tags (0.00s)
    --- PASS: TestSupportsTags/resource_type_doesn't_support_tags (0.00s)
PASS
ok  	github.com/jckuester/awsls/resource	0.064s
=== RUN   TestAcc_ProfilesAndRegions
    helper.go:58: env TEST_AWS_PROFILE1 not set, therefore using the following default value: myaccount1
    helper.go:58: env TEST_AWS_PROFILE2 not set, therefore using the following default value: myaccount2
    helper.go:58: env TEST_AWS_REGION1 not set, therefore using the following default value: us-west-2
    helper.go:58: env TEST_AWS_REGION2 not set, therefore using the following default value: us-east-1
TestAcc_ProfilesAndRegions 2021-03-14T04:22:06Z retry.go:72: terraform [init -upgrade=false -backend-config=bucket=awsls-testacc-tfstate-492043 -backend-config=key=awsls-testacc-mvkwzq.tfstate -backend-config=profile=myaccount1 -backend-config=region=us-west-2 -backend-config=encrypt=1 -no-color]
TestAcc_ProfilesAndRegions 2021-03-14T04:22:06Z command.go:87: Running command terraform with args [init -upgrade=false -backend-config=bucket=awsls-testacc-tfstate-492043 -backend-config=key=awsls-testacc-mvkwzq.tfstate -backend-config=profile=myaccount1 -backend-config=region=us-west-2 -backend-config=encrypt=1 -no-color]
TestAcc_ProfilesAndRegions 2021-03-14T04:22:06Z retry.go:80: Returning due to fatal error: FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
    apply.go:14: 
        	Error Trace:	apply.go:14
        	            				acc_test.go:40
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
        	Test:       	TestAcc_ProfilesAndRegions
TestAcc_ProfilesAndRegions 2021-03-14T04:22:06Z retry.go:72: terraform [destroy -auto-approve -input=false -lock=false -var profile1=myaccount1 -var profile2=myaccount2 -var region1=us-west-2 -var region2=us-east-1 -no-color]
TestAcc_ProfilesAndRegions 2021-03-14T04:22:06Z command.go:87: Running command terraform with args [destroy -auto-approve -input=false -lock=false -var profile1=myaccount1 -var profile2=myaccount2 -var region1=us-west-2 -var region2=us-east-1 -no-color]
TestAcc_ProfilesAndRegions 2021-03-14T04:22:06Z retry.go:80: Returning due to fatal error: FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
    destroy.go:12: 
        	Error Trace:	destroy.go:12
        	            				panic.go:636
        	            				testing.go:732
        	            				apply.go:14
        	            				acc_test.go:40
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
        	Test:       	TestAcc_ProfilesAndRegions
--- FAIL: TestAcc_ProfilesAndRegions (0.00s)
=== RUN   TestAcc_Attributes
    helper.go:58: env TEST_AWS_PROFILE1 not set, therefore using the following default value: myaccount1
    helper.go:58: env TEST_AWS_PROFILE2 not set, therefore using the following default value: myaccount2
    helper.go:58: env TEST_AWS_REGION1 not set, therefore using the following default value: us-west-2
    helper.go:58: env TEST_AWS_REGION2 not set, therefore using the following default value: us-east-1
TestAcc_Attributes 2021-03-14T04:22:06Z retry.go:72: terraform [init -upgrade=false -backend-config=bucket=awsls-testacc-tfstate-492043 -backend-config=key=awsls-testacc-uomy6c.tfstate -backend-config=profile=myaccount1 -backend-config=region=us-west-2 -backend-config=encrypt=1 -no-color]
TestAcc_Attributes 2021-03-14T04:22:06Z command.go:87: Running command terraform with args [init -upgrade=false -backend-config=bucket=awsls-testacc-tfstate-492043 -backend-config=key=awsls-testacc-uomy6c.tfstate -backend-config=profile=myaccount1 -backend-config=region=us-west-2 -backend-config=encrypt=1 -no-color]
TestAcc_Attributes 2021-03-14T04:22:06Z retry.go:80: Returning due to fatal error: FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
    apply.go:14: 
        	Error Trace:	apply.go:14
        	            				acc_test.go:171
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
        	Test:       	TestAcc_Attributes
TestAcc_Attributes 2021-03-14T04:22:06Z retry.go:72: terraform [destroy -auto-approve -input=false -lock=false -var profile=myaccount1 -var region=us-west-2 -var name=awsls-testacc-uomy6c -no-color]
TestAcc_Attributes 2021-03-14T04:22:06Z command.go:87: Running command terraform with args [destroy -auto-approve -input=false -lock=false -var profile=myaccount1 -var region=us-west-2 -var name=awsls-testacc-uomy6c -no-color]
TestAcc_Attributes 2021-03-14T04:22:06Z retry.go:80: Returning due to fatal error: FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
    destroy.go:12: 
        	Error Trace:	destroy.go:12
        	            				panic.go:636
        	            				testing.go:732
        	            				apply.go:14
        	            				acc_test.go:171
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
        	Test:       	TestAcc_Attributes
--- FAIL: TestAcc_Attributes (0.00s)
=== RUN   TestAcc_ResourceTypeGlobPattern
    helper.go:58: env TEST_AWS_PROFILE1 not set, therefore using the following default value: myaccount1
    helper.go:58: env TEST_AWS_PROFILE2 not set, therefore using the following default value: myaccount2
    helper.go:58: env TEST_AWS_REGION1 not set, therefore using the following default value: us-west-2
    helper.go:58: env TEST_AWS_REGION2 not set, therefore using the following default value: us-east-1
TestAcc_ResourceTypeGlobPattern 2021-03-14T04:22:06Z retry.go:72: terraform [init -upgrade=false -backend-config=bucket=awsls-testacc-tfstate-492043 -backend-config=key=awsls-testacc-ecuozn.tfstate -backend-config=profile=myaccount1 -backend-config=region=us-west-2 -backend-config=encrypt=1 -no-color]
TestAcc_ResourceTypeGlobPattern 2021-03-14T04:22:06Z command.go:87: Running command terraform with args [init -upgrade=false -backend-config=bucket=awsls-testacc-tfstate-492043 -backend-config=key=awsls-testacc-ecuozn.tfstate -backend-config=profile=myaccount1 -backend-config=region=us-west-2 -backend-config=encrypt=1 -no-color]
TestAcc_ResourceTypeGlobPattern 2021-03-14T04:22:06Z retry.go:80: Returning due to fatal error: FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
    apply.go:14: 
        	Error Trace:	apply.go:14
        	            				acc_test.go:270
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
        	Test:       	TestAcc_ResourceTypeGlobPattern
TestAcc_ResourceTypeGlobPattern 2021-03-14T04:22:06Z retry.go:72: terraform [destroy -auto-approve -input=false -lock=false -var name=awsls-testacc-ecuozn -var profile=myaccount1 -var region=us-west-2 -no-color]
TestAcc_ResourceTypeGlobPattern 2021-03-14T04:22:06Z command.go:87: Running command terraform with args [destroy -auto-approve -input=false -lock=false -var name=awsls-testacc-ecuozn -var profile=myaccount1 -var region=us-west-2 -no-color]
TestAcc_ResourceTypeGlobPattern 2021-03-14T04:22:06Z retry.go:80: Returning due to fatal error: FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
    destroy.go:12: 
        	Error Trace:	destroy.go:12
        	            				panic.go:636
        	            				testing.go:732
        	            				apply.go:14
        	            				acc_test.go:270
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: exec: "terraform": executable file not found in $PATH}
        	Test:       	TestAcc_ResourceTypeGlobPattern
--- FAIL: TestAcc_ResourceTypeGlobPattern (0.00s)
=== RUN   TestAcc_NonExistingResourceType

Error: no resource type found: aws_foo


--- PASS: TestAcc_NonExistingResourceType (20.72s)
=== RUN   TestAcc_UnsupportedResourceType

   • resource type not (yet) supported: aws_opsworks_mysql_layer
Error: no resource type found: aws_opsworks_mysql_layer


--- PASS: TestAcc_UnsupportedResourceType (17.35s)
=== RUN   TestAcc_Version

version: dev
commit: ?
built at: ?
using: go1.15.8


--- PASS: TestAcc_Version (17.55s)
FAIL
FAIL	github.com/jckuester/awsls/test	55.656s
FAIL
make: *** [Makefile:35: test-all] Error 1

jckuester · 2021-03-14T07:42:33Z

Sorry for not giving you more details. make ci runs integration tests where you need an AWS account for. Simply running go build is sufficient for you to create the binary in the working directory.

iainelder · 2021-03-14T10:19:39Z

Simply running go build is sufficient for you to create the binary in the working directory.

Okay! I'll try it when I get a moment.

That might be obvious to a gopher, but I'm new to the Go ecosystem. Thanks for your patience ☺️

iainelder · 2021-03-16T16:34:29Z

Yes, my build from branch #29 can use my SSO profile!

$ ./awsls --version

version: dev
commit: ?
built at: ?
using: go1.15.8

$ ./awsls --profiles example --regions eu-west-1 aws_network_interface

TYPE                    ID                      PROFILE   REGION      CREATED   
aws_network_interface   eni-01234567890123456   example   eu-west-1   N/A       
aws_network_interface   eni-01234567890123457   example   eu-west-1   N/A       
aws_network_interface   eni-01234567890123458   example   eu-west-1   N/A       
...

Thanks for your work on this.

When can we expect this to get a proper release?

jckuester · 2021-03-21T18:47:11Z

Hi @iainelder, sorry for my late response. I'll try to do my best and let you know; I haven't found much time yet to finish the needed refactoring properly.

iainelder · 2021-03-22T22:13:08Z

No problem. We have day jobs and lives away from the kyeboard too 😄

In the meantime I can use my dev build.

jckuester · 2021-04-11T11:57:27Z

Hi @iainelder, thanks for your patience here :) I am almost done and hope to make a release tonight for this PR (including also the JSON output feature and the bug you reported about redownloading the Terraform provider).

iainelder · 2021-04-12T21:12:27Z

Thanks for the effort on this! Looking forward to the new features :-)

jckuester · 2021-04-17T11:10:30Z

Finally released in v0.10.0. There was a failing test that needed to be fixed, which delayed the release again, sorry.

If you have the chance to spread the word about the new awsls release on Twitter or elsewhere, this would be appreciated 🙂

iainelder · 2021-04-19T14:50:37Z

@jckuester Thanks! I'll check as soon as I can and let you know.

awsls is a great tool for ad-hoc inventory analysis, and I recommend it to everyone I work with when it's appropriate to do so :-)

iainelder · 2021-04-19T18:10:09Z

@jckuester The SSO support is working great in v0.10.0. Thanks so much for the work!

jckuester mentioned this issue Mar 6, 2021

Upgrade to Terraform AWS Provider v3.31.0 #28

Merged

jckuester closed this as completed Apr 24, 2021

jckuester mentioned this issue Sep 4, 2021

Used aws sso for profile #41

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support AWS SSO #27

Support AWS SSO #27

iainelder commented Feb 25, 2021

jckuester commented Feb 26, 2021

jckuester commented Mar 6, 2021

iainelder commented Mar 6, 2021

jckuester commented Mar 6, 2021 •

edited

Loading

iainelder commented Mar 7, 2021 •

edited

Loading

jckuester commented Mar 8, 2021

iainelder commented Mar 13, 2021

jckuester commented Mar 13, 2021

iainelder commented Mar 14, 2021 •

edited

Loading

jckuester commented Mar 14, 2021

iainelder commented Mar 14, 2021

iainelder commented Mar 16, 2021

jckuester commented Mar 21, 2021 •

edited

Loading

iainelder commented Mar 22, 2021 •

edited

Loading

jckuester commented Apr 11, 2021

iainelder commented Apr 12, 2021

jckuester commented Apr 17, 2021

iainelder commented Apr 19, 2021

iainelder commented Apr 19, 2021

Support AWS SSO #27

Support AWS SSO #27

Comments

iainelder commented Feb 25, 2021

jckuester commented Feb 26, 2021

jckuester commented Mar 6, 2021

iainelder commented Mar 6, 2021

jckuester commented Mar 6, 2021 • edited Loading

iainelder commented Mar 7, 2021 • edited Loading

jckuester commented Mar 8, 2021

iainelder commented Mar 13, 2021

jckuester commented Mar 13, 2021

iainelder commented Mar 14, 2021 • edited Loading

jckuester commented Mar 14, 2021

iainelder commented Mar 14, 2021

iainelder commented Mar 16, 2021

jckuester commented Mar 21, 2021 • edited Loading

iainelder commented Mar 22, 2021 • edited Loading

jckuester commented Apr 11, 2021

iainelder commented Apr 12, 2021

jckuester commented Apr 17, 2021

iainelder commented Apr 19, 2021

iainelder commented Apr 19, 2021

jckuester commented Mar 6, 2021 •

edited

Loading

iainelder commented Mar 7, 2021 •

edited

Loading

iainelder commented Mar 14, 2021 •

edited

Loading

jckuester commented Mar 21, 2021 •

edited

Loading

iainelder commented Mar 22, 2021 •

edited

Loading