s3-url-service

Redirect bearer token requests with presigned S3 urls.

Overview

Would you like to serve private content from an AWS S3 Bucket but aren't satisfied with any of the often complex and confusing security/access/user/bucket policies? This service does not reverse proxy data through your server. It dynamically generates a presigned link and forwards the requester directly to S3 allowing the best of both worlds:

1. Full control over authorization (e.g. custom claims in JWT Bearer tokens etc.)
2. No wasting bandwidth by streaming content through your server

Build

docker build -t ubergarm/s3-url-service .

Runtime Configuration

Environment Variable	Description	Default
JWT_SECRET	The plain text HMAC-SHA256 symmetric secret key	secret
JWT_CREDENTIALS_REQUIRED	'true' or 'false' to enforce valid JWT credentials in request	false
EXPIRES	Link expiration and redirect cache duration (in seconds)	2592000 (30 days in seconds)
AWS_DEFAULT_REGION	AWS region	us-east-1
AWS_ACCESS_KEY_ID	AWS ID credentials	n/a
AWS_SECRET_ACCESS_KEY	AWS SECRET credentials	n/a
AWS_SSE_KMS_KEY_ID	AWS-KMS Key Id (if unset, SSE-KMS is not enabled)	n/a

Run

Export your AWS credentials as environment variables then:

docker run --rm \
            -it \
            -p 8080:8080 \
            -e JWT_SECRET=secret \
            -e JWT_CREDENTIALS_REQUIRED=false \
            -e EXPIRES=86400 \
            -e AWS_DEFAULT_REGION \
            -e AWS_ACCESS_KEY_ID \
            -e AWS_SECRET_ACCESS_KEY \
            -e AWS_SSE_KMS_KEY_ID \
            ubergarm/s3-url-service

Alternatively, you can stash your environment variables on disk by copying the env/dotenv template to env/.env and editing that file. Then run by mounting the env dir in your docker container:

docker run --rm \
            -it \
            -p 8080:8080 \
            -v ${PWD}/env:/app/env \
            ubergarm/s3-url-service

Optionally you can add -v $PWD:/app to test without rebuilding etc...

Test

Download content with credentials as Authorization header Bearer token:

#apt-get install -y httpie || brew install httpie
http --follow --print HBhb localhost:8080/s3_bucket_name/s3_key_value Authorization:"Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ"

Download content with credentials as URL argument token:

http --follow --print HBhb localhost:8080/s3_bucket_name/s3_key_value?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ

Download content with credentials as Cookie token:

http --follow --print HBhb localhost:8080/s3_bucket_name/s3_key_value "Cookie:token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ"

Upload content without any credentials:

#apt-get install -y curl || brew install curl
curl -v -L -T test.txt localhost:8080/s3_bucket_name/s3_key_value

AWS S3 Bucket Policy

There are many ways to get a set of credentials with permissions to access a given AWS S3 bucket. This is just one such possible example.

You can assign any existing AWS IAM user/role permissions to a 3rd party bucket. Change the principal to match your credentials and the resource to match the target bucket. The ListBucket action is optional. (Make sure your user/role has access to S3 from its attached IAM policy as well.)

{
	"Version": "2012-10-17",
	"Id": "RedirectServiceBucketPolicy",
	"Statement": [
		{
			"Sid": "GetPutListObjects",
			"Effect": "Allow",
			"Principal": {
				"AWS": "arn:aws:iam::012345678901:user/redirect-service"
			},
			"Action": [
				"s3:GetObject",
				"s3:PutObject",
				"s3:ListBucket"
			],
			"Resource": [
				"arn:aws:s3:::target_bucket_name",
				"arn:aws:s3:::target_bucket_name/*"
			]
		}
	]
}

NOTE this policy will allow uploading, but if the IAM is from a 3rd party account the permissions will be set at the object level, have a different owner, and in general not work like you might expect.

TODO

• Implement /:bucket/:key endpoint regular expression
• Implement aws-sdk presigned links
• Plumb in scaffolding for JWT Bearer tokens
• Test download
• Test upload
• Give example S3 Bucket Policy
• Pass in caching parameters as environment variables
• Cleanup how container starts
• Look more closely at http vs https support
• Find way to cleanup duplicated code
• Support multiple credentials/buckets secured with JWT claims (you can open a PR for this one! ;) )

References

• restify
• restify-jwt
• jwt.io
• aws-sdk
• AWS Authentication
• AWS CLI presign
• AWS SSE-KMS