Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oci-proxy: Add AWS S3 bucket for registry-sandbox.k8s.io #3671

Merged

Conversation

ameukam
Copy link
Member

@ameukam ameukam commented Apr 25, 2022

Related to:

Ensure a AWS S3 bucket exists so we can test IP based redirection of
archeio.

The bucket contains a copy of the images layers
served by k8s.gcr.io.

The bucket:

  • is world readable
  • only allow HTTPS connections
  • only allow HTTP methods GET and HEAD
  • has versioning enabled

Another private bucket is created for access logging.

Signed-off-by: Arnaud Meukam [email protected]

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 25, 2022
@k8s-ci-robot k8s-ci-robot requested review from spiffxp and thockin April 25, 2022 05:07
@k8s-ci-robot k8s-ci-robot added sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Apr 25, 2022
@ameukam
Copy link
Member Author

ameukam commented Apr 25, 2022

@endocrimes @sftim still WIP but looking for early review. 🙇🏾

@ameukam ameukam force-pushed the aws-account-registry-k8s-io-sandbox branch from a86fcfa to 2b5ff55 Compare April 25, 2022 16:54
Copy link
Contributor

@sftim sftim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I added some feedback, mainly to suggest not using ACLs for the logs bucket.

Comment on lines 29 to 84
resource "aws_s3_bucket_acl" "access_log" {
provider = aws.origin

bucket = aws_s3_bucket.access_log.id
acl = "log-delivery-write"
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than using this ACL, I recommend using a bucket policy combined with a bucket that is set to bucket owner enforced—in Terraform, you'd use aws_s3_bucket_ownership_controls, similar to what's done for the serving bucket.

Look under https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-ownership-migrating-acls-prerequisites.html#object-ownership-server-access-logs for a heading To migrate bucket ACL permissions for the S3 log delivery group to the logging service principal in a bucket policy and use a policy like that, with a suitable aws:SourceArn condition. That'll let S3 logs write to the logs bucket.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


# Objects are deleted after 90 days
expiration {
days = 90
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd put the number of days into a local

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

*/

terraform {
required_version = "~> 1.1.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
required_version = "~> 1.1.0"
required_version = "~> 1.1.2"

Early patch releases in this series have a bug - details in https://github.com/hashicorp/terraform/releases/tag/v1.1.2
I would pin to a version that has that bug fixed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use tfswitch locally. it will detect the version and automatically pick the latest version. I'm not sure we want to pick a specific version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do recommend this pin. We can't be sure that another contributor is also using the same tfswitch setup.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


provider "aws" {
profile = "default"
region = "us-west-2"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this region? It might be worth mentioning that in a comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not specific reason. Since this is a sandbox environment supposed to be gone in the future I randomly picked one.

@sftim
Copy link
Contributor

sftim commented Apr 26, 2022

BTW:, if it's useful to have, we can generate events (EventBridge) on:

  • bucket policy changes
  • object writes (PutObject, DeleteObject, DeleteObjectVersion)

and, for example, push these to Slack. Or push a summary every 5 minutes.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/log-s3-data-events.html is an outline of how.

@ameukam
Copy link
Member Author

ameukam commented Apr 26, 2022

BTW:, if it's useful to have, we can generate events (EventBridge) on:

* bucket policy changes

* object writes (`PutObject`, `DeleteObject`, `DeleteObjectVersion`)

and, for example, push these to Slack. Or push a summary every 5 minutes.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/log-s3-data-events.html is an outline of how.

@sftim working on it. Wanted to do it as a followup. (don't have enough bandwidth and the moment)

Related to:
  - kubernetes#3620

Ensure a AWS S3 bucket exists so we can test ip based redirection of
archeio.

The bucket contains a copy for the images layers
served by k8s.gcr.io

The bucket:
- is world readable
- only allow HTTPS connections
- only allow HTTP methods GET and HEAD
- has versioning enabled

Another private bucket is created for access logging.

Signed-off-by: Arnaud Meukam <[email protected]>
@ameukam ameukam force-pushed the aws-account-registry-k8s-io-sandbox branch from 2b5ff55 to 9e62b56 Compare April 30, 2022 13:00
@ameukam ameukam changed the title [WIP] oci-proxy: Add AWS S3 bucket for registry-sandbox.k8s.io oci-proxy: Add AWS S3 bucket for registry-sandbox.k8s.io May 3, 2022
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 3, 2022
@dims
Copy link
Member

dims commented May 12, 2022

/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 12, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ameukam, dims

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 563af90 into kubernetes:main May 12, 2022
@k8s-ci-robot k8s-ci-robot added this to the v1.25 milestone May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
Development

Successfully merging this pull request may close these issues.

4 participants