Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build sandbox infrastructure for images-layers #3620

Closed
4 tasks done
ameukam opened this issue Apr 13, 2022 · 13 comments
Closed
4 tasks done

build sandbox infrastructure for images-layers #3620

ameukam opened this issue Apr 13, 2022 · 13 comments
Assignees
@ameukam
Labels
area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. sig/release Categorizes an issue or PR as relevant to SIG Release.

Comments

@ameukam
Copy link
Member

ameukam commented Apr 13, 2022 •
edited
Loading

Part of:

Context

Build a development environment to explore AWS S3 capabilities in order to achieve goals defined in the design doc

See: https://github.com/kubernetes/k8s.io/wiki/New-Registry-url-for-Kubernetes-(registry.k8s.io)

Implementation

Reuse the account created in cncf/credits#1 as a sandbox environment image layers promotion to AWS.

Required:

  • Create a S3 bucket for testing
    • The bucket already exists and a full sync was already done. I need to recreate the bucket to respect naming convention and do a full copy again.
    • Ensure the bucket is world-readable
    • Allow only HTTPs connections to the bucket
  • [X] Give access to @jaypipes and @kubernetes/release-engineering and associate them AWSS3FullAccess policy.

Non-Required:

  • Enable AWS Config (experimental purpose)
  • Enable AWS Cloudtrail (experimental purpose)
  • Enable AWS GuardDuty (experimental purpose)
  • Ensure access logs is enabled for the bucket (experimental purpose)
  • Ensure versioning is enabled (experimental purpose)
  • Ensure a notification is send when a object is deleted (experimental purpose)

Update:

This AWS account is for this infrastructure is temporary and will be replaced. In the meantime. The HCL code for it will be in https://github.com/kubernetes/k8s.io/tree/main/infra/aws/terraform.

/assign @ameukam
/sig release
/sig k8s-infra
/kind feature
/area artifacts
/priority important-soon
/milestone v1.24
@ameukam ameukam added the sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. label Apr 13, 2022
@k8s-ci-robot k8s-ci-robot added sig/release Categorizes an issue or PR as relevant to SIG Release. kind/feature Categorizes issue or PR as related to a new feature. labels Apr 13, 2022
@k8s-ci-robot k8s-ci-robot added this to the v1.24 milestone Apr 13, 2022
@k8s-ci-robot k8s-ci-robot added area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Apr 13, 2022
@ameukam
Copy link
Member Author

ameukam commented Apr 13, 2022

cc @jhoblitt @BobyMCbobs

@ameukam ameukam moved this from Backlog to Implementation TODO in registry.k8s.io (SIG K8S Infra) Apr 13, 2022
@BenTheElder BenTheElder mentioned this issue Apr 15, 2022
Closed
@BobyMCbobs BobyMCbobs mentioned this issue Apr 19, 2022
Merged
@ameukam
Copy link
Member Author

ameukam commented Apr 22, 2022

cc @sftim

@sftim
Copy link
Contributor

sftim commented Apr 22, 2022

What information can we add to this issue to make it easier for a newcomer to help?

@ameukam
Copy link
Member Author

ameukam commented Apr 22, 2022

What information can we add to this issue to make it easier for a newcomer to help?

I think we need to provide more context about what we are trying achieve. I updated the description. But this issue is targeted for individuals with good knowledge of AWS.

@sftim
Copy link
Contributor

sftim commented Apr 22, 2022

To clarify: can we help someone who is confident in AWS, but new to the infrastructure SIG?

@ameukam
Copy link
Member Author

ameukam commented Apr 23, 2022

To clarify: can we help someone who is confident in AWS, but new to the infrastructure SIG?

@sftim short answer: Yes. How to onboard can be defined over time. For AWS, I would say the only requirement is knowledge of the HCL but it's not a strong requirement.

ameukam added a commit to ameukam/k8s.io that referenced this issue Apr 25, 2022
@ameukam
oci-proxy: Add AWS S3 bucket for registry-sandbox.k8s.io
a86fcfa 
Related to:
  - kubernetes#3620

Ensure a AWS S3 bucket exists so we can test ip based redirection of
archeio.

The bucket contains a copy for the images layers
served by k8s.gcr.io

The bucket:
- is world readable
- only allow HTTPS connections
- only allow HTTP methods GET and HEAD
- has versioning enabled

Another private bucket is created for access logging.

Signed-off-by: Arnaud Meukam <[email protected]>
@ameukam ameukam mentioned this issue Apr 25, 2022
Merged
ameukam added a commit to ameukam/k8s.io that referenced this issue Apr 25, 2022
@ameukam
oci-proxy: Add AWS S3 bucket for registry-sandbox.k8s.io
2b5ff55 
Related to:
  - kubernetes#3620

Ensure a AWS S3 bucket exists so we can test ip based redirection of
archeio.

The bucket contains a copy for the images layers
served by k8s.gcr.io

The bucket:
- is world readable
- only allow HTTPS connections
- only allow HTTP methods GET and HEAD
- has versioning enabled

Another private bucket is created for access logging.

Signed-off-by: Arnaud Meukam <[email protected]>
@ameukam ameukam moved this to In Progress in SIG K8S Infra Apr 25, 2022
@BobyMCbobs BobyMCbobs mentioned this issue Apr 26, 2022
Closed
25 tasks
ameukam added a commit to ameukam/k8s.io that referenced this issue Apr 30, 2022
@ameukam
oci-proxy: Add AWS S3 bucket for registry-sandbox.k8s.io
9e62b56 
Related to:
  - kubernetes#3620

Ensure a AWS S3 bucket exists so we can test ip based redirection of
archeio.

The bucket contains a copy for the images layers
served by k8s.gcr.io

The bucket:
- is world readable
- only allow HTTPS connections
- only allow HTTP methods GET and HEAD
- has versioning enabled

Another private bucket is created for access logging.

Signed-off-by: Arnaud Meukam <[email protected]>
@ameukam
Copy link
Member Author

ameukam commented May 12, 2022

/milestone v1.25

@k8s-ci-robot k8s-ci-robot modified the milestones: v1.24, v1.25 May 12, 2022
@sftim
Copy link
Contributor

sftim commented Jun 1, 2022

Give access to @jaypipes and https://github.com/orgs/kubernetes/teams/release-engineering and associate them AWSS3FullAccess policy.

Is AWSS3FullAccess the right policy to grant? I'd expect to have a role that is allowed to run Terraform to deploy buckets, and another role that is allowed to add and remove objects.

@ameukam
Copy link
Member Author

ameukam commented Jun 22, 2022

Give access to @jaypipes and https://github.com/orgs/kubernetes/teams/release-engineering and associate them AWSS3FullAccess policy.

Is AWSS3FullAccess the right policy to grant? I'd expect to have a role that is allowed to run Terraform to deploy buckets, and another role that is allowed to add and remove objects.

The main intent is to provide an environment with some autonomy for the contributors to implement the tooling needed to achieve the goal defined in kubernetes-sigs/promo-tools#533. I'm open to different propositions.

@ameukam ameukam mentioned this issue Jul 5, 2022
Closed
@ameukam
Copy link
Member Author

ameukam commented Aug 19, 2022

/milestone clear

@k8s-ci-robot k8s-ci-robot removed this from the v1.25 milestone Aug 19, 2022
@BobyMCbobs
Copy link
Member

@ameukam, I believe we've got an account for the buckets for registry.k8s.io; can this issue be closed?

@riaankleinhans riaankleinhans moved this from Implementation TODO to Breakdown TODO in registry.k8s.io (SIG K8S Infra) Sep 15, 2022
@ameukam
Copy link
Member Author

ameukam commented Oct 5, 2022

@ameukam, I believe we've got an account for the buckets for registry.k8s.io; can this issue be closed?

Yes. The AWS account provided for this issue is still used for registry-sandbox.k8s.io.

/close

@k8s-ci-robot
Copy link
Contributor

@ameukam: Closing this issue.

In response to this:

@ameukam, I believe we've got an account for the buckets for registry.k8s.io; can this issue be closed?

Yes. The AWS account provided for this issue is still used for registry-sandbox.k8s.io.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Repository owner moved this from In Progress to Done in SIG K8S Infra Oct 5, 2022
Repository owner moved this from Breakdown TODO to Audit & Logging Issues / PRs in registry.k8s.io (SIG K8S Infra) Oct 5, 2022
@ameukam ameukam moved this from Audit & Logging Issues / PRs to Validation Done in registry.k8s.io (SIG K8S Infra) Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ameukam ameukam

Labels
area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. sig/release Categorizes an issue or PR as relevant to SIG Release.
Projects
SIG K8S Infra
Status: Done
registry.k8s.io (SIG K8S Infra)
Status: Done
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ameukam @BobyMCbobs @k8s-ci-robot @sftim