*: add EKS periodic jobs, upgrade "awstester" vendor #9940

gyuho · 2018-10-27T00:24:30Z

Upgrade "awstester" dependency
Add "eks" to "kubetest --deployment" flag
Add initial EKS e2e tests as periodic

krzyzacy · 2018-10-27T00:26:57Z

instead of kubernetes/eks, this probably need to be under kubernetes/sig-foo

does eks belongs to sig-aws?

gyuho · 2018-10-27T00:28:35Z

@krzyzacy @BenTheElder @spiffxp Yeah, I thought about it while I was writing this based on https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-gcp/sig-gcp-gke-config.yaml.

What do you think @d-nishi ?

krzyzacy · 2018-10-27T00:31:24Z

Also instead of a postsubmit, I would run it as a periodic job instead. You shouldn't need to checkout any repo?

BenTheElder · 2018-10-27T00:34:34Z

config/jobs/kubernetes/eks/k8s-tip-aws-eks-1.10.yaml

+  kubernetes/kubernetes:
+
+  # Run Kubernetes e2e tests with EKS prod build 1.10
+  - name: pull-kubernetes-e2e-tip-aws-eks-1.10-prod


we use pull- for presubmits, this should be something like ci-

BenTheElder · 2018-10-27T00:35:54Z

config/jobs/kubernetes/eks/k8s-1.10-aws-eks-1.10.yaml

+  kubernetes/kubernetes:
+
+  # Run Kubernetes 1.10 branch e2e tests with EKS prod build 1.10
+  - name: pull-kubernetes-e2e-1.10-aws-eks-1.10-prod


same here. https://github.com/kubernetes/test-infra/pull/9940/files#r228694772

BenTheElder · 2018-10-27T00:37:53Z

Does this need to be a post-submit? What about a periodic / cron job?
Most (nearly all) of our CI end to end testing consumes (downloads and uses) continuously pre-built binaries, which cuts out a lot of compute / memory in the build cluster.

gyuho · 2018-10-27T01:16:38Z

@BenTheElder @krzyzacy

consumes (downloads and uses) continuously pre-built

I see. Yes, then we can switch to periodic jobs. Since this is the very beginning to us, we do not want to use all the computing resources for every merge :)

Will update this PR shortly.

d-nishi · 2018-10-28T18:13:53Z

@gyuho - this can be sig-aws

@justinsb are you ok supporting the idea of kubernetes/sig-aws? we are integrating CI signal for the SIG-AWS subprojects. Thoughts please?

gyuho · 2018-10-29T18:53:54Z

@krzyzacy @BenTheElder Addressed. Now these are periodic jobs. PTAL. Thanks!

Signed-off-by: Gyuho Lee <[email protected]>

justinsb · 2018-10-30T01:08:40Z

I'm fine with the name change cc @d-nishi !

krzyzacy

I would make all shared env into one preset. Also maybe you can merge the three jobs into a single yaml file?

krzyzacy · 2018-10-30T01:11:43Z

config/jobs/kubernetes/sig-aws/eks/k8s-1.10-aws-eks-1.10.yaml

+    preset-service-account: "true"
+    preset-k8s-ssh: "true"
+    preset-ci-kubernetes-e2e-1-10-aws-eks-1-10: "true"
+


babynit: unnecessary new line

Removed. Thanks.

krzyzacy · 2018-10-30T01:12:21Z

config/jobs/kubernetes/sig-aws/eks/k8s-1.10-aws-eks-1.10.yaml

+  name: ci-kubernetes-e2e-1-10-aws-eks-1-10-prod
+  labels:
+    preset-service-account: "true"
+    preset-k8s-ssh: "true"


you won't need the ssh key as that's for gce

Removed. Thanks.

krzyzacy · 2018-10-30T01:12:32Z

config/jobs/kubernetes/sig-aws/eks/k8s-1.10-aws-eks-1.10.yaml

+
+  spec:
+    containers:
+    - image: 607362164682.dkr.ecr.us-west-2.amazonaws.com/awstester-e2e:20181029-141d7d02b498


is this image public?

This will be accessible with the credential that's used here. We can make it public if that's required :)

Here's the Dockerfile for that image https://github.com/aws/awstester/blob/master/scripts/awstester-e2e.Dockerfile.

hummm, wonder how prow will be able to access it as the credential is still in the secret. ImagePulling happens in pod init time.

or we can try and see :-)

@krzyzacy Just listed it as a public image. Let's try! Thanks!

Signed-off-by: Gyuho Lee <[email protected]>

krzyzacy

/lgtm

k8s-ci-robot · 2018-10-30T03:59:12Z

LGTM label has been added.

Git tree hash: 7eb26c82a316e6c2d27ce735b5c610fb0da6efaf

k8s-ci-robot · 2018-10-30T03:59:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gyuho, krzyzacy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [krzyzacy]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2018-10-30T04:06:54Z

@gyuho: Updated the job-config configmap using the following files:

key k8s-aws-eks-1.10.yaml using file config/jobs/kubernetes/sig-aws/eks/k8s-aws-eks-1.10.yaml

In response to this:

ref. #9814

Upgrade "awstester" dependency

Add "eks" to "kubetest --deployment" flag

Add initial EKS e2e tests as periodic

/cc @krzyzacy @d-nishi

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot requested review from d-nishi and krzyzacy October 27, 2018 00:24

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. area/config Issues or PRs related to code in /config labels Oct 27, 2018

BenTheElder reviewed Oct 27, 2018

View reviewed changes

gyuho mentioned this pull request Oct 28, 2018

kubetest: add "EKS" provider #9814

Closed

12 tasks

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 28, 2018

gyuho changed the title *: add EKS post-submit jobs, upgrade "awstester" vendor *: add EKS periodic jobs, upgrade "awstester" vendor Oct 28, 2018

k8s-ci-robot added the area/testgrid label Oct 28, 2018

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 29, 2018

gyuho added 2 commits October 29, 2018 17:23

vendor: update "awstester" to 0.1.1

b77a748

Signed-off-by: Gyuho Lee <[email protected]>

kubetest/main.go: add "eks" to "--deployment" flag description

832d308

Signed-off-by: Gyuho Lee <[email protected]>

krzyzacy reviewed Oct 30, 2018

View reviewed changes

gyuho added 2 commits October 29, 2018 18:25

config/jobs/kubernetes/sig-aws/eks: initial commit

64d5ca1

Signed-off-by: Gyuho Lee <[email protected]>

testgrid: add EKS jobs

97065dd

Signed-off-by: Gyuho Lee <[email protected]>

krzyzacy approved these changes Oct 30, 2018

View reviewed changes

k8s-ci-robot assigned krzyzacy Oct 30, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 30, 2018

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 30, 2018

k8s-ci-robot merged commit 27957ef into kubernetes:master Oct 30, 2018

gyuho deleted the eks branch October 30, 2018 04:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: add EKS periodic jobs, upgrade "awstester" vendor #9940

*: add EKS periodic jobs, upgrade "awstester" vendor #9940

gyuho commented Oct 27, 2018 •

edited

Loading

krzyzacy commented Oct 27, 2018

gyuho commented Oct 27, 2018

krzyzacy commented Oct 27, 2018

BenTheElder Oct 27, 2018

BenTheElder Oct 27, 2018

BenTheElder commented Oct 27, 2018

gyuho commented Oct 27, 2018

d-nishi commented Oct 28, 2018 •

edited

Loading

gyuho commented Oct 29, 2018

justinsb commented Oct 30, 2018

krzyzacy left a comment

krzyzacy Oct 30, 2018

gyuho Oct 30, 2018

krzyzacy Oct 30, 2018

gyuho Oct 30, 2018

krzyzacy Oct 30, 2018

gyuho Oct 30, 2018 •

edited

Loading

gyuho Oct 30, 2018

krzyzacy Oct 30, 2018

krzyzacy Oct 30, 2018

gyuho Oct 30, 2018

krzyzacy left a comment

k8s-ci-robot commented Oct 30, 2018

k8s-ci-robot commented Oct 30, 2018

k8s-ci-robot commented Oct 30, 2018

*: add EKS periodic jobs, upgrade "awstester" vendor #9940

*: add EKS periodic jobs, upgrade "awstester" vendor #9940

Conversation

gyuho commented Oct 27, 2018 • edited Loading

krzyzacy commented Oct 27, 2018

gyuho commented Oct 27, 2018

krzyzacy commented Oct 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BenTheElder commented Oct 27, 2018

gyuho commented Oct 27, 2018

d-nishi commented Oct 28, 2018 • edited Loading

gyuho commented Oct 29, 2018

justinsb commented Oct 30, 2018

krzyzacy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gyuho Oct 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

krzyzacy left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Oct 30, 2018

k8s-ci-robot commented Oct 30, 2018

k8s-ci-robot commented Oct 30, 2018

gyuho commented Oct 27, 2018 •

edited

Loading

d-nishi commented Oct 28, 2018 •

edited

Loading

gyuho Oct 30, 2018 •

edited

Loading