You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
1. What kops version are you running? The command kops version, will display
this information.
Trying to upgrade from a v1.14.2 cluster to v1.16.0
2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.
v1.14.9
3. What cloud provider are you using?
AWS
4. What commands did you run? What is the simplest way to reproduce this issue?
kops update cluster --yes --phase assets
5. What happened after the commands executed?
A number of assets failed to be successfully mirrored due to the below message:
W0310 12:14:04.307921 1498 executor.go:128] error running task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz" (6m48s remaining to succeed): unable to transfer "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz": the sha value in "s3://<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz.sha256" does not match "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz" calculated value "sha1:a997fd4809d96df84cf438efeff9bdd1bc8c69f2"
The assets failing to publish were as follows:
nodeup
kubelet
protokube.tar.gz
kops
utils.tar.gz
kubectl
6. What did you expect to happen?
The artefacts to be successfully mirrored to the private S3 bucket.
From following the code through I think the issue was introduced in 1.15-alpha by #7468 as we now will check against sha256 hashes if they exist, but compare them against SHA1 hashes always.
**7. Please provide your cluster manifest. Execute
Believe this is N/A so leaving out for now.
**8. Please run the commands with most verbose logging by adding the -v 10 flag.
I0310 12:13:43.864736 1498 executor.go:143] No progress made, sleeping before retrying 7 failed task(s)
I0310 12:13:53.864975 1498 executor.go:103] Tasks: 220 done / 227 total; 7 can run
I0310 12:13:53.865024 1498 executor.go:176] Executing task "CopyFile/https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl": *assettasks.CopyFile {"Name":"https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl","SourceFile":"https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl","TargetFile":"https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl","SHA":"d2a31e87c5f6deced4ba8899f9c465e54822f0cd146f32ea83cb1daafa5d9c4f","Lifecycle":"Sync"}
I0310 12:13:53.865051 1498 executor.go:176] Executing task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz": *assettasks.CopyFile {"Name":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz","SourceFile":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz","TargetFile":"https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz","SHA":"e473a232468db76a63663055ea6ed1e12f5d7e0357285cef080a78cb0361dee0","Lifecycle":"Sync"}
I0310 12:13:53.865122 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl.sha256
I0310 12:13:53.865120 1498 executor.go:176] Executing task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/utils.tar.gz": *assettasks.CopyFile {"Name":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/utils.tar.gz","SourceFile":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/utils.tar.gz","TargetFile":"https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz","SHA":"f0474049e5f5e98f60f83961827d5468f2f707a5f7446e77535e267b7312200e","Lifecycle":"Sync"}
I0310 12:13:53.865162 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz.sha256
I0310 12:13:53.865206 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz.sha256
I0310 12:13:53.865182 1498 executor.go:176] Executing task "CopyFile/https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet": *assettasks.CopyFile {"Name":"https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet","SourceFile":"https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet","TargetFile":"https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet","SHA":"01c35041ffbae9a13baa4b7790066a9274ef2dd32a8fd5b3d7c03944fe9c1af7","Lifecycle":"Sync"}
I0310 12:13:53.865060 1498 executor.go:176] Executing task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/darwin/amd64/kops": *assettasks.CopyFile {"Name":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/darwin/amd64/kops","SourceFile":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/darwin/amd64/kops","TargetFile":"https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops","SHA":"2ec48b66ea18859407a9b9121b6acbedc9182db3e95578ee879c333a53f69748","Lifecycle":"Sync"}
I0310 12:13:53.865111 1498 executor.go:176] Executing task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/nodeup": *assettasks.CopyFile {"Name":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/nodeup","SourceFile":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/nodeup","TargetFile":"https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup","SHA":"06630fce88dc39b52a9373c3fa006e3b9f74b50d7c72be3be025bb90669d8f14","Lifecycle":"Sync"}
I0310 12:13:53.865303 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops.sha256
I0310 12:13:53.865280 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet.sha256
I0310 12:13:53.865099 1498 executor.go:176] Executing task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/kops": *assettasks.CopyFile {"Name":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/kops","SourceFile":"https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/kops","TargetFile":"https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops","SHA":"dca9c93161d50d67f316e6e45ee86c2870bfeff0c5f07a10f8ae5bb847976a80","Lifecycle":"Sync"}
I0310 12:13:53.865312 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup.sha256
I0310 12:13:53.865377 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops.sha256
I0310 12:13:53.875870 1498 context.go:247] retrying after error file does not exist
I0310 12:13:53.885333 1498 context.go:247] retrying after error file does not exist
I0310 12:13:53.886458 1498 context.go:247] retrying after error file does not exist
I0310 12:13:53.890416 1498 context.go:247] retrying after error file does not exist
I0310 12:13:53.891264 1498 context.go:247] retrying after error file does not exist
I0310 12:13:53.892106 1498 context.go:247] retrying after error file does not exist
I0310 12:13:53.896861 1498 context.go:247] retrying after error file does not exist
I0310 12:13:54.376125 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup.sha256
I0310 12:13:54.385582 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz.sha256
I0310 12:13:54.386680 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/release/v1.14.9/bin/linux/amd64/kubectl.sha256
I0310 12:13:54.390615 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops.sha256
I0310 12:13:54.391009 1498 context.go:247] retrying after error file does not exist
I0310 12:13:54.391474 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet.sha256
I0310 12:13:54.392307 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops.sha256
I0310 12:13:54.394967 1498 context.go:247] retrying after error file does not exist
I0310 12:13:54.397058 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz.sha256
I0310 12:13:54.399714 1498 context.go:247] retrying after error file does not exist
I0310 12:13:54.405636 1498 context.go:247] retrying after error file does not exist
I0310 12:13:54.414912 1498 context.go:247] retrying after error file does not exist
I0310 12:13:54.415427 1498 context.go:247] retrying after error file does not exist
I0310 12:13:54.415902 1498 context.go:247] retrying after error file does not exist
I0310 12:13:55.391225 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz.sha256
I0310 12:13:55.395185 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl.sha256
I0310 12:13:55.395904 1498 context.go:247] retrying after error file does not exist
I0310 12:13:55.399891 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet.sha256
I0310 12:13:55.403996 1498 context.go:247] retrying after error file does not exist
I0310 12:13:55.405840 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops.sha256
I0310 12:13:55.410339 1498 context.go:247] retrying after error file does not exist
I0310 12:13:55.415117 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup.sha256
I0310 12:13:55.415598 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz.sha256
I0310 12:13:55.416073 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops.sha256
I0310 12:13:55.420813 1498 context.go:247] retrying after error file does not exist
I0310 12:13:55.424477 1498 context.go:247] retrying after error file does not exist
I0310 12:13:55.425519 1498 context.go:247] retrying after error file does not exist
I0310 12:13:55.437650 1498 context.go:247] retrying after error file does not exist
I0310 12:13:57.396240 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz.sha256
I0310 12:13:57.399569 1498 context.go:247] retrying after error file does not exist
I0310 12:13:57.404237 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet.sha256
I0310 12:13:57.408488 1498 context.go:247] retrying after error file does not exist
I0310 12:13:57.410558 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops.sha256
I0310 12:13:57.414848 1498 context.go:247] retrying after error file does not exist
I0310 12:13:57.421018 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup.sha256
I0310 12:13:57.424389 1498 context.go:247] retrying after error file does not exist
I0310 12:13:57.424682 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl.sha256
I0310 12:13:57.425695 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz.sha256
I0310 12:13:57.428648 1498 context.go:247] retrying after error file does not exist
I0310 12:13:57.429933 1498 context.go:247] retrying after error file does not exist
I0310 12:13:57.437942 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops.sha256
I0310 12:13:57.442436 1498 context.go:247] retrying after error file does not exist
I0310 12:14:01.399834 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz.sha256
I0310 12:14:01.404078 1498 context.go:251] hit maximum retries 5 with error file does not exist
I0310 12:14:01.404098 1498 copyfile.go:78] unable to download: "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz.sha256", assuming target file is not present, and if not present may not be an error: file does not exist
I0310 12:14:01.404169 1498 copyfile.go:130] copying bits from "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/utils.tar.gz" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz"
I0310 12:14:01.404189 1498 context.go:179] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/utils.tar.gz
I0310 12:14:01.408764 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet.sha256
I0310 12:14:01.414491 1498 context.go:251] hit maximum retries 5 with error file does not exist
I0310 12:14:01.414506 1498 copyfile.go:78] unable to download: "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet.sha256", assuming target file is not present, and if not present may not be an error: file does not exist
I0310 12:14:01.414551 1498 copyfile.go:130] copying bits from "https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet"
I0310 12:14:01.414572 1498 context.go:179] Performing HTTP request: GET https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet
I0310 12:14:01.415096 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops.sha256
I0310 12:14:01.419244 1498 context.go:251] hit maximum retries 5 with error file does not exist
I0310 12:14:01.419259 1498 copyfile.go:78] unable to download: "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops.sha256", assuming target file is not present, and if not present may not be an error: file does not exist
I0310 12:14:01.419306 1498 copyfile.go:130] copying bits from "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/darwin/amd64/kops" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops"
I0310 12:14:01.419325 1498 context.go:179] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.16.0/darwin/amd64/kops
I0310 12:14:01.424626 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup.sha256
I0310 12:14:01.428447 1498 context.go:251] hit maximum retries 5 with error file does not exist
I0310 12:14:01.428462 1498 copyfile.go:78] unable to download: "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup.sha256", assuming target file is not present, and if not present may not be an error: file does not exist
I0310 12:14:01.428504 1498 copyfile.go:130] copying bits from "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/nodeup" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup"
I0310 12:14:01.428523 1498 context.go:179] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/nodeup
I0310 12:14:01.428839 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl.sha256
I0310 12:14:01.430186 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz.sha256
I0310 12:14:01.433025 1498 context.go:251] hit maximum retries 5 with error file does not exist
I0310 12:14:01.433041 1498 copyfile.go:78] unable to download: "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl.sha256", assuming target file is not present, and if not present may not be an error: file does not exist
I0310 12:14:01.433083 1498 copyfile.go:130] copying bits from "https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl"
I0310 12:14:01.433103 1498 context.go:179] Performing HTTP request: GET https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl
I0310 12:14:01.434215 1498 context.go:251] hit maximum retries 5 with error file does not exist
I0310 12:14:01.434230 1498 copyfile.go:78] unable to download: "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz.sha256", assuming target file is not present, and if not present may not be an error: file does not exist
I0310 12:14:01.434280 1498 copyfile.go:130] copying bits from "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz"
I0310 12:14:01.434302 1498 context.go:179] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz
I0310 12:14:01.442673 1498 context.go:179] Performing HTTP request: GET https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops.sha256
I0310 12:14:01.446919 1498 context.go:251] hit maximum retries 5 with error file does not exist
I0310 12:14:01.446934 1498 copyfile.go:78] unable to download: "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops.sha256", assuming target file is not present, and if not present may not be an error: file does not exist
I0310 12:14:01.446979 1498 copyfile.go:130] copying bits from "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/kops" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops"
I0310 12:14:01.447000 1498 context.go:179] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/kops
W0310 12:14:04.307870 1498 executor.go:128] error running task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/nodeup" (6m48s remaining to succeed): unable to transfer "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/nodeup" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup": the sha value in "s3://<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/nodeup.sha256" does not match "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/nodeup" calculated value "sha1:517f7b5e821b8d49424e94d0d35f33adcf17bdc5"
W0310 12:14:04.307904 1498 executor.go:128] error running task "CopyFile/https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet" (6m48s remaining to succeed): unable to transfer "https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet": the sha value in "s3://<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet.sha256" does not match "https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubelet" calculated value "sha1:f84c48d5368d6f1d687b530b673d2d093c12fe56"
W0310 12:14:04.307921 1498 executor.go:128] error running task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz" (6m48s remaining to succeed): unable to transfer "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz": the sha value in "s3://<BUCKET_NAME_REDACTED>/kops/1.16.0/images/protokube.tar.gz.sha256" does not match "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/images/protokube.tar.gz" calculated value "sha1:a997fd4809d96df84cf438efeff9bdd1bc8c69f2"
W0310 12:14:04.307940 1498 executor.go:128] error running task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/kops" (6m48s remaining to succeed): unable to transfer "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/kops" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops": the sha value in "s3://<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/kops.sha256" does not match "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/kops" calculated value "sha1:95295b67781a57d9c1a578e1dd1a67c5052a6f03"
W0310 12:14:04.307958 1498 executor.go:128] error running task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/darwin/amd64/kops" (6m48s remaining to succeed): unable to transfer "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/darwin/amd64/kops" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops": the sha value in "s3://<BUCKET_NAME_REDACTED>/kops/1.16.0/darwin/amd64/kops.sha256" does not match "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/darwin/amd64/kops" calculated value "sha1:c8044a249f4373f941135f72c9f1a82292442806"
W0310 12:14:04.307974 1498 executor.go:128] error running task "CopyFile/https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/utils.tar.gz" (6m48s remaining to succeed): unable to transfer "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/utils.tar.gz" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz": the sha value in "s3://<BUCKET_NAME_REDACTED>/kops/1.16.0/linux/amd64/utils.tar.gz.sha256" does not match "https://kubeupv2.s3.amazonaws.com/kops/1.16.0/linux/amd64/utils.tar.gz" calculated value "sha1:13f380f3c5f2df41b419e1eadaf50f886021a3fc"
W0310 12:14:04.307991 1498 executor.go:128] error running task "CopyFile/https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl" (6m48s remaining to succeed): unable to transfer "https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl" to "https://s3-eu-west-1.amazonaws.com/<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl": the sha value in "s3://<BUCKET_NAME_REDACTED>/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl.sha256" does not match "https://storage.googleapis.com/kubernetes-release/release/v1.14.9/bin/linux/amd64/kubectl" calculated value "sha1:90579134a6ae9693cb39fb2c6150de1b36e8af76"
I0310 12:14:04.308006 1498 executor.go:143] No progress made, sleeping before retrying 7 failed task(s)
9. Anything else do we need to know?
Believe this will be resolved by #8694 - testing now.
The text was updated successfully, but these errors were encountered:
1. What
kops
version are you running? The commandkops version
, will displaythis information.
Trying to upgrade from a v1.14.2 cluster to v1.16.0
2. What Kubernetes version are you running?
kubectl version
will print theversion if a cluster is running or provide the Kubernetes version specified as
a
kops
flag.v1.14.9
3. What cloud provider are you using?
AWS
4. What commands did you run? What is the simplest way to reproduce this issue?
kops update cluster --yes --phase assets
5. What happened after the commands executed?
A number of assets failed to be successfully mirrored due to the below message:
The assets failing to publish were as follows:
6. What did you expect to happen?
The artefacts to be successfully mirrored to the private S3 bucket.
From following the code through I think the issue was introduced in 1.15-alpha by #7468 as we now will check against
sha256
hashes if they exist, but compare them against SHA1 hashes always.**7. Please provide your cluster manifest. Execute
Believe this is N/A so leaving out for now.
**8. Please run the commands with most verbose logging by adding the
-v 10
flag.9. Anything else do we need to know?
Believe this will be resolved by #8694 - testing now.
The text was updated successfully, but these errors were encountered: