Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot upgrade any cluster from 1.18 to 1.19 #11045

Closed
vkryzh opened this issue Mar 14, 2021 · 1 comment
Closed

Cannot upgrade any cluster from 1.18 to 1.19 #11045

vkryzh opened this issue Mar 14, 2021 · 1 comment

Comments

@vkryzh
Copy link

vkryzh commented Mar 14, 2021 •
edited
Loading

1. What kops version are you running? The command kops version, will display
this information.
1.19.1/1.20beta2

2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.
1.18.16->1.9.x

3. What cloud provider are you using?
AWS

4. What commands did you run? What is the simplest way to reproduce this issue?
I upgrade cluster version from 1.18.16 to 1.19.7, do kops update and rolling-update for master node.

5. What happened after the commands executed?
All nodes are in not-ready status, master is not in the list.

6. What did you expect to happen?
Master joins the cluster, can start nodes upgrade.

7. Please provide your cluster manifest. Execute
kops get --name my.example.com -o yaml to display your cluster manifest.
You may want to remove your cluster name and other sensitive information.

apiVersion: kops.k8s.io/v1alpha2
kind: Cluster
metadata:
  creationTimestamp: "2019-06-20T14:32:07Z"
  generation: 72
  name: xxxxxxxxxxxxxxxxxxxxxxxxx
spec:
  additionalPolicies:
    node: "[\n    {\n        \"Effect\": \"Allow\",\n        \"Action\": [\n            \"ec2:DescribeLaunchTemplateVersions\",\n            \"autoscaling:DescribeAutoScalingGroups\",\n            \"autoscaling:DescribeAutoScalingInstances\",\n            \"autoscaling:SetDesiredCapacity\",\n            \"autoscaling:DescribeLaunchConfigurations\",\n            \"autoscaling:DescribeTags\",\n            \"autoscaling:TerminateInstanceInAutoScalingGroup\"\n        ],\n        \"Resource\": [\n            \"*\"\n        ]\n    },\n    {\n        \"Effect\": \"Allow\",\n        \"Action\": [\n            \"route53:ChangeResourceRecordSets\"\n        ],\n        \"Resource\": [\n            \"arn:aws:route53:::hostedzone/*\"\n        ]\n    },\n    {\n        \"Effect\": \"Allow\",\n        \"Action\": [\n            \"route53:ListHostedZones\",\n            \"route53:ListResourceRecordSets\"\n        ],\n        \"Resource\": [\n            \"*\"\n        ]\n    },\n    {\n        \"Effect\": \"Allow\",\n        \"Action\": [\n            \"ec2:DescribeAddresses\",\n            \"ec2:AssociateAddress\",\n           \"ec2:TerminateInstances\"\n        ],\n        \"Resource\": \"*\"\n    }\n]  \n"
  api:
    loadBalancer:
      class: Classic
      type: Internal
  authorization:
    rbac: {}
  channel: stable
  cloudProvider: aws
  configBase: s3://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  docker:
    registryMirrors:
    - https://mirror.gcr.io
  etcdClusters:
  - cpuRequest: 500m
    etcdMembers:
    - instanceGroup: master-us-east-1a
      name: a
    memoryRequest: 1Gi
    name: main
  - cpuRequest: 100m
    etcdMembers:
    - instanceGroup: master-us-east-1a
      name: a
    memoryRequest: 256Mi
    name: events
  iam:
    allowContainerRegistry: true
    legacy: false
  kubeAPIServer:
    anonymousAuth: false
    authorizationMode: RBAC
    featureGates:
      TTLAfterFinished: "true"
    oidcClientID: dev-k8s-authenticator
    oidcGroupsClaim: groups
    oidcIssuerURL: xxxxxxxxxxxxxxxxxxxxxxxxx
    oidcUsernameClaim: email
  kubeControllerManager:
    featureGates:
      TTLAfterFinished: "true"
    horizontalPodAutoscalerDownscaleDelay: 3m0s
    horizontalPodAutoscalerSyncPeriod: 10s
    horizontalPodAutoscalerTolerance: 50m
    horizontalPodAutoscalerUpscaleDelay: 20s
  kubeDNS:
    externalCoreFile: |
      .:53 {
        errors
        health {
            lameduck 5s
        }
        ready
        autopath @kubernetes
        kubernetes cluster.local in-addr.arpa ip6.arpa {
            pods verified
            fallthrough in-addr.arpa ip6.arpa
            ttl 30
        }
        prometheus :9153
        forward . /etc/resolv.conf {
            max_fails 0
        }

        loop
        cache 300 {
            prefetch 1 10m 80%
        }
        loadbalance
        reload
      }
      amazonaws.com.:53 {
        errors
        loadbalance round_robin
        prometheus 0.0.0.0:9153
        loop
        forward . 1.1.1.1 8.8.8.8  {
        }
        cache 3600 {
            success 19968
            prefetch 1 10m 80%
        }
      }
    memoryLimit: 1Gi
    nodeLocalDNS:
      cpuRequest: 50m
      enabled: true
      memoryRequest: 64Mi
    provider: CoreDNS
  kubeProxy:
    metricsBindAddress: 0.0.0.0
  kubelet:
    anonymousAuth: false
    authenticationTokenWebhook: true
    authorizationMode: Webhook
    cpuManagerPolicy: static
    evictionHard: memory.available<256Mi,nodefs.available<10%,nodefs.inodesFree<5%,imagefs.available<10%,imagefs.inodesFree<5%
    evictionMaxPodGracePeriod: 30
    evictionMinimumReclaim: memory.available=256Mi
    kubeReserved:
      cpu: 400m
      memory: 512Mi
    maxPods: 250
    systemReserved:
      cpu: 400m
      memory: 512Mi
  kubernetesApiAccess:
  - 10.0.0.0/8
  kubernetesVersion: 1.18.16
  masterInternalName: api.internal.xxxxxxxxxxxxxxxxxxxxxxxxxx
  masterPublicName: api.xxxxxxxxxxxxxxxxxxxx
  networkCIDR: xxxxxxxxxxxxxx
  networkID: xxxxxxxxxxxxxxxxxx
  networking:
    weave:
      mtu: 8912
      netExtraArgs: --log-level=info
      npcExtraArgs: --log-level=info
  nonMasqueradeCIDR: xxxxxxxxxxxxxxxxx
  sshAccess:
  - xxxxxxxxxxxxxxxxxxxxxx
  subnets:
  - cidr: xxxxxxxxxxxxxx
    id: xxxxxxxxxxxxxxxxxxxxxxx
    name: us-east-1a
    type: Private
    zone: us-east-1a
  - cidr: xxxxxxxxxxxxxxxxxxx
    id: xxxxxxxxxxxxxxxxxxx
    name: us-east-1b
    type: Private
    zone: us-east-1b
  - cidr: xxxxxxxxxxxxxxxxxxxx
    id: xxxxxxxxxxxxxxxxxxxxx
    name: us-east-1c
    type: Private
    zone: us-east-1c
  - cidr: xxxxxxxxxxxxxxxxxxx
    id: xxxxxxxxxxxxxxxxxxxxxxxx
    name: public-us-east-1a
    type: Public
    zone: us-east-1a
  - cidr: xxxxxxxxxxxxxxxxxx
    id: xxxxxxxxxxxxxxxxxxxx
    name: public-us-east-1b
    type: Public
    zone: us-east-1b
  - cidr: xxxxxxxxxxxxxxxxxxxx
    id: xxxxxxxxxxxxxxxxxxx
    name: public-us-east-1c
    type: Public
    zone: us-east-1c
  topology:
    dns:
      type: Private
    masters: private
    nodes: public

8. Some logs
kubelet on master:

Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.635504    5690 kubelet_node_status.go:362] Setting node annotation to enable volume controller attach/detach
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.635535    5690 kubelet_node_status.go:410] Adding node label from cloud provider: beta.kubernetes.io/instance-type=r5a.2xlarge
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.635543    5690 kubelet_node_status.go:412] Adding node label from cloud provider: node.kubernetes.io/instance-type=r5a.2xlarge
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.635550    5690 kubelet_node_status.go:423] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/zone=us-east-1a
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.635556    5690 kubelet_node_status.go:425] Adding node label from cloud provider: topology.kubernetes.io/zone=us-east-1a
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.635564    5690 kubelet_node_status.go:429] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/region=us-east-1
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.635570    5690 kubelet_node_status.go:431] Adding node label from cloud provider: topology.kubernetes.io/region=us-east-1
Mar 14 16:34:09 ip-************** kubelet[5690]: E0314 16:34:09.660940    5690 kubelet.go:2186] node "ip-**************.ec2.internal" not found
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.673904    5690 kubelet_node_status.go:554] Recording NodeHasSufficientMemory event message for node ip-**************.ec2.internal
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.673951    5690 kubelet_node_status.go:554] Recording NodeHasNoDiskPressure event message for node ip-**************.ec2.internal
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.673961    5690 kubelet_node_status.go:554] Recording NodeHasSufficientPID event message for node ip-**************.ec2.internal
Mar 14 16:34:09 ip-************** kubelet[5690]: I0314 16:34:09.673984    5690 kubelet_node_status.go:71] Attempting to register node ip-**************.ec2.internal
Mar 14 16:34:09 ip-************** kubelet[5690]: E0314 16:34:09.676165    5690 kubelet_node_status.go:93] Unable to register node "ip-**************.ec2.internal" with API server: nodes is forbidden: User "system:node:ip-**************.ec2.internal" cannot create resource "nodes" in API group "" at the cluster scope
Mar 14 16:34:09 ip-************** kubelet[5690]: E0314 16:34:09.683236    5690 controller.go:136] failed to ensure node lease exists, will retry in 7s, error: leases.coordination.k8s.io "ip-**************.ec2.internal" is forbidden: User "system:node:ip-**************.ec2.internal" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Mar 14 16:34:09 ip-************** kubelet[5690]: E0314 16:34:09.761100    5690 kubelet.go:2186] node "ip-**************.ec2.internal" not found
Mar 14 16:34:09 ip-************** kubelet[5690]: E0314 16:34:09.804354    5690 reflector.go:127] k8s.io/kubernetes/pkg/kubelet/kubelet.go:438: Failed to watch *v1.Node: failed to list *v1.Node: nodes "ip-**************.ec2.internal" is forbidden: User "system:node:ip-**************.ec2.internal" cannot list resource "nodes" in API group "" at the cluster scope
Mar 14 16:34:09 ip-************** kubelet[5690]: E0314 16:34:09.861257    5690 kubelet.go:2186] node "ip-**************.ec2.internal" not found
Mar 14 16:34:09 ip-************** kubelet[5690]: E0314 16:34:09.961408    5690 kubelet.go:2186] node "ip-**************.ec2.internal" not found
Mar 14 16:34:10 ip-************** kubelet[5690]: E0314 16:34:10.061521    5690 kubelet.go:2186] node "ip-**************.ec2.internal" not found
Mar 14 16:34:10 ip-************** kubelet[5690]: W0314 16:34:10.123236    5690 cni.go:239] Unable to update cni config: no networks found in /etc/cni/net.d/
Mar 14 16:34:10 ip-************** kubelet[5690]: I0314 16:34:10.155873    5690 csi_plugin.go:994] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-**************.ec2.internal" is forbidden: User "system:node:ip-**************.ec2.internal" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope

kops-nodeup on master:

[File//srv/kubernetes]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576609    1429 topological_sort.go:73]         File//etc/kubernetes/pki/kube-apiserver/etcd-client.key:        [File//etc/kubernetes/pki/kube-apiserver IssueCert/etcd-client]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576616    1429 topological_sort.go:73]         Package/bridge-utils:        [UpdatePackages]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576622    1429 topological_sort.go:73]         File//etc/kubernetes/kube-apiserver-healthcheck/secrets/client.crt:        [UserTask/kube-apiserver-healthcheck File//etc/kubernetes/kube-apiserver-healthcheck/secrets IssueCert/kube-apiserver-healthcheck]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576633    1429 topological_sort.go:73]         Package/wget:        [UpdatePackages]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576640    1429 topological_sort.go:73]         File//usr/local/bin/kubectl:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576646    1429 topological_sort.go:73]         File//etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.key:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576652    1429 topological_sort.go:73]         Service/protokube.service:        [File//usr/bin/runc File//etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.key File//etc/sysctl.d/99-k8s-general.conf File//etc/kubernetes/kube-apiserver-healthcheck/secrets/client.crt Package/libseccomp2 File//etc/logrotate.d/kube-scheduler File//etc/kubernetes/manifests/kube-scheduler.manifest File//etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.crt File//var/log/kube-proxy.log File//opt/cni/bin/vlan File//usr/share/doc/containerd/apache.txt File//home/ubuntu/.kube/config File//srv/kubernetes/ca.key Package/cgroupfs-mount UserTask/kops-controller Package/wget File//etc/logrotate.d/kubelet File//srv/kubernetes/apiserver-aggregator-ca.crt File//srv/kubernetes/kubelet-api.crt File//etc/sysconfig/docker File//usr/local/bin/kubectl File//etc/logrotate.d/docker Package/iptables File//etc/kubernetes/pki/kube-apiserver/etcd-ca.crt File//etc/logrotate.d/kube-apiserver Package/nfs-common File//etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.key File//etc/systemd/timesyncd.conf File//srv/kubernetes/apiserver-aggregator.crt File//etc/kubernetes/kube-apiserver-healthcheck/secrets/ca.crt File//etc/kubernetes/manifests/kube-proxy.manifest File//etc/kubernetes/kops-controller/ca-key.pem File//opt/cni/bin/bandwidth Package/logrotate File//etc/kubernetes/manifests/etcd-main.manifest Package/ebtables File//opt/kops/bin/iptables-setup File//var/lib/kube-controller-manager/kubeconfig File//usr/bin/docker-proxy File//etc/logrotate.d/kube-proxy UserTask/kube-apiserver-healthcheck File//etc/kubernetes/pki/kube-apiserver File//opt/cni/bin/loopback Package/ethtool Package/libltdl7 File//etc/kubernetes/manifests/kube-controller-manager.manifest File//etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.key File//etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.key File//opt/cni/bin/portmap GroupTask/docker File//etc/kubernetes/kube-apiserver-healthcheck/secrets File//opt/cni/bin/sbr File//srv/kubernetes File//etc/logrotate.d/kube-controller-manager File//etc/kubernetes/manifests/etcd-events.manifest File//etc/kubernetes/kops-controller/kops-controller.key Package/curl File//srv/kubernetes/ca.crt File//srv/kubernetes/apiserver-aggregator.key File//usr/bin/docker-init File//usr/bin/containerd File//etc/logrotate.d/kube-addons File//etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.key File//etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.key File//opt/cni/bin/dhcp File//etc/sysconfig/kubelet Package/util-linux File//etc/kubernetes/kube-apiserver-healthcheck/secrets/client.key File//srv/kubernetes/kubelet-api.key File//etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.crt File//opt/cni/bin/host-device File//etc/kubernetes/cloud.config File//usr/local/bin/kubelet File//var/lib/kubelet File//var/log/kube-controller-manager.log File//etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.crt File//opt/cni/bin/firewall File//srv/kubernetes/assets File//etc/kubernetes/manifests/kube-apiserver.manifest File//etc/kubernetes/kops-controller/ca.pem Package/perl File//opt/cni/bin/macvlan UpdatePackages File//opt/cni/bin/ptp File//opt/cni/bin/tuning Package/netcat-traditional File//etc/containerd/config-kops.toml File//var/lib/kops/kubeconfig File//opt/cni/bin/ipvlan File//opt/cni/bin/static Package/git File//etc/sysconfig/containerd File//etc/logrotate.d/etcd-events File//var/log/kube-apiserver.log File//var/log/kube-scheduler.log File//usr/bin/ctr File//srv/kubernetes/server.key File//var/lib/kube-scheduler/config.yaml File//etc/kubernetes/kops-controller File//etc/apt/apt.conf.d/20auto-upgrades Package/unattended-upgrades File//usr/bin/docker Package/conntrack File//etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.crt Package/apt-transport-https File//etc/kubernetes/manifests File//srv/kubernetes/known_tokens.csv File//opt/cni/bin/bridge File//home/ubuntu/.kube/ File//var/lib/kube-scheduler/kubeconfig File//opt/cni/bin/host-local File//etc/logrotate.d/etcd Package/libapparmor1 File//srv/kubernetes/service-account.key File//etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.crt File//etc/kubernetes/pki/kube-apiserver/etcd-client.key File//srv/kubernetes/kubelet-server.crt File//srv/kubernetes/kubelet-server.key File//etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.crt Package/python-apt File//usr/bin/dockerd File//var/lib/kubectl/kubeconfig Package/bridge-utils File//etc/kubernetes/kops-controller/kops-controller.crt File//etc/kubernetes/pki/kube-apiserver/etcd-client.crt File//usr/bin/containerd-shim File//usr/share/doc/docker/apache.txt File//etc/cni/net.d/ Package/pigz Package/socat File//srv/kubernetes/server.crt File//opt/cni/bin/flannel]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576702    1429 topological_sort.go:73]         File//etc/kubernetes/pki/kube-apiserver:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576709    1429 topological_sort.go:73]         Package/libltdl7:        [UpdatePackages]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576715    1429 topological_sort.go:73]         File//usr/bin/containerd-shim:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576721    1429 topological_sort.go:73]         KubeConfig/kube-controller-manager:        [IssueCert/kube-controller-manager IssueCert/kube-controller-manager IssueCert/kube-controller-manager]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576729    1429 topological_sort.go:73]         File//usr/bin/docker:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576738    1429 topological_sort.go:73]         IssueCert/kubecfg:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576744    1429 topological_sort.go:73]         File//etc/containerd/config-kops.toml:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576749    1429 topological_sort.go:73]         File//opt/cni/bin/bandwidth:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576754    1429 topological_sort.go:73]         LoadImageTask/protokube:        [Service/containerd.service Service/docker.service]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576759    1429 topological_sort.go:73]         File//srv/kubernetes/ca.crt:        [File//srv/kubernetes]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576763    1429 topological_sort.go:73]         File//etc/kubernetes/kube-apiserver-healthcheck/secrets/client.key:        [UserTask/kube-apiserver-healthcheck File//etc/kubernetes/kube-apiserver-healthcheck/secrets IssueCert/kube-apiserver-healthcheck]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576768    1429 topological_sort.go:73]         IssueCert/kubelet-server:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576773    1429 topological_sort.go:73]         File//opt/cni/bin/flannel:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576777    1429 topological_sort.go:73]         File//home/ubuntu/.kube/config:        [File//home/ubuntu/.kube/ KubeConfig/kubecfg]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576782    1429 topological_sort.go:73]         File//etc/logrotate.d/kube-scheduler:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576788    1429 topological_sort.go:73]         File//etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.crt:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576797    1429 topological_sort.go:73]         Service/systemd-timesyncd:        [File//opt/cni/bin/ptp File//opt/cni/bin/tuning Package/netcat-traditional File//etc/containerd/config-kops.toml File//var/lib/kops/kubeconfig File//opt/cni/bin/ipvlan File//opt/cni/bin/static Package/git File//etc/sysconfig/containerd File//etc/logrotate.d/etcd-events File//var/log/kube-apiserver.log File//var/log/kube-scheduler.log File//usr/bin/ctr File//srv/kubernetes/server.key File//var/lib/kube-scheduler/config.yaml File//etc/kubernetes/kops-controller File//etc/apt/apt.conf.d/20auto-upgrades Package/unattended-upgrades File//usr/bin/docker Package/conntrack File//etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.crt Package/apt-transport-https File//etc/kubernetes/manifests File//srv/kubernetes/known_tokens.csv File//opt/cni/bin/bridge File//home/ubuntu/.kube/ File//var/lib/kube-scheduler/kubeconfig File//opt/cni/bin/host-local File//etc/logrotate.d/etcd Package/libapparmor1 File//srv/kubernetes/service-account.key File//etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.crt File//etc/kubernetes/pki/kube-apiserver/etcd-client.key File//srv/kubernetes/kubelet-server.crt File//srv/kubernetes/kubelet-server.key File//etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.crt Package/python-apt File//usr/bin/dockerd File//var/lib/kubectl/kubeconfig Package/bridge-utils File//etc/kubernetes/kops-controller/kops-controller.crt File//etc/kubernetes/pki/kube-apiserver/etcd-client.crt File//usr/bin/containerd-shim File//usr/share/doc/docker/apache.txt File//etc/cni/net.d/ Package/pigz Package/socat File//srv/kubernetes/server.crt File//opt/cni/bin/flannel File//usr/bin/runc File//etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.key File//etc/sysctl.d/99-k8s-general.conf File//etc/kubernetes/kube-apiserver-healthcheck/secrets/client.crt Package/libseccomp2 File//etc/logrotate.d/kube-scheduler File//etc/kubernetes/manifests/kube-scheduler.manifest File//etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.crt File//var/log/kube-proxy.log File//opt/cni/bin/vlan File//usr/share/doc/containerd/apache.txt File//home/ubuntu/.kube/config File//srv/kubernetes/ca.key Package/cgroupfs-mount UserTask/kops-controller Package/wget File//etc/logrotate.d/kubelet File//srv/kubernetes/apiserver-aggregator-ca.crt File//srv/kubernetes/kubelet-api.crt File//etc/sysconfig/docker File//usr/local/bin/kubectl File//etc/logrotate.d/docker Package/iptables File//etc/kubernetes/pki/kube-apiserver/etcd-ca.crt File//etc/logrotate.d/kube-apiserver Package/nfs-common File//etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.key File//etc/systemd/timesyncd.conf File//srv/kubernetes/apiserver-aggregator.crt File//etc/kubernetes/kube-apiserver-healthcheck/secrets/ca.crt File//etc/kubernetes/manifests/kube-proxy.manifest File//etc/kubernetes/kops-controller/ca-key.pem File//opt/cni/bin/bandwidth Package/logrotate File//etc/kubernetes/manifests/etcd-main.manifest Package/ebtables File//opt/kops/bin/iptables-setup File//var/lib/kube-controller-manager/kubeconfig File//usr/bin/docker-proxy File//etc/logrotate.d/kube-proxy UserTask/kube-apiserver-healthcheck File//etc/kubernetes/pki/kube-apiserver File//opt/cni/bin/loopback Package/ethtool Package/libltdl7 File//etc/kubernetes/manifests/kube-controller-manager.manifest File//etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.key File//etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.key File//opt/cni/bin/portmap GroupTask/docker File//etc/kubernetes/kube-apiserver-healthcheck/secrets File//opt/cni/bin/sbr File//srv/kubernetes File//etc/logrotate.d/kube-controller-manager File//etc/kubernetes/manifests/etcd-events.manifest File//etc/kubernetes/kops-controller/kops-controller.key Package/curl File//srv/kubernetes/ca.crt File//srv/kubernetes/apiserver-aggregator.key File//usr/bin/docker-init File//usr/bin/containerd File//etc/logrotate.d/kube-addons File//etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.key File//etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.key File//opt/cni/bin/dhcp File//etc/sysconfig/kubelet Package/util-linux File//etc/kubernetes/kube-apiserver-healthcheck/secrets/client.key File//srv/kubernetes/kubelet-api.key File//etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.crt File//opt/cni/bin/host-device File//etc/kubernetes/cloud.config File//usr/local/bin/kubelet File//var/lib/kubelet File//var/log/kube-controller-manager.log File//etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.crt File//opt/cni/bin/firewall File//srv/kubernetes/assets File//etc/kubernetes/manifests/kube-apiserver.manifest File//etc/kubernetes/kops-controller/ca.pem Package/perl File//opt/cni/bin/macvlan UpdatePackages]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576848    1429 topological_sort.go:73]         File//etc/kubernetes/kops-controller/kops-controller.crt:        [UserTask/kops-controller File//etc/kubernetes/kops-controller IssueCert/kops-controller]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576854    1429 topological_sort.go:73]         File//srv/kubernetes/server.crt:        [File//srv/kubernetes IssueCert/master]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576860    1429 topological_sort.go:73]         File//etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.key:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576867    1429 topological_sort.go:73]         File//opt/cni/bin/host-local:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576871    1429 topological_sort.go:73]         IssueCert/kube-proxy:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576875    1429 topological_sort.go:73]         File//opt/cni/bin/bridge:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576880    1429 topological_sort.go:73]         File//srv/kubernetes/kubelet-server.crt:        [File//srv/kubernetes IssueCert/kubelet-server]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576885    1429 topological_sort.go:73]         File//usr/share/doc/containerd/apache.txt:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576889    1429 topological_sort.go:73]         UserTask/kops-controller:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576893    1429 topological_sort.go:73]         Package/perl:        [UpdatePackages]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576897    1429 topological_sort.go:73]         File//home/ubuntu/.kube/:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576902    1429 topological_sort.go:73]         File//etc/kubernetes/kops-controller/ca-key.pem:        [UserTask/kops-controller File//etc/kubernetes/kops-controller]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576906    1429 topological_sort.go:73]         File//etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.crt:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576911    1429 topological_sort.go:73]         File//etc/kubernetes/manifests/kube-scheduler.manifest:        [File//etc/kubernetes/manifests]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576915    1429 topological_sort.go:73]         IssueCert/kube-scheduler:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576919    1429 topological_sort.go:73]         IssueCert/kubelet-api:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576923    1429 topological_sort.go:73]         IssueCert/apiserver-aggregator:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576928    1429 topological_sort.go:73]         File//etc/sysconfig/kubelet:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576934    1429 topological_sort.go:73]         File//etc/kubernetes/cloud.config:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576939    1429 topological_sort.go:73]         Package/socat:        [UpdatePackages]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.576946    1429 topological_sort.go:73]         File//etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.key:        []
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577165    1429 executor.go:111] Tasks: 0 done / 163 total; 88 can run
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577244    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.crt": File: "/etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577313    1429 executor.go:186] Executing task "File//opt/cni/bin/loopback": File: "/opt/cni/bin/loopback"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577306    1429 executor.go:186] Executing task "File//opt/cni/bin/tuning": File: "/opt/cni/bin/tuning"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577363    1429 executor.go:186] Executing task "File//etc/logrotate.d/etcd-events": File: "/etc/logrotate.d/etcd-events"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577373    1429 executor.go:186] Executing task "IssueCert/kubelet": IssueCert: kubelet
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577386    1429 executor.go:186] Executing task "File//etc/logrotate.d/kube-proxy": File: "/etc/logrotate.d/kube-proxy"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577386    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.key": File: "/etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577409    1429 executor.go:186] Executing task "File//var/log/kube-apiserver.log": File: "/var/log/kube-apiserver.log"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577921    1429 executor.go:186] Executing task "File//opt/cni/bin/vlan": File: "/opt/cni/bin/vlan"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577924    1429 executor.go:186] Executing task "IssueCert/kube-controller-manager": IssueCert: kube-controller-manager
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577935    1429 executor.go:186] Executing task "IssueCert/master": IssueCert: master
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577951    1429 executor.go:186] Executing task "File//var/log/kube-controller-manager.log": File: "/var/log/kube-controller-manager.log"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577962    1429 files.go:58] Writing file "/opt/cni/bin/loopback"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577422    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/kube-apiserver": File: "/etc/kubernetes/pki/kube-apiserver"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578042    1429 files.go:58] Writing file "/var/log/kube-apiserver.log"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578128    1429 issue_cert.go:155] signing certificate for "master"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578135    1429 issue_cert.go:155] signing certificate for "kube-controller-manager"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577435    1429 executor.go:186] Executing task "File//srv/kubernetes": File: "/srv/kubernetes"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578191    1429 files.go:114] Changing file mode for "/var/log/kube-apiserver.log" to -r--------
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578226    1429 files.go:58] Writing file "/opt/cni/bin/tuning"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577473    1429 executor.go:186] Executing task "File//etc/logrotate.d/kubelet": File: "/etc/logrotate.d/kubelet"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578226    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578226    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578285    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.crt": File: "/etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578373    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578101    1429 files.go:58] Writing file "/var/log/kube-controller-manager.log"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578687    1429 files.go:114] Changing file mode for "/var/log/kube-controller-manager.log" to -r--------
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577476    1429 executor.go:186] Executing task "File//etc/kubernetes/kube-apiserver-healthcheck/secrets": File: "/etc/kubernetes/kube-apiserver-healthcheck/secrets"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577503    1429 executor.go:186] Executing task "File//usr/bin/docker": File: "/usr/bin/docker"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578990    1429 files.go:58] Writing file "/usr/bin/docker"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577489    1429 executor.go:186] Executing task "IssueCert/kube-apiserver-healthcheck": IssueCert: kube-apiserver-healthcheck
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577516    1429 executor.go:186] Executing task "File//etc/systemd/timesyncd.conf": File: "/etc/systemd/timesyncd.conf"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.579220    1429 issue_cert.go:155] signing certificate for "kube-apiserver-healthcheck"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.579270    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577449    1429 executor.go:186] Executing task "File//usr/bin/containerd": File: "/usr/bin/containerd"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.579538    1429 changes.go:81] Field changed "Contents" actual="#  This file is part of systemd.\n#\n#  systemd is free software; you can redistribute it and/or modify it\n#  under the terms of the GNU Lesser General Public License as published by\n#  the Free Software Foundation; either version 2.1 of the License, or\n#  (... (truncated)" expected="# Built by Kops - do NOT edit\n\n[Time]\nNTP=169.254.169.123\n"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577509    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.key": File: "/etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.579635    1429 files.go:58] Writing file "/etc/systemd/timesyncd.conf"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.579716    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.579575    1429 files.go:58] Writing file "/usr/bin/containerd"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.579818    1429 files.go:114] Changing file mode for "/etc/systemd/timesyncd.conf" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577542    1429 executor.go:186] Executing task "File//etc/sysconfig/kubelet": File: "/etc/sysconfig/kubelet"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577554    1429 executor.go:186] Executing task "File//usr/bin/ctr": File: "/usr/bin/ctr"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577566    1429 executor.go:186] Executing task "File//etc/kubernetes/kops-controller": File: "/etc/kubernetes/kops-controller"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577567    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.key": File: "/etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577543    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.crt": File: "/etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.579909    1429 files.go:58] Writing file "/etc/logrotate.d/etcd-events"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.579984    1429 files.go:58] Writing file "/etc/sysconfig/kubelet"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.580011    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.580063    1429 files.go:114] Changing file mode for "/etc/sysconfig/kubelet" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.580063    1429 files.go:114] Changing file mode for "/etc/logrotate.d/etcd-events" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577601    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.crt": File: "/etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.580140    1429 files.go:58] Writing file "/usr/bin/ctr"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.580176    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.580216    1429 files.go:58] Writing file "/etc/logrotate.d/kube-proxy"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577618    1429 executor.go:186] Executing task "File//var/lib/kubelet": File: "/var/lib/kubelet"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.580320    1429 files.go:114] Changing file mode for "/etc/logrotate.d/kube-proxy" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577630    1429 executor.go:186] Executing task "File//opt/cni/bin/host-local": File: "/opt/cni/bin/host-local"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.580438    1429 files.go:58] Writing file "/opt/cni/bin/host-local"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.580455    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-main/etcd-manager-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.583883    1429 files.go:114] Changing file mode for "/opt/cni/bin/loopback" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.583972    1429 files.go:114] Changing file mode for "/opt/cni/bin/tuning" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.584128    1429 files.go:114] Changing file mode for "/opt/cni/bin/host-local" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577625    1429 executor.go:186] Executing task "File//etc/kubernetes/cloud.config": File: "/etc/kubernetes/cloud.config"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.584300    1429 files.go:58] Writing file "/etc/kubernetes/cloud.config"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.584391    1429 files.go:114] Changing file mode for "/etc/kubernetes/cloud.config" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577564    1429 executor.go:186] Executing task "File//etc/sysconfig/docker": File: "/etc/sysconfig/docker"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.584453    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.584613    1429 files.go:58] Writing file "/etc/sysconfig/docker"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.584692    1429 files.go:114] Changing file mode for "/etc/sysconfig/docker" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577631    1429 executor.go:186] Executing task "File//opt/cni/bin/host-device": File: "/opt/cni/bin/host-device"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.584809    1429 files.go:58] Writing file "/opt/cni/bin/host-device"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578991    1429 files.go:58] Writing file "/etc/logrotate.d/kubelet"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.585339    1429 files.go:114] Changing file mode for "/etc/logrotate.d/kubelet" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577645    1429 issue_cert.go:155] signing certificate for "kubelet"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.585408    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577654    1429 executor.go:186] Executing task "File//opt/cni/bin/portmap": File: "/opt/cni/bin/portmap"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.586438    1429 files.go:58] Writing file "/opt/cni/bin/portmap"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.589343    1429 files.go:114] Changing file mode for "/opt/cni/bin/host-device" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577642    1429 executor.go:186] Executing task "File//usr/share/doc/docker/apache.txt": File: "/usr/share/doc/docker/apache.txt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577667    1429 executor.go:186] Executing task "File//usr/bin/docker-init": File: "/usr/bin/docker-init"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.590245    1429 files.go:58] Writing file "/usr/bin/docker-init"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577657    1429 executor.go:186] Executing task "UpdatePackages": UpdatePackages
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577646    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.key": File: "/etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.590687    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-main/etcd-peers-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577693    1429 executor.go:186] Executing task "File//opt/cni/bin/ipvlan": File: "/opt/cni/bin/ipvlan"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.590931    1429 files.go:58] Writing file "/opt/cni/bin/ipvlan"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591099    1429 files.go:114] Changing file mode for "/usr/bin/docker-init" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577707    1429 executor.go:186] Executing task "File//etc/logrotate.d/kube-apiserver": File: "/etc/logrotate.d/kube-apiserver"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591226    1429 files.go:58] Writing file "/etc/logrotate.d/kube-apiserver"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591299    1429 files.go:114] Changing file mode for "/etc/logrotate.d/kube-apiserver" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577701    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.key": File: "/etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591455    1429 files.go:114] Changing file mode for "/opt/cni/bin/portmap" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591476    1429 update_packages.go:82] running command [apt-get update]
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591485    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577721    1429 executor.go:186] Executing task "File//etc/sysconfig/containerd": File: "/etc/sysconfig/containerd"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591659    1429 files.go:58] Writing file "/etc/sysconfig/containerd"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591727    1429 files.go:114] Changing file mode for "/etc/sysconfig/containerd" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577708    1429 executor.go:186] Executing task "File//etc/logrotate.d/etcd": File: "/etc/logrotate.d/etcd"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591836    1429 files.go:58] Writing file "/etc/logrotate.d/etcd"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.591917    1429 files.go:114] Changing file mode for "/etc/logrotate.d/etcd" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577741    1429 executor.go:186] Executing task "File//opt/cni/bin/flannel": File: "/opt/cni/bin/flannel"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.592074    1429 files.go:58] Writing file "/opt/cni/bin/flannel"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577738    1429 executor.go:186] Executing task "IssueCert/kube-scheduler": IssueCert: kube-scheduler
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.592717    1429 issue_cert.go:155] signing certificate for "kube-scheduler"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.592800    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.593073    1429 files.go:58] Writing file "/usr/share/doc/docker/apache.txt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.593236    1429 files.go:114] Changing file mode for "/usr/share/doc/docker/apache.txt" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577754    1429 executor.go:186] Executing task "File//etc/logrotate.d/kube-scheduler": File: "/etc/logrotate.d/kube-scheduler"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577697    1429 executor.go:186] Executing task "File//usr/share/doc/containerd/apache.txt": File: "/usr/share/doc/containerd/apache.txt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.593887    1429 files.go:58] Writing file "/etc/logrotate.d/kube-scheduler"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577753    1429 executor.go:186] Executing task "IssueCert/etcd-client": IssueCert: etcd-client
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.594009    1429 files.go:114] Changing file mode for "/etc/logrotate.d/kube-scheduler" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.594190    1429 issue_cert.go:155] signing certificate for "etcd-client"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.594238    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/etcd-clients-ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577765    1429 executor.go:186] Executing task "IssueCert/kops": IssueCert: kops
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.594892    1429 issue_cert.go:155] signing certificate for "kops"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.594928    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577766    1429 executor.go:186] Executing task "File//etc/cni/net.d/": File: "/etc/cni/net.d/"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577663    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.crt": File: "/etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.595375    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-main/etcd-clients-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.595435    1429 files.go:114] Changing file mode for "/opt/cni/bin/ipvlan" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.595456    1429 files.go:58] Writing file "/usr/share/doc/containerd/apache.txt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577765    1429 executor.go:186] Executing task "File//usr/local/bin/kubectl": File: "/usr/local/bin/kubectl"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.595566    1429 files.go:114] Changing file mode for "/usr/share/doc/containerd/apache.txt" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.595574    1429 files.go:58] Writing file "/usr/local/bin/kubectl"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577770    1429 executor.go:186] Executing task "File//usr/bin/dockerd": File: "/usr/bin/dockerd"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577780    1429 executor.go:186] Executing task "File//home/ubuntu/.kube/": File: "/home/ubuntu/.kube/"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577779    1429 executor.go:186] Executing task "IssueCert/kubecfg": IssueCert: kubecfg
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.596072    1429 files_owner.go:68] Changing file owner/group for "/home/ubuntu/.kube/" to ubuntu:ubuntu
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.596195    1429 files.go:58] Writing file "/usr/bin/dockerd"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.596525    1429 issue_cert.go:155] signing certificate for "kubecfg"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.596626    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577779    1429 executor.go:186] Executing task "IssueCert/kubelet-server": IssueCert: kubelet-server
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.597411    1429 issue_cert.go:155] signing certificate for "kubelet-server"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.597473    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577771    1429 executor.go:186] Executing task "File//etc/sysctl.d/99-k8s-general.conf": File: "/etc/sysctl.d/99-k8s-general.conf"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577775    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-events/etcd-manager-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577793    1429 executor.go:186] Executing task "File//usr/local/bin/kubelet": File: "/usr/local/bin/kubelet"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.598139    1429 files.go:58] Writing file "/usr/local/bin/kubelet"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577785    1429 executor.go:186] Executing task "File//etc/logrotate.d/kube-controller-manager": File: "/etc/logrotate.d/kube-controller-manager"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.598280    1429 files.go:58] Writing file "/etc/logrotate.d/kube-controller-manager"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.598369    1429 files.go:114] Changing file mode for "/etc/logrotate.d/kube-controller-manager" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577796    1429 executor.go:186] Executing task "File//etc/apt/apt.conf.d/20auto-upgrades": File: "/etc/apt/apt.conf.d/20auto-upgrades"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577798    1429 executor.go:186] Executing task "File//opt/cni/bin/ptp": File: "/opt/cni/bin/ptp"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577805    1429 executor.go:186] Executing task "File//opt/cni/bin/static": File: "/opt/cni/bin/static"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.598801    1429 files.go:58] Writing file "/opt/cni/bin/static"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.598126    1429 files.go:114] Changing file mode for "/opt/cni/bin/flannel" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577812    1429 executor.go:186] Executing task "IssueCert/kube-proxy": IssueCert: kube-proxy
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.599372    1429 issue_cert.go:155] signing certificate for "kube-proxy"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577815    1429 executor.go:186] Executing task "File//etc/containerd/config-kops.toml": File: "/etc/containerd/config-kops.toml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577836    1429 executor.go:186] Executing task "File//etc/logrotate.d/kube-addons": File: "/etc/logrotate.d/kube-addons"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577843    1429 executor.go:186] Executing task "File//opt/cni/bin/bridge": File: "/opt/cni/bin/bridge"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.600398    1429 files.go:58] Writing file "/opt/cni/bin/bridge"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.600538    1429 files.go:114] Changing file mode for "/usr/bin/ctr" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577839    1429 executor.go:186] Executing task "File//opt/kops/bin/iptables-setup": File: "/opt/kops/bin/iptables-setup"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577856    1429 executor.go:186] Executing task "File//opt/cni/bin/dhcp": File: "/opt/cni/bin/dhcp"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.600958    1429 files.go:58] Writing file "/opt/cni/bin/ptp"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.600977    1429 files.go:58] Writing file "/opt/cni/bin/dhcp"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577851    1429 executor.go:186] Executing task "IssueCert/kubelet-api": IssueCert: kubelet-api
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577856    1429 executor.go:186] Executing task "File//opt/cni/bin/bandwidth": File: "/opt/cni/bin/bandwidth"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.601659    1429 files.go:58] Writing file "/opt/cni/bin/bandwidth"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.598693    1429 files.go:58] Writing file "/etc/sysctl.d/99-k8s-general.conf"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577856    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.key": File: "/etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577873    1429 executor.go:186] Executing task "IssueCert/kops-controller": IssueCert: kops-controller
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577873    1429 executor.go:186] Executing task "File//opt/cni/bin/firewall": File: "/opt/cni/bin/firewall"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577874    1429 executor.go:186] Executing task "UserTask/kube-apiserver-healthcheck": User: kube-apiserver-healthcheck
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577864    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.crt": File: "/etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577870    1429 executor.go:186] Executing task "File//usr/bin/containerd-shim": File: "/usr/bin/containerd-shim"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606787    1429 files.go:58] Writing file "/opt/cni/bin/firewall"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606871    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-events/etcd-clients-ca.key"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606881    1429 files.go:114] Changing file mode for "/etc/sysctl.d/99-k8s-general.conf" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606910    1429 file.go:267] Changed; will execute OnChangeExecute command: "sysctl --system"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.607164    1429 user.go:97] Creating user "kube-apiserver-healthcheck"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577885    1429 executor.go:186] Executing task "IssueCert/apiserver-aggregator": IssueCert: apiserver-aggregator
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577888    1429 executor.go:186] Executing task "File//opt/cni/bin/sbr": File: "/opt/cni/bin/sbr"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577895    1429 executor.go:186] Executing task "File//var/lib/kube-scheduler/config.yaml": File: "/var/lib/kube-scheduler/config.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.607648    1429 files.go:114] Changing file mode for "/opt/cni/bin/bandwidth" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.607684    1429 issue_cert.go:155] signing certificate for "apiserver-aggregator"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577887    1429 executor.go:186] Executing task "File//usr/bin/runc": File: "/usr/bin/runc"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577869    1429 executor.go:186] Executing task "GroupTask/docker": Group: docker
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577900    1429 executor.go:186] Executing task "File//var/log/kube-proxy.log": File: "/var/log/kube-proxy.log"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577906    1429 executor.go:186] Executing task "File//etc/kubernetes/manifests": File: "/etc/kubernetes/manifests"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.607725    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/apiserver-aggregator-ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.578015    1429 files.go:58] Writing file "/opt/cni/bin/vlan"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577530    1429 executor.go:186] Executing task "File//opt/cni/bin/macvlan": File: "/opt/cni/bin/macvlan"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577614    1429 executor.go:186] Executing task "File//var/log/kube-scheduler.log": File: "/var/log/kube-scheduler.log"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577535    1429 executor.go:186] Executing task "File//etc/logrotate.d/docker": File: "/etc/logrotate.d/docker"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577765    1429 executor.go:186] Executing task "File//usr/bin/docker-proxy": File: "/usr/bin/docker-proxy"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606417    1429 files.go:114] Changing file mode for "/opt/cni/bin/ptp" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.577811    1429 executor.go:186] Executing task "UserTask/kops-controller": User: kops-controller
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606662    1429 issue_cert.go:155] signing certificate for "kubelet-api"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.607837    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606702    1429 files.go:58] Writing file "/opt/kops/bin/iptables-setup"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.608053    1429 files.go:114] Changing file mode for "/opt/kops/bin/iptables-setup" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606722    1429 files.go:58] Writing file "/etc/containerd/config-kops.toml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606733    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.608172    1429 files.go:114] Changing file mode for "/etc/containerd/config-kops.toml" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606747    1429 files.go:114] Changing file mode for "/opt/cni/bin/static" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.606765    1429 files.go:58] Writing file "/etc/logrotate.d/kube-addons"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.608545    1429 files.go:114] Changing file mode for "/etc/logrotate.d/kube-addons" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.613942    1429 files.go:114] Changing file mode for "/opt/cni/bin/firewall" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.614133    1429 issue_cert.go:155] signing certificate for "kops-controller"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.614211    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/issued/ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.623481    1429 files.go:58] Writing file "/etc/kubernetes/pki/etcd-manager-events/etcd-peers-ca.crt"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.624125    1429 changes.go:81] Field changed "Contents" actual="APT::Periodic::Update-Package-Lists \"1\";\nAPT::Periodic::Unattended-Upgrade \"1\";\n" expected="APT::Periodic::Update-Package-Lists \"1\";\nAPT::Periodic::Unattended-Upgrade \"1\";\n\nAPT::Periodic::AutocleanInterval \"7\";\n"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.624351    1429 files.go:58] Writing file "/etc/apt/apt.conf.d/20auto-upgrades"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.624683    1429 files.go:114] Changing file mode for "/etc/apt/apt.conf.d/20auto-upgrades" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.624709    1429 files.go:58] Writing file "/usr/bin/containerd-shim"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.624949    1429 files.go:58] Writing file "/opt/cni/bin/sbr"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.625215    1429 files.go:58] Writing file "/var/lib/kube-scheduler/config.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.625265    1429 files.go:58] Writing file "/usr/bin/runc"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.625156    1429 files.go:114] Changing file mode for "/opt/cni/bin/dhcp" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.624728    1429 user.go:99] running command: useradd -u 10012 -s /sbin/nologin -d /etc/kubernetes/kube-apiserver-healthcheck/secrets kube-apiserver-healthcheck
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.625081    1429 files.go:114] Changing file mode for "/opt/cni/bin/bridge" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.625774    1429 group.go:94] Creating group "docker"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.625846    1429 group.go:96] running command: groupadd --system docker
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.625973    1429 files.go:58] Writing file "/var/log/kube-proxy.log"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.626099    1429 files.go:114] Changing file mode for "/var/log/kube-proxy.log" to -r--------
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.629568    1429 files.go:114] Changing file mode for "/opt/cni/bin/sbr" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.629918    1429 files.go:58] Writing file "/opt/cni/bin/macvlan"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.630970    1429 files.go:114] Changing file mode for "/opt/cni/bin/vlan" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.631166    1429 files.go:58] Writing file "/var/log/kube-scheduler.log"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.631287    1429 files.go:114] Changing file mode for "/var/log/kube-scheduler.log" to -r--------
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.631399    1429 files.go:58] Writing file "/etc/logrotate.d/docker"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.631479    1429 files.go:114] Changing file mode for "/etc/logrotate.d/docker" to -rw-r--r--
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.631576    1429 files.go:58] Writing file "/usr/bin/docker-proxy"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.634497    1429 files.go:114] Changing file mode for "/usr/bin/containerd-shim" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.634828    1429 files.go:114] Changing file mode for "/usr/bin/docker-proxy" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.635064    1429 user.go:97] Creating user "kops-controller"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.635276    1429 user.go:99] running command: useradd -u 10011 -s /sbin/nologin kops-controller
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.635431    1429 files.go:114] Changing file mode for "/opt/cni/bin/macvlan" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.635344    1429 files.go:114] Changing file mode for "/usr/bin/runc" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.637170    1429 files.go:114] Changing file mode for "/usr/bin/containerd" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.638653    1429 files.go:114] Changing file mode for "/var/lib/kube-scheduler/config.yaml" to -r--------
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.640928    1429 files.go:114] Changing file mode for "/usr/bin/docker" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** useradd[1458]: new group: name=kube-apiserver-healthcheck, GID=10012
Mar 14 16:30:09 ip-************** useradd[1458]: new user: name=kube-apiserver-healthcheck, UID=10012, GID=10012, home=/etc/kubernetes/kube-apiserver-healthcheck/secrets, shell=/sbin/nologin, from=none
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.660944    1429 files.go:114] Changing file mode for "/usr/local/bin/kubectl" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.665740    1429 files.go:114] Changing file mode for "/usr/bin/dockerd" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.688222    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/private/etcd-clients-ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.693578    1429 s3fs.go:290] Reading file "s3://state.**********************/**********************/pki/private/apiserver-aggregator-ca/keyset.yaml"
Mar 14 16:30:09 ip-************** nodeup[1429]: I0314 16:30:09.731297    1429 files.go:114] Changing file mode for "/usr/local/bin/kubelet" to -rwxr-xr-x
Mar 14 16:30:09 ip-************** groupadd[1459]: group added to /etc/group: name=docker, GID=998
Mar 14 16:30:09 ip-************** groupadd[1459]: group added to /etc/gshadow: name=docker
Mar 14 16:30:09 ip-************** groupadd[1459]: new group: name=docker, GID=998
Mar 14 16:30:09 ip-************** useradd[1460]: new group: name=kops-controller, GID=10011
Mar 14 16:30:09 ip-************** useradd[1460]: new user: name=kops-controller, UID=10011, GID=10011, home=/home/kops-controller, shell=/sbin/nologin, from=none
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126252    1429 executor.go:111] Tasks: 88 done / 163 total; 58 can run
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126395    1429 executor.go:186] Executing task "File//etc/kubernetes/manifests/etcd-main.manifest": File: "/etc/kubernetes/manifests/etcd-main.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126420    1429 executor.go:186] Executing task "File//srv/kubernetes/kubelet-api.key": File: "/srv/kubernetes/kubelet-api.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126426    1429 executor.go:186] Executing task "Package/bridge-utils": Package: bridge-utils
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126445    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/kube-apiserver/etcd-client.key": File: "/etc/kubernetes/pki/kube-apiserver/etcd-client.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126487    1429 executor.go:186] Executing task "File//srv/kubernetes/ca.key": File: "/srv/kubernetes/ca.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126522    1429 executor.go:186] Executing task "File//srv/kubernetes/kubelet-api.crt": File: "/srv/kubernetes/kubelet-api.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126533    1429 files.go:58] Writing file "/srv/kubernetes/kubelet-api.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126565    1429 executor.go:186] Executing task "File//etc/kubernetes/manifests/kube-proxy.manifest": File: "/etc/kubernetes/manifests/kube-proxy.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126589    1429 files.go:58] Writing file "/etc/kubernetes/pki/kube-apiserver/etcd-client.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126594    1429 executor.go:186] Executing task "File//srv/kubernetes/server.key": File: "/srv/kubernetes/server.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126616    1429 files.go:58] Writing file "/srv/kubernetes/ca.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126623    1429 executor.go:186] Executing task "Package/perl": Package: perl
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126634    1429 executor.go:186] Executing task "Package/conntrack": Package: conntrack
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126644    1429 files.go:58] Writing file "/etc/kubernetes/manifests/kube-proxy.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126693    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W conntrack
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126707    1429 executor.go:186] Executing task "File//srv/kubernetes/service-account.key": File: "/srv/kubernetes/service-account.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126533    1429 files.go:58] Writing file "/etc/kubernetes/manifests/etcd-main.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126727    1429 files.go:58] Writing file "/srv/kubernetes/server.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126737    1429 executor.go:186] Executing task "File//srv/kubernetes/assets": File: "/srv/kubernetes/assets"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126745    1429 executor.go:186] Executing task "File//etc/kubernetes/kops-controller/kops-controller.key": File: "/etc/kubernetes/kops-controller/kops-controller.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126764    1429 executor.go:186] Executing task "Package/cgroupfs-mount": Package: cgroupfs-mount
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126786    1429 files.go:114] Changing file mode for "/etc/kubernetes/manifests/etcd-main.manifest" to -r--r-----
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126803    1429 executor.go:186] Executing task "Package/pigz": Package: pigz
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126818    1429 files.go:58] Writing file "/etc/kubernetes/kops-controller/kops-controller.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126820    1429 executor.go:186] Executing task "Package/socat": Package: socat
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126827    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W cgroupfs-mount
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126853    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W pigz
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126860    1429 executor.go:186] Executing task "File//srv/kubernetes/kubelet-server.key": File: "/srv/kubernetes/kubelet-server.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126620    1429 files.go:58] Writing file "/srv/kubernetes/kubelet-api.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126880    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W socat
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126887    1429 executor.go:186] Executing task "Package/libapparmor1": Package: libapparmor1
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126788    1429 files.go:58] Writing file "/srv/kubernetes/service-account.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126934    1429 files.go:58] Writing file "/srv/kubernetes/kubelet-server.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126936    1429 executor.go:186] Executing task "Package/netcat-traditional": Package: netcat-traditional
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126943    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W libapparmor1
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126950    1429 files.go:114] Changing file mode for "/srv/kubernetes/kubelet-api.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126719    1429 files.go:114] Changing file mode for "/etc/kubernetes/manifests/kube-proxy.manifest" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126986    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W netcat-traditional
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126982    1429 executor.go:186] Executing task "File//etc/kubernetes/manifests/kube-scheduler.manifest": File: "/etc/kubernetes/manifests/kube-scheduler.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126992    1429 files_owner.go:68] Changing file owner/group for "/etc/kubernetes/kops-controller/kops-controller.key" to kops-controller:
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126999    1429 executor.go:186] Executing task "Package/python-apt": Package: python-apt
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127018    1429 executor.go:186] Executing task "KubeConfig/kube-scheduler": KubeConfig: kube-scheduler
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127043    1429 executor.go:186] Executing task "KubeConfig/kops": KubeConfig: kops
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127058    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W python-apt
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127061    1429 executor.go:186] Executing task "File//srv/kubernetes/server.crt": File: "/srv/kubernetes/server.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127073    1429 executor.go:186] Executing task "File//etc/kubernetes/kops-controller/kops-controller.crt": File: "/etc/kubernetes/kops-controller/kops-controller.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127092    1429 executor.go:186] Executing task "File//srv/kubernetes/ca.crt": File: "/srv/kubernetes/ca.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127109    1429 executor.go:186] Executing task "Package/iptables": Package: iptables
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127134    1429 files.go:58] Writing file "/srv/kubernetes/server.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127142    1429 executor.go:186] Executing task "File//etc/kubernetes/kube-apiserver-healthcheck/secrets/ca.crt": File: "/etc/kubernetes/kube-apiserver-healthcheck/secrets/ca.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127153    1429 files.go:58] Writing file "/srv/kubernetes/ca.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127164    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W iptables
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127161    1429 executor.go:186] Executing task "File//srv/kubernetes/apiserver-aggregator-ca.crt": File: "/srv/kubernetes/apiserver-aggregator-ca.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126998    1429 executor.go:186] Executing task "KubeConfig/kube-proxy": KubeConfig: kube-proxy
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127207    1429 files.go:114] Changing file mode for "/srv/kubernetes/server.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127224    1429 files.go:58] Writing file "/etc/kubernetes/kube-apiserver-healthcheck/secrets/ca.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126746    1429 executor.go:186] Executing task "Package/apt-transport-https": Package: apt-transport-https
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127241    1429 files.go:58] Writing file "/srv/kubernetes/apiserver-aggregator-ca.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127257    1429 executor.go:186] Executing task "KubeConfig/kube-controller-manager": KubeConfig: kube-controller-manager
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127284    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W apt-transport-https
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127309    1429 files.go:114] Changing file mode for "/etc/kubernetes/kube-apiserver-healthcheck/secrets/ca.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127316    1429 files.go:114] Changing file mode for "/srv/kubernetes/apiserver-aggregator-ca.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127239    1429 executor.go:186] Executing task "File//etc/kubernetes/kops-controller/ca.pem": File: "/etc/kubernetes/kops-controller/ca.pem"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127351    1429 executor.go:186] Executing task "Package/unattended-upgrades": Package: unattended-upgrades
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127382    1429 files_owner.go:68] Changing file owner/group for "/etc/kubernetes/kube-apiserver-healthcheck/secrets/ca.crt" to kube-apiserver-healthcheck:
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127404    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W unattended-upgrades
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127419    1429 executor.go:186] Executing task "File//etc/kubernetes/kube-apiserver-healthcheck/secrets/client.key": File: "/etc/kubernetes/kube-apiserver-healthcheck/secrets/client.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126609    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W bridge-utils
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127062    1429 files.go:58] Writing file "/etc/kubernetes/manifests/kube-scheduler.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127403    1429 files.go:58] Writing file "/etc/kubernetes/kops-controller/ca.pem"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127499    1429 files.go:58] Writing file "/etc/kubernetes/kube-apiserver-healthcheck/secrets/client.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127519    1429 files.go:114] Changing file mode for "/etc/kubernetes/manifests/kube-scheduler.manifest" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126390    1429 executor.go:186] Executing task "Package/libltdl7": Package: libltdl7
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127586    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W libltdl7
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127605    1429 files_owner.go:68] Changing file owner/group for "/etc/kubernetes/kops-controller/ca.pem" to kops-controller:
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127076    1429 executor.go:186] Executing task "File//etc/kubernetes/kops-controller/ca-key.pem": File: "/etc/kubernetes/kops-controller/ca-key.pem"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127652    1429 files_owner.go:68] Changing file owner/group for "/etc/kubernetes/kube-apiserver-healthcheck/secrets/client.key" to kube-apiserver-healthcheck:
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127683    1429 executor.go:186] Executing task "KubeConfig/kubecfg": KubeConfig: kubecfg
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127690    1429 files.go:58] Writing file "/etc/kubernetes/kops-controller/ca-key.pem"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127724    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/kube-apiserver/etcd-client.crt": File: "/etc/kubernetes/pki/kube-apiserver/etcd-client.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127797    1429 files.go:58] Writing file "/etc/kubernetes/pki/kube-apiserver/etcd-client.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127812    1429 files_owner.go:68] Changing file owner/group for "/etc/kubernetes/kops-controller/ca-key.pem" to kops-controller:
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127902    1429 executor.go:186] Executing task "Package/logrotate": Package: logrotate
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127927    1429 files.go:114] Changing file mode for "/etc/kubernetes/pki/kube-apiserver/etcd-client.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127956    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W logrotate
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127961    1429 executor.go:186] Executing task "Package/curl": Package: curl
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128015    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W curl
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128024    1429 executor.go:186] Executing task "Package/nfs-common": Package: nfs-common
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128067    1429 executor.go:186] Executing task "File//srv/kubernetes/kubelet-server.crt": File: "/srv/kubernetes/kubelet-server.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128113    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W nfs-common
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128146    1429 files.go:58] Writing file "/srv/kubernetes/kubelet-server.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128169    1429 executor.go:186] Executing task "File//etc/kubernetes/manifests/kube-apiserver.manifest": File: "/etc/kubernetes/manifests/kube-apiserver.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.127148    1429 files.go:58] Writing file "/etc/kubernetes/kops-controller/kops-controller.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128229    1429 files.go:114] Changing file mode for "/srv/kubernetes/kubelet-server.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128244    1429 files.go:58] Writing file "/etc/kubernetes/manifests/kube-apiserver.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128245    1429 executor.go:186] Executing task "File//srv/kubernetes/apiserver-aggregator.crt": File: "/srv/kubernetes/apiserver-aggregator.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128261    1429 executor.go:186] Executing task "Package/ethtool": Package: ethtool
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128299    1429 files.go:114] Changing file mode for "/etc/kubernetes/kops-controller/kops-controller.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128309    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W ethtool
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128318    1429 files.go:58] Writing file "/srv/kubernetes/apiserver-aggregator.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128337    1429 executor.go:186] Executing task "File//etc/kubernetes/manifests/etcd-events.manifest": File: "/etc/kubernetes/manifests/etcd-events.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128359    1429 executor.go:186] Executing task "Package/wget": Package: wget
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128384    1429 files_owner.go:68] Changing file owner/group for "/etc/kubernetes/kops-controller/kops-controller.crt" to kops-controller:
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128412    1429 files.go:114] Changing file mode for "/srv/kubernetes/apiserver-aggregator.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128430    1429 executor.go:186] Executing task "Package/libseccomp2": Package: libseccomp2
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128432    1429 executor.go:186] Executing task "File//srv/kubernetes/apiserver-aggregator.key": File: "/srv/kubernetes/apiserver-aggregator.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128446    1429 executor.go:186] Executing task "File//etc/kubernetes/kube-apiserver-healthcheck/secrets/client.crt": File: "/etc/kubernetes/kube-apiserver-healthcheck/secrets/client.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128472    1429 executor.go:186] Executing task "File//etc/kubernetes/manifests/kube-controller-manager.manifest": File: "/etc/kubernetes/manifests/kube-controller-manager.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128501    1429 executor.go:186] Executing task "Package/ebtables": Package: ebtables
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128516    1429 executor.go:186] Executing task "File//srv/kubernetes/known_tokens.csv": File: "/srv/kubernetes/known_tokens.csv"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128523    1429 files.go:58] Writing file "/etc/kubernetes/kube-apiserver-healthcheck/secrets/client.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128555    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W ebtables
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128576    1429 files.go:58] Writing file "/etc/kubernetes/manifests/kube-controller-manager.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128589    1429 files.go:58] Writing file "/srv/kubernetes/known_tokens.csv"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128503    1429 files.go:58] Writing file "/srv/kubernetes/apiserver-aggregator.key"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128606    1429 files.go:114] Changing file mode for "/etc/kubernetes/kube-apiserver-healthcheck/secrets/client.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128649    1429 files.go:114] Changing file mode for "/etc/kubernetes/manifests/kube-controller-manager.manifest" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128417    1429 files.go:58] Writing file "/etc/kubernetes/manifests/etcd-events.manifest"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128674    1429 executor.go:186] Executing task "KubeConfig/kubelet": KubeConfig: kubelet
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128679    1429 files_owner.go:68] Changing file owner/group for "/etc/kubernetes/kube-apiserver-healthcheck/secrets/client.crt" to kube-apiserver-healthcheck:
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128484    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W libseccomp2
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128706    1429 executor.go:186] Executing task "Package/git": Package: git
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128745    1429 files.go:114] Changing file mode for "/etc/kubernetes/manifests/etcd-events.manifest" to -r--r-----
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126693    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W perl
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128320    1429 files.go:114] Changing file mode for "/etc/kubernetes/manifests/kube-apiserver.manifest" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.126543    1429 executor.go:186] Executing task "File//etc/kubernetes/pki/kube-apiserver/etcd-ca.crt": File: "/etc/kubernetes/pki/kube-apiserver/etcd-ca.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.129571    1429 files.go:58] Writing file "/etc/kubernetes/pki/kube-apiserver/etcd-ca.crt"
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.129659    1429 files.go:114] Changing file mode for "/etc/kubernetes/pki/kube-apiserver/etcd-ca.crt" to -rw-r--r--
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.129701    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W git
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.128420    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W wget
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.129786    1429 executor.go:186] Executing task "Package/util-linux": Package: util-linux
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.129857    1429 package.go:152] Listing installed packages: dpkg-query -f ${db:Status-Abbrev}${Version}\n -W util-linux
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.141439    1429 package.go:282] Installing package "conntrack" (dependencies: [])
Mar 14 16:30:19 ip-************** nodeup[1429]: I0314 16:30:19.141500    1429 package.go:340] running command [apt-get install --yes --no-install-recommends conntrack]
Mar 14 16:30:23 ip-************** nodeup[1429]: I0314 16:30:23.542337    1429 package.go:282] Installing package "libseccomp2" (dependencies: [])
Mar 14 16:30:23 ip-************** nodeup[1429]: I0314 16:30:23.542388    1429 package.go:340] running command [apt-get install --yes --no-install-recommends libseccomp2]
Mar 14 16:30:24 ip-************** nodeup[1429]: I0314 16:30:24.008528    1429 package.go:282] Installing package "perl" (dependencies: [])
Mar 14 16:30:24 ip-************** nodeup[1429]: I0314 16:30:24.008571    1429 package.go:340] running command [apt-get install --yes --no-install-recommends perl]
Mar 14 16:30:24 ip-************** nodeup[1429]: I0314 16:30:24.489323    1429 package.go:282] Installing package "pigz" (dependencies: [])
Mar 14 16:30:24 ip-************** nodeup[1429]: I0314 16:30:24.489361    1429 package.go:340] running command [apt-get install --yes --no-install-recommends pigz]
Mar 14 16:30:28 ip-************** nodeup[1429]: I0314 16:30:28.325697    1429 package.go:282] Installing package "socat" (dependencies: [])
Mar 14 16:30:28 ip-************** nodeup[1429]: I0314 16:30:28.325741    1429 package.go:340] running command [apt-get install --yes --no-install-recommends socat]
Mar 14 16:30:32 ip-************** nodeup[1429]: I0314 16:30:32.116500    1429 package.go:282] Installing package "libapparmor1" (dependencies: [])
Mar 14 16:30:32 ip-************** nodeup[1429]: I0314 16:30:32.116543    1429 package.go:340] running command [apt-get install --yes --no-install-recommends libapparmor1]
Mar 14 16:30:32 ip-************** nodeup[1429]: I0314 16:30:32.601008    1429 package.go:282] Installing package "netcat-traditional" (dependencies: [])
Mar 14 16:30:32 ip-************** nodeup[1429]: I0314 16:30:32.601048    1429 package.go:340] running command [apt-get install --yes --no-install-recommends netcat-traditional]
Mar 14 16:30:36 ip-************** nodeup[1429]: I0314 16:30:36.382925    1429 package.go:282] Installing package "iptables" (dependencies: [])
Mar 14 16:30:36 ip-************** nodeup[1429]: I0314 16:30:36.382966    1429 package.go:340] running command [apt-get install --yes --no-install-recommends iptables]
Mar 14 16:30:36 ip-************** nodeup[1429]: I0314 16:30:36.864211    1429 package.go:282] Installing package "python-apt" (dependencies: [])
Mar 14 16:30:36 ip-************** nodeup[1429]: I0314 16:30:36.864250    1429 package.go:340] running command [apt-get install --yes --no-install-recommends python-apt]
Mar 14 16:30:49 ip-************** nodeup[1429]: I0314 16:30:49.375185    1429 package.go:282] Installing package "apt-transport-https" (dependencies: [])
Mar 14 16:30:49 ip-************** nodeup[1429]: I0314 16:30:49.375232    1429 package.go:340] running command [apt-get install --yes --no-install-recommends apt-transport-https]
Mar 14 16:30:52 ip-************** nodeup[1429]: I0314 16:30:52.876509    1429 package.go:282] Installing package "unattended-upgrades" (dependencies: [])
Mar 14 16:30:52 ip-************** nodeup[1429]: I0314 16:30:52.876551    1429 package.go:340] running command [apt-get install --yes --no-install-recommends unattended-upgrades]
Mar 14 16:30:53 ip-************** nodeup[1429]: I0314 16:30:53.346730    1429 package.go:282] Installing package "bridge-utils" (dependencies: [])
Mar 14 16:30:53 ip-************** nodeup[1429]: I0314 16:30:53.346778    1429 package.go:340] running command [apt-get install --yes --no-install-recommends bridge-utils]
Mar 14 16:30:57 ip-************** nodeup[1429]: I0314 16:30:57.059665    1429 package.go:282] Installing package "libltdl7" (dependencies: [])
Mar 14 16:30:57 ip-************** nodeup[1429]: I0314 16:30:57.059706    1429 package.go:340] running command [apt-get install --yes --no-install-recommends libltdl7]
Mar 14 16:30:57 ip-************** nodeup[1429]: I0314 16:30:57.525817    1429 package.go:282] Installing package "logrotate" (dependencies: [])
Mar 14 16:30:57 ip-************** nodeup[1429]: I0314 16:30:57.525857    1429 package.go:340] running command [apt-get install --yes --no-install-recommends logrotate]
Mar 14 16:30:57 ip-************** nodeup[1429]: I0314 16:30:57.995021    1429 package.go:282] Installing package "curl" (dependencies: [])
Mar 14 16:30:57 ip-************** nodeup[1429]: I0314 16:30:57.995065    1429 package.go:340] running command [apt-get install --yes --no-install-recommends curl]
Mar 14 16:30:58 ip-************** nodeup[1429]: I0314 16:30:58.458332    1429 package.go:282] Installing package "nfs-common" (dependencies: [])
Mar 14 16:30:58 ip-************** nodeup[1429]: I0314 16:30:58.458383    1429 package.go:340] running command [apt-get install --yes --no-install-recommends nfs-common]
Mar 14 16:30:59 ip-************** useradd[3458]: new user: name=_rpc, UID=113, GID=65534, home=/run/rpcbind, shell=/usr/sbin/nologin, from=none
Mar 14 16:30:59 ip-************** usermod[3466]: change user '_rpc' password
Mar 14 16:31:01 ip-************** useradd[3830]: new user: name=statd, UID=114, GID=65534, home=/var/lib/nfs, shell=/usr/sbin/nologin, from=none
Mar 14 16:31:01 ip-************** usermod[3838]: change user 'statd' password
Mar 14 16:31:01 ip-************** chage[3845]: changed password expiry for statd
Mar 14 16:31:06 ip-************** nodeup[1429]: I0314 16:31:06.296826    1429 package.go:282] Installing package "ethtool" (dependencies: [])
Mar 14 16:31:06 ip-************** nodeup[1429]: I0314 16:31:06.296868    1429 package.go:340] running command [apt-get install --yes --no-install-recommends ethtool]
Mar 14 16:31:06 ip-************** nodeup[1429]: I0314 16:31:06.747981    1429 package.go:282] Installing package "ebtables" (dependencies: [])
Mar 14 16:31:06 ip-************** nodeup[1429]: I0314 16:31:06.748022    1429 package.go:340] running command [apt-get install --yes --no-install-recommends ebtables]
Mar 14 16:31:10 ip-************** nodeup[1429]: I0314 16:31:10.712850    1429 package.go:282] Installing package "wget" (dependencies: [])
Mar 14 16:31:10 ip-************** nodeup[1429]: I0314 16:31:10.712895    1429 package.go:340] running command [apt-get install --yes --no-install-recommends wget]
Mar 14 16:31:11 ip-************** nodeup[1429]: I0314 16:31:11.178003    1429 package.go:282] Installing package "cgroupfs-mount" (dependencies: [])
Mar 14 16:31:11 ip-************** nodeup[1429]: I0314 16:31:11.178044    1429 package.go:340] running command [apt-get install --yes --no-install-recommends cgroupfs-mount]
Mar 14 16:31:15 ip-************** nodeup[1429]: I0314 16:31:15.364922    1429 package.go:282] Installing package "git" (dependencies: [])
Mar 14 16:31:15 ip-************** nodeup[1429]: I0314 16:31:15.364974    1429 package.go:340] running command [apt-get install --yes --no-install-recommends git]
Mar 14 16:31:19 ip-************** nodeup[1429]: I0314 16:31:19.765013    1429 package.go:282] Installing package "util-linux" (dependencies: [])
Mar 14 16:31:19 ip-************** nodeup[1429]: I0314 16:31:19.765075    1429 package.go:340] running command [apt-get install --yes --no-install-recommends util-linux]
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327187    1429 executor.go:111] Tasks: 146 done / 163 total; 7 can run
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327280    1429 executor.go:186] Executing task "File//var/lib/kube-proxy/kubeconfig": File: "/var/lib/kube-proxy/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327306    1429 executor.go:186] Executing task "File//home/ubuntu/.kube/config": File: "/home/ubuntu/.kube/config"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327314    1429 executor.go:186] Executing task "File//var/lib/kube-scheduler/kubeconfig": File: "/var/lib/kube-scheduler/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327340    1429 executor.go:186] Executing task "File//var/lib/kubelet/kubeconfig": File: "/var/lib/kubelet/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327358    1429 executor.go:186] Executing task "File//var/lib/kube-controller-manager/kubeconfig": File: "/var/lib/kube-controller-manager/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327419    1429 files.go:58] Writing file "/home/ubuntu/.kube/config"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327415    1429 executor.go:186] Executing task "File//var/lib/kubectl/kubeconfig": File: "/var/lib/kubectl/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327443    1429 files.go:58] Writing file "/var/lib/kubelet/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327475    1429 files.go:58] Writing file "/var/lib/kube-proxy/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327509    1429 files.go:58] Writing file "/var/lib/kube-controller-manager/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327525    1429 files.go:58] Writing file "/var/lib/kube-scheduler/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327529    1429 files.go:114] Changing file mode for "/home/ubuntu/.kube/config" to -r--------
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327539    1429 files.go:114] Changing file mode for "/var/lib/kubelet/kubeconfig" to -r--------
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327554    1429 files.go:114] Changing file mode for "/var/lib/kube-proxy/kubeconfig" to -r--------
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327281    1429 executor.go:186] Executing task "File//var/lib/kops/kubeconfig": File: "/var/lib/kops/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327581    1429 files.go:114] Changing file mode for "/var/lib/kube-controller-manager/kubeconfig" to -r--------
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327601    1429 files.go:58] Writing file "/var/lib/kubectl/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327621    1429 files.go:114] Changing file mode for "/var/lib/kube-scheduler/kubeconfig" to -r--------
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327669    1429 files.go:114] Changing file mode for "/var/lib/kubectl/kubeconfig" to -r--------
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327678    1429 files_owner.go:68] Changing file owner/group for "/home/ubuntu/.kube/config" to ubuntu:ubuntu
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327687    1429 files.go:58] Writing file "/var/lib/kops/kubeconfig"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327765    1429 files.go:114] Changing file mode for "/var/lib/kops/kubeconfig" to -r--------
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327854    1429 executor.go:111] Tasks: 153 done / 163 total; 9 can run
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327899    1429 executor.go:186] Executing task "Service/docker.service": Service: docker.service
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327916    1429 executor.go:186] Executing task "Service/systemd-timesyncd": Service: systemd-timesyncd
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327939    1429 executor.go:186] Executing task "Service/kubelet.service": Service: kubelet.service
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327940    1429 executor.go:186] Executing task "Service/logrotate.service": Service: logrotate.service
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327967    1429 executor.go:186] Executing task "Service/protokube.service": Service: protokube.service
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327969    1429 executor.go:186] Executing task "Service/docker.socket": Service: docker.socket
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327970    1429 executor.go:186] Executing task "Service/containerd.service": Service: containerd.service
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.327989    1429 executor.go:186] Executing task "Service/kubernetes-iptables-setup.service": Service: kubernetes-iptables-setup.service
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328000    1429 executor.go:186] Executing task "Service/logrotate.timer": Service: logrotate.timer
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328086    1429 service.go:123] querying state of service "logrotate.timer"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328100    1429 changes.go:81] Field changed "Definition" actual="<nil>" expected="[Unit]\nDescription=Docker Application Container Engine\nDocumentation=https://docs.docker.com\nAfter=network-online.target firewalld.service containerd.service\nWants=network-online.target\nRequires=docker.socket\n\n[Service]\nEnvironmentFile=/etc/sysconfig/docker\nEnvironmentFile=/etc/environment\nType=notify\nExecStart=/usr/bin/dockerd -H fd:// \"$DOCKER_OPTS\"\nExecReload=/bin/kill -s HUP $MAINPID\nTimeoutSec=0\nRestartSec=2\nRestart=always\nStartLimitBurst=3\nStartLimitInterval=60s\nLimitNOFILE=infinity\nLimitNPROC=infinity\nLimitCORE=infinity\nTasksMax=infinity\nDelegate=yes\nKillMode=process\nOOMScoreAdjust=-500\n\n[Install]\nWantedBy=multi-user.target\n"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328149    1429 changes.go:81] Field changed "Running" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328163    1429 changes.go:81] Field changed "Enabled" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328177    1429 changes.go:81] Field changed "ManageState" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328192    1429 changes.go:81] Field changed "SmartRestart" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328200    1429 changes.go:81] Field changed "Running" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328213    1429 changes.go:81] Field changed "Enabled" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328226    1429 changes.go:81] Field changed "ManageState" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328239    1429 changes.go:81] Field changed "SmartRestart" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328308    1429 service.go:349] Restarting service "systemd-timesyncd"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328310    1429 files.go:58] Writing file "/lib/systemd/system/docker.service"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328299    1429 changes.go:81] Field changed "Definition" actual="<nil>" expected="[Unit]\nDescription=Kubernetes Kubelet Server\nDocumentation=https://github.com/kubernetes/kubernetes\nAfter=docker.service\n\n[Service]\nEnvironmentFile=/etc/sysconfig/kubelet\nExecStart=/usr/local/bin/kubelet \"$DAEMON_ARGS\"\nRestart=always\nRestartSec=2s\nStartLimitInterval=0\nKillMode=process\nUser=root\nCPUAccounting=true\nMemoryAccounting=true\n"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328322    1429 service.go:123] querying state of service "logrotate.service"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328337    1429 changes.go:81] Field changed "Running" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328353    1429 changes.go:81] Field changed "Enabled" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328367    1429 changes.go:81] Field changed "ManageState" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328381    1429 changes.go:81] Field changed "SmartRestart" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328403    1429 files.go:114] Changing file mode for "/lib/systemd/system/docker.service" to -rw-r--r--
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328380    1429 changes.go:81] Field changed "Definition" actual="<nil>" expected="[Unit]\nDescription=Kubernetes Protokube Service\nDocumentation=https://github.com/kubernetes/kops\n\n[Service]\nExecStartPre=-/usr/bin/docker stop protokube\nExecStartPre=-/usr/bin/docker rm protokube\nExecStart=/usr/bin/docker run --net=host --pid=host --privileged --volume /:/rootfs --env KUBECONFIG=/rootfs/var/lib/kops/kubeconfig --volume /bin:/bin:ro --volume /lib:/lib:ro --volume /sbin:/sbin:ro --volume /usr/bin:/usr/bin:ro --volume /var/run/dbus:/var/run/dbus --volume /run/systemd:/run/systemd --volume /lib64:/lib64:ro --volume /usr/local/bin:/opt/kops/bin:ro --env PATH=/opt/kops/bin:/usr/bin:/sbin:/bin  --env 'AWS_REGION=us-east-1'  --name protokube protokube:1.20.0-beta.2 /protokube --bootstrap-master-node-labels=true --channels=s3://state.**********************/**********************/addons/bootstrap-channel.yaml --cloud=aws --containerized=true --dns-internal-suffix=.internal.********************** --dns=aws-route53 --initialize-rbac=true --manage-etcd=false --master=true --node-name=ip-**************.ec2.internal --remove-dns-names=etcd-a.internal.**********************,etcd-events-a.internal.********************** --v=4 --zone=*/Z04573671OIZ21LVFGLE4\nRestart=always\nRestartSec=3s\nStartLimitInterval=0\n\n[Install]\nWantedBy=multi-user.target\n"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328434    1429 service.go:286] Reloading systemd configuration
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328447    1429 files.go:58] Writing file "/lib/systemd/system/kubelet.service"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328453    1429 changes.go:81] Field changed "Running" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328467    1429 changes.go:81] Field changed "Enabled" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328480    1429 changes.go:81] Field changed "ManageState" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328494    1429 changes.go:81] Field changed "SmartRestart" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328516    1429 changes.go:81] Field changed "Definition" actual="<nil>" expected="[Unit]\nDescription=Docker Socket for the API\nPartOf=docker.service\n\n[Socket]\nListenStream=/var/run/docker.sock\nSocketMode=0660\nSocketUser=root\nSocketGroup=docker\n\n[Install]\nWantedBy=sockets.target\n"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328539    1429 files.go:114] Changing file mode for "/lib/systemd/system/kubelet.service" to -rw-r--r--
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328548    1429 changes.go:81] Field changed "Running" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328562    1429 changes.go:81] Field changed "Enabled" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328564    1429 files.go:58] Writing file "/lib/systemd/system/protokube.service"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328572    1429 service.go:286] Reloading systemd configuration
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328575    1429 changes.go:81] Field changed "ManageState" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328588    1429 changes.go:81] Field changed "SmartRestart" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328588    1429 changes.go:81] Field changed "Definition" actual="<nil>" expected="[Unit]\nDescription=containerd container runtime\nDocumentation=https://containerd.io\nAfter=network.target local-fs.target\n\n[Service]\nEnvironmentFile=/etc/sysconfig/containerd\nEnvironmentFile=/etc/environment\nExecStartPre=-/sbin/modprobe overlay\nExecStart=/usr/bin/containerd -c /etc/containerd/config-kops.toml \"$CONTAINERD_OPTS\"\nType=notify\nDelegate=yes\nKillMode=process\nRestart=always\nRestartSec=5\nLimitNPROC=infinity\nLimitCORE=infinity\nLimitNOFILE=infinity\nTasksMax=infinity\nOOMScoreAdjust=-999\n\n[Install]\nWantedBy=multi-user.target\n"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328624    1429 changes.go:81] Field changed "Running" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328637    1429 changes.go:81] Field changed "Enabled" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328648    1429 files.go:58] Writing file "/lib/systemd/system/docker.socket"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328650    1429 changes.go:81] Field changed "ManageState" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328652    1429 files.go:114] Changing file mode for "/lib/systemd/system/protokube.service" to -rw-r--r--
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328664    1429 changes.go:81] Field changed "SmartRestart" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328684    1429 service.go:286] Reloading systemd configuration
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328702    1429 changes.go:81] Field changed "Definition" actual="<nil>" expected="[Unit]\nDescription=Configure iptables for kubernetes\nDocumentation=https://github.com/kubernetes/kops\nBefore=network.target\n\n[Service]\nType=oneshot\nRemainAfterExit=yes\nExecStart=/opt/kops/bin/iptables-setup\n\n[Install]\nWantedBy=basic.target\n"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328729    1429 files.go:58] Writing file "/lib/systemd/system/containerd.service"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328731    1429 changes.go:81] Field changed "Running" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328734    1429 files.go:114] Changing file mode for "/lib/systemd/system/docker.socket" to -rw-r--r--
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328749    1429 changes.go:81] Field changed "Enabled" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328762    1429 service.go:286] Reloading systemd configuration
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328762    1429 changes.go:81] Field changed "ManageState" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328779    1429 changes.go:81] Field changed "SmartRestart" actual="<nil>" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328805    1429 files.go:114] Changing file mode for "/lib/systemd/system/containerd.service" to -rw-r--r--
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328835    1429 service.go:286] Reloading systemd configuration
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328843    1429 files.go:58] Writing file "/lib/systemd/system/kubernetes-iptables-setup.service"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328925    1429 files.go:114] Changing file mode for "/lib/systemd/system/kubernetes-iptables-setup.service" to -rw-r--r--
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.328956    1429 service.go:286] Reloading systemd configuration
Mar 14 16:31:20 ip-************** nodeup[1429]: W0314 16:31:20.340125    1429 service.go:221] Unknown WantedBy="timers.target"; will treat as not enabled
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.340188    1429 changes.go:81] Field changed "Definition" actual="[Unit]\nDescription=Daily rotation of log files\nDocumentation=man:logrotate(8) man:logrotate.conf(5)\n\n[Timer]\nOnCalendar=daily\nAccuracySec=12h\nPersistent=true\n\n[Install]\nWantedBy=timers.target\n" expected="[Unit]\nDescription=Hourly Log Rotation\n\n[Timer]\nOnCalendar=hourly\n"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.340225    1429 changes.go:81] Field changed "Enabled" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.340335    1429 files.go:58] Writing file "/lib/systemd/system/logrotate.timer"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.341186    1429 files.go:114] Changing file mode for "/lib/systemd/system/logrotate.timer" to -rw-r--r--
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.341243    1429 service.go:286] Reloading systemd configuration
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.347368    1429 changes.go:81] Field changed "Definition" actual="[Unit]\nDescription=Rotate log files\nDocumentation=man:logrotate(8) man:logrotate.conf(5)\nConditionACPower=true\n\n[Service]\nType=oneshot\nExecStart=/usr/sbin/logrotate /etc/logrotate.conf\n\n# performance options\nNice=19\nIOSchedulingClass=best-effort\nIOSchedulingPriority=7\n\n# hardening options\n#  details: https://www.freedesktop.org/software/systemd/man/systemd.exec.html\n#  no ProtectHome for userdir logs\n#  no PrivateNetwork for mail deliviery\n#  no ProtectKernelTunables for working SELinux with systemd older than 235\n#  no MemoryDenyWriteExecute for gzip on i686\nPrivateDevices=true\nPrivateTmp=true\nProtectControlGroups=true\nProtectKernelModules=true\nProtectSystem=full\nRestrictRealtime=true\n" expected="[Unit]\nDescription=Rotate and Compress System Logs\n\n[Service]\nExecStart=/usr/sbin/logrotate /etc/logrotate.conf\n"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.347443    1429 changes.go:81] Field changed "Running" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.347454    1429 changes.go:81] Field changed "Enabled" actual="false" expected="true"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.347545    1429 files.go:58] Writing file "/lib/systemd/system/logrotate.service"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.347736    1429 files.go:114] Changing file mode for "/lib/systemd/system/logrotate.service" to -rw-r--r--
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.347780    1429 service.go:286] Reloading systemd configuration
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.571667    1429 service.go:349] Restarting service "docker.service"
Mar 14 16:31:20 ip-************** nodeup[1429]: I0314 16:31:20.808067    1429 service.go:349] Restarting service "kubelet.service"
Mar 14 16:31:21 ip-************** nodeup[1429]: I0314 16:31:21.012418    1429 service.go:349] Restarting service "protokube.service"
Mar 14 16:31:21 ip-************** nodeup[1429]: I0314 16:31:21.268578    1429 service.go:349] Restarting service "docker.socket"
Mar 14 16:31:21 ip-************** nodeup[1429]: I0314 16:31:21.481222    1429 service.go:349] Restarting service "containerd.service"
Mar 14 16:31:21 ip-************** nodeup[1429]: I0314 16:31:21.716416    1429 service.go:349] Restarting service "kubernetes-iptables-setup.service"
Mar 14 16:31:21 ip-************** nodeup[1429]: I0314 16:31:21.959781    1429 service.go:123] querying state of service "logrotate.timer"
Mar 14 16:31:22 ip-************** nodeup[1429]: I0314 16:31:22.181514    1429 service.go:349] Restarting service "logrotate.service"
Mar 14 16:31:22 ip-************** nodeup[1429]: W0314 16:31:22.198260    1429 service.go:331] service was running, but did not have ExecMainStartTimestamp: "logrotate.timer"
Mar 14 16:31:22 ip-************** nodeup[1429]: I0314 16:31:22.198300    1429 service.go:360] Enabling service "logrotate.timer"
Mar 14 16:31:22 ip-************** nodeup[1429]: I0314 16:31:22.536478    1429 service.go:360] Enabling service "logrotate.service"
Mar 14 16:31:22 ip-************** nodeup[1429]: I0314 16:31:22.565301    1429 service.go:360] Enabling service "docker.socket"
Mar 14 16:31:22 ip-************** nodeup[1429]: I0314 16:31:22.571576    1429 service.go:360] Enabling service "kubernetes-iptables-setup.service"
Mar 14 16:31:22 ip-************** nodeup[1429]: I0314 16:31:22.871620    1429 service.go:360] Enabling service "containerd.service"
Mar 14 16:31:23 ip-************** nodeup[1429]: I0314 16:31:23.756121    1429 service.go:360] Enabling service "systemd-timesyncd"
Mar 14 16:31:24 ip-************** nodeup[1429]: I0314 16:31:24.072382    1429 service.go:360] Enabling service "docker.service"
Mar 14 16:31:24 ip-************** nodeup[1429]: I0314 16:31:24.077383    1429 service.go:360] Enabling service "kubelet.service"
Mar 14 16:31:24 ip-************** nodeup[1429]: I0314 16:31:24.336202    1429 service.go:360] Enabling service "protokube.service"
Mar 14 16:31:24 ip-************** nodeup[1429]: I0314 16:31:24.803341    1429 executor.go:111] Tasks: 162 done / 163 total; 1 can run
Mar 14 16:31:24 ip-************** nodeup[1429]: I0314 16:31:24.803410    1429 executor.go:186] Executing task "LoadImageTask/protokube": LoadImageTask: [https://artifacts.k8s.io/binaries/kops/1.20.0-beta.2/images/protokube-amd64.tar.gz https://github.com/kubernetes/kops/releases/download/v1.20.0-beta.2/images-protokube-amd64.tar.gz https://kubeupv2.s3.amazonaws.com/kops/1.20.0-beta.2/images/protokube-amd64.tar.gz]
Mar 14 16:31:24 ip-************** nodeup[1429]: W0314 16:31:24.803520    1429 load_image.go:77] LoadImageTask checking if image present not yet implemented
Mar 14 16:31:24 ip-************** nodeup[1429]: I0314 16:31:24.803704    1429 http.go:78] Downloading "https://artifacts.k8s.io/binaries/kops/1.20.0-beta.2/images/protokube-amd64.tar.gz"
Mar 14 16:31:24 ip-************** nodeup[1429]: W0314 16:31:24.878115    1429 load_image.go:112] error downloading url "https://artifacts.k8s.io/binaries/kops/1.20.0-beta.2/images/protokube-amd64.tar.gz": error response from "https://artifacts.k8s.io/binaries/kops/1.20.0-beta.2/images/protokube-amd64.tar.gz": HTTP 404
Mar 14 16:31:24 ip-************** nodeup[1429]: I0314 16:31:24.878210    1429 files.go:137] Hash did not match for "/var/cache/nodeup/sha256:b407d467303305f0bab44bc27e17f70b5dd479d563f27b65e7bca06325088530_https___artifacts_k8s_io_binaries_kops_1_20_0-beta_2_images_protokube-amd64_tar_gz": actual=sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 vs expected=sha256:b407d467303305f0bab44bc27e17f70b5dd479d563f27b65e7bca06325088530
Mar 14 16:31:24 ip-************** nodeup[1429]: I0314 16:31:24.878260    1429 http.go:78] Downloading "https://github.com/kubernetes/kops/releases/download/v1.20.0-beta.2/images-protokube-amd64.tar.gz"
Mar 14 16:31:26 ip-************** nodeup[1429]: I0314 16:31:26.223966    1429 files.go:134] Hash matched for "/var/cache/nodeup/sha256:b407d467303305f0bab44bc27e17f70b5dd479d563f27b65e7bca06325088530_https___artifacts_k8s_io_binaries_kops_1_20_0-beta_2_images_protokube-amd64_tar_gz": sha256:b407d467303305f0bab44bc27e17f70b5dd479d563f27b65e7bca06325088530
Mar 14 16:31:28 ip-************** nodeup[1429]: I0314 16:31:28.634645    1429 load_image.go:159] running command docker load -i /tmp/loadimage332668703/https___artifacts_k8s_io_binaries_kops_1_20_0-beta_2_images_protokube-amd64_tar_gz
Mar 14 16:31:32 ip-************** nodeup[1429]: I0314 16:31:32.009656    1429 executor.go:111] Tasks: 163 done / 163 total; 0 can run
Mar 14 16:31:32 ip-************** nodeup[1429]: I0314 16:31:32.009720    1429 context.go:91] deleting temp dir: "/tmp/deploy510302547"
Mar 14 16:31:32 ip-************** nodeup[1429]: success
Mar 14 16:31:32 ip-************** systemd[1]: kops-configuration.service: Succeeded.
Mar 14 16:31:32 ip-************** systemd[1]: Finished Run kops bootstrap (nodeup).

9. Anything else do we need to know?
None of kubelets can authorize on API server. On a new master /var/lib/kubelet/pki doesn't exist. After rolling back to 1.18 everything works fine.
@vkryzh
Copy link
Author

vkryzh commented Mar 16, 2021

This issue was relevant. #11014

@vkryzh vkryzh closed this as completed Mar 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vkryzh