Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor remove node to allow removing dead nodes and etcd members #5009

Merged
merged 1 commit into from
Aug 7, 2019
Merged

Refactor remove node to allow removing dead nodes and etcd members #5009

merged 1 commit into from
Aug 7, 2019

Conversation

mattymo
Copy link
Contributor

@mattymo mattymo commented Jul 25, 2019

/kind feature

This enables purging etcd nodes as well.

@k8s-ci-robot k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Jul 25, 2019
@mattymo
Copy link
Contributor Author

mattymo commented Jul 25, 2019

ci check this

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 25, 2019
@k8s-ci-robot k8s-ci-robot requested review from Miouge1 and mirwan July 25, 2019 14:30
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mattymo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 25, 2019
@mattymo
Refactor remove node to allow removing dead nodes and etcd members
0aacf69 
Change-Id: I1c59249f08f16d0f6fd60df6ab61f17a0a7df189
riverzhang
riverzhang reviewed Aug 6, 2019
View reviewed changes
roles/remove-node/post-remove/tasks/main.yml
delegate_to: "{{ groups['etcd']|first }}"
when: inventory_hostname in groups['etcd']

- name: Remove etcd member from cluster
Copy link
Contributor

@riverzhang riverzhang Aug 6, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are 3 nodes etcd cluster, if we remove 2 etcd nodes. The kubernetes cluster will not be available.
So we should remind the user in the documentation. If you delete the etcd cluster more than half the number.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will actually function if you remove node 3, then node 2. It will scale down to 1 and then you could rebuild from that

@Atoms
Copy link
Member

Atoms commented Aug 7, 2019

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 7, 2019
@k8s-ci-robot k8s-ci-robot merged commit a44235d into kubernetes-sigs:master Aug 7, 2019
erulabs added a commit to kubesail/kubespray that referenced this pull request Aug 9, 2019
@erulabs
Merge branch 'master' of https://github.com/kubernetes-sigs/kubespray
30b3278 
* 'master' of https://github.com/kubernetes-sigs/kubespray:
  Add missing coredns tag. (kubernetes-sigs#5054)
  Bump minimum K8S version to 1.14 (kubernetes-sigs#5055)
  multus | fix use last version (kubernetes-sigs#5041)
  Fix variable for rbd_provisioner_user_secret (kubernetes-sigs#5042)
  go to k8s 1.15.2, update nodelocaldns to latest bugfix release (kubernetes-sigs#5048)
  Refactor calico route reflector to run in k8s cluster (kubernetes-sigs#4975)
  Fix check for removing etcd member (kubernetes-sigs#5051)
  Refactor remove node to allow removing dead nodes and etcd members (kubernetes-sigs#5009)
  Allow etcd member join by checking cluster health only on first etcd (kubernetes-sigs#5032)
  Ansible version bump for CVE-2019-10156 (kubernetes-sigs#5050)
  Add ability to setup virtual ip for ingress-controller (kubernetes-sigs#5044)
  Optionally refresh kubeadm token every time (kubernetes-sigs#5045)
  Upgrade Cilium network plugin to v1.5.5. (kubernetes-sigs#5014)
  Optionally refresh kubeadm token every time (kubernetes-sigs#5043)
LuckySB pushed a commit to southbridgeio/kubespray that referenced this pull request Aug 13, 2019
@mattymo @LuckySB
Refactor remove node to allow removing dead nodes and etcd members (k…
6fb7b7f 
…ubernetes-sigs#5009)

Change-Id: I1c59249f08f16d0f6fd60df6ab61f17a0a7df189
b23prodtm added a commit to b23prodtm/kubespray that referenced this pull request Aug 20, 2019
@b23prodtm
Pr/21 (#22)
4ca7a26 
* Fixup etcdctl download for etcd kubeadm mode (kubernetes-sigs#4991)

Change-Id: I8d8e59a97823390f40e8810905ca52430329f040

* configure docker_options directly with template (kubernetes-sigs#4912)

* Remove unused handlers from Flannel CNI (kubernetes-sigs#4984)

* Only reload docker when is_atomic for Flannel

* Remove unused handlers from Flannel CNI

* update to kubernetes 1.15.1 (kubernetes-sigs#4989)

* update to kubernetes 1.15.1

* Revert to sonobuoy 0.15.0

* update test timeout from 3 to 5 minutes

* add ansible_hostname and ansible_fqdn to apiserver_sans (kubernetes-sigs#4990)

* Restart canal after scale or upgrade. Just like PR#4531, but for canal (kubernetes-sigs#4992)

* Update calico-typha deployment to address v3.7.x changes (kubernetes-sigs#5003)

* Update calico-typha deployment to address v3.7.x changes

So that calico-typha works for Calico v3.7.x.

* Apply changes for v3.7.x only.

* fix broken ubuntu containerd engine (kubernetes-sigs#5002)

* move flexvolume plugin directory creation to preinstall (kubernetes-sigs#4999)

* move flexvolume plugin directory creation to preinstall

* changes per pr feedback

* Additional files and dirs to remove when running reset (kubernetes-sigs#5000)

* Update critools to v1.15.0 (kubernetes-sigs#5016)

* Update CoreDNS to 1.6.0 (kubernetes-sigs#5021)

* fix broken link (kubernetes-sigs#5023)

* add Kube-OVN cni to kubespray (kubernetes-sigs#5020)

* Add packet_ubuntu18-flannel-containerd (kubernetes-sigs#5004)

* Add an option to reserve resources for OS system daemons (kubernetes-sigs#5007)

* Add CoreDNS endpoint_pod_names option (kubernetes-sigs#5012)

* [contrib/heketi]: tear down additions and fixes. Heketi updated to version 9 (kubernetes-sigs#5027)

* lvm packages removal during tear down skipped by default
  * lvm utils execution PATH fixed for CentOS/RH
  * Heketi updated to the latest version 9

Signed-off-by: Vitaliy Dmitriev <[email protected]>

* Fix small typo (kubernetes-sigs#5029)

* Always create bash_completion.d folder (kubernetes-sigs#5039)

* Fix mistakes in downloads docs (kubernetes-sigs#5038)

* Refactor containerd ubuntu setup and remove redundant tasks (kubernetes-sigs#5015)

* Fix commands for using experimental kubeadm control plane (kubernetes-sigs#5006)

* Optionally refresh kubeadm token every time (kubernetes-sigs#5043)

Change-Id: I278cb14aa93abf20160cc001f69e2f472504e6d8

* Upgrade Cilium network plugin to v1.5.5. (kubernetes-sigs#5014)

* Needs an additional cilium-operator deployment.
  * Added option to enable hostPort mappings.

* Optionally refresh kubeadm token every time (kubernetes-sigs#5045)

Change-Id: I278cb14aa93abf20160cc001f69e2f472504e6d8

* Add ability to setup virtual ip for ingress-controller (kubernetes-sigs#5044)

* Ansible version bump for CVE-2019-10156 (kubernetes-sigs#5050)

* Allow etcd member join by checking cluster health only on first etcd (kubernetes-sigs#5032)

Change-Id: I9cc01cef3a437893225e2d9f58495826bbce7be9

* Refactor remove node to allow removing dead nodes and etcd members (kubernetes-sigs#5009)

Change-Id: I1c59249f08f16d0f6fd60df6ab61f17a0a7df189

* Fix check for removing etcd member (kubernetes-sigs#5051)

Change-Id: Ib27d051ff111f813097a9b33a86465a2a30a6db0

* Refactor calico route reflector to run in k8s cluster (kubernetes-sigs#4975)

* Refactor calico-rr to run in k8s cluster with taint

Change-Id: I75a3169ff5b36ce8302fc7ef1c32d3eb697b5afa

* add preinstall checks

* rework calico/rr role

Change-Id: I2f0a7e6cb77cf91ad4a615923680760d2e5d9ca8

* add empty calico-rr group

Change-Id: I006c0a60db9b72d02245bf8fdfabcf982144a5ad

* go to k8s 1.15.2, update nodelocaldns to latest bugfix release (kubernetes-sigs#5048)

* Fix variable for rbd_provisioner_user_secret (kubernetes-sigs#5042)

* Update main.yml

* fix dead link 404

* multus | fix use last version (kubernetes-sigs#5041)

* Bump minimum K8S version to 1.14 (kubernetes-sigs#5055)

Signed-off-by: Craig Rodrigues <[email protected]>

* Add missing coredns tag. (kubernetes-sigs#5054)

* Always download coredns images with kubeadm (kubernetes-sigs#5071)

Fixes situation when using manual mode because it
tries to download coredns v1.3.1 from the same
image repository where kubernetes images are
downloaded from.

Change-Id: Ibbec8a72c8162ce8befa74e2013a268737ea5f8a

* Allow to configure number of kube-masters (kubernetes-sigs#5073)

Change-Id: Ia3f30a1216b3ea063cd72c839ef6dff753cf10c6

* Fix broken containerd pinning on Ubuntu (kubernetes-sigs#5072)

* move kube-ovn images to dockerhub (kubernetes-sigs#5063)

* Set initial kubeadm token if specified in kubeadm init (kubernetes-sigs#5057)

Change-Id: I7fd94ec6d195af60d237b3cfe91668ca1f707d26

* Ingress nginx (kubernetes-sigs#5066)

* remove svc-default-backend

* update ingress-nginx clusterrole

* Disable moderator (kubernetes-sigs#5069)

* Test the CI

* Disable CI moderator

* Upgrade ingress nginx to 0.25.1 (kubernetes-sigs#5081)

* upgrade Helm to 2.14.3 (kubernetes-sigs#5075)

Signed-off-by: Bart Verwilst <[email protected]>

* remove resource-container default value for kube-proxy (kubernetes-sigs#4994)

* fix openstack_cacert conditional (kubernetes-sigs#5078)

* fix word errors in downloads.md (kubernetes-sigs#5083)

Signed-off-by: Guangming Wang <[email protected]>

* cleanup vars.md typos words (kubernetes-sigs#5086)

Signed-off-by: Guangming Wang <[email protected]>

* add cluster name into filer if specifed in environment variable (kubernetes-sigs#5085)

* Updated Openstack to terraform 0.12 (kubernetes-sigs#5062)

* update openstack to terraform 0.12(.5)

* replace cluter.tf with cluster.tfvars

* update README.md to terraform 0.12

* update Openstack CI tests to use terraform 0.12

* specify terraform version in openstack README

* gitlab CI to copy cluster.tfvars in case of openstack provider

* The terraform/openstack dynamic inventory can read
tfstate v4 (generated by terraform 0.12) and convert them internally
ro v3 (as generated by terraform 0.11.x).

Additionally the script has been updated to Python 3.

* Kube-router annotate.yml: Use group 'k8s-cluster' instead of 'all' (kubernetes-sigs#5087) (kubernetes-sigs#5088)

* add cinder max attached volumes (kubernetes-sigs#5089)

* Merge branch 'master' into pr/21
b23prodtm added a commit to b23prodtm/kubespray that referenced this pull request Aug 20, 2019
@b23prodtm
Upstream upd (#21)
e01bad9 
* Fixup etcdctl download for etcd kubeadm mode (kubernetes-sigs#4991)

Change-Id: I8d8e59a97823390f40e8810905ca52430329f040

* configure docker_options directly with template (kubernetes-sigs#4912)

* Remove unused handlers from Flannel CNI (kubernetes-sigs#4984)

* Only reload docker when is_atomic for Flannel

* Remove unused handlers from Flannel CNI

* update to kubernetes 1.15.1 (kubernetes-sigs#4989)

* update to kubernetes 1.15.1

* Revert to sonobuoy 0.15.0

* update test timeout from 3 to 5 minutes

* add ansible_hostname and ansible_fqdn to apiserver_sans (kubernetes-sigs#4990)

* Restart canal after scale or upgrade. Just like PR#4531, but for canal (kubernetes-sigs#4992)

* Update calico-typha deployment to address v3.7.x changes (kubernetes-sigs#5003)

* Update calico-typha deployment to address v3.7.x changes

So that calico-typha works for Calico v3.7.x.

* Apply changes for v3.7.x only.

* fix broken ubuntu containerd engine (kubernetes-sigs#5002)

* move flexvolume plugin directory creation to preinstall (kubernetes-sigs#4999)

* move flexvolume plugin directory creation to preinstall

* changes per pr feedback

* Additional files and dirs to remove when running reset (kubernetes-sigs#5000)

* Update critools to v1.15.0 (kubernetes-sigs#5016)

* Update CoreDNS to 1.6.0 (kubernetes-sigs#5021)

* fix broken link (kubernetes-sigs#5023)

* add Kube-OVN cni to kubespray (kubernetes-sigs#5020)

* Add packet_ubuntu18-flannel-containerd (kubernetes-sigs#5004)

* Add an option to reserve resources for OS system daemons (kubernetes-sigs#5007)

* Add CoreDNS endpoint_pod_names option (kubernetes-sigs#5012)

* [contrib/heketi]: tear down additions and fixes. Heketi updated to version 9 (kubernetes-sigs#5027)

* lvm packages removal during tear down skipped by default
  * lvm utils execution PATH fixed for CentOS/RH
  * Heketi updated to the latest version 9

Signed-off-by: Vitaliy Dmitriev <[email protected]>

* Fix small typo (kubernetes-sigs#5029)

* Always create bash_completion.d folder (kubernetes-sigs#5039)

* Fix mistakes in downloads docs (kubernetes-sigs#5038)

* Refactor containerd ubuntu setup and remove redundant tasks (kubernetes-sigs#5015)

* Fix commands for using experimental kubeadm control plane (kubernetes-sigs#5006)

* Optionally refresh kubeadm token every time (kubernetes-sigs#5043)

Change-Id: I278cb14aa93abf20160cc001f69e2f472504e6d8

* Upgrade Cilium network plugin to v1.5.5. (kubernetes-sigs#5014)

* Needs an additional cilium-operator deployment.
  * Added option to enable hostPort mappings.

* Optionally refresh kubeadm token every time (kubernetes-sigs#5045)

Change-Id: I278cb14aa93abf20160cc001f69e2f472504e6d8

* Add ability to setup virtual ip for ingress-controller (kubernetes-sigs#5044)

* Ansible version bump for CVE-2019-10156 (kubernetes-sigs#5050)

* Allow etcd member join by checking cluster health only on first etcd (kubernetes-sigs#5032)

Change-Id: I9cc01cef3a437893225e2d9f58495826bbce7be9

* Refactor remove node to allow removing dead nodes and etcd members (kubernetes-sigs#5009)

Change-Id: I1c59249f08f16d0f6fd60df6ab61f17a0a7df189

* Fix check for removing etcd member (kubernetes-sigs#5051)

Change-Id: Ib27d051ff111f813097a9b33a86465a2a30a6db0

* Refactor calico route reflector to run in k8s cluster (kubernetes-sigs#4975)

* Refactor calico-rr to run in k8s cluster with taint

Change-Id: I75a3169ff5b36ce8302fc7ef1c32d3eb697b5afa

* add preinstall checks

* rework calico/rr role

Change-Id: I2f0a7e6cb77cf91ad4a615923680760d2e5d9ca8

* add empty calico-rr group

Change-Id: I006c0a60db9b72d02245bf8fdfabcf982144a5ad

* go to k8s 1.15.2, update nodelocaldns to latest bugfix release (kubernetes-sigs#5048)

* Fix variable for rbd_provisioner_user_secret (kubernetes-sigs#5042)

* Update main.yml

* fix dead link 404

* multus | fix use last version (kubernetes-sigs#5041)

* Bump minimum K8S version to 1.14 (kubernetes-sigs#5055)

Signed-off-by: Craig Rodrigues <[email protected]>

* Add missing coredns tag. (kubernetes-sigs#5054)

* Always download coredns images with kubeadm (kubernetes-sigs#5071)

Fixes situation when using manual mode because it
tries to download coredns v1.3.1 from the same
image repository where kubernetes images are
downloaded from.

Change-Id: Ibbec8a72c8162ce8befa74e2013a268737ea5f8a

* Allow to configure number of kube-masters (kubernetes-sigs#5073)

Change-Id: Ia3f30a1216b3ea063cd72c839ef6dff753cf10c6

* Fix broken containerd pinning on Ubuntu (kubernetes-sigs#5072)

* move kube-ovn images to dockerhub (kubernetes-sigs#5063)

* Set initial kubeadm token if specified in kubeadm init (kubernetes-sigs#5057)

Change-Id: I7fd94ec6d195af60d237b3cfe91668ca1f707d26

* Ingress nginx (kubernetes-sigs#5066)

* remove svc-default-backend

* update ingress-nginx clusterrole

* Disable moderator (kubernetes-sigs#5069)

* Test the CI

* Disable CI moderator

* Upgrade ingress nginx to 0.25.1 (kubernetes-sigs#5081)

* upgrade Helm to 2.14.3 (kubernetes-sigs#5075)

Signed-off-by: Bart Verwilst <[email protected]>

* remove resource-container default value for kube-proxy (kubernetes-sigs#4994)

* fix openstack_cacert conditional (kubernetes-sigs#5078)

* fix word errors in downloads.md (kubernetes-sigs#5083)

Signed-off-by: Guangming Wang <[email protected]>

* cleanup vars.md typos words (kubernetes-sigs#5086)

Signed-off-by: Guangming Wang <[email protected]>

* add cluster name into filer if specifed in environment variable (kubernetes-sigs#5085)

* Updated Openstack to terraform 0.12 (kubernetes-sigs#5062)

* update openstack to terraform 0.12(.5)

* replace cluter.tf with cluster.tfvars

* update README.md to terraform 0.12

* update Openstack CI tests to use terraform 0.12

* specify terraform version in openstack README

* gitlab CI to copy cluster.tfvars in case of openstack provider

* The terraform/openstack dynamic inventory can read
tfstate v4 (generated by terraform 0.12) and convert them internally
ro v3 (as generated by terraform 0.11.x).

Additionally the script has been updated to Python 3.

* Kube-router annotate.yml: Use group 'k8s-cluster' instead of 'all' (kubernetes-sigs#5087) (kubernetes-sigs#5088)

* add cinder max attached volumes (kubernetes-sigs#5089)
@Colstuwjx Colstuwjx mentioned this pull request Sep 30, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Atoms Atoms Atoms left review comments

@riverzhang riverzhang riverzhang left review comments

@Miouge1 Miouge1 Awaiting requested review from Miouge1

@mirwan mirwan Awaiting requested review from mirwan

Assignees

@Atoms Atoms

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mattymo @k8s-ci-robot @Atoms @riverzhang