From e07f822527ab312bb417c37770fcf84f8896fab7 Mon Sep 17 00:00:00 2001 From: Gergely Nemeth Date: Mon, 18 Jun 2018 16:44:16 -0400 Subject: [PATCH 1/5] docs(kubernetes): explain readiness probe --- README.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/README.md b/README.md index 5019414..dcb9033 100644 --- a/README.md +++ b/README.md @@ -103,6 +103,32 @@ terminus(server, options); server.listen(PORT || 3000); ``` +## How to set Terminus up with Kubernetes? + +When a pod is deleted, Kubernetes will notify it and will wait for `gracePeriod` seconds before killing it. + +During that time window (30 seconds by default), the pod is in the `terminating` state and will be removed from any Services by a controller. The pod itself needs to catch the `SIGTERM` signal, and start failing any readiness probes. + +During this time, it is possible that load-balancers don't remove the pods "in time", and when the pod dies, it kills live connections. + +To make sure you don't lose any connections, we recommend delaying the shutdown with the number of milliseconds that's defined by the readiness probe in your deployment configuration. To help with this, terminus exposes an option called `beforeShutdown` that takes any Promise-returning function. + +```javascript +function beforeShutdown () { + // given your readiness probes run every 5 second + // may worth using a bigger number so you won't + // run into any race conditions + return Promise(resolve => { + setTimeout(5000, resolve) + }) +} +terminus(server, { + beforeShutdown +}) +``` + +[Learn more](https://github.com/kubernetes/contrib/issues/1140#issuecomment-231641402) + ## Limited Windows support Due to inherent platform limitations, `terminus` has limited support for Windows. From 8f71c9200ec500294bc8d53ea4dc7fa0da6e651c Mon Sep 17 00:00:00 2001 From: Gergely Nemeth Date: Fri, 29 Jun 2018 11:01:07 -0700 Subject: [PATCH 2/5] docs(kubernetes): explain readiness probe - nits --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index dcb9033..0009f0f 100644 --- a/README.md +++ b/README.md @@ -105,7 +105,7 @@ server.listen(PORT || 3000); ## How to set Terminus up with Kubernetes? -When a pod is deleted, Kubernetes will notify it and will wait for `gracePeriod` seconds before killing it. +When a pod is deleted, Kubernetes will notify it and wait for `gracePeriod` seconds before killing it. During that time window (30 seconds by default), the pod is in the `terminating` state and will be removed from any Services by a controller. The pod itself needs to catch the `SIGTERM` signal, and start failing any readiness probes. @@ -116,10 +116,10 @@ To make sure you don't lose any connections, we recommend delaying the shutdown ```javascript function beforeShutdown () { // given your readiness probes run every 5 second - // may worth using a bigger number so you won't + // may be worth using a bigger number so you won't // run into any race conditions return Promise(resolve => { - setTimeout(5000, resolve) + setTimeout(resolve, 5000) }) } terminus(server, { From 4fb5a06f284a893233e76c2df4f990f4b45e6aa4 Mon Sep 17 00:00:00 2001 From: Gergely Nemeth Date: Fri, 29 Jun 2018 11:09:33 -0700 Subject: [PATCH 3/5] docs(kubernetes): explain readiness probe --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 0009f0f..82f0672 100644 --- a/README.md +++ b/README.md @@ -107,9 +107,11 @@ server.listen(PORT || 3000); When a pod is deleted, Kubernetes will notify it and wait for `gracePeriod` seconds before killing it. -During that time window (30 seconds by default), the pod is in the `terminating` state and will be removed from any Services by a controller. The pod itself needs to catch the `SIGTERM` signal, and start failing any readiness probes. +During that time window (30 seconds by default), the pod is in the `terminating` state and will be removed from any Services by a controller. The pod itself needs to catch the `SIGTERM` signal and start failing any readiness probes. -During this time, it is possible that load-balancers don't remove the pods "in time", and when the pod dies, it kills live connections. +During this time, it is possible that load-balancers don't remove the pods "in time", and when the pod dies, it kills live connections. + +> If the ingress controller you use route via the Service, it is not an issue for your case. At the time of this writing, we use the nginx ingress controller which routes traffic directly to the pods. To make sure you don't lose any connections, we recommend delaying the shutdown with the number of milliseconds that's defined by the readiness probe in your deployment configuration. To help with this, terminus exposes an option called `beforeShutdown` that takes any Promise-returning function. @@ -135,4 +137,4 @@ Due to inherent platform limitations, `terminus` has limited support for Windows You can expect `SIGINT` to work, as well as `SIGBREAK` and to some extent `SIGHUP`. However `SIGTERM` will never work on Windows because killing a process in the task manager is unconditional, i.e., there's no way for an application to detect or prevent it. Here's some relevant documentation from [`libuv`](https://github.com/libuv/libuv) to learn more about what `SIGINT`, `SIGBREAK` etc. signify and what's supported on Windows - http://docs.libuv.org/en/v1.x/signal.html. -Also see https://nodejs.org/api/process.html#process_signal_events. +Also, see https://nodejs.org/api/process.html#process_signal_events. From 6ad8f714ad92070a9d1bb8bdba52668f418c15cc Mon Sep 17 00:00:00 2001 From: Gergely Nemeth Date: Fri, 29 Jun 2018 11:39:26 -0700 Subject: [PATCH 4/5] docs(kubernetes): explain readiness probe --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 82f0672..9ebbcef 100644 --- a/README.md +++ b/README.md @@ -105,13 +105,13 @@ server.listen(PORT || 3000); ## How to set Terminus up with Kubernetes? -When a pod is deleted, Kubernetes will notify it and wait for `gracePeriod` seconds before killing it. +When a Pod is deleted, Kubernetes will notify it and wait for `gracePeriod` seconds before killing it. -During that time window (30 seconds by default), the pod is in the `terminating` state and will be removed from any Services by a controller. The pod itself needs to catch the `SIGTERM` signal and start failing any readiness probes. +During that time window (30 seconds by default), the Pod is in the `terminating` state and will be removed from any Services by a controller. The Pod itself needs to catch the `SIGTERM` signal and start failing any readiness probes. -During this time, it is possible that load-balancers don't remove the pods "in time", and when the pod dies, it kills live connections. +> If the ingress controller you use route via the Service, it is not an issue for your case. At the time of this writing, we use the nginx ingress controller which routes traffic directly to the Pods. -> If the ingress controller you use route via the Service, it is not an issue for your case. At the time of this writing, we use the nginx ingress controller which routes traffic directly to the pods. +During this time, it is possible that load-balancers (like the nginx ingress controller) don't remove the Pods "in time", and when the Pod dies, it kills live connections. To make sure you don't lose any connections, we recommend delaying the shutdown with the number of milliseconds that's defined by the readiness probe in your deployment configuration. To help with this, terminus exposes an option called `beforeShutdown` that takes any Promise-returning function. From 9148dd88d676b0173f559ca387c6cf8cadf6076b Mon Sep 17 00:00:00 2001 From: Gergely Nemeth Date: Fri, 29 Jun 2018 12:35:50 -0700 Subject: [PATCH 5/5] docs(kubernetes): explain readiness probe --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9ebbcef..f6db4ea 100644 --- a/README.md +++ b/README.md @@ -105,7 +105,7 @@ server.listen(PORT || 3000); ## How to set Terminus up with Kubernetes? -When a Pod is deleted, Kubernetes will notify it and wait for `gracePeriod` seconds before killing it. +When Kubernetes or a user deletes a Pod, Kubernetes will notify it and wait for `gracePeriod` seconds before killing it. During that time window (30 seconds by default), the Pod is in the `terminating` state and will be removed from any Services by a controller. The Pod itself needs to catch the `SIGTERM` signal and start failing any readiness probes.