From 38819b358f346f0514829314141e4e74e1bd0f15 Mon Sep 17 00:00:00 2001 From: Steven Bal Date: Thu, 24 Oct 2024 15:45:11 +0200 Subject: [PATCH] :memo: [#403] Document Notificaties API retry behavior --- docs/admin/notifications.rst | 45 ++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/docs/admin/notifications.rst b/docs/admin/notifications.rst index 8f9ebeac..3faa29b0 100644 --- a/docs/admin/notifications.rst +++ b/docs/admin/notifications.rst @@ -61,6 +61,51 @@ please refer to the `Open Zaak authorization documentation`_. For more information on how to set up subscriptions in Open Notificaties, please refer to the `Open Notificaties subscription documentation`_. +Automatic retry for notifications +================================= + +By default, sending notifications has automatic retry behaviour, i.e. if the notification +publishing task has failed, it will automatically be rescheduled/tried again until the maximum +retry limit has been reached. + +Autoretry explanation and configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Retry behaviour is implemented using binary exponential backoff with a delay factor, +the formula to calculate the time to wait until the next retry is as follows: + +.. math:: + + t = \text{backoff_factor} * 2^c + +where `t` is time in seconds and `c` is the number of retries that have been performed already. + +In the **Configuratie > Notificatiescomponentconfiguratie** admin page the autoretry settings +can be configured: + +* **Notification delivery max retries**: the maximum number of retries the task queue + will do if sending a notification has failed. Default is ``5``. +* **Notification delivery retry backoff**: a boolean or a number. If this option is set to + ``True``, autoretries will be delayed following the rules of binary exponential backoff. If + this option is set to a number, it is used as a delay factor. Default is ``3``. +* **Notification delivery retry backoff max**: an integer, specifying number of seconds. + If ``Notification delivery retry backoff`` is enabled, this option will set a maximum + delay in seconds between task autoretries. Default is ``48`` seconds. + +With the assumption that the requests are done immediately we can model the notification +tasks schedule with the default configurations: + +1. At 0s the request to create an Object is made, the notification task is scheduled, picked up + by worker and failed +2. At 3s with 3s delay the first retry happens (``2^0`` * ``Notification delivery retry backoff``) +3. At 9s with 6s delay - the second retry (``2^1`` * ``Notification delivery retry backoff``) +4. At 21s with 12s delay - the third retry +5. At 45s with 24s delay - the fourth retry +6. At 1m33s with 48s delay - the fifth retry, which is the last one. + +So if the Notification API is up after 1 min of downtime the default configuration can handle it +automatically. + .. _documentation: https://open-notificaties.readthedocs.io .. _Open Zaak authorization documentation: https://open-zaak.readthedocs.io/en/stable/manual/api-authorizations.html .. _Open Notificaties subscription documentation: https://open-notificaties.readthedocs.io/en/stable/manual/subscriptions.html