From 0424734c268aea36312117335b92ea79bff2c8ee Mon Sep 17 00:00:00 2001 From: John Whitlock Date: Tue, 16 Apr 2024 09:30:27 -0500 Subject: [PATCH] Update Django docs (#109) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Specify that the middleware serves the views Since the DockerflowMiddleware serves the health monitoring views, it is unneccessary to add them to urls.py * Expand DOCKERFLOW_VERSION_CALLBACK docs Add links to other documentation. Fix the signature of the callback function. Move so that the section is alpha-sorted. Fixup version callback * Add DOCKERFLOW_REQUEST_ID_HEADER_NAME docs * Add DOCKERFLOW_SUMMARY_LOG_QUERYSTRING docs * Add DEBUG docs * Add SILENCED_SYSTEM_CHECKS docs * Add DOCKERFLOW_HEARTBEAT_FAILED_STATUS_CODE docs * MIDDLEWARE_CLASSES was removed in Django 2.0, 2017 * Add dockerflow.checks.register to log config * Use dockerflow as logger name These logs should propagate up, so use more general logging level. * Fix name of RequestIdLogFilter * Add SECURE_REDIRECT_EXEMPT docs * More MIDDLEWARE_CLASSES removal * Less bear 🐻 * Fix name of RequestIdLogFilter --- docs/django.rst | 143 +++++++++++++++++++++++++++++++++++++++-------- docs/fastapi.rst | 4 +- docs/flask.rst | 4 +- docs/logging.rst | 4 +- docs/sanic.rst | 4 +- 5 files changed, 128 insertions(+), 31 deletions(-) diff --git a/docs/django.rst b/docs/django.rst index d4b61d9..3a52a82 100644 --- a/docs/django.rst +++ b/docs/django.rst @@ -7,9 +7,6 @@ Django projects that want to follow the Dockerflow specs: - A Python logging formatter following the `mozlog`_ format to be used in the ``LOGGING`` setting. -- A middleware to emit `request.summary`_ log records based on request specific - data. - - Views for health monitoring: - ``/__version__`` - Serves a ``version.json`` file @@ -19,6 +16,8 @@ Django projects that want to follow the Dockerflow specs: - ``/__lbheartbeat__`` - Retuns a HTTP 200 response +- A middleware to emit `request.summary`_ log records based on request specific + data, and to serve the health monitoring views. - Signals for passed and failed heartbeats. @@ -43,20 +42,32 @@ To install ``python-dockerflow``'s Django support please follow these steps: .. seealso:: :ref:`django-versions` for more information -#. Add the ``DockerflowMiddleware`` to your ``MIDDLEWARE_CLASSES`` or - ``MIDDLEWARE`` setting:: +#. Add the ``DockerflowMiddleware`` to your ``MIDDLEWARE`` setting:: - MIDDLEWARE_CLASSES = ( + MIDDLEWARE = [ # ... + # 'django.middleware.security.SecurityMiddleware', 'dockerflow.django.middleware.DockerflowMiddleware', # ... - ) + ] + +#. (Optional) Add the healthcheck views to SECURE_REDIRECT_EXEMPT_, so they can + be used as `Kubernetes liveness checks`_:: + + SECURE_REDIRECT_EXEMPT = [ + r"^__version__/?$", + r"^__heartbeat__/?$", + r"^__lbheartbeat__/?$", + ] #. :ref:`Configure logging ` to use the :class:`~dockerflow.logging.JsonLogFormatter` logging formatter for the ``request.summary`` logger (you may have to extend your existing logging configuration!). +.. _`Kubernetes liveness checks`: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/ +.. _SECURE_REDIRECT_EXEMPT: https://docs.djangoproject.com/en/stable/ref/settings/#secure-redirect-exempt + .. _django-config: Configuration @@ -255,6 +266,7 @@ Health monitoring Health monitoring happens via three different views following the Dockerflow_ spec: +.. _http_get_version: .. http:get:: /__version__ The view that serves the :ref:`version information `. @@ -284,16 +296,25 @@ spec: :statuscode 200: no error :statuscode 404: a version.json wasn't found +.. _http_get_heartbeat: .. http:get:: /__heartbeat__ The heartbeat view will go through the list of configured Dockerflow checks in the :ref:`DOCKERFLOW_CHECKS` setting, run each check, and, if - `settings.DEBUG` is `True`, add their results to a JSON response. + ``settings.DEBUG`` is ``True``, add their results to a JSON response. The view will return HTTP responses with either a status code of 200 if all checks ran successfully or 500 if there was one or more warnings or errors returned by the checks. + The check processes will log to ``dockerflow.checks.register``. Failed + checks that cause the heartbeat to fail are logged at ``ERROR`` level + or higher. Successful checks are logged at ``INFO`` level and higher. + The check setup process is logged at the ``DEBUG`` level. Since failure + details are omitted with ``DEBUG=False``, this logger should emit logs + at ``WARNING`` or ``ERROR`` level in production, so that the logs can + be used to diagnose heartbeat failures. + **Custom Dockerflow checks:** To write your own custom Dockerflow checks, please follow the documentation @@ -311,7 +332,7 @@ spec: GET /__heartbeat__ HTTP/1.1 Host: example.com - **Example response**: + **Example response, DEBUG=True**: .. sourcecode:: http @@ -336,6 +357,18 @@ spec: } } + **Example response, DEBUG=False**: + + .. sourcecode:: http + + HTTP/1.1 500 Internal Server Error + Vary: Accept-Encoding + Content-Type: application/json + + { + "status": "warning" + } + :statuscode 200: no error, with potential warnings :statuscode 500: there was an error @@ -388,7 +421,7 @@ configure **at least** the ``request.summary`` logger that way:: }, 'filters': { 'request_id': { - '()': 'dockerflow.logging.RequestIdFilter', + '()': 'dockerflow.logging.RequestIdLogFilter', }, }, 'handlers': { @@ -404,10 +437,15 @@ configure **at least** the ``request.summary`` logger that way:: 'handlers': ['console'], 'level': 'DEBUG', }, + 'dockerflow': { + 'handlers': ['console'], + 'level': 'WARNING', + }, } } -In order to include querystrings in the request summary log, set this flag in settings: +In order to include querystrings in the request summary log, set +:ref:`this flag ` in settings: .. code-block:: python @@ -427,15 +465,17 @@ The *MozLog* formatter will output ``Fields`` application-specific fields. It ca extra={"phase": "started", "host": host, "port": port} ) +.. _requests_correlation_id: Requests Correlation ID ~~~~~~~~~~~~~~~~~~~~~~~ A unique request ID is read from the ``X-Request-ID`` request header, and a UUID4 value is generated if unset. -Leveraging the ``RequestIdFilter`` in logging configuration as shown above will add a ``rid`` field into the ``Fields`` entry of all log messages. +Leveraging the ``RequestIdLogFilter`` in logging configuration as shown above will add a ``rid`` field into the ``Fields`` entry of all log messages. -The header name to obtain the request ID can be customized in settings: +The header name to obtain the request ID can be +`customized in settings `_: .. code-block:: python @@ -460,9 +500,9 @@ in your Django projet: STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles') -#. Add the middleware to your ``MIDDLEWARE`` (or ``MIDDLEWARE_CLASSES``) setting:: +#. Add the middleware to your ``MIDDLEWARE`` setting:: - MIDDLEWARE_CLASSES = [ + MIDDLEWARE = [ # 'django.middleware.security.SecurityMiddleware', 'whitenoise.middleware.WhiteNoiseMiddleware', # ... @@ -491,16 +531,19 @@ the section about `Using WhiteNoise with Django`_ in its documentation. Settings -------- -.. _DOCKERFLOW_VERSION_CALLBACK: +``DEBUG`` +~~~~~~~~~ -``DOCKERFLOW_VERSION_CALLBACK`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The standard Django setting DEBUG_ is referenced by the +:ref:`__heartbeat__` view. If it is set to ``True``, then: + +- Django's deployment checks are run. These are the additional checks ran by + including the ``--deploy`` flag, such as ``python manage.py check --deploy``. -The dotted import path for the callable that -returns the content to return under ``/__version__``. +- The ``checks`` and ``details`` objects are omitted from the JSON response, + leaving only the ``status`` of ``ok``, ``warning`` or ``error``. -Defaults to ``'dockerflow.version.get_version'`` which will be passed the -``BASE_DIR`` setting by default. +.. _DEBUG: https://docs.djangoproject.com/en/stable/ref/settings/#debug .. _DOCKERFLOW_CHECKS: @@ -508,7 +551,8 @@ Defaults to ``'dockerflow.version.get_version'`` which will be passed the ~~~~~~~~~~~~~~~~~~~~~ A list of dotted import paths to register during -Django setup, to be used in the rendering of the ``/__heartbeat__`` view. +Django setup, to be used in the rendering of the +:ref:`__heartbeat__` view. Defaults to: .. code-block:: python @@ -517,3 +561,56 @@ Defaults to: 'dockerflow.django.checks.check_database_connected', 'dockerflow.django.checks.check_migrations_applied', ] + +.. _DOCKERFLOW_HEARTBEAT_FAILED_STATUS_CODE: + +``DOCKERFLOW_HEARTBEAT_FAILED_STATUS_CODE`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the :ref:`__heartbeat__` view, this setting +is used to set the status code when a check fails at ``error`` or higher. +If unset, the default is ``500`` for an Internal Server Error. + +.. _DOCKERFLOW_REQUEST_ID_HEADER_NAME: + +``DOCKERFLOW_REQUEST_ID_HEADER_NAME`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The case-insenstive name of the HTTP header referenced for identifying a +request. The default is `X-Request-ID`, used by the +`Heroku router `_. +If the header is not set by the incoming request, a UUID is generated +for the :ref:`requests correlation ID`. + +A good value is the header name used by your deployment infrastructure. +For example, the Google Cloud Platform sets the W3C standard ``traceparent`` +header as well as a legacy ``X-Cloud-Trace-Context`` header for +`trace context `_. + +.. _DOCKERFLOW_SUMMARY_LOG_QUERYSTRING: + +``DOCKERFLOW_SUMMARY_LOG_QUERYSTRING`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If set to ``True``, then the request summary log will include the querystring. +This defaults to ``False``, in case there is user-sensitive information in +some querystrings. + +.. _DOCKERFLOW_VERSION_CALLBACK: + + +``DOCKERFLOW_VERSION_CALLBACK`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The dotted import path for the callable that takes a +`HttpRequest `_ +and returns the :ref:`version content` to return under +:ref:`__version__`. This defaults to ``dockerflow.version.get_version``. + +``SILENCED_SYSTEM_CHECKS`` +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The standard Django setting SILENCED_SYSTEM_CHECKS_ is used by the +:ref:`__heartbeat__` view to omit the named checks. + +.. _SILENCED_SYSTEM_CHECKS: https://docs.djangoproject.com/en/stable/ref/settings/#silenced-system-checks diff --git a/docs/fastapi.rst b/docs/fastapi.rst index fe4fe75..fe2e734 100644 --- a/docs/fastapi.rst +++ b/docs/fastapi.rst @@ -300,7 +300,7 @@ for at least the ``request.summary`` logger: }, 'filters': { 'request_id': { - '()': 'dockerflow.logging.RequestIdFilter', + '()': 'dockerflow.logging.RequestIdLogFilter', }, }, 'handlers': { @@ -346,7 +346,7 @@ Requests Correlation ID A unique request ID is read from the ``X-Request-ID`` request header, and a UUID4 value is generated if unset. -Leveraging the ``RequestIdFilter`` in logging configuration as shown above will add a ``rid`` field into the ``Fields`` entry of all log messages. +Leveraging the ``RequestIdLogFilter`` in logging configuration as shown above will add a ``rid`` field into the ``Fields`` entry of all log messages. The header name to obtain the request ID can be customized in settings: diff --git a/docs/flask.rst b/docs/flask.rst index c77834e..c36d2c4 100644 --- a/docs/flask.rst +++ b/docs/flask.rst @@ -443,7 +443,7 @@ for at least the ``request.summary`` logger:: }, 'filters': { 'request_id': { - '()': 'dockerflow.logging.RequestIdFilter', + '()': 'dockerflow.logging.RequestIdLogFilter', }, }, 'handlers': { @@ -486,7 +486,7 @@ Requests Correlation ID A unique request ID is read from the ``X-Request-ID`` request header, and a UUID4 value is generated if unset. -Leveraging the ``RequestIdFilter`` in logging configuration as shown above will add a ``rid`` field into the ``Fields`` entry of all log messages. +Leveraging the ``RequestIdLogFilter`` in logging configuration as shown above will add a ``rid`` field into the ``Fields`` entry of all log messages. The header name to obtain the request ID can be customized in settings: diff --git a/docs/logging.rst b/docs/logging.rst index 1b6c255..d1a8dea 100644 --- a/docs/logging.rst +++ b/docs/logging.rst @@ -41,7 +41,7 @@ this:: }, 'filters': { 'request_id': { - '()': 'dockerflow.logging.RequestIdFilter', + '()': 'dockerflow.logging.RequestIdLogFilter', }, }, 'handlers': { @@ -90,7 +90,7 @@ thing as the dictionary based configuratio above: keys = request_id [filter_request_id] - class = dockerflow.logging.RequestIdFilter + class = dockerflow.logging.RequestIdLogFilter [logger_root] level = INFO diff --git a/docs/sanic.rst b/docs/sanic.rst index 034272f..08a97b5 100644 --- a/docs/sanic.rst +++ b/docs/sanic.rst @@ -423,7 +423,7 @@ for at least the ``request.summary`` logger:: }, 'filters': { 'request_id': { - '()': 'dockerflow.logging.RequestIdFilter', + '()': 'dockerflow.logging.RequestIdLogFilter', }, }, 'handlers': { @@ -485,7 +485,7 @@ Requests Correlation ID A unique request ID is read from the ``X-Request-ID`` request header, and a UUID4 value is generated if unset. -Leveraging the ``RequestIdFilter`` in logging configuration as shown above will add a ``rid`` field into the ``Fields`` entry of all log messages. +Leveraging the ``RequestIdLogFilter`` in logging configuration as shown above will add a ``rid`` field into the ``Fields`` entry of all log messages. The header name to obtain the request ID can be customized in settings: