From 42568a5e01cdeaeb3c40c22150921bbdcacaf976 Mon Sep 17 00:00:00 2001 From: Navin Karkera Date: Thu, 25 May 2023 10:00:00 +0530 Subject: [PATCH] fixup! docs: adr for enabling producing to event bus via settings --- ...11-producing-to-event-bus-via-settings.rst | 76 ++++++------------- 1 file changed, 22 insertions(+), 54 deletions(-) diff --git a/docs/decisions/0011-producing-to-event-bus-via-settings.rst b/docs/decisions/0011-producing-to-event-bus-via-settings.rst index e12cc855..13dcb2c9 100644 --- a/docs/decisions/0011-producing-to-event-bus-via-settings.rst +++ b/docs/decisions/0011-producing-to-event-bus-via-settings.rst @@ -9,31 +9,19 @@ Status Context ******* -Since the initial implementation of the event bus allowed only a single event -type to be published to a topic, details like topic name, consumer group name, -and consumer name were configured via code. The current implementation of -openedx-events does not actually push any events to the underlying -implementations like `edx-event-bus-kafka`_ and `edx-event-bus-redis`_. The -event-producing application is expected to create a signal handler (since -openedx-events subclasses Django signals) to catch the event and push it into -the event bus. Some examples of the handlers can be found `here`_. - -.. _here: https://github.com/openedx/edx-platform/blob/27b8d2f68d5dfaf84755e7d7f8dccc97ce3be509/cms/djangoapps/contentstore/signals/handlers.py#L162-L210 +Since the initial implementation of the event bus allowed only a single event type to be published to a topic, details like topic name, consumer group name, and consumer name were configured via code. The current implementation of openedx-events does not actually push any events to the underlying implementations like `edx-event-bus-kafka`_ and `edx-event-bus-redis`_. The event-producing application is expected to create a signal handler (since openedx-events subclasses Django signals) to catch the event and push it into the event bus. Some examples of the handlers: `handlers example`_. + +.. _handlers example: https://github.com/openedx/edx-platform/blob/27b8d2f68d5dfaf84755e7d7f8dccc97ce3be509/cms/djangoapps/contentstore/signals/handlers.py#L162-L210 .. _edx-event-bus-kafka: https://github.com/openedx/event-bus-kafka .. _edx-event-bus-redis: https://github.com/openedx/event-bus-redis -This ADR aims to propose a solution for configuring the details, like the topic. -name, consumer group name, etc. via Django settings as well as pushing events to -the event bus without requiring the producing application to write additional -handlers. +This ADR aims to propose a solution for configuring the details, like the topic name, consumer group name, etc. via Django settings as well as pushing events to the event bus without requiring the producing application to write additional handlers. Decision ******** -Create a generic signal handler to push events to the event bus. This handler -should be attached to or connected to the signals that are enabled in Django settings. -of the application. The configuration format will be as shown below: +Create a generic signal handler to push events to the event bus. This handler should be attached to or connected to the signals that are enabled in Django settings. The configuration format will be as shown below: .. code-block:: python @@ -49,58 +37,38 @@ of the application. The configuration format will be as shown below: 'event_key_field': 'xblock_info.usage_key', 'event_data_key': 'xblock_info', 'enabled': True, - } + }, } -This configuration will be read in openedx_events ``apps.OpenedxEventsConfig(AppConfig).ready`` -method and a generic signal handler will be connected to the event_types (keys) -listed in the configuration after validating its format. +This configuration will be read in openedx_events ``apps.OpenedxEventsConfig(AppConfig).ready`` method and a generic signal handler will be connected to the event_types (keys) listed in the configuration after validating its format. .. code-block:: python def ready(self): load_all_signals() - signals_config = getattr(settings, "OPEN_EDX_EVENTS_CONFIG", {}) - # validate signals_config before moving to next step - for event_type, config in signals_config.items(): - signal = OpenEdxPublicSignal.get_signal_by_type(event_type) - if config.get("enabled", True): - signal.connect(general_signal_handler, dispatch_uid=event_type) - else: - signal.disconnect(general_signal_handler, dispatch_uid=event_type) - return super().ready() - -The generic handler will again read the configuration and get details for -event or signal triggered and push it to the event bus. + config = read_config() + for configured_signal in config: + connect_or_disconnect_handlers(configured_signal) + +The generic handler will again read the configuration and get details for event or signal triggered and push it to the event bus. .. code-block:: python def general_signal_handler(sender, signal, **kwargs): - signal_config = getattr(settings, "OPEN_EDX_EVENTS_CONFIG", {}).get(signal.event_type) - if signal_config: - if isinstance(signal_config["topics"], str): - topics = [signal_config['topics']] - else: - topics = signal_config['topics'] - for topic in topics: - get_producer().send( - signal=signal, - topic=topic, - event_key_field=signal_config['event_key_field'], - event_data={signal_config['event_data_key']: kwargs[signal_config['event_data_key']]}, - event_metadata=kwargs['metadata'], - ) + config = read_config(signal) + for topic in topics: + push_event_to_implementation(signal, config) Consequences ************ -* Applications producing to event_bus can push events without requiring any - additional code. -* The configuration is a dictionary, which makes it flexible but difficult to - enforce schemas and maintain them. -* Producing as well as consuming applications will be required to add - ``openedx_events`` as an application in their ``INSTALLED_APPS`` settings. + +* Applications producing to event_bus can push events without requiring any additional code. +* Users can push related events to the same topic, which will allow them to run a single consumer and process the events in the correct order. +* The configuration is a dictionary, which makes it flexible but difficult to enforce schemas and maintain them. +* Producing as well as consuming applications will be required to add ``openedx_events`` as an application in their ``INSTALLED_APPS`` settings. Rejected Alternatives ********************* -None +* Implementing configurable handlers in the host applications will require repeating the code in each host application. +* Following the current way of using fixed handlers will force users to create multiple consumer processes for consuming multiple events, and hence the order of execution is not guaranteed across events.