Skip to content

Commit

Permalink
fixup! docs: adr for enabling producing to event bus via settings
Browse files Browse the repository at this point in the history
  • Loading branch information
navinkarkera committed Jun 6, 2023
1 parent eace1ee commit 636ed8c
Showing 1 changed file with 31 additions and 15 deletions.
46 changes: 31 additions & 15 deletions docs/decisions/0011-producing-to-event-bus-via-settings.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,14 @@
Status
******

**Provisional** 2023-05-23
**Accepted** 2023-06-05

Context
*******

Since the initial implementation of the event bus allowed only a single event type to be published to a topic, details like topic name, consumer group name, and consumer name were configured via code. The current implementation of openedx-events does not actually push any events to the underlying implementations like `edx-event-bus-kafka`_ and `edx-event-bus-redis`_. The event-producing application is expected to create a signal handler (since openedx-events subclasses Django signals) to catch the event and push it into the event bus. Some examples of the handlers: `handlers example`_.
The initial implementation of the event bus allowed only a single event type to be published to a topic, with details like topic name, consumer group name, and consumer name were configured via code. We weren't sure what the api would finally look like, and whether the event bus config would live as part of the definition of the event, so we just went with explicit code that we could iterate upon.

The current implementation of openedx-events does not actually push any events to the underlying implementations like `edx-event-bus-kafka`_ and `edx-event-bus-redis`_. The event-producing application is expected to create a signal handler (since openedx-events subclasses Django signals) to catch the event and push it into the event bus. Some examples of the handlers: `handlers example`_.

.. _handlers example: https://github.com/openedx/edx-platform/blob/27b8d2f68d5dfaf84755e7d7f8dccc97ce3be509/cms/djangoapps/contentstore/signals/handlers.py#L162-L210
.. _edx-event-bus-kafka: https://github.com/openedx/event-bus-kafka
Expand All @@ -21,26 +23,35 @@ This ADR aims to propose a solution for configuring the details, like the topic
Decision
********

Update constructor of ``OpenEdxPublicSignal`` class to optionally accept ``topics`` and ``event_key_field`` and store them. These values can be then used as default while pushing them to event bus implementations. Add ``topics`` and ``event_key_field`` for existing signal definitions in openedx_events repository. Once this ADR is implemented, make ``topics`` and ``event_key_field`` required in ``OpenEdxPublicSignal`` constructor.

Create a generic signal handler to push events to the event bus. This handler should be attached to or connected to the signals that are enabled in Django settings. The configuration format will be as shown below:

.. code-block:: python
OPEN_EDX_EVENTS_CONFIG = {
# .. setting_name: EVENT_BUS_PRODUCER_CONFIG
# .. setting_default: By default all signals are configured using initialization values from signal but not enabled.
# .. setting_description: Dictionary specifying event_types (keys) and its configuration dictionary as values.
# Each configuration dictionary contains a flag denoting whether the event will be enabled or disabled and
# optionally details like topics/stream names where the event will be pushed to, event_key_field which is
# and period delimited string path to event data field to use as event key.
# Note: The topic names should not include environment prefix as it will be dynamically added based on
# EVENT_BUS_TOPIC_PREFIX setting.
EVENT_BUS_PRODUCER_CONFIG = {
'org.openedx.content_authoring.xblock.deleted.v1': {
'topics': 'xblock-status',
'event_key_field': 'xblock_info.usage_key',
'event_data_key': 'xblock_info',
'event_key_field': 'xblock_info.usage_key', # override default event_key_field
'enabled': True,
},
'org.openedx.content_authoring.xblock.published.v1': {
'topics': ['xblock-status', 'xblock-published'],
'event_key_field': 'xblock_info.usage_key',
'event_data_key': 'xblock_info',
'topics': ['content-authoring-xblock-lifecycle', 'xblock-published'], # override default topics
'enabled': True,
},
'org.openedx.content_authoring.xblock.duplicated.v1': {
'enabled': True, # enable this event and use default values
},
}
This configuration will be read in openedx_events ``apps.OpenedxEventsConfig(AppConfig).ready`` method and a generic signal handler will be connected to the event_types (keys) listed in the configuration after validating its format.
This configuration will be read and combined with the default config in openedx_events ``apps.OpenedxEventsConfig(AppConfig).ready`` method and a generic signal handler will be connected to the event_types (keys) listed in the configuration after validating its format.

.. code-block:: python
Expand All @@ -49,8 +60,9 @@ This configuration will be read in openedx_events ``apps.OpenedxEventsConfig(App
config = read_config()
# validate the config dictionary and raise errors if any
validate_config()
for configured_signal in config:
connect_or_disconnect_handlers(configured_signal)
for signal in all_signals:
if (topic and event_key_field is configured) and enabled:
connect_or_disconnect_handlers(configured_signal)
The generic handler will again read the configuration and get details for event or signal triggered and push it to the event bus.

Expand All @@ -61,16 +73,20 @@ The generic handler will again read the configuration and get details for event
for topic in topics:
push_event_to_implementation(signal, config)
Optionally, add a function to ``OpenEdxPublicSignal`` to return default configuration for documentating them.

Consequences
************

* Applications producing to event_bus can push events without requiring any additional code.
* Users can push related events to the same topic, which will allow them to run a single consumer and process the events in the correct order.
* Users can push multiple related event_types to the same topic, which will allow them to run a single consumer and process the events in the correct order.
* The configuration is a dictionary, which makes it flexible but difficult to enforce schemas and maintain them.
* Producing as well as consuming applications will be required to add ``openedx_events`` as an application in their ``INSTALLED_APPS`` settings.
* Producing applications similar to consuming applications will be required to add ``openedx_events`` as an application in their ``INSTALLED_APPS`` settings.
* Adding details like ``topic`` and ``event_key_field`` to signal definition will keep configuration compact if user does not want to override them.

Rejected Alternatives
*********************

* Implementing configurable handlers in the host applications will require repeating the code in each host application.
* Following the current way of using fixed handlers will force users to create multiple consumer processes for consuming multiple events, and hence the order of execution is not guaranteed across events.
* Following the current way of using fixed handlers will restricts users ability to combine events in a single topic based on their preference.

0 comments on commit 636ed8c

Please sign in to comment.