Skip to content

Commit

Permalink
fixup! docs: adr for enabling producing to event bus via settings
Browse files Browse the repository at this point in the history
  • Loading branch information
navinkarkera committed May 25, 2023
1 parent 4a5563e commit 42568a5
Showing 1 changed file with 22 additions and 54 deletions.
76 changes: 22 additions & 54 deletions docs/decisions/0011-producing-to-event-bus-via-settings.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,31 +9,19 @@ Status
Context
*******

Since the initial implementation of the event bus allowed only a single event
type to be published to a topic, details like topic name, consumer group name,
and consumer name were configured via code. The current implementation of
openedx-events does not actually push any events to the underlying
implementations like `edx-event-bus-kafka`_ and `edx-event-bus-redis`_. The
event-producing application is expected to create a signal handler (since
openedx-events subclasses Django signals) to catch the event and push it into
the event bus. Some examples of the handlers can be found `here`_.

.. _here: https://github.com/openedx/edx-platform/blob/27b8d2f68d5dfaf84755e7d7f8dccc97ce3be509/cms/djangoapps/contentstore/signals/handlers.py#L162-L210
Since the initial implementation of the event bus allowed only a single event type to be published to a topic, details like topic name, consumer group name, and consumer name were configured via code. The current implementation of openedx-events does not actually push any events to the underlying implementations like `edx-event-bus-kafka`_ and `edx-event-bus-redis`_. The event-producing application is expected to create a signal handler (since openedx-events subclasses Django signals) to catch the event and push it into the event bus. Some examples of the handlers: `handlers example`_.

.. _handlers example: https://github.com/openedx/edx-platform/blob/27b8d2f68d5dfaf84755e7d7f8dccc97ce3be509/cms/djangoapps/contentstore/signals/handlers.py#L162-L210
.. _edx-event-bus-kafka: https://github.com/openedx/event-bus-kafka
.. _edx-event-bus-redis: https://github.com/openedx/event-bus-redis

This ADR aims to propose a solution for configuring the details, like the topic.
name, consumer group name, etc. via Django settings as well as pushing events to
the event bus without requiring the producing application to write additional
handlers.
This ADR aims to propose a solution for configuring the details, like the topic name, consumer group name, etc. via Django settings as well as pushing events to the event bus without requiring the producing application to write additional handlers.


Decision
********

Create a generic signal handler to push events to the event bus. This handler
should be attached to or connected to the signals that are enabled in Django settings.
of the application. The configuration format will be as shown below:
Create a generic signal handler to push events to the event bus. This handler should be attached to or connected to the signals that are enabled in Django settings. The configuration format will be as shown below:

.. code-block:: python
Expand All @@ -49,58 +37,38 @@ of the application. The configuration format will be as shown below:
'event_key_field': 'xblock_info.usage_key',
'event_data_key': 'xblock_info',
'enabled': True,
}
},
}
This configuration will be read in openedx_events ``apps.OpenedxEventsConfig(AppConfig).ready``
method and a generic signal handler will be connected to the event_types (keys)
listed in the configuration after validating its format.
This configuration will be read in openedx_events ``apps.OpenedxEventsConfig(AppConfig).ready`` method and a generic signal handler will be connected to the event_types (keys) listed in the configuration after validating its format.

.. code-block:: python
def ready(self):
load_all_signals()
signals_config = getattr(settings, "OPEN_EDX_EVENTS_CONFIG", {})
# validate signals_config before moving to next step
for event_type, config in signals_config.items():
signal = OpenEdxPublicSignal.get_signal_by_type(event_type)
if config.get("enabled", True):
signal.connect(general_signal_handler, dispatch_uid=event_type)
else:
signal.disconnect(general_signal_handler, dispatch_uid=event_type)
return super().ready()
The generic handler will again read the configuration and get details for
event or signal triggered and push it to the event bus.
config = read_config()
for configured_signal in config:
connect_or_disconnect_handlers(configured_signal)
The generic handler will again read the configuration and get details for event or signal triggered and push it to the event bus.

.. code-block:: python
def general_signal_handler(sender, signal, **kwargs):
signal_config = getattr(settings, "OPEN_EDX_EVENTS_CONFIG", {}).get(signal.event_type)
if signal_config:
if isinstance(signal_config["topics"], str):
topics = [signal_config['topics']]
else:
topics = signal_config['topics']
for topic in topics:
get_producer().send(
signal=signal,
topic=topic,
event_key_field=signal_config['event_key_field'],
event_data={signal_config['event_data_key']: kwargs[signal_config['event_data_key']]},
event_metadata=kwargs['metadata'],
)
config = read_config(signal)
for topic in topics:
push_event_to_implementation(signal, config)
Consequences
************
* Applications producing to event_bus can push events without requiring any
additional code.
* The configuration is a dictionary, which makes it flexible but difficult to
enforce schemas and maintain them.
* Producing as well as consuming applications will be required to add
``openedx_events`` as an application in their ``INSTALLED_APPS`` settings.

* Applications producing to event_bus can push events without requiring any additional code.
* Users can push related events to the same topic, which will allow them to run a single consumer and process the events in the correct order.
* The configuration is a dictionary, which makes it flexible but difficult to enforce schemas and maintain them.
* Producing as well as consuming applications will be required to add ``openedx_events`` as an application in their ``INSTALLED_APPS`` settings.

Rejected Alternatives
*********************

None
* Implementing configurable handlers in the host applications will require repeating the code in each host application.
* Following the current way of using fixed handlers will force users to create multiple consumer processes for consuming multiple events, and hence the order of execution is not guaranteed across events.

0 comments on commit 42568a5

Please sign in to comment.