diff --git a/docs/how-tos/add-new-implementation.rst b/docs/how-tos/add-new-implementation.rst new file mode 100644 index 00000000..f0be7af8 --- /dev/null +++ b/docs/how-tos/add-new-implementation.rst @@ -0,0 +1,31 @@ +How to add a new concrete implementation of the event bus +========================================================= + +Context +------- + +Though the initial Open edX event bus was implemented using Kafka, the ultimate goal is for maintainers to be able to swap out different technologies like Redis or Pulsar. See `How to start using the Event bus `_ for more information. Existing Concrete Implementations + +- `Kafka `_ +- `Redis Streams `_ + +Producing +--------- + +There should be a producer class that inherits from `EventBusProducer `_ in openedx-events. + +The defined ``send`` method is meant to be called from within a signal receiver in the producing service. + +Consuming +--------- + +At a high level, the consumer should be a process that takes the signals and events from the broker and emits the signal with the event. There should be a consumer class that inherits from `EventBusConsumer `_ in openedx-events. + +The consumer class then needs to implement ``consume_indefinitely`` loop, which will stay running and listen to events as they come in. + +If you are doing this within Django, one thing to be aware of is Django will not do any of the automatic connection cleanup that it usually does per request. This means that if your database restarts while the ``consume_indefinitely`` loop is running, Django may try to hold on to a defunct connection. To address this, we have included an `utility function <../../openedx_events/tooling.py#L323>`_ in openedx-events which needs to called before processing any signal. Checkout `consumer.py `_ in event bus redis implementation. We’re also resetting the RequestCache in the utility function, and there may be later, more comprehensive changes.. + +Abstraction tickets +------------------- + +The known remaining work for a fully abstracted event bus is captured in the `Abstraction tickets `_ diff --git a/docs/how-tos/index.rst b/docs/how-tos/index.rst index f8908b33..132baca5 100644 --- a/docs/how-tos/index.rst +++ b/docs/how-tos/index.rst @@ -9,3 +9,4 @@ How-tos adding-events-to-a-service adding-events-to-event-bus using-events + add-new-implementation