Skip to content

Commit

Permalink
docs: simplify consuming docs in how-to
Browse files Browse the repository at this point in the history
  • Loading branch information
navinkarkera committed Dec 8, 2023
1 parent ce90633 commit bafcb7a
Show file tree
Hide file tree
Showing 4 changed files with 9 additions and 5 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Change Log
Unreleased
----------

[9.2.1] - 2023-12-08
[9.3.0] - 2023-12-08
--------------------
Added
~~~~~
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ How to add a new concrete implementation of the event bus
Context
-------

Though the initial Open edX event bus was implemented using Kafka, the ultimate goal is for maintainers to be able to swap out different technologies like Redis or Pulsar. See `How to start using the Event bus <https://openedx.atlassian.net/wiki/spaces/AC/pages/3508699151>`_ for more information. Existing Concrete Implementations
Here is a list of the existing concrete implementations of the event bus:

- `Kafka <https://github.com/openedx/event-bus-kafka>`_
- `Redis Streams <https://github.com/openedx/event-bus-redis>`_

This how-to is to help you add a new concrete implementation, for example using Pulsar or some other technology.

Producing
---------

Expand All @@ -23,7 +25,9 @@ At a high level, the consumer should be a process that takes the signals and eve

The consumer class then needs to implement ``consume_indefinitely`` loop, which will stay running and listen to events as they come in.

If you are doing this within Django, one thing to be aware of is Django will not do any of the automatic connection cleanup that it usually does per request. This means that if your database restarts while the ``consume_indefinitely`` loop is running, Django may try to hold on to a defunct connection. To address this, we have included an `utility function <../../openedx_events/tooling.py#L323>`_ in openedx-events which needs to called before processing any signal. Checkout `consumer.py <https://github.com/openedx/event-bus-redis/blob/main/edx_event_bus_redis/internal/consumer.py>`_ in event bus redis implementation. We’re also resetting the RequestCache in the utility function, and there may be later, more comprehensive changes..
We have included an `utility function <../../openedx_events/tooling.py#L323>`_ in openedx-events which needs to be called before processing any signal. Currently, it reconnects the db connection if required as well as clears RequestCache and there may be later, more comprehensive changes. These steps mimic some setup/teardown that is normally performed by Django in its request/response based architecture.

Checkout `consumer.py <https://github.com/openedx/event-bus-redis/blob/main/edx_event_bus_redis/internal/consumer.py>`_ in event bus redis implementation.

Abstraction tickets
-------------------
Expand Down
2 changes: 1 addition & 1 deletion docs/how-tos/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ How-tos
adding-events-to-a-service
adding-events-to-event-bus
using-events
add-new-implementation
add-new-event-bus-concrete-implementation
2 changes: 1 addition & 1 deletion openedx_events/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@
more information about the project.
"""

__version__ = "9.2.1"
__version__ = "9.3.0"

0 comments on commit bafcb7a

Please sign in to comment.