From 470b0452f611a1cd9a3cce61811f01c8cad104ee Mon Sep 17 00:00:00 2001 From: Jillian Vogel Date: Fri, 19 Jan 2024 00:23:39 +1030 Subject: [PATCH 1/2] docs: Adds ADR 11. Course and block data --- docs/decisions/0011_course_block_data.rst | 68 +++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 docs/decisions/0011_course_block_data.rst diff --git a/docs/decisions/0011_course_block_data.rst b/docs/decisions/0011_course_block_data.rst new file mode 100644 index 0000000..61d5b4c --- /dev/null +++ b/docs/decisions/0011_course_block_data.rst @@ -0,0 +1,68 @@ +11. Course and block data +######################### + +Status +****** + +Accepted + +Context +******* + +Aspects needs a way to access up-to-date course and block metadata in order to display human-friendly labels and data to +instructors in a way that makes sense in the context of their course. + +The tracking event data that Aspects processes contains "object IDs", or string representations of UsageKeys that +identify the piece of content the event relates to. But these UsageKeys are not very legible to humans, and the metadata +for that content (display title of the block, title of the course it appears in, its position in the course, etc.) is +part of the context where these events occur, and is not stored in the events themselves. + +Example: + +*"As a course author, I need to know how many times the videos in my course have been viewed so I can see which videos are +the most helpful (or the most confusing)."* + +Aspects receives tracking events every time someone watches a video, and so the videos watched can be counted purely +from these tracking events. However, videos which haven't been watched won't appear in the tracking events. To include +these unwatched videos in our chart, Aspects needs a way to query a course for all the video blocks it contains. + +Open edX sends signals (which, confusingly, are also called "events", see OEP-41) when some actions are taken on the +platform, including publishing a course, or updating a course outline. These signals can be used to tell Aspects when +its course or block data needs to be updated. + +Decision +******** + +Use Open edX's "course published" signal to trigger updating course outline and block relationship data stored in +Clickhouse. This approach is supported by `OEP-50 Hooks extension framework`_, and will be compatible with deployments +that use the Event Bus to manage signals sent between applications. + +Consequences +************ + +#. Create an "event sink" which uses course and/or block signals from within the LMS/CMS to trigger course and block + data synchronization to Clickhouse. +#. This "event sink" will run as a plugin on Open edX, and so has access to its models and data, which it can query and + package up for insertion in Clickhouse. +#. Aspects will create custom views into this course and block data to ensure that the most relevant information is + easily available to the datasets. + +Rejected Alternatives +********************* + +**Use course graph** + +Previously Aspects used the `coursegraph`_ application in Open edX as the source of truth for course outline and block +relationships. + +The synchronization script had to be run regularly in order to keep the data up-to-date. + +However as the Aspects project has grown, course data isn't the only Open edX data that Aspects needs a copy of. So the +more general solution of "event sinks" was preferred. + +References +********** + +.. OEP-41 Asynchronous Server Event Message Format: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0041-arch-async-server-event-messaging.html +.. OEP-50 Hooks extension framework: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0050-hooks-extension-framework.html +.. OEP-52 Event Bus Architecture: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0052-arch-event-bus-architecture.html From b529d6c1363d29e58e9ebdb9a1f54fd234379f09 Mon Sep 17 00:00:00 2001 From: Jillian Vogel Date: Tue, 23 Jan 2024 10:52:15 +1030 Subject: [PATCH 2/2] docs: fixed broken links --- docs/decisions/0011_course_block_data.rst | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/docs/decisions/0011_course_block_data.rst b/docs/decisions/0011_course_block_data.rst index 61d5b4c..a72ff6a 100644 --- a/docs/decisions/0011_course_block_data.rst +++ b/docs/decisions/0011_course_block_data.rst @@ -26,7 +26,7 @@ Aspects receives tracking events every time someone watches a video, and so the from these tracking events. However, videos which haven't been watched won't appear in the tracking events. To include these unwatched videos in our chart, Aspects needs a way to query a course for all the video blocks it contains. -Open edX sends signals (which, confusingly, are also called "events", see OEP-41) when some actions are taken on the +Open edX sends signals (which, confusingly, are also called "events", see `OEP-41`_) when some actions are taken on the platform, including publishing a course, or updating a course outline. These signals can be used to tell Aspects when its course or block data needs to be updated. @@ -34,8 +34,8 @@ Decision ******** Use Open edX's "course published" signal to trigger updating course outline and block relationship data stored in -Clickhouse. This approach is supported by `OEP-50 Hooks extension framework`_, and will be compatible with deployments -that use the Event Bus to manage signals sent between applications. +Clickhouse. This approach is supported by `OEP-50`_ Hooks extension framework, and will be compatible with deployments +that use the Event Bus (see `OEP-52`_) to manage signals sent between applications. Consequences ************ @@ -63,6 +63,11 @@ more general solution of "event sinks" was preferred. References ********** -.. OEP-41 Asynchronous Server Event Message Format: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0041-arch-async-server-event-messaging.html -.. OEP-50 Hooks extension framework: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0050-hooks-extension-framework.html -.. OEP-52 Event Bus Architecture: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0052-arch-event-bus-architecture.html +* `OEP-41`_ Asynchronous Server Event Message Format +* `OEP-50`_ Hooks Extension Framework +* `OEP-52`_ Event Bus Architecture + +.. _OEP-41: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0041-arch-async-server-event-messaging.html +.. _OEP-50: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0050-hooks-extension-framework.html +.. _OEP-52: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0052-arch-event-bus-architecture.html +.. _coursegraph: https://github.com/openedx/edx-platform/tree/master/cms/djangoapps/coursegraph