Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: replaces references to OARS with Aspects in the ADRs #104

Merged
merged 1 commit into from
Jan 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/decisions/0001-open-analytics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ privacy, scalability, budget, and expertise.
Decision
********

We will create the Open Analytics Reference System (OARS) that combines existing open-source projects
We will create the Aspects Analytics system (Aspects) that combines existing open-source projects
into a preconfigured bundle that can be easily deployed using Tutor.

These projects will include:
Expand Down
2 changes: 1 addition & 1 deletion docs/decisions/0002_xapi.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ development of the `event-routing-backends`_ package as the method of implementa
Decision
********

The Open Analytics Reference System (OARS) will support xAPI as its default standard. Tracking
The Aspects Analytics system (Aspects) will support xAPI as its default standard. Tracking
log events will be transformed to xAPI statements by event-routing-backends package and sent to an
xAPI-supporting Learning Record Store (LRS). The specific LRS and backing database chosen will be the
subject of future ADRs.
Expand Down
6 changes: 3 additions & 3 deletions docs/decisions/0003_superset.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@ to manage.
Decision
********

The Open Analytics Reference System (OARS) will use `Apache Superset`_ as its primary data visualization
The Aspects Analytics system (Aspects) will use `Apache Superset`_ as its primary data visualization
and exploration tool. When developing a plan to support analytics in Open edX installs we have looked
for a solution that offered at least the level of functionality we were able to get from the Insights
user interface, as well as the following:

- Open source
- Deployable and configurable to work with OARS using Tutor
- Deployable and configurable to work with Aspects using Tutor
- Able to use the LMS as an authentication and authorization provider
- Capable of handling large quantities of data
- Minimal new technology / expertise needed beyond the usual Open edX stack
Expand All @@ -48,7 +48,7 @@ or expensive paid solutions.
Consequences
************

Superset will be integrated into OARS via a Tutor plugin that:
Superset will be integrated into Aspects via a Tutor plugin that:

- Allows it to share the existing Tutor redis and MySQL services
- Integrates it with our chosen analytic database (detailed in a future ADR)
Expand Down
14 changes: 7 additions & 7 deletions docs/decisions/0004_clickhouse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@ Additionally as MySQL is a transactional database it is not optimized for a larg
workload, creating issues at scale that meant more pre-processing (and therefore data delay).

Technology in the analytics space has evolved rapidly in the 10 years since Insights was designed,
unlocking new capabilities that were not possible at the time. In development of the OARS system we
unlocking new capabilities that were not possible at the time. In development of the Aspects system we
have an opportunity simplify and improve our collection and use of analytics data, as well as greatly
lower the skill and cost requirements of running such a system.

Decision
********

The Open Analytics Reference System (OARS) will use `ClickHouse`_ as its analytic database. This
The Aspects Analytics system (Aspects) will use `ClickHouse`_ as its analytic database. This
decision is rooted in the desire to provide high performance results on inexpensive hardware, while
maintaining the ability to scale up to large data sets and removing as much complexity, domain specific
knowledge, and pre-processing of the data as possible.
Expand All @@ -37,13 +37,13 @@ and advanced use cases, security, and high performance on commodity hardware. Cl
as well as:

- Being open-source (Apache 2 license)
- Being deployable and configurable to work with OARS using Tutor
- Being deployable and configurable to work with Aspects using Tutor
- Having row based access controls
- Able to store and query JSON documents, such as our xAPI statements
- Using a familiar SQL interface, no new or esoteric languages needed
- Requiring minimal new expertise for basic installation and configuration of a small instance
- Flexible configuration to scale up to very large datasets for large instances
- Compatible with other OARS technology (Superset, xAPI, Python)
- Compatible with other Aspects technology (Superset, xAPI, Python)
- Able to efficiently read and write from a single instance in most cases
- Offering a cloud service for providers who prefer that to running their own, or who need advanced
scaling
Expand All @@ -55,15 +55,15 @@ as well as:
Consequences
************

ClickHouse will be integrated into OARS via a Tutor plugin, either its own or as part of the LRS
plugin. It will be configured with a simple schema that should be sufficient for most installs,
ClickHouse will be integrated into Aspects via the Aspects Tutor plugin.
It will be configured with a simple schema that should be sufficient for most installs,
users for the LRS and Superset, and with networking and permissions configured appropriately
such that all of these pieces work together "out of the box".

A single ClickHouse instance should support a medium sized Open edX instance on commodity hardware,
including supporting our default reporting queries from Superset. More advanced deployments may be
required for large scale deployments, extremely intensive reports, or novel use cases outside the scope
of OARS v1.
of Aspects v1.

With proper configuration we expect there will be no need for schema management or additional data
pipelines for our default reports. Further ADRs will cover the specifics of these decisions.
Expand Down
18 changes: 9 additions & 9 deletions docs/decisions/0005_ralph.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@ Context
The xAPI specification adopted in :doc:`0002_xapi` uses a learning record store (LRS) as a
storage destination for xAPI statements. More information about LRSs can be found at `xapi.com`_.
There are many xAPI compatible LRSs, each with their own pluses and minuses, and supporting different
data storage backends. In discussions with the community while developing the OARS architecture
data storage backends. In discussions with the community while developing the Aspects architecture
no community preference for an LRS was forthcoming.

.. _xapi.com: https://xapi.com/learning-record-store/

Decision
********

The Open Analytics Reference System (OARS) will use `Ralph`_ as its LRS. This decision was made for
The Aspects Analytics system (Aspects) will use `Ralph`_ as its LRS. This decision was made for
several reasons:

- It is a toolkit already rooted in the Open edX ecosystem, and developed by a long-term Open edX partner (OpenFUN)
Expand All @@ -37,19 +37,19 @@ several reasons:
Consequences
************

Ralph will be integrated into OARS via a Tutor plugin. It will be configured to write xAPI statements to a
single table, and other OARS components will manage the downstream reporting schema separately.
Ralph will be integrated into Aspects via the Aspects Tutor plugin. It will be configured to write xAPI statements to a
single table, and other Aspects components will manage the downstream reporting schema separately.

A single Ralph instance should support a medium sized Open edX instance on commodity hardware,
including writing to ClickHouse while it is under normal reporting load for a small to medium Open edX install.
Large scale deployments, extremely intensive reports, or novel use cases are outside the scope of OARS v1.
Large scale deployments, extremely intensive reports, or novel use cases are outside the scope of Aspects v1.

Ralph does not currently support the entire xAPI LRS specification, only the `statements` endpoint, which is
all that we require for OARS v1. OpenFUN has been happy to receive pull requests and would welcome new contributions if further endpoints need
all that we require for Aspects v1. OpenFUN has been happy to receive pull requests and would welcome new contributions if further endpoints need
to be developed.

As Ralph's `statements` endpoint is xAPI compliant, other LRSs can be used for xAPI storage, however doing so
would likely mean losing the other OARS functionality (Superset reporting) if it does not support ClickHouse
would likely mean losing the other Aspects functionality (Superset reporting) if it does not support ClickHouse
for storage.


Expand All @@ -59,14 +59,14 @@ Rejected Alternatives
Off the shelf LRSs
------------------
No other xAPI LRS had the technology choices we were looking for, or had performance or licensing
restrictions that did not meet our desired levels for OARS. For instance, Learning Locker has an open
restrictions that did not meet our desired levels for Aspects. For instance, Learning Locker has an open
source version of their LRS, however it does not seem to be supported in any way. The xAPI reference
implementation is in Django, which is a familiar technology but is written as a transactional system
against MySQL and would not scale in the ways that we want. Other systems were cloud offerings or
otherwise licensed in ways that might restrict community adoption.

Writing our own
---------------
While the parts of the xAPI specification that we wish to use for OARS v1 are relatively simple, writing
While the parts of the xAPI specification that we wish to use for Aspects v1 are relatively simple, writing
and maintaining our own project was additional work that seemed to have little benefit over working with
an existing partner's existing project to mutual benefit.
22 changes: 11 additions & 11 deletions docs/decisions/0006_areas_of_responsibility.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,25 +9,25 @@ Accepted
Context
*******

The Open Analytics Reference System (OARS) consists of several pieces of technology working together via
The Aspects Analytics system (Aspects) consists of several pieces of technology working together via
some configuration and scripting. Decisions around where to store data and configuration need to be made
so that developers understand where to make changes and look for the causes of issues. These decisions
can have wide ranging impact on things such as extensibility and configurability of the system as a whole.

The number of plugins and systems in play can make it difficult to know where to look for configuration
or add new features. This ADR hopes to address this by offering the following guiding principles:

- Each service-based Tutor plugin should be able to be run the service separately from OARS, and in different
- Each service-based Tutor plugin should be able to be run the service separately from Aspects, and in different
configurations
- Opinionated configuration and non-LMS service associations belong in the tutor-contrib-oars plugin
- Opinionated configuration and non-LMS service associations belong in the tutor-contrib-aspects plugin


Decisions
*********

The service-based Tutor plugins (``tutor-contrib-clickhouse``, ``tutor-contrib-ralph``, ``tutor-contrib-superset``)
will each be able to be run outside the OARS context, such that if members of the community want to use
them in a modified way, or override OARS defaults, that will be possible. It is still assumed that the
will each be able to be run outside the Aspects context, such that if members of the community want to use
them in a modified way, or override Aspects defaults, that will be possible. It is still assumed that the
plugins will be used in an Open edX context and they will integrate themselves with the platform where
necessary.

Expand All @@ -53,10 +53,10 @@ Specifically:
(course staff, global staff, superuser, etc)
#. Create row-based filters for queries to enforce course-level permissions (Note:
since there are no datasets yet there is nothing to connect them to, but
they are created here as they would likely be useful for a non-OARS use case)
they are created here as they would likely be useful for a non-Aspects use case)
#. NOT create any ClickHouse databases, users, or tables

The process of tying these plugins together is managed by the ``tutor-contrib-oars`` plugin, which
The process of tying these plugins together is managed by the ``tutor-contrib-aspects`` plugin, which
does not provide a running service, but rather the opinionated configuration of the above services
and commands to manage them:

Expand All @@ -75,18 +75,18 @@ and commands to manage them:
#. Attach the row level course filter to the appropriate datasets
#. In LMS:
#. Add event-routing-backends as a Python requirement
#. Create an OARS user, necessary for the next step...
#. Create an Aspects user, necessary for the next step...
#. Configure event-routing-backends to use xAPI and send statements to Ralph
using the LMS user created in ``tutor-contrib-ralph``)

The OARS plugin will also be the conduit for running dbt via Tutor "do" commands. The
The Aspects plugin will also be the conduit for running dbt via Tutor "do" commands. The
dbt integration will be covered in a separate ADR.

Consequences
************

* The service-related plugins (ClickHouse, Ralph, Superset) can be run together or separately
with configurations different from the OARS defaults
with configurations different from the Aspects defaults
* A separate set of plugins could be created to replaces pieces of the stack with different
technologies (ex. replace Superset with Tableau, or ClickHouse with other supported Ralph
backend)
Expand All @@ -104,5 +104,5 @@ and created a different level of confusion about whether a ClickHouse user for R
in the ClickHouse plugin or Ralph's. It was similarly confusing for shared resources like the
ClickHouse schema which Ralph writes to part of, but the rest is managed elsewhere.

This would also leak OARS implementation details into all of the plugins, reducing their
This would also leak Aspects implementation details into all of the plugins, reducing their
flexibility for other use cases.