-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add Meilisearch-compatible search engine #162
Conversation
Unit test command was failing because manage.py was pointing to an incorrect module by default.
Thanks for the pull request, @regisb! What's next?Please work through the following steps to get your changes ready for engineering review: 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. 🔘 Let us know that your PR is ready for review:Who will review my changes?This repository is currently maintained by Where can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:
When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
This command makes it possible to compile requirements without upgrading them. Note that the `make upgrade` command still works with this change.
The goal of this change is to introduce a search engine that is compatible with the edx-search API but that uses Meilisearch instead of Elasticsearch. That way, we can replace one by the other across edx-platform by simply changing a single SEARCH_ENGINE django setting. There are a couple of differences between Meilisearch and Elasticsearch: 1. Filterable attributes must be defined explicitly. 2. No support for datetime objects, which must be converted to timestamps (with an extra field to store the timezone). 3. No special characters allowed in the primary key values, such that we must hash course IDs before we can use them as primary key values. Note that this PR does not introduce any breaking change. This is an opt-in engine that anyone is free to use. There is some setup work for every search feature: see the engine module documentation for more information. See the corresponding conversation here: openedx/frontend-app-authoring#1334 (comment)
0be358e
to
fa0fc71
Compare
I will add more unit tests to improve the test coverage later. In the meantime, this PR is ready for review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Works a treat @regisb . Thank you for building this!
- I tested this on my tutor dev stack:
- ran
make compile-requirements
to test the requirements & Makefile change - installed requirements in virtualenv and ran
./manage.py migrate
to test change tomanage.py
- enabled courseware search and this new engine during launch, according to the instructions in search/meilisearch.py
- created a new course in Studio with content
- published the course
- searched the course from the learning MFE and saw my published content.
- ran
- I read through the code
-
I checked for accessibility issuesN/A - Includes documentation
- User-facing strings are extracted for translation
@itsjeyd @mphilbrick211 Backstage says this repo is unmaintained, but we'd really like to get this change into Sumac. CC @feanil since you touched it last 😄 |
Jill, you are faster than light. |
@pomegranited I think you did the right thing by pinging @feanil :) When a repo is unmaintained we usually need to get help from someone that's part of Axim's technical team to move forward. |
Know why? Because somebody built this amazing development and production build system that allows me test changes reliably and easily 😄 |
@@ -7,7 +7,7 @@ | |||
import sys | |||
|
|||
if __name__ == "__main__": | |||
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "edxsearch.settings") | |||
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[question]: Why was this done? Why do we have these separate settings modules at all?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was to fix an existing issue with that manage.py file -- it was referencing a non-existent file. AFAIK, you'd only use this to run a dev server locally, not for anything real.
Closing this in favor of #164, which adds more unit tests and some settings file related cleanup. |
The goal of this change is to introduce a search engine that is
compatible with the edx-search API but that uses Meilisearch instead of
Elasticsearch. That way, we can replace one by the other across
edx-platform by simply changing a single SEARCH_ENGINE django setting.
There are a couple of differences between Meilisearch and Elasticsearch:
timestamps (with an extra field to store the timezone).
must hash course IDs before we can use them as primary key values.
Note that this PR does not introduce any breaking change. This is an
opt-in engine that anyone is free to use. There is some setup work for
every search feature: see the engine module documentation for more
information.
See the corresponding conversation here:
openedx/frontend-app-authoring#1334 (comment)
This PR replaces the corresponding one in edx-platform: openedx/edx-platform#35650