Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prototype Typesense search support #245

Open
ormsbee opened this issue Dec 17, 2024 · 2 comments
Open

Prototype Typesense search support #245

ormsbee opened this issue Dec 17, 2024 · 2 comments

Comments

@ormsbee
Copy link

ormsbee commented Dec 17, 2024

Acceptance Criteria

  • Prototype a search abstraction layer for content libraries.
  • Prototype a Typesense backend.
  • Refactor Meilisearch support to be a backend.
  • Create rough estimates and development stories for full support.
  • (Question: Should a Tutor plugin be part of this story?)

The scope of this abstraction layer would be to work on browser-oriented search engines like Meilisearch. This would not try to stretch to cover more traditional search engines like Elasticsearch, since doing so would be much more work and present performance concerns.

The biggest challenge is likely relate to tagging (quote from @bradenmacdonald):

Most current usages of Meilisearch are pretty straightforward search/filter that would be relatively easy to map onto TypeSense APIs or Algolia for that matter. But when it comes to filtering using hierarchical tags, which can themselves be filtered by a keyword, the implementation is quite complex and Meilisearch-specific - so that's part that would be trickiest to abstract and/or reimplement. https://github.com/openedx/frontend-app-authoring/blob/b110b6bdc9216759d509c660c025bfb8c6b973d8/src/search-manager/data/api.ts#L304-L506

Background

MIT has expressed concerns about Meilisearch's lack of failover/high-availability. While this feature is on the Meilisearch roadmap, it does not look like it will be prioritized in the near future.

At the same time, Algolia is an extremely popular commercial search engine that Meilisearch modeled its API on top of. While nobody has expressed interest in using Algolia yet, it is a strong long term possibility.

Testing

@blarghmatey, @pdpinch: Before this work kicks off, could you please verify that Typesense will be an acceptable backend? If there's early validation you need to do at the prototype step, I'd like to get a sense for that would look like on your side.

@ormsbee ormsbee converted this from a draft issue Dec 17, 2024
@ormsbee ormsbee changed the title Proof-of-concept TypeSense search backend implementation Prototype TypeSense search backend implementation Dec 17, 2024
@ormsbee ormsbee changed the title Prototype TypeSense search backend implementation Prototype Typesense search backend implementation Dec 17, 2024
@ormsbee ormsbee changed the title Prototype Typesense search backend implementation Prototype Typesense search support Dec 17, 2024
@ormsbee
Copy link
Author

ormsbee commented Dec 17, 2024

FYI: @jmakowski1123, @kdmccormick

@bradenmacdonald
Copy link
Contributor

I'd also like to know more about the root concern here: is it concern that learners and authors won't have access to search features (or e.g. the content libraries UI) during occasional downtime of a non-replicated search engine? And/or is it concern that writes (updates to the search index) will be lost during such downtime?

In other words, if we had a feature for queueing writes so they weren't ever lost, even if the search engine was occasionally unavailable, would that be "good enough"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In grooming
Development

No branches or pull requests

2 participants