Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Support multilangual search #94

Merged
merged 6 commits into from
Jul 20, 2023

Conversation

syphax-bouazzouni
Copy link

@syphax-bouazzouni syphax-bouazzouni commented Jul 14, 2023

Context

This PR update and implement the multi-lingual search. With it, we can now for these three properties of Classes/Concepts: prefLabel, synonym, and definition.

  1. Search in all languages.
  2. Search specifically in one language (e.g. fr, en, ...).
  3. Search specifically in values with no language.

This was made possible by doing two main things;

  1. Update our SOLR Schema to have dynamic fields: to save all the values in different languages. For example, a resource that has labels in French, English and not tagged, we will have in SOLR the fields prefLabel_fr for those in French, prefLabel_en for those in English, prefLabel_none for those with no languages, and lastly prefLabel for all the values in all languages mixed see 0c4ba8c).

    Those dynamic fields will let us do requests like this .search(prefLabel_fr:Plante) to search only the label in French. And .search(prefLabel:Plante) to search in all the languages.

  2. Requesting and indexing all the languages: The work done in Feature: Add request all languages unit test #93 let us fetch all the data in all the languages when we ask for it. Those requested data can then be indexed and saved in their corresponding language dynamic field. ( dynamic fields are created only if values exist in that language, creating only the fields needed) (7aaf942)

Changes

  • Add test for multilingual terms indexation and search (4f09fb6)
  • Update SOLR terms schema to add dynamic fields for values in all langs (0c4ba8c)
  • Make the indexation use map_attributes with include_languages enabled (7aaf942)
  • Override the Ressource to_hash method in Class model to have the option include_languages (9dd8db7)
  • Update the indexed document of a class model to include the dynamic fields for values in other languages (8280ded)

@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/support-multi-langual-search branch from 766108a to cba5930 Compare July 14, 2023 23:58
@syphax-bouazzouni
Copy link
Author

This PR to work in the server we need to update the server SOLR schema.xml file that is in this path /srv/ontoportal/data/solr/config/term_search

@syphax-bouazzouni syphax-bouazzouni merged commit 50619e6 into development Jul 20, 2023
syphax-bouazzouni added a commit that referenced this pull request Sep 5, 2023
…ngual-search

Feature: Support multilangual search
syphax-bouazzouni added a commit that referenced this pull request Sep 5, 2023
…ngual-search

Feature: Support multilangual search
syphax-bouazzouni added a commit that referenced this pull request Dec 5, 2023
#96)

* fix  json serializer nil exception (#79)

if hash was missing @context key the json serializer throwed a nil exception

* Merge pull request #87 from ontoportal-lirmm/fix/achived-submission-space-optimization

Fix: Archived submission space optimization

* Merge pull request #71 from ontoportal-lirmm/feature/support-multilingual-read-one-language-from-request-parameter

Feature: Support multilingual - Add read one language from request parameter tests

* Merge pull request #73 from ontoportal-lirmm/feature/add-request-language-to-context

Feature/add request language to context

* fix  json serializer nil exception (#79)

if hash was missing @context key the json serializer throwed a nil exception

* Merge pull request #83 from ontoportal-lirmm/feature/add-request-language-to-context

Feature: add request language to context follow up

* Merge pull request #93 from ontoportal-lirmm/support-multi-lang-part-02

Feature: Add request all languages unit test

* Merge pull request #94 from ontoportal-lirmm/feature/support-multi-langual-search

Feature: Support multilangual search

---------

Co-authored-by: Raimi Solorzano Niederhausen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant