Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SKU not searched in the autocomplete separately #829

Closed
facundocapua opened this issue Mar 20, 2018 · 4 comments
Closed

SKU not searched in the autocomplete separately #829

facundocapua opened this issue Mar 20, 2018 · 4 comments

Comments

@facundocapua
Copy link

I have a requirement in a project that the SKU should match partially in the autocomplete, so my approach was the following.

I added an analyzer for the SKU field in this way:
elasticsuite_analysis.xml:

<?xml version="1.0"?>
<analysis xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:noNamespaceSchemaLocation="urn:magento:module:Smile_ElasticsuiteCore:etc/elasticsuite_analysis.xsd">

    <filters>
        <filter name="autocomplete_filter" type="ngram" language="default">
            <min_gram>3</min_gram>
            <max_gram>20</max_gram>
        </filter>
    </filters>

    <analyzers>
        <analyzer name="autocomplete" tokenizer="whitespace" language="default">
            <filters>
                <filter ref="lowercase" />
                <filter ref="ascii_folding" />
                <filter ref="trim" />
                <filter ref="elision" />
                <filter ref="word_delimiter" />
                <filter ref="autocomplete_filter" />
            </filters>
            <char_filters>
                <char_filter ref="html_strip" />
            </char_filters>
        </analyzer>
    </analyzers>
</analysis>

And elasticsuite_indices.xml:

<?xml version="1.0"?>
<indices xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="urn:magento:module:Smile_ElasticsuiteCore:etc/elasticsuite_indices.xsd">

    <index identifier="catalog_product" defaultSearchType="product">
        <type name="product" idFieldName="entity_id">
            <mapping>
                <field name="sku" type="string">
                    <isSearchable>1</isSearchable>
                    <isUsedForSortBy>1</isUsedForSortBy>
                    <isUsedInSpellcheck>1</isUsedInSpellcheck>
                    <defaultSearchAnalyzer>autocomplete</defaultSearchAnalyzer>
                </field>
            </mapping>
        </type>
    </index>
</indices>

The mapping was done as expected, however I'm not getting the SKU in the search query, this is an extract of the generated query:

"multi_match": {
                      "query": "2114630000",
                      "fields": [
                        "sku^16",
                        "spelling.whitespace^4",
                        "name.whitespace^16",
                        "at_modelo.whitespace^20",
                        "option_text_at_color.whitespace^12",
                        "option_text_at_capacidad.whitespace^8"
                      ],
                      "minimum_should_match": "100%",
                      "tie_breaker": 1,
                      "boost": 1,
                      "type": "best_fields",
                      "cutoff_frequency": 0.15,
                      "fuzziness": "AUTO",
                      "prefix_length": 1,
                      "max_expansions": 10
                    }

Doing some debugging I see that the code removed the SKU from the fields to search is this line:

$canAddField = $defaultField === null || $field->getSearchWeight() !== 1;

I have a few questions:

  • Is my approach correct for solving the business requirement?
  • Why are the fields with weight == 1 ignored in the query?

Thanks a lot and congratulations for such a great module!

@romainruaud
Copy link
Collaborator

Hello,

first of all, thank you for your congratulations and for using our module, you're welcome.

And from what I can see, you understood well how you are able to extend index properties via the XML file, which is fine.

A question here, did you find it on the developer documentation (on our wiki), or did you dig on it by yourself ?

As far as i can see, you are having the SKU in your search query, I see "sku^16" so the field is taken into account.

the fields with weight = 1 are "not" really ignored, they are removed from the query because all searchable fields are copied into the "search" meta-field. So there is no point adding them to the query with a weight of 1. Only fields with weight >1 are explicitely added to the query to ensure their matches scores are properly computed (with their weight multiplicator).

You will say that in your case, we are not querying the "search" field but the "spelling" one. This is due to the fact that prior your request, Elasticsuite engine did calculate that this query has to be a "spellchecked" one, due to the fact that no exact matching are found into the index for your query string.

The "spelling" field work the same way than the "search" one, except it contains only a copy of the fields that are "is_used_in_spellcheck=1".

But coming back to your issue, it seems to be a duplicate of #797 which has been fixed on the master branch. You can see the code in the PR #810

Maybe you could give a try to dev-master branch to see if it fixes your issue.

Do not hesitate if you have any other questions.

Best regards

@facundocapua
Copy link
Author

Thank @romainruaud for you quick and complete response!!

Answering your first question, it was a mix of reading the documentation and then checking the code. For me, in many cases it is easier to read the code rather than the documentation (even more when is a well written code as yours)

I was having some issues in my approach with this

private function loadSpellingType(RequestInterface $request)

The problem is that in case it is not an exact match, the type was SPELLING_TYPE_MOST_FUZZY and then the query builder was calling this:

$query = $this->getSpellcheckedQuery($containerConfig, $queryText, $spellingType, $boost);

And then the analyzer was always the whitespace instead of the autocomplete defined by me.

Now I'm testing adding the code of the PR you suggested via a patch.

Just for curiosity, do you have an estimated date for the release of version 2.6.0?

Thanks a lot for your help, it was really helpful!

@afoucret
Copy link
Contributor

Hi @facundocapua,

The partial sku search as implemented in #810 is intended to work on part of sku when changing the character type.

As an example if you have 124A-EE1, you will search for 124, A EE and 1.
In your case, you wish to seek for an arbitrary part in the SKU. It is not excpected and will be hard to implement using ElasticSearch.

You will have to write a plugin for the spellchecker to detect if a fuzzy search is not part of an actual sku.

We do not plan anything for this because we plan to rewrite completely the spellchecker in the future. I have some concerns about your autocomplete analyzer and think it can result in a weaker precision for the overall search.

@flexsingit
Copy link

we have installed and set up elasticsuite with Magento 2.3 and elastics search is working pretty good when we tested with some SKU. but there are some instances of the search in which it is not returning the result as expected. For example, when we search with the ISD-TN450 it is returning the correct results but when we search with the ISD-TN45 it is not returning any result.

We have checked the demo at http://demo.magento-elastic-suite.io/ and search with the keyword WT09 it is returning the exact result but when we search with the keyword WT0 it is not showing any result. but the product exists and showing on the website. Please suggest if we are missing something in our configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants