Error on Magento 2 Reindex with Opensearch 2.4.1-1

[thunder809]

Magento with ElasticSuite fails rebuilding the index with newest opensearch installation.

Error is in "vendor/smile/elasticsuite/src/module-elasticsuite-core/Index/Bulk/BulkRequest.php"

This is my temporary fix, to get indexer to work again:
removing '_type' => '_doc' from every occasion:

- $this->bulkData[] = ['index' => ['_index' => $index->getName(), '_type' => '_doc', '_id' => $docId]];
+ $this->bulkData[] = ['index' => ['_index' => $index->getName(),  '_id' => $docId]];

Preconditions

Magento Version : CE 2.4.5-p1

ElasticSuite Version : 2.10.12.1

Environment : Developer / Default

Third party modules :

Steps to reproduce

1. use opensearch as search engine
2. run php bin/magento indexer:reindex in console

Expected result

1. Reindex should work without any problems

Actual result

An exception is thrown:

Catalog Search index process error during indexation process:
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400}
Amasty Customer Group Catalog Rule index has been rebuilt successfully in 00:00:00
Amasty Customer Group Catalog Rule index has been rebuilt successfully in 00:00:00
Amasty Customer Group Catalog Rule index has been rebuilt successfully in 00:00:00
ElasticSuite Category Indexing index process error during indexation process:
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400}
ElasticSuite Thesaurus Indexing index has been rebuilt successfully in 00:00:00
PHP Fatal error: Uncaught Elasticsearch\Common\Exceptions\BadRequest400Exception: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400} in /srv/http/magento2_dev.localhost/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:693
Stack trace:
#0 /srv/http/magento2_dev.localhost/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(333): Elasticsearch\Connections\Connection->process4xxError(Array, Array, Array)
#1 /srv/http/magento2_dev.localhost/vendor/react/promise/src/FulfilledPromise.php(28): Elasticsearch\Connections\Connection->Elasticsearch\Connections{closure}(Array)
#2 /srv/http/magento2_dev.localhost/vendor/react/promise/src/Promise.php(134): React\Promise\FulfilledPromise->then(Object(Closure), NULL)
#3 /srv/http/magento2_dev.localhost/vendor/react/promise/src/Promise.php(168): React\Promise\Promise::React\Promise{closure}(Object(React\Promise\FulfilledPromise))
#4 /srv/http/magento2_dev.localhost/vendor/react/promise/src/Promise.php(231): React\Promise\Promise->settle(Object(React\Promise\FulfilledPromise))
#5 /srv/http/magento2_dev.localhost/vendor/react/promise/src/Deferred.php(36): React\Promise\Promise::React\Promise{closure}(Array)
#6 /srv/http/magento2_dev.localhost/vendor/ezimuel/ringphp/src/Client/CurlMultiHandler.php(246): React\Promise\Deferred->resolve(Array)
#7 /srv/http/magento2_dev.localhost/vendor/ezimuel/ringphp/src/Client/CurlMultiHandler.php(137): GuzzleHttp\Ring\Client\CurlMultiHandler->processMessages()
#8 /srv/http/magento2_dev.localhost/vendor/ezimuel/ringphp/src/Future/BaseFutureTrait.php(127): GuzzleHttp\Ring\Client\CurlMultiHandler->execute()
#9 /srv/http/magento2_dev.localhost/vendor/ezimuel/ringphp/src/Future/BaseFutureTrait.php(58): GuzzleHttp\Ring\Future\FutureArray->invokeWait()
#10 /srv/http/magento2_dev.localhost/vendor/ezimuel/ringphp/src/Future/BaseFutureTrait.php(127): GuzzleHttp\Ring\Future\FutureArray->wait()
#11 /srv/http/magento2_dev.localhost/vendor/ezimuel/ringphp/src/Future/BaseFutureTrait.php(58): GuzzleHttp\Ring\Future\FutureArray->invokeWait()
#12 /srv/http/magento2_dev.localhost/vendor/ezimuel/ringphp/src/Future/MagicFutureTrait.php(30): GuzzleHttp\Ring\Future\FutureArray->wait()
#13 /srv/http/magento2_dev.localhost/vendor/ezimuel/ringphp/src/Future/FutureArray.php(27): GuzzleHttp\Ring\Future\FutureArray->__get('_value')
#14 /srv/http/magento2_dev.localhost/vendor/smile/elasticsuite/src/module-elasticsuite-core/Index/AsyncIndexOperation.php(113): GuzzleHttp\Ring\Future\FutureArray->offsetGet('items')
#15 /srv/http/magento2_dev.localhost/vendor/smile/elasticsuite/src/module-elasticsuite-core/Index/AsyncIndexOperation.php(130): Smile\ElasticsuiteCore\Index\AsyncIndexOperation->resolveFutureBulks()
#16 /srv/http/magento2_dev.localhost/vendor/smile/elasticsuite/src/module-elasticsuite-core/Indexer/GenericIndexerHandler.php(128): Smile\ElasticsuiteCore\Index\AsyncIndexOperation->refreshIndex(Object(Smile\ElasticsuiteCore\Index\Index))
#17 /srv/http/magento2_dev.localhost/vendor/magento/module-catalog-search/Model/Indexer/Fulltext.php(176): Smile\ElasticsuiteCore\Indexer\GenericIndexerHandler->saveIndex(Array, Object(Generator))
#18 /srv/http/magento2_dev.localhost/generated/code/Magento/CatalogSearch/Model/Indexer/Fulltext/Interceptor.php(32): Magento\CatalogSearch\Model\Indexer\Fulltext->executeByDimensions(Array, NULL)
#19 /srv/http/magento2_dev.localhost/vendor/magento/module-catalog-search/Model/Indexer/Fulltext.php(236): Magento\CatalogSearch\Model\Indexer\Fulltext\Interceptor->executeByDimensions(Array)
#20 [internal function]: Magento\CatalogSearch\Model\Indexer\Fulltext->Magento\CatalogSearch\Model\Indexer{closure}()
#21 /srv/http/magento2_dev.localhost/vendor/magento/module-indexer/Model/ProcessManager.php(88): call_user_func(Object(Closure))
#22 /srv/http/magento2_dev.localhost/vendor/magento/module-indexer/Model/ProcessManager.php(75): Magento\Indexer\Model\ProcessManager->simpleThreadExecute(Array)
#23 /srv/http/magento2_dev.localhost/vendor/magento/module-catalog-search/Model/Indexer/Fulltext.php(239): Magento\Indexer\Model\ProcessManager->execute(Array)
#24 /srv/http/magento2_dev.localhost/vendor/magento/framework/Interception/Interceptor.php(58): Magento\CatalogSearch\Model\Indexer\Fulltext->executeFull()
#25 /srv/http/magento2_dev.localhost/vendor/magento/framework/Interception/Interceptor.php(138): Magento\CatalogSearch\Model\Indexer\Fulltext\Interceptor->___callParent('executeFull', Array)
#26 /srv/http/magento2_dev.localhost/vendor/magento/framework/Interception/Interceptor.php(153): Magento\CatalogSearch\Model\Indexer\Fulltext\Interceptor->Magento\Framework\Interception{closure}()
#27 /srv/http/magento2_dev.localhost/generated/code/Magento/CatalogSearch/Model/Indexer/Fulltext/Interceptor.php(41): Magento\CatalogSearch\Model\Indexer\Fulltext\Interceptor->___callPlugins('executeFull', Array, Array)
#28 /srv/http/magento2_dev.localhost/vendor/magento/module-indexer/Model/Indexer.php(443): Magento\CatalogSearch\Model\Indexer\Fulltext\Interceptor->executeFull()
#29 /srv/http/magento2_dev.localhost/vendor/magento/module-indexer/Model/Indexer/DependencyDecorator.php(268): Magento\Indexer\Model\Indexer->reindexAll()
#30 /srv/http/magento2_dev.localhost/vendor/magento/module-indexer/Console/Command/IndexerReindexCommand.php(110): Magento\Indexer\Model\Indexer\DependencyDecorator->reindexAll()
#31 /srv/http/magento2_dev.localhost/vendor/symfony/console/Command/Command.php(255): Magento\Indexer\Console\Command\IndexerReindexCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#32 /srv/http/magento2_dev.localhost/vendor/magento/framework/Interception/Interceptor.php(58): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#33 /srv/http/magento2_dev.localhost/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Indexer\Console\Command\IndexerReindexCommand\Interceptor->___callParent('run', Array)
#34 /srv/http/magento2_dev.localhost/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Indexer\Console\Command\IndexerReindexCommand\Interceptor->Magento\Framework\Interception{closure}(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#35 /srv/http/magento2_dev.localhost/generated/code/Magento/Indexer/Console/Command/IndexerReindexCommand/Interceptor.php(86): Magento\Indexer\Console\Command\IndexerReindexCommand\Interceptor->___callPlugins('run', Array, Array)
#36 /srv/http/magento2_dev.localhost/vendor/symfony/console/Application.php(1021): Magento\Indexer\Console\Command\IndexerReindexCommand\Interceptor->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#37 /srv/http/magento2_dev.localhost/vendor/symfony/console/Application.php(275): Symfony\Component\Console\Application->doRunCommand(Object(Magento\Indexer\Console\Command\IndexerReindexCommand\Interceptor), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#38 /srv/http/magento2_dev.localhost/vendor/magento/framework/Console/Cli.php(116): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#39 /srv/http/magento2_dev.localhost/vendor/symfony/console/Application.php(149): Magento\Framework\Console\Cli->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#40 /srv/http/magento2_dev.localhost/bin/magento(23): Symfony\Component\Console\Application->run()
#41 {main}
thrown in /srv/http/magento2_dev.localhost/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php on line 693

[vahonc]

Hello @thunder809,

According to the system requirements of Magento 2.4.5/2.4.5-p1, it supports OpenSearch 1.2 version. Therefore I'm not sure about the compatibility of M2 with the OpenSearch 2.4.1-1 version.

At least we don't have such information and have not tested ElasticSuite compatibility with this version.

Could you try to test this issue on the environment with OpenSearch 1.2 version?

BR,
Vadym

[github-actions]

This issue was waiting update from the author for too long. Without any update, we are unfortunately not sure how to resolve this issue. We are therefore reluctantly going to close this bug for now. Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away! Thanks for your contribution.

[speller]

Magento 2.4.6 is now compatible with OpenSearch 2.x. Is it a good time to add support for OS 2.x versions to ElasticSuite as well?

[romainruaud]

it has been done in the latest 2.11.x version.

Regards

[speller]

Yeah, thanks.
But version 2.11 doesn't support Magento 2.4.5. I can't install it: smile/elasticsuite 2.11.0 requires magento/framework >=103.0.6 -> found magento/framework[103.0.6] but the package is fixed to 103.0.5

[vahonc]

@speller,

Does Magento 2.4.5 support the OpenSearch 2.x?

It looks like nope, according to this system-requirements.

BR,
Vadym

[speller]

@vahonc Is ElasticSuite requiring the core OpenSearch/ElasticSearch modules? I can't see any in the composer.json file. As I understand, ElasticSuite replaces the core modules. In this case, does it make sense to refer to the core ES/OS compatibility?

[romainruaud]

Hi @speller , Elasticsuite does not depend on Magento "core" implementation.

But we are tied to the version of the connector they use : https://github.com/magento/magento2/blob/2.4-develop/composer.json#L44

If you want to use Opensearch2 / Elasticsearch 8, you'll have to use the client library with version 8.x.

Magento 2.4.6 allow this, but prior versions were requiring the Elasticsearch client 6 or 7.

So we would not have been able to add ES 8 / OS 2 support to those old versions, this would have require some composer "hacks" which is not something we were willing to do.

Regards

[speller]

@romainruaud got it, thank you for the explanation!