Multiple time "catalog_product index does not exist yet. Make sure everything is reindexed"

[menardavid]

Preconditions

Recently upgraded to M2.4.4p2 with ElasticSuite 2.10.10 and since, we see a lot of occurence of the mentioned error.

Magento Version : Magento EE 2.4.4p2 on Cloud

ElasticSuite Version : 2.10.10

Environment : Production

Third party modules :

• smile/module-elasticsuite-cms-search
• comwrap/module-elasticsuite-blog-search

[menardavid]

Here is a graphic in NewRelic. We see that is it flat mostly during the night and the weekend but automatically starts back on monday around 8 almost every week day.

[romainruaud]

Do you have a complete stack trace of the error ?

I'm curious to see what is the root of this call.

Such message means that you are not having an Elasticsearch index for products, which is probably not the case, otherwise the website himself should be completely non-functioning.

Regards

[menardavid]

@romainruaud - Here is the full stack trace. If you need more information, let us know.

[2022-11-10T14:08:04.681466+00:00] report.CRITICAL: LogicException: catalog_product index does not exist yet. Make sure everything is reindexed. in /app/hh47pligx5ajk/vendor/smile/elasticsuite/src/module-elasticsuite-core/Index/IndexOperation.php:118
184.145.176.250 - - [2022-11-10T14:08:04+00:00] "GET /fr/store/pointofsale/ajaxgetfavorite/?_=1668089280948 HTTP/1.1" 200 432 "https://www.saq.com/fr/nouveautes/nouveautes-cellier" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"
Stack trace:
#0 /app/hh47pligx5ajk/vendor/smile/elasticsuite/src/module-elasticsuite-core/Search/Request/ContainerConfiguration.php(193): Smile\ElasticsuiteCore\Index\IndexOperation->getIndexByName('catalog_product', 0)
#1 /app/hh47pligx5ajk/vendor/smile/elasticsuite/src/module-elasticsuite-core/Search/Request/ContainerConfiguration.php(94): Smile\ElasticsuiteCore\Search\Request\ContainerConfiguration->getIndex()
#2 /app/hh47pligx5ajk/vendor/smile/elasticsuite/src/module-elasticsuite-core/Search/Request/Builder.php(161): Smile\ElasticsuiteCore\Search\Request\ContainerConfiguration->getIndexName()
#3 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(58): Smile\ElasticsuiteCore\Search\Request\Builder->create(0, 'catalog_view_co...', 0, 0, NULL, Array, Array, Array, Array)
#4 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(138): Smile\ElasticsuiteCore\Search\Request\Builder\Interceptor->___callParent('create', Array)
#5 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(153): Smile\ElasticsuiteCore\Search\Request\Builder\Interceptor->Magento\Framework\Interception{closure}(0, 'catalog_view_co...', 0, 0, NULL, Array, Array, Array, Array)
#6 /app/hh47pligx5ajk/generated/code/Smile/ElasticsuiteCore/Search/Request/Builder/Interceptor.php(23): Smile\ElasticsuiteCore\Search\Request\Builder\Interceptor->___callPlugins('create', Array, Array)
#7 /app/hh47pligx5ajk/vendor/smile/elasticsuite/src/module-elasticsuite-catalog/Model/ResourceModel/Product/Fulltext/Collection.php(634): Smile\ElasticsuiteCore\Search\Request\Builder\Interceptor->create(0, 'catalog_view_co...', 0, 0, NULL, Array, Array, Array, Array)
#8 /app/hh47pligx5ajk/vendor/smile/elasticsuite/src/module-elasticsuite-catalog/Model/ResourceModel/Product/Fulltext/Collection.php(195): Smile\ElasticsuiteCatalog\Model\ResourceModel\Product\Fulltext\Collection->loadProductCounts()
#9 /app/hh47pligx5ajk/app/design/frontend/Saq/saq/Magento_CatalogWidget/templates/product/widget/content/grid.phtml(29): Smile\ElasticsuiteCatalog\Model\ResourceModel\Product\Fulltext\Collection->getSize()
#10 /app/hh47pligx5ajk/vendor/magento/framework/View/TemplateEngine/Php.php(71): include('/app/hh47pligx5...')
#11 /app/hh47pligx5ajk/vendor/magento/framework/View/Element/Template.php(263): Magento\Framework\View\TemplateEngine\Php->render(Object(Saq\Catalog\Preference\Catalog\Block\Product\ProductsList\Interceptor), '/app/hh47pligx5...', Array)
#12 /app/hh47pligx5ajk/vendor/magento/framework/View/Element/Template.php(293): Magento\Framework\View\Element\Template->fetchView('/app/hh47pligx5...')
#13 /app/hh47pligx5ajk/vendor/magento/framework/View/Element/AbstractBlock.php(1095): Magento\Framework\View\Element\Template->_toHtml()
#14 /app/hh47pligx5ajk/vendor/magento/framework/Cache/LockGuardedCacheLoader.php(136): Magento\Framework\View\Element\AbstractBlock->Magento\Framework\View\Element{closure}()
#15 /app/hh47pligx5ajk/vendor/magento/framework/View/Element/AbstractBlock.php(1117): Magento\Framework\Cache\LockGuardedCacheLoader->lockedLoadData('BLOCK_e2fab1184...', Object(Closure), Object(Closure), Object(Closure))
#16 /app/hh47pligx5ajk/vendor/magento/framework/View/Element/AbstractBlock.php(660): Magento\Framework\View\Element\AbstractBlock->_loadCache()
#17 /app/hh47pligx5ajk/vendor/magento/module-widget/Model/Template/Filter.php(156): Magento\Framework\View\Element\AbstractBlock->toHtml()
#18 /app/hh47pligx5ajk/vendor/magento/module-widget/Model/Template/Filter.php(167): Magento\Widget\Model\Template\Filter->generateWidget(Array)
#19 [internal function]: Magento\Widget\Model\Template\Filter->widgetDirective(Array)
#20 /app/hh47pligx5ajk/vendor/magento/framework/Filter/DirectiveProcessor/LegacyDirective.php(42): ReflectionMethod->invokeArgs(Object(Magento\Widget\Model\Template\Filter\Interceptor), Array)
#21 /app/hh47pligx5ajk/vendor/magento/framework/Filter/Template.php(275): Magento\Framework\Filter\DirectiveProcessor\LegacyDirective->process(Array, Object(Magento\Widget\Model\Template\Filter\Interceptor), Array)
#22 /app/hh47pligx5ajk/vendor/magento/framework/Filter/Template.php(209): Magento\Framework\Filter\Template->processDirectives('<style>#html-bo...')
#23 /app/hh47pligx5ajk/vendor/magento/module-email/Model/Template/Filter.php(1106): Magento\Framework\Filter\Template->filter('<style>#html-bo...')
#24 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Email\Model\Template\Filter->filter('<style>#html-bo...')
#25 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Widget\Model\Template\Filter\Interceptor->___callParent('filter', Array)
#26 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Widget\Model\Template\Filter\Interceptor->Magento\Framework\Interception{closure}('<style>#html-bo...')
#27 /app/hh47pligx5ajk/generated/code/Magento/Widget/Model/Template/Filter/Interceptor.php(32): Magento\Widget\Model\Template\Filter\Interceptor->___callPlugins('filter', Array, Array)
#28 /app/hh47pligx5ajk/vendor/comwrap/module-elasticsuite-blog-search/Model/Post/Indexer/Fulltext/Action/Full.php(78): Magento\Widget\Model\Template\Filter\Interceptor->filter('<style>#html-bo...')
#29 /app/hh47pligx5ajk/vendor/comwrap/module-elasticsuite-blog-search/Model/Post/Indexer/Fulltext/Action/Full.php(46): Comwrap\ElasticsuiteBlog\Model\Post\Indexer\Fulltext\Action\Full->processPostData(Array)
#30 /app/hh47pligx5ajk/vendor/magento/framework/Indexer/SaveHandler/Batch.php(20): Comwrap\ElasticsuiteBlog\Model\Post\Indexer\Fulltext\Action\Full->rebuildStoreIndex(1)
#31 /app/hh47pligx5ajk/vendor/smile/elasticsuite/src/module-elasticsuite-core/Indexer/GenericIndexerHandler.php(115): Magento\Framework\Indexer\SaveHandler\Batch->getItems(Object(Generator), 1000)
#32 /app/hh47pligx5ajk/vendor/comwrap/module-elasticsuite-blog-search/Model/Post/Indexer/Fulltext.php(91): Smile\ElasticsuiteCore\Indexer\GenericIndexerHandler->saveIndex(Array, Object(Generator))
#33 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(58): Comwrap\ElasticsuiteBlog\Model\Post\Indexer\Fulltext->executeFull()
#34 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(138): Comwrap\ElasticsuiteBlog\Model\Post\Indexer\Fulltext\Interceptor->___callParent('executeFull', Array)
#35 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(153): Comwrap\ElasticsuiteBlog\Model\Post\Indexer\Fulltext\Interceptor->Magento\Framework\Interception{closure}()
#36 /app/hh47pligx5ajk/generated/code/Comwrap/ElasticsuiteBlog/Model/Post/Indexer/Fulltext/Interceptor.php(23): Comwrap\ElasticsuiteBlog\Model\Post\Indexer\Fulltext\Interceptor->___callPlugins('executeFull', Array, Array)
#37 /app/hh47pligx5ajk/vendor/magento/module-indexer/Model/Indexer.php(443): Comwrap\ElasticsuiteBlog\Model\Post\Indexer\Fulltext\Interceptor->executeFull()
#38 /app/hh47pligx5ajk/vendor/magento/module-indexer/Model/Indexer/DependencyDecorator.php(268): Magento\Indexer\Model\Indexer->reindexAll()
#39 /app/hh47pligx5ajk/vendor/magento/module-indexer/Model/Processor.php(88): Magento\Indexer\Model\Indexer\DependencyDecorator->reindexAll()
#40 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Indexer\Model\Processor->reindexAllInvalid()
#41 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Indexer\Model\Processor\Interceptor->___callParent('reindexAllInval...', Array)
#42 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Indexer\Model\Processor\Interceptor->Magento\Framework\Interception{closure}()
#43 /app/hh47pligx5ajk/generated/code/Magento/Indexer/Model/Processor/Interceptor.php(23): Magento\Indexer\Model\Processor\Interceptor->___callPlugins('reindexAllInval...', Array, Array)
#44 /app/hh47pligx5ajk/vendor/magento/module-indexer/Cron/ReindexAllInvalid.php(31): Magento\Indexer\Model\Processor\Interceptor->reindexAllInvalid()
#45 [internal function]: Magento\Indexer\Cron\ReindexAllInvalid->execute(Object(Magento\Cron\Model\Schedule))
#46 /app/hh47pligx5ajk/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(368): call_user_func_array(Array, Array)
#47 /app/hh47pligx5ajk/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(879): Magento\Cron\Observer\ProcessCronQueueObserver->_runJob(1668089220, 1668089283, Array, Object(Magento\Cron\Model\Schedule), 'index')
#48 /app/hh47pligx5ajk/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(840): Magento\Cron\Observer\ProcessCronQueueObserver->tryRunJob(1668089220, 1668089283, Array, Object(Magento\Cron\Model\Schedule), 'index')
#49 /app/hh47pligx5ajk/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(280): Magento\Cron\Observer\ProcessCronQueueObserver->processPendingJobs('index', Array, 1668089283)
#50 /app/hh47pligx5ajk/vendor/magento/framework/Event/Invoker/InvokerDefault.php(88): Magento\Cron\Observer\ProcessCronQueueObserver->execute(Object(Magento\Framework\Event\Observer))
#51 /app/hh47pligx5ajk/vendor/magento/framework/Event/Invoker/InvokerDefault.php(74): Magento\Framework\Event\Invoker\InvokerDefault->_callObserverMethod(Object(Magento\Cron\Observer\ProcessCronQueueObserver), Object(Magento\Framework\Event\Observer))
#52 /app/hh47pligx5ajk/vendor/magento/module-staging/Model/Event/Manager.php(97): Magento\Framework\Event\Invoker\InvokerDefault->dispatch(Array, Object(Magento\Framework\Event\Observer))
#53 /app/hh47pligx5ajk/generated/code/Magento/Staging/Model/Event/Manager/Proxy.php(95): Magento\Staging\Model\Event\Manager->dispatch('default', Array)
#54 /app/hh47pligx5ajk/vendor/magento/framework/App/Cron.php(86): Magento\Staging\Model\Event\Manager\Proxy->dispatch('default')
#55 /app/hh47pligx5ajk/vendor/magento/module-cron/Console/Command/CronCommand.php(117): Magento\Framework\App\Cron->launch()
#56 /app/hh47pligx5ajk/vendor/symfony/console/Command/Command.php(255): Magento\Cron\Console\Command\CronCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#57 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(58): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#58 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Cron\Console\Command\CronCommand\Interceptor->___callParent('run', Array)
#59 /app/hh47pligx5ajk/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Cron\Console\Command\CronCommand\Interceptor->Magento\Framework\Interception{closure}(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#60 /app/hh47pligx5ajk/generated/code/Magento/Cron/Console/Command/CronCommand/Interceptor.php(23): Magento\Cron\Console\Command\CronCommand\Interceptor->___callPlugins('run', Array, Array)
#61 /app/hh47pligx5ajk/vendor/symfony/console/Application.php(1009): Magento\Cron\Console\Command\CronCommand\Interceptor->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#62 /app/hh47pligx5ajk/vendor/symfony/console/Application.php(273): Symfony\Component\Console\Application->doRunCommand(Object(Magento\Cron\Console\Command\CronCommand\Interceptor), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#63 /app/hh47pligx5ajk/vendor/magento/framework/Console/Cli.php(115): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#64 /app/hh47pligx5ajk/vendor/symfony/console/Application.php(149): Magento\Framework\Console\Cli->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#65 /app/hh47pligx5ajk/bin/magento(23): Symfony\Component\Console\Application->run()
#66 {main} [] []

[romainruaud]

OK I think I understand.

It's a cronjob or a command that is reindexing the comwrap-blog contents into Elasticsearch.

It seems that you have blog posts that are containing widgets.

When indexing the blog post, the Comwrap_ElasticsuiteBlog module tries to render the widget, here :

https://github.com/comwrap/Comwrap_ElasticsuiteBlog/blob/master/Model/Post/Indexer/Fulltext/Action/Full.php#L78

And to render it, it tries to emulate the frontend, so it ends up doing an Elasticsearch query.

This query fails, probably because it tries to instantiate a search request on the "admin" website, since it's done in a "cron / cli" context.

I'm not sure it's relevant to render widgets prior to indexing blogpost content.

This blog post is also probably missing in the search index for blog posts.

As this is done by a third party extension, you should probably try to contact ComWrap to check if they could fix this.

Regards

[menardavid]

Thank you for the trail, We will start there and keep you posted.

[romainruaud]

I close, since it's due to a third party module.