URL Rewrite issues occur very often /catalog/product/view/id/711/s/product-name/category/16/

[kayintveen]

Summary (*)

For a long time we are struggling with URL Rewrites. When adding products and categories very often we get url structures like these. /catalog/product/view/id/711/s/product-name/category/16/

We had it with 2.0.0 and we even did a clean install of 2.1 and re-added (via import csv) the 360 products. We added the products to the categories by hand.

We noticed there is a solution by adding the category or product url by hand. but of course this is not something we want to do for 300 products and somewhere aroudn 25 categories.

We added a french and german store as well and those are 100% bad.
The main store now is like 70% okay, but still 30% is not correct.

Examples (*)

How is this possible.

What we did to try to solve it is

• Search for duplicate urls in the database (none found)
• Search for urls not added to the main store (we had this before, but with our re-import into the main store this is solved)

Edit, i have found some more information.
We have a site with multiple storeviews, i notice that our first store has problems with the url. when i search the url key in the url_rewrite database i do find the url for all store's (2,3,4,5,6,7,8,9,10) excpect for store. 1.

Should't there be a store 0 url key ??

Steps to reproduce
-> We first created 9 different storeviews (one store, one website)
-> We imported around 300 records (simple products) with specified url keys (but also tried without) via the import product functionality in the back-end
-> After that we notice a lot of inconsistencies with the url keys.

Proposed solution

[andimov]

@kayintveen
Thank you for reporting!
Please, format this issue according to the Issue reporting guidelines: with steps to reproduce, actual result and expected result.

[kayintveen]

@andimov Thank you, i just entered the steps to reproduce.

[valibus]

Same problem here on 2.1.
We had clean url for product and after importing a csv with those 3 colums:
|SKU|storeview|category|

we got bad formated url like kayintveen told you.

Did you plan to fix it, it's realy realy bad for seo.

[jtw372]

Same problem here on 2.1.1.

[ps202]

confirmed.

[veloraven]

@kayintveen it seems that you describe to issues in this report:

• with urls format
• with missed url rewrites for store 1
  Please create separate GitHub issue for the second case.

[twistedindustries]

@veloraven Is there any fix for 2.0.X yet?

[kayintveen]

We still have this huge issue and not one update on this.
We solved it now by adding url rewrites by hand on the url_rewrite table. but everytime we add a store or something these custom rows all get dropped. making it a huge issue for us now!

Also added a Stack Overflow question of the last time we had it. http://magento.stackexchange.com/questions/158552/magento-2-url-rewrites-get-removed-when-a-indexation-is-triggered-or-something

[farhadfrz]

We have the same issue, 20000 products in two store views, but some url rewrites are being deleted for store 1. We have tried to regenerate url rewrites manually and from admin but after few hours they are being deleted!! Any solution??

[hostep]

Possibly related issue: #8786

[thlassche]

I developed a workaround for this, a CLI command that can be used in cron, that regenerates all rewrites. Send me a message if you need it.

[thisfiore]

@thlassche can you send me a solution the workaround ?

[IlnitskiyArtem]

Hi, @kayintveen. Internal ticket MAGETWO-64295, is in our issue backlog.

[royvanos]

@thlassche can you send me the workaround? Thanks!

[orlangur]

Seems to be fixed in 2.1.8 (1222362).

[kunzi]

Do we need to re-import products for the problem to be fixed? Suddenly we have this problem after updating to 2.1.7 and then saving category. We lost all url rewrites for saved category and categories under that + for all products. Upgrading 2.1.8 did nothing and re-saving categories does not fix this issue.

[caramidaovidiu]

You can use this solution and you won't have any more problems with the URL rewrites.
https://webpanda-solutions.com/url-rewrites-regenerate-and-customize.html

[simonworkhouse]

Or alternatively, you could use these solutions:

• https://www.prestashop.com/
• https://sylius.com/
• Anything that's not Magento
• etc...

[mille2018]

I have used web panda extension it doesn't work for large product only good for 5 to 10 products. Magento should mix this problem asap or lose customers

[caramidaovidiu]

It can process all products at once with the cli command

…

On Fri, 12 Jul 2019 at 11:24, mille2018 ***@***.***> wrote: I have used web panda extension it doesn't work for large product only good for 5 to 10 products. Magento should mix this problem asap or lose customers — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5863?email_source=notifications&email_token=ALZ4JLMUH2BLKHNUN6GEYOLP7A5TZA5CNFSM4CK4USMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZZCRTA#issuecomment-510798028>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALZ4JLILKJAYYMCGCF45DU3P7A5TZANCNFSM4CK4USMA> .

[ghost]

@caramidaovidiu

Normally i would say Earning money by a bug in a opensource platform should be avoided at all costs.
In case of magento, you almost need to use this because Magento team itself seems to be unable to resolve the problems.

@simonworkhouse

1. A little bit cynical but if i test things and i dont see a problem it is not there.
   "If you have never had rewrite issues, you either haven't been paying attention or you haven't been using Magento."

2. I am using magento for 5 years now

3. Yes i agree Magento is a mess: "The only recommendation I can give to anyone is to avoid Magento at all costs, there is no need to subject oneself to the buggy, poorly architected mess that it is."

4."The rewrite system in Magento needs to be entirely rebuilt and this time they need some experienced developers giving the whole system careful consideration.:

Yes, They have to call some (real) experts and pay one of them a fortune to resolve these opensource issues. Everything is solved by then.

Without the money no one is considering helping a community anymore.

Who is willing to try and help a community out please feel free to analyze and dissect magento 2
The whole community would be thankful if someone repairs all the issues like URL rewrites and other well known problems in the latest final version. Also take a look on mobile devices because there are issues too.

If I had such a skill set i already did such an operation in my free time.
But unfortunately I am not skilled enough and i have not the time to learn.

I certainly have the feeling Adobe is going to pull the plug someday in the near future from the Opensource edition. Cost effective it is a pain in there ass. If they wanted it to be a success they had already intervened and did the necessary steps to get things done fast to Clean the name of Magento opensource and make it a success. Nothing points out this will ever happen.
So for now we have to deal with what we have because changing platform is expansive an too much time consuming (for me).

[caramidaovidiu]

There are free extensions online that do the basic url regeneration. The webpanda one does more than that and the support is also very important. You won't get that with a free one and most of the people who have this kind of problem will need to support even with the extension working.

[WinstonN]

@magento-engcom-team why has this issue been closed?

[angelflo]

same question :
@magento-engcom-team why has this issue been closed? !!!

[chrisbadley]

This still occurs on Magento 2.3.2, I can actually sit there fetching the records from the url_rewrite table with MySQL Workbench and watch entries in the table vanish before my eyes, on a staging environment, with no transaction, imports or visits occurring at the time. Nothing is in the log files.

This could result in masses of products returning 404 errors from search results and decimate a websites SEO. 3 years of URL rewrite issues and Magento have still not got to grips with these type of occurrences.

[simonmaass]

@chrisbadley can also confirm this in 2.3.2 aswell!

[chrisbadley]

@chrisbadley can also confirm this in 2.3.2 aswell!

Hi Simon, let me know if you have any extension that add things to categories automatically. I may have the solution for you.

[simonmaass]

@chrisbadley hey Chris :) Would love to know what you did to solve the problem!

[chrisbadley]

@chrisbadley hey Chris :) Would love to know what you did to solve the problem!

Simon, List the extensions you have installed and I can tell you which one could potentially be causing the problem.

[angelflo]

Hi @magento-engcom-team ,

Is it possible to have a REAL explanation about this bug ?
1/ Why do you stay this ticket marked as closed whereas bug is still here ?
2/ Are you currently working to solve this multisite problem ?

Thanks a lot to take in consideration

[ihor-sviziev]

Hi @angelflo,

Unfortunately we still don't have exact steps to reproduce of this issue. Unfortunately this issue reproducing randomly and it's really hard to identify it.

If you have clear steps to reproduce - please report separate issue for that.

Thank you!

[simonmaass]

@chrisbadley these are our 3rd party extension in the composer.json:

"splendidinternet/mage2-locale-de-de": "^1.54",
"magepal/magento2-gmailsmtpapp": "^2.6.4",
"algolia/algoliasearch-magento-2": "^1.11.3",
"mirasvit/module-reports": "^1.3.29",
"elgentos/regenerate-catalog-urls": "^0.2.0",
"fooman/pdfcustomiser-m2": "^8.2.2",
"firebear/importexport": "^3.1.7",
"stripeofficial/stripe-payments": "^2.0.10",
"magepal/magento2-googletagmanager": "dev-master",
"semaio/magento2-configimportexport": "dev-develop",
"staempfli/magento2-module-seo": "^1.7",
"dhl/module-shipping-m2": "^0.10.1",
"mageplaza/module-delete-orders": "^1.2",
"trustedshops/trustedshops": "^1.0.4",
"xtento/productexport": "^2.10",
"xtento/customordernumber": "^2.1",

[chrisbadley]

@chrisbadley these are our 3rd party extension in the composer.json:

"splendidinternet/mage2-locale-de-de": "^1.54",
"magepal/magento2-gmailsmtpapp": "^2.6.4",
"algolia/algoliasearch-magento-2": "^1.11.3",
"mirasvit/module-reports": "^1.3.29",
"elgentos/regenerate-catalog-urls": "^0.2.0",
"fooman/pdfcustomiser-m2": "^8.2.2",
"firebear/importexport": "^3.1.7",
"stripeofficial/stripe-payments": "^2.0.10",
"magepal/magento2-googletagmanager": "dev-master",
"semaio/magento2-configimportexport": "dev-develop",
"staempfli/magento2-module-seo": "^1.7",
"dhl/module-shipping-m2": "^0.10.1",
"mageplaza/module-delete-orders": "^1.2",
"trustedshops/trustedshops": "^1.0.4",
"xtento/productexport": "^2.10",
"xtento/customordernumber": "^2.1",

Not really what I was hoping to see Simon, having said that, I would certainly start looking into the code for firebearstudio Import & Export, especially if you are running this via a cron.

My problem was an extension that automatically assigned products to a category based on attributes. I found this in a helper class:

 foreach ($collection as $product) {
        foreach ($product->getStoreIds() as $storeId) {
            $this->urlPersist->deleteByData([
                \Magento\UrlRewrite\Service\V1\Data\UrlRewrite::ENTITY_ID => $product->getId(),
                \Magento\UrlRewrite\Service\V1\Data\UrlRewrite::ENTITY_TYPE => \Magento\CatalogUrlRewrite\Model\ProductUrlRewriteGenerator::ENTITY_TYPE,
                \Magento\UrlRewrite\Service\V1\Data\UrlRewrite::REDIRECT_TYPE => 0,
                \Magento\UrlRewrite\Service\V1\Data\UrlRewrite::STORE_ID => $storeId
            ]);
            try {
                $this->urlPersist->replace($this->productUrlRewriteGenerator->generate($product));
            } catch (\Exception $e) {

            }
        }
    }

Once I commented this out, the problem vanished.

I hope you find the resolution to your problem, good luck with your project, sorry I cannot be of more help.

[simonmaass]

@chrisbadley thanks so much for your insight!

I'll keep looking and update here if i find something!

[Eddcapone]

Same Problem on 2.4.1

[TheObiOne]

Any updates on this?

We're also experiencing this on 2.4.1-p1

Very inconsistent and can't pick up a trend.

[piuga]

This seems related to #23275 (comment)
We also encountered the broken URL when a product was updated from "Not visible individually" to "Catalog, Search" visibility. Once the product is marked as not visible, the entries for this product from url_rewrite table are removed.
Related also to: #19293