Error when trying to update a culture variant property to invariant

[jreurink]

Which Umbraco version are you using?

8.8.2

Bug summary

We have an Umbraco on-premise installation with a large amount (thousands) of culture variant nodes. We are trying to update some culture variant properties to invariant, but we get a server error during saving. The error is as follows:
System.ArgumentException: An item with the same key has already been added.

Stack trace:
stacktrace.txt

Specifics

We've found the issue after some digging around in the source of Umbraco.Core, in the 'RenormalizeDocumentEditedFlags' method of the ContentTypeRepositoryBase, in this statement:

var docCultureVariationsToUpdate = editedLanguageVersions.InGroupsOf(2000)
.SelectMany(_ => Database.Fetch<DocumentCultureVariationDto>(
Sql().Select<DocumentCultureVariationDto>().From<DocumentCultureVariationDto>()
.WhereIn<DocumentCultureVariationDto>(x => x.LanguageId, editedLanguageVersions.Keys.Select(x => x.langId).ToList())
.WhereIn<DocumentCultureVariationDto>(x => x.NodeId, editedLanguageVersions.Keys.Select(x => x.nodeId))))
//convert to dictionary with the same key type
.ToDictionary(x => (x.NodeId, (int?)x.LanguageId), x => x);

The collection of editedLanguageVersions contains more than 2000 items, which causes the subsequent Database.Fetch call to duplicate its results, which causes the .ToDictionary call to fail.

Steps to reproduce

The issue only occurs when there are more than 2000 items in the collection of editedLanguageVersions, which requires a lot of content nodes, at least a thousand for a site with two languages. An easy way to reproduce it is by altering the source of Umbraco.Core and copying the updated dll in your project. By changing the line
var docCultureVariationsToUpdate = editedLanguageVersions.InGroupsOf(2000) to var docCultureVariationsToUpdate = editedLanguageVersions.InGroupsOf(5), the bug will be triggered with a lot less content.

After that the steps to reproduce are:

• Create a culture-variant ('Allow vary by culture' = true) document type with a minimum of two available cultures.
• Create a culture-variant property on the document type.
• Create some content nodes of the document type and publish them. Make sure to fill the culture-variant property for all culture variants.
  • The amount of content needed depends on the value used in editedLanguageVersions.InGroupsOf(5), if it's 5, 3-4 nodes should be sufficient.
• Set 'Allow vary by culture' to false on the property, but NOT on the document type.
• Save the document type.
• Exception!
  

Expected result / actual result

We expect the document type to be updated correctly, but instead it throws an unhandled exception.

[emma-hq]

Hi @jreurink

Thanks for all the info here. This is going to be an interesting one to try and reproduce so thanks for the tip off as to how to do that easily. We'll take a look and let you know what we come up with.

Emma

[jreurink]

We've just run into a second issue in the same 'RenormalizeDocumentEditedFlags' method, but this time it's a SQL error: System.Data.SqlClient.SqlException (0x80131904): The incoming request has too many parameters. The server supports a maximum of 2100 parameters. Reduce the number of parameters and resend the request.

Apparently the SQL statement that's generated contains too many parameters for SQL Server to handle.
This is the stack trace of the error: sql_error_stacktrace.txt

@emma-hq Are there any updates on the issue on your end?

[emma-hq]

Hey @jreunik,

We've taken a look at what's happening here and yes, we are going to label it as up-for-grabs. You've really done the heavy lifting here, narrowing down the issue to exactly where it lives. Perhaps you'd be up for submitting a pr? Either way, I'll pop the label on and see if anyone is up for making a contribution to fix it.

Thanks again for your reporting and all the work you've done documenting the issue too,

Emma

[umbrabot]

Hi @jreurink,

We're writing to let you know that we would love some help with this issue. We feel that this issue is ideal to flag for a community member to work on it. Once flagged here, folk looking for issues to work on will know to look at yours. Of course, please feel free work on this yourself ;-). If there are any changes to this status, we'll be sure to let you know.

For more information about issues and states, have a look at this blog post

Thanks muchly, from your friendly Umbraco GitHub bot :-)

[umbrabot]

Hiya @jreurink,

Just wanted to let you know that we noticed that this issue got a bit stale and might not be relevant any more.

We will close this issue for now but we're happy to open it up again if you think it's still relevant (for example: it's a feature request that's not yet implemented, or it's a bug that's not yet been fixed).

To open it this issue up again, you can write @umbrabot still relevant in a new comment as the first line. It would be super helpful for us if on the next line you could let us know why you think it's still relevant.

For example:

@umbrabot still relevant
This bug can still be reproduced in version x.y.z

This will reopen the issue in the next few hours.

Thanks, from your friendly Umbraco GitHub bot 🤖 🙂