Use a different way to store MCP reports instead of orderable child nodes

[Joscorbe]

We detected a case of the execution of MCP that generated a huge report of more than 30k rows. The report is generated using orderable child nodes, which are extremely inefficient when the amount of nodes is too big.

When a node is added or removed, the list is written entirely, updating the sequence of the orderable child nodes in a hidden property. This means, the bigger the list is, the more expensive the operation gets. When this happens, the previous list becomes garbage, which will be cleanup eventually by the Revision Garbage Collector.

Our proposal is to store the report in a different way, avoiding the creation of this content structure, for example a binary property, that would be a lot more efficient.

In our example, the revision garbage collection was failing with a timeout because with the amount of garbage it was unable to make any progress. We needed to scale up the to make it pass and clean up.

[davidjgonzalez]

@Joscorbe would you be interested in contributing? The original MCP author isn't active in this project anymore :(

[joerghoh]

The naive approach of replacing the nt:unstructured nodetype in the row node by oak:unstructured is not going to work, because otherwise the resulting report has a changed (random) order. As the nodes below rows (which contain the actual row data) are numbered, it would be possible to reconstruct the order from there.

But as suggested it's probably more efficient not to store the report in this format alltogether, but rather store the resulting file.

[joerghoh]

#2669 for the PR

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Joscorbe]

I'm closing this issue, since it seems resolved (and tagged wrongly as wontfix).