Demonstrate saved object import stalling and Kibana server becoming unresponsive #118916
Conversation
nreese
changed the title
💔 Build FailedFailed CI Steps
Test Failures
Metrics [docs]
HistoryTo update your PR or re-run it, just comment with: |
|
Seems like migrations go into an endless loop because the document migrator never sets a For the visualization and index-pattern we set But for the dashboard it stays undefined so and we never break out of the while loop: I've always felt like a I think what happens is if any object has a migrationVersion higher than the reference migration version (8.0.0) then we always break here so we never get to |
|
@rudolf Thanks for investigating. What are the next steps to resolve this problem? |
The thing is, the document in the import file does specify When collecting the objects in *** importSavedObjectsFromStream [
{
migrationVersion: { dashboard: '8.1.0' },
attributes: { /* */ },
coreMigrationVersion: '8.1.0', // <---
id: 'my-dashboard',
references: [ [Object] ],
type: 'dashboard',
updated_at: '2019-01-22T19:32:47.232Z',
version: 'WzEzLDJd'
}
]But when reaching I tracked down where we're loosing this value, and it's inside the call of kibana/src/core/server/saved_objects/service/lib/repository.ts Lines 520 to 535 in 349434e (Note: I think this needs to be fixed, but I'm not sure 100% this will be sufficient, as when importing documents from older versions, cc @jportner we'll need your insight here given you're the one who implemented this feature. |
|
I believe I found the problem. Inside the document migrator, it takes all applicable types of transforms ("reference", "convert", and "migrate") and loops over them, applying each one to the saved object. The "reference" transforms are applied based on the object's Anyway, we technically support consumer-defined migrations that increase the object's Edit: the comment chain on that PR is too long, so this link doesn't actually show you the discussion -- here's a screenshot: That logic incorrectly broke out of the loop for "reference" transforms. It should only break out of the loop for the other two types of transforms. This meant that in a certain situation where a saved object's I tested this solution locally and it seems to do the trick: diff --git a/src/core/server/saved_objects/migrations/core/document_migrator.ts b/src/core/server/saved_objects/migrations/core/document_migrator.ts
index da16dbc5e69..b405497eeed 100644
--- a/src/core/server/saved_objects/migrations/core/document_migrator.ts
+++ b/src/core/server/saved_objects/migrations/core/document_migrator.ts
@@ -754,7 +754,7 @@ function migrateProp(
for (const { version, transform, transformType } of applicableTransforms(migrations, doc, prop)) {
const currentVersion = propVersion(doc, prop);
- if (currentVersion && Semver.gt(currentVersion, version)) {
+ if (currentVersion && Semver.gt(currentVersion, version) && transformType !== 'reference') {
// the previous transform function increased the object's migrationVersion; break out of the loop
break;
}We intentionally omit |
|
I created an issue to track this bug: #120129 I'll submit a PR to fix it and link it to that issue. |
While working on #118617, there is a failing functional test Copy Saved Objects to Space. @jportner stated that saved object copying is just using export and import saved object APIs and we were able to produce the same problem by just importing the saved objects from the test.
@TinaHeiligers @rudolf I have opened this PR to isolate the changes from #118617 to demonstrate saved object import failure. To view the problem, run this PR and import the saved object found below. Notice how the import never finishes and the Kibana server is now unresponsive. Not sure if this is related to #117879 but sounds similar.
ndjson export of "A Dashboard" used in failing test copy_saved_objects.ts