[SO migration] fail the migration if unknown types are encountered #118300
Conversation
pgayvallet
added
project:ResilientSavedObjectMigrations
|
@elasticmachine merge upstream |
|
retest |
|
@elasticmachine merge upstream |
| // Do not transform documents of unknown types | ||
| if (raw?._source?.type && !knownTypes.has(raw._source.type)) { | ||
| processedDocs.push(raw); | ||
| } else if (serializer.isRawSavedObject(raw, options)) { |
There was a problem hiding this comment.
This part of the revert was not strictly necessary, but it felt safer to get back to the behavior where migrating unknown doc fails, to be consistent with our strategy to fail the migration in such case (even if going that far should never occur)
| { | ||
| oss: true, | ||
| oss: false, |
There was a problem hiding this comment.
Some test datasets contains objects of types registered from xpack plugins, I had to enable xpack for those tests to avoid re-generating the dataset, as the tests were failing due to the behavior reintroduced in this PR.
| @@ -122,7 +122,7 @@ describe('migration v2', () => { | |||
| expect( | |||
| records.find((rec) => | |||
| rec.message.startsWith( | |||
| `Unable to complete saved object migrations for the [.kibana] index: The document with _id "canvas-workpad-template:workpad-template-061d7868-2b4e-4dc8-8bf7-3772b52926e5" is 1715275 bytes which exceeds the configured maximum batch size of 1015275 bytes. To proceed, please increase the 'migrations.maxBatchSizeBytes' Kibana configuration option and ensure that the Elasticsearch 'http.max_content_length' configuration option is set to an equal or larger value.` | |||
| `Unable to complete saved object migrations for the [.kibana] index: The document with _id "canvas-workpad-template:workpad-template-061d7868-2b4e-4dc8-8bf7-3772b52926e5" is 1715274 bytes which exceeds the configured maximum batch size of 1015275 bytes. To proceed, please increase the 'migrations.maxBatchSizeBytes' Kibana configuration option and ensure that the Elasticsearch 'http.max_content_length' configuration option is set to an equal or larger value.` | |||
There was a problem hiding this comment.
I have NO idea why, but enabling xpack changed the size of the document by one byte 🤷
There was a problem hiding this comment.
An extra newline maybe or some tiny change between node versions?
|
Pinging @elastic/kibana-core (Team:Core) |
…rt-on-unknown-so-types
|
@pgayvallet shouldn't "process" be "proceed" in the release note? i.e.
or
|
Definitely should, thanks. |
…rt-on-unknown-so-types
|
@elasticmachine merge upstream |
💚 Build Succeeded
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
pgayvallet
added
the
auto-backport
…lastic#118300) * manually revert elastic#105213 * enabled xpack to register types for some IT tests * fiz doc size Co-authored-by: Kibana Machine <[email protected]>
💚 Backport successful
This backport PR will be merged automatically after passing CI. |
…118300) (#118690) * manually revert #105213 * enabled xpack to register types for some IT tests * fiz doc size Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Pierre Gayvallet <[email protected]>
…118300) * manually revert #105213 * enabled xpack to register types for some IT tests * fiz doc size Co-authored-by: Kibana Machine <[email protected]>
Summary
Fix #107678 by manually reverting #105213
Revert SO migration behavior to what it was before #105213 by having it fail fast when unknown SO types are encountered during the
CHECK_UNKNOWN_DOCUMENTSstepChecklist
Release Note
Encountering unknown saved object types during a version migration will now cause Kibana to fail the upgrade. To proceed with the migration, it will be required to either re-enable the plugins owning them, or to delete the documents from the previous version's index.