Namespaces can be lost when importing multi-NS objects

[pgayvallet]

Related to #164243

Types that were moved from single to multiple namespace types during 8.0.0 can loose their namespace information when being imported without type version specified (which is the case for assets installed by fleet packages), if they don't define any migration for an higher version.

We confirmed from #164243 that index-pattern is impacted, but there may be others.

Basically, if for a given type, convertToMultiNamespaceTypeVersion is set and is higher than the latest registered migration, as it is the case for index-pattern:

kibana/src/plugins/data_views/server/saved_objects/data_views.ts

Line 19 in 21351df

convertToMultiNamespaceTypeVersion: '8.0.0',

kibana/src/plugins/data_views/server/saved_objects/index_pattern_migrations.ts

Lines 60 to 64 in ca2d1d6

	export const indexPatternSavedObjectTypeMigrations = {
	'6.5.0': flow(migrateAttributeTypeAndAttributeTypeMeta),
	'7.6.0': flow(migrateSubTypeAndParentFieldProperties),
	'7.11.0': flow(addAllowNoIndex),
	};

Then, when imported without specifying typeMigrationVersion, the latest migration version is set on the document (e.g 7.11.0).

Which causes the migration system to pickup the document for migration at next restart, which tries to reapply the "convert to multi-NS" migration again, causing the existing namespaces field to be erased and replaced with the default value of [ "default" ].

However, when creating a document of the type using the create API, the max of the two values is correctly used (e.g 8.0.0 for index-pattern), so the behavior between import and create seems different here. This could be related to the usage of the convertNamespaceTypes option?

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[pgayvallet]

For instance, on main:

Creating an index-pattern via

POST /api/saved_objects/index-pattern/id-1
{
  "attributes": {}
}

creates the following document:

{
        "_index": ".kibana_analytics_8.11.0_001",
        "_id": "index-pattern:id-1",
        "_score": 2.5314267,
        "_source": {
          "index-pattern": {},
          "type": "index-pattern",
          "references": [],
          "managed": false,
          "namespaces": [
            "not-default"
          ],
          "coreMigrationVersion": "8.8.0",
          "typeMigrationVersion": "8.0.0", // <-- good
          "updated_at": "2023-08-22T14:17:08.526Z",
          "created_at": "2023-08-22T14:17:08.526Z"
        }
},

when importing

{
  "type":"index-pattern",
  "id":"logs-*",
  "attributes":{"title":"logs-*","timeFieldName":"@timestamp","allowNoIndex":true},
  "references":[]
}

creates

{
        "_index": ".kibana_analytics_8.11.0_001",
        "_id": "index-pattern:logs-*",
        "_score": 2.5314267,
        "_source": {
          "index-pattern": {
            "title": "logs-*",
            "timeFieldName": "@timestamp",
            "allowNoIndex": true
          },
          "type": "index-pattern",
          "references": [],
          "managed": true,
          "namespaces": [
            "not-default"
          ],
          "coreMigrationVersion": "8.8.0",
          "typeMigrationVersion": "7.11.0", // <-- wrong
          "updated_at": "2023-08-22T14:11:57.708Z",
          "created_at": "2023-08-22T14:11:57.708Z"
        }
      },