Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split .kibana system index into multiple indices #154444

Closed
wants to merge 29 commits into from
Closed

Split .kibana system index into multiple indices #154444

Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
050034d
Allow relocating SO to different indices during migration
gsoldevila Mar 31, 2023
30cffcb
Revert changes in ES archiver
gsoldevila Mar 31, 2023
dffba15
Merge branch 'main' into kbn-104081-split-multiple-indices-logic
gsoldevila Mar 31, 2023
75dcf23
Revert correctly ^^
gsoldevila Mar 31, 2023
c0f8249
Fix failing integration tests, add a new one to test the split
gsoldevila Apr 3, 2023
cd0f5d0
Merge branch 'main' into kbn-104081-split-multiple-indices-logic
gsoldevila Apr 3, 2023
942c026
Merge branch 'main' into kbn-104081-split-multiple-indices-logic
gsoldevila Apr 4, 2023
dfc8045
Define MAIN_SAVED_OBJECT_INDEX in @kbn/core-saved-objects-server
gsoldevila Apr 4, 2023
279a2ed
Merge branch 'main' into kbn-104081-split-multiple-indices-logic
gsoldevila Apr 4, 2023
59c2908
Merge branch 'main' into kbn-104081-split-multiple-indices-logic
gsoldevila Apr 5, 2023
14e9f8e
Relocate 'cases' saved objects to '.kibana_cases' index
gsoldevila Apr 5, 2023
03bdb74
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Apr 5, 2023
05dea55
Fix integration tests
gsoldevila Apr 6, 2023
3e3d4b9
Make `indexPattern: string` again
gsoldevila Apr 6, 2023
c895981
Merge branch 'main' into kbn-104081-split-multiple-indices-logic
gsoldevila Apr 6, 2023
6c10618
Merge branch 'kbn-104081-split-multiple-indices-logic' into kbn-10408…
gsoldevila Apr 6, 2023
be3541e
Fix typo: involed => involved
gsoldevila Apr 6, 2023
087742d
Address some PR remarks
gsoldevila Apr 11, 2023
fa57995
Merge branch 'main' into kbn-104081-split-multiple-indices-logic
gsoldevila Apr 11, 2023
d1c1055
Merge branch 'kbn-104081-split-multiple-indices-logic' into kbn-10408…
gsoldevila Apr 11, 2023
6483764
Fix incorrect call to esClient.get(); use .kibana_cases index
gsoldevila Apr 11, 2023
2046c27
Update mappings with .kibana_ to fake that reindex split is complete
rudolf Apr 11, 2023
4efdb10
Address some of the PR remarks
gsoldevila Apr 11, 2023
9725176
Take into account targetMappings._meta when merging
gsoldevila Apr 11, 2023
ee3e52d
Merge branch 'main' into kbn-104081-split-multiple-indices-logic
gsoldevila Apr 11, 2023
ce58e44
Merge branch 'kbn-104081-split-multiple-indices-logic' into kbn-10408…
gsoldevila Apr 11, 2023
288c4b4
Fix outdated integration test
gsoldevila Apr 11, 2023
342c400
remove unused var
pgayvallet Apr 12, 2023
d0bb843
Merge remote-tracking branch 'upstream/main' into kbn-104081-split-mu…
pgayvallet Apr 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ...ages/core/saved-objects/core-saved-objects-migration-server-internal/src/model/helpers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ export function mergeMigrationMappingPropertyHashes(
return {
...targetMappings,
_meta: {
...targetMappings._meta,
Copy link
Contributor

@pgayvallet pgayvallet Apr 12, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no unit tests for this mergeMigrationMappingPropertyHashes helper. We should add some (and assert the behavior of this change/fix)

Should this be done in the current PR or on the 'upstream' one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should do it on the upstream PR.

But first I wanted to check whether this mergeMigrationMappingPropertyHashes operation is still necessary.
If we want to use these hashes as a basis to construct the indexTypesMap, I'm afraid the merge can be problematic. @rudolf WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use these in checkTargetMappings as a way to determine if mappings are up to date. If we don't merge the _meta it means there could be plugins which have applied some mappings that we don't know about. This would cause us to go to UPDATE_TARGET_MAPPINGS_PROPERTIES which would be unecessary but not a problem. So I don't see an obvious problem right now.

migrationMappingPropertyHashes: {
...indexMappings._meta?.migrationMappingPropertyHashes,
...targetMappings._meta?.migrationMappingPropertyHashes,
Expand Down