Skip to content

Commit

Permalink
[8.13] [Index Management][Mappings editor] Allow multi-fields support…
Browse files Browse the repository at this point in the history
… for most field types (#178032) (#178337)

# Backport

This will backport the following commits from `main` to `8.13`:
- [[Index Management][Mappings editor] Allow multi-fields support for
most field types
(#178032)](#178032)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Yulia
Čech","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-03-08T16:57:20Z","message":"[Index
Management][Mappings editor] Allow multi-fields support for most field
types (#178032)\n\n## Summary\r\nFixes
https://github.com/elastic/kibana/issues/177822\r\nFixes
https://github.com/elastic/kibana/issues/112879\r\n\r\nThis PR allows
adding multi-fields to most mapping fields except the\r\nfollowing
list\r\n> flattened\r\n> aggregate_metric_double\r\n> join\r\n>
constant_keyword\r\n> shape\r\n> geo_shape\r\n> point\r\n\r\nThis fixes
a bug with mutli-fields being dropped from a field when the\r\nUI was
not supporting mutli-fields for that field type.\r\n\r\n### How to test
\r\nAdding multi-fields\r\n1. Navigate to Index Management -> Index
templates or Component\r\ntemplates tab\r\n2. Start creating a template
and get to the Mappings step in the wizard\r\n3. Try out different type
fields and check that it's possible to add\r\nmutli-fields\r\n\r\nNot
dropping multi-fields\r\n1. Create an index template \r\n```\r\nPUT
_index_template/test\r\n{\r\n \"template\": {\r\n \"mappings\": {\r\n
\"properties\": {\r\n \"command_line\": {\r\n \"fields\": {\r\n
\"text\": {\r\n \"type\": \"match_only_text\"\r\n }\r\n },\r\n \"type\":
\"wildcard\"\r\n }\r\n }\r\n }\r\n },\r\n \"index_patterns\": [\r\n
\"test\"\r\n ],\r\n \"data_stream\": {},\r\n \"allow_auto_create\":
false\r\n}\r\n```\r\n2. Navigate to Index Management -> Index
templates\r\n3. Find the created template, open it and save without
changes\r\n4. Check that the nested multi-fields are still present in
the field \r\n`GET _index_template/test `\r\n\r\n### Screenshots
\r\n<img width=\"907\" alt=\"Screenshot 2024-03-07 at 17 29
03\"\r\nsrc=\"https://github.com/elastic/kibana/assets/6585477/9a16d24d-b68d-479d-928b-58a5ec5c9147\">\r\n<img
width=\"904\" alt=\"Screenshot 2024-03-07 at 17 30
27\"\r\nsrc=\"https://github.com/elastic/kibana/assets/6585477/121d5946-7bad-4ffb-afa3-c33c81b793a6\">\r\n\r\n\r\n\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [ ] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [ ] Any UI touched in this PR is
usable by keyboard only (learn more\r\nabout [keyboard
accessibility](https://webaim.org/techniques/keyboard/))\r\n- [ ] Any UI
touched in this PR does not create any new axe failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"0fe6a9349576b8ddb1d03c2223dc6179eb2dcd33","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Feature:Index
Management","Team:Deployment
Management","release_note:skip","v8.13.0","v8.14.0"],"title":"[Index
Management][Mappings editor] Allow multi-fields support for most field
types","number":178032,"url":"https://github.com/elastic/kibana/pull/178032","mergeCommit":{"message":"[Index
Management][Mappings editor] Allow multi-fields support for most field
types (#178032)\n\n## Summary\r\nFixes
https://github.com/elastic/kibana/issues/177822\r\nFixes
https://github.com/elastic/kibana/issues/112879\r\n\r\nThis PR allows
adding multi-fields to most mapping fields except the\r\nfollowing
list\r\n> flattened\r\n> aggregate_metric_double\r\n> join\r\n>
constant_keyword\r\n> shape\r\n> geo_shape\r\n> point\r\n\r\nThis fixes
a bug with mutli-fields being dropped from a field when the\r\nUI was
not supporting mutli-fields for that field type.\r\n\r\n### How to test
\r\nAdding multi-fields\r\n1. Navigate to Index Management -> Index
templates or Component\r\ntemplates tab\r\n2. Start creating a template
and get to the Mappings step in the wizard\r\n3. Try out different type
fields and check that it's possible to add\r\nmutli-fields\r\n\r\nNot
dropping multi-fields\r\n1. Create an index template \r\n```\r\nPUT
_index_template/test\r\n{\r\n \"template\": {\r\n \"mappings\": {\r\n
\"properties\": {\r\n \"command_line\": {\r\n \"fields\": {\r\n
\"text\": {\r\n \"type\": \"match_only_text\"\r\n }\r\n },\r\n \"type\":
\"wildcard\"\r\n }\r\n }\r\n }\r\n },\r\n \"index_patterns\": [\r\n
\"test\"\r\n ],\r\n \"data_stream\": {},\r\n \"allow_auto_create\":
false\r\n}\r\n```\r\n2. Navigate to Index Management -> Index
templates\r\n3. Find the created template, open it and save without
changes\r\n4. Check that the nested multi-fields are still present in
the field \r\n`GET _index_template/test `\r\n\r\n### Screenshots
\r\n<img width=\"907\" alt=\"Screenshot 2024-03-07 at 17 29
03\"\r\nsrc=\"https://github.com/elastic/kibana/assets/6585477/9a16d24d-b68d-479d-928b-58a5ec5c9147\">\r\n<img
width=\"904\" alt=\"Screenshot 2024-03-07 at 17 30
27\"\r\nsrc=\"https://github.com/elastic/kibana/assets/6585477/121d5946-7bad-4ffb-afa3-c33c81b793a6\">\r\n\r\n\r\n\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [ ] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [ ] Any UI touched in this PR is
usable by keyboard only (learn more\r\nabout [keyboard
accessibility](https://webaim.org/techniques/keyboard/))\r\n- [ ] Any UI
touched in this PR does not create any new axe failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"0fe6a9349576b8ddb1d03c2223dc6179eb2dcd33"}},"sourceBranch":"main","suggestedTargetBranches":["8.13"],"targetPullRequestStates":[{"branch":"8.13","label":"v8.13.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/178032","number":178032,"mergeCommit":{"message":"[Index
Management][Mappings editor] Allow multi-fields support for most field
types (#178032)\n\n## Summary\r\nFixes
https://github.com/elastic/kibana/issues/177822\r\nFixes
https://github.com/elastic/kibana/issues/112879\r\n\r\nThis PR allows
adding multi-fields to most mapping fields except the\r\nfollowing
list\r\n> flattened\r\n> aggregate_metric_double\r\n> join\r\n>
constant_keyword\r\n> shape\r\n> geo_shape\r\n> point\r\n\r\nThis fixes
a bug with mutli-fields being dropped from a field when the\r\nUI was
not supporting mutli-fields for that field type.\r\n\r\n### How to test
\r\nAdding multi-fields\r\n1. Navigate to Index Management -> Index
templates or Component\r\ntemplates tab\r\n2. Start creating a template
and get to the Mappings step in the wizard\r\n3. Try out different type
fields and check that it's possible to add\r\nmutli-fields\r\n\r\nNot
dropping multi-fields\r\n1. Create an index template \r\n```\r\nPUT
_index_template/test\r\n{\r\n \"template\": {\r\n \"mappings\": {\r\n
\"properties\": {\r\n \"command_line\": {\r\n \"fields\": {\r\n
\"text\": {\r\n \"type\": \"match_only_text\"\r\n }\r\n },\r\n \"type\":
\"wildcard\"\r\n }\r\n }\r\n }\r\n },\r\n \"index_patterns\": [\r\n
\"test\"\r\n ],\r\n \"data_stream\": {},\r\n \"allow_auto_create\":
false\r\n}\r\n```\r\n2. Navigate to Index Management -> Index
templates\r\n3. Find the created template, open it and save without
changes\r\n4. Check that the nested multi-fields are still present in
the field \r\n`GET _index_template/test `\r\n\r\n### Screenshots
\r\n<img width=\"907\" alt=\"Screenshot 2024-03-07 at 17 29
03\"\r\nsrc=\"https://github.com/elastic/kibana/assets/6585477/9a16d24d-b68d-479d-928b-58a5ec5c9147\">\r\n<img
width=\"904\" alt=\"Screenshot 2024-03-07 at 17 30
27\"\r\nsrc=\"https://github.com/elastic/kibana/assets/6585477/121d5946-7bad-4ffb-afa3-c33c81b793a6\">\r\n\r\n\r\n\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [ ] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [ ] Any UI touched in this PR is
usable by keyboard only (learn more\r\nabout [keyboard
accessibility](https://webaim.org/techniques/keyboard/))\r\n- [ ] Any UI
touched in this PR does not create any new axe failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"0fe6a9349576b8ddb1d03c2223dc6179eb2dcd33"}}]}]
BACKPORT-->

Co-authored-by: Yulia Čech <[email protected]>
  • Loading branch information
kibanamachine and yuliacech authored Mar 8, 2024
1 parent 50406b4 commit a75e5d4
Show file tree
Hide file tree
Showing 3 changed files with 75 additions and 11 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -436,4 +436,46 @@ describe('Mappings editor: core', () => {
expect(data).toEqual(updatedMappings);
});
});

describe('multi-fields support', () => {
it('allows multi-fields for most types', async () => {
const value = {
properties: {
name1: {
type: 'wildcard',
},
},
};
await act(async () => {
testBed = setup({ onChange: onChangeHandler, value });
});

const { component, exists } = testBed;
component.update();
expect(exists('addMultiFieldButton')).toBe(true);
});

it('keeps the fields property in the field', async () => {
const value = {
properties: {
name1: {
type: 'wildcard',
fields: {
text: {
type: 'match_only_text',
},
},
},
},
};
await act(async () => {
testBed = setup({ onChange: onChangeHandler, value });
});

const { component } = testBed;
component.update();
({ data } = await getMappingsEditorData(component));
expect(data).toEqual(value);
});
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -79,15 +79,26 @@ describe('utils', () => {
});

describe('getFieldMeta', () => {
test('returns "canHaveMultiFields:true" for text data type', () => {
expect(getFieldMeta({ name: 'text_field', type: 'text' }).canHaveMultiFields).toEqual(true);
});
test('returns "canHaveMultiFields:true" for keyword data type', () => {
expect(getFieldMeta({ name: 'keyword_field', type: 'keyword' }).canHaveMultiFields).toEqual(
true
);
});
test('returns "canHaveMultiFields:true" for IP data type', () => {
expect(getFieldMeta({ name: 'ip_field', type: 'ip' })).toEqual({
canHaveChildFields: false,
canHaveMultiFields: true,
childFieldsName: 'fields',
hasChildFields: false,
hasMultiFields: false,
isExpanded: false,
});
expect(getFieldMeta({ name: 'ip_field', type: 'ip' }).canHaveMultiFields).toEqual(true);
});
test('returns "canHaveMultiFields:true" for wildcard data type', () => {
expect(getFieldMeta({ name: 'wildcard_field', type: 'wildcard' }).canHaveMultiFields).toEqual(
true
);
});
test('returns "canHaveMultiFields:false" for flattened data type', () => {
expect(
getFieldMeta({ name: 'flattened_field', type: 'flattened' }).canHaveMultiFields
).toEqual(false);
});
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,24 @@ import { TreeItem } from '../components/tree';

export const getUniqueId = () => uuidv4();

const fieldsWithoutMultiFields: DataType[] = [
// @ts-expect-error aggregate_metric_double is not yet supported by the editor
'aggregate_metric_double',
'constant_keyword',
'flattened',
'geo_shape',
'join',
'percolator',
'point',
'shape',
];
export const getChildFieldsName = (dataType: DataType): ChildFieldName | undefined => {
if (dataType === 'text' || dataType === 'keyword' || dataType === 'ip') {
return 'fields';
if (fieldsWithoutMultiFields.includes(dataType)) {
return undefined;
} else if (dataType === 'object' || dataType === 'nested') {
return 'properties';
}
return undefined;
return 'fields';
};

export const getFieldMeta = (field: Field, isMultiField?: boolean): FieldMeta => {
Expand Down

0 comments on commit a75e5d4

Please sign in to comment.