[Index Patterns] Fix return saved index pattern object #101051

Dosant · 2021-06-01T13:10:04Z

Summary

While working on runtime fields REST API I noticed the following problem:

const indexPattern1 = await indexPatterns.createAndSave(specWithFields);
const indexPattern2 = await indexPatterns.get(spec.id);

expect(indexPattern1.fields).toEqual(indexPattern2.fields) // BOOM failed

This pr makes so returned indexPattern object from createAndSave is the same that would be later returned from get. Also fixes inconsistencies in tests.

Further open question I'd like to discuss later. Separate from this pr:

Does it make sense to allow to create any fields except scripted fields? Looks likes any other fields would be overridden by the es index. Such fields are also not saved to SO:

kibana/src/plugins/data/common/index_patterns/index_patterns/index_pattern.ts

Lines 324 to 326 in f7698bd

    
           fields: this.fields 
        
             ? JSON.stringify(this.fields.filter((field) => field.scripted)) 
        
             : undefined,

Considering that only scripted fields are saved, how allowNoIndex option supposed to work then? Looks like currently, it is not possible to create a non-scripted/non-runtime field if a field is not in es index?

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
This was checked for cross-browser compatibility

For maintainers

This was checked for breaking API changes and was labeled appropriately

Dosant · 2021-06-01T14:16:03Z

src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts

    if (this.savedObjectsCache) {
      this.savedObjectsCache.push(response as SavedObject<IndexPatternSavedObjectAttrs>);
    }
-    return indexPattern;
+    return createdIndexPattern;


This is the main change. I want returned index pattern to go through the all transformations that are happening when saved objects is saved

src/plugins/data/server/index_patterns/routes/create_index_pattern.ts

Dosant · 2021-06-01T14:49:13Z

test/api_integration/apis/index_patterns/index_pattern_crud/create_index_pattern/main.ts

      });

-      expect(response.status).to.be(200);
-      expect(response.body.index_pattern.title).to.be(title);
-      expect(response.body.index_pattern.fields.foo.name).to.be('foo');


After the functional change this test stopped working because foo field was actually never saved into saved object

mattkime

Changes look good although I think we could preserve version as part of the index pattern spec.

Dosant · 2021-06-01T15:08:35Z

@mattkime, I reverted functional version change, will create a bug issue to follow up with problems I noticed with it

elasticmachine · 2021-06-01T15:09:15Z

Pinging @elastic/kibana-app-services (Team:AppServices)

mattkime

changes look good and work well. nice cleanup!

Dosant · 2021-06-01T18:51:23Z

@elasticmachine merge upstream

kibanamachine · 2021-06-01T20:37:40Z

💚 Build Succeeded

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`data`	820.8KB	821.3KB	+466.0B

Unknown metric groups

References to deprecated APIs

id	before	after	diff
`canvas`	29	25	-4
`crossClusterReplication`	8	6	-2
`fleet`	22	20	-2
`globalSearch`	4	2	-2
`indexManagement`	12	7	-5
`infra`	256	146	-110
`lens`	67	45	-22
`licensing`	18	15	-3
`lists`	239	236	-3
`maps`	286	208	-78
`ml`	121	115	-6
`monitoring`	109	56	-53
`securitySolution`	390	346	-44
`stackAlerts`	101	95	-6
total			-340

History

💚 Build #128386 succeeded 39476cf
💛 Build #128362 was flaky e6642c4
💔 Build #128346 failed f150fad
💛 Build #128333 was flaky 9dfe69f
💔 Build #128360 failed 315868f

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

#101134) * [Index Patterns] Fix return saved index pattern object (#101051) * update docs

…sens/kibana into reporting/new-png-pdf-report-type * 'reporting/new-png-pdf-report-type' of github.com:jloleysens/kibana: (46 commits) [Security Solution] Add Ransomware canary advanced policy option (elastic#101068) [Exploratory view] Core web vitals (elastic#100320) [Security solution][Endpoint] Add unit tests for fleet event filters/trusted apps cards (elastic#101034) [Lens] Use a setter function for the dimension panel (elastic#101123) [Index Patterns] Fix return saved index pattern object (elastic#101051) [CI] For PRs, build TS refs before public api docs check (elastic#100791) [Maps] fix line and polygon label regression (elastic#101085) Migrate CCR to new ES JS client. (elastic#100131) [Canvas] Switch Canvas to use React Router (elastic#100579) [Expressions] Use table column ID instead of name when set (elastic#99724) [DOCS] Updates docs landing page (elastic#100749) [DOCS] Corrects typo in step 3 (elastic#101079) [DOCS] Updates runtime example in Discover (elastic#100926) Migrate kibana.autocomplete config to data plugin (elastic#100586) [Uptime] New width/delay definition for waterfall sidebar item tooltip (elastic#100147) [FTR] Use importExport for saved_object/basic archive (elastic#100244) [Fleet] Better input for multi text input in agent policy builder (elastic#101020) [CI] Buildkite support with Baseline pipeline (elastic#100492) [Reporting/Telemetry] Do not send telemetry if we are in screenshot mode (elastic#100388) Create API keys with metadata (elastic#100682) ...

* master: (68 commits) Unskip advanced settings a11y test (elastic#100558) [App Search] Crawler Landing Page (elastic#100822) [DOCS] Clarify when to use kbn clean (elastic#101155) change label behavior (elastic#100991) skip flaky suite (elastic#101126) Fix cases plugin ownership (elastic#101073) [Home] Adding file upload to add data page (elastic#100863) [ML] Functional tests - reenable categorization tests (elastic#101137) [DOCS] Adds server.uuid to settings docs (elastic#101121) Fix newsfeed unread notifications always on when reloading Kibana (elastic#100357) [Lens] Time shift metrics (elastic#98781) [Deprecations service] make `correctiveActions.manualSteps` required (elastic#100997) Add "Risk Matrix" section to the PR template (elastic#100649) [Maps] spatially filter by all geo fields (elastic#100735) [Security Solution] Add Ransomware canary advanced policy option (elastic#101068) [Exploratory view] Core web vitals (elastic#100320) [Security solution][Endpoint] Add unit tests for fleet event filters/trusted apps cards (elastic#101034) [Lens] Use a setter function for the dimension panel (elastic#101123) [Index Patterns] Fix return saved index pattern object (elastic#101051) [CI] For PRs, build TS refs before public api docs check (elastic#100791) ...

Dosant added 5 commits June 1, 2021 15:09

fix field save issue

9dfe69f

improve

f150fad

improve

22a7e49

improve

315868f

improve

e6642c4

Dosant commented Jun 1, 2021

View reviewed changes

Dosant changed the title ~~fix field save issue~~ [Index Patterns] Fix return saved index pattern object Jun 1, 2021

Dosant commented Jun 1, 2021

View reviewed changes

src/plugins/data/server/index_patterns/routes/create_index_pattern.ts Show resolved Hide resolved

Dosant commented Jun 1, 2021

View reviewed changes

revert version

39476cf

mattkime requested changes Jun 1, 2021

View reviewed changes

Dosant requested a review from mattkime June 1, 2021 15:08

Dosant added release_note:skip Skip the PR/issue when compiling release notes Team:AppServices v7.14.0 v8.0.0 Feature:Data Views Data Views code and UI - index patterns before 8.0 labels Jun 1, 2021

Dosant marked this pull request as ready for review June 1, 2021 15:09

Dosant requested a review from a team as a code owner June 1, 2021 15:09

mattkime approved these changes Jun 1, 2021

View reviewed changes

Merge branch 'master' into dev/fields-save-issue

268fdf8

Dosant merged commit 0060701 into elastic:master Jun 2, 2021

Dosant mentioned this pull request Jun 2, 2021

[7.x] [Index Patterns] Fix return saved index pattern object (#101051) #101134

Merged

Dosant added a commit to Dosant/kibana that referenced this pull request Jun 2, 2021

[Index Patterns] Fix return saved index pattern object (elastic#101051)

979f37f

Dosant mentioned this pull request Jun 2, 2021

[Index Patterns] Version when creating index pattern #101142

Closed

Dosant added a commit that referenced this pull request Jun 2, 2021

[7.x] [Index Patterns] Fix return saved index pattern object (#101051) (

4202cb4

#101134) * [Index Patterns] Fix return saved index pattern object (#101051) * update docs

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Index Patterns] Fix return saved index pattern object #101051

[Index Patterns] Fix return saved index pattern object #101051

Dosant commented Jun 1, 2021 •

edited

Loading

Dosant Jun 1, 2021

Dosant Jun 1, 2021

mattkime left a comment

Dosant commented Jun 1, 2021

elasticmachine commented Jun 1, 2021

mattkime left a comment

Dosant commented Jun 1, 2021

kibanamachine commented Jun 1, 2021

References to deprecated APIs

	fields: this.fields
	? JSON.stringify(this.fields.filter((field) => field.scripted))
	: undefined,

[Index Patterns] Fix return saved index pattern object #101051

[Index Patterns] Fix return saved index pattern object #101051

Conversation

Dosant commented Jun 1, 2021 • edited Loading

Summary

Further open question I'd like to discuss later. Separate from this pr:

Checklist

For maintainers

Dosant Jun 1, 2021

Choose a reason for hiding this comment

Dosant Jun 1, 2021

Choose a reason for hiding this comment

mattkime left a comment

Choose a reason for hiding this comment

Dosant commented Jun 1, 2021

elasticmachine commented Jun 1, 2021

mattkime left a comment

Choose a reason for hiding this comment

Dosant commented Jun 1, 2021

kibanamachine commented Jun 1, 2021

💚 Build Succeeded

Metrics [docs]

Page load bundle

References to deprecated APIs

History

Dosant commented Jun 1, 2021 •

edited

Loading