Remove `data_enhanced` plugin #122075

Dosant · 2021-12-28T12:31:59Z

Summary

Close #119321

This pr removes the data_enhanced plugin and moves its code to the data plugin
The _enhanced plugins were created as a license workaround which is no longer needed and we can clean it up.

Most of the moved code is search-sessions code. That code moved to data plugin and data plugin now has to depend on security and task_manager (src plugins can depend on x-pack plugins as an optional dep). In some places this dependency created circular dependencies which were fixed by fixing the dependence to be on dataViews plugin instead of data plugin.

Risk Matrix

Risk	Probability	Severity	Mitigation/Notes
Check that search-session work. Including the configs and task manager tasks	Mid	High	Most of the moved code is search-session related

For maintainers

This was checked for breaking API changes and was labeled appropriately

Release notes

xpack.data_enhanced.* kibana.yml config params are deprecated and data.* counterparts should be used instead

Dosant · 2021-12-30T14:22:28Z

@elasticmachine merge upstream

…move-data-enhanced

Dosant · 2022-04-14T09:29:02Z

@elasticmachine merge upstream

…move-data-enhanced

Dosant · 2022-04-20T14:46:22Z

src/plugins/telemetry/schema/oss_plugins.json

@@ -154,6 +154,19 @@
        }
      }
    },
+    "search-session": {


@elastic/telemetry this config was moved from x-pack to oss. Is it safe to do? Is there anything else to be aware of?

I'll let @Bamieh or @afharo confirm, but I think it's fine

yup! all good! I don't think it changes the resulting output :)

Dosant · 2022-04-20T14:49:10Z

tsconfig.types.json

  },
  "include": [
    "src/core/server/index.ts",
    "src/core/public/index.ts",
-    "src/plugins/data/server/index.ts",


After moving the code from data_enhanced to data this type generation for docs stopped working because data now references code from x-pack (task_manager, security) and this changed the output folder structure of this types.

Since we don't use these types for app services plugins generation anymore, I just removed them. I added the rootDir to preserve the previous file tree output structure.

Dosant · 2022-04-20T14:53:42Z

src/plugins/data/server/config_deprecations.ts

@@ -19,4 +19,7 @@ export const autocompleteConfigDeprecationProvider: ConfigDeprecationProvider =
  renameFromRoot('kibana.autocompleteTimeout', 'data.autocomplete.valueSuggestions.timeout', {
    level: 'warning',
  }),
+  renameFromRoot('xpack.data_enhanced.search.sessions', 'data.search.sessions', {


session config has to be changed

@Dosant to allowlist in cloud

Dosant · 2022-04-21T11:47:29Z

@elasticmachine merge upstream

lukasolson

Code changes LGTM and things seem to be working properly (even having sessions from master & then checking out this PR).

lukasolson · 2022-04-21T16:49:31Z

src/plugins/data/server/search/session/session_service.ts

+      checkPersistedCompletedSessionExpiration
+    );
+
+    this.setupCompleted = true;


I'm not sure I understand why this is now necessary. Could you explain?

Mostly a precaution, because task_manager plugin is optional, we have to handle a code branch where search_service doesn't have access to task_manager and it can't initialize session_service:

kibana/src/plugins/data/server/search/search_service.ts

Lines 159 to 165 in 1bdbafe

if (taskManager) {

this.sessionService.setup(core, { taskManager, security });

} else {

// this should never happen in real world, but

// taskManager and security are optional deps because they are in x-pack

this.logger.debug('Skipping sessionService setup because taskManager is not available');

}

This makes it possible so that setup is never called and I wanted a clear error in case start is called by mistake in this case

lukasolson · 2022-04-21T16:58:42Z

x-pack/plugins/runtime_fields/public/types.ts

-  data: DataPublicPluginStart;
-}
+// eslint-disable-next-line @typescript-eslint/no-empty-interface
+export interface StartPlugins {}


Was this unused?

…move-data-enhanced

ymao1

Response Ops changes LGTM

YulNaumenko

Security Solution changes LGTM

Kerry350

Infra changes LGTM.

ppisljar

LGTM

…move-data-enhanced

…/kibana into d/2021-12-22-remove-data-enhanced

afharo

LGTM

afharo · 2022-04-28T09:53:37Z

src/plugins/telemetry/schema/oss_plugins.json

@@ -154,6 +154,19 @@
        }
      }
    },
+    "search-session": {


yup! all good! I don't think it changes the resulting output :)

Dosant · 2022-04-28T12:29:41Z

@elasticmachine merge upstream

elasticmachine · 2022-04-28T12:30:46Z

Pinging @elastic/kibana-app-services (Team:AppServicesSv)

Dosant · 2022-04-29T07:20:18Z

@elasticmachine merge upstream

Dosant · 2022-04-29T09:47:27Z

@elasticmachine merge upstream

patrykkopycinski · 2022-04-29T11:05:48Z

@elasticmachine merge upstream

patrykkopycinski · 2022-04-29T13:01:44Z

@elasticmachine merge upstream

kibana-ci · 2022-04-29T14:19:08Z

💛 Build succeeded, but was flaky

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`data`	456	506	+50
`dataEnhanced`	57	-	-57
total			-7

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`data`	2802	2851	+49
`dataEnhanced`	16	-	-16
total			+33

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`data`	15.1KB	52.4KB	+37.3KB
`dataEnhanced`	45.4KB	-	-45.4KB
total			-8.1KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`data`	18	19	+1
`dataEnhanced`	2	-	-2
total			-1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`data`	410.9KB	416.3KB	+5.4KB
`dataEnhanced`	9.3KB	-	-9.3KB
`esUiShared`	130.3KB	130.3KB	-21.0B
total			-3.9KB

Unknown metric groups

API count

id	before	after	diff
`data`	3414	3463	+49
`dataEnhanced`	16	-	-16
total			+33

async chunk count

id	before	after	diff
`data`	2	4	+2
`dataEnhanced`	2	-	-2
total			-0

ESLint disabled line counts

id	before	after	diff
`data`	39	57	+18
`dataEnhanced`	18	-	-18
`runtimeFields`	2	3	+1
total			+1

References to deprecated APIs

id	before	after	diff
`data`	375	382	+7
`dataEnhanced`	49	-	-49
total			-42

Total ESLint disabled count

id	before	after	diff
`data`	43	61	+18
`dataEnhanced`	18	-	-18
`runtimeFields`	2	3	+1
total			+1

Unreferenced deprecated APIs

id	before	after	diff
`data`	56	57	+1

History

💔 Build #41418 failed 03ae82f
💔 Build #41402 failed f6327d9
💔 Build #41376 failed 5fd2964
💛 Build #41152 was flaky 032d04a
💔 Build #41100 failed e315b38

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Code moved into `data` plugin

This was referenced Jan 4, 2022

remove deps on data plugin #121955

Closed

[data.search] remove data_enhanced #119321

Closed

Dosant mentioned this pull request Jan 25, 2022

data dependent on security breaks tests #123720

Closed

remove data_enhanced

83640bc

Dosant force-pushed the d/2021-12-22-remove-data-enhanced branch from 1d61349 to 83640bc Compare April 6, 2022 13:27

Dosant added 7 commits April 6, 2022 16:10

fix docs generation

758f5ea

fix test

b04a82d

fix tests

cd2ad4d

Merge branch 'main' of github.com:elastic/kibana into d/2021-12-22-re…

051b1a6

…move-data-enhanced

Merge branch 'main' of github.com:elastic/kibana into d/2021-12-22-re…

503c731

…move-data-enhanced

Merge branch 'main' of github.com:elastic/kibana into d/2021-12-22-re…

7ad56d4

…move-data-enhanced

more fixes

b52fd3f

kibanamachine and others added 8 commits April 14, 2022 04:29

Merge branch 'main' into d/2021-12-22-remove-data-enhanced

141bce5

Merge branch 'main' of github.com:elastic/kibana into d/2021-12-22-re…

5844657

…move-data-enhanced

fix lint errors

0ce1dd7

remove data-enhacned mapping

79d8fa8

lint fixes

3bd62f6

more lint fixes

2bd8bea

fix i18n

f74d27c

Merge branch 'main' of github.com:elastic/kibana into d/2021-12-22-re…

fc809ff

…move-data-enhanced

Dosant changed the title ~~D/2021 12 22 remove data enhanced~~ Remove data_enhanced plugin Apr 20, 2022

Dosant commented Apr 20, 2022

View reviewed changes

Dosant requested a review from lukasolson April 20, 2022 14:57

Merge branch 'main' into d/2021-12-22-remove-data-enhanced

1bdbafe

lukasolson approved these changes Apr 21, 2022

View reviewed changes

Merge branch 'main' of github.com:elastic/kibana into d/2021-12-22-re…

c23c4b3

…move-data-enhanced

ymao1 approved these changes Apr 25, 2022

View reviewed changes

YulNaumenko approved these changes Apr 25, 2022

View reviewed changes

Kerry350 self-requested a review April 26, 2022 12:48

Kerry350 approved these changes Apr 27, 2022

View reviewed changes

ppisljar approved these changes Apr 28, 2022

View reviewed changes

Dosant added 3 commits April 28, 2022 11:41

Merge branch 'main' of github.com:elastic/kibana into d/2021-12-22-re…

65d3156

…move-data-enhanced

update kibana-docker variables

d44eaa8

Merge branch 'd/2021-12-22-remove-data-enhanced' of github.com:Dosant…

e315b38

…/kibana into d/2021-12-22-remove-data-enhanced

afharo approved these changes Apr 28, 2022

View reviewed changes

Merge branch 'main' into d/2021-12-22-remove-data-enhanced

032d04a

Dosant added the Team:AppServicesSv label Apr 28, 2022

Dosant added Feature:Search Querying infrastructure in Kibana v8.3.0 release_note:deprecation labels Apr 28, 2022

Merge branch 'main' into d/2021-12-22-remove-data-enhanced

5fd2964

Merge branch 'main' into d/2021-12-22-remove-data-enhanced

f6327d9

Merge branch 'main' into d/2021-12-22-remove-data-enhanced

03ae82f

Merge branch 'main' into d/2021-12-22-remove-data-enhanced

e2c6c68

Dosant merged commit e603d92 into elastic:main Apr 29, 2022

kibanamachine added the backport:skip This commit does not require backporting label Apr 29, 2022

kertal pushed a commit to kertal/kibana that referenced this pull request May 24, 2022

Remove data_enhanced plugin (elastic#122075)

1b6e2d0

Code moved into `data` plugin

lukasolson mentioned this pull request Nov 9, 2022

Merge discover & discover_enhanced plugins #144936

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove `data_enhanced` plugin #122075

Remove `data_enhanced` plugin #122075

Dosant commented Dec 28, 2021 •

edited

Loading

Dosant commented Dec 30, 2021

Dosant commented Apr 14, 2022

Dosant Apr 20, 2022

pgayvallet Apr 26, 2022

afharo Apr 28, 2022

Dosant Apr 20, 2022

Dosant Apr 20, 2022

Dosant commented Apr 21, 2022

lukasolson left a comment

lukasolson Apr 21, 2022

Dosant Apr 25, 2022 •

edited

Loading

lukasolson Apr 21, 2022

Dosant Apr 25, 2022

ymao1 left a comment

YulNaumenko left a comment

Kerry350 left a comment

ppisljar left a comment

afharo left a comment

afharo Apr 28, 2022

Dosant commented Apr 28, 2022

elasticmachine commented Apr 28, 2022

Dosant commented Apr 29, 2022

Dosant commented Apr 29, 2022

patrykkopycinski commented Apr 29, 2022

patrykkopycinski commented Apr 29, 2022

kibana-ci commented Apr 29, 2022

API count

async chunk count

ESLint disabled line counts

References to deprecated APIs

Total ESLint disabled count

Unreferenced deprecated APIs

	if (taskManager) {
	this.sessionService.setup(core, { taskManager, security });
	} else {
	// this should never happen in real world, but
	// taskManager and security are optional deps because they are in x-pack
	this.logger.debug('Skipping sessionService setup because taskManager is not available');
	}

Remove data_enhanced plugin #122075

Remove data_enhanced plugin #122075

Conversation

Dosant commented Dec 28, 2021 • edited Loading

Summary

Risk Matrix

For maintainers

Release notes

Dosant commented Dec 30, 2021

Dosant commented Apr 14, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Dosant commented Apr 21, 2022

lukasolson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Dosant Apr 25, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ymao1 left a comment

Choose a reason for hiding this comment

YulNaumenko left a comment

Choose a reason for hiding this comment

Kerry350 left a comment

Choose a reason for hiding this comment

ppisljar left a comment

Choose a reason for hiding this comment

afharo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Dosant commented Apr 28, 2022

elasticmachine commented Apr 28, 2022

Dosant commented Apr 29, 2022

Dosant commented Apr 29, 2022

patrykkopycinski commented Apr 29, 2022

patrykkopycinski commented Apr 29, 2022

kibana-ci commented Apr 29, 2022

💛 Build succeeded, but was flaky

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

async chunk count

ESLint disabled line counts

References to deprecated APIs

Total ESLint disabled count

Unreferenced deprecated APIs

History

Remove `data_enhanced` plugin #122075

Remove `data_enhanced` plugin #122075

Dosant commented Dec 28, 2021 •

edited

Loading

Dosant Apr 25, 2022 •

edited

Loading