[NP Kibana Migrations ] kibana plugin home #50444

TinaHeiligers · 2019-11-12T23:53:26Z

Summary

Refers to #50224
Creates a new plugin on the Kibana Platform for Home that will contain:

Tutorials API and registration
Features Catalogue
(and potentially)
sample data as well.

This PR only handles the Tutorial API and registration.

TODO:

Update docs for creating a tutorial: https://www.elastic.co/guide/en/kibana/master/add-data-guide.html#_creating_a_new_tutorial

Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~- [ ] This was checked for cross-browser compatibility, including a check against IE11~~
~~- [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support~~

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
~~- [ ] This was checked for keyboard-only and screenreader accessibility~~

For maintainers

This was checked for breaking API changes and was labeled appropriately
This includes a feature addition or change that requires a release note and was labeled appropriately

Dev docs

The api to register new tutorials was moved to the new platform. You are now able to add new tutorials by creating a plugin in the new platform, adding a dependency to home in its kibana.json and using the tutorials.registerTutorial method in the setup lifecycle:

class MyPlugin {
  setup(core: CoreSetup, plugins: { home: HomeServerPluginSetup }) {
    home.tutorials.registerTutorial(() => ({ /* tutorial definition */ }));
  }
}

It is still possible to register tutorials from within the legacy platform by calling the same method exposed on the server object:

server.newPlatform.setup.plugins.home.tutorials.registerTutorial(() => ({ /* tutorial definition */ }));

…ver_tutorials_api

…orm tutorial plugin registrations

TinaHeiligers · 2019-11-12T23:54:08Z

@flash1293 What labels do we add to NP migration PR'S?

flash1293 · 2019-11-13T08:27:59Z

@TinaHeiligers Feature: NP Migration

…orials

…gin_home

… tutorials service

TinaHeiligers · 2019-11-14T17:04:05Z

@legrego @kobelb, we're in the process of migrating the tutorials registry api to the new platform and was hoping you'd be able to help with availability of getSpaceId in the New Platform on the request context:
From the original request in the issue:

Spaces relies on access to the raw Hapi object to get the space ID in x-pack/plugins/spaces/server/lib/spaces_tutorial_context_factory.ts. This will need to be refactored before tutorials can be moved. When will the getSpaceId be available in the New Platform? We expect it to be available on the request context?

TinaHeiligers · 2019-11-14T17:06:47Z

@aaronjcaldwell could you help with the following blocker?

The EMS tutorial (src/legacy/core_plugins/kibana/server/tutorials/ems/index.js) is relying on the global config object to access server.basePath and map.emsLandingPageUrl. The basePath can be accessed via core.http.basePath.get but the map config will need to be provided by the Maps plugin.

TinaHeiligers · 2019-11-14T17:08:24Z

@ogupte is this something you could help with the following blocker?

The APM tutorial (src/legacy/core_plugins/kibana/server/tutorials/apm/index.js) is relying on the legacy global config object to get access to the apm_oss.indexPattern and xpack.apm.ui.enabled keys. This will need to be refactored to get these configs directly from the apm_oss and apm plugins' setup contract.

…gin_home

TinaHeiligers · 2019-11-16T00:10:29Z

The spaces changes look good. Regarding Joe's comment above:

The other two problems we can tackle later in a subsequent PR - if everything is using the new API we can remove the legacy mixin and also remove the legacy tutorial context factory call out of the spaces plugin.

Are you aware of anything else using the legacy context factory? I think it was originally introduced solely for the Spaces plugin, so we might be ready to remove it altogether? What do you think?

I don't think the legacy context factory is being used elsewhere but would prefer to verify this with the @elastic/kibana-platform team.

elasticmachine · 2019-11-16T01:01:05Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 93d67d6

pgayvallet

I will let @eliperelman properly review, but it's looking good for me!

A few minor points and questions:

src/plugins/home/server/plugin.ts

src/legacy/core_plugins/kibana/public/home/load_tutorials.js

pgayvallet · 2019-11-16T12:05:01Z

src/plugins/home/server/lib/tutorial_schema.ts

+const dashboardSchema = Joi.object({
+  id: Joi.string().required(), // Dashboard saved object id
+  linkLabel: Joi.string().when('isOverview', {
+    is: true,
+    then: Joi.required(),
+  }),
+  // Is this an Overview / Entry Point dashboard?
+  isOverview: Joi.boolean().required(),
+});


Is there a reason to be using joi directly instead of @kbn/config-schema?

That's the way it was prior to the move and I didn't want to introduce too many changes at once.
I did start but then ran into validations that aren't obviously available in @kbn/config-schema.
e.g. How do I go about validating an empty object for

query: { match_all: {}, }, query: { bool: { filter: { term: { 'agent.type': 'auditbeat', }, }, }, }, query: { bool: { filter: [ { term: { 'processor.event': 'onboarding' } }, { range: { 'observer.version_major': { gte: 7 } } }, ], }, },

WIth Joi validation, this is given as:

query: joi.object(),

I'll be happy to continue this effort with some assistance ;-) There's a throw-away, draft PR linked: TinaHeiligers#11

Ok, thanks for the answer. Not everything in joi is available in config-schema, so migration may be not that obvious. Will try to look at the migration PR. joi is fine for now.

…gin_home

elasticmachine · 2019-11-16T16:59:00Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 1d79358

…s api http routes for legacy platform and NP

elasticmachine · 2019-11-16T21:39:11Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 8d49809

pgayvallet

LGTM for me. Ideally would still like a second opinion from other assigned platform reviewers.

pgayvallet · 2019-11-17T10:51:25Z

src/legacy/core_plugins/kibana/public/home/load_tutorials.js

+const baseUrl = getServices().addBasePath('/api/kibana/home/tutorials_LP');
+const baseUrlNP = getServices().addBasePath('/api/kibana/home/tutorials');


NIT: I would also rename the consts accordingly but this is very minor

pgayvallet · 2019-11-17T11:01:14Z

src/legacy/core_plugins/kibana/server/tutorials/register.js

+  server.newPlatform.setup.plugins.home.tutorials.registerTutorial(iptablesLogsSpecProvider);
+  server.newPlatform.setup.plugins.home.tutorials.registerTutorial(ciscoLogsSpecProvider);
+  server.newPlatform.setup.plugins.home.tutorials.registerTutorial(envoyproxyLogsSpecProvider);
+  server.newPlatform.setup.plugins.home.tutorials.registerTutorial(couchdbMetricsSpecProvider);
  server.registerTutorial(emsBoundariesSpecProvider);


So apm and ems tutorials are the only ones still relying on legacy things and can't currently be migrated? Is this in the scope of the PR or will this be done in a next step when the plugins are ready?

That will be done as a follow up PR where we move the tutorials themselves into relevant folders. The idea is that teams/apps that own a product will move their tutorial and registration into their plugin. Tutorials that aren't app specific will remain as Kibana-type tutorials and those I'll move to the home plugin.

pgayvallet · 2019-11-17T11:01:27Z

src/plugins/home/server/services/tutorials_registry.ts

+        const emptyContext = {};
+        const { error } = Joi.validate(specProvider(emptyContext), tutorialSchema);


NIT: Joi.validate(specProvider({}), tutorialSchema);?

That's the code as it was prior to the migration. If there's something specific you would like me to change that to, please let me know.

…gin_home

elasticmachine · 2019-11-17T18:02:21Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 7f553fb

flash1293

Tested and works fine - space id shows up in the tutorial

…gin_home

eliperelman

Looks good, just had a couple nits. 👍

eliperelman · 2019-11-18T16:54:31Z

src/legacy/core_plugins/kibana/public/home/load_tutorials.js

@@ -21,29 +21,44 @@ import _ from 'lodash';
 import { getServices } from './kibana_services';
 import { i18n } from '@kbn/i18n';

+const baseUrlLP = getServices().addBasePath('/api/kibana/home/tutorials_LP');


Are we ok with using uppercase in this filename? I believe all other files in the repo are lowercase.

eliperelman · 2019-11-18T16:58:46Z

src/legacy/core_plugins/kibana/server/tutorials/register.js

-  server.registerTutorial(uwsgiMetricsSpecProvider);
-  server.registerTutorial(netflowSpecProvider);
-  server.registerTutorial(traefikLogsSpecProvider);
+  server.newPlatform.setup.plugins.home.tutorials.registerTutorial(systemLogsSpecProvider);


Do you feel like it would be beneficial for succinctness to loop through these?

[ systemMetricsSpecProvider, /* ... */ cockroachdbMetricsSpecProvider, ].forEach(provider => server.newPlatform.setup.plugins.home.tutorials.registerTutorial(provider) );

These tutorial registrations need to remain separate at this stage because they are going to move to their specific plugins (e.g. the apm tutorial registration will move to the apm plugin (or one of their plugins). When we've identified all the non-specific tutorials, we can consider looping over them. If you happen to know which ones the new home plugin should house (the non-app specific ones), please let us know!

eliperelman · 2019-11-18T17:01:20Z

src/plugins/home/kibana.json

@@ -0,0 +1,6 @@
+{
+  "id": "home",
+  "version": "kibana",


I mentioned this over zoom, but I think we should make this version field consistent with other plugins to be semver, and also use kibanaVersion:

"version": "0.0.1", "kibanaVersion": "kibana",

The work in this plugin is not yet complete and we can address this change in the next PR (#50838) that builds on this one.

eliperelman · 2019-11-18T17:06:30Z

src/plugins/home/server/plugin.ts

+
+  public setup(core: CoreSetup) {
+    return {
+      tutorials: { ...this.tutorialsRegistry.setup(core) },


Can I ask why we need to spread this instead of passing it directly?

Suggested change

tutorials: { ...this.tutorialsRegistry.setup(core) },

tutorials: this.tutorialsRegistry.setup(core),

We need to expose all the methods exposed in the setup of the service.

eliperelman · 2019-11-18T17:06:45Z

src/plugins/home/server/plugin.ts

+
+  public start() {
+    return {
+      tutorials: { ...this.tutorialsRegistry.start() },


Suggested change

tutorials: { ...this.tutorialsRegistry.start() },

tutorials: this.tutorialsRegistry.start(),

Here, we spread the start contract for consistency with the setup method, should any further methods be added to the start contract of the service in the future.

elasticmachine · 2019-11-18T18:15:54Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: d5a120e

* [NP:Kibana:homeAPI] initializes a new plugin (WIP) * Typing * New plugin tutorials not needing server working * Retains legacy tutorial registration and adds new route for new platform tutorial plugin registrations * Adds comment on where to pre-register general non-plugin specific tutorials * Converts TutorialsPlugin to TutorialsRegistry service in new Home plugin * Changes call to location of registerTutorial in registerTutorials * Adds console log for the home plugin that's returning an empty object * Removes async from setup and start methods in the home plugin and the tutorials service * Starts writing tests and creating mocks * Adds basic tests for TutorialRegistry service * Adds basic tests for TutorialRegistry service * Adds test for route * Adds mocks and tests for the home plugin * Adds home plugin to security plugin and registers scoped tutorials service * Removes incorrect addition to src core server * Fixes type errors * Deletes unused code * Deletes duplicate golangMetricsSpecProvider registration * Nests tutorials service in a tutorials key in the home plugin, changes api http routes for legacy platform and NP * Changes url variable names

…-fallback * 'master' of github.com:elastic/kibana: (116 commits) [Maps] move apply global filter settting from layer to source (elastic#50523) [SIEM] Fix: Empty `Source` / `Destination` shown when only ports are populated (elastic#50843) [Maps] Delay vector tile layer syncing until spritesheet is loaded (elastic#48955) [Maps] prevent users from overflowing URL when filtering by shape (elastic#50747) [DOCS] Mark Beats central management as discontinued (elastic#49423) [page_objects/common_page] convert to ts (elastic#50771) [NP Kibana Migrations ] kibana plugin home (elastic#50444) [DOCS] Shareables naming convention (elastic#50497) [ML] DF Analytics - auto-populate model_memory_limit (elastic#50714) Increase alerting test stability and reduce flakiness (elastic#50246) [ML] Remaning new_job_new folder (elastic#50917) [Telemetry] Show opt-in changes for OSS users (elastic#50831) [ML] Fix lat_long anomalies table links menu and value formatting (elastic#50916) [Dev] Fix serialising a really big string (elastic#50915) Better explanation about the Prettier recommendation (extension vs. NPM module) (elastic#50629) [Monitoring] Use a basic monitoring user for tests (elastic#47865) [Monitoring] Gracefully handle issue with filebeat indices (elastic#48929) [Monitoring] Improve permissions required around setup mode (elastic#50421) Additional validation for elasticsearch username (elastic#48247) Revert changes to use_kibana_ui_setting (elastic#50877) ... # Conflicts: # src/legacy/core_plugins/console/server/request.test.ts

* [NP Kibana Migrations ] kibana plugin home (#50444)

TinaHeiligers added 5 commits November 11, 2019 17:11

[NP:Kibana:homeAPI] initializes a new plugin (WIP)

6faa981

Merge branch 'master' of github.com:elastic/kibana into NP_kibana_ser…

a24e7f2

…ver_tutorials_api

Typing

eeeaae6

New plugin tutorials not needing server working

7a78b55

Retains legacy tutorial registration and adds new route for new platf…

dfa5e34

…orm tutorial plugin registrations

TinaHeiligers mentioned this pull request Nov 12, 2019

Draft: [Kibana Platform Migrations: Home Plugin ] kibana server tutorials api #50442

Closed

7 tasks

TinaHeiligers added 7 commits November 13, 2019 08:56

Adds comment on where to pre-register general non-plugin specific tut…

31548ef

…orials

Merge branch 'master' of github.com:elastic/kibana into NP_kibana_plu…

560e7d2

…gin_home

Converts TutorialsPlugin to TutorialsRegistry service in new Home plugin

c22d787

Changes call to location of registerTutorial in registerTutorials

1741e3c

Adds console log for the home plugin that's returning an empty object

aca40b9

Removes async from setup and start methods in the home plugin and the…

10d477a

… tutorials service

Starts writing tests and creating mocks

de06bc0

TinaHeiligers added 6 commits November 14, 2019 12:47

Adds basic tests for TutorialRegistry service

7a7ce9a

Adds basic tests for TutorialRegistry service

fcedb41

Merge branch 'master' of github.com:elastic/kibana into NP_kibana_plu…

a77aced

…gin_home

Merge branch 'master' of github.com:elastic/kibana into NP_kibana_plu…

1862438

…gin_home

Adds test for route

628e410

Adds mocks and tests for the home plugin

cc978bd

TinaHeiligers marked this pull request as ready for review November 14, 2019 23:38

TinaHeiligers requested a review from a team as a code owner November 14, 2019 23:38

TinaHeiligers requested review from flash1293 and joshdover November 14, 2019 23:38

TinaHeiligers added Feature:NP Migration Team:Visualizations Visualization editors, elastic-charts and infrastructure labels Nov 14, 2019

pgayvallet reviewed Nov 16, 2019

View reviewed changes

Merge branch 'master' of github.com:elastic/kibana into NP_kibana_plu…

1d79358

…gin_home

Nests tutorials service in a tutorials key in the home plugin, change…

8d49809

…s api http routes for legacy platform and NP

pgayvallet approved these changes Nov 17, 2019

View reviewed changes

TinaHeiligers added 2 commits November 17, 2019 09:35

Changes url variable names

7f553fb

Merge branch 'master' of github.com:elastic/kibana into NP_kibana_plu…

cb1a9f3

…gin_home

flash1293 mentioned this pull request Nov 18, 2019

Np kibana plugin home feature catalogue #50838

Merged

4 tasks

flash1293 added v7.6.0 and removed v7.7.0 labels Nov 18, 2019

flash1293 approved these changes Nov 18, 2019

View reviewed changes

Merge branch 'master' of github.com:elastic/kibana into NP_kibana_plu…

d5a120e

…gin_home

legrego approved these changes Nov 18, 2019

View reviewed changes

eliperelman approved these changes Nov 18, 2019

View reviewed changes

TinaHeiligers merged commit 04425ff into elastic:master Nov 18, 2019

nickofthyme mentioned this pull request Nov 18, 2019

[7.x] [NP Kibana Migrations ] kibana plugin home (#50444) #50961

Closed

TinaHeiligers mentioned this pull request Nov 18, 2019

[7.x] [NP Kibana Migrations ] kibana plugin home (#50444) #50970

Closed

TinaHeiligers mentioned this pull request Nov 18, 2019

Tina heiligers backport/7.x/pr 50444 #50974

Merged

TinaHeiligers added a commit that referenced this pull request Nov 18, 2019

Tina heiligers backport/7.x/pr 50444 (#50974)

6553d13

* [NP Kibana Migrations ] kibana plugin home (#50444)

kindsun mentioned this pull request Nov 26, 2019

[Maps] Add basic NP shim to Maps app and expose map config #51683

Merged

TinaHeiligers deleted the NP_kibana_plugin_home branch October 14, 2020 15:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[NP Kibana Migrations ] kibana plugin home #50444

[NP Kibana Migrations ] kibana plugin home #50444

TinaHeiligers commented Nov 12, 2019 •

edited by flash1293

Loading

TinaHeiligers commented Nov 12, 2019

flash1293 commented Nov 13, 2019

TinaHeiligers commented Nov 14, 2019

TinaHeiligers commented Nov 14, 2019

TinaHeiligers commented Nov 14, 2019

TinaHeiligers commented Nov 16, 2019

elasticmachine commented Nov 16, 2019

pgayvallet left a comment

pgayvallet Nov 16, 2019

TinaHeiligers Nov 16, 2019 •

edited

Loading

pgayvallet Nov 17, 2019

elasticmachine commented Nov 16, 2019

elasticmachine commented Nov 16, 2019

pgayvallet left a comment

pgayvallet Nov 17, 2019

pgayvallet Nov 17, 2019

TinaHeiligers Nov 17, 2019 •

edited

Loading

pgayvallet Nov 17, 2019

TinaHeiligers Nov 17, 2019 •

edited

Loading

elasticmachine commented Nov 17, 2019

flash1293 left a comment

eliperelman left a comment

eliperelman Nov 18, 2019

eliperelman Nov 18, 2019

TinaHeiligers Nov 19, 2019

eliperelman Nov 18, 2019

TinaHeiligers Nov 19, 2019

eliperelman Nov 18, 2019

TinaHeiligers Nov 19, 2019

eliperelman Nov 18, 2019

TinaHeiligers Nov 19, 2019

elasticmachine commented Nov 18, 2019

		const baseUrl = getServices().addBasePath('/api/kibana/home/tutorials_LP');
		const baseUrlNP = getServices().addBasePath('/api/kibana/home/tutorials');

		const emptyContext = {};
		const { error } = Joi.validate(specProvider(emptyContext), tutorialSchema);

	tutorials: { ...this.tutorialsRegistry.setup(core) },
	tutorials: this.tutorialsRegistry.setup(core),

	tutorials: { ...this.tutorialsRegistry.start() },
	tutorials: this.tutorialsRegistry.start(),

[NP Kibana Migrations ] kibana plugin home #50444

[NP Kibana Migrations ] kibana plugin home #50444

Conversation

TinaHeiligers commented Nov 12, 2019 • edited by flash1293 Loading

Summary

Checklist

For maintainers

Dev docs

TinaHeiligers commented Nov 12, 2019

flash1293 commented Nov 13, 2019

TinaHeiligers commented Nov 14, 2019

TinaHeiligers commented Nov 14, 2019

TinaHeiligers commented Nov 14, 2019

TinaHeiligers commented Nov 16, 2019

elasticmachine commented Nov 16, 2019

💚 Build Succeeded

pgayvallet left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TinaHeiligers Nov 16, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Nov 16, 2019

💚 Build Succeeded

elasticmachine commented Nov 16, 2019

💚 Build Succeeded

pgayvallet left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TinaHeiligers Nov 17, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TinaHeiligers Nov 17, 2019 • edited Loading

Choose a reason for hiding this comment

elasticmachine commented Nov 17, 2019

💚 Build Succeeded

flash1293 left a comment

Choose a reason for hiding this comment

eliperelman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Nov 18, 2019

💚 Build Succeeded

TinaHeiligers commented Nov 12, 2019 •

edited by flash1293

Loading

TinaHeiligers Nov 16, 2019 •

edited

Loading

TinaHeiligers Nov 17, 2019 •

edited

Loading

TinaHeiligers Nov 17, 2019 •

edited

Loading