[7.x] Spaces - New Platform Migration, Step 1 (#35429) (#39270)

* crude test updates

* remove custom server typedef

* allow spaces to aquire security plugin after init

* split CoreSetup into CoreSetup and PluginsSetup

* move interfaces to new plugin

* init interceptors in legacy plugin

* fix import

* add placeholder kibana.json

* use NP Elasticsearch service instead of legacy ES Plugin

* cleanup imports

* don't destructure the es client

* introduce request facade

* document reason for getSecurity

* prefer relative imports from src/core

* fix typo in filename: inteceptors --> interceptors

* fix imports; remove stray ts-ignore

* improve typings for spaces client

* rename InterfaceExcept --> Omit

* don't use legacy config in NP

* additional comment

* shim NP config service

* fix merge from master

* revert relative imports into src/core and src/legacy

* shim capabilities modifier into new platform

* removing placeholder kibana.json

* fix prettier problem

* temporary: patch NP 'setUrl'

* migrate onRequest interceptor to NP, without tests

* fix ts error

* testing and deps cleanup for onRequestInterceptor

* replace spaces's usages of request.getBasePath with http.getBasePathFor

* add explicit timeouts for jest interceptor tests

* attempt to fix imports

* use NP logging instead of faked implementation

* revert stray yarn.lock change

* attempt to stablize and fix tests

* update jest config to include src/core/server/mocks

* fix plugin config typings

* add service tests

* fix merge

* allow spaces service to also work with legacy requests

* update interfaces to confirm to new internal/external API convention

* re-enable some post auth interceptor tests

* add explicit timeouts for tests

* prefer modifyUrl instead of manual url modification

* update logger shim to conform to PluginInitializerContext

* remove spaces ConfigClass

* don't weaken type declaration for scoped cluster client calls

* remove legacy server from SpacesCoreSetup

* remove spaces service cache

* remove legacy server as an interceptor dependency

* use modifyUrl on the raw request too

* remove unused import

* cleanup typings

* replace onRequest interceptor with new onPreAuth interceptor

* fix onPostAuth tests

* temporarily copy modifyUrl into spaces plugin

* fix mock export

* fix merge from master

* spaces scopedClient always uses updated ES client and config

* improve typings for usage collector

* rename isLegacyRequest -> isFakeRequest

* use updated NP base path API

* remove commented code

* only expose scoped spaces client

* use OptionalPlugin instead of getSecurity

* update imports of Saved Objects Service to use new src/core/server location

* update core docs

docs/development/core/server/kibana-plugin-server.md

@@ -53,6 +53,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
 |  [SavedObjectsBulkCreateObject](./kibana-plugin-server.savedobjectsbulkcreateobject.md) |  |
 |  [SavedObjectsBulkGetObject](./kibana-plugin-server.savedobjectsbulkgetobject.md) |  |
 |  [SavedObjectsBulkResponse](./kibana-plugin-server.savedobjectsbulkresponse.md) |  |
+|  [SavedObjectsClientWrapperOptions](./kibana-plugin-server.savedobjectsclientwrapperoptions.md) |  |
 |  [SavedObjectsCreateOptions](./kibana-plugin-server.savedobjectscreateoptions.md) |  |
 |  [SavedObjectsFindOptions](./kibana-plugin-server.savedobjectsfindoptions.md) |  |
 |  [SavedObjectsFindResponse](./kibana-plugin-server.savedobjectsfindresponse.md) |  |
@@ -81,4 +82,5 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
 |  [RecursiveReadonly](./kibana-plugin-server.recursivereadonly.md) |  |
 |  [RouteMethod](./kibana-plugin-server.routemethod.md) | The set of common HTTP methods supported by Kibana routing. |
 |  [SavedObjectsClientContract](./kibana-plugin-server.savedobjectsclientcontract.md) | \#\# SavedObjectsClient errors<!-- -->Since the SavedObjectsClient has its hands in everything we are a little paranoid about the way we present errors back to to application code. Ideally, all errors will be either:<!-- -->1. Caused by bad implementation (ie. undefined is not a function) and as such unpredictable 2. An error that has been classified and decorated appropriately by the decorators in [SavedObjectsErrorHelpers](./kibana-plugin-server.savedobjectserrorhelpers.md)<!-- -->Type 1 errors are inevitable, but since all expected/handle-able errors should be Type 2 the <code>isXYZError()</code> helpers exposed at <code>SavedObjectsErrorHelpers</code> should be used to understand and manage error responses from the <code>SavedObjectsClient</code>.<!-- -->Type 2 errors are decorated versions of the source error, so if the elasticsearch client threw an error it will be decorated based on its type. That means that rather than looking for <code>error.body.error.type</code> or doing substring checks on <code>error.body.error.reason</code>, just use the helpers to understand the meaning of the error:<!-- -->\`\`\`<!-- -->js if (SavedObjectsErrorHelpers.isNotFoundError(error)) { // handle 404 }<!-- -->if (SavedObjectsErrorHelpers.isNotAuthorizedError(error)) { // 401 handling should be automatic, but in case you wanted to know }<!-- -->// always rethrow the error unless you handle it throw error; \`\`\`<!-- -->\#\#\# 404s from missing index<!-- -->From the perspective of application code and APIs the SavedObjectsClient is a black box that persists objects. One of the internal details that users have no control over is that we use an elasticsearch index for persistance and that index might be missing.<!-- -->At the time of writing we are in the process of transitioning away from the operating assumption that the SavedObjects index is always available. Part of this transition is handling errors resulting from an index missing. These used to trigger a 500 error in most cases, and in others cause 404s with different error messages.<!-- -->From my (Spencer) perspective, a 404 from the SavedObjectsApi is a 404; The object the request/call was targeting could not be found. This is why \#14141 takes special care to ensure that 404 errors are generic and don't distinguish between index missing or document missing.<!-- -->\#\#\# 503s from missing index<!-- -->Unlike all other methods, create requests are supposed to succeed even when the Kibana index does not exist because it will be automatically created by elasticsearch. When that is not the case it is because Elasticsearch's <code>action.auto_create_index</code> setting prevents it from being created automatically so we throw a special 503 with the intention of informing the user that their Elasticsearch settings need to be updated.<!-- -->See [SavedObjectsErrorHelpers](./kibana-plugin-server.savedobjectserrorhelpers.md) |
+|  [SavedObjectsClientWrapperFactory](./kibana-plugin-server.savedobjectsclientwrapperfactory.md) |  |
 

docs/development/core/server/kibana-plugin-server.savedobjectsclientwrapperfactory.md

@@ -0,0 +1,11 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-server](./kibana-plugin-server.md) &gt; [SavedObjectsClientWrapperFactory](./kibana-plugin-server.savedobjectsclientwrapperfactory.md)
+
+## SavedObjectsClientWrapperFactory type
+
+<b>Signature:</b>
+
+```typescript
+export declare type SavedObjectsClientWrapperFactory<Request = unknown> = (options: SavedObjectsClientWrapperOptions<Request>) => SavedObjectsClientContract;
+```

docs/development/core/server/kibana-plugin-server.savedobjectsclientwrapperoptions.client.md

@@ -0,0 +1,11 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-server](./kibana-plugin-server.md) &gt; [SavedObjectsClientWrapperOptions](./kibana-plugin-server.savedobjectsclientwrapperoptions.md) &gt; [client](./kibana-plugin-server.savedobjectsclientwrapperoptions.client.md)
+
+## SavedObjectsClientWrapperOptions.client property
+
+<b>Signature:</b>
+
+```typescript
+client: SavedObjectsClientContract;
+```

docs/development/core/server/kibana-plugin-server.savedobjectsclientwrapperoptions.md

@@ -0,0 +1,19 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-server](./kibana-plugin-server.md) &gt; [SavedObjectsClientWrapperOptions](./kibana-plugin-server.savedobjectsclientwrapperoptions.md)
+
+## SavedObjectsClientWrapperOptions interface
+
+<b>Signature:</b>
+
+```typescript
+export interface SavedObjectsClientWrapperOptions<Request = unknown> 
+```
+
+## Properties
+
+|  Property | Type | Description |
+|  --- | --- | --- |
+|  [client](./kibana-plugin-server.savedobjectsclientwrapperoptions.client.md) | <code>SavedObjectsClientContract</code> |  |
+|  [request](./kibana-plugin-server.savedobjectsclientwrapperoptions.request.md) | <code>Request</code> |  |
+

docs/development/core/server/kibana-plugin-server.savedobjectsclientwrapperoptions.request.md

@@ -0,0 +1,11 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-server](./kibana-plugin-server.md) &gt; [SavedObjectsClientWrapperOptions](./kibana-plugin-server.savedobjectsclientwrapperoptions.md) &gt; [request](./kibana-plugin-server.savedobjectsclientwrapperoptions.request.md)
+
+## SavedObjectsClientWrapperOptions.request property
+
+<b>Signature:</b>
+
+```typescript
+request: Request;
+```

src/core/server/elasticsearch/scoped_cluster_client.ts

@@ -43,7 +43,10 @@ export class ScopedClusterClient {
     private readonly internalAPICaller: APICaller,
     private readonly scopedAPICaller: APICaller,
     private readonly headers?: Headers
-  ) {}
+  ) {
+    this.callAsCurrentUser = this.callAsCurrentUser.bind(this);
+    this.callAsInternalUser = this.callAsInternalUser.bind(this);
+  }
 
   /**
    * Calls specified `endpoint` with provided `clientParams` on behalf of the

src/core/server/index.ts

@@ -91,6 +91,8 @@ export {
   SavedObjectsClient,
   SavedObjectsClientContract,
   SavedObjectsCreateOptions,
+  SavedObjectsClientWrapperFactory,
+  SavedObjectsClientWrapperOptions,
   SavedObjectsErrorHelpers,
   SavedObjectsFindOptions,
   SavedObjectsFindResponse,

src/core/server/saved_objects/service/index.ts

@@ -38,6 +38,7 @@ export {
   SavedObjectsRepository,
   ScopedSavedObjectsClientProvider,
   SavedObjectsClientWrapperFactory,
+  SavedObjectsClientWrapperOptions,
   SavedObjectsErrorHelpers,
 } from './lib';
 

src/core/server/server.api.md

@@ -497,6 +497,21 @@ export class SavedObjectsClient {
 // @public
 export type SavedObjectsClientContract = Pick<SavedObjectsClient, keyof SavedObjectsClient>;
 
+// Warning: (ae-missing-release-tag) "SavedObjectsClientWrapperFactory" is exported by the package, but it is missing a release tag (@alpha, @beta, @public, or @internal)
+// 
+// @public (undocumented)
+export type SavedObjectsClientWrapperFactory<Request = unknown> = (options: SavedObjectsClientWrapperOptions<Request>) => SavedObjectsClientContract;
+
+// Warning: (ae-missing-release-tag) "SavedObjectsClientWrapperOptions" is exported by the package, but it is missing a release tag (@alpha, @beta, @public, or @internal)
+// 
+// @public (undocumented)
+export interface SavedObjectsClientWrapperOptions<Request = unknown> {
+    // (undocumented)
+    client: SavedObjectsClientContract;
+    // (undocumented)
+    request: Request;
+}
+
 // @public (undocumented)
 export interface SavedObjectsCreateOptions extends SavedObjectsBaseOptions {
     id?: string;

src/legacy/server/http/setup_base_path_provider.js

@@ -18,11 +18,6 @@
  */
 
 export function setupBasePathProvider(kbnServer) {
-  kbnServer.server.decorate('request', 'setBasePath', function (basePath) {
-    const request = this;
-    kbnServer.newPlatform.setup.core.http.basePath.set(request, basePath);
-  });
-
   kbnServer.server.decorate('request', 'getBasePath', function () {
     const request = this;
     return kbnServer.newPlatform.setup.core.http.basePath.get(request);

x-pack/plugins/security/index.js

@@ -33,7 +33,7 @@ import {
 import { watchStatusAndLicenseToInitialize } from '../../server/lib/watch_status_and_license_to_initialize';
 import { SecureSavedObjectsClientWrapper } from './server/lib/saved_objects_client/secure_saved_objects_client_wrapper';
 import { deepFreeze } from './server/lib/deep_freeze';
-import { createOptionalPlugin } from './server/lib/optional_plugin';
+import { createOptionalPlugin } from '../../server/lib/optional_plugin';
 
 export const security = (kibana) => new kibana.Plugin({
   id: 'security',

x-pack/plugins/security/server/lib/authorization/check_privileges_dynamically.test.ts

@@ -5,7 +5,7 @@
  */
 
 import { SpacesPlugin } from '../../../../spaces/types';
-import { OptionalPlugin } from '../optional_plugin';
+import { OptionalPlugin } from '../../../../../server/lib/optional_plugin';
 import { checkPrivilegesDynamicallyWithRequestFactory } from './check_privileges_dynamically';
 
 test(`checkPrivileges.atSpace when spaces is enabled`, async () => {

x-pack/plugins/security/server/lib/authorization/check_privileges_dynamically.ts

@@ -13,7 +13,7 @@ import { CheckPrivilegesAtResourceResponse, CheckPrivilegesWithRequest } from '.
  */
 
 import { SpacesPlugin } from '../../../../spaces/types';
-import { OptionalPlugin } from '../optional_plugin';
+import { OptionalPlugin } from '../../../../../server/lib/optional_plugin';
 
 export type CheckPrivilegesDynamically = (
   privilegeOrPrivileges: string | string[]

x-pack/plugins/security/server/lib/authorization/service.ts

@@ -10,7 +10,7 @@ import { getClient } from '../../../../../server/lib/get_client_shield';
 import { SpacesPlugin } from '../../../../spaces/types';
 import { XPackFeature, XPackMainPlugin } from '../../../../xpack_main/xpack_main';
 import { APPLICATION_PREFIX } from '../../../common/constants';
-import { OptionalPlugin } from '../optional_plugin';
+import { OptionalPlugin } from '../../../../../server/lib/optional_plugin';
 import { Actions, actionsFactory } from './actions';
 import { CheckPrivilegesWithRequest, checkPrivilegesWithRequestFactory } from './check_privileges';
 import {

x-pack/plugins/security/server/routes/api/external/privileges/get.test.ts

@@ -7,6 +7,7 @@ import Boom from 'boom';
 import { Server } from 'hapi';
 import { RawKibanaPrivileges } from '../../../../../common/model';
 import { initGetPrivilegesApi } from './get';
+import { AuthorizationService } from '../../../../lib/authorization/service';
 
 const createRawKibanaPrivileges: () => RawKibanaPrivileges = () => {
   return {
@@ -37,13 +38,13 @@ const createMockServer = () => {
   const mockServer = new Server({ debug: false, port: 8080 });
 
   mockServer.plugins.security = {
-    authorization: {
+    authorization: ({
       privileges: {
         get: jest.fn().mockImplementation(() => {
           return createRawKibanaPrivileges();
         }),
       },
-    },
+    } as unknown) as AuthorizationService,
   } as any;
   return mockServer;
 };

x-pack/plugins/spaces/index.ts

@@ -4,32 +4,25 @@
  * you may not use this file except in compliance with the Elastic License.
  */
 
+import * as Rx from 'rxjs';
 import { resolve } from 'path';
-
-import { SavedObjectsService } from 'src/core/server';
-import { Request, Server } from 'hapi';
+import KbnServer, { Server } from 'src/legacy/server/kbn_server';
+import { createOptionalPlugin } from '../../server/lib/optional_plugin';
 // @ts-ignore
 import { AuditLogger } from '../../server/lib/audit_logger';
-// @ts-ignore
-import { watchStatusAndLicenseToInitialize } from '../../server/lib/watch_status_and_license_to_initialize';
 import mappings from './mappings.json';
-import { SpacesAuditLogger } from './server/lib/audit_logger';
-import { checkLicense } from './server/lib/check_license';
-import { createDefaultSpace } from './server/lib/create_default_space';
-import { createSpacesService } from './server/lib/create_spaces_service';
 import { wrapError } from './server/lib/errors';
 import { getActiveSpace } from './server/lib/get_active_space';
 import { getSpaceSelectorUrl } from './server/lib/get_space_selector_url';
-import { getSpacesUsageCollector } from './server/lib/get_spaces_usage_collector';
 import { migrateToKibana660 } from './server/lib/migrations';
-import { initSpacesRequestInterceptors } from './server/lib/request_inteceptors';
-import { spacesSavedObjectsClientWrapperFactory } from './server/lib/saved_objects_client/saved_objects_client_wrapper_factory';
-import { SpacesClient } from './server/lib/spaces_client';
-import { createSpacesTutorialContextFactory } from './server/lib/spaces_tutorial_context_factory';
-import { toggleUICapabilities } from './server/lib/toggle_ui_capabilities';
-import { initExternalSpacesApi } from './server/routes/api/external';
-import { initInternalApis } from './server/routes/api/v1';
-
+import { plugin } from './server/new_platform';
+import {
+  SpacesInitializerContext,
+  SpacesCoreSetup,
+  SpacesHttpServiceSetup,
+} from './server/new_platform/plugin';
+import { initSpacesRequestInterceptors } from './server/lib/request_interceptors';
+import { SecurityPlugin } from '../security';
 export const spaces = (kibana: Record<string, any>) =>
   new kibana.Plugin({
     id: 'spaces',
@@ -95,7 +88,7 @@ export const spaces = (kibana: Record<string, any>) =>
         request: Record<string, any>,
         server: Record<string, any>
       ) {
-        const spacesClient = server.plugins.spaces.spacesClient.getScopedClient(request);
+        const spacesClient = await server.plugins.spaces.getScopedSpacesClient(request);
         try {
           vars.activeSpace = {
             valid: true,
@@ -117,86 +110,77 @@ export const spaces = (kibana: Record<string, any>) =>
     },
 
     async init(server: Server) {
-      const thisPlugin = this;
-      const xpackMainPlugin = server.plugins.xpack_main;
-
-      watchStatusAndLicenseToInitialize(xpackMainPlugin, thisPlugin, async () => {
-        await createDefaultSpace(server);
-      });
+      const kbnServer = (server as unknown) as KbnServer;
+      const initializerContext = ({
+        legacyConfig: server.config(),
+        config: {
+          create: () => {
+            return Rx.of({
+              maxSpaces: server.config().get('xpack.spaces.maxSpaces'),
+            });
+          },
+        },
+        logger: {
+          get(...contextParts: string[]) {
+            return kbnServer.newPlatform.coreContext.logger.get(
+              'plugins',
+              'spaces',
+              ...contextParts
+            );
+          },
+        },
+      } as unknown) as SpacesInitializerContext;
 
-      // Register a function that is called whenever the xpack info changes,
-      // to re-compute the license check results for this plugin.
-      xpackMainPlugin.info
-        .feature(thisPlugin.id)
-        .registerLicenseCheckResultsGenerator(checkLicense);
+      const spacesHttpService: SpacesHttpServiceSetup = {
+        ...kbnServer.newPlatform.setup.core.http,
+        route: server.route.bind(server),
+      };
 
-      const spacesService = createSpacesService(server);
-      server.expose('getSpaceId', (request: any) => spacesService.getSpaceId(request));
+      const core: SpacesCoreSetup = {
+        http: spacesHttpService,
+        elasticsearch: kbnServer.newPlatform.setup.core.elasticsearch,
+        savedObjects: server.savedObjects,
+        usage: server.usage,
+        tutorial: {
+          addScopedTutorialContextFactory: server.addScopedTutorialContextFactory,
+        },
+        capabilities: {
+          registerCapabilitiesModifier: server.registerCapabilitiesModifier,
+        },
+        auditLogger: {
+          create: (pluginId: string) =>
+            new AuditLogger(server, pluginId, server.config(), server.plugins.xpack_main.info),
+        },
+      };
 
-      const config = server.config();
+      const plugins = {
+        xpackMain: server.plugins.xpack_main,
+        // TODO: Spaces has a circular dependency with Security right now.
+        // Security is not yet available when init runs, so this is wrapped in an optional function for the time being.
+        security: createOptionalPlugin<SecurityPlugin>(
+          server.config(),
+          'xpack.security',
+          server.plugins,
+          'security'
+        ),
+        spaces: this,
+      };
 
-      const spacesAuditLogger = new SpacesAuditLogger(
-        new AuditLogger(server, 'spaces', config, xpackMainPlugin.info)
-      );
+      const { spacesService, log } = await plugin(initializerContext).setup(core, plugins);
 
-      server.expose('spacesClient', {
-        getScopedClient: (request: Request) => {
-          const adminCluster = server.plugins.elasticsearch.getCluster('admin');
-          const { callWithRequest, callWithInternalUser } = adminCluster;
-          const callCluster = callWithRequest.bind(adminCluster, request);
-          const { savedObjects } = server;
-          const internalRepository = savedObjects.getSavedObjectsRepository(callWithInternalUser);
-          const callWithRequestRepository = savedObjects.getSavedObjectsRepository(callCluster);
-          const authorization = server.plugins.security
-            ? server.plugins.security.authorization
-            : null;
-          return new SpacesClient(
-            spacesAuditLogger,
-            (message: string) => {
-              server.log(['spaces', 'debug'], message);
-            },
-            authorization,
-            callWithRequestRepository,
-            server.config(),
-            internalRepository,
-            request
-          );
+      initSpacesRequestInterceptors({
+        config: initializerContext.legacyConfig,
+        http: core.http,
+        getHiddenUiAppById: server.getHiddenUiAppById,
+        onPostAuth: handler => {
+          server.ext('onPostAuth', handler);
         },
+        log,
+        spacesService,
+        xpackMain: plugins.xpackMain,
       });
 
-      const {
-        addScopedSavedObjectsClientWrapperFactory,
-        types,
-      } = server.savedObjects as SavedObjectsService;
-      addScopedSavedObjectsClientWrapperFactory(
-        Number.MAX_SAFE_INTEGER - 1,
-        spacesSavedObjectsClientWrapperFactory(spacesService, types)
-      );
-
-      server.addScopedTutorialContextFactory(createSpacesTutorialContextFactory(spacesService));
-
-      initInternalApis(server);
-      initExternalSpacesApi(server);
-
-      initSpacesRequestInterceptors(server);
-
-      // Register a function with server to manage the collection of usage stats
-      server.usage.collectorSet.register(getSpacesUsageCollector(server));
-
-      server.registerCapabilitiesModifier(async (request, uiCapabilities) => {
-        const spacesClient = server.plugins.spaces.spacesClient.getScopedClient(request);
-        try {
-          const activeSpace = await getActiveSpace(
-            spacesClient,
-            request.getBasePath(),
-            server.config().get('server.basePath')
-          );
-
-          const features = server.plugins.xpack_main.getFeatures();
-          return toggleUICapabilities(features, uiCapabilities, activeSpace);
-        } catch (e) {
-          return uiCapabilities;
-        }
-      });
+      server.expose('getSpaceId', (request: any) => spacesService.getSpaceId(request));
+      server.expose('getScopedSpacesClient', spacesService.scopedClient);
     },
   });