[Drift] Enable chat globally + A/B test for pages where the chat was …

…available before (elastic#167069)

## Summary

Close elastic#159691

[Requirements](https://docs.google.com/document/d/1uXgyDIGuIqkYXmavdMTpBgQEiOP07ObJYb7GNq5GiSE/edit#heading=h.okl11rz12ytg)
[A/B test
description](https://docs.google.com/document/d/1yzfZF8mtlRNH4X6HjosD6Exh24zAhN__LR_zh3DOzKw/edit?usp=sharing)

[Figma](https://www.figma.com/file/WGhmfgyy9FBOltLtycfGPE/Getting-started?type=design&node-id=92-44804&mode=design&t=mwbIexn5Fs754HQz-0)
Testing - see _testing_ section below for more details
(https://dosant-pr-167374-d-2023-06-14-global-drift-with-experiment.kbndev.co/.
elastic/changeme)

This PR enables cloud chat (Drift) globally. This is done by adding a
custom chat button in the Kibana header which manually toggles Drift
widget. We attempt to manually position the widget to the top of the
screen so it pops up close to the chat button that triggered it.

Previously Drift chat was available only on specific pages like
Solutions onboarding pages, integrations, setup guides as a regular chat
widget with the floating chat bubble in the bottom right corner. We
couldn't enable it on all pages, because on a lot of them the floating
chat bottom would have overlapped the application UI.

We also were asked to add [an a/b
test](https://docs.google.com/document/d/1yzfZF8mtlRNH4X6HjosD6Exh24zAhN__LR_zh3DOzKw/edit?usp=sharing):
 - A: The chat button appears in the header for all pages (new)
- B: The chat button appears as floating action button in the
bottom-right corner on pages where Drift was previously available
(solutions onboarding pages, integrations, setup guides)

### Screenshots / Videos

#### Global Chat in the header

![Screenshot 2023-09-25 at 10 30
41](https://github.com/elastic/kibana/assets/7784120/dba3b3da-4e90-4d6c-a5d2-99123aa8c753)
![Screenshot 2023-09-25 at 10 30
45](https://github.com/elastic/kibana/assets/7784120/752d05e4-cc85-458e-8216-f75529c2bac7)

#### The tour on the first appearance 

![Screenshot 2023-09-25 at 10 55
42](https://github.com/elastic/kibana/assets/7784120/c0958095-f724-4b69-a149-04a0aec8e083)

#### (Part of A/B test) Drift in the header on new pages and as floating
action button on old pages


https://github.com/elastic/kibana/assets/7784120/0386ccbd-ab6c-4eb2-a57b-f9324fcf73eb

### Implementation notes

- **We still enable Drift only for trial users + gap window**
- We exposed additional APIs from drift iframe to manually control its
visibility and react to more events
elastic/cloud#118761. This changes are required
for the code in this PR to work. ~The updated frame code wasn't deployed
yet.~ the changes were deployed
- We use [`playbookFired` event
](https://devdocs.drift.com/docs/drift-events#playbook-fired) to know if
Drift chat should be visible for the current user. We show the button in
the header only when it fires.
- To react to the event and to display the button, we have to kick of
Drift iframe initialization first
- This means Drift codes loads before we show the button and before user
interacts with it (Only when Drift is enabled, meaning, only for trial
users + gap window)
- Subsequent launches or opens of the same playbook will not re-trigger
the `playbookFired` event, I used local storage flag to workaround this
and show the live chat button, but it also has it's own edge case. As an
alternative we can always show the chat button and don't rely on the
playbook event, more details here:
https://docs.google.com/document/d/1j313mVOIz19Rkoj8TDFWaLc7Pgk_ByBBKJFNIC-jbyc/edit?usp=sharing.
For now was decided to rely on the `playbookFired` event.
- A/B: **to support for both new and old implementation, I had to
refactor the old one from drop-in chat to global chat that is controlled
by list of hardcoded URLs.** This is not ideal, but this allows two
implementations to co-exist with much tech-debt and we plan to get rid
of this after the a/b test.
- When we navigate between pages with different implementations, Drift
re-initializes itself (just like in old implementation), this
performance debt should go away when we get rid of the a/b test.
- When end-to-end testing the a/b experiment, I found a bug in the a/b
test setup in Kibana elastic#167240
this needs to be addressed separately for a/b test to work properly.
- When the user receives a message from Drift - the custom "Live Chat"
button doesn't indicate that there is a new message. This needs a follow
up, @Dosant to create an issue


### Testing

Version with the experiment where on old pages Drift appears as a
floating chat bubble -
https://dosant-pr-167374-d-2023-06-14-global-drift-with-experiment.kbndev.co/
elastic/changeme

> [!NOTE]  
> If the live chat button doesn't appear, it is likely because the
playbook was recently activated by someone else. you can workaround this
for testing by creating a different user or by setting
`cloudChatPlaybookFiredOnce : true` to localstorage. This issue and
mitigation is described in details in the "implementation details"
section


#### To test locally: 

```
xpack.cloud.id: 'some-id'
xpack.cloud.trial_end_date: '2023-09-21T00:00:00.000Z'
xpack.cloud_integrations.chat.trialBuffer: 45
xpack.cloud.chat.enabled: true

xpack.cloud.chatIdentitySecret: <pls react out> 
xpack.cloud.chat.chatURL: https://elasticcloud-production-chat-us-east-1.s3.amazonaws.com/drift-iframe.html

xpack.cloud_integrations.experiments.flag_overrides:
  "cloud-chat.chat-variant": "bubble" or "header"

```

.github/CODEOWNERS

@@ -69,7 +69,6 @@ packages/kbn-ci-stats-shipper-cli @elastic/kibana-operations
 packages/kbn-cli-dev-mode @elastic/kibana-operations
 packages/cloud @elastic/kibana-core
 x-pack/plugins/cloud_integrations/cloud_chat @elastic/kibana-core
-x-pack/plugins/cloud_integrations/cloud_chat_provider @elastic/kibana-core
 x-pack/plugins/cloud_integrations/cloud_data_migration @elastic/platform-onboarding
 x-pack/plugins/cloud_defend @elastic/kibana-cloud-security-posture
 x-pack/plugins/cloud_integrations/cloud_experiments @elastic/kibana-core

docs/developer/plugin-list.asciidoc

@@ -478,10 +478,6 @@ The plugin exposes the static DefaultEditorController class to consume.
 |Integrates with DriftChat in order to provide live support to our Elastic Cloud users. This plugin should only run on Elastic Cloud.
 
 
-|{kib-repo}blob/{branch}/x-pack/plugins/cloud_integrations/cloud_chat_provider/README.md[cloudChatProvider]
-|This plugin exists as a workaround for using cloudChat plugin in plugins which can't have a direct dependency on security plugin.
-
-
 |{kib-repo}blob/{branch}/x-pack/plugins/cloud_integrations/cloud_data_migration/README.md[cloudDataMigration]
 |Static migration page where self-managed users can see text/copy about migrating to Elastic Cloud
 

package.json

@@ -175,7 +175,6 @@
     "@kbn/charts-plugin": "link:src/plugins/charts",
     "@kbn/cloud": "link:packages/cloud",
     "@kbn/cloud-chat-plugin": "link:x-pack/plugins/cloud_integrations/cloud_chat",
-    "@kbn/cloud-chat-provider-plugin": "link:x-pack/plugins/cloud_integrations/cloud_chat_provider",
     "@kbn/cloud-data-migration-plugin": "link:x-pack/plugins/cloud_integrations/cloud_data_migration",
     "@kbn/cloud-defend-plugin": "link:x-pack/plugins/cloud_defend",
     "@kbn/cloud-experiments-plugin": "link:x-pack/plugins/cloud_integrations/cloud_experiments",

packages/core/application/core-application-browser-internal/src/application_service.test.mocks.ts

@@ -7,6 +7,7 @@
  */
 
 import { capabilitiesServiceMock } from '@kbn/core-capabilities-browser-mocks';
+import { Observable } from 'rxjs';
 
 export const MockCapabilitiesService = capabilitiesServiceMock.create();
 export const CapabilitiesServiceConstructor = jest
@@ -26,7 +27,7 @@ jest.doMock('history', () => ({
 }));
 
 export const parseAppUrlMock = jest.fn();
-export const getLocationObservableMock = jest.fn();
+export const getLocationObservableMock = jest.fn(() => new Observable());
 jest.doMock('./utils', () => {
   const original = jest.requireActual('./utils');
 

packages/core/application/core-application-browser-internal/src/application_service.tsx

@@ -112,6 +112,7 @@ export class ApplicationService {
   private stop$ = new Subject<void>();
   private registrationClosed = false;
   private history?: History<any>;
+  private location$?: Observable<string>;
   private navigate?: (url: string, state: unknown, replace: boolean) => void;
   private openInNewTab?: (url: string) => void;
   private redirectTo?: (url: string) => void;
@@ -136,10 +137,10 @@ export class ApplicationService {
         }),
       });
 
-    const location$ = getLocationObservable(window.location, this.history);
+    this.location$ = getLocationObservable(window.location, this.history);
     registerAnalyticsContextProvider({
       analytics,
-      location$,
+      location$: this.location$,
     });
 
     this.navigate = (url, state, replace) => {
@@ -311,6 +312,7 @@ export class ApplicationService {
         shareReplay(1)
       ),
       capabilities,
+      currentLocation$: this.location$!.pipe(takeUntil(this.stop$)),
       currentAppId$: this.currentAppId$.pipe(
         filter((appId) => appId !== undefined),
         distinctUntilChanged(),

packages/core/application/core-application-browser-mocks/src/application_service.mock.ts

@@ -40,10 +40,12 @@ const createInternalSetupContractMock = (): jest.Mocked<InternalApplicationSetup
 
 const createStartContractMock = (): jest.Mocked<ApplicationStart> => {
   const currentAppId$ = new Subject<string | undefined>();
+  const currentLocation$ = new Subject<string>();
 
   return {
     applications$: new BehaviorSubject<Map<string, PublicAppInfo>>(new Map()),
     currentAppId$: currentAppId$.asObservable(),
+    currentLocation$: currentLocation$.asObservable(),
     capabilities: capabilitiesServiceMock.createStartContract().capabilities,
     navigateToApp: jest.fn(),
     navigateToUrl: jest.fn(),
@@ -79,11 +81,13 @@ const createInternalStartContractMock = (
   const currentAppId$ = currentAppId
     ? new BehaviorSubject<string | undefined>(currentAppId)
     : new Subject<string | undefined>();
+  const currentLocation$ = new Subject<string>();
 
   return {
     applications$: new BehaviorSubject<Map<string, PublicAppInfo>>(new Map()),
     capabilities: capabilitiesServiceMock.createStartContract().capabilities,
     currentAppId$: currentAppId$.asObservable(),
+    currentLocation$: currentLocation$.asObservable(),
     currentActionMenu$: new BehaviorSubject<MountPoint | undefined>(undefined),
     getComponent: jest.fn(),
     getUrlForApp: jest.fn(),

packages/core/application/core-application-browser/src/contracts.ts

@@ -134,6 +134,11 @@ export interface ApplicationStart {
    * An observable that emits the current application id and each subsequent id update.
    */
   currentAppId$: Observable<string | undefined>;
+
+  /**
+   * An observable that emits the current path#hash and each subsequent update using the global history instance
+   */
+  currentLocation$: Observable<string>;
 }
 
 /**

packages/core/plugins/core-plugins-browser-internal/src/plugin_context.ts

@@ -115,6 +115,7 @@ export function createPluginStartContext<
       navigateToApp: deps.application.navigateToApp,
       navigateToUrl: deps.application.navigateToUrl,
       getUrlForApp: deps.application.getUrlForApp,
+      currentLocation$: deps.application.currentLocation$,
     },
     customBranding: deps.customBranding,
     docLinks: deps.docLinks,

packages/kbn-optimizer/limits.yml

@@ -12,7 +12,6 @@ pageLoadAssetSize:
   charts: 55000
   cloud: 21076
   cloudChat: 19894
-  cloudChatProvider: 17114
   cloudDataMigration: 19170
   cloudDefend: 18697
   cloudExperiments: 59358

src/plugins/discover/public/__mocks__/start_contract.ts

@@ -21,10 +21,12 @@ const capabilities = deepFreeze({
 
 export const createStartContractMock = (): jest.Mocked<ApplicationStart> => {
   const currentAppId$ = new Subject<string | undefined>();
+  const currentLocation$ = new Subject<string>();
 
   return {
     applications$: new BehaviorSubject<Map<string, PublicAppInfo>>(new Map()),
     currentAppId$: currentAppId$.asObservable(),
+    currentLocation$: currentLocation$.asObservable(),
     capabilities,
     navigateToApp: jest.fn(),
     navigateToUrl: jest.fn(),

src/plugins/home/kibana.jsonc

@@ -6,20 +6,8 @@
     "id": "home",
     "server": true,
     "browser": true,
-    "requiredPlugins": [
-      "dataViews",
-      "share",
-      "urlForwarding"
-    ],
-    "optionalPlugins": [
-      "usageCollection",
-      "customIntegrations",
-      "cloud",
-      "guidedOnboarding",
-      "cloudChatProvider"
-    ],
-    "requiredBundles": [
-      "kibanaReact"
-    ]
+    "requiredPlugins": ["dataViews", "share", "urlForwarding"],
+    "optionalPlugins": ["usageCollection", "customIntegrations", "cloud", "guidedOnboarding"],
+    "requiredBundles": ["kibanaReact"]
   }
 }

src/plugins/home/public/application/components/guided_onboarding/getting_started.tsx

@@ -43,8 +43,7 @@ const skipText = i18n.translate('home.guidedOnboarding.gettingStarted.skip.butto
 });
 
 export const GettingStarted = () => {
-  const { application, trackUiMetric, chrome, guidedOnboardingService, cloud, cloudChat } =
-    getServices();
+  const { application, trackUiMetric, chrome, guidedOnboardingService, cloud } = getServices();
 
   const [guidesState, setGuidesState] = useState<GuideState[]>([]);
   const [isLoading, setIsLoading] = useState<boolean>(false);
@@ -227,7 +226,6 @@ export const GettingStarted = () => {
             {skipText}
           </EuiLink>
         </div>
-        {cloudChat?.Chat && <cloudChat.Chat />}
       </EuiPageTemplate.Section>
     </KibanaPageTemplate>
   );

src/plugins/home/public/application/kibana_services.ts

@@ -22,7 +22,6 @@ import { DataViewsContract } from '@kbn/data-views-plugin/public';
 import { SharePluginSetup } from '@kbn/share-plugin/public';
 import { GuidedOnboardingApi } from '@kbn/guided-onboarding-plugin/public';
 import { CloudSetup } from '@kbn/cloud-plugin/public';
-import { CloudChatProviderPluginStart } from '@kbn/cloud-chat-provider-plugin/public';
 import { TutorialService } from '../services/tutorials';
 import { AddDataService } from '../services/add_data';
 import { FeatureCatalogueRegistry } from '../services/feature_catalogue';
@@ -54,7 +53,6 @@ export interface HomeKibanaServices {
   welcomeService: WelcomeService;
   guidedOnboardingService?: GuidedOnboardingApi;
   cloud?: CloudSetup;
-  cloudChat?: CloudChatProviderPluginStart;
 }
 
 let services: HomeKibanaServices | null = null;

src/plugins/home/public/plugin.ts

@@ -22,7 +22,6 @@ import type { GuidedOnboardingPluginStart } from '@kbn/guided-onboarding-plugin/
 import { AppNavLinkStatus } from '@kbn/core/public';
 import { SharePluginSetup } from '@kbn/share-plugin/public';
 import type { CloudSetup } from '@kbn/cloud-plugin/public';
-import type { CloudChatProviderPluginStart } from '@kbn/cloud-chat-provider-plugin/public';
 import { PLUGIN_ID, HOME_APP_BASE_PATH } from '../common/constants';
 import { setServices } from './application/kibana_services';
 import { ConfigSchema } from '../config';
@@ -43,7 +42,6 @@ export interface HomePluginStartDependencies {
   dataViews: DataViewsPublicPluginStart;
   urlForwarding: UrlForwardingStart;
   guidedOnboarding: GuidedOnboardingPluginStart;
-  cloudChatProvider?: CloudChatProviderPluginStart;
 }
 
 export interface HomePluginSetupDependencies {
@@ -82,10 +80,8 @@ export class HomePublicPlugin
         const trackUiMetric = usageCollection
           ? usageCollection.reportUiCounter.bind(usageCollection, 'Kibana_home')
           : () => {};
-        const [
-          coreStart,
-          { dataViews, urlForwarding: urlForwardingStart, guidedOnboarding, cloudChatProvider },
-        ] = await core.getStartServices();
+        const [coreStart, { dataViews, urlForwarding: urlForwardingStart, guidedOnboarding }] =
+          await core.getStartServices();
         setServices({
           share,
           trackUiMetric,
@@ -110,7 +106,6 @@ export class HomePublicPlugin
           welcomeService: this.welcomeService,
           guidedOnboardingService: guidedOnboarding.guidedOnboardingApi,
           cloud,
-          cloudChat: cloudChatProvider,
         });
         coreStart.chrome.docTitle.change(
           i18n.translate('home.pageTitle', { defaultMessage: 'Home' })

src/plugins/home/tsconfig.json

@@ -30,7 +30,6 @@
     "@kbn/ebt-tools",
     "@kbn/core-analytics-server",
     "@kbn/storybook",
-    "@kbn/cloud-chat-provider-plugin",
     "@kbn/shared-ux-router",
     "@kbn/core-http-common",
   ],

tsconfig.base.json

@@ -132,8 +132,6 @@
       "@kbn/cloud/*": ["packages/cloud/*"],
       "@kbn/cloud-chat-plugin": ["x-pack/plugins/cloud_integrations/cloud_chat"],
       "@kbn/cloud-chat-plugin/*": ["x-pack/plugins/cloud_integrations/cloud_chat/*"],
-      "@kbn/cloud-chat-provider-plugin": ["x-pack/plugins/cloud_integrations/cloud_chat_provider"],
-      "@kbn/cloud-chat-provider-plugin/*": ["x-pack/plugins/cloud_integrations/cloud_chat_provider/*"],
       "@kbn/cloud-data-migration-plugin": ["x-pack/plugins/cloud_integrations/cloud_data_migration"],
       "@kbn/cloud-data-migration-plugin/*": ["x-pack/plugins/cloud_integrations/cloud_data_migration/*"],
       "@kbn/cloud-defend-plugin": ["x-pack/plugins/cloud_defend"],

x-pack/plugins/cloud_integrations/cloud_chat/.storybook/decorator.tsx

@@ -13,6 +13,7 @@ import { ServicesProvider, CloudChatServices } from '../public/services';
 const services: CloudChatServices = {
   chat: {
     chatURL: 'https://elasticcloud-production-chat-us-east-1.s3.amazonaws.com/drift-iframe.html',
+    chatVariant: 'bubble',
     user: {
       id: 'user-id',
       email: '[email protected]',

x-pack/plugins/cloud_integrations/cloud_chat/common/types.ts

@@ -5,8 +5,11 @@
  * 2.0.
  */
 
+export type ChatVariant = 'header' | 'bubble';
+
 export interface GetChatUserDataResponseBody {
   token: string;
   email: string;
   id: string;
+  chatVariant: ChatVariant;
 }

x-pack/plugins/cloud_integrations/cloud_chat/kibana.jsonc

@@ -13,11 +13,13 @@
       "chat"
     ],
     "requiredPlugins": [
-      "cloud",
-      "cloudChatProvider"
+      "cloud"
+    ],
+    "requiredBundles": [
     ],
     "optionalPlugins": [
-      "security"
+      "security",
+      "cloudExperiments"
     ]
   }
 }