[Feature] Match discover look and feel (#1135)

* remove unused files Signed-off-by: Eric Wei <[email protected]> * missing snapshots Signed-off-by: Eric Wei <[email protected]> * remove unused files Signed-off-by: Eric Wei <[email protected]> * move sidebar out of explorer for match look and feel Signed-off-by: Eric <[email protected]> * sidebar panel always with full height Signed-off-by: Eric <[email protected]> * adjust padding Signed-off-by: Eric <[email protected]> * make search components wrappable Signed-off-by: Eric <[email protected]> * adjust noresult/novisualize page Signed-off-by: Eric <[email protected]> * add padding to datagrid Signed-off-by: Eric <[email protected]> * restructured sidebar fields to use exact same spacing/paneling as discover, so that field type icons look the same and field action buttons disappear if not hovered over Signed-off-by: Paul Sebastian <[email protected]> * address datasource persistency issue Signed-off-by: Eric <[email protected]> * code changes preparing for integration with app analytics Signed-off-by: Eric <[email protected]> * resolve page crash and hide datasource for app analytics Signed-off-by: Eric <[email protected]> * saved object supports datasources, query language, and direct query Signed-off-by: Eric <[email protected]> * change saved object icon Signed-off-by: Eric <[email protected]> * bulk delete for saved query Signed-off-by: Eric <[email protected]> * change data grid to separate sort/page query to not interfere with other components Signed-off-by: Paul Sebastian <[email protected]> * basic drag and drop added Signed-off-by: Paul Sebastian <[email protected]> * viz config changes Signed-off-by: Paul Sebastian <[email protected]> * add margin to resizable Signed-off-by: Paul Sebastian <[email protected]> * wrap countdistribution with panel Signed-off-by: Eric <[email protected]> * pr requested changes Signed-off-by: Paul Sebastian <[email protected]> * remove panel wrapper for inner countdistribution Signed-off-by: Eric <[email protected]> * change datasource selector to only showopensearch/ default cluste Signed-off-by: Eric <[email protected]> * remove unused files Signed-off-by: Eric Wei <[email protected]> * missing snapshots Signed-off-by: Eric Wei <[email protected]> * remove unused files Signed-off-by: Eric Wei <[email protected]> * url datasource data prsing and clean up Signed-off-by: Eric <[email protected]> * minor format Signed-off-by: Eric <[email protected]> * query bar issue when loading direct query saved object Signed-off-by: Eric <[email protected]> * minor cleanup Signed-off-by: Eric <[email protected]> * update snapshots Signed-off-by: Eric <[email protected]> * fix minor query bar issue and add types to searchMeta slice Signed-off-by: Eric <[email protected]> * stop polling when load from saved object and few related minor changes Signed-off-by: Eric <[email protected]> * fix polling cleanup issue Signed-off-by: Eric <[email protected]> * save will not trigger reload Signed-off-by: Eric <[email protected]> * add language enum Signed-off-by: Eric <[email protected]> * direct query status indicator Signed-off-by: Paul Sebastian <[email protected]> * turn default timestamp/pattern mark to badge and on hover Signed-off-by: Paul Sebastian <[email protected]> * flyout resize button margin fix Signed-off-by: Paul Sebastian <[email protected]> * direct query data grid fixes Signed-off-by: Paul Sebastian <[email protected]> * pr changes and added test Signed-off-by: Paul Sebastian <[email protected]> * removed console logs Signed-off-by: Paul Sebastian <[email protected]> * searchMetaData type change Signed-off-by: Paul Sebastian <[email protected]> * use datasource variable Signed-off-by: Paul Sebastian <[email protected]> * fixed types and reduced redundancy Signed-off-by: Paul Sebastian <[email protected]> * fixed when data grix shows Signed-off-by: Paul Sebastian <[email protected]> * simplify condition checking for datagrid Signed-off-by: Eric <[email protected]> * update snapshot Signed-off-by: Eric <[email protected]> --------- Signed-off-by: Eric Wei <[email protected]> Signed-off-by: Eric <[email protected]> Signed-off-by: Paul Sebastian <[email protected]> Co-authored-by: Paul Sebastian <[email protected]> (cherry picked from commit ea2fb2c) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-project · Oct 25, 2023 · f519720 · f519720
1 parent e81cc9c
commit f519720
Show file tree

Hide file tree

Showing 56 changed files with 10,687 additions and 12,030 deletions.
diff --git a/common/constants/data_sources.ts b/common/constants/data_sources.ts
@@ -0,0 +1,16 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+export const DATA_SOURCE_NAME_URL_PARAM_KEY = 'datasourceName';
+export const DATA_SOURCE_TYPE_URL_PARAM_KEY = 'datasourceType';
+export const DEFAULT_DATA_SOURCE_TYPE = 'DEFAULT_INDEX_PATTERNS';
+export const DEFAULT_DATA_SOURCE_NAME = 'Default cluster';
+export const DEFAULT_DATA_SOURCE_OBSERVABILITY_DISPLAY_NAME = 'OpenSearch';
+export const DEFAULT_DATA_SOURCE_TYPE_NAME = 'Default Group';
+export const enum QUERY_LANGUAGE {
+  PPL = 'PPL',
+  SQL = 'SQL',
+  DQL = 'DQL',
+}
diff --git a/common/constants/shared.ts b/common/constants/shared.ts
@@ -242,6 +242,7 @@ export const WAITING_TIME_ON_USER_ACTIONS = 300;
 export const VISUALIZATION_ERROR = {
   NO_DATA: 'No data found.',
   INVALID_DATA: 'Invalid visualization data',
+  NO_SERIES: 'Add a field to start',
 };
 
 export const S3_DATASOURCE_TYPE = 'S3_DATASOURCE';
diff --git a/common/types/explorer.ts b/common/types/explorer.ts
@@ -38,6 +38,7 @@ import {
   SavedObjectsStart,
 } from '../../../../src/core/public/saved_objects';
 import { ChromeBreadcrumb } from '../../../../src/core/public/chrome';
+import { DataSourceType } from '../../../../src/plugins/data/public';
 
 export interface IQueryTab {
   id: string;
@@ -145,13 +146,23 @@ export interface IExplorerProps {
   queryManager?: QueryManager;
 }
 
-export interface SavedQuery {
+export interface SelectedDataSource {
+  label: string;
+  name: string;
+  value: string;
+  type: string;
+  ds?: DataSourceType;
+}
+
+export interface SavedQuery extends SavedObjectAttributes {
   description: string;
   name: string;
   query: string;
   selected_date_range: { start: string; end: string; text: string };
   selected_fields: { text: string; tokens: IField[] };
   selected_timestamp: IField;
+  dataSources: string; // list of type SelectedDataSources that is stringified
+  queryLang: string;
 }
 
 export interface SavedVisualization extends SavedObjectAttributes {
@@ -166,6 +177,8 @@ export interface SavedVisualization extends SavedObjectAttributes {
   user_configs?: string;
   units_of_measure?: string;
   application_id?: string;
+  dataSources: string; // list of type SelectedDataSources that is stringified
+  queryLang: string;
 }
 
 export interface ExplorerDataType {
@@ -406,3 +419,11 @@ export interface GridSortingColumn {
   id: string;
   direction: 'asc' | 'desc';
 }
+
+export enum DirectQueryLoadingStatus {
+  SUCCESS = 'SUCCESS',
+  FAILED = 'FAILED',
+  RUNNING = 'RUNNING',
+  SCHEDULED = 'SCHEDULED',
+  CANCELED = 'CANCELED',
+}
diff --git a/common/types/observability_saved_object_attributes.ts b/common/types/observability_saved_object_attributes.ts
@@ -4,10 +4,14 @@
  */
 
 import { SavedObjectAttributes } from '../../../../src/core/types';
-import { SavedVisualization } from './explorer';
+import { SavedQuery, SavedVisualization } from './explorer';
 
 export const VISUALIZATION_SAVED_OBJECT = 'observability-visualization';
-export const OBSERVABILTY_SAVED_OBJECTS = [VISUALIZATION_SAVED_OBJECT] as const;
+export const SEARCH_SAVED_OBJECT = 'observability-search';
+export const OBSERVABILTY_SAVED_OBJECTS = [
+  VISUALIZATION_SAVED_OBJECT,
+  SEARCH_SAVED_OBJECT,
+] as const;
 export const SAVED_OBJECT_VERSION = 1;
 
 export interface VisualizationSavedObjectAttributes extends SavedObjectAttributes {
@@ -17,3 +21,11 @@ export interface VisualizationSavedObjectAttributes extends SavedObjectAttribute
   createdTimeMs: number;
   savedVisualization: SavedVisualization;
 }
+
+export interface SearchSavedObjectAttributes extends SavedObjectAttributes {
+  title: string;
+  description: string;
+  version: number;
+  createdTimeMs: number;
+  savedQuery: SavedQuery;
+}
diff --git a/common/utils/index.ts b/common/utils/index.ts
@@ -10,6 +10,7 @@ export {
   buildRawQuery,
   composeFinalQuery,
   removeBacktick,
+  getSavingCommonParams,
 } from '../../public/components/common/query_utils';
 
 export * from './core_services';
diff --git a/public/components/application_analytics/components/application.tsx b/public/components/application_analytics/components/application.tsx
@@ -116,6 +116,7 @@ export function Application(props: AppDetailProps) {
     callback,
     queryManager,
     mode,
+    dataSourcePluggables,
   } = props;
   const [application, setApplication] = useState<ApplicationType>({
     id: '',
@@ -371,6 +372,7 @@ export function Application(props: AppDetailProps) {
         callbackInApp={callbackInApp}
         queryManager={queryManager}
         curSelectedTabId={selectedTabId}
+        dataSourcePluggables={dataSourcePluggables}
       />
     );
   };

diff --git a/public/components/application_analytics/home.tsx b/public/components/application_analytics/home.tsx
@@ -73,6 +73,7 @@ export const Home = (props: HomeProps) => {
     chrome,
     notifications,
     queryManager,
+    dataSourcePluggables,
   } = props;
   const [triggerSwitchToEvent, setTriggerSwitchToEvent] = useState(0);
   const dispatch = useDispatch();
@@ -140,6 +141,7 @@ export const Home = (props: HomeProps) => {
     setEndTime,
     mode: 'data_prepper',
     dataPrepperIndicesExist: indicesExist,
+    dataSourcePluggables,
   };
 
   const setToast = (title: string, color = 'success', text?: ReactChild) => {

diff --git a/public/components/common/field_icon/field_icon.tsx b/public/components/common/field_icon/field_icon.tsx
@@ -50,7 +50,7 @@ export const typeToEuiIconMap: Partial<Record<string, EuiTokenProps>> = {
 export function FieldIcon({
   type,
   label,
-  size = 'l',
+  size = 's',
   scripted,
   className,
   ...rest

diff --git a/public/components/common/query_utils/index.ts b/public/components/common/query_utils/index.ts
@@ -6,17 +6,20 @@
 import dateMath from '@elastic/datemath';
 import { Moment } from 'moment-timezone';
 import { isEmpty } from 'lodash';
-import moment from 'moment';
+import { SearchMetaData } from 'public/components/event_analytics/redux/slices/search_meta_data_slice';
 import {
-  DATE_PICKER_FORMAT,
   PPL_DEFAULT_PATTERN_REGEX_FILETER,
+  SELECTED_DATE_RANGE,
+  SELECTED_FIELDS,
+  SELECTED_TIMESTAMP,
 } from '../../../../common/constants/explorer';
 import {
   PPL_DATE_FORMAT,
   PPL_INDEX_INSERT_POINT_REGEX,
   PPL_INDEX_REGEX,
   PPL_NEWLINE_REGEX,
 } from '../../../../common/constants/shared';
+import { IExplorerFields, IQuery } from '../../../../common/types/explorer';
 
 /*
  * "Query Utils" This file contains different reused functions in operational panels
@@ -186,9 +189,6 @@ export const preprocessQuery = ({
 
   if (!start || !end) return finalQuery;
 
-  const formattedStart = moment(start).utc().format(DATE_PICKER_FORMAT);
-  const formattedEnd = moment(end).utc().format(DATE_PICKER_FORMAT);
-
   const promQLTokens = parsePromQLIntoKeywords(rawQuery);
 
   if (promQLTokens?.connection) {
@@ -245,24 +245,12 @@ export const buildPatternsQuery = (
   return finalQuery;
 };
 
-export const buildQuery = (baseQuery: string, currQuery: string) => {
-  let fullQuery: string;
-  if (baseQuery) {
-    fullQuery = baseQuery;
-    if (currQuery) {
-      fullQuery += '| ' + currQuery;
-    }
-  } else {
-    fullQuery = currQuery;
-  }
-  return fullQuery;
-};
+export const buildQuery = (baseQuery: string, currQuery: string) => baseQuery + '| ' + currQuery;
 
-export const buildRawQuery = (query: any, appBaseQuery: string) => {
-  const rawQueryStr = (query.rawQuery as string).includes(appBaseQuery)
-    ? query.rawQuery
-    : buildQuery(appBaseQuery, query.rawQuery);
-  return rawQueryStr;
+export const buildRawQuery = (query: IQuery, appBaseQuery: string) => {
+  if (appBaseQuery && !query.rawQuery.includes(appBaseQuery))
+    return buildQuery(appBaseQuery, query.rawQuery);
+  return query.rawQuery;
 };
 
 export const composeFinalQuery = (
@@ -294,3 +282,28 @@ export const removeBacktick = (stringContainsBacktick: string) => {
   if (!stringContainsBacktick) return '';
   return stringContainsBacktick.replace(/`/g, '');
 };
+
+export const getSavingCommonParams = (
+  queryState: IQuery,
+  appBaseQuery: string,
+  fields: IExplorerFields,
+  savingTitle: string,
+  explorerSearchMeta: SearchMetaData
+) => {
+  return {
+    dataSources: JSON.stringify([
+      {
+        name: explorerSearchMeta.datasources?.[0]?.name || '',
+        type: explorerSearchMeta.datasources?.[0]?.type || '',
+        label: explorerSearchMeta.datasources?.[0]?.label || '',
+        value: explorerSearchMeta.datasources?.[0]?.value || '',
+      },
+    ]),
+    queryLang: explorerSearchMeta.lang,
+    query: buildRawQuery(queryState, appBaseQuery),
+    fields: fields[SELECTED_FIELDS],
+    dateRange: queryState[SELECTED_DATE_RANGE],
+    name: savingTitle,
+    timestamp: queryState[SELECTED_TIMESTAMP],
+  };
+};
diff --git a/public/components/common/search/search.tsx b/public/components/common/search/search.tsx
@@ -20,24 +20,21 @@ import {
 } from '@elastic/eui';
 import { isEqual } from 'lodash';
 import React, { useEffect, useState } from 'react';
-import { useDispatch, useSelector } from 'react-redux';
-import { APP_ANALYTICS_TAB_ID_REGEX, RAW_QUERY } from '../../../../common/constants/explorer';
+import { useDispatch } from 'react-redux';
+import { APP_ANALYTICS_TAB_ID_REGEX } from '../../../../common/constants/explorer';
 import { PPL_SPAN_REGEX } from '../../../../common/constants/shared';
 import { uiSettingsService } from '../../../../common/utils';
 import { useFetchEvents } from '../../../components/event_analytics/hooks';
-import { changeQuery } from '../../../components/event_analytics/redux/slices/query_slice';
 import { usePolling } from '../../../components/hooks/use_polling';
 import { coreRefs } from '../../../framework/core_refs';
 import { SQLService } from '../../../services/requests/sql';
 import { SavePanel } from '../../event_analytics/explorer/save_panel';
-import {
-  selectSearchMetaData,
-  update as updateSearchMetaData,
-} from '../../event_analytics/redux/slices/search_meta_data_slice';
+import { update as updateSearchMetaData } from '../../event_analytics/redux/slices/search_meta_data_slice';
 import { PPLReferenceFlyout } from '../helpers';
 import { LiveTailButton, StopLiveButton } from '../live_tail/live_tail_button';
 import { Autocomplete } from './autocomplete';
 import { DatePicker } from './date_picker';
+import { QUERY_LANGUAGE } from '../../../../common/constants/data_sources';
 export interface IQueryBarProps {
   query: string;
   tempQuery: string;
@@ -97,14 +94,12 @@ export const Search = (props: any) => {
     setIsQueryRunning,
   } = props;
 
-  const explorerSearchMetadata = useSelector(selectSearchMetaData)[tabId];
   const dispatch = useDispatch();
   const appLogEvents = tabId.match(APP_ANALYTICS_TAB_ID_REGEX);
   const [isSavePanelOpen, setIsSavePanelOpen] = useState(false);
   const [isLanguagePopoverOpen, setLanguagePopoverOpen] = useState(false);
   const [isFlyoutVisible, setIsFlyoutVisible] = useState(false);
-  const [queryLang, setQueryLang] = useState('PPL');
-  const [jobId, setJobId] = useState('');
+  const [queryLang, setQueryLang] = useState(QUERY_LANGUAGE.PPL);
   const sqlService = new SQLService(coreRefs.http);
   const { application } = coreRefs;
 
@@ -119,7 +114,7 @@ export const Search = (props: any) => {
   }, 5000);
 
   const requestParams = { tabId };
-  const { getLiveTail, getEvents, getAvailableFields, dispatchOnGettingHis } = useFetchEvents({
+  const { dispatchOnGettingHis } = useFetchEvents({
     pplService: new SQLService(coreRefs.http),
     requestParams,
   });
@@ -163,10 +158,9 @@ export const Search = (props: any) => {
   );
 
   const handleQueryLanguageChange = (lang: string) => {
-    if (lang === 'DQL') {
-      return application!.navigateToUrl(
-        `../app/data-explorer/discover#?_a=(discover:(columns:!(_source),isDirty:!f,sort:!()),metadata:(indexPattern:'${explorerSearchMetadata.datasources[0].value}',view:discover))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_q=(filters:!(),query:(language:kuery,query:''))`
-      );
+    if (lang === QUERY_LANGUAGE.DQL) {
+      application!.navigateToUrl('../app/data-explorer/discover');
+      return;
     }
     dispatch(
       updateSearchMetaData({
@@ -187,10 +181,16 @@ export const Search = (props: any) => {
   };
 
   const languagePopOverItems = [
-    <EuiContextMenuItem key="PPL" onClick={() => handleQueryLanguageChange('PPL')}>
+    <EuiContextMenuItem
+      key={QUERY_LANGUAGE.PPL}
+      onClick={() => handleQueryLanguageChange(QUERY_LANGUAGE.PPL)}
+    >
       PPL
     </EuiContextMenuItem>,
-    <EuiContextMenuItem key="DQL" onClick={() => handleQueryLanguageChange('DQL')}>
+    <EuiContextMenuItem
+      key={QUERY_LANGUAGE.DQL}
+      onClick={() => handleQueryLanguageChange(QUERY_LANGUAGE.DQL)}
+    >
       DQL
     </EuiContextMenuItem>,
   ];
@@ -215,24 +215,9 @@ export const Search = (props: any) => {
     }
   }, [pollingResult, pollingError]);
 
-  useEffect(() => {
-    if (explorerSearchMetadata.datasources?.[0]?.type === 'DEFAULT_INDEX_PATTERNS') {
-      const queryWithSelectedSource = `source = ${explorerSearchMetadata.datasources[0].label}`;
-      handleQueryChange(queryWithSelectedSource);
-      dispatch(
-        changeQuery({
-          tabId,
-          query: {
-            [RAW_QUERY]: queryWithSelectedSource,
-          },
-        })
-      );
-    }
-  }, [explorerSearchMetadata.datasources]);
-
   return (
     <div className="globalQueryBar">
-      <EuiFlexGroup gutterSize="s" justifyContent="flexStart" alignItems="flexStart">
+      <EuiFlexGroup gutterSize="s" justifyContent="flexStart" alignItems="flexStart" wrap>
         {appLogEvents && (
           <EuiFlexItem style={{ minWidth: 110 }} grow={false}>
             <EuiToolTip position="top" content={baseQuery}>
@@ -242,19 +227,21 @@ export const Search = (props: any) => {
             </EuiToolTip>
           </EuiFlexItem>
         )}
-        <EuiFlexItem key="lang-selector" className="search-area lang-selector" grow={false}>
-          <EuiPopover
-            id="smallContextMenuExample"
-            button={languagePopOverButton}
-            isOpen={isLanguagePopoverOpen}
-            closePopover={closeLanguagePopover}
-            panelPaddingSize="none"
-            anchorPosition="downLeft"
-          >
-            <EuiContextMenuPanel size="s" items={languagePopOverItems} />
-          </EuiPopover>
-        </EuiFlexItem>
-        <EuiFlexItem key="search-bar" className="search-area" grow={5}>
+        {!appLogEvents && (
+          <EuiFlexItem key="lang-selector" className="search-area lang-selector" grow={false}>
+            <EuiPopover
+              id="smallContextMenuExample"
+              button={languagePopOverButton}
+              isOpen={isLanguagePopoverOpen}
+              closePopover={closeLanguagePopover}
+              panelPaddingSize="none"
+              anchorPosition="downLeft"
+            >
+              <EuiContextMenuPanel size="s" items={languagePopOverItems} />
+            </EuiPopover>
+          </EuiFlexItem>
+        )}
+        <EuiFlexItem key="search-bar" className="search-area" grow={5} style={{ minWidth: 400 }}>
           <Autocomplete
             key={'autocomplete-search-bar'}
             query={query}