Skip to content

Commit

Permalink
Expressions refactor (#54342)
Browse files Browse the repository at this point in the history
* feat: ๐ŸŽธ add UiComponent interface

* feat: ๐ŸŽธ add adapter for react => ui components and back

* refactor: ๐Ÿ’ก move registries to shared /common folder

* feat: ๐ŸŽธ create expressions service state contaienr

* chore: ๐Ÿค– export some symbols

* feat: ๐ŸŽธ add Executor class

* test: ๐Ÿ’ add simple integration test

* feat: ๐ŸŽธ move registries into executor

* feat: ๐ŸŽธ add initial implementation of Execution

* feat: ๐ŸŽธ move Executor's state container into a signle file

* refactor: ๐Ÿ’ก move createError() to /common folder

* feat: ๐ŸŽธ use Executor in plugin definition

* refactor: ๐Ÿ’ก rename handlers to FunctionHandlers

* feat: ๐ŸŽธ improve function typings

* feat: ๐ŸŽธ move types and func in sep folder, improve Execution

* refactor: ๐Ÿ’ก cleanup expression_types folder

* refactor: ๐Ÿ’ก improve typing names of expression types

* refactor: ๐Ÿ’ก remove lodash from ExpressionType and improve types

* test: ๐Ÿ’ add ExpressionType tests

* refactor: ๐Ÿ’ก remove function wrappers around types

* refactor: ๐Ÿ’ก move functions to /common

* test: ๐Ÿ’ improve expression function tests

* feat: ๐ŸŽธ create /parser folder

* refactor: ๐Ÿ’ก move function types into /expression_functions dir

* refactor: ๐Ÿ’ก improve parser setup

* refactor: ๐Ÿ’ก fix export structure and move args into expr_func

* test: ๐Ÿ’ add ExpressionFunctionParameter tests

* fix: ๐Ÿ› fix executor types and imports

* refactor: ๐Ÿ’ก move getByAlias to plugin, fix Execution types

* feat: ๐ŸŽธ add function for argument parsing

* test: ๐Ÿ’ add Executor type tests

* test: ๐Ÿ’ add executor function and context tests

* test: ๐Ÿ’ check that Executor returns Execution

* test: ๐Ÿ’ add basic tests for Execution

* test: ๐Ÿ’ add basic test for execution of a chain of functions

* test: ๐Ÿ’ add "mult" function tot tests

* feat: ๐ŸŽธ create separate expression_renderer folder

* feat: ๐ŸŽธ use new executor in public plugin

* feat: ๐ŸŽธ remove renderers from executor, add result to execution

* fix: ๐Ÿ› fix Kibana TypeScript errors

* test: ๐Ÿ’ add file to write integration tests for expr plugin

* refactor: ๐Ÿ’ก move state_containers to /common

* refactor: ๐Ÿ’ก move /parser to /ast and inline format() function

* refactor: ๐Ÿ’ก remove remaining @kbn/interpreter imports

* feat: ๐ŸŽธ better handling and typing for Executor context

* feat: ๐ŸŽธ use Executor.run function in plugin

* fix: ๐Ÿ› fix TypeScript type errors

* test: ๐Ÿ’ move integration tests into one file

* feat: ๐ŸŽธ create ExpressionsService

* chore: ๐Ÿค– clean up legacy code

* feat: ๐ŸŽธ use ExpressionsService in /public

* refactor: ๐Ÿ’ก move inspector adapters to /common

* feat: ๐ŸŽธ improve execution

* feat: ๐ŸŽธ add state to execution state and don't clone AST

* test: ๐Ÿ’ add tests for Execution object

* test: ๐Ÿ’ improve expression test helpers

* test: ๐Ÿ’ add Execution tests

* refactor: ๐Ÿ’ก improve required argument checking

* fix: ๐Ÿ› fix Kibana TypeScript errors

* test: ๐Ÿ’ add ExpressionsService unit tests

* fix: ๐Ÿ› fix Expression plugin TypeScript types

* refactor: ๐Ÿ’ก prefix React component with React*

* fix: ๐Ÿ› fix X-Pack TypeScript errors

* fix: ๐Ÿ› fix test TypeScript errors

* fix: ๐Ÿ› fix issues preventing loading

* feat: ๐ŸŽธ remove getInitialContext() handler

* fix: ๐Ÿ› fix TypeScript errors

* chore: ๐Ÿค– remove uicomponent interface

* chore: ๐Ÿค– remove missing import

* fix: ๐Ÿ› correctly handle .query in "kibana" expression function

* refactor: ๐Ÿ’ก call first arg in expression functions "input"

* fix: ๐Ÿ› do not free Execution state container

* test: ๐Ÿ’ fix tests after refactor

* test: ๐Ÿ’ fix more tests after refactor

* fix: ๐Ÿ› remove redundant export

* test: ๐Ÿ’ update intepreter_functional test shapshots

* fix: ๐Ÿ› relax "kibana" function throwing on missin gsearch ctx

* refactor: ๐Ÿ’ก don't use ExpressionAST interface in Canvas

* docs: โœ๏ธ improve ExpressionRenderer JSDocs

* refactor: ๐Ÿ’ก rename context.types to inputTypes in internal fn

* refactor: ๐Ÿ’ก replace context.types by unknown in ExprFuncDef

* refactor: ๐Ÿ’ก improve expression function definitions in OSS

* fix: ๐Ÿ› correctly set name on metric_vis_fn

* refactor: ๐Ÿ’ก improve Lens definitions of expression functions

* refactor: ๐Ÿ’ก improve Canvas expression function definitions

* test: ๐Ÿ’ add createMockExecutionContext() helper

* refactor: ๐Ÿ’ก add some type to events$ observable in expr handler

* feat: ๐ŸŽธ add types to observables in data handler

* refactor: ๐Ÿ’ก use inputTypes in canvas

* fix: ๐Ÿ› fix interpreter grammer generation script

* feat: ๐ŸŽธ allow array in getByAlias

* test: ๐Ÿ’ simplify test function specs

* test: ๐Ÿ’ fix autocomplete tests

* fix: ๐Ÿ› use correct expression types and NP getFunctions() API

* refactor: ๐Ÿ’ก use NP expressions to get renderer

* fix: ๐Ÿ› use context.types on server-side Canvas function defs

* refactor: ๐Ÿ’ก use NP API to register Canvas renderers

* feat: ๐ŸŽธ use NP API to get types

* style: ๐Ÿ’„ minor formatting changes

* feat: ๐ŸŽธ use NP API to get expression functions

* fix: ๐Ÿ› fix Canvas workpads

* test: ๐Ÿ’ add missing mock functions

* refactor: ๐Ÿ’ก improve Lens func definition argument types

* fix: ๐Ÿ› fix Lens type error

* feat: ๐ŸŽธ make lens datatable work again

* feat: ๐ŸŽธ bootstrap ExpressionsService on server-side

* feat: ๐ŸŽธ expose more registry related functions in contract

* feat: ๐ŸŽธ add environment: server to server-side expressions

* docs: โœ๏ธ add documentation

* test: ๐Ÿ’ add missing Jest mocks

* fix: ๐Ÿ› correct TypeScript type

* docs: โœ๏ธ improve documentation

* fix: ๐Ÿ› make FunctionHelpDict type contain only Canvas functions

* fix: ๐Ÿ› fix merge conflict

* test: ๐Ÿ’ fix expression mocks
  • Loading branch information
streamich authored Feb 11, 2020
1 parent 9fd75db commit 454d0a2
Show file tree
Hide file tree
Showing 304 changed files with 5,272 additions and 3,031 deletions.
2 changes: 1 addition & 1 deletion packages/kbn-interpreter/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"version": "1.0.0",
"license": "Apache-2.0",
"scripts": {
"interpreter:peg": "pegjs common/lib/grammar.peg",
"interpreter:peg": "pegjs src/common/lib/grammar.peg",
"build": "node scripts/build",
"kbn:bootstrap": "node scripts/build --dev",
"kbn:watch": "node scripts/build --dev --watch"
Expand Down
4 changes: 1 addition & 3 deletions packages/kbn-interpreter/src/common/lib/registry.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,7 @@ export class Registry {
}

register(fn) {
if (typeof fn !== 'function') throw new Error(`Register requires an function`);

const obj = fn();
const obj = typeof fn === 'function' ? fn() : fn;

if (typeof obj !== 'object' || !obj[this._prop]) {
throw new Error(`Registered functions must return an object with a ${this._prop} property`);
Expand Down
1 change: 1 addition & 0 deletions packages/kbn-utility-types/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,4 @@ type B = UnwrapPromise<A>; // string
- `ShallowPromise<T>` &mdash; Same as `Promise` type, but it flat maps the wrapped type.
- `UnwrapObservable<T>` &mdash; Returns wrapped type of an observable.
- `UnwrapPromise<T>` &mdash; Returns wrapped type of a promise.
- `UnwrapPromiseOrReturn<T>` &mdash; Returns wrapped type of a promise or the type itself, if it isn't a promise.
5 changes: 5 additions & 0 deletions packages/kbn-utility-types/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@ export type ShallowPromise<T> = T extends Promise<infer U> ? Promise<U> : Promis
*/
export type UnwrapPromise<T extends Promise<any>> = PromiseType<T>;

/**
* Returns wrapped type of a promise, or returns type as is, if it is not a promise.
*/
export type UnwrapPromiseOrReturn<T> = T extends Promise<infer U> ? U : T;

/**
* Minimal interface for an object resembling an `Observable`.
*/
Expand Down
23 changes: 10 additions & 13 deletions src/legacy/core_plugins/data/public/search/expressions/esaggs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import { createFormat } from 'ui/visualize/loader/pipeline_helpers/utilities';
import {
KibanaContext,
KibanaDatatable,
ExpressionFunction,
ExpressionFunctionDefinition,
KibanaDatatableColumn,
} from 'src/plugins/expressions/public';
import {
Expand Down Expand Up @@ -66,7 +66,8 @@ export interface RequestHandlerParams {

const name = 'esaggs';

type Context = KibanaContext | null;
type Input = KibanaContext | null;
type Output = Promise<KibanaDatatable>;

interface Arguments {
index: string;
Expand All @@ -76,8 +77,6 @@ interface Arguments {
aggConfigs: string;
}

type Return = Promise<KibanaDatatable>;

const handleCourierRequest = async ({
searchSource,
aggs,
Expand Down Expand Up @@ -221,12 +220,10 @@ const handleCourierRequest = async ({
return (searchSource as any).tabifiedResponse;
};

export const esaggs = (): ExpressionFunction<typeof name, Context, Arguments, Return> => ({
export const esaggs = (): ExpressionFunctionDefinition<typeof name, Input, Arguments, Output> => ({
name,
type: 'kibana_datatable',
context: {
types: ['kibana_context', 'null'],
},
inputTypes: ['kibana_context', 'null'],
help: i18n.translate('data.functions.esaggs.help', {
defaultMessage: 'Run AggConfig aggregation',
}),
Expand Down Expand Up @@ -256,7 +253,7 @@ export const esaggs = (): ExpressionFunction<typeof name, Context, Arguments, Re
help: '',
},
},
async fn(context, args, { inspectorAdapters, abortSignal }) {
async fn(input, args, { inspectorAdapters, abortSignal }) {
const indexPatterns = getIndexPatterns();
const { filterManager } = getQueryService();

Expand All @@ -272,13 +269,13 @@ export const esaggs = (): ExpressionFunction<typeof name, Context, Arguments, Re
const response = await handleCourierRequest({
searchSource,
aggs,
timeRange: get(context, 'timeRange', undefined),
query: get(context, 'query', undefined),
filters: get(context, 'filters', undefined),
timeRange: get(input, 'timeRange', undefined),
query: get(input, 'query', undefined),
filters: get(input, 'filters', undefined),
forceFetch: true,
metricsAtAllLevels: args.metricsAtAllLevels,
partialRows: args.partialRows,
inspectorAdapters,
inspectorAdapters: inspectorAdapters as Adapters,
filterManager,
abortSignal: (abortSignal as unknown) as AbortSignal,
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@
import { createInputControlVisFn } from './input_control_fn';

// eslint-disable-next-line
import { functionWrapper } from '../../../../plugins/expressions/public/functions/tests/utils';
import { functionWrapper } from '../../../../plugins/expressions/common/expression_functions/specs/tests/utils';

jest.mock('./legacy_imports.ts');

describe('interpreter/functions#input_control_vis', () => {
const fn = functionWrapper(createInputControlVisFn);
const fn = functionWrapper(createInputControlVisFn());
const visConfig = {
controls: [
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,11 @@
import { i18n } from '@kbn/i18n';

import {
ExpressionFunction,
ExpressionFunctionDefinition,
KibanaDatatable,
Render,
} from '../../../../plugins/expressions/public';

const name = 'input_control_vis';

type Context = KibanaDatatable;

interface Arguments {
visConfig: string;
}
Expand All @@ -40,19 +36,15 @@ interface RenderValue {
visConfig: VisParams;
}

type Return = Promise<Render<RenderValue>>;

export const createInputControlVisFn = (): ExpressionFunction<
typeof name,
Context,
export const createInputControlVisFn = (): ExpressionFunctionDefinition<
'input_control_vis',
KibanaDatatable,
Arguments,
Return
Render<RenderValue>
> => ({
name: 'input_control_vis',
type: 'render',
context: {
types: [],
},
inputTypes: [],
help: i18n.translate('inputControl.function.help', {
defaultMessage: 'Input control visualization',
}),
Expand All @@ -63,7 +55,7 @@ export const createInputControlVisFn = (): ExpressionFunction<
help: '',
},
},
async fn(context, args) {
fn(input, args) {
const params = JSON.parse(args.visConfig);
return {
type: 'render',
Expand Down
20 changes: 0 additions & 20 deletions src/legacy/core_plugins/interpreter/README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,2 @@
Interpreter legacy plugin has been migrated to the New Platform. Use
`expressions` New Platform plugin instead.

In the New Platform:

```ts
class MyPlugin {
setup(core, { expressions }) {
expressions.registerFunction(myFunction);
}
start(core, { expressions }) {
}
}
```

In the Legacy Platform:

```ts
import { npSetup, npStart } from 'ui/new_platform';

npSetup.plugins.expressions.registerFunction(myFunction);
```
7 changes: 2 additions & 5 deletions src/legacy/core_plugins/interpreter/public/interpreter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,7 @@ import 'uiExports/interpreter';
import { register, registryFactory } from '@kbn/interpreter/common';
import { npSetup } from 'ui/new_platform';
import { registries } from './registries';
import {
ExpressionInterpretWithHandlers,
ExpressionExecutor,
} from '../../../../plugins/expressions/public';
import { Executor, ExpressionExecutor } from '../../../../plugins/expressions/public';

// Expose kbnInterpreter.register(specs) and kbnInterpreter.registries() globally so that plugins
// can register without a transpile step.
Expand All @@ -46,7 +43,7 @@ export const getInterpreter = async () => {
};

// TODO: This function will be left behind in the legacy platform.
export const interpretAst: ExpressionInterpretWithHandlers = async (ast, context, handlers) => {
export const interpretAst: Executor['run'] = async (ast, context, handlers) => {
const { interpreter } = await getInterpreter();
return await interpreter.interpretAst(ast, context, handlers);
};
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@
*/

// eslint-disable-next-line
import { functionWrapper } from '../../../../plugins/expressions/public/functions/tests/utils';
import { functionWrapper } from '../../../../plugins/expressions/common/expression_functions/specs/tests/utils';
import { createRegionMapFn } from './region_map_fn';

jest.mock('ui/new_platform');

describe('interpreter/functions#regionmap', () => {
const fn = functionWrapper(createRegionMapFn);
const fn = functionWrapper(createRegionMapFn());
const context = {
type: 'kibana_datatable',
rows: [{ 'col-0-1': 0 }],
Expand Down
4 changes: 2 additions & 2 deletions src/legacy/core_plugins/tile_map/public/tilemap_fn.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
*/

// eslint-disable-next-line
import { functionWrapper } from '../../../../plugins/expressions/public/functions/tests/utils';
import { functionWrapper } from '../../../../plugins/expressions/common/expression_functions/specs/tests/utils';
import { createTileMapFn } from './tile_map_fn';

jest.mock('ui/new_platform');
Expand All @@ -40,7 +40,7 @@ jest.mock('ui/vis/map/convert_to_geojson', () => ({
import { convertToGeoJson } from 'ui/vis/map/convert_to_geojson';

describe('interpreter/functions#tilemap', () => {
const fn = functionWrapper(createTileMapFn);
const fn = functionWrapper(createTileMapFn());
const context = {
type: 'kibana_datatable',
rows: [{ 'col-0-1': 0 }],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
*/

// eslint-disable-next-line
import { functionWrapper } from '../../../../plugins/expressions/public/functions/tests/utils';
import { functionWrapper } from '../../../../plugins/expressions/common/expression_functions/specs/tests/utils';
import { createMarkdownVisFn } from './markdown_fn';

describe('interpreter/functions#markdown', () => {
const fn = functionWrapper(createMarkdownVisFn);
const fn = functionWrapper(createMarkdownVisFn());
const args = {
font: { spec: { fontSize: 12 } },
openLinksInNewTab: true,
Expand Down
24 changes: 8 additions & 16 deletions src/legacy/core_plugins/vis_type_markdown/public/markdown_fn.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,31 +18,23 @@
*/

import { i18n } from '@kbn/i18n';
import { ExpressionFunction, Render } from '../../../../plugins/expressions/public';
import { ExpressionFunctionDefinition, Render } from '../../../../plugins/expressions/public';
import { Arguments, MarkdownVisParams } from './types';

const name = 'markdownVis';

type Context = undefined;

interface RenderValue {
visType: 'markdown';
visConfig: MarkdownVisParams;
}

type Return = Promise<Render<RenderValue>>;

export const createMarkdownVisFn = (): ExpressionFunction<
typeof name,
Context,
export const createMarkdownVisFn = (): ExpressionFunctionDefinition<
'markdownVis',
unknown,
Arguments,
Return
Render<RenderValue>
> => ({
name,
name: 'markdownVis',
type: 'render',
context: {
types: [],
},
inputTypes: [],
help: i18n.translate('visTypeMarkdown.function.help', {
defaultMessage: 'Markdown visualization',
}),
Expand Down Expand Up @@ -70,7 +62,7 @@ export const createMarkdownVisFn = (): ExpressionFunction<
}),
},
},
async fn(context, args) {
fn(input, args) {
return {
type: 'render',
as: 'visualization',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,21 +19,19 @@

import { last, findIndex, isNaN } from 'lodash';
import React, { Component } from 'react';

import { isColorDark } from '@elastic/eui';

import { getFormat } from '../legacy_imports';
import { MetricVisValue } from './metric_vis_value';
import { Input } from '../metric_vis_fn';
import { FieldFormatsContentType, IFieldFormat } from '../../../../../plugins/data/public';
import { Context } from '../metric_vis_fn';
import { KibanaDatatable } from '../../../../../plugins/expressions/public';
import { getHeatmapColors } from '../../../../../plugins/charts/public';
import { VisParams, MetricVisMetric } from '../types';
import { SchemaConfig, Vis } from '../../../visualizations/public';

export interface MetricVisComponentProps {
visParams: VisParams;
visData: Context;
visData: Input;
vis: Vis;
renderComplete: () => void;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,12 @@

import { createMetricVisFn } from './metric_vis_fn';
// eslint-disable-next-line @kbn/eslint/no-restricted-paths
import { functionWrapper } from '../../../../plugins/expressions/public/functions/tests/utils';
import { functionWrapper } from '../../../../plugins/expressions/common/expression_functions/specs/tests/utils';

jest.mock('ui/new_platform');

describe('interpreter/functions#metric', () => {
const fn = functionWrapper(createMetricVisFn);
const fn = functionWrapper(createMetricVisFn());
const context = {
type: 'kibana_datatable',
rows: [{ 'col-0-1': 0 }],
Expand Down
Loading

0 comments on commit 454d0a2

Please sign in to comment.