[MDS] Full Timeline Support

[huyaboo]

Proposal

As of today, Timeline does not support multiple datasource or MDS). #6009 provides most of the building blocks to support Timeline. However, to fully integrate Timeline in MDS, there is a need to support exporting/importing to another datasource. With #5712, users can specify which datasource to import from. Timeline should seamlessly import from different datasources.

High-Level Approach

Timeline, like other visualizations, stores visualization configuration info in the visState. An example is shown below:

{
  title: '(Timeline) Avg bytes over time',
  type: 'timelion',
  aggs: [],
  params: {
    expression: '.opensearch(opensearch_dashboards_sample_data_logs, metric=avg:bytes, timefield=@timestamp).lines(show=true).points(show=true).yaxis(label="Average bytes")',
    interval: 'auto'
  }
}

As shown, the visState stores the entire expression as a string. To properly support import/export, the data source references can be appended when the visualization is saved to the datasource by extracting out the data_source_name from the expression. Then, upon import to a specified data source, the data source id reference is appended, and any local cluster queries can be modified to use this new data source. This is a similar approach to #5975. However, to implement this approach will require significant effort. There are two main challenges:

1. How to append data source references
2. How to update the expression string safely

Challenge 1: Appending data source references

To reliably determine the data_source_name from the expression, the expression will have to be parsed. However, since Timeline syntax can specify named arguments, unnamed arguments, or a combination of the two, this will be a challenge to parse. Some investigation will need to be done on how the .opensearch() function resolves the input values to arguments.

Challenge 2: Changing the expression so that it uses a specified data source

Unlike other visualization types, Timeline syntax is specific to this visualization. To define this syntax, Timeline makes use of pegjs, a JS implementation of a PEG parser. A PEG parser provides a way to parse an input from a grammar. Think of PEG parsers as context-free-grammar (CFG) except that PEG parses are unambiguous in their parse and always parse an input string to the same parse tree. In Timeline's case, the grammar can be found in chain.peg.

OpenSearch-Dashboards/src/plugins/vis_type_timeline/common/chain.peg

Lines 1 to 3 in 46b17e4

	/*
	* Timeline syntax parser
	*/

The expression is parsed in the parseSheet() function when the request is being processed:

OpenSearch-Dashboards/src/plugins/vis_type_timeline/server/handlers/lib/parse_sheet.js

Lines 39 to 60 in 46b17e4

	export default function parseSheet(sheet) {
	return _.map(sheet, function (plot) {
	try {
	return Parser.parse(plot).tree;
	} catch (e) {
	if (e.expected) {
	throw new Error(
	i18n.translate('timeline.serverSideErrors.sheetParseErrorMessage', {
	defaultMessage: 'Expected: {expectedDescription} at character {column}',
	description: 'This would be for example: "Expected: a quote at character 5"',
	values: {
	expectedDescription: e.expected[0].description,
	column: e.column,
	},
	})
	);
	} else {
	throw e;
	}
	}
	});
	}

One major hurdle with import/export logic is that expression will have to be modified if export/import to another data source is supported. To modify this expression in a structured manner, the expression will have to be parsed. However, pegjs only supports an asymmetric parse (see this closed issue). In other words, it is not possible with pegjs to parse a string into a parse object, modify that object, and transform that object back into a string.

// Supported
expression string -> parse object

// Refer to Challenge 1
modify parse object with data_source_name

// NOT Supported
parse object -> expression string

This means that the reconstruction of the expression back from the parse object will require a manual implementation to modify the text and store it in the visState. This is possible to accomplish but will take effort to implement and test and there are some unknowns about behavior with existing visualizations.

Alternatives

We do not support import to a specified datasource. While there is the possibility of visualizations breaking upon import, it was already the case because if timeline visualizations specify a data source, upon import, the user will need to re-authenticate the data source reference. Basically, importing a Timeline visualization in MDS world will require more effort from the user.

[zhongnansu]

@huyaboo are we good to close this?