Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate the "Master" nomenclature #1318

Closed
8 tasks done
tlfeng opened this issue Mar 4, 2022 · 7 comments
Closed
8 tasks done

Deprecate the "Master" nomenclature #1318

tlfeng opened this issue Mar 4, 2022 · 7 comments

Comments

@tlfeng
Copy link

tlfeng commented Mar 4, 2022

Background

Coming from opensearch-project/OpenSearch#2589.

OpenSearch repository is going to replace the terminology "master" with "cluster manager".
issue: opensearch-project/OpenSearch#472 (comment), with the plan for its terminology replacement.

Although the existing usages with "master" will be supported in OpenSearch version 2.x to keep the backwards compatibility, please prepare for the nomenclature change in advance, and replace all the usages with "master" terminology in the code base.
All the OpenSearch REST APIs and settings that contain "master" terminology will be deprecated in 2.0, and alternative usages will be added.

Solution

Replace the terminology "master" with "cluster manager".

When being compatible with OpenSearch 2.0:

  • Replace "master" in code comment and internal variable / function name
  • Replace OpenSearch REST API that contains "master" with the provided inclusive REST API
  • Replace "master" in documentation

Tasks

@tlfeng tlfeng added the enhancement New feature or request label Mar 4, 2022
@tmarkley tmarkley self-assigned this Apr 15, 2022
@dblock dblock added the campaign Parent issues of OpenSearch release campaigns. label Apr 18, 2022
@dblock dblock removed the campaign Parent issues of OpenSearch release campaigns. label Apr 18, 2022
@dblock dblock changed the title Change the "Master" nomenclature Deprecate the "Master" nomenclature Apr 18, 2022
@tlfeng
Copy link
Author

tlfeng commented Apr 18, 2022

About Dashboards plugin, I looked through and did a search for "master" keyword in all the repositories of Dashboards plugins.
Only Anomaly Detection Dashboards plugin has got a usage in the CI workflow for setting up a testing Docker image: https://github.com/opensearch-project/anomaly-detection-dashboards-plugin/blob/1.3.1.0/.github/configurations/docker-compose.yml#L10
The issue to track above opensearch-project/anomaly-detection-dashboards-plugin#235

@dblock
Copy link
Member

dblock commented Apr 18, 2022

For 2.0, at a minimum please ensure that the plugin is not calling any deprecated APIs in core or another plugin, and confirm below, then remove the 2.0.0 label. If you have time, do the complete deprecation as described in this issue.

@tmarkley
Copy link
Contributor

tmarkley commented Apr 19, 2022

Dashboards is calling some of the now-deprecated APIs, but we cannot make all of those changes and have them tested by 2.0 unless we push out the release date. There is a significant amount of work required in Dashboards and the JS client.

Edit: currently working on a list of the required changes.

@tmarkley tmarkley added v2.1.0 and removed v2.0.0 labels Apr 19, 2022
@tmarkley
Copy link
Contributor

tmarkley commented Apr 19, 2022

Expectations are:

  1. If plugins/Dashboards are using deprecated APIs from the engine, we should remove them in 2.0.
  2. If plugins/Dashboards are providing APIs with non-inclusive words, we should deprecate them in 2.0 too.
  3. If plugins/Dashboards are using non-inclusive words internally (code, comment, etc), it is okay to make the change by 2.1. But 2.0 is still preferred.

@tmarkley
Copy link
Contributor

tmarkley commented Apr 20, 2022

Non-inclusive instances of master

OpenSearch APIs

Cluster master_timeout - Blocked by opensearch-project/opensearch-js/pull/222

"description":"Return local information, do not retrieve the state from master node (default: false)"
},
"master_timeout":{
"type":"time",
"description":"Explicit operation timeout for connection to master node"

cat master - Blocked by opensearch-project/opensearch-js/pull/222

(endpoint: 'cat.master', params: CatCommonParams, options?: LegacyCallAPIOptions): ReturnType<Client['cat']['master']>;

(endpoint: 'cat.master', params: CatCommonParams, options?: LegacyCallAPIOptions): ReturnType<Client['cat']['master']>;

"cat.master": {
"url_params": {
"format": "",
"local": "__flag__",
"master_timeout": "",
"h": [],
"help": "__flag__",
"s": [],
"v": "__flag__"
},
"methods": [
"GET"
],
"patterns": [
"_cat/master"
],
"documentation": "https://opensearch.org/docs/latest/opensearch/rest-api/cat/cat-master/"

apiVersion: master

* Version of the OpenSearch (6.7, 7.1 or `master`) client will be connecting to.

const mockOpenSearchClientConfig = { apiVersion: 'opensearch-client-master' };

apiVersion: 'master',
customHeaders: { xsrf: 'something' },
logQueries: false,
sniffOnStart: false,
sniffOnConnectionFault: false,
hosts: ['http://localhost/opensearch'],
requestHeadersWhitelist: [],
},
logger.get()
)
).toMatchInlineSnapshot(`
Object {
"apiVersion": "master",
"hosts": Array [
Object {
"headers": Object {
"x-opensearch-product-origin": "opensearch-dashboards",
"xsrf": "something",
},
"host": "localhost",
"path": "/opensearch",
"port": "80",
"protocol": "http:",
"query": null,
},
],
"keepAlive": true,
"log": [Function],
"sniffOnConnectionFault": false,
"sniffOnStart": false,
}
`);
});
test('parses fully specified config', () => {
const opensearchConfig: LegacyOpenSearchClientConfig = {
apiVersion: 'v7.0.0',
customHeaders: { xsrf: 'something' },
logQueries: true,
sniffOnStart: true,
sniffOnConnectionFault: true,
hosts: [
'http://localhost/opensearch',
'http://domain.com:1234/opensearch',
'https://opensearch.local',
],
requestHeadersWhitelist: [],
username: 'opensearch',
password: 'changeme',
pingTimeout: 12345,
requestTimeout: 54321,
sniffInterval: 11223344,
ssl: {
verificationMode: 'certificate',
certificateAuthorities: ['content-of-ca-path-1', 'content-of-ca-path-2'],
certificate: 'content-of-certificate-path',
key: 'content-of-key-path',
keyPassphrase: 'key-pass',
alwaysPresentCertificate: true,
},
};
const opensearchClientConfig = parseOpenSearchClientConfig(opensearchConfig, logger.get());
// Check that original references aren't used.
for (const host of opensearchClientConfig.hosts) {
expect(opensearchConfig.customHeaders).not.toBe(host.headers);
}
expect(opensearchConfig.ssl).not.toBe(opensearchClientConfig.ssl);
expect(opensearchClientConfig).toMatchInlineSnapshot(`
Object {
"apiVersion": "v7.0.0",
"hosts": Array [
Object {
"auth": "opensearch:changeme",
"headers": Object {
"x-opensearch-product-origin": "opensearch-dashboards",
"xsrf": "something",
},
"host": "localhost",
"path": "/opensearch",
"port": "80",
"protocol": "http:",
"query": null,
},
Object {
"auth": "opensearch:changeme",
"headers": Object {
"x-opensearch-product-origin": "opensearch-dashboards",
"xsrf": "something",
},
"host": "domain.com",
"path": "/opensearch",
"port": "1234",
"protocol": "http:",
"query": null,
},
Object {
"auth": "opensearch:changeme",
"headers": Object {
"x-opensearch-product-origin": "opensearch-dashboards",
"xsrf": "something",
},
"host": "opensearch.local",
"path": "/",
"port": "443",
"protocol": "https:",
"query": null,
},
],
"keepAlive": true,
"log": [Function],
"pingTimeout": 12345,
"requestTimeout": 54321,
"sniffInterval": 11223344,
"sniffOnConnectionFault": true,
"sniffOnStart": true,
"ssl": Object {
"ca": Array [
"content-of-ca-path-1",
"content-of-ca-path-2",
],
"cert": "content-of-certificate-path",
"checkServerIdentity": [Function],
"key": "content-of-key-path",
"passphrase": "key-pass",
"rejectUnauthorized": true,
},
}
`);
});
test('parses config timeouts of moment.Duration type', () => {
expect(
parseOpenSearchClientConfig(
{
apiVersion: 'master',
customHeaders: { xsrf: 'something' },
logQueries: false,
sniffOnStart: false,
sniffOnConnectionFault: false,
pingTimeout: duration(100, 'ms'),
requestTimeout: duration(30, 's'),
sniffInterval: duration(1, 'minute'),
hosts: ['http://localhost:9200/opensearch'],
requestHeadersWhitelist: [],
},
logger.get()
)
).toMatchInlineSnapshot(`
Object {
"apiVersion": "master",
"hosts": Array [
Object {
"headers": Object {
"x-opensearch-product-origin": "opensearch-dashboards",
"xsrf": "something",
},
"host": "localhost",
"path": "/opensearch",
"port": "9200",
"protocol": "http:",
"query": null,
},
],
"keepAlive": true,
"log": [Function],
"pingTimeout": 100,
"requestTimeout": 30000,
"sniffInterval": 60000,
"sniffOnConnectionFault": false,
"sniffOnStart": false,
}
`);
});
describe('#auth', () => {
test('is not set if #auth = false even if username and password are provided', () => {
expect(
parseOpenSearchClientConfig(
{
apiVersion: 'v7.0.0',
customHeaders: { xsrf: 'something' },
logQueries: true,
sniffOnStart: true,
sniffOnConnectionFault: true,
hosts: ['http://user:password@localhost/opensearch', 'https://opensearch.local'],
username: 'opensearch',
password: 'changeme',
requestHeadersWhitelist: [],
},
logger.get(),
{ auth: false }
)
).toMatchInlineSnapshot(`
Object {
"apiVersion": "v7.0.0",
"hosts": Array [
Object {
"headers": Object {
"x-opensearch-product-origin": "opensearch-dashboards",
"xsrf": "something",
},
"host": "localhost",
"path": "/opensearch",
"port": "80",
"protocol": "http:",
"query": null,
},
Object {
"headers": Object {
"x-opensearch-product-origin": "opensearch-dashboards",
"xsrf": "something",
},
"host": "opensearch.local",
"path": "/",
"port": "443",
"protocol": "https:",
"query": null,
},
],
"keepAlive": true,
"log": [Function],
"sniffOnConnectionFault": true,
"sniffOnStart": true,
}
`);
});
test('is not set if username is not specified', () => {
expect(
parseOpenSearchClientConfig(
{
apiVersion: 'v7.0.0',
customHeaders: { xsrf: 'something' },
logQueries: true,
sniffOnStart: true,
sniffOnConnectionFault: true,
hosts: ['https://opensearch.local'],
requestHeadersWhitelist: [],
password: 'changeme',
},
logger.get(),
{ auth: true }
)
).toMatchInlineSnapshot(`
Object {
"apiVersion": "v7.0.0",
"hosts": Array [
Object {
"headers": Object {
"x-opensearch-product-origin": "opensearch-dashboards",
"xsrf": "something",
},
"host": "opensearch.local",
"path": "/",
"port": "443",
"protocol": "https:",
"query": null,
},
],
"keepAlive": true,
"log": [Function],
"sniffOnConnectionFault": true,
"sniffOnStart": true,
}
`);
});
test('is not set if password is not specified', () => {
expect(
parseOpenSearchClientConfig(
{
apiVersion: 'v7.0.0',
customHeaders: { xsrf: 'something' },
logQueries: true,
sniffOnStart: true,
sniffOnConnectionFault: true,
hosts: ['https://opensearch.local'],
requestHeadersWhitelist: [],
username: 'opensearch',
},
logger.get(),
{ auth: true }
)
).toMatchInlineSnapshot(`
Object {
"apiVersion": "v7.0.0",
"hosts": Array [
Object {
"headers": Object {
"x-opensearch-product-origin": "opensearch-dashboards",
"xsrf": "something",
},
"host": "opensearch.local",
"path": "/",
"port": "443",
"protocol": "https:",
"query": null,
},
],
"keepAlive": true,
"log": [Function],
"sniffOnConnectionFault": true,
"sniffOnStart": true,
}
`);
});
});
describe('#customHeaders', () => {
test('override the default headers', () => {
const headerKey = Object.keys(DEFAULT_HEADERS)[0];
const parsedConfig = parseOpenSearchClientConfig(
{
apiVersion: 'master',
customHeaders: { [headerKey]: 'foo' },
logQueries: false,
sniffOnStart: false,
sniffOnConnectionFault: false,
hosts: ['http://localhost/opensearch'],
requestHeadersWhitelist: [],
},
logger.get()
);
expect(parsedConfig.hosts[0].headers).toEqual({
[headerKey]: 'foo',
});
});
});
describe('#log', () => {
test('default logger with #logQueries = false', () => {
const parsedConfig = parseOpenSearchClientConfig(
{
apiVersion: 'master',
customHeaders: { xsrf: 'something' },
logQueries: false,
sniffOnStart: false,
sniffOnConnectionFault: false,
hosts: ['http://localhost/opensearch'],
requestHeadersWhitelist: [],
},
logger.get()
);
const Logger = new parsedConfig.log();
Logger.error('some-error');
Logger.warning('some-warning');
Logger.trace('some-trace');
Logger.info('some-info');
Logger.debug('some-debug');
expect(typeof Logger.close).toBe('function');
expect(loggingSystemMock.collect(logger)).toMatchInlineSnapshot(`
Object {
"debug": Array [],
"error": Array [
Array [
"some-error",
],
],
"fatal": Array [],
"info": Array [],
"log": Array [],
"trace": Array [],
"warn": Array [
Array [
"some-warning",
],
],
}
`);
});
test('default logger with #logQueries = true', () => {
const parsedConfig = parseOpenSearchClientConfig(
{
apiVersion: 'master',
customHeaders: { xsrf: 'something' },
logQueries: true,
sniffOnStart: false,
sniffOnConnectionFault: false,
hosts: ['http://localhost/opensearch'],
requestHeadersWhitelist: [],
},
logger.get()
);
const Logger = new parsedConfig.log();
Logger.error('some-error');
Logger.warning('some-warning');
Logger.trace('METHOD', { path: '/some-path' }, '?query=2', 'unknown', '304');
Logger.info('some-info');
Logger.debug('some-debug');
expect(typeof Logger.close).toBe('function');
expect(loggingSystemMock.collect(logger)).toMatchInlineSnapshot(`
Object {
"debug": Array [
Array [
"304
METHOD /some-path
?query=2",
Object {
"tags": Array [
"query",
],
},
],
],
"error": Array [
Array [
"some-error",
],
],
"fatal": Array [],
"info": Array [],
"log": Array [],
"trace": Array [],
"warn": Array [
Array [
"some-warning",
],
],
}
`);
});
test('custom logger', () => {
const customLogger = jest.fn();
const parsedConfig = parseOpenSearchClientConfig(
{
apiVersion: 'master',

master nodes - Blocked by opensearch-project/opensearch-js/pull/222

['_local', '_master', 'data:true', 'data:false', 'master:true', 'master:false'],

OpenSearch Dashboards APIs

Cluster manager setupMaster

// forward a reloadLoggingConfig message to master

cluster.setupMaster({

Other Code

isDevClusterMaster config

isDevClusterMaster: boolean;
}
/** @internal */
export interface CliArgs {
dev: boolean;
envName?: string;
quiet: boolean;
silent: boolean;
watch: boolean;
repl: boolean;
basePath: boolean;
/** @deprecated use disableOptimizer to know if the @osd/optimizer is disabled in development */
optimize?: boolean;
runExamples: boolean;
disableOptimizer: boolean;
cache: boolean;
dist: boolean;
}
/** @internal */
export interface RawPackageInfo {
branch: string;
version: string;
build: {
distributable?: boolean;
number: number;
sha: string;
};
}
export class Env {
/**
* @internal
*/
public static createDefault(repoRoot: string, options: EnvOptions, pkg?: RawPackageInfo): Env {
if (!pkg) {
pkg = loadJsonFile.sync(join(repoRoot, 'package.json')) as RawPackageInfo;
}
return new Env(repoRoot, pkg, options);
}
/** @internal */
public readonly configDir: string;
/** @internal */
public readonly binDir: string;
/** @internal */
public readonly logDir: string;
/** @internal */
public readonly pluginSearchPaths: readonly string[];
/**
* Information about OpenSearch Dashboards package (version, build number etc.).
*/
public readonly packageInfo: Readonly<PackageInfo>;
/**
* Mode OpenSearch Dashboards currently run in (development or production).
*/
public readonly mode: Readonly<EnvironmentMode>;
/**
* Arguments provided through command line.
* @internal
*/
public readonly cliArgs: Readonly<CliArgs>;
/**
* Paths to the configuration files.
* @internal
*/
public readonly configs: readonly string[];
/**
* Indicates that this OpenSearch Dashboards instance is run as development Node Cluster master.
* @internal
*/
public readonly isDevClusterMaster: boolean;
/**
* @internal
*/
constructor(public readonly homeDir: string, pkg: RawPackageInfo, options: EnvOptions) {
this.configDir = resolve(this.homeDir, 'config');
this.binDir = resolve(this.homeDir, 'bin');
this.logDir = resolve(this.homeDir, 'log');
/**
* BEWARE: this needs to stay roughly synchronized with the @osd/optimizer
* `packages/osd-optimizer/src/optimizer_config.ts` determines the paths
* that should be searched for plugins to build
*/
this.pluginSearchPaths = [
resolve(this.homeDir, 'src', 'plugins'),
resolve(this.homeDir, 'plugins'),
...(options.cliArgs.runExamples ? [resolve(this.homeDir, 'examples')] : []),
resolve(this.homeDir, '..', 'opensearch-dashboards-extra'),
];
this.cliArgs = Object.freeze(options.cliArgs);
this.configs = Object.freeze(options.configs);
this.isDevClusterMaster = options.isDevClusterMaster;

isDevClusterMaster:
options.isDevClusterMaster !== undefined ? options.isDevClusterMaster : false,

"isDevClusterMaster": false,
"logDir": "/test/opensearchDashboardsRoot/log",
"mode": Object {
"dev": true,
"name": "development",
"prod": false,
},
"packageInfo": Object {
"branch": "feature-v1",
"buildNum": 9007199254740991,
"buildSha": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"dist": false,
"version": "v1",
},
"pluginSearchPaths": Array [
"/test/opensearchDashboardsRoot/src/plugins",
"/test/opensearchDashboardsRoot/plugins",
"/test/opensearchDashboardsRoot/../opensearch-dashboards-extra",
],
}
`;
exports[`correctly creates default environment if \`--env.name\` is supplied.: prod env properties 1`] = `
Env {
"binDir": "/test/opensearchDashboardsRoot/bin",
"cliArgs": Object {
"basePath": false,
"cache": true,
"dev": false,
"disableOptimizer": true,
"dist": false,
"envName": "production",
"quiet": false,
"repl": false,
"runExamples": false,
"silent": false,
"watch": false,
},
"configDir": "/test/opensearchDashboardsRoot/config",
"configs": Array [
"/some/other/path/some-opensearch-dashboards.yml",
],
"homeDir": "/test/opensearchDashboardsRoot",
"isDevClusterMaster": false,
"logDir": "/test/opensearchDashboardsRoot/log",
"mode": Object {
"dev": false,
"name": "production",
"prod": true,
},
"packageInfo": Object {
"branch": "feature-v1",
"buildNum": 9007199254740991,
"buildSha": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"dist": false,
"version": "v1",
},
"pluginSearchPaths": Array [
"/test/opensearchDashboardsRoot/src/plugins",
"/test/opensearchDashboardsRoot/plugins",
"/test/opensearchDashboardsRoot/../opensearch-dashboards-extra",
],
}
`;
exports[`correctly creates default environment in dev mode.: env properties 1`] = `
Env {
"binDir": "/test/opensearchDashboardsRoot/bin",
"cliArgs": Object {
"basePath": false,
"cache": true,
"dev": true,
"disableOptimizer": true,
"dist": false,
"quiet": false,
"repl": false,
"runExamples": false,
"silent": false,
"watch": false,
},
"configDir": "/test/opensearchDashboardsRoot/config",
"configs": Array [
"/test/cwd/config/opensearch_dashboards.yml",
],
"homeDir": "/test/opensearchDashboardsRoot",
"isDevClusterMaster": true,
"logDir": "/test/opensearchDashboardsRoot/log",
"mode": Object {
"dev": true,
"name": "development",
"prod": false,
},
"packageInfo": Object {
"branch": "some-branch",
"buildNum": 9007199254740991,
"buildSha": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"dist": false,
"version": "some-version",
},
"pluginSearchPaths": Array [
"/test/opensearchDashboardsRoot/src/plugins",
"/test/opensearchDashboardsRoot/plugins",
"/test/opensearchDashboardsRoot/../opensearch-dashboards-extra",
],
}
`;
exports[`correctly creates default environment in prod distributable mode.: env properties 1`] = `
Env {
"binDir": "/test/opensearchDashboardsRoot/bin",
"cliArgs": Object {
"basePath": false,
"cache": true,
"dev": false,
"disableOptimizer": true,
"dist": false,
"quiet": false,
"repl": false,
"runExamples": false,
"silent": false,
"watch": false,
},
"configDir": "/test/opensearchDashboardsRoot/config",
"configs": Array [
"/some/other/path/some-opensearch-dashboards.yml",
],
"homeDir": "/test/opensearchDashboardsRoot",
"isDevClusterMaster": false,
"logDir": "/test/opensearchDashboardsRoot/log",
"mode": Object {
"dev": false,
"name": "production",
"prod": true,
},
"packageInfo": Object {
"branch": "feature-v1",
"buildNum": 100,
"buildSha": "feature-v1-build-sha",
"dist": true,
"version": "v1",
},
"pluginSearchPaths": Array [
"/test/opensearchDashboardsRoot/src/plugins",
"/test/opensearchDashboardsRoot/plugins",
"/test/opensearchDashboardsRoot/../opensearch-dashboards-extra",
],
}
`;
exports[`correctly creates default environment in prod non-distributable mode.: env properties 1`] = `
Env {
"binDir": "/test/opensearchDashboardsRoot/bin",
"cliArgs": Object {
"basePath": false,
"cache": true,
"dev": false,
"disableOptimizer": true,
"dist": false,
"quiet": false,
"repl": false,
"runExamples": false,
"silent": false,
"watch": false,
},
"configDir": "/test/opensearchDashboardsRoot/config",
"configs": Array [
"/some/other/path/some-opensearch-dashboards.yml",
],
"homeDir": "/test/opensearchDashboardsRoot",
"isDevClusterMaster": false,
"logDir": "/test/opensearchDashboardsRoot/log",
"mode": Object {
"dev": false,
"name": "production",
"prod": true,
},
"packageInfo": Object {
"branch": "feature-v1",
"buildNum": 9007199254740991,
"buildSha": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"dist": false,
"version": "v1",
},
"pluginSearchPaths": Array [
"/test/opensearchDashboardsRoot/src/plugins",
"/test/opensearchDashboardsRoot/plugins",
"/test/opensearchDashboardsRoot/../opensearch-dashboards-extra",
],
}
`;
exports[`correctly creates environment with constructor.: env properties 1`] = `
Env {
"binDir": "/some/home/dir/bin",
"cliArgs": Object {
"basePath": false,
"cache": true,
"dev": false,
"disableOptimizer": true,
"dist": false,
"quiet": false,
"repl": false,
"runExamples": false,
"silent": false,
"watch": false,
},
"configDir": "/some/home/dir/config",
"configs": Array [
"/some/other/path/some-opensearch-dashboards.yml",
],
"homeDir": "/some/home/dir",
"isDevClusterMaster": false,

import { isMaster } from 'cluster';
import { CliArgs, Env, RawConfigService } from './config';
import { Root } from './root';
import { CriticalError } from './errors';
interface OpenSearchDashboardsFeatures {
// Indicates whether we can run OpenSearch Dashboards in a so called cluster mode in which
// OpenSearch Dashboards is run as a "worker" process together with optimizer "worker" process
// that are orchestrated by the "master" process (dev mode only feature).
isClusterModeSupported: boolean;
// Indicates whether we can run OpenSearch Dashboards in REPL mode (dev mode only feature).
isReplModeSupported: boolean;
}
interface BootstrapArgs {
configs: string[];
cliArgs: CliArgs;
applyConfigOverrides: (config: Record<string, any>) => Record<string, any>;
features: OpenSearchDashboardsFeatures;
}
/**
*
* @internal
* @param param0 - options
*/
export async function bootstrap({
configs,
cliArgs,
applyConfigOverrides,
features,
}: BootstrapArgs) {
if (cliArgs.repl && !features.isReplModeSupported) {
onRootShutdown('OpenSearch Dashboards REPL mode can only be run in development mode.');
}
if (cliArgs.optimize) {
// --optimize is deprecated and does nothing now, avoid starting up and just shutdown
return;
}
// `bootstrap` is exported from the `src/core/server/index` module,
// meaning that any test importing, implicitly or explicitly, anything concrete
// from `core/server` will load `dev-utils`. As some tests are mocking the `fs` package,
// and as `REPO_ROOT` is initialized on the fly when importing `dev-utils` and requires
// the `fs` package, it causes failures. This is why we use a dynamic `require` here.
// eslint-disable-next-line @typescript-eslint/no-var-requires
const { REPO_ROOT } = require('@osd/utils');
const env = Env.createDefault(REPO_ROOT, {
configs,
cliArgs,
isDevClusterMaster: isMaster && cliArgs.dev && features.isClusterModeSupported,

test('does not start http server if process is dev cluster master', async () => {
const configService = createConfigService();
const httpServer = {
isListening: () => false,
setup: jest.fn().mockReturnValue({}),
start: jest.fn(),
stop: noop,
};
mockHttpServer.mockImplementation(() => httpServer);
const service = new HttpService({
coreId,
configService,
env: Env.createDefault(REPO_ROOT, getEnvOptions({ isDevClusterMaster: true })),

* 2. When the process is run as dev cluster master in which case cluster manager
* will fork a dedicated process where http service will be set up instead.
* @internal
* */
private shouldListen(config: HttpConfig) {
return !this.coreContext.env.isDevClusterMaster && config.autoListen;

describe('once LegacyService is set up in `devClusterMaster` mode', () => {
beforeEach(() => {
configService.atPath.mockImplementation((path) => {
return new BehaviorSubject(
path === 'dev' ? { basePathProxyTargetPort: 100500 } : { basePath: '/abc' }
);
});
});
test('creates ClusterManager without base path proxy.', async () => {
const devClusterLegacyService = new LegacyService({
coreId,
env: Env.createDefault(
REPO_ROOT,
getEnvOptions({
cliArgs: { silent: true, basePath: false },
isDevClusterMaster: true,
})
),
logger,
configService: configService as any,
});
await devClusterLegacyService.setupLegacyConfig();
await devClusterLegacyService.setup(setupDeps);
await devClusterLegacyService.start(startDeps);
expect(MockClusterManager).toHaveBeenCalledTimes(1);
expect(MockClusterManager).toHaveBeenCalledWith(
expect.objectContaining({ silent: true, basePath: false }),
expect.objectContaining({
get: expect.any(Function),
set: expect.any(Function),
}),
undefined
);
});
test('creates ClusterManager with base path proxy.', async () => {
const devClusterLegacyService = new LegacyService({
coreId,
env: Env.createDefault(
REPO_ROOT,
getEnvOptions({
cliArgs: { quiet: true, basePath: true },
isDevClusterMaster: true,

if (this.coreContext.env.isDevClusterMaster) {

const initialize = config.initialize && !this.coreContext.env.isDevClusterMaster;

import { isMaster, isWorker } from 'cluster';
import { Server } from '@hapi/hapi';
import { LogRotator } from './log_rotator';
import { OpenSearchDashboardsConfig } from '../../osd_server';
let logRotator: LogRotator;
export async function setupLoggingRotate(server: Server, config: OpenSearchDashboardsConfig) {
// If log rotate is not enabled we skip
if (!config.get('logging.rotate.enabled')) {
return;
}
// We just want to start the logging rotate service once
// and we choose to use the master (prod) or the worker server (dev)
if (!isMaster && isWorker && process.env.osdWorkerType !== 'server') {
return;
}
// We don't want to run logging rotate server if
// we are not logging to a file
if (config.get('logging.dest') === 'stdout') {
server.log(
['warning', 'logging:rotate'],
'Log rotation is enabled but logging.dest is configured for stdout. Set logging.dest to a file for this setting to take effect.'
);
return;
}
// Enable Logging Rotate Service
// We need the master process and it can

import { isMaster } from 'cluster';
import fs from 'fs';
import { Server } from '@hapi/hapi';
import { throttle } from 'lodash';
import { tmpdir } from 'os';
import { basename, dirname, join, sep } from 'path';
import { Observable } from 'rxjs';
import { first } from 'rxjs/operators';
import { promisify } from 'util';
import { OpenSearchDashboardsConfig } from '../../osd_server';
const mkdirAsync = promisify(fs.mkdir);
const readdirAsync = promisify(fs.readdir);
const renameAsync = promisify(fs.rename);
const statAsync = promisify(fs.stat);
const unlinkAsync = promisify(fs.unlink);
const writeFileAsync = promisify(fs.writeFile);
export class LogRotator {
private readonly config: OpenSearchDashboardsConfig;
private readonly log: Server['log'];
public logFilePath: string;
public everyBytes: number;
public keepFiles: number;
public running: boolean;
private logFileSize: number;
public isRotating: boolean;
public throttledRotate: () => void;
public stalker: chokidar.FSWatcher | null;
public usePolling: boolean;
public pollingInterval: number;
private stalkerUsePollingPolicyTestTimeout: NodeJS.Timeout | null;
public shouldUsePolling: boolean;
constructor(config: OpenSearchDashboardsConfig, server: Server) {
this.config = config;
this.log = server.log.bind(server);
this.logFilePath = config.get('logging.dest');
this.everyBytes = config.get('logging.rotate.everyBytes');
this.keepFiles = config.get('logging.rotate.keepFiles');
this.running = false;
this.logFileSize = 0;
this.isRotating = false;
this.throttledRotate = throttle(async () => await this._rotate(), 5000);
this.stalker = null;
this.usePolling = config.get('logging.rotate.usePolling');
this.pollingInterval = config.get('logging.rotate.pollingInterval');
this.shouldUsePolling = false;
this.stalkerUsePollingPolicyTestTimeout = null;
}
async start() {
if (this.running) {
return;
}
this.running = true;
// create exit listener for cleanup purposes
this._createExitListener();
// call rotate on startup
await this._callRotateOnStartup();
// init log file size monitor
await this._startLogFileSizeMonitor();
}
stop = () => {
if (!this.running) {
return;
}
// cleanup exit listener
this._deleteExitListener();
// stop log file size monitor
this._stopLogFileSizeMonitor();
this.running = false;
};
async _shouldUsePolling() {
try {
// Setup a test file in order to try the fs env
// and understand if we need to usePolling or not
const tempFileDir = tmpdir();
const tempFile = join(tempFileDir, 'osd_log_rotation_use_polling_test_file.log');
await mkdirAsync(tempFileDir, { recursive: true });
await writeFileAsync(tempFile, '');
// setup fs.watch for the temp test file
const testWatcher = fs.watch(tempFile, { persistent: false });
// await writeFileAsync(tempFile, 'test');
const usePollingTest$ = new Observable<boolean>((observer) => {
// observable complete function
const completeFn = (completeStatus: boolean) => {
if (this.stalkerUsePollingPolicyTestTimeout) {
clearTimeout(this.stalkerUsePollingPolicyTestTimeout);
}
testWatcher.close();
observer.next(completeStatus);
observer.complete();
};
// setup conditions that would fire the observable
this.stalkerUsePollingPolicyTestTimeout = setTimeout(() => completeFn(true), 15000);
testWatcher.on('change', () => completeFn(false));
testWatcher.on('error', () => completeFn(true));
// fire test watcher events
setTimeout(() => {
fs.writeFileSync(tempFile, 'test');
}, 0);
});
// wait for the first observable result and consider it as the result
// for our use polling test
const usePollingTestResult = await usePollingTest$.pipe(first()).toPromise();
// delete the temp file used for the test
await unlinkAsync(tempFile);
return usePollingTestResult;
} catch {
return true;
}
}
async _startLogFileSizeMonitor() {
this.usePolling = this.config.get('logging.rotate.usePolling');
this.shouldUsePolling = await this._shouldUsePolling();
if (this.usePolling && !this.shouldUsePolling) {
this.log(
['warning', 'logging:rotate'],
'Looks like your current environment support a faster algorithm then polling. You can try to disable `usePolling`'
);
}
if (!this.usePolling && this.shouldUsePolling) {
this.log(
['error', 'logging:rotate'],
'Looks like within your current environment you need to use polling in order to enable log rotator. Please enable `usePolling`'
);
}
this.stalker = chokidar.watch(this.logFilePath, {
ignoreInitial: true,
awaitWriteFinish: false,
useFsEvents: false,
usePolling: this.usePolling,
interval: this.pollingInterval,
binaryInterval: this.pollingInterval,
alwaysStat: true,
atomic: false,
});
this.stalker.on('change', this._logFileSizeMonitorHandler);
}
_logFileSizeMonitorHandler = async (filename: string, stats: fs.Stats) => {
if (!filename || !stats) {
return;
}
this.logFileSize = stats.size || 0;
await this.throttledRotate();
};
_stopLogFileSizeMonitor() {
if (!this.stalker) {
return;
}
this.stalker.close();
if (this.stalkerUsePollingPolicyTestTimeout) {
clearTimeout(this.stalkerUsePollingPolicyTestTimeout);
}
}
_createExitListener() {
process.on('exit', this.stop);
}
_deleteExitListener() {
process.removeListener('exit', this.stop);
}
async _getLogFileSizeAndCreateIfNeeded() {
try {
const logFileStats = await statAsync(this.logFilePath);
return logFileStats.size;
} catch {
// touch the file to make the watcher being able to register
// change events
await writeFileAsync(this.logFilePath, '');
return 0;
}
}
async _callRotateOnStartup() {
this.logFileSize = await this._getLogFileSizeAndCreateIfNeeded();
await this._rotate();
}
_shouldRotate() {
// should rotate evaluation
// 1. should rotate if current log size exceeds
// the defined one on everyBytes
// 2. should not rotate if is already rotating or if any
// of the conditions on 1. do not apply
if (this.isRotating) {
return false;
}
return this.logFileSize >= this.everyBytes;
}
async _rotate() {
if (!this._shouldRotate()) {
return;
}
await this._rotateNow();
}
async _rotateNow() {
// rotate process
// 1. get rotated files metadata (list of log rotated files present on the log folder, numerical sorted)
// 2. delete last file
// 3. rename all files to the correct index +1
// 4. rename + compress current log into 1
// 5. send SIGHUP to reload log config
// rotate process is starting
this.isRotating = true;
// get rotated files metadata
const foundRotatedFiles = await this._readRotatedFilesMetadata();
// delete number of rotated files exceeding the keepFiles limit setting
const rotatedFiles: string[] = await this._deleteFoundRotatedFilesAboveKeepFilesLimit(
foundRotatedFiles
);
// delete last file
await this._deleteLastRotatedFile(rotatedFiles);
// rename all files to correct index + 1
// and normalize numbering if by some reason
// (for example log file deletion) that numbering
// was interrupted
await this._renameRotatedFilesByOne(rotatedFiles);
// rename current log into 0
await this._rotateCurrentLogFile();
// send SIGHUP to reload log configuration
this._sendReloadLogConfigSignal();
// Reset log file size
this.logFileSize = 0;
// rotate process is finished
this.isRotating = false;
}
async _readRotatedFilesMetadata() {
const logFileBaseName = basename(this.logFilePath);
const logFilesFolder = dirname(this.logFilePath);
const foundLogFiles: string[] = await readdirAsync(logFilesFolder);
return (
foundLogFiles
.filter((file) => new RegExp(`${logFileBaseName}\\.\\d`).test(file))
// we use .slice(-1) here in order to retrieve the last number match in the read filenames
.sort((a, b) => Number(a.match(/(\d+)/g)!.slice(-1)) - Number(b.match(/(\d+)/g)!.slice(-1)))
.map((filename) => `${logFilesFolder}${sep}${filename}`)
);
}
async _deleteFoundRotatedFilesAboveKeepFilesLimit(foundRotatedFiles: string[]) {
if (foundRotatedFiles.length <= this.keepFiles) {
return foundRotatedFiles;
}
const finalRotatedFiles = foundRotatedFiles.slice(0, this.keepFiles);
const rotatedFilesToDelete = foundRotatedFiles.slice(
finalRotatedFiles.length,
foundRotatedFiles.length
);
await Promise.all(
rotatedFilesToDelete.map((rotatedFilePath: string) => unlinkAsync(rotatedFilePath))
);
return finalRotatedFiles;
}
async _deleteLastRotatedFile(rotatedFiles: string[]) {
if (rotatedFiles.length < this.keepFiles) {
return;
}
const lastFilePath: string = rotatedFiles.pop() as string;
await unlinkAsync(lastFilePath);
}
async _renameRotatedFilesByOne(rotatedFiles: string[]) {
const logFileBaseName = basename(this.logFilePath);
const logFilesFolder = dirname(this.logFilePath);
for (let i = rotatedFiles.length - 1; i >= 0; i--) {
const oldFilePath = rotatedFiles[i];
const newFilePath = `${logFilesFolder}${sep}${logFileBaseName}.${i + 1}`;
await renameAsync(oldFilePath, newFilePath);
}
}
async _rotateCurrentLogFile() {
const newFilePath = `${this.logFilePath}.0`;
await renameAsync(this.logFilePath, newFilePath);
}
_sendReloadLogConfigSignal() {
if (isMaster) {

Missing documentation references

clusterRoute: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-reroute.html`,
clusterState: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-state.html`,
clusterStats: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-stats.html`,
clusterPending: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-pending.html`,
},
mappingTypes: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/current/mapping-types.html`,
moduleScripting: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/modules-scripting.html`,
indexAPI: {
indexAnalyze: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-analyze.html`,
indexClearCache: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-clearcache.html`,
indexClone: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-clone-index.html`,
indexSynced: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-synced-flush-api.html`,
indexFlush: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-flush.html`,
indexForceMerge: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-forcemerge.html`,
indexSetting: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-get-settings.html`,
indexUpgrade: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-upgrade.html`,
indexUpdateSetting: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-update-settings.html`,
indexRecovery: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-recovery.html`,
indexRefresh: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-refresh.html`,
indexRollover: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-rollover-index.html`,
indexSegment: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-segments.html`,
indexShardStore: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-shards-stores.html`,
indexShrink: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-shrink-index.html`,
indexSplit: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-split-index.html`,
indexStats: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-stats.html`,
indexGetFieldMapping: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-get-field-mapping.html`,
indexGetMapping: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-get-mapping.html`,
indexOpenClose: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-open-close.html`,
indexPutMapping: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-put-mapping.html`,
indexSearchValidate: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-validate.html`,
},
ingest: {
deletePipeline: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/delete-pipeline-api.html`,
getPipeline: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/get-pipeline-api.html`,
putPipeline: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/put-pipeline-api.html`,
simulatePipeline: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/simulate-pipeline-api.html`,
grokProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/grok-processor.html`,
appendProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/append-processor.html`,
bytesProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/bytes-processor.html`,
ingestCircleProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/ingest-circle-processor.html`,
csvProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/csv-processor.html`,
convertProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/convert-processor.html`,
dataProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/date-processor.html`,
dataIndexNamProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/date-index-name-processor.html`,
dissectProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/dissect-processor.html`,
dotExpandProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/dot-expand-processor.html`,
dropProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/drop-processor.html`,
failProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/fail-processor.html`,
foreachProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/foreach-processor.html`,
geoIPProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/geoip-processor.html`,
gusbProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/gsub-processor.html`,
htmlstripProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/htmlstrip-processor.html`,
inferenceProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/inference-processor.html`,
joinProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/join-processor.html`,
jsonProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/json-processor.html`,
kvProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/kv-processor.html`,
lowecaseProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/lowercase-processor.html`,
pipelineProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/pipeline-processor.html`,
removeProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/remove-processor.html`,
renameProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/rename-processor.html`,
scriptProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/script-processor.html`,
setProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/set-processor.html`,
securityUserProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/ingest-node-set-security-user-processor.html`,
splitProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/split-processor.html`,
sortProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/sort-processor.html`,
trimProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/trim-processor.html`,
uppercaseProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/uppercase-processor.html`,
urldecodeProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/urldecode-processor.html`,
userAgentProcessor: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/user-agent-processor.html`,
},
nodes: {
info: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-nodes-info.html`,
hotThreads: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-nodes-hot-threads.html`,
reloadSecuritySetting: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/secure-settings.html#reloadable-secure-settings`,
// Possible here but no details: https://opensearch.org/docs/latest/opensearch/popular-api/#get-node-statistics
nodeStats: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-nodes-stats.html`,
usage: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-nodes-usage.html`,
},
reIndex: {
// Missing _rethrottle
rethrottle: `https://opensearch.org/docs/latest/opensearch/rest-api/document-apis/reindex/`,
},
timelineDeprecation: `https://opensearch.org/docs/latest/guide/en/kibana/master/dashboard.html#timeline-deprecation`,
apmServer: `https://opensearch.org/downloads/apm/apm-server`,
tutorial: {
loadDataTutorial: `https://opensearch.org/guide/en/kibana/current/tutorial-load-dataset.html`,
visualizeTutorial: `https://opensearch.org/guide/en/kibana/current/tutorial-visualizing.html`,
},
scroll: {
clear_scroll: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-request-body.html#_clear_scroll_api`,
},
documentAPI: {
// missing `rethrottle` info
delete_by_query: `https://opensearch.org/docs/latest/opensearch/rest-api/document-apis/delete-by-query/`,
multiTermVector: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/docs-multi-termvectors.html`,
termVector: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/docs-termvectors.html`,
update_by_query_rethrottle: `https://opensearch.org/docs/latest/opensearch/rest-api/document-apis/update-by-query/`,
},
filed_caps: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-field-caps.html`,
painless_execute: `https://opensearch.org/docs/latest/guide/en/elasticsearch/painless/master/painless-execute-api.html`,
search: {
search: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-search.html`,
searchRankEval: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-rank-eval.html`,
searchShards: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-shards.html`,
searchFieldCap: `https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-field-caps.html`,

// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/append-processor.html
const appendProcessorDefinition = {
append: {
__template: {
field: '',
value: [],
},
field: '',
value: [],
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/bytes-processor.html
const bytesProcessorDefinition = {
bytes: {
__template: {
field: '',
},
field: '',
target_field: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/ingest-circle-processor.html
const circleProcessorDefinition = {
circle: {
__template: {
field: '',
error_distance: '',
shape_type: '',
},
field: '',
target_field: '',
error_distance: '',
shape_type: {
__one_of: ['geo_shape', 'shape'],
},
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/csv-processor.html
const csvProcessorDefinition = {
csv: {
__template: {
field: '',
target_fields: [''],
},
field: '',
target_fields: [''],
separator: '',
quote: '',
empty_value: '',
trim: {
__one_of: [true, false],
},
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/convert-processor.html
const convertProcessorDefinition = {
convert: {
__template: {
field: '',
type: '',
},
field: '',
type: {
__one_of: ['integer', 'float', 'string', 'boolean', 'auto'],
},
target_field: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/date-processor.html
const dateProcessorDefinition = {
date: {
__template: {
field: '',
formats: [],
},
field: '',
target_field: '@timestamp',
formats: [],
timezone: 'UTC',
locale: 'ENGLISH',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/date-index-name-processor.html
const dateIndexNameProcessorDefinition = {
date_index_name: {
__template: {
field: '',
date_rounding: '',
},
field: '',
date_rounding: {
__one_of: ['y', 'M', 'w', 'd', 'h', 'm', 's'],
},
date_formats: [],
timezone: 'UTC',
locale: 'ENGLISH',
index_name_format: 'yyyy-MM-dd',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/dissect-processor.html
const dissectProcessorDefinition = {
dissect: {
__template: {
field: '',
pattern: '',
},
field: '',
pattern: '',
append_separator: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/dot-expand-processor.html
const dotExpanderProcessorDefinition = {
dot_expander: {
__template: {
field: '',
},
field: '',
path: '',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/drop-processor.html
const dropProcessorDefinition = {
drop: {
__template: {},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/fail-processor.html
const failProcessorDefinition = {
fail: {
__template: {
message: '',
},
message: '',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/foreach-processor.html
const foreachProcessorDefinition = {
foreach: {
__template: {
field: '',
processor: {},
},
field: '',
processor: {
__scope_link: '_processor',
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/geoip-processor.html
const geoipProcessorDefinition = {
geoip: {
__template: {
field: '',
},
field: '',
target_field: '',
database_file: '',
properties: [''],
ignore_missing: {
__one_of: [false, true],
},
first_only: {
__one_of: [false, true],
},
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/grok-processor.html
const grokProcessorDefinition = {
grok: {
__template: {
field: '',
patterns: [],
},
field: '',
patterns: [],
pattern_definitions: {},
trace_match: {
__one_of: [false, true],
},
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/gsub-processor.html
const gsubProcessorDefinition = {
gsub: {
__template: {
field: '',
pattern: '',
replacement: '',
},
field: '',
pattern: '',
replacement: '',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/htmlstrip-processor.html
const htmlStripProcessorDefinition = {
html_strip: {
__template: {
field: '',
},
field: '',
target_field: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/inference-processor.html
const inferenceProcessorDefinition = {
inference: {
__template: {
model_id: '',
field_map: {},
inference_config: {},
},
model_id: '',
field_map: {},
inference_config: {},
target_field: '',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/join-processor.html
const joinProcessorDefinition = {
join: {
__template: {
field: '',
separator: '',
},
field: '',
separator: '',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/json-processor.html
const jsonProcessorDefinition = {
json: {
__template: {
field: '',
},
field: '',
target_field: '',
add_to_root: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/kv-processor.html
const kvProcessorDefinition = {
kv: {
__template: {
field: '',
field_split: '',
value_split: '',
},
field: '',
field_split: '',
value_split: '',
target_field: '',
include_keys: [],
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/lowercase-processor.html
const lowercaseProcessorDefinition = {
lowercase: {
__template: {
field: '',
},
field: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/pipeline-processor.html
const pipelineProcessorDefinition = {
pipeline: {
__template: {
name: '',
},
name: '',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/remove-processor.html
const removeProcessorDefinition = {
remove: {
__template: {
field: '',
},
field: '',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/rename-processor.html
const renameProcessorDefinition = {
rename: {
__template: {
field: '',
target_field: '',
},
field: '',
target_field: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/script-processor.html
const scriptProcessorDefinition = {
script: {
__template: {},
lang: 'painless',
file: '',
id: '',
source: '',
params: {},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/set-processor.html
const setProcessorDefinition = {
set: {
__template: {
field: '',
value: '',
},
field: '',
value: '',
override: {
__one_of: [true, false],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/ingest-node-set-security-user-processor.html
const setSecurityUserProcessorDefinition = {
set_security_user: {
__template: {
field: '',
},
field: '',
properties: [''],
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/split-processor.html
const splitProcessorDefinition = {
split: {
__template: {
field: '',
separator: '',
},
field: '',
separator: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/sort-processor.html
const sortProcessorDefinition = {
sort: {
__template: {
field: '',
},
field: '',
order: 'asc',
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/trim-processor.html
const trimProcessorDefinition = {
trim: {
__template: {
field: '',
},
field: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/uppercase-processor.html
const uppercaseProcessorDefinition = {
uppercase: {
__template: {
field: '',
},
field: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/urldecode-processor.html
const urlDecodeProcessorDefinition = {
urldecode: {
__template: {
field: '',
},
field: '',
target_field: '',
ignore_missing: {
__one_of: [false, true],
},
...commonPipelineParams,
},
};
// Based on https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/user-agent-processor.html

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-pending.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-reroute.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-state.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-stats.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/modules-scripting.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-field-caps.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/modules-scripting.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-analyze.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-clearcache.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-clone-index.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-synced-flush-api.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-flush.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-forcemerge.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-get-field-mapping.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-get-mapping.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-get-settings.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-upgrade.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-open-close.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-put-mapping.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-update-settings.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-recovery.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-refresh.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-rollover-index.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-segments.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-shards-stores.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-shrink-index.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-split-index.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-stats.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/indices-upgrade.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-validate.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/delete-pipeline-api.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/get-pipeline-api.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/grok-processor.html#grok-processor-rest-get"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/put-pipeline-api.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/simulate-pipeline-api.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/docs-multi-termvectors.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-nodes-hot-threads.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-nodes-info.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/secure-settings.html#reloadable-secure-settings"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-nodes-stats.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/cluster-nodes-usage.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/modules-scripting.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-rank-eval.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/painless/master/painless-execute-api.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/search-shards.html"

"documentation": "https://opensearch.org/docs/latest/guide/en/elasticsearch/reference/master/docs-termvectors.html"

Incorrect references to master branch

'https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/plugins/bfetch/README.md',

href: 'https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/foo/README.md',

'https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/embeddable/README.md',

{
label: 'IRouter API docs',
href:
'https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/docs/development/core/server/opensearch-dashboards-plugin-core-server.irouter.md',
iconType: 'logoGithub',
target: '_blank',
size: 's',
},
{
label: 'HttpHandler (core.http.fetch) API docs',
href:
'https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/docs/development/core/public/opensearch-dashboards-plugin-core-public.httphandler.md',
iconType: 'logoGithub',
target: '_blank',
size: 's',
},
{
label: 'Conventions',
href:
'https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/DEVELOPER_GUIDE.md#api-endpoints',
iconType: 'logoGithub',
target: '_blank',
size: 's',
},

{
label: 'IRouter',
href:
'https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/docs/development/core/server/opensearch-dashboards-plugin-core-server.irouter.md',
iconType: 'logoGithub',
target: '_blank',
size: 's',
},
{
label: 'HttpHandler (core.http.fetch)',
href:
'https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/docs/development/core/public/opensearch-dashboards-plugin-core-public.httphandler.md',
iconType: 'logoGithub',
target: '_blank',
size: 's',
},

See the [opensearchDashboards contributing guide](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/CONTRIBUTING.md) for instructions setting up your development environment.

developerExamples.register({
appId: 'stateContainersExampleBrowserHistory',
title: 'State containers using browser history',
description: `An example todo app that uses browser history and state container utilities like createStateContainerReactHelpers,
createStateContainer, createOsdUrlStateStorage, createSessionStorageStateStorage,
syncStates and getStateFromOsdUrl to keep state in sync with the URL. Change some parameters, navigate away and then back, and the
state should be preserved.`,
links: [
{
label: 'README',
href:
'https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/opensearch_dashboards_utils/docs/state_containers/README.md',
iconType: 'logoGithub',
size: 's',
target: '_blank',
},
],
});
developerExamples.register({
appId: 'stateContainersExampleHashHistory',
title: 'State containers using hash history',
description: `An example todo app that uses hash history and state container utilities like createStateContainerReactHelpers,
createStateContainer, createOsdUrlStateStorage, createSessionStorageStateStorage,
syncStates and getStateFromOsdUrl to keep state in sync with the URL. Change some parameters, navigate away and then back, and the
state should be preserved.`,
links: [
{
label: 'README',
href:
'https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/opensearch_dashboards_utils/docs/state_containers/README.md',
iconType: 'logoGithub',
size: 's',
target: '_blank',
},
],
});
developerExamples.register({
appId: PLUGIN_ID,
title: 'Sync state from a query bar with the url',
description: `Shows how to use data.syncQueryStateWitUrl in combination with state container utilities from opensearch_dashboards_utils to
show a query bar that stores state in the url and is kept in sync.
`,
links: [
{
label: 'README',
href:
'https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/plugins/data/public/query/state_sync/README.md',
iconType: 'logoGithub',
size: 's',
target: '_blank',
},
],
});

'https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/plugins/ui_actions/README.md',

'https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/plugins/share/public/url_generators/README.md',

// https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/ROADMAP.md

// use version 2 of the resolver interface, https://github.com/benmosher/eslint-plugin-import/blob/master/resolvers/README.md#interfaceversion--number

"url": "https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/packages/osd-eslint-import-resolver-opensearch-dashboards"

See [the resolvers docs](https://github.com/benmosher/eslint-plugin-import#resolvers) or the [resolver spec](https://github.com/benmosher/eslint-plugin-import/blob/master/resolvers/README.md#resolvesource-file-config---found-boolean-path-string-) for more details.

[Here](https://github.com/yahoo/intl-messageformat/blob/master/src/core.js#L62)
you can find default format options used as the prototype of the formats
provided to the constructor.
Creating instances of `IntlMessageFormat` is expensive.
[Intl-format-cache](https://github.com/yahoo/intl-format-cache)
library is simply to make it easier to create a cache of format
instances of a particular type to aid in their reuse. Under the
hood, this package creates a cache key based on the arguments passed
to the memoized constructor.
```js
import memoizeIntlConstructor from 'intl-format-cache';
const getMessageFormat = memoizeIntlConstructor(IntlMessageFormat);
```
## Vanilla JS
`Intl-messageformat` package assumes that the
[Intl](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl)
global object exists in the runtime. `Intl` is present in all modern
browsers and Node.js 0.10+. In order to load i18n engine
in Node.js we should simply `import` this module (in Node.js, the
[data](https://github.com/yahoo/intl-messageformat/tree/master/dist/locale-data)

* {@link https://github.com/yahoo/intl-messageformat/blob/master/src/core.js#L62}

To target the current development version of OpenSearch Dashboards just use the default `master` branch.
```sh
node scripts/generate_plugin --name my_plugin_name -y
# generates a plugin in `plugins/my_plugin_name`
```
To target 1.0, use the `1.0` branch.
```sh
git checkout 1.x
yarn osd bootstrap # always bootstrap when switching branches
node scripts/generate_plugin --name my_plugin_name -y
# generates a plugin for OpenSearch Dashboards 1.0 in `../opensearch-dashboards-extra/my_plugin_name`
```
The generate script supports a few flags; run it with the `--help` flag to learn more.
```sh
node scripts/generate_plugin --help
```
## Updating
Since the Plugin Generator is now a part of the OpenSearch Dashboards repo, when you update your local checkout of the OpenSearch Dashboards repository and `bootstrap` everything should be up to date!
> ***NOTE:*** These commands should be run from the OpenSearch Dashboards repo, and `upstream` is our convention for the git remote that references https://github.com/opensearch-project/OpenSearch-Dashboards.git, unless you added this remote you might need to use `origin`.
```sh
git pull upstream master

guide](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/CONTRIBUTING.md) for instructions

// https://github.com/isaacs/node-glob/blob/master/common.js#L104

\ mini utility to convert [OpenSearch's REST spec](https://github.com/opensearch-project/OpenSearch/blob/master/rest-api-spec) to Console's (OpenSearch Dashboards) autocomplete format.
It is used to semi-manually update Console's autocompletion rules.
### Retrieving the spec
If you don't have a copy of the OpenSearch repo on your machine, follow these steps to clone only the rest API specs
```
mkdir opensearch-spec && cd opensearch-spec
git init
git remote add origin https://github.com/opensearch-project/OpenSearch
git config core.sparsecheckout true
echo "rest-api-spec/src/main/resources/rest-api-spec/api/*\nx-pack/plugin/src/test/resources/rest-api-spec/api/*" > .git/info/sparse-checkout
git pull --depth=1 origin master

* Since https://github.com/elastic/elasticsearch/pull/42346 has been merged into ES master

at /var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/node_modules/selenium-webdriver/lib/webdriver.js:834:17
at process._tickCallback (internal/process/next_tick.js:68:7)
at lastError (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/test/common/services/retry/retry_for_success.ts:28:9)
- at onFailure (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/test/common/services/retry/retry_for_success.ts:68:13)]]›
+ at onFailure (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/test/common/services/retry/retry_for_success.ts:68:13)
‹/failure›
‹/testcase›
‹testcase name="maps app &quot;after all&quot; hook" classname="Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/maps" time="0.179" metadata-json="{&quot;messages&quot;:[&quot;foo&quot;],&quot;screenshots&quot;:[{&quot;name&quot;:&quot;failure[dashboard app using current data dashboard snapshots compare TSVB snapshot]&quot;,&quot;url&quot;:&quot;https://storage.googleapis.com/kibana-ci-artifacts/jobs/elastic+kibana+7.x/1632/kibana-oss-tests/test/functional/screenshots/failure/dashboard%20app%20using%20current%20data%20dashboard%20snapshots%20compare%20TSVB%20snapshot.png&quot;}]}"›
‹system-out›
- ‹![CDATA[[00:00:00] │
+ [00:00:00] │
[00:07:04] └-: maps app
...
-]]›
+
‹/system-out›
‹failure›
- ‹![CDATA[{ NoSuchSessionError: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used.
+ { NoSuchSessionError: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used.
at promise.finally (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/node_modules/selenium-webdriver/lib/webdriver.js:726:38)
at Object.thenFinally [as finally] (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/node_modules/selenium-webdriver/lib/promise.js:124:12)
- at process._tickCallback (internal/process/next_tick.js:68:7) name: 'NoSuchSessionError', remoteStacktrace: '' }]]›
+ at process._tickCallback (internal/process/next_tick.js:68:7) name: 'NoSuchSessionError', remoteStacktrace: '' }
‹/failure›
‹/testcase›
‹testcase name="InfraOps app feature controls infrastructure security global infrastructure all privileges shows infrastructure navlink" classname="Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/infra/feature_controls/infrastructure_security·ts"›
‹system-out›
- ‹![CDATA[[00:00:00] │
+ [00:00:00] │
[00:05:13] └-: InfraOps app
...
-]]›
+
‹/system-out›
‹skipped/›
‹/testcase›
‹testcase name="machine learning anomaly detection saved search with lucene query job creation opens the advanced section" classname="Firefox XPack UI Functional Tests.x-pack/test/functional/apps/machine_learning/anomaly_detection/saved_search_job·ts" time="6.040"›
- ‹system-out›‹![CDATA[[00:21:57] └-: machine learning...]]›‹/system-out›
- ‹failure›‹![CDATA[{ NoSuchSessionError: Tried to run command without establishing a connection
+ ‹system-out›[00:21:57] └-: machine learning...‹/system-out›
+ ‹failure›{ NoSuchSessionError: Tried to run command without establishing a connection
at Object.throwDecodedError (/dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/error.js:550:15)
at parseHttpResponse (/dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/http.js:563:13)
at Executor.execute (/dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/http.js:489:26)
- at process._tickCallback (internal/process/next_tick.js:68:7) name: 'NoSuchSessionError', remoteStacktrace: '' }]]›‹/failure›
+ at process._tickCallback (internal/process/next_tick.js:68:7) name: 'NoSuchSessionError', remoteStacktrace: '' }‹/failure›
‹/testcase›
‹/testsuite›
-‹/testsuites›
+‹/testsuites›
\\ No newline at end of file
`);
});
it('rewrites jest reports with minimal changes', async () => {
const xml = await addMessagesToReport({
report: await parseTestReport(JEST_REPORT),
messages: [
{
classname: 'X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp',
name: 'launcher can reconnect if process died',
message: 'foo bar',
},
],
log,
reportPath: Path.resolve(__dirname, './__fixtures__/jest_report.xml'),
});
expect(createPatch('jest.xml', JEST_REPORT, xml, { context: 0 })).toMatchInlineSnapshot(`
Index: jest.xml
===================================================================
--- jest.xml [object Object]
+++ jest.xml
@@ -3,13 +3,17 @@
‹testsuite name="x-pack/legacy/plugins/code/server/lsp/abstract_launcher.test.ts" timestamp="2019-06-07T03:42:21" time="14.504" tests="5" failures="1" skipped="0" file="/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-intake/node/immutable/kibana/x-pack/legacy/plugins/code/server/lsp/abstract_launcher.test.ts"›
‹testcase classname="X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp" name="launcher can start and end a process" time="1.316"/›
‹testcase classname="X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp" name="launcher can force kill the process if langServer can not exit" time="3.182"/›
‹testcase classname="X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp" name="launcher can reconnect if process died" time="7.060"›
- ‹failure›
- ‹![CDATA[TypeError: Cannot read property '0' of undefined
- at Object.‹anonymous›.test (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-intake/node/immutable/kibana/x-pack/legacy/plugins/code/server/lsp/abstract_launcher.test.ts:166:10)]]›
- ‹/failure›
+ ‹failure›‹![CDATA[
+ TypeError: Cannot read property '0' of undefined
+ at Object.‹anonymous›.test (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-intake/node/immutable/kibana/x-pack/legacy/plugins/code/server/lsp/abstract_launcher.test.ts:166:10)
+ ]]›‹/failure›
+ ‹system-out›Failed Tests Reporter:
+ - foo bar
+
+‹/system-out›
‹/testcase›
‹testcase classname="X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp" name="passive launcher can start and end a process" time="0.435"/›
‹testcase classname="X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp" name="passive launcher should restart a process if a process died before connected" time="1.502"/›
‹/testsuite›
-‹/testsuites›
+‹/testsuites›
\\ No newline at end of file
`);
});
it('rewrites mocha reports with minimal changes', async () => {
const xml = await addMessagesToReport({
report: await parseTestReport(MOCHA_REPORT),
messages: [
{
name: 'code in multiple nodes "before all" hook',
classname: 'X-Pack Mocha Tests.x-pack/legacy/plugins/code/server/__tests__/multi_node·ts',
message: 'foo bar',
},
],
log,
reportPath: Path.resolve(__dirname, './__fixtures__/mocha_report.xml'),
});
expect(createPatch('mocha.xml', MOCHA_REPORT, xml, { context: 0 })).toMatchInlineSnapshot(`
Index: mocha.xml
===================================================================
--- mocha.xml [object Object]
+++ mocha.xml
@@ -1,13 +1,16 @@
‹?xml version="1.0" encoding="utf-8"?›
‹testsuites›
‹testsuite timestamp="2019-06-13T23:29:36" time="30.739" tests="1444" failures="2" skipped="3"›
‹testcase name="code in multiple nodes &quot;before all&quot; hook" classname="X-Pack Mocha Tests.x-pack/legacy/plugins/code/server/__tests__/multi_node·ts" time="0.121"›
- ‹system-out›
- ‹![CDATA[]]›
+ ‹system-out›Failed Tests Reporter:
+ - foo bar
+
+
+
‹/system-out›
- ‹failure›
- ‹![CDATA[Error: Unable to read artifact info from https://artifacts-api.opensearch.org/v1/versions/8.0.0-SNAPSHOT/builds/latest/projects/elasticsearch: Service Temporarily Unavailable
+ ‹failure›‹![CDATA[
+ Error: Unable to read artifact info from https://artifacts-api.opensearch.org/v1/versions/8.0.0-SNAPSHOT/builds/latest/projects/elasticsearch: Service Temporarily Unavailable
‹html›
‹head›‹title›503 Service Temporarily Unavailable‹/title›‹/head›
‹body bgcolor="white"›
‹center›‹h1›503 Service Temporarily Unavailable‹/h1›‹/center›
@@ -15,24 +18,24 @@
‹/body›
‹/html›
at Function.getSnapshot (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-intake/node/immutable/kibana/packages/osd-opensearch/src/artifact.js:95:13)

at /var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/node_modules/selenium-webdriver/lib/webdriver.js:834:17
at process._tickCallback (internal/process/next_tick.js:68:7)
at lastError (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/test/common/services/retry/retry_for_success.ts:28:9)
at onFailure (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/test/common/services/retry/retry_for_success.ts:68:13)
",
"likelyIrrelevant": false,
"name": "maps app maps loaded from sample data ecommerce \\"before all\\" hook",
"time": "154.378",
},
Object {
"classname": "Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/maps",
"failure": "
{ NoSuchSessionError: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used.
at promise.finally (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/node_modules/selenium-webdriver/lib/webdriver.js:726:38)
at Object.thenFinally [as finally] (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/node_modules/selenium-webdriver/lib/promise.js:124:12)
at process._tickCallback (internal/process/next_tick.js:68:7) name: 'NoSuchSessionError', remoteStacktrace: '' }
",
"likelyIrrelevant": true,
"metadata-json": "{\\"messages\\":[\\"foo\\"],\\"screenshots\\":[{\\"name\\":\\"failure[dashboard app using current data dashboard snapshots compare TSVB snapshot]\\",\\"url\\":\\"https://storage.googleapis.com/kibana-ci-artifacts/jobs/elastic+kibana+7.x/1632/kibana-oss-tests/test/functional/screenshots/failure/dashboard%20app%20using%20current%20data%20dashboard%20snapshots%20compare%20TSVB%20snapshot.png\\"}]}",
"name": "maps app \\"after all\\" hook",
"time": "0.179",
},
Object {
"classname": "Firefox XPack UI Functional Tests.x-pack/test/functional/apps/machine_learning/anomaly_detection/saved_search_job·ts",
"failure": "{ NoSuchSessionError: Tried to run command without establishing a connection
at Object.throwDecodedError (/dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/error.js:550:15)
at parseHttpResponse (/dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/http.js:563:13)
at Executor.execute (/dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/http.js:489:26)
at process._tickCallback (internal/process/next_tick.js:68:7) name: 'NoSuchSessionError', remoteStacktrace: '' }",
"likelyIrrelevant": true,
"name": "machine learning anomaly detection saved search with lucene query job creation opens the advanced section",
"time": "6.040",
},
]
`);
});
it('discovers failures in jest report', async () => {
const failures = getFailures(await parseTestReport(JEST_REPORT));
expect(failures).toMatchInlineSnapshot(`
Array [
Object {
"classname": "X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp",
"failure": "
TypeError: Cannot read property '0' of undefined
at Object.<anonymous>.test (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-intake/node/immutable/kibana/x-pack/legacy/plugins/code/server/lsp/abstract_launcher.test.ts:166:10)
",
"likelyIrrelevant": false,
"name": "launcher can reconnect if process died",
"time": "7.060",
},
]
`);
});
it('discovers failures in mocha report', async () => {
const failures = getFailures(await parseTestReport(MOCHA_REPORT));
expect(failures).toMatchInlineSnapshot(`
Array [
Object {
"classname": "X-Pack Mocha Tests.x-pack/legacy/plugins/code/server/__tests__/multi_node·ts",
"failure": "
Error: Unable to read artifact info from https://artifacts-api.opensearch.org/v1/versions/8.0.0-SNAPSHOT/builds/latest/projects/elasticsearch: Service Temporarily Unavailable
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body bgcolor=\\"white\\">
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.13.7</center>
</body>
</html>
at Function.getSnapshot (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-intake/node/immutable/kibana/packages/osd-opensearch/src/artifact.js:95:13)

const isMasterOrVersion = branch === 'master' || branch.match(/^\d+\.(x|\d+)$/);
if (!isMasterOrVersion || isPr) {
log.info('Failure issues only created on master/version branch jobs');

at /var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/node_modules/selenium-webdriver/lib/webdriver.js:834:17
at process._tickCallback (internal/process/next_tick.js:68:7)
at lastError (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/test/common/services/retry/retry_for_success.ts:28:9)
at onFailure (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/test/common/services/retry/retry_for_success.ts:68:13)]]>
</failure>
</testcase>
<testcase name="maps app &quot;after all&quot; hook" classname="Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/maps" time="0.179" metadata-json="{&quot;messages&quot;:[&quot;foo&quot;],&quot;screenshots&quot;:[{&quot;name&quot;:&quot;failure[dashboard app using current data dashboard snapshots compare TSVB snapshot]&quot;,&quot;url&quot;:&quot;https://storage.googleapis.com/kibana-ci-artifacts/jobs/elastic+kibana+7.x/1632/kibana-oss-tests/test/functional/screenshots/failure/dashboard%20app%20using%20current%20data%20dashboard%20snapshots%20compare%20TSVB%20snapshot.png&quot;}]}">
<system-out>
<![CDATA[[00:00:00] │
[00:07:04] └-: maps app
...
]]>
</system-out>
<failure>
<![CDATA[{ NoSuchSessionError: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used.
at promise.finally (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/node_modules/selenium-webdriver/lib/webdriver.js:726:38)
at Object.thenFinally [as finally] (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-ciGroup7/node/immutable/kibana/node_modules/selenium-webdriver/lib/promise.js:124:12)

<testsuite name="x-pack/legacy/plugins/code/server/lsp/abstract_launcher.test.ts" timestamp="2019-06-07T03:42:21" time="14.504" tests="5" failures="1" skipped="0" file="/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-intake/node/immutable/kibana/x-pack/legacy/plugins/code/server/lsp/abstract_launcher.test.ts">
<testcase classname="X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp" name="launcher can start and end a process" time="1.316"/>
<testcase classname="X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp" name="launcher can force kill the process if langServer can not exit" time="3.182"/>
<testcase classname="X-Pack Jest Tests.x-pack/legacy/plugins/code/server/lsp" name="launcher can reconnect if process died" time="7.060">
<failure>
<![CDATA[TypeError: Cannot read property '0' of undefined
at Object.<anonymous>.test (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-intake/node/immutable/kibana/x-pack/legacy/plugins/code/server/lsp/abstract_launcher.test.ts:166:10)]]>

at Function.getSnapshot (/var/lib/jenkins/workspace/elastic+kibana+master/JOB/x-pack-intake/node/immutable/kibana/packages/osd-opensearch/src/artifact.js:95:13)

- [Changelog](https://github.com/Stuk/jszip/blob/master/CHANGES.md)

* Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)

* Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)

// https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/legacy/server/config/schema.js

echo "the OpenSearch Dashboards branch (first on your fork and then upstream) before building from master."

// percy snapshots always target pkg.branch, so that feature branches can be based on master/7.x/etc.

// https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/core/server/http/http_config.ts

Many example embeddables are implemented and registered [here](https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/examples/embeddable_examples). They can be played around with and explored [in the Embeddable Explorer example plugin](https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/examples/embeddable_explorer). Just run OpenSearch Dashboards with
```
yarn start --run-examples
```
and navigate to the Embeddable explorer app.
There is also an example of rendering dashboard container outside of dashboard app [here](https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/examples/dashboard_embeddable_examples).

See the [OpenSearch Dashboards contributing guide](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/CONTRIBUTING.md) for instructions setting up your development environment.

* Refer to {@link https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/plugins/opensearch_dashboards_utils/docs/state_containers/react.md | guide} for details

* Refer to {@link https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/opensearch_dashboards_utils/docs/state_containers | guides and examples} for more info

* [api reference](https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/opensearch_dashboards_utils/docs/state_containers)

* They are designed to work together with {@link https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/opensearch_dashboards_utils/docs/state_containers | state containers}. But state containers are not required.
*
* State syncing utilities include:
*
* *{@link syncState} util which:
* * Subscribes to state changes and pushes them to state storage.
* * Optionally subscribes to state storage changes and pushes them to state.
* * Two types of storages compatible with `syncState`:
* * {@link IOsdUrlStateStorage} - Serializes state and persists it to URL's query param in rison or hashed format.
* Listens for state updates in the URL and pushes them back to state.
* * {@link ISessionStorageStateStorage} - Serializes state and persists it to browser storage.
*
* Refer {@link https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/opensearch_dashboards_utils/docs/state_sync | here} for a complete guide and examples.

- [api reference](https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/opensearch_dashboards_utils/docs/state_sync)

* Go {@link https://github.com/opensearch-project/OpenSearch-Dashboards/tree/master/src/plugins/opensearch_dashboards_utils/docs/state_sync | here} for a complete guide and examples.

* {@link https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/plugins/opensearch_dashboards_utils/docs/state_sync/storages/osd_url_storage.md | Refer to this guide for more info}

* {@link https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/plugins/opensearch_dashboards_utils/docs/state_sync/storages/session_storage.md | guide}
* @public
*/
export interface ISessionStorageStateStorage extends IStateStorage {
set: <State>(key: string, state: State) => void;
get: <State = unknown>(key: string) => State | null;
}
/**
* TODO: Update link
* Creates {@link ISessionStorageStateStorage}
* {@link https://github.com/opensearch-project/OpenSearch-Dashboards/blob/master/src/plugins/opensearch_dashboards_utils/docs/state_sync/storages/session_storage.md | guide}

/** The React Router **initial entries** setting ([see documentation](https://github.com/ReactTraining/react-router/blob/master/packages/react-router/docs/api/MemoryRouter.md)) */
initialEntries?: string[];
/** The React Router **initial index** setting ([see documentation](https://github.com/ReactTraining/react-router/blob/master/packages/react-router/docs/api/MemoryRouter.md)) */

CMD git pull origin master && ./build_docs.pl --doc /home/kibana/ascii_docs/index.asciidoc --out /home/kibana/html_docs --chunk=1

Comments/variables

async function getLicenseFromLocalOrMaster(opensearchClient: OpenSearchClient) {
// Fetching the local license is cheaper than getting it from the master node and good enough
const { license } = await fetchLicense(opensearchClient, true).catch(async (err) => {
if (cachedLicense) {
try {
// Fallback to the master node's license info
const response = await fetchLicense(opensearchClient, false);
return response;
} catch (masterError) {
if ([400, 404].includes(masterError.statusCode)) {
// If the master node does not have a license, we can assume there is no license
cachedLicense = undefined;
} else {
throw err;
}
}
}
return { license: void 0 };
});
if (license) {
cachedLicense = license;
}
return license;
}
export const getLocalLicense: LicenseGetter = async (clustersDetails, { opensearchClient }) => {
const license = await getLicenseFromLocalOrMaster(opensearchClient);

* Using _cluster/state/nodes to retrieve the cluster_id of each node from master node which is considered to be a lightweight operation
* else if the nodes have different cluster_ids then fan out the request to all nodes
* else there are no nodes in the cluster
*/
const sharedClusterId =
state.body.nodes.length > 0
? get(state.body.nodes[0], `attributes.${healthcheckAttributeName}`, null)
: null;
return sharedClusterId === null ||
state.body.nodes.find(
(node: any) => sharedClusterId !== get(node, `attributes.${healthcheckAttributeName}`, null)
)
? null
: '_local';
} catch (e) {
return null;
}
};
export interface PollOpenSearchNodesVersionOptions {
internalClient: OpenSearchClient;
optimizedHealthcheckId?: string;
log: Logger;
opensearchDashboardsVersion: string;
ignoreVersionMismatch: boolean;
opensearchVersionCheckInterval: number;
}
interface NodeInfo {
version: string;
ip: string;
http?: {
publish_address: string;
};
name: string;
}
export interface NodesInfo {
nodes: {
[key: string]: NodeInfo;
};
}
export interface NodesVersionCompatibility {
isCompatible: boolean;
message?: string;
incompatibleNodes: NodeInfo[];
warningNodes: NodeInfo[];
opensearchDashboardsVersion: string;
nodesInfoRequestError?: Error;
}
function getHumanizedNodeName(node: NodeInfo) {
const publishAddress = node?.http?.publish_address + ' ' || '';
return 'v' + node.version + ' @ ' + publishAddress + '(' + node.ip + ')';
}
export function mapNodesVersionCompatibility(
nodesInfo: NodesInfo,
opensearchDashboardsVersion: string,
ignoreVersionMismatch: boolean
): NodesVersionCompatibility {
if (Object.keys(nodesInfo.nodes ?? {}).length === 0) {
return {
isCompatible: false,
message: 'Unable to retrieve version information from OpenSearch nodes.',
incompatibleNodes: [],
warningNodes: [],
opensearchDashboardsVersion,
};
}
const nodes = Object.keys(nodesInfo.nodes)
.sort() // Sorting ensures a stable node ordering for comparison
.map((key) => nodesInfo.nodes[key])
.map((node) => Object.assign({}, node, { name: getHumanizedNodeName(node) }));
// Aggregate incompatible OpenSearch nodes.
const incompatibleNodes = nodes.filter(
(node) =>
!opensearchVersionCompatibleWithOpenSearchDashboards(
node.version,
opensearchDashboardsVersion
)
);
// Aggregate OpenSearch nodes which should prompt a OpenSearch Dashboards upgrade. It's acceptable
// if OpenSearch and OpenSearch Dashboards versions are not the same as long as they are not
// incompatible, but we should warn about it.
// Ignore version qualifiers https://github.com/elastic/elasticsearch/issues/36859
const warningNodes = nodes.filter(
(node) =>
!opensearchVersionEqualsOpenSearchDashboards(node.version, opensearchDashboardsVersion)
);
// Note: If incompatible and warning nodes are present `message` only contains
// an incompatibility notice.
let message;
if (incompatibleNodes.length > 0) {
const incompatibleNodeNames = incompatibleNodes.map((node) => node.name).join(', ');
if (ignoreVersionMismatch) {
message = `Ignoring version incompatibility between OpenSearch Dashboards v${opensearchDashboardsVersion} and the following OpenSearch nodes: ${incompatibleNodeNames}`;
} else {
message = `This version of OpenSearch Dashboards (v${opensearchDashboardsVersion}) is incompatible with the following OpenSearch nodes in your cluster: ${incompatibleNodeNames}`;
}
} else if (warningNodes.length > 0) {
const warningNodeNames = warningNodes.map((node) => node.name).join(', ');
message =
`You're running OpenSearch Dashboards ${opensearchDashboardsVersion} with some different versions of ` +
'OpenSearch. Update OpenSearch Dashboards or OpenSearch to the same ' +
`version to prevent compatibility issues: ${warningNodeNames}`;
}
return {
isCompatible: ignoreVersionMismatch || incompatibleNodes.length === 0,
message,
incompatibleNodes,
warningNodes,
opensearchDashboardsVersion,
};
}
// Returns true if two NodesVersionCompatibility entries match
function compareNodes(prev: NodesVersionCompatibility, curr: NodesVersionCompatibility) {
const nodesEqual = (n: NodeInfo, m: NodeInfo) => n.ip === m.ip && n.version === m.version;
return (
curr.isCompatible === prev.isCompatible &&
curr.incompatibleNodes.length === prev.incompatibleNodes.length &&
curr.warningNodes.length === prev.warningNodes.length &&
curr.incompatibleNodes.every((node, i) => nodesEqual(node, prev.incompatibleNodes[i])) &&
curr.warningNodes.every((node, i) => nodesEqual(node, prev.warningNodes[i]))
);
}
export const pollOpenSearchNodesVersion = ({
internalClient,
optimizedHealthcheckId,
log,
opensearchDashboardsVersion,
ignoreVersionMismatch,
opensearchVersionCheckInterval: healthCheckInterval,
}: PollOpenSearchNodesVersionOptions): Observable<NodesVersionCompatibility> => {
log.debug('Checking OpenSearch version');
return timer(0, healthCheckInterval).pipe(
exhaustMap(() => {
/*
* Originally, Dashboards queries OpenSearch cluster to get the version info of each node and check the version compatibility with each node.
* The /nodes request could fail even one node in cluster fail to response
* For better dashboards resilience, the behaviour is changed to only query the local node when all the nodes have the same cluster_id
* Using _cluster/state/nodes to retrieve the cluster_id of each node from the master node

@ananzh
Copy link
Member

ananzh commented Apr 21, 2022

The nodejs client "Master" deprecation is summarized in opensearch-project/opensearch-js#221

@kavilla kavilla added v2.1.0 and removed v2.0.0 labels Apr 29, 2022
@tmarkley tmarkley removed their assignment May 16, 2022
@kavilla
Copy link
Member

kavilla commented Jun 29, 2022

Master has been deprecated. Closing this issue. New issue to track the removal of these deprecated terms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants