Skip to content

Commit

Permalink
Merge branch 'master' of github.com:elastic/kibana into dev/state-man…
Browse files Browse the repository at this point in the history
…agment/hashed-item-store
  • Loading branch information
Dosant committed Dec 5, 2019
2 parents 6a95be9 + abaf009 commit ee7ad45
Show file tree
Hide file tree
Showing 318 changed files with 10,131 additions and 3,145 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ disabledPlugins
webpackstats.json
/config/*
!/config/kibana.yml
!/config/apm.js
coverage
selenium
.babel_register_cache.json
Expand Down
18 changes: 17 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ A high level overview of our contributing guidelines.
- [Internationalization](#internationalization)
- [Testing and Building](#testing-and-building)
- [Debugging server code](#debugging-server-code)
- [Instrumenting with Elastic APM](#instrumenting-with-elastic-apm)
- [Debugging Unit Tests](#debugging-unit-tests)
- [Unit Testing Plugins](#unit-testing-plugins)
- [Cross-browser compatibility](#cross-browser-compatibility)
Expand Down Expand Up @@ -374,6 +375,21 @@ macOS users on a machine with a discrete graphics card may see significant speed
### Debugging Server Code
`yarn debug` will start the server with Node's inspect flag. Kibana's development mode will start three processes on ports `9229`, `9230`, and `9231`. Chrome's developer tools need to be configured to connect to all three connections. Add `localhost:<port>` for each Kibana process in Chrome's developer tools connection tab.

### Instrumenting with Elastic APM
Kibana ships with the [Elastic APM Node.js Agent](https://github.com/elastic/apm-agent-nodejs) built-in for debugging purposes.

Its default configuration is meant to be used by core Kibana developers only, but it can easily be re-configured to your needs.
In its default configuration it's disabled and will, once enabled, send APM data to a centrally managed Elasticsearch cluster accessible only to Elastic employees.

To change the location where data is sent, use the [`serverUrl`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html#server-url) APM config option.
To activate the APM agent, use the [`active`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html#active) APM config option.

All config options can be set either via environment variables, or by creating an appropriate config file under `config/apm.dev.js`.
For more information about configuring the APM agent, please refer to [the documentation](https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuring-the-agent.html).

Once the agent is active, it will trace all incoming HTTP requests to Kibana, monitor for errors, and collect process-level metrics.
The collected data will be sent to the APM Server and is viewable in the APM UI in Kibana.

### Unit testing frameworks
Kibana is migrating unit testing from Mocha to Jest. Legacy unit tests still
exist in Mocha but all new unit tests should be written in Jest. Mocha tests
Expand All @@ -389,7 +405,7 @@ The following table outlines possible test file locations and how to invoke them
| Jest | `src/**/*.test.js`<br>`src/**/*.test.ts` | `node scripts/jest -t regexp [test path]` |
| Jest (integration) | `**/integration_tests/**/*.test.js` | `node scripts/jest_integration -t regexp [test path]` |
| Mocha | `src/**/__tests__/**/*.js`<br>`!src/**/public/__tests__/*.js`<br>`packages/kbn-datemath/test/**/*.js`<br>`packages/kbn-dev-utils/src/**/__tests__/**/*.js`<br>`tasks/**/__tests__/**/*.js` | `node scripts/mocha --grep=regexp [test path]` |
| Functional | `test/*integration/**/config.js`<br>`test/*functional/**/config.js` | `node scripts/functional_tests_server --config test/[directory]/config.js`<br>`node scripts/functional_test_runner --config test/[directory]/config.js --grep=regexp` |
| Functional | `test/*integration/**/config.js`<br>`test/*functional/**/config.js`<br>`test/accessibility/config.js` | `node scripts/functional_tests_server --config test/[directory]/config.js`<br>`node scripts/functional_test_runner --config test/[directory]/config.js --grep=regexp` |
| Karma | `src/**/public/__tests__/*.js` | `npm run test:dev` |

For X-Pack tests located in `x-pack/` see [X-Pack Testing](x-pack/README.md#testing)
Expand Down
81 changes: 81 additions & 0 deletions config/apm.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

/**
* DO NOT EDIT THIS FILE!
*
* This file contains the configuration for the Elastic APM instrumentaion of
* Kibana itself and is only intented to be used during development of Kibana.
*
* Instrumentation is turned off by default. Once activated it will send APM
* data to an Elasticsearch cluster accessible by Elastic employees.
*
* To modify the configuration, either use environment variables, or create a
* file named `config/apm.dev.js`, which exports a config object as described
* in the docs.
*
* For an overview over the available configuration files, see:
* https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html
*
* For general information about Elastic APM, see:
* https://www.elastic.co/guide/en/apm/get-started/current/index.html
*/

const { readFileSync } = require('fs');
const { join } = require('path');
const { execSync } = require('child_process');
const merge = require('lodash.merge');

module.exports = merge({
active: false,
serverUrl: 'https://f1542b814f674090afd914960583265f.apm.us-central1.gcp.cloud.es.io:443',
// The secretToken below is intended to be hardcoded in this file even though
// it makes it public. This is not a security/privacy issue. Normally we'd
// instead disable the need for a secretToken in the APM Server config where
// the data is transmitted to, but due to how it's being hosted, it's easier,
// for now, to simply leave it in.
secretToken: 'R0Gjg46pE9K9wGestd',
globalLabels: {},
centralConfig: false,
logUncaughtExceptions: true
}, devConfig());

const rev = gitRev();
if (rev !== null) module.exports.globalLabels.git_rev = rev;

try {
const filename = join(__dirname, '..', 'data', 'uuid');
module.exports.globalLabels.kibana_uuid = readFileSync(filename, 'utf-8');
} catch (e) {} // eslint-disable-line no-empty

function gitRev() {
try {
return execSync('git rev-parse --short HEAD', { encoding: 'utf-8' }).trim();
} catch (e) {
return null;
}
}

function devConfig() {
try {
return require('./apm.dev'); // eslint-disable-line import/no-unresolved
} catch (e) {
return {};
}
}
3 changes: 2 additions & 1 deletion docs/developer/core/development-functional-tests.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ When run without any arguments the `FunctionalTestRunner` automatically loads th
* `--config test/functional/config.js` starts Elasticsearch and Kibana servers with the WebDriver tests configured to run in Chrome.
* `--config test/functional/config.firefox.js` starts Elasticsearch and Kibana servers with the WebDriver tests configured to run in Firefox.
* `--config test/api_integration/config.js` starts Elasticsearch and Kibana servers with the api integration tests configuration.
* `--config test/accessibility/config.ts` starts Elasticsearch and Kibana servers with the WebDriver tests configured to run an accessibility audit using https://www.deque.com/axe/[axe].

There are also command line flags for `--bail` and `--grep`, which behave just like their mocha counterparts. For instance, use `--grep=foo` to run only tests that match a regular expression.

Expand Down Expand Up @@ -362,7 +363,7 @@ Full list of services that are used in functional tests can be found here: {blob
** Source: {blob}test/functional/services/remote/remote.ts[test/functional/services/remote/remote.ts]
** Instance of https://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/index_exports_WebDriver.html[WebDriver] class
** Responsible for all communication with the browser
** To perform browser actions, use `remote` service
** To perform browser actions, use `remote` service
** For searching and manipulating with DOM elements, use `testSubjects` and `find` services
** See the https://seleniumhq.github.io/selenium/docs/api/javascript/[selenium-webdriver docs] for the full API.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ Search for objects
<b>Signature:</b>

```typescript
find: <T extends SavedObjectAttributes>(options: Pick<SavedObjectFindOptionsServer, "search" | "filter" | "type" | "fields" | "searchFields" | "defaultSearchOperator" | "hasReference" | "sortField" | "page" | "perPage">) => Promise<SavedObjectsFindResponsePublic<T>>;
find: <T extends SavedObjectAttributes>(options: Pick<SavedObjectFindOptionsServer, "search" | "filter" | "type" | "page" | "fields" | "searchFields" | "defaultSearchOperator" | "hasReference" | "sortField" | "perPage">) => Promise<SavedObjectsFindResponsePublic<T>>;
```
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ export declare class SavedObjectsClient
| [bulkGet](./kibana-plugin-public.savedobjectsclient.bulkget.md) | | <code>(objects?: {</code><br/><code> id: string;</code><br/><code> type: string;</code><br/><code> }[]) =&gt; Promise&lt;SavedObjectsBatchResponse&lt;SavedObjectAttributes&gt;&gt;</code> | Returns an array of objects by id |
| [create](./kibana-plugin-public.savedobjectsclient.create.md) | | <code>&lt;T extends SavedObjectAttributes&gt;(type: string, attributes: T, options?: SavedObjectsCreateOptions) =&gt; Promise&lt;SimpleSavedObject&lt;T&gt;&gt;</code> | Persists an object |
| [delete](./kibana-plugin-public.savedobjectsclient.delete.md) | | <code>(type: string, id: string) =&gt; Promise&lt;{}&gt;</code> | Deletes an object |
| [find](./kibana-plugin-public.savedobjectsclient.find.md) | | <code>&lt;T extends SavedObjectAttributes&gt;(options: Pick&lt;SavedObjectFindOptionsServer, &quot;search&quot; &#124; &quot;filter&quot; &#124; &quot;type&quot; &#124; &quot;fields&quot; &#124; &quot;searchFields&quot; &#124; &quot;defaultSearchOperator&quot; &#124; &quot;hasReference&quot; &#124; &quot;sortField&quot; &#124; &quot;page&quot; &#124; &quot;perPage&quot;&gt;) =&gt; Promise&lt;SavedObjectsFindResponsePublic&lt;T&gt;&gt;</code> | Search for objects |
| [find](./kibana-plugin-public.savedobjectsclient.find.md) | | <code>&lt;T extends SavedObjectAttributes&gt;(options: Pick&lt;SavedObjectFindOptionsServer, &quot;search&quot; &#124; &quot;filter&quot; &#124; &quot;type&quot; &#124; &quot;page&quot; &#124; &quot;fields&quot; &#124; &quot;searchFields&quot; &#124; &quot;defaultSearchOperator&quot; &#124; &quot;hasReference&quot; &#124; &quot;sortField&quot; &#124; &quot;perPage&quot;&gt;) =&gt; Promise&lt;SavedObjectsFindResponsePublic&lt;T&gt;&gt;</code> | Search for objects |
| [get](./kibana-plugin-public.savedobjectsclient.get.md) | | <code>&lt;T extends SavedObjectAttributes&gt;(type: string, id: string) =&gt; Promise&lt;SimpleSavedObject&lt;T&gt;&gt;</code> | Fetches a single object |

## Methods
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@

```typescript
config: {
legacy: {
globalConfig$: Observable<SharedGlobalConfig>;
};
create: <T = ConfigSchema>() => Observable<T>;
createIfExists: <T = ConfigSchema>() => Observable<T | undefined>;
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ export interface PluginInitializerContext<ConfigSchema = unknown>

| Property | Type | Description |
| --- | --- | --- |
| [config](./kibana-plugin-server.plugininitializercontext.config.md) | <code>{</code><br/><code> create: &lt;T = ConfigSchema&gt;() =&gt; Observable&lt;T&gt;;</code><br/><code> createIfExists: &lt;T = ConfigSchema&gt;() =&gt; Observable&lt;T &#124; undefined&gt;;</code><br/><code> }</code> | |
| [config](./kibana-plugin-server.plugininitializercontext.config.md) | <code>{</code><br/><code> legacy: {</code><br/><code> globalConfig$: Observable&lt;SharedGlobalConfig&gt;;</code><br/><code> };</code><br/><code> create: &lt;T = ConfigSchema&gt;() =&gt; Observable&lt;T&gt;;</code><br/><code> createIfExists: &lt;T = ConfigSchema&gt;() =&gt; Observable&lt;T &#124; undefined&gt;;</code><br/><code> }</code> | |
| [env](./kibana-plugin-server.plugininitializercontext.env.md) | <code>{</code><br/><code> mode: EnvironmentMode;</code><br/><code> packageInfo: Readonly&lt;PackageInfo&gt;;</code><br/><code> }</code> | |
| [logger](./kibana-plugin-server.plugininitializercontext.logger.md) | <code>LoggerFactory</code> | |
| [opaqueId](./kibana-plugin-server.plugininitializercontext.opaqueid.md) | <code>PluginOpaqueId</code> | |
Expand Down
10 changes: 7 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
"typespec": "typings-tester --config x-pack/legacy/plugins/canvas/public/lib/aeroelastic/tsconfig.json x-pack/legacy/plugins/canvas/public/lib/aeroelastic/__fixtures__/typescript/typespec_tests.ts",
"checkLicenses": "node scripts/check_licenses --dev",
"build": "node scripts/build --all-platforms",
"start": "node --trace-warnings --trace-deprecation scripts/kibana --dev ",
"start": "node --trace-warnings --throw-deprecation scripts/kibana --dev",
"debug": "node --nolazy --inspect scripts/kibana --dev",
"debug-break": "node --nolazy --inspect-brk scripts/kibana --dev",
"karma": "karma start",
Expand Down Expand Up @@ -159,6 +159,7 @@
"d3-cloud": "1.2.5",
"deepmerge": "^4.2.2",
"del": "^5.1.0",
"elastic-apm-node": "^3.2.0",
"elasticsearch": "^16.5.0",
"elasticsearch-browser": "^16.5.0",
"encode-uri-query": "1.0.1",
Expand Down Expand Up @@ -203,6 +204,7 @@
"minimatch": "^3.0.4",
"moment": "^2.24.0",
"moment-timezone": "^0.5.27",
"monaco-editor": "~0.17.0",
"mustache": "2.3.2",
"ngreact": "0.5.1",
"node-fetch": "1.7.3",
Expand All @@ -221,7 +223,9 @@
"react-grid-layout": "^0.16.2",
"react-input-range": "^1.3.0",
"react-markdown": "^3.4.1",
"react-monaco-editor": "~0.27.0",
"react-redux": "^5.1.2",
"react-resize-detector": "^4.2.0",
"react-router-dom": "^4.3.1",
"react-sizeme": "^2.3.6",
"reactcss": "1.2.3",
Expand Down Expand Up @@ -334,6 +338,7 @@
"@types/react": "^16.9.11",
"@types/react-dom": "^16.9.4",
"@types/react-redux": "^6.0.6",
"@types/react-resize-detector": "^4.0.1",
"@types/react-router-dom": "^4.3.1",
"@types/react-virtualized": "^9.18.7",
"@types/redux": "^3.6.31",
Expand Down Expand Up @@ -405,7 +410,6 @@
"gulp-sourcemaps": "2.6.5",
"has-ansi": "^3.0.0",
"iedriver": "^3.14.1",
"image-diff": "1.6.3",
"intl-messageformat-parser": "^1.4.0",
"is-path-inside": "^2.1.0",
"istanbul-instrumenter-loader": "3.0.1",
Expand Down Expand Up @@ -433,7 +437,7 @@
"node-sass": "^4.9.4",
"normalize-path": "^3.0.0",
"nyc": "^14.1.1",
"pixelmatch": "4.0.2",
"pixelmatch": "^5.1.0",
"pkg-up": "^2.0.0",
"pngjs": "^3.4.0",
"postcss": "^7.0.5",
Expand Down
17 changes: 11 additions & 6 deletions packages/kbn-babel-code-parser/src/strategies.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
import { canRequire } from './can_require';
import { dependenciesVisitorsGenerator } from './visitors';
import { dirname, isAbsolute, resolve } from 'path';
import { builtinModules } from 'module';

export function _calculateTopLevelDependency(inputDep, outputDep = '') {
// The path separator will be always the forward slash
Expand Down Expand Up @@ -48,14 +49,18 @@ export function _calculateTopLevelDependency(inputDep, outputDep = '') {
return _calculateTopLevelDependency(depSplitPaths.join(pathSeparator), outputDep);
}

export async function dependenciesParseStrategy(cwd, parseSingleFile, mainEntry, wasParsed, results) {
// Retrieve native nodeJS modules
const natives = process.binding('natives');

export async function dependenciesParseStrategy(
cwd,
parseSingleFile,
mainEntry,
wasParsed,
results
) {
// Get dependencies from a single file and filter
// out node native modules from the result
const dependencies = (await parseSingleFile(mainEntry, dependenciesVisitorsGenerator))
.filter(dep => !natives[dep]);
const dependencies = (await parseSingleFile(mainEntry, dependenciesVisitorsGenerator)).filter(
dep => !builtinModules.includes(dep)
);

// Return the list of all the new entries found into
// the current mainEntry that we could use to look for
Expand Down
12 changes: 1 addition & 11 deletions packages/kbn-plugin-helpers/tasks/build/rewrite_package_json.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,19 +41,9 @@ module.exports = function rewritePackage(buildSource, buildVersion, kibanaVersio
delete pkg.scripts;
delete pkg.devDependencies;

file.contents = toBuffer(JSON.stringify(pkg, null, 2));
file.contents = Buffer.from(JSON.stringify(pkg, null, 2));
}

return file;
});
};

function toBuffer(string) {
if (typeof Buffer.from === 'function') {
return Buffer.from(string, 'utf8');
} else {
// this was deprecated in node v5 in favor
// of Buffer.from(string, encoding)
return new Buffer(string, 'utf8');
}
}
1 change: 1 addition & 0 deletions scripts/kibana.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,6 @@
* under the License.
*/

require('../src/apm')(process.env.ELASTIC_APM_PROXY_SERVICE_NAME || 'kibana-proxy');
require('../src/setup_node_env');
require('../src/cli/cli');
39 changes: 39 additions & 0 deletions src/apm.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

const { existsSync } = require('fs');
const { join } = require('path');
const { name, version } = require('../package.json');

module.exports = function (serviceName = name) {
if (process.env.kbnWorkerType === 'optmzr') return;

const conf = {
serviceName: `${serviceName}-${version.replace(/\./g, '_')}`
};

if (configFileExists()) conf.configFile = 'config/apm.js';
else conf.active = false;

require('elastic-apm-node').start(conf);
};

function configFileExists() {
return existsSync(join(__dirname, '..', 'config', 'apm.js'));
}
2 changes: 1 addition & 1 deletion src/cli/cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
*/

import _ from 'lodash';
import { pkg } from '../legacy/utils';
import { pkg } from '../core/server/utils';
import Command from './command';
import serveCommand from './serve/serve';

Expand Down
2 changes: 1 addition & 1 deletion src/cli/cluster/cluster_manager.js
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ export default class ClusterManager {

setupWatching(extraPaths, pluginInternalDirsIgnore) {
const chokidar = require('chokidar');
const { fromRoot } = require('../../legacy/utils');
const { fromRoot } = require('../../core/server/utils');

const watchPaths = [
fromRoot('src/core'),
Expand Down
3 changes: 2 additions & 1 deletion src/cli/cluster/worker.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ import _ from 'lodash';
import cluster from 'cluster';
import { EventEmitter } from 'events';

import { BinderFor, fromRoot } from '../../legacy/utils';
import { BinderFor } from '../../legacy/utils';
import { fromRoot } from '../../core/server/utils';

const cliPath = fromRoot('src/cli');
const baseArgs = _.difference(process.argv.slice(2), ['--no-watch']);
Expand Down
1 change: 1 addition & 0 deletions src/cli/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,6 @@
* under the License.
*/

require('../apm')();
require('../setup_node_env');
require('./cli');
Loading

0 comments on commit ee7ad45

Please sign in to comment.