Merge remote-tracking branch 'upstream/master' into feature/lens

.github/CODEOWNERS

@@ -19,6 +19,11 @@
 # Machine Learning
 /x-pack/plugins/ml/  @elastic/ml-ui
 
+# Operations
+/src/dev/ @elastic/kibana-operations
+/src/setup_node_env/ @elastic/kibana-operations
+/src/optimize/ @elastic/kibana-operations
+
 # Platform
 /src/core/  @elastic/kibana-platform
 

CONTRIBUTING.md

@@ -356,7 +356,7 @@ Test runner arguments:
     `node scripts/jest -t 'stops both admin and data clients' src/core/server/elasticsearch/elasticsearch_service.test.ts`
   - Run the api integration test case whose description matches the given string:
     `node scripts/functional_tests_server --config test/api_integration/config.js`
-    `node scripts/functional_tests_runner --config test/api_integration/config.js --grep='should return 404 if id does not match any sample data sets'`
+    `node scripts/functional_test_runner --config test/api_integration/config.js --grep='should return 404 if id does not match any sample data sets'`
 
 ### Debugging Unit Tests
 

docs/dev-tools/grokdebugger/index.asciidoc

@@ -1,25 +1,119 @@
 [role="xpack"]
 [[xpack-grokdebugger]]
-== Debugging Grok Expressions
+== Debugging grok expressions
 
-Grok is a pattern matching syntax that you can use to parse arbitrary text and
-structure it. Grok is perfect for parsing syslog logs, apache and other
-webserver logs, mysql logs, and in general, any log format that is generally
-written for humans and not computer consumption.
+You can build and debug grok patterns in the {kib} *Grok Debugger*
+before you use them in your data processing pipelines. Grok is a pattern 
+matching syntax that you can use to parse arbitrary text and
+structure it. Grok is good for parsing syslog, apache, and other
+webserver logs, mysql logs, and in general, any log format that is
+written for human consumption. 
 
 Grok patterns are supported in the ingest node
 {ref}/grok-processor.html[grok processor] and the Logstash
-{logstash-ref}/plugins-filters-grok.html[grok filter]. The Elastic Stack ships
-with over 120 reusable grok patterns. See
-https://github.com/elastic/elasticsearch/tree/master/modules/ingest-common/src/main/resources/patterns[Ingest node grok patterns] and https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns[Logstash grok patterns]
-for the full list of patterns.
+{logstash-ref}/plugins-filters-grok.html[grok filter]. See 
+{logstash-ref}/plugins-filters-grok.html#_grok_basics[grok basics]
+for more information on the syntax for a grok pattern. 
 
-You can build and debug grok patterns in the Grok Debugger tool in {kib}
-before you use them in your data processing pipelines. Because
+The Elastic Stack ships
+with more than 120 reusable grok patterns. See
+https://github.com/elastic/elasticsearch/tree/master/libs/grok/src/main/resources/patterns[Ingest node grok patterns] and https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns[Logstash grok patterns]
+for the complete list of patterns.
+
+Because
 ingest node and Logstash share the same grok implementation and pattern
-libraries, any grok pattern that you create in the Grok Debugger will work
+libraries, any grok pattern that you create in the *Grok Debugger* will work
 in ingest node and Logstash.
 
+[float]
+[[grokdebugger-getting-started]]
+=== Getting started with the Grok Debugger
+
+This example walks you through using the *Grok Debugger*. This tool
+is automatically enabled in {kib}. 
+
+NOTE: If you're using {security}, you must have the `manage_pipeline`
+permission to use the Grok Debugger.
+
+. In the side navigation, click *DevTools*, then open the *Grok Debugger*. 
+. In *Sample Data*, enter a message that is representative of the data that you
+want to parse. For example:
++
+[source,ruby]
+-------------------------------------------------------------------------------
+55.3.244.1 GET /index.html 15824 0.043
+-------------------------------------------------------------------------------
+
+. In *Grok Pattern*, enter the grok pattern that you want to apply to the data.
++
+To parse the log line in this example, use:
++
+[source,ruby]
+-------------------------------------------------------------------------------
+%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}
+-------------------------------------------------------------------------------
+
+. Click **Simulate**.
++
+You'll see the simulated event that results from applying the grok
+pattern.
++
+[role="screenshot"]
 image::dev-tools/grokdebugger/images/grok-debugger-overview.png["Grok Debugger"]
 
-include::getting-started.asciidoc[]
+
+//TODO: Update LS and ingest node docs with pointers to the new grok debugger. Replace references to the Heroku app.
+
+[float]
+[[grokdebugger-custom-patterns]]
+=== Testing custom patterns
+
+If the default grok pattern dictionary doesn't contain the patterns you need,
+you can define, test, and debug custom patterns using the Grok Debugger.
+
+Custom patterns that you enter in the Grok Debugger are not saved. Custom patterns
+are only available for the current debugging session and have no side effects.
+
+Follow this example to define a custom pattern.
+
+. In *Sample Data*, enter the following sample message:
++
+[source,ruby]
+-------------------------------------------------------------------------------
+Jan  1 06:25:43 mailserver14 postfix/cleanup[21403]: BEF25A72965: message-id=<[email protected]>
+-------------------------------------------------------------------------------
+
+. Enter this grok pattern:
++
+[source,ruby]
+-------------------------------------------------------------------------------
+%{SYSLOGBASE} %{POSTFIX_QUEUEID:queue_id}: %{MSG:syslog_message}
+-------------------------------------------------------------------------------
++
+Notice that the grok pattern references custom patterns called `POSTFIX_QUEUEID`
+and `MSG`.
+
+. Expand **Custom Patterns** and enter pattern definitions for the custom
+patterns that you want to use in the grok expression. You must specify each pattern definition
+on its own line.
++
+For this example, you must specify pattern definitions
+for `POSTFIX_QUEUEID` and `MSG`:
++
+[source,ruby]
+-------------------------------------------------------------------------------
+POSTFIX_QUEUEID [0-9A-F]{10,11}
+MSG message-id=<%{GREEDYDATA}>
+-------------------------------------------------------------------------------
+
+. Click **Simulate**.
++
+You'll see the simulated output event that results from applying
+the grok pattern that contains the custom pattern:
++
+[role="screenshot"]
+image::dev-tools/grokdebugger/images/grok-debugger-custom-pattern.png["Debugging a custom pattern"]
++
+If an error occurs, you can continue iterating over
+the custom pattern until the output matches the event
+that you expect.

docs/development/core/public/kibana-plugin-public.coresetup.md

@@ -4,7 +4,7 @@
 
 ## CoreSetup interface
 
-Core services exposed to the start lifecycle
+Core services exposed to the setup lifecycle
 
 <b>Signature:</b>
 

docs/development/core/public/kibana-plugin-public.corestart.http.md

@@ -0,0 +1,13 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index) &gt; [kibana-plugin-public](./kibana-plugin-public.md) &gt; [CoreStart](./kibana-plugin-public.corestart.md) &gt; [http](./kibana-plugin-public.corestart.http.md)
+
+## CoreStart.http property
+
+[HttpStart](./kibana-plugin-public.httpstart.md)
+
+<b>Signature:</b>
+
+```typescript
+http: HttpStart;
+```

docs/development/core/public/kibana-plugin-public.corestart.md

@@ -4,6 +4,8 @@
 
 ## CoreStart interface
 
+Core services exposed to the start lifecycle
+
 <b>Signature:</b>
 
 ```typescript
@@ -16,6 +18,7 @@ export interface CoreStart
 |  --- | --- | --- |
 |  [application](./kibana-plugin-public.corestart.application.md) | <code>ApplicationStart</code> | [ApplicationStart](./kibana-plugin-public.applicationstart.md) |
 |  [basePath](./kibana-plugin-public.corestart.basepath.md) | <code>BasePathStart</code> | [BasePathStart](./kibana-plugin-public.basepathstart.md) |
+|  [http](./kibana-plugin-public.corestart.http.md) | <code>HttpStart</code> | [HttpStart](./kibana-plugin-public.httpstart.md) |
 |  [i18n](./kibana-plugin-public.corestart.i18n.md) | <code>I18nStart</code> | [I18nStart](./kibana-plugin-public.i18nstart.md) |
 |  [injectedMetadata](./kibana-plugin-public.corestart.injectedmetadata.md) | <code>InjectedMetadataStart</code> | [InjectedMetadataStart](./kibana-plugin-public.injectedmetadatastart.md) |
 |  [notifications](./kibana-plugin-public.corestart.notifications.md) | <code>NotificationsStart</code> | [NotificationsStart](./kibana-plugin-public.notificationsstart.md) |

docs/development/core/public/kibana-plugin-public.fatalerrorinfo.md

@@ -0,0 +1,21 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index) &gt; [kibana-plugin-public](./kibana-plugin-public.md) &gt; [FatalErrorInfo](./kibana-plugin-public.fatalerrorinfo.md)
+
+## FatalErrorInfo interface
+
+Represents the `message` and `stack` of a fatal Error
+
+<b>Signature:</b>
+
+```typescript
+export interface FatalErrorInfo 
+```
+
+## Properties
+
+|  Property | Type | Description |
+|  --- | --- | --- |
+|  [message](./kibana-plugin-public.fatalerrorinfo.message.md) | <code>string</code> |  |
+|  [stack](./kibana-plugin-public.fatalerrorinfo.stack.md) | <code>string &#124; undefined</code> |  |
+

docs/development/core/public/kibana-plugin-public.fatalerrorinfo.message.md

@@ -0,0 +1,11 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index) &gt; [kibana-plugin-public](./kibana-plugin-public.md) &gt; [FatalErrorInfo](./kibana-plugin-public.fatalerrorinfo.md) &gt; [message](./kibana-plugin-public.fatalerrorinfo.message.md)
+
+## FatalErrorInfo.message property
+
+<b>Signature:</b>
+
+```typescript
+message: string;
+```

docs/development/core/public/kibana-plugin-public.fatalerrorinfo.stack.md

@@ -0,0 +1,11 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index) &gt; [kibana-plugin-public](./kibana-plugin-public.md) &gt; [FatalErrorInfo](./kibana-plugin-public.fatalerrorinfo.md) &gt; [stack](./kibana-plugin-public.fatalerrorinfo.stack.md)
+
+## FatalErrorInfo.stack property
+
+<b>Signature:</b>
+
+```typescript
+stack: string | undefined;
+```

docs/development/core/public/kibana-plugin-public.fatalerrorssetup.get$.md

@@ -9,5 +9,5 @@ An Observable that will emit whenever a fatal error is added with `add()`
 <b>Signature:</b>
 
 ```typescript
-get$: () => Rx.Observable<ErrorInfo>;
+get$: () => Rx.Observable<FatalErrorInfo>;
 ```

docs/development/core/public/kibana-plugin-public.fatalerrorssetup.md

@@ -17,5 +17,5 @@ export interface FatalErrorsSetup
 |  Property | Type | Description |
 |  --- | --- | --- |
 |  [add](./kibana-plugin-public.fatalerrorssetup.add.md) | <code>(error: string &#124; Error, source?: string) =&gt; never</code> | Add a new fatal error. This will stop the Kibana Public Core and display a fatal error screen with details about the Kibana build and the error. |
-|  [get$](./kibana-plugin-public.fatalerrorssetup.get$.md) | <code>() =&gt; Rx.Observable&lt;ErrorInfo&gt;</code> | An Observable that will emit whenever a fatal error is added with <code>add()</code> |
+|  [get$](./kibana-plugin-public.fatalerrorssetup.get$.md) | <code>() =&gt; Rx.Observable&lt;FatalErrorInfo&gt;</code> | An Observable that will emit whenever a fatal error is added with <code>add()</code> |