Merge branch 'master' into uptime_improve-monitor-charts-query

docs/api.asciidoc

@@ -34,6 +34,7 @@ NOTE: You cannot access these APIs via the Console in Kibana.
 * <<dashboard-import-api>>
 * <<logstash-configuration-management-api>>
 * <<url-shortening-api>>
+* <<upgrade-assistant-api>>
 --
 
 include::api/spaces-management.asciidoc[]
@@ -42,4 +43,4 @@ include::api/saved-objects.asciidoc[]
 include::api/dashboard-import.asciidoc[]
 include::api/logstash-configuration-management.asciidoc[]
 include::api/url-shortening.asciidoc[]
-
+include::api/upgrade-assistant.asciidoc[]

docs/api/upgrade-assistant.asciidoc

@@ -0,0 +1,15 @@
+[role="xpack"]
+[[upgrade-assistant-api]]
+== Upgrade Assistant API
+
+experimental[This API is *experimental* and may be changed or removed completely in a future release. The underlying Upgrade Assistant concepts are stable, but the APIs for managing Upgrade Assistant are currently experimental.]
+
+The Upgrade Assistant API allows you to check the upgrade status of your Elasticsearch cluster
+and reindex indices that were created in the previous major version. The assistant helps you prepare
+for the next major version of Elasticsearch.
+
+* <<upgrade-assistant-api-status>>
+* <<upgrade-assistant-api-reindexing>>
+
+include::upgrade-assistant/status.asciidoc[]
+include::upgrade-assistant/reindexing.asciidoc[]

docs/api/upgrade-assistant/reindexing.asciidoc

@@ -0,0 +1,154 @@
+[[upgrade-assistant-api-reindexing]]
+=== Reindex API
+
+experimental[This API is *experimental* and may be changed or removed completely in a future release. The underlying Upgrade Assistant concepts are stable, but the APIs for managing Upgrade Assistant are currently experimental.]
+
+When checking the upgrade status, some indices will have the `reindex` paramaeter set to `true`. You can use this Reindexing API to reindex these indices.
+
+==== Start or resume a reindex
+
+===== Request
+
+To start a new reindex or resume a paused reindex, submit a POST request to the `/api/upgrade_assistant/reindex/<indexName>` endpoint:
+
+Note: You cannot access this endpoint via the Console in Kibana.
+
+[source,js]
+--------------------------------------------------
+POST /api/upgrade_assistant/reindex/myIndex
+--------------------------------------------------
+// KIBANA
+
+===== Response
+
+A successful call returns a response code of `200` and a response body
+containing a JSON structure similar to the following example:
+
+[source,js]
+--------------------------------------------------
+{
+  "indexName": ".ml-state",
+  "newIndexName": ".reindexed-v7-ml-state",
+  "status": 0,
+  "lastCompletedStep": 0,
+  "reindexTaskId": null,
+  "reindexTaskPercComplete": null,
+  "errorMessage": null
+}
+--------------------------------------------------
+
+See the next section for an explanation of each field.
+
+==== Check the status of a reindex
+
+===== Request
+
+Once a reindex is started, you can check the status of the reindex operation by submitting a GET request to the `/api/upgrade_assistant/reindex/<indexName>` endpoint:
+
+Note: You cannot access this endpoint via the Console in Kibana.
+
+[source,js]
+--------------------------------------------------
+GET /api/upgrade_assistant/reindex/myIndex
+--------------------------------------------------
+// KIBANA
+
+===== Response
+
+A successful call returns a response code of `200` and a response body
+containing a JSON structure similar to the following example:
+
+[source,js]
+--------------------------------------------------
+{
+  "reindexOp": {
+    "indexName": ".ml-state",
+    "newIndexName": ".reindexed-v7-ml-state", <1>
+    "status": 0, <2>
+    "lastCompletedStep": 40, <3>
+    "reindexTaskId": "QprwvTMzRQ2MLWOW22oQ4Q:11819", <4>
+    "reindexTaskPercComplete": 0.3, <5>
+    "errorMessage": null <6>
+  },
+  "warnings": [], <7>
+  "hasRequiredPrivileges": true <8>
+}
+--------------------------------------------------
+
+<1> Name of the new index that is being created.
+<2> Current status of the reindex. See the <<status-code,status code table>> for details.
+<3> Last successfully completed step of the reindex. See the <<step-code,step code table>> table for details.
+<4> Task ID of the reindex task in Elasticsearch. Only present if reindexing has started.
+<5> Percentage of how far the reindexing task in Elasticsearch has progressed, in decimal from from 0 to 1.
+<6> Error that caused the reindex to fail, if it failed.
+<7> An array of any warning codes explaining what changes are required for this reindex. See the <<warning-code,warning code table>> for details.
+<8> Whether or not the current user has required privileges to reindex this index. Returns `true` if Security is not available or disabled.
+
+[[status-code]]
+===== Status code
+
+The `status` field corresponds to these statuses:
+
+[horizontal]
+0:: in progress
+1:: completed
+2:: failed
+3:: paused
+4:: cancelled
+
+[[step-code]]
+===== Step code
+
+The `lastCompletedStep` field corresponds to these steps:
+
+[horizontal]
+0::  The reindex operation has been created in Kibana.
+10:: Index group services stopped. Only applies to some system indices.
+20:: index set to readonly
+30:: The new destination index has been created.
+40:: The reindex task in Elasticsearch has started.
+50:: The reindex task in Elasticsearch has completed.
+60:: Aliases have been created to point to the new index, and the old index has been deleted.
+70:: Index group services have been resumed. Only applies to some system indices.
+
+[[warning-code]]
+===== Warning code
+
+The `warnings` field corresponds to an array of integers for these warnings:
+
+[horizontal]
+0:: The `_all` meta field will be removed.
+1:: Any coerced boolean values will be converted in the source document (example: `yes`, `1`, `off`).
+2:: Documents will be converted to support Elastic Common Schema. Only applies to APM indices created in 6.x.
+
+===== Paused reindexes
+
+If the Kibana node that started the reindex is shutdown or restarted, the reindex will go into a paused state after some time.
+To resume the reindex, you must submit a new POST request to the `/api/upgrade_assistant/reindex/<indexName>` endpoint.
+
+==== Cancel a reindex
+
+===== Request
+
+You can cancel reindexes that are waiting for the Elasticsearch reindex task to complete (`lastCompletedStep` set to `40`).
+To cancel a reindex, submit a POST request to the `/api/upgrade_assistant/reindex/<indexName>/cancel` endpoint:
+
+Note: You cannot access this endpoint via the Console in Kibana.
+
+[source,js]
+--------------------------------------------------
+POST /api/upgrade_assistant/reindex/myIndex/cancel
+--------------------------------------------------
+// KIBANA
+
+===== Response
+
+A successful call returns a response code of `200` and a response body
+containing a JSON structure similar to the following example:
+
+[source,js]
+--------------------------------------------------
+{
+  "acknowledged": true
+}
+--------------------------------------------------

docs/api/upgrade-assistant/status.asciidoc

@@ -0,0 +1,48 @@
+[[upgrade-assistant-api-status]]
+=== Upgrade Readiness Status
+
+experimental[This API is *experimental* and may be changed or removed completely in a future release. The underlying Upgrade Assistant concepts are stable, but the APIs for managing Upgrade Assistant are currently experimental.]
+
+==== Request
+
+To check the status of your cluster, submit a GET request to the `/api/upgrade_assistant/status` endpoint:
+
+Note: You cannot access this endpoint via the Console in Kibana.
+
+[source,js]
+--------------------------------------------------
+GET /api/upgrade_assistant/status
+--------------------------------------------------
+// KIBANA
+
+==== Response
+
+A successful call returns a response code of `200` and a response body
+containing a JSON structure similar to the following example:
+
+[source,js]
+--------------------------------------------------
+{
+  "readyForUpgrade": false,
+  "cluster": [
+    {
+      "message": "Cluster deprecated issue",
+      "details": "...",
+      "level": "warning",
+      "url": "https://docs.elastic.co/..."
+    }
+  ],
+  "indices": [
+    {
+      "message": "Index was created before 6.0",
+      "details": "...",
+      "index": "myIndex",
+      "level": "critical",
+      "reindex": true, <1>
+      "url": "https://docs.elastic.co/..."
+    }
+  ]
+}
+--------------------------------------------------
+
+<1> You can fix indices with the `reindex` attribute set to `true` using the <<upgrade-assistant-api-reindexing>>.

docs/management/managing-indices.asciidoc

@@ -1,30 +1,59 @@
 [[managing-indices]]
 == Managing Indices
 
-The *Index Management* UI enables you to view index settings,
-mappings, and statistics and perform management operations.
-These include refreshing, flushing, clearing the cache, merging segments,
-and closing or deleting indices. The UI provides a convenient way to
+*Index Management* enables you to view index settings,
+mappings, and statistics and perform index-level operations.
+These include refreshing, flushing, clearing the cache, force merging segments,
+freezing indices, and more. *Index Management* provides a convenient way to
 perform bulk operations on multiple indices.
 
-To open the UI, select *Management > Index Management*. If security is enabled,
-you must have the the `monitor` cluster privilege and the `view_index_metadata` 
+To access this feature, go to *Management > {es} > Index Management*. 
+If security is enabled,
+you must have the `monitor` cluster privilege and the `view_index_metadata` 
 and `manage` index privileges to view the data.  See 
 {xpack-ref}/security-privileges.html[Security Privileges] for more
 information.
 
+*Index Management* uses badges to make users aware when an index is {ref}/frozen-indices.html[frozen], 
+a {ref}/ccr-put-follow.html[follower index], 
+or a {ref}/rollup-get-rollup-index-caps.html[rollup index]. 
+Clicking a badge filters for all indices of that type. 
+
 [role="screenshot"]
-image::images/management-index-management.png[Index Management UI]
+image::images/management_index_labels.png[Index Management UI]
+
+Clicking the name of an index displays the index summary and provides access to
+the index settings, mapping, and statistics. 
+
+From the *Manage* menu, you can perform these index-level operations on either 
+a single or multiple indices:
+
+* *Close the index*. Blocks the index from read/write operations. 
+A closed index exists in the cluster, but doesn't consume resources 
+other than disk space. If you reopen a closed index, it goes through the 
+normal recovery process. 
+
+* *Force merge the index*. Reduces the number of segments in your shard by 
+merging smaller files and clearing deleted ones. Only force merge a read-only index.
+
+* *Refresh the index*. Writes the operations in the indexing buffer to the 
+filesystem cache. This is automatically done once per second. Forcing a manual 
+refresh is useful during testing, but should not be routinely done in 
+production because it has a performance impact.
+
+* *Clear the index cache*. Clears all caches associated with the index. 
+
+* *Flush the index*. Frees memory by syncing the filesystem cache to disk and 
+clearing the cache. Once the sync is complete, the internal transaction log is reset.
+
+* *Freeze the index*. Makes the index read-only and reduces its memory footprint 
+by moving shards to disk. {ref}/frozen-indices.html[Frozen indices] remain 
+searchable, but queries take longer.
 
-Click the name of an index to display the index summary and access
-the index settings, mapping, and statistics. The *Manage* menu in the
-lower right of the index pane enables you to manage
-the selected index.
+* *Delete the index*. Permanently removes the index and all of it's documents.
 
-To perform bulk operations, select the checkboxes of the indices you want to
-modify and choose an operation from the *Manage indices* menu
-next to the query bar. To select all
-indices, select the checkbox in the *Name* header.
+* *Add an lifecycle policy*.  Specifies a policy for managing the lifecycle of the 
+index.
 
 For information about the available management operations,
-see {ref}/indices.html[Indices APIs] in the Elasticsearch Reference.
+see {ref}/indices.html[Indices APIs].

docs/visualize/vega.asciidoc

@@ -8,6 +8,10 @@ into Kibana, either standalone, or on top of a map.  To see Vega in action,
 watch this
 https://www.youtube.com/watch?v=lQGCipY3th8[short introduction video].
 
+Currently Vega version 4.4 and VegaLite version 2.6 are supported.
+
+NOTE: In Vega it is possible to load data dynamically, e.g. by setting signals as data urls. This is not supported in Kibana as all data is fetched at once prior to passing it to the Vega renderer.
+
 
 [[vega-quick-demo]]
 === Getting Started with Vega

package.json

@@ -234,8 +234,8 @@
     "uuid": "3.0.1",
     "val-loader": "^1.1.1",
     "validate-npm-package-name": "2.2.2",
-    "vega-lib": "^3.3.1",
-    "vega-lite": "^2.4.0",
+    "vega-lib": "^4.4.0",
+    "vega-lite": "^2.6.0",
     "vega-schema-url-parser": "1.0.0",
     "vega-tooltip": "^0.9.14",
     "vision": "^5.3.3",