[Docs] All Rollup docs experimental, agg limitations, clarify DeleteJ…

…ob (#31299) - All rollup pages should be marked as experimental instead of just the top page - While the job config docs state which aggregations are allowed, adding a section which specifically details this in one place is more convenient for the user - Add a clarification that the DeleteJob API does not delete the rollup data, just the rollup job.
elastic · Jun 13, 2018 · 1e41d60 · 1e41d60
1 parent 43631e8
commit 1e41d60
Show file tree

Hide file tree

Showing 15 changed files with 81 additions and 3 deletions.
diff --git a/x-pack/docs/en/rest-api/rollup/delete-job.asciidoc b/x-pack/docs/en/rest-api/rollup/delete-job.asciidoc
@@ -5,9 +5,37 @@
 <titleabbrev>Delete Job</titleabbrev>
 ++++
 
+experimental[]
+
 This API deletes an existing rollup job.  The job can be started or stopped, in both cases it will be deleted.  Attempting
 to delete a non-existing job will throw an exception
 
+.Deleting the job does not delete rolled up data
+**********************************
+When a job is deleted, that only removes the process that is actively monitoring and rolling up data.
+It does not delete any previously rolled up data.  This is by design; a user may wish to roll up a static dataset.  Because
+the dataset is static, once it has been fully rolled up there is no need to keep the indexing Rollup job around (as there
+will be no new data).  So the job may be deleted, leaving behind the rolled up data for analysis.
+
+If you wish to also remove the rollup data, and the rollup index only contains the data for a single job, you can simply
+delete the whole rollup index.  If the rollup index stores data from several jobs, you must issue a Delete-By-Query that
+targets the Rollup job's ID in the rollup index:
+
+
+[source,js]
+--------------------------------------------------
+POST my_rollup_index/_delete_by_query
+{
+  "query": {
+    "term": {
+      "_rollup.id": "the_rollup_job_id"
+    }
+  }
+}
+--------------------------------------------------
+// NOTCONSOLE
+
+**********************************
 ==== Request
 
 `DELETE _xpack/rollup/job/<job_id>`

diff --git a/x-pack/docs/en/rest-api/rollup/get-job.asciidoc b/x-pack/docs/en/rest-api/rollup/get-job.asciidoc
@@ -5,6 +5,8 @@
 <titleabbrev>Get Job</titleabbrev>
 ++++
 
+experimental[]
+
 This API returns the configuration, stats and status of rollup jobs.  The API can return the details for a single job,
 or for all jobs.
 

diff --git a/x-pack/docs/en/rest-api/rollup/put-job.asciidoc b/x-pack/docs/en/rest-api/rollup/put-job.asciidoc
@@ -5,6 +5,8 @@
 <titleabbrev>Create Job</titleabbrev>
 ++++
 
+experimental[]
+
 This API enables you to create a rollup job.  The job will be created in a `STOPPED` state, and must be
 started with the <<rollup-start-job,Start Job API>>.
 

diff --git a/x-pack/docs/en/rest-api/rollup/rollup-caps.asciidoc b/x-pack/docs/en/rest-api/rollup/rollup-caps.asciidoc
@@ -5,6 +5,8 @@
 <titleabbrev>Get Rollup Caps</titleabbrev>
 ++++
 
+experimental[]
+
 This API returns the rollup capabilities that have been configured for an index or index pattern.  This API is useful
 because a rollup job is often configured to rollup only a subset of fields from the source index.  Furthermore, only
 certain aggregations can be configured for various fields, leading to a limited subset of functionality depending on

diff --git a/x-pack/docs/en/rest-api/rollup/rollup-job-config.asciidoc b/x-pack/docs/en/rest-api/rollup/rollup-job-config.asciidoc
@@ -2,6 +2,8 @@
 [[rollup-job-config]]
 === Rollup Job Configuration
 
+experimental[]
+
 The Rollup Job Configuration contains all the details about how the rollup job should run, when it indexes documents,
 and what future queries will be able to execute against the rollup index.
 

diff --git a/x-pack/docs/en/rest-api/rollup/rollup-search.asciidoc b/x-pack/docs/en/rest-api/rollup/rollup-search.asciidoc
@@ -5,6 +5,8 @@
 <titleabbrev>Rollup Search</titleabbrev>
 ++++
 
+experimental[]
+
 The Rollup Search endpoint allows searching rolled-up data using the standard query DSL.  The Rollup Search endpoint
 is needed because, internally, rolled-up documents utilize a different document structure than the original data.  The
 Rollup Search endpoint rewrites standard query DSL into a format that matches the rollup documents, then takes the response

diff --git a/x-pack/docs/en/rest-api/rollup/start-job.asciidoc b/x-pack/docs/en/rest-api/rollup/start-job.asciidoc
@@ -5,6 +5,8 @@
 <titleabbrev>Start Job</titleabbrev>
 ++++
 
+experimental[]
+
 This API starts an existing, stopped rollup job.  If the job does not exist an exception will be thrown.
 Starting an already started job has no action.
 

diff --git a/x-pack/docs/en/rest-api/rollup/stop-job.asciidoc b/x-pack/docs/en/rest-api/rollup/stop-job.asciidoc
@@ -5,6 +5,8 @@
 <titleabbrev>Stop Job</titleabbrev>
 ++++
 
+experimental[]
+
 This API stops an existing, started rollup job.  If the job does not exist an exception will be thrown.
 Stopping an already stopped job has no action.
 

diff --git a/x-pack/docs/en/rollup/api-quickref.asciidoc b/x-pack/docs/en/rollup/api-quickref.asciidoc
@@ -1,6 +1,8 @@
 [[rollup-api-quickref]]
 == API Quick Reference
 
+experimental[]
+
 Most {rollup} endpoints have the following base:
 
 [source,js]

diff --git a/x-pack/docs/en/rollup/index.asciidoc b/x-pack/docs/en/rollup/index.asciidoc
@@ -18,7 +18,8 @@ for analysis, but at a fraction of the storage cost of raw data.
 * <<rollup-getting-started,Getting Started>>
 * <<rollup-api-quickref, API Quick Reference>>
 * <<rollup-understanding-groups,Understanding Rollup Grouping>>
-* <<rollup-search-limitations,Limitations of Rollup Search>>
+* <<rollup-agg-limitations,Rollup aggregation limitations>>
+* <<rollup-search-limitations,Rollup Search limitations>>
 
 
 --
@@ -27,4 +28,5 @@ include::overview.asciidoc[]
 include::api-quickref.asciidoc[]
 include::rollup-getting-started.asciidoc[]
 include::understanding-groups.asciidoc[]
+include::rollup-agg-limitations.asciidoc[]
 include::rollup-search-limitations.asciidoc[]
diff --git a/x-pack/docs/en/rollup/overview.asciidoc b/x-pack/docs/en/rollup/overview.asciidoc
@@ -1,6 +1,8 @@
 [[rollup-overview]]
 == Overview
 
+experimental[]
+
 Time-based data (documents that are predominantly identified by their timestamp) often have associated retention policies
 to manage data growth.  For example, your system may be generating 500,000 documents every second.  That will generate
 43 million documents per day, and nearly 16 billion documents a year.

diff --git a/x-pack/docs/en/rollup/rollup-agg-limitations.asciidoc b/x-pack/docs/en/rollup/rollup-agg-limitations.asciidoc
@@ -0,0 +1,24 @@
+[[rollup-agg-limitations]]
+== Rollup Aggregation Limitations
+
+experimental[]
+
+There are some limitations to how fields can be rolled up / aggregated.  This page highlights the major limitations so that
+you are aware of them.
+
+[float]
+=== Limited aggregation components
+
+The Rollup functionality allows fields to be grouped with the following aggregations:
+
+- Date Histogram aggregation
+- Histogram aggregation
+- Terms aggregation
+
+And the following metrics are allowed to be specified for numeric fields:
+
+- Min aggregation
+- Max aggregation
+- Sum aggregation
+- Average aggregation
+- Value Count aggregation
diff --git a/x-pack/docs/en/rollup/rollup-getting-started.asciidoc b/x-pack/docs/en/rollup/rollup-getting-started.asciidoc
@@ -1,6 +1,8 @@
 [[rollup-getting-started]]
 == Getting Started
 
+experimental[]
+
 To use the Rollup feature, you need to create one or more "Rollup Jobs".  These jobs run continuously in the background
 and rollup the index or indices that you specify, placing the rolled documents in a secondary index (also of your choosing).
 

diff --git a/x-pack/docs/en/rollup/rollup-search-limitations.asciidoc b/x-pack/docs/en/rollup/rollup-search-limitations.asciidoc
@@ -1,6 +1,8 @@
 [[rollup-search-limitations]]
 == Rollup Search Limitations
 
+experimental[]
+
 While we feel the Rollup function is extremely flexible, the nature of summarizing data means there will be some limitations.  Once
 live data is thrown away, you will always lose some flexibility.
 
@@ -100,8 +102,8 @@ The Rollup functionality allows `query`'s in the search request, but with a limi
 - MatchAll Query
 - Any compound query (Boolean, Boosting, ConstantScore, etc)
 
-Furthermore, these queries can only use fields that were also saved in the rollup job.  If you wish to filter on a keyword `hostname` field,
-that field must have been configured in the rollup job under a `terms` grouping.
+Furthermore, these queries can only use fields that were also saved in the rollup job as a `group`.
+If you wish to filter on a keyword `hostname` field, that field must have been configured in the rollup job under a `terms` grouping.
 
 If you attempt to use an unsupported query, or the query references a field that wasn't configured in the rollup job, an exception will be
 thrown.  We expect the list of support queries to grow over time as more are implemented.

diff --git a/x-pack/docs/en/rollup/understanding-groups.asciidoc b/x-pack/docs/en/rollup/understanding-groups.asciidoc
@@ -1,6 +1,8 @@
 [[rollup-understanding-groups]]
 == Understanding Groups
 
+experimental[]
+
 To preserve flexibility, Rollup Jobs are defined based on how future queries may need to use the data.  Traditionally, systems force
 the admin to make decisions about what metrics to rollup and on what interval.  E.g. The average of `cpu_time` on an hourly basis.  This
 is limiting; if, at a future date, the admin wishes to see the average of `cpu_time` on an hourly basis _and partitioned by `host_name`_,