mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-03-09 14:34:43 +00:00
[Docs] All Rollup docs experimental, agg limitations, clarify DeleteJob (#31299)
- All rollup pages should be marked as experimental instead of just the top page - While the job config docs state which aggregations are allowed, adding a section which specifically details this in one place is more convenient for the user - Add a clarification that the DeleteJob API does not delete the rollup data, just the rollup job.
This commit is contained in:
parent
664903a70a
commit
d4262de83a
@ -5,9 +5,37 @@
|
||||
<titleabbrev>Delete Job</titleabbrev>
|
||||
++++
|
||||
|
||||
experimental[]
|
||||
|
||||
This API deletes an existing rollup job. The job can be started or stopped, in both cases it will be deleted. Attempting
|
||||
to delete a non-existing job will throw an exception
|
||||
|
||||
.Deleting the job does not delete rolled up data
|
||||
**********************************
|
||||
When a job is deleted, that only removes the process that is actively monitoring and rolling up data.
|
||||
It does not delete any previously rolled up data. This is by design; a user may wish to roll up a static dataset. Because
|
||||
the dataset is static, once it has been fully rolled up there is no need to keep the indexing Rollup job around (as there
|
||||
will be no new data). So the job may be deleted, leaving behind the rolled up data for analysis.
|
||||
|
||||
If you wish to also remove the rollup data, and the rollup index only contains the data for a single job, you can simply
|
||||
delete the whole rollup index. If the rollup index stores data from several jobs, you must issue a Delete-By-Query that
|
||||
targets the Rollup job's ID in the rollup index:
|
||||
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
POST my_rollup_index/_delete_by_query
|
||||
{
|
||||
"query": {
|
||||
"term": {
|
||||
"_rollup.id": "the_rollup_job_id"
|
||||
}
|
||||
}
|
||||
}
|
||||
--------------------------------------------------
|
||||
// NOTCONSOLE
|
||||
|
||||
**********************************
|
||||
==== Request
|
||||
|
||||
`DELETE _xpack/rollup/job/<job_id>`
|
||||
|
@ -5,6 +5,8 @@
|
||||
<titleabbrev>Get Job</titleabbrev>
|
||||
++++
|
||||
|
||||
experimental[]
|
||||
|
||||
This API returns the configuration, stats and status of rollup jobs. The API can return the details for a single job,
|
||||
or for all jobs.
|
||||
|
||||
|
@ -5,6 +5,8 @@
|
||||
<titleabbrev>Create Job</titleabbrev>
|
||||
++++
|
||||
|
||||
experimental[]
|
||||
|
||||
This API enables you to create a rollup job. The job will be created in a `STOPPED` state, and must be
|
||||
started with the <<rollup-start-job,Start Job API>>.
|
||||
|
||||
|
@ -5,6 +5,8 @@
|
||||
<titleabbrev>Get Rollup Caps</titleabbrev>
|
||||
++++
|
||||
|
||||
experimental[]
|
||||
|
||||
This API returns the rollup capabilities that have been configured for an index or index pattern. This API is useful
|
||||
because a rollup job is often configured to rollup only a subset of fields from the source index. Furthermore, only
|
||||
certain aggregations can be configured for various fields, leading to a limited subset of functionality depending on
|
||||
|
@ -2,6 +2,8 @@
|
||||
[[rollup-job-config]]
|
||||
=== Rollup Job Configuration
|
||||
|
||||
experimental[]
|
||||
|
||||
The Rollup Job Configuration contains all the details about how the rollup job should run, when it indexes documents,
|
||||
and what future queries will be able to execute against the rollup index.
|
||||
|
||||
|
@ -5,6 +5,8 @@
|
||||
<titleabbrev>Rollup Search</titleabbrev>
|
||||
++++
|
||||
|
||||
experimental[]
|
||||
|
||||
The Rollup Search endpoint allows searching rolled-up data using the standard query DSL. The Rollup Search endpoint
|
||||
is needed because, internally, rolled-up documents utilize a different document structure than the original data. The
|
||||
Rollup Search endpoint rewrites standard query DSL into a format that matches the rollup documents, then takes the response
|
||||
|
@ -5,6 +5,8 @@
|
||||
<titleabbrev>Start Job</titleabbrev>
|
||||
++++
|
||||
|
||||
experimental[]
|
||||
|
||||
This API starts an existing, stopped rollup job. If the job does not exist an exception will be thrown.
|
||||
Starting an already started job has no action.
|
||||
|
||||
|
@ -5,6 +5,8 @@
|
||||
<titleabbrev>Stop Job</titleabbrev>
|
||||
++++
|
||||
|
||||
experimental[]
|
||||
|
||||
This API stops an existing, started rollup job. If the job does not exist an exception will be thrown.
|
||||
Stopping an already stopped job has no action.
|
||||
|
||||
|
@ -1,6 +1,8 @@
|
||||
[[rollup-api-quickref]]
|
||||
== API Quick Reference
|
||||
|
||||
experimental[]
|
||||
|
||||
Most {rollup} endpoints have the following base:
|
||||
|
||||
[source,js]
|
||||
|
@ -18,7 +18,8 @@ for analysis, but at a fraction of the storage cost of raw data.
|
||||
* <<rollup-getting-started,Getting Started>>
|
||||
* <<rollup-api-quickref, API Quick Reference>>
|
||||
* <<rollup-understanding-groups,Understanding Rollup Grouping>>
|
||||
* <<rollup-search-limitations,Limitations of Rollup Search>>
|
||||
* <<rollup-agg-limitations,Rollup aggregation limitations>>
|
||||
* <<rollup-search-limitations,Rollup Search limitations>>
|
||||
|
||||
|
||||
--
|
||||
@ -27,4 +28,5 @@ include::overview.asciidoc[]
|
||||
include::api-quickref.asciidoc[]
|
||||
include::rollup-getting-started.asciidoc[]
|
||||
include::understanding-groups.asciidoc[]
|
||||
include::rollup-agg-limitations.asciidoc[]
|
||||
include::rollup-search-limitations.asciidoc[]
|
@ -1,6 +1,8 @@
|
||||
[[rollup-overview]]
|
||||
== Overview
|
||||
|
||||
experimental[]
|
||||
|
||||
Time-based data (documents that are predominantly identified by their timestamp) often have associated retention policies
|
||||
to manage data growth. For example, your system may be generating 500,000 documents every second. That will generate
|
||||
43 million documents per day, and nearly 16 billion documents a year.
|
||||
|
24
x-pack/docs/en/rollup/rollup-agg-limitations.asciidoc
Normal file
24
x-pack/docs/en/rollup/rollup-agg-limitations.asciidoc
Normal file
@ -0,0 +1,24 @@
|
||||
[[rollup-agg-limitations]]
|
||||
== Rollup Aggregation Limitations
|
||||
|
||||
experimental[]
|
||||
|
||||
There are some limitations to how fields can be rolled up / aggregated. This page highlights the major limitations so that
|
||||
you are aware of them.
|
||||
|
||||
[float]
|
||||
=== Limited aggregation components
|
||||
|
||||
The Rollup functionality allows fields to be grouped with the following aggregations:
|
||||
|
||||
- Date Histogram aggregation
|
||||
- Histogram aggregation
|
||||
- Terms aggregation
|
||||
|
||||
And the following metrics are allowed to be specified for numeric fields:
|
||||
|
||||
- Min aggregation
|
||||
- Max aggregation
|
||||
- Sum aggregation
|
||||
- Average aggregation
|
||||
- Value Count aggregation
|
@ -1,6 +1,8 @@
|
||||
[[rollup-getting-started]]
|
||||
== Getting Started
|
||||
|
||||
experimental[]
|
||||
|
||||
To use the Rollup feature, you need to create one or more "Rollup Jobs". These jobs run continuously in the background
|
||||
and rollup the index or indices that you specify, placing the rolled documents in a secondary index (also of your choosing).
|
||||
|
||||
|
@ -1,6 +1,8 @@
|
||||
[[rollup-search-limitations]]
|
||||
== Rollup Search Limitations
|
||||
|
||||
experimental[]
|
||||
|
||||
While we feel the Rollup function is extremely flexible, the nature of summarizing data means there will be some limitations. Once
|
||||
live data is thrown away, you will always lose some flexibility.
|
||||
|
||||
@ -100,8 +102,8 @@ The Rollup functionality allows `query`'s in the search request, but with a limi
|
||||
- MatchAll Query
|
||||
- Any compound query (Boolean, Boosting, ConstantScore, etc)
|
||||
|
||||
Furthermore, these queries can only use fields that were also saved in the rollup job. If you wish to filter on a keyword `hostname` field,
|
||||
that field must have been configured in the rollup job under a `terms` grouping.
|
||||
Furthermore, these queries can only use fields that were also saved in the rollup job as a `group`.
|
||||
If you wish to filter on a keyword `hostname` field, that field must have been configured in the rollup job under a `terms` grouping.
|
||||
|
||||
If you attempt to use an unsupported query, or the query references a field that wasn't configured in the rollup job, an exception will be
|
||||
thrown. We expect the list of support queries to grow over time as more are implemented.
|
||||
|
@ -1,6 +1,8 @@
|
||||
[[rollup-understanding-groups]]
|
||||
== Understanding Groups
|
||||
|
||||
experimental[]
|
||||
|
||||
To preserve flexibility, Rollup Jobs are defined based on how future queries may need to use the data. Traditionally, systems force
|
||||
the admin to make decisions about what metrics to rollup and on what interval. E.g. The average of `cpu_time` on an hourly basis. This
|
||||
is limiting; if, at a future date, the admin wishes to see the average of `cpu_time` on an hourly basis _and partitioned by `host_name`_,
|
||||
|
Loading…
x
Reference in New Issue
Block a user