81 lines
3.8 KiB
Plaintext
81 lines
3.8 KiB
Plaintext
[role="xpack"]
|
|
[[transform-overview]]
|
|
=== {transform-cap} overview
|
|
++++
|
|
<titleabbrev>Overview</titleabbrev>
|
|
++++
|
|
|
|
You can use {transforms} to _pivot_ your data into a new entity-centric index.
|
|
By transforming and summarizing your data, it becomes possible to visualize and
|
|
analyze it in alternative and interesting ways.
|
|
|
|
A lot of {es} indices are organized as a stream of events: each event is an
|
|
individual document, for example a single item purchase. {transforms-cap} enable
|
|
you to summarize this data, bringing it into an organized, more
|
|
analysis-friendly format. For example, you can summarize all the purchases of a
|
|
single customer.
|
|
|
|
{transforms-cap} enable you to define a pivot, which is a set of
|
|
features that transform the index into a different, more digestible format.
|
|
Pivoting results in a summary of your data in a new index.
|
|
|
|
To define a pivot, first you select one or more fields that you will use to
|
|
group your data. You can select categorical fields (terms) and numerical fields
|
|
for grouping. If you use numerical fields, the field values are bucketed using
|
|
an interval that you specify.
|
|
|
|
The second step is deciding how you want to aggregate the grouped data. When
|
|
using aggregations, you practically ask questions about the index. There are
|
|
different types of aggregations, each with its own purpose and output. To learn
|
|
more about the supported aggregations and group-by fields, see
|
|
{ref}/transform-resource.html[{transform-cap} resources].
|
|
|
|
As an optional step, you can also add a query to further limit the scope of the
|
|
aggregation.
|
|
|
|
The {transform} performs a composite aggregation that paginates through all the
|
|
data defined by the source index query. The output of the aggregation is stored
|
|
in a destination index. Each time the {transform} queries the source index, it
|
|
creates a _checkpoint_. You can decide whether you want the {transform} to run
|
|
once (batch {transform}) or continuously ({ctransform}). A batch {transform} is a
|
|
single operation that has a single checkpoint. {ctransforms-cap} continually
|
|
increment and process checkpoints as new source data is ingested.
|
|
|
|
.Example
|
|
|
|
Imagine that you run a webshop that sells clothes. Every order creates a
|
|
document that contains a unique order ID, the name and the category of the
|
|
ordered product, its price, the ordered quantity, the exact date of the order,
|
|
and some customer information (name, gender, location, etc). Your dataset
|
|
contains all the transactions from last year.
|
|
|
|
If you want to check the sales in the different categories in your last fiscal
|
|
year, define a {transform} that groups the data by the product categories
|
|
(women's shoes, men's clothing, etc.) and the order date. Use the last year as
|
|
the interval for the order date. Then add a sum aggregation on the ordered
|
|
quantity. The result is an entity-centric index that shows the number of sold
|
|
items in every product category in the last year.
|
|
|
|
[role="screenshot"]
|
|
image::images/pivot-preview.jpg["Example of a {transform} pivot in {kib}"]
|
|
|
|
IMPORTANT: The {transform} leaves your source index intact. It
|
|
creates a new index that is dedicated to the transformed data.
|
|
|
|
|
|
[[transform-performance]]
|
|
==== Performance considerations
|
|
|
|
{transforms-cap} perform search aggregations on the source
|
|
indices then index the results into the destination index. Therefore, a
|
|
{transform} never takes less time than the cumulated duration of the
|
|
aggregation that it performs and the indexing process.
|
|
|
|
For better performance, make sure that your search aggregations and queries are
|
|
optimized, so they don't process unnecessary data.
|
|
|
|
NOTE: When you use <<search-aggregations-bucket-datehistogram-aggregation>>, the
|
|
queries are not considered optimal as they run through a significant amount of
|
|
data. For this reason, {transforms} performing date histogram aggregations take
|
|
longer to run.
|