In Elasticsearch, a pipeline aggregation is a type of aggregation that operates on the results of other aggregations. Pipeline aggregations are used to perform additional calculations on the results of one or more aggregations, or to generate new metrics based on the results of existing aggregations.
There are several types of pipeline aggregations in Elasticsearch, including:
1. Average bucket pipeline aggregation: This aggregation calculates the average of a metric on a per-bucket basis. It is used to calculate the average of a metric across multiple buckets generated by a bucket aggregation.
2. Cumulative sum pipeline aggregation: This aggregation calculates a running total of a metric on a per-bucket basis. It is used to generate a cumulative sum of a metric over time.
3. Moving average pipeline aggregation: This aggregation calculates a moving average of a metric on a per-bucket basis. It is used to calculate a rolling average of a metric over time.
Pipeline aggregations are defined using the Elasticsearch Query DSL, and can be included in a search query using the “aggs” parameter. Here’s an example of a search query that includes a terms aggregation and a moving average pipeline aggregation:
{
"aggs": {
"by_date": {
"date_histogram": {
"field": "timestamp",
"interval": "day"
},
"aggs": {
"total_sales": {
"sum": {
"field": "sales"
}
},
"moving_average": {
"moving_avg": {
"buckets_path": "total_sales"
}
}
}
}
}
}
In this example, the search query includes a date histogram aggregation that groups documents by day based on the “timestamp” field, as well as a sum aggregation that calculates the total sales for each day. The moving average pipeline aggregation then calculates a moving average of the total sales over a window of three days.
By using pipeline aggregations, you can perform additional calculations on the results of other aggregations, allowing you to generate more complex metrics and gain deeper insights into your data.