Multiple aggregations in one Elasticsearch query let you answer several questions in a single request - per-bucket metrics, parallel groupings, and post-processing of bucket outputs. There are three composition patterns: sibling aggregations at the same level, nested sub-aggregations inside a parent bucket, and pipeline aggregations that operate on the output of other aggregations. Combining them is how production dashboards extract multi-metric breakdowns without hammering the cluster with separate queries.
The Three Composition Patterns
The aggs block of a search supports unlimited siblings at any level, sub-aggregations under bucket aggregations, and pipeline aggregations that reference other aggregations by path. Pick the pattern that matches the question:
| Pattern | Use when |
|---|---|
| Sibling | Independent metrics or groupings over the same query, returned together. |
| Nested sub-aggregation | Compute metrics per bucket of a parent bucket aggregation. |
| Pipeline (parent) | Compute a value from each bucket of the parent (e.g. running total). |
| Pipeline (sibling) | Aggregate across all buckets of a sibling (e.g. grand total). |
Aggregations always run on documents matching the outer query. Filtering inside an aggregation requires a filter, filters, or nested aggregation - not a query.
Example: Sibling, Nested, and Pipeline Together
GET /sales/_search
{
"size": 0,
"query": { "range": { "order_date": { "gte": "now-90d" } } },
"aggs": {
"by_category": {
"terms": { "field": "category.keyword", "size": 10 },
"aggs": {
"avg_price": { "avg": { "field": "price" } },
"max_price": { "max": { "field": "price" } },
"sample": { "top_hits": { "size": 1, "sort": [{ "order_date": "desc" }] } }
}
},
"total_revenue": { "sum": { "field": "amount" } },
"revenue_by_category_total": {
"sum_bucket": { "buckets_path": "by_category>avg_price" }
}
}
}
This query returns: per-category average and max prices plus the latest sample, the global revenue total, and a pipeline sum across the per-category averages - all in one round trip.
Performance and Bucket Limits
Elasticsearch enforces search.max_buckets (default 65,536) as the total number of buckets in the response across all aggregations. Nested bucket aggregations multiply: a terms agg with 100 buckets containing a date_histogram with 200 buckets produces 100 x 200 = 20,000 buckets, and a second-level nested terms with 10 buckets pushes it to 200,000 and fails.
Sibling metric aggregations are cheap; nested bucket aggregations are expensive. Order nesting from lowest to highest cardinality where the data permits it, since the outer aggregation determines how many times the inner one is executed. Use filter and filters aggregations to scope sub-aggregations to a subset without running an entire second query.
| Pattern | Typical cost |
|---|---|
| 1 terms + 5 sibling metrics | Linear in matched docs, low memory. |
| 1 terms + 1 nested terms | Multiplied bucket count, watch search.max_buckets. |
| Pipeline aggs | Constant additional cost, runs on already-reduced buckets. |
| Cardinality inside high-cardinality terms | Heap-heavy; each bucket holds its own HLL sketch. |
Combining many aggregations inside a single search is a common cause of slow queries, high heap usage, and circuit-breaker trips in Elasticsearch. Hand-walking the slow log to figure out which composed aggregation - sibling, nested, or pipeline - is the actual driver of latency in a multi-aggs dashboard query is exactly the loop Pulse runs continuously.
Common Mistakes
- Letting nested terms aggregations multiply past
search.max_buckets. Addsize,min_doc_count, or restructure into siblings. - Using nested terms when you want unique combinations - use multi_terms instead.
- Building a sum/avg per bucket and a global sum/avg separately, then realizing a pipeline
sum_bucket/avg_bucketwould have given the second result for free. - Forgetting to set
"size": 0on the outer search when only aggregation results are needed - wastes I/O fetching hits. - Ordering parent terms by a deeply-nested sub-aggregation - bucket order error compounds and top-N becomes noisy.
Diagnose Heavy Multi-Aggregation Queries with Pulse
Pulse is an AI DBA for Elasticsearch and OpenSearch that continuously profiles production query traffic. For multi-aggregation searches specifically, Pulse:
- Identifies search requests where nested bucket aggregations multiply toward
search.max_buckets(default 65,536) and flags those that are already returningtoo_many_buckets_exception - Pinpoints which sibling, nested, or pipeline aggregation inside a composed
aggstree is dominating per-request cost - the part you cannot see from the top-level slow log alone - Flags cardinality sub-aggregations nested under high-cardinality terms parents, where each bucket holds its own HLL sketch
- Detects searches missing
"size": 0while only aggregation results are needed, wasting I/O fetching hits - Traces each slow multi-aggs query back to the calling service via slow-log and APM correlation
- Recommends concrete restructures: replace a second computed metric with a sibling sum_bucket, flatten a nested terms into multi_terms for unique combinations, switch deep enumeration to a composite aggregation, reorder parent buckets by ascending cardinality, and scope sub-aggregations with
filter/filtersinstead of a second search - Tracks heap, circuit breaker, and latency impact after the change ships
This converts the manual aggregation-tree debugging loop into a continuous optimization workflow.
Frequently Asked Questions
Q: Can I combine metric, bucket, and pipeline aggregations in one query?
A: Yes. The three aggregation kinds compose freely - sibling, nested, and pipeline patterns can all appear in the same aggs tree. Pipeline aggregations must follow their referenced aggregations in the response order, which Elasticsearch handles automatically as long as the path is valid.
Q: How many levels of nesting are practical?
A: There is no hard limit, but every additional level multiplies the bucket count. Two or three levels is typical; four or more usually indicates the dashboard should be redesigned, often by replacing nesting with multi_terms or a composite aggregation.
Q: How do I limit aggregation scope without changing the main query?
A: Wrap the affected aggregations in a filter (single condition) or filters (multiple named conditions) aggregation. This is cheaper than running a second search.
Q: Can I sort bucket aggregations by sub-aggregation values?
A: Yes. Set order on the parent bucket aggregation to { "<sub_agg_name>": "desc" }. Be aware that ordering by a sub-aggregation amplifies the top-N error inherent to terms aggregations.
Q: How do I get totals across all buckets in one query?
A: Add a sibling sum_bucket pipeline aggregation pointing at the per-bucket metric. It runs on the already-reduced bucket list and returns a single scalar.
Q: How can I handle high-cardinality fields safely in multi-aggregation queries?
A: Cap size on terms aggregations, prefer cardinality for distinct-count needs, and consider the composite aggregation for exhaustive enumeration with bounded memory.
Q: How do I find which aggregation in a multi-aggs dashboard query is dominating latency?
A: Pulse profiles Elasticsearch and OpenSearch slow logs, decomposes composed aggs trees to identify the specific sibling, nested, or pipeline aggregation driving cost, attributes each query to the calling dashboard or service, and recommends restructures - sibling pipelines, multi_terms flattening, composite replacement, and "size": 0 fixes.
Related Reading
- Terms Aggregation: the most common parent bucket aggregation.
- Composite Aggregation: paginated alternative to nested terms.
- Sum Bucket Aggregation: grand totals across sibling buckets.
- Date Histogram Aggregation: time-series bucket parent.
- Cardinality Aggregation: distinct counts inside nested buckets.
- Elasticsearch Query Language: the DSL all aggregations run inside.
