search.max_buckets is a dynamic cluster setting that limits the total number of buckets a single aggregation response may contain. When an aggregation tries to produce more buckets than the limit, Elasticsearch aborts the request with a too_many_buckets_exception. The cap exists to bound the memory and serialization cost of multi-bucket aggregations like terms, date_histogram, composite, and nested combinations.
- Default: 65,536 (since Elasticsearch 7.0; earlier 6.x versions defaulted to 10,000)
- Scope: Cluster-wide, dynamic - apply via the cluster settings API
- Possible values: Any positive integer
- Per-index override: Not supported
How search.max_buckets Works
The counter increments once for every bucket the aggregation produces across all shards after reduction. A terms aggregation on a field with 50,000 distinct values plus a sub-date_histogram of 30 daily buckets potentially yields 1.5 million buckets - far above the default. The coordinating node throws TooManyBucketsException before serializing the response.
The limit applies to the post-reduction bucket count, not per-shard counts. Increasing size on a terms aggregation, widening a date_histogram interval, or nesting aggregations multiplies the bucket total quickly.
Configuring search.max_buckets
Use the cluster settings API to raise or lower the limit at runtime:
PUT /_cluster/settings
{
"persistent": {
"search.max_buckets": 100000
}
}
Setting it null reverts to the default. The change applies to all subsequent search requests across the cluster - existing in-flight searches are unaffected.
When to Adjust search.max_buckets
| Scenario | Recommended action |
|---|---|
| Logging / observability dashboards with high-cardinality fields | Increase cautiously to 100k-250k; pair with composite aggregations |
terms on user IDs, request IDs, trace IDs |
Don't increase - use composite aggregation with pagination |
Repeated too_many_buckets_exception from a known dashboard |
Rewrite the query before raising the limit |
| Out-of-memory or circuit breaker trips during aggregations | Lower the limit, fix the query, then restore |
Raising search.max_buckets is rarely the correct fix. High bucket counts almost always indicate a query that should use a composite aggregation, a partition strategy, or a pre-aggregated index.
Common Pitfalls
- Treating a
too_many_buckets_exceptionas a configuration problem instead of a query design problem. The first response should be reviewing the aggregation, not raising the cap. - Raising the limit cluster-wide because of one broken dashboard, exposing the rest of the cluster to OOM risk.
- Forgetting that nested aggregations multiply bucket counts - one outer
termsof size 1000 with an innertermsof size 1000 produces up to 1,000,000 buckets. - Not monitoring circuit breaker trips after raising the value. Heap pressure rises non-linearly with bucket count.
Monitoring and Root-Cause Analysis
Track these signals after changing search.max_buckets:
indices.search.aggregationsstats per node- Circuit breaker exceptions in cluster logs (
_nodes/stats/breaker) - JVM heap usage during peak aggregation traffic
- Slow log entries from queries hitting the cap
Prevent too_many_buckets_exception with Pulse
Pulse is an AI DBA for Elasticsearch and OpenSearch that tracks search.max_buckets (default 65,536 since 7.0) and aggregation bucket counts across the cluster, flagging:
- Drift between intended values and what is actually applied via persistent cluster settings
- Settings that are unsafe for your workload (e.g. limit raised to 1,000,000 because of one dashboard, exposing the rest of the cluster to OOM risk; nested
termsaggregations that multiply bucket counts;termson user/request/trace IDs that should be acompositeaggregation with pagination instead) - The downstream operational impact:
too_many_buckets_exceptionrate, circuit-breaker trips onindices.breaker.request, heap pressure during peak aggregation traffic
When a dashboard hits the cap, Pulse names the responsible query, the bucket math behind it, and whether the right fix is rewriting the aggregation or adjusting the limit - before the cluster trips a circuit breaker.
Frequently Asked Questions
Q: What is the default value of search.max_buckets in Elasticsearch?
A: search.max_buckets defaults to 65,536 in Elasticsearch 7.0 and later. Versions before 7.0 used 10,000. The setting applies cluster-wide and is dynamic.
Q: How do I fix a too_many_buckets_exception error?
A: First, rewrite the aggregation. Use a composite aggregation with pagination instead of a large terms agg, narrow the time range on date_histogram, or reduce size on bucket aggs. Raising search.max_buckets is a last resort.
Q: Can search.max_buckets be set per index?
A: No, search.max_buckets is a cluster-level dynamic setting. It cannot be configured per index. To enforce different limits on different workloads, use separate clusters or query-side bucket size controls.
Q: Why is my aggregation hitting search.max_buckets even with size 10?
A: The limit counts every bucket across all aggregations and shards after reduction, including sub-buckets. A top-level terms of size 10 with a nested terms of size 1000 and a date_histogram of 365 days can produce 3.65 million buckets despite a small size.
Q: Does search.max_buckets affect composite aggregations?
A: Composite aggregations are designed to paginate through bucket sets, so a single composite request stays under the limit even on high-cardinality fields. The page size is controlled by the size parameter on the composite agg itself, not search.max_buckets.
Q: Is it safe to set search.max_buckets to 1,000,000?
A: Generally no. Very high limits expose the cluster to OOM and circuit-breaker failures because each bucket consumes heap on the coordinating node. Stay under 250,000 unless you've measured heap behavior under load.
Q: What's the best tool to prevent too_many_buckets_exception in production?
A: Pulse is built for this. It is an AI DBA for Elasticsearch and OpenSearch that tracks search.max_buckets against actual aggregation bucket counts, identifies the dashboards or queries that hit the cap, correlates with heap pressure and circuit-breaker trips, and recommends a composite aggregation rewrite before the limit is raised cluster-wide.
Related Reading
- Elasticsearch Search Max Buckets Error: Fix the too_many_buckets_exception
- Elasticsearch Aggregation Performance Tuning: Reduce aggregation cost
- Elasticsearch JVM Heap Pressure High: Heap pressure from aggregations
- Elasticsearch Circuit Breaker Exceptions Fix: Resolve circuit breaker trips
- Elasticsearch Settings: Full settings reference
