The cluster.max_shards_per_node setting caps how many open shards (primary plus replica) Elasticsearch will allocate to a single non-frozen data node. The default is 1000 since Elasticsearch 7.0. When a create-index or open-index request would push any node over this limit, Elasticsearch rejects it with validation_exception: Validation Failed: 1: this action would add [N] total shards, but this cluster currently has [M]/[max] maximum shards open. Raising the limit treats the symptom; the right fix is almost always fewer, larger indices.
Definition
cluster.max_shards_per_node is a dynamic cluster-level setting that bounds the open-shard count per data node. Closed indices, frozen indices, and searchable snapshots do not count against the limit. Frozen nodes have their own ceiling, cluster.max_shards_per_node.frozen (default 3000).
The product cluster.max_shards_per_node * <number_of_data_nodes> is the cluster-wide ceiling. A 6-node cluster with the default limit can hold 6000 open shards before new indices fail.
Default and Range
| Property | Value |
|---|---|
| Default | 1000 (since Elasticsearch 7.0) |
| Type | Integer, dynamic cluster setting |
| Scope | Cluster |
| Affects | Open primary plus replica shards on regular data nodes |
| Counterpart | cluster.max_shards_per_node.frozen (default 3000) for frozen data nodes |
Elasticsearch's own guidance is to target 20 shards per GB of JVM heap. With a maxed-out 31 GB heap, that translates to ~620 shards as a healthy upper bound, well below the default 1000. The 1000 figure is a safety net, not a target.
How to Change It
Update dynamically through the cluster settings API:
# Raise the limit to 1500 cluster-wide
PUT /_cluster/settings
{
"persistent": {
"cluster.max_shards_per_node": 1500
}
}
Verify with:
GET /_cluster/settings?include_defaults=true&filter_path=*.cluster.max_shards_per_node
To return to the default, set the value to null:
PUT /_cluster/settings
{ "persistent": { "cluster.max_shards_per_node": null } }
What to Do When You Hit the Limit
Hitting validation_exception does not mean the cluster is full of useful data. In most production incidents the root cause is one of:
- Time-series indices accumulating without an ILM delete phase. Confirm with
GET /_cat/indices?v&s=index- sort by name and look for ancient indices with low storage. - One primary shard per day per low-volume tenant. Many small indices burn shard budget. Consolidate into shared indices keyed by tenant_id, or switch to data streams.
- Too high a default
number_of_shardsin an index template. A template settingnumber_of_shards: 5for indices that hold 2 GB of data wastes 4 shards per index. - A bulk reindex that doubles total shard count for the migration window. Plan the reindex to delete the source after cutover, not before.
Fix the underlying issue with shrink, delete, or rollover before raising the limit. Raising the cap without addressing the cause buys time and accumulates heap pressure.
Operational Impact
Each open shard costs roughly 1-2 MB of JVM heap for cluster state, plus heap for filter caches, segment metadata, and fielddata. At the default 1000 shards per node, a node with a 30 GB heap will spend 2-4 GB just on shard overhead before serving a single query. Symptoms of being close to the practical (not configured) ceiling:
- Slow cluster state propagation;
GET /_cluster/statetakes seconds. - High GC time on the master node.
- Long node-join times when a node restarts.
_cat/recoveryshowing many tiny shards in motion after every restart.
Common Mistakes
- Treating 1000 as a target. It is a safety limit. Aim for 20 shards per GB of heap.
- Raising the limit during an outage. This hides the real problem (oversharding) and makes the next crisis worse.
- Forgetting closed indices when calculating headroom. Closed indices don't count toward the limit, but reopening them does. Plan capacity assuming everything is open.
- Different limits across environments. Keep dev and prod consistent so capacity issues surface before production.
Prevent the max-shards-per-node Validation Exception with Pulse
Pulse is an AI DBA for Elasticsearch and OpenSearch that tracks cluster.max_shards_per_node (default 1000) and cluster.max_shards_per_node.frozen (default 3000) against actual open-shard counts on every data node, flagging:
- Drift between the configured cap and the recommended ceiling of 20 shards per GB of heap
- Settings that are unsafe for your workload (e.g. cap raised to 2000 while heap is at 16 GB, an ILM policy that has stopped deleting old daily indices, or an index template with
number_of_shards: 5for indices that hold under 5 GB) - The downstream operational impact: cluster state propagation time, master GC pressure, node-join time after restart
Pulse names the underlying cause - oversharding, missing ILM delete phase, wrong template default - before the next validation_exception: Validation Failed: this action would add [N] total shards blocks an index creation.
Frequently Asked Questions
Q: What is the default value of cluster.max_shards_per_node in Elasticsearch?
A: The default is 1000 since Elasticsearch 7.0. This counts open primary plus replica shards on regular data nodes. Frozen data nodes have a separate setting, cluster.max_shards_per_node.frozen, which defaults to 3000.
Q: Can I change cluster.max_shards_per_node without restarting Elasticsearch?
A: Yes. cluster.max_shards_per_node is a dynamic cluster setting. Update it through PUT /_cluster/settings and the change takes effect immediately across the cluster.
Q: Do closed indices count toward cluster.max_shards_per_node?
A: No. Only open shards count. Closed indices, frozen indices, and searchable snapshot indices are excluded. Reopening a closed index re-adds its shards to the count and can trigger the limit.
Q: What error do I get when I exceed cluster.max_shards_per_node?
A: Elasticsearch rejects the create-index or open-index call with validation_exception: Validation Failed: this action would add [N] total shards, but this cluster currently has [M]/[max] maximum shards open.
Q: Should I raise cluster.max_shards_per_node to fix too many shards errors?
A: Usually no. Raising the cap hides the problem. The right response is to consolidate small indices, delete or roll over old data, and tighten number_of_shards defaults in index templates. Raise the limit only after confirming the remaining shards are appropriately sized (10-50 GB each).
Q: How does cluster.max_shards_per_node relate to JVM heap?
A: Each open shard consumes roughly 1-2 MB of heap for cluster state plus more for caches. Elasticsearch recommends staying under 20 shards per GB of heap. The default 1000 is a hard ceiling, not the recommended operating point.
Q: What's the best tool to prevent hitting cluster.max_shards_per_node?
A: Pulse is built for this. It is an AI DBA for Elasticsearch and OpenSearch that tracks open-shard counts against the configured cap and the 20-shards-per-GB heap guideline, flags ILM policies that have stopped deleting old indices, and recommends consolidation or template fixes before the next index creation fails with validation_exception.
