The cluster.max_shards_per_node.frozen setting in Elasticsearch controls the maximum number of shards (primary and replica) allowed on a single node for frozen indices. This setting helps prevent a node from becoming overloaded with too many frozen shards, which could impact performance and stability.
- Default value: 3000
- Possible values: Any positive integer
- Recommendation: Adjust based on your hardware capabilities and specific use case
This setting is specific to frozen indices, which are a special type of index designed for infrequently accessed, historical data. Frozen indices consume minimal resources when not in use, making them ideal for long-term storage of large amounts of data.
The default value of 3000 is generally suitable for most use cases. However, you may need to adjust this value based on your specific hardware configuration, data volume, and query patterns.
Example
To change the cluster.max_shards_per_node.frozen setting using the cluster settings API:
PUT _cluster/settings
{
"persistent": {
"cluster.max_shards_per_node.frozen": 4000
}
}
You might want to increase this value if you have powerful nodes capable of handling more frozen shards or if you have a large number of frozen indices that need to be distributed across your cluster. Conversely, you might want to decrease this value if you're experiencing performance issues related to too many frozen shards on a single node.
Common Issues and Misuses
- Setting the value too high can lead to nodes becoming overloaded, potentially causing out-of-memory errors or slow performance.
- Setting the value too low can prevent the cluster from allocating all necessary shards, leading to unassigned shards and incomplete data distribution.
Do's and Don'ts
- Do monitor your cluster's performance and adjust this setting as needed.
- Do consider your hardware capabilities when setting this value.
- Don't set this value arbitrarily high without understanding the potential consequences.
- Don't ignore this setting when working with large numbers of frozen indices.
Frequently Asked Questions
Q: How does this setting differ from the regular cluster.max_shards_per_node setting?
A: The cluster.max_shards_per_node.frozen setting applies specifically to frozen indices, while cluster.max_shards_per_node applies to all other index types. This allows for more granular control over shard allocation for different types of indices.
Q: Can I set different values for different nodes in my cluster?
A: No, this is a cluster-wide setting. It applies uniformly to all nodes in the cluster capable of holding frozen indices.
Q: What happens if I try to allocate more frozen shards than this setting allows?
A: Elasticsearch will not allocate additional frozen shards to a node that has reached its limit. This can result in unassigned shards in your cluster.
Q: How does this setting interact with the index lifecycle management (ILM) freeze action?
A: When ILM freezes an index, it will respect this setting. If freezing an index would cause a node to exceed its frozen shard limit, the freeze action may be delayed or the shards may be allocated to a different node.
Q: Should I adjust this setting if I'm not using frozen indices?
A: If you're not using frozen indices in your cluster, this setting will have no effect. You can leave it at its default value.
