The cluster.routing.allocation.node_concurrent_incoming_recoveries setting in Elasticsearch controls the number of concurrent incoming shard recoveries allowed on a single node. This setting is crucial for managing the resource utilization during cluster rebalancing and recovery operations.
Description
- Default Value: 2
- Possible Values: Any positive integer
- Recommendation: The default value is suitable for most scenarios, but it can be adjusted based on the cluster's hardware capabilities and recovery requirements.
This setting limits the number of shards that can be concurrently recovered on a single node from other nodes. It helps prevent overwhelming a node with too many simultaneous recovery operations, which could impact its performance and stability.
Example
To change the value of this setting using the cluster settings API:
PUT _cluster/settings
{
"persistent": {
"cluster.routing.allocation.node_concurrent_incoming_recoveries": 3
}
}
You might want to increase this value if you have powerful nodes with ample resources and need to speed up the recovery process during large-scale rebalancing or when adding new nodes. However, be cautious as setting it too high can overload nodes and negatively impact cluster performance.
Common Issues and Misuses
- Setting the value too high can lead to excessive resource consumption on nodes, potentially causing performance issues or node instability.
- Setting the value too low can significantly slow down the recovery process, especially in large clusters or during major rebalancing operations.
Do's and Don'ts
- Do monitor node performance and recovery times when adjusting this setting.
- Do consider the hardware capabilities of your nodes when changing this value.
- Don't set this value excessively high without careful testing and monitoring.
- Don't forget to adjust related settings like
cluster.routing.allocation.node_concurrent_recoveriesfor a balanced configuration.
Frequently Asked Questions
Q: How does this setting differ from cluster.routing.allocation.node_concurrent_recoveries?
A: While node_concurrent_incoming_recoveries specifically limits incoming recoveries, node_concurrent_recoveries sets an overall limit for both incoming and outgoing recoveries on a node.
Q: Can changing this setting impact cluster stability?
A: Yes, setting it too high can overload nodes and potentially cause stability issues. It's important to change this setting cautiously and monitor the effects.
Q: Should I adjust this setting in a production environment?
A: It's generally safe to adjust in production, but always test changes in a staging environment first and implement them during low-traffic periods.
Q: How does this setting affect the overall recovery speed of the cluster?
A: Increasing this value can potentially speed up overall cluster recovery by allowing more parallel recoveries, but it also increases the load on individual nodes.
Q: Is there a recommended value for large clusters?
A: The optimal value depends on your specific hardware and cluster configuration. For large clusters with powerful nodes, values between 2 and 4 are often suitable, but always test and monitor before settling on a value.
