The index.routing.allocation.include._id setting in Elasticsearch is used to control shard allocation based on node IDs. It allows you to specify which nodes are eligible to host shards of a particular index by including specific node IDs.
- Default value: Not set (empty)
- Possible values: Comma-separated list of node IDs
- Recommendations: Use this setting when you need fine-grained control over shard placement on specific nodes.
This setting is part of the shard allocation filtering mechanism in Elasticsearch. When set, it ensures that shards of the index are only allocated to nodes whose IDs match the specified values. This can be useful for scenarios where you want to place certain indices on specific nodes for performance, hardware, or organizational reasons.
Example
To set the index.routing.allocation.include._id for an existing index:
PUT /my-index/_settings
{
"index.routing.allocation.include._id": "node1,node2,node3"
}
This example restricts the allocation of shards for my-index to only the nodes with IDs node1, node2, and node3.
Reason for change: You might want to use this setting when you have specialized nodes (e.g., nodes with SSDs) and want to ensure that certain indices are only allocated to these nodes.
Effect: After applying this setting, Elasticsearch will attempt to move the shards of the index to the specified nodes if they are not already there. If none of the specified nodes are available, the shards may become unassigned.
Common Issues or Misuses
- Over-constraining shard allocation, leading to unassigned shards if specified nodes are unavailable
- Forgetting to update the setting when adding or removing nodes, potentially causing allocation imbalances
Do's and Don'ts
Do's:
- Use this setting in conjunction with other allocation settings for more flexible control
- Regularly review and update the setting as your cluster topology changes
- Consider using this for specialized hardware allocation (e.g., faster storage for frequently accessed indices)
Don'ts:
- Don't use this setting as the sole means of controlling data locality; consider other factors like rack awareness
- Avoid setting this for all indices unless you have a very specific cluster architecture in mind
- Don't forget to remove or update the setting when decommissioning nodes
Frequently Asked Questions
Q: How does index.routing.allocation.include._id differ from node.attr based allocation?
A: While both can be used for shard allocation control, index.routing.allocation.include._id directly specifies node IDs, whereas node attributes allow for more flexible, tag-based allocation that can be easier to manage in larger, dynamic clusters.
Q: Can I use wildcards or regular expressions with this setting?
A: No, index.routing.allocation.include._id only accepts exact node IDs. For more flexible matching, consider using node attributes or other allocation settings.
Q: What happens if I specify a node ID that doesn't exist in my cluster?
A: Non-existent node IDs are simply ignored. The setting will only affect allocation for the valid node IDs specified.
Q: Is it possible to use this setting in combination with exclude settings?
A: Yes, you can combine include and exclude settings. Elasticsearch will allocate shards to nodes that match the include criteria but do not match any exclude criteria.
Q: How does this setting interact with the cluster-wide shard allocation settings?
A: This index-level setting takes precedence over cluster-wide allocation settings. However, it still respects other constraints like the total number of shards per node.
