The index.routing.allocation.include setting in Elasticsearch is used to control shard allocation based on node attributes on an index-level. It allows you to specify which nodes are eligible to host shards of a particular index.
- Default value: Not set (no restrictions)
- Possible values: A comma-separated list of key-value pairs (e.g.,
tag:value1,value2) - Recommendation: Use this setting to distribute shards across nodes with specific attributes for better control over data placement and hardware utilization.
This setting works in conjunction with node attributes. Shards will only be allocated to nodes that have at least one of the specified values for the given attribute.
Example
To set the index.routing.allocation.include for an existing index:
PUT /my-index/_settings
{
"index.routing.allocation.include": {
"rack": "rack1,rack2"
}
}
This example restricts shard allocation for my-index to nodes with the rack attribute set to either rack1 or rack2.
Reason for change: You might want to ensure that certain indices are only allocated to specific nodes, perhaps for performance reasons or to keep sensitive data on particular hardware.
Effect: Shards of the index will only be allocated to nodes matching the specified attributes, potentially causing reallocation of existing shards.
Common Issues or Misuses
- Over-restricting shard allocation, leading to unassigned shards if no nodes match the criteria
- Forgetting to set the corresponding node attributes, resulting in unexpected allocation behavior
- Using conflicting include/exclude settings, which may prevent shard allocation
Do's and Don'ts
- Do use this setting in conjunction with
index.routing.allocation.excludeandindex.routing.allocation.requirefor fine-grained control - Do ensure that enough nodes have the specified attributes to maintain desired replication
- Don't use overly complex allocation rules that may be difficult to manage or debug
- Don't forget to update node attributes when changing hardware or node roles
Frequently Asked Questions
Q: How does index.routing.allocation.include differ from index.routing.allocation.require?
A: include allows allocation if any of the specified values match, while require demands all specified values to match for allocation to occur.
Q: Can I use wildcards in the attribute values?
A: Yes, you can use wildcards. For example, rack:rack* would match any rack attribute starting with "rack".
Q: What happens if no nodes match the include criteria?
A: Shards will remain unassigned until nodes matching the criteria become available or the allocation settings are changed.
Q: How can I view the current allocation settings for an index?
A: Use the Get Index Settings API: GET /my-index/_settings/index.routing.allocation.*
Q: Is it possible to use multiple attributes in the include setting?
A: Yes, you can specify multiple attributes, e.g., {"rack": "rack1,rack2", "size": "large,medium"}.
