What is index.routing.allocation.include.?
The index.routing.allocation.include.<custom_attribute> setting in Elasticsearch is used to control shard allocation based on custom node attributes. It allows you to specify which nodes are eligible to host shards of a particular index based on custom attributes assigned to those nodes.
Description
- Default value: Not set
- Possible values: Comma-separated list of attribute values
- Recommendation: Use this setting to create logical groupings of nodes and control shard placement for specific indices
This setting is part of the index-level shard allocation settings. It works in conjunction with custom node attributes that you define. When set, Elasticsearch will only allocate shards of the index to nodes that have matching attribute values.
Version Information
This setting has been available since early versions of Elasticsearch and continues to be supported in current versions.
Example
To set the index.routing.allocation.include.<custom_attribute> for an existing index:
PUT /my-index/_settings
{
"index.routing.allocation.include.data_type": "hot,warm"
}
In this example, we're setting the index to only be allocated on nodes with the custom attribute data_type set to either "hot" or "warm".
Reason for change: You might want to use this setting to implement hot-warm-cold architectures, where certain indices are placed on faster (hot) nodes, while others are on slower (warm or cold) nodes.
Effect: After applying this setting, Elasticsearch will attempt to move shards of the index to nodes that match the specified attribute values, potentially triggering shard relocations.
Common Issues or Misuses
- Setting overly restrictive allocation rules that prevent proper shard distribution
- Forgetting to set the corresponding node attributes, resulting in unallocated shards
- Using too many custom attributes, which can complicate cluster management
Do's and Don'ts
Do's:
- Use this setting to implement data tiering strategies
- Combine with other allocation settings for fine-grained control
- Document your custom attributes and allocation policies
Don'ts:
- Don't use this setting without a clear allocation strategy
- Avoid changing allocation settings frequently, as it can lead to excessive shard movements
- Don't rely solely on this setting for critical data placement; consider using multiple allocation rules
Frequently Asked Questions
Q: How does this setting differ from index.routing.allocation.require.
A: While include allows allocation if any of the specified values match, require mandates that all specified values must match for allocation to occur.
Q: Can I use wildcards in the attribute values?
A: Yes, you can use wildcards. For example, "data_*" would match nodes with attributes like "data_hot", "data_warm", etc.
Q: What happens if no nodes match the specified attribute values?
A: If no nodes match the criteria, the shards will remain unassigned until matching nodes become available or the allocation rules are changed.
Q: How does this setting interact with other allocation settings?
A: This setting is combined with other allocation settings using AND logic. All conditions must be met for a shard to be allocated to a node.
Q: Can I use this setting to isolate indices for specific customers in a multi-tenant setup?
A: Yes, this is a common use case. You can assign customer-specific attributes to nodes and use this setting to ensure each customer's data is allocated to the appropriate nodes.
