The index.routing.allocation.exclude._tier_preference setting in Elasticsearch controls which data tiers an index should not be allocated to. It allows you to exclude specific tiers from the allocation process, providing fine-grained control over where your index data is stored within your cluster.
- Default value: Not set
- Possible values: Comma-separated list of tier names (e.g., "data_hot,data_warm,data_cold,data_frozen")
- Recommendations: Use this setting judiciously to ensure proper data distribution across tiers based on your performance and cost requirements.
This setting is part of Elasticsearch's data tiers feature, which was introduced in version 7.10.0. It works in conjunction with other tier preference settings to manage the lifecycle of your data across different storage tiers.
Example
To exclude an index from being allocated to the "data_cold" and "data_frozen" tiers, you can use the following cluster settings API call:
PUT /my-index/_settings
{
"index.routing.allocation.exclude._tier_preference": "data_cold,data_frozen"
}
This change would ensure that the index "my-index" is only allocated to nodes in the hot or warm tiers, which might be desirable for frequently accessed or performance-sensitive data.
Common Issues and Misuses
- Overuse of exclusions can lead to unbalanced cluster states or prevent proper index allocation.
- Conflicting settings between include, require, and exclude preferences can cause unexpected behavior.
- Forgetting to update this setting as your data lifecycle changes may result in suboptimal resource utilization.
Do's and Don'ts
- Do use this setting in conjunction with ILM (Index Lifecycle Management) policies for automated tier transitions.
- Do consider the performance implications of excluding certain tiers for frequently accessed indices.
- Don't set conflicting tier preferences that could prevent index allocation altogether.
- Don't forget to review and update these settings as your data access patterns evolve.
Frequently Asked Questions
Q: How does this setting interact with index.routing.allocation.include._tier_preference?
A: While exclude defines tiers where the index should not be allocated, include specifies tiers where it can be allocated. If both are set, Elasticsearch will allocate the index to tiers that are included but not excluded.
Q: Can I use this setting to move an index between tiers?
A: This setting alone doesn't move indices. It's best used with ILM policies or the move API for controlled transitions between tiers.
Q: What happens if I exclude all available tiers?
A: If all available tiers are excluded, the index will become unallocated and its shards will remain unassigned until the setting is changed or new compatible nodes are added to the cluster.
Q: How does this setting affect existing shard allocations?
A: Changing this setting may trigger shard relocation if current allocations violate the new rules. Elasticsearch will move shards to comply with the updated tier preferences.
Q: Is this setting dynamic, or does it require a restart?
A: This is a dynamic setting that can be updated on a live index without requiring a restart of the cluster or the index.
