The cluster.routing.allocation.disk.watermark.flood_stage.frozen setting controls the disk usage threshold for frozen indices at which Elasticsearch will attempt to relocate or cancel allocation of frozen shards on a node.
Description
- Default value: 95%
- Possible values: Percentage (e.g., 95%) or byte values (e.g., 20GB)
- Recommendations: Set this value higher than the regular flood stage watermark but lower than 100% to allow for some buffer.
This setting is part of the disk-based shard allocation mechanism in Elasticsearch. It specifically targets frozen indices, which are indices that have been moved to a more compressed, read-only state to save resources. When the disk usage on a node reaches this threshold for frozen indices, Elasticsearch will attempt to relocate frozen shards from that node or prevent new frozen shard allocations to it.
Example
To change this setting using the cluster settings API:
PUT _cluster/settings
{
"persistent": {
"cluster.routing.allocation.disk.watermark.flood_stage.frozen": "97%"
}
}
You might want to change this setting if you have a large number of frozen indices and want to fine-tune when Elasticsearch takes action to prevent disk full errors. Increasing this value allows for more disk usage before Elasticsearch intervenes, while decreasing it provides a larger safety margin.
Common Issues
- Setting this value too high can lead to disk full errors if Elasticsearch doesn't have enough time to relocate shards.
- Setting it too low might cause unnecessary shard relocations, potentially impacting cluster performance.
Do's and Don'ts
- Do monitor disk usage alongside this setting to ensure it's appropriate for your cluster.
- Do set this value higher than the regular flood stage watermark.
- Don't set this value to 100%, as it leaves no room for error or temporary spikes in disk usage.
- Don't ignore warnings related to this watermark being reached.
Frequently Asked Questions
Q: How does this setting differ from the regular flood stage watermark?
A: This setting is specific to frozen indices, which typically consume less resources and are accessed less frequently than active indices. It allows for higher disk usage before taking action on frozen shards.
Q: What happens when this watermark is reached?
A: Elasticsearch will attempt to relocate frozen shards from the affected node to other nodes with more available disk space. If relocation is not possible, it may prevent new frozen shard allocations to the node.
Q: Can this setting be changed dynamically?
A: Yes, this setting can be updated dynamically using the cluster settings API without requiring a cluster restart.
Q: How does this setting interact with other disk-based allocation settings?
A: This setting works in conjunction with other disk watermark settings but is specifically for frozen indices. It should be set higher than the regular flood stage watermark but lower than 100%.
Q: Should I use a percentage or absolute value for this setting?
A: Either can be used. Percentages are often more flexible as they adapt to different disk sizes across nodes, while absolute values can be useful for ensuring a specific amount of free space regardless of total disk size.
