Elasticsearch cluster.blocks.read_only_allow_delete Setting

The cluster.blocks.read_only_allow_delete setting in Elasticsearch is a crucial cluster-wide setting that controls whether the cluster enters a read-only state while still allowing delete operations. This setting is primarily used as a protective measure when the cluster is running low on disk space.

• Default Value: false
• Possible Values: true or false
• Recommendations: It's recommended to leave this setting at its default value and allow Elasticsearch to manage it automatically based on disk space thresholds.

When set to true, this setting puts the entire cluster into a read-only state where new documents cannot be indexed, and existing documents cannot be updated. However, it still allows delete operations, which can help free up disk space in emergency situations.

Example

To manually set the cluster to read-only mode allowing deletes:

PUT _cluster/settings
{
  "persistent": {
    "cluster.blocks.read_only_allow_delete": true
  }
}

This might be necessary if you need to prevent new data from being written while you investigate and resolve disk space issues. Once the disk space problem is resolved, you can revert the setting:

PUT _cluster/settings
{
  "persistent": {
    "cluster.blocks.read_only_allow_delete": null
  }
}

Common Issues and Misuses

1. Forgetting to reset the setting after resolving disk space issues, leaving the cluster in a read-only state.
2. Manually setting this to true without understanding its implications on ongoing operations.
3. Relying on this setting as a primary method of managing disk space instead of proper capacity planning.

Do's and Don'ts

Do's:

• Monitor disk space regularly to prevent reaching critical levels.
• Use this setting as a last resort for protecting your cluster from running out of disk space.
• Always check this setting if you unexpectedly can't write to your cluster.

Don'ts:

• Don't manually set this to true unless absolutely necessary.
• Don't forget to reset it to false once disk space issues are resolved.
• Don't ignore warnings about low disk space that may lead to this setting being automatically applied.

Frequently Asked Questions

Q: What triggers Elasticsearch to automatically set cluster.blocks.read_only_allow_delete to true?
A: Elasticsearch automatically sets this to true when any data node in the cluster reaches the flood-stage disk watermark, which by default is 95% disk usage.

Q: Can I still perform search operations when this setting is true?
A: Yes, search operations are still allowed. The cluster enters a read-only state, but existing data can still be read and searched.

Q: Why does Elasticsearch allow delete operations when in this read-only state?
A: Delete operations are allowed to provide a way to free up disk space, which is crucial when the cluster is in a state where it's running out of storage.

Q: How can I prevent Elasticsearch from automatically setting this to true?
A: Proper disk space management is key. Monitor your disk usage, add more storage when needed, and consider adjusting your disk watermark settings if necessary.

Q: After resolving disk space issues, why am I still unable to index new documents?
A: You may need to manually reset the cluster.blocks.read_only_allow_delete setting to false or null, as it doesn't automatically revert once disk space is freed up.