The cluster.lifecycle.default.rollover setting in Elasticsearch controls the default rollover configuration for index lifecycle management (ILM) policies. It determines the conditions under which a new index is created and the current index is rolled over when no specific rollover action is defined in an ILM policy.
- Default value:
{"max_age": "30d", "max_size": "50gb", "max_docs": 100000000} - Possible values: A JSON object containing one or more of the following keys:
max_age,max_size, andmax_docs - Recommendation: Adjust based on your specific use case and data growth patterns
This setting provides a cluster-wide default for rollover conditions when not explicitly defined in individual ILM policies. It helps ensure that indices don't grow too large or too old, which can impact performance and manageability.
This setting is available in Elasticsearch 7.0 and later versions.
Example
To change the default rollover settings using the cluster settings API:
PUT _cluster/settings
{
"persistent": {
"cluster.lifecycle.default.rollover": {
"max_age": "14d",
"max_size": "30gb",
"max_docs": 50000000
}
}
}
In this example, we're changing the default rollover conditions to create a new index when the current index reaches 14 days old, 30GB in size, or 50 million documents, whichever comes first. This might be desirable for environments with faster data growth or stricter retention policies.
Common Issues and Misuses
- Setting values too low can lead to excessive index creation and management overhead
- Setting values too high may result in oversized indices that are difficult to manage or recover
- Forgetting to account for this default when designing ILM policies can lead to unexpected rollover behavior
Do's and Don'ts
Do's:
- Regularly review and adjust this setting based on your data patterns and infrastructure capabilities
- Consider different rollover strategies for different types of indices (e.g., logs vs. metrics)
- Use this setting as a safety net, but define specific rollover actions in ILM policies for more granular control
Don'ts:
- Don't set extreme values that could lead to index management issues
- Don't rely solely on this default setting for critical indices; always define specific ILM policies
- Don't forget to monitor the effects of changes to this setting on your cluster's performance and storage utilization
Frequently Asked Questions
Q: How does the cluster.lifecycle.default.rollover setting interact with ILM policies?
A: This setting provides default rollover conditions that are used when an ILM policy doesn't specify its own rollover action. If a policy defines its own rollover conditions, those will take precedence over the cluster-wide default.
Q: Can I disable one of the default rollover conditions?
A: Yes, you can omit any of the max_age, max_size, or max_docs keys from the setting. For example, if you only want to rollover based on size and document count, you can exclude the max_age condition.
Q: How often are these rollover conditions checked?
A: Elasticsearch periodically checks indices against their rollover conditions. The frequency of these checks can be adjusted using the indices.lifecycle.poll_interval setting, which defaults to 10 minutes.
Q: Will changing this setting affect existing indices?
A: Changing this setting will only affect new indices or those that haven't yet reached their rollover conditions. Existing indices that have already been created or rolled over won't be immediately impacted.
Q: How can I monitor when rollovers occur due to these default conditions?
A: You can monitor rollover events through Elasticsearch's indexing and ILM-related logs. Additionally, you can use the Index Management UI in Kibana or the ILM Explain API to view the status of indices and their lifecycle progress.
