Elasticsearch indices.lifecycle.rollover.only_if_has_documents Setting

The indices.lifecycle.rollover.only_if_has_documents setting in Elasticsearch controls whether Index Lifecycle Management (ILM) should roll over an index only if it contains documents. This setting helps prevent the creation of unnecessary empty indices during the rollover process.

• Default Value: true
• Possible Values: true or false
• Recommendation: Keep the default value unless you have a specific use case for rolling over empty indices.

This setting is particularly useful in scenarios where you want to ensure that new indices are created only when there's actual data to be stored. By default, Elasticsearch will not roll over an index if it's empty, which helps in maintaining a cleaner index structure and reduces unnecessary resource consumption.

This setting is available in Elasticsearch 7.10 and later versions.

Example

To change this setting using the cluster settings API:

PUT _cluster/settings
{
  "persistent": {
    "indices.lifecycle.rollover.only_if_has_documents": false
  }
}

Changing this setting to false would allow empty indices to be rolled over. This might be useful in specific scenarios, such as when you want to maintain a consistent time-based index pattern regardless of data presence.

Common Issues or Misuses

• Setting this to false without a clear reason can lead to the creation of many empty indices, potentially wasting storage and increasing management overhead.
• Misunderstanding this setting might lead to confusion when expecting rollovers for empty indices that don't occur by default.

Do's and Don'ts

• Do keep this setting true unless you have a specific need for rolling over empty indices.
• Do consider this setting when troubleshooting unexpected behavior in your ILM rollover policies.
• Don't set this to false without understanding the implications on your index management and storage usage.
• Don't ignore this setting when designing your ILM policies, especially if you expect periods of no data ingestion.

Frequently Asked Questions

Q: Why isn't my index rolling over even though the rollover conditions are met?
A: If the index is empty and indices.lifecycle.rollover.only_if_has_documents is set to true (default), the rollover won't occur. Ensure your index has documents or consider changing this setting if you need empty indices to roll over.

Q: Can I set this value differently for specific indices?
A: No, this is a cluster-level setting that applies to all indices managed by ILM. You cannot configure it on a per-index basis.

Q: How does this setting interact with other rollover conditions like max_age or max_size?
A: This setting takes precedence. Even if other rollover conditions are met, an empty index won't roll over if this setting is true.

Q: Will changing this setting affect existing indices?
A: The new setting value will be applied to future rollover attempts. Existing indices won't be immediately affected, but their next rollover check will use the new setting.

Q: How can I verify if an index rollover was skipped due to this setting?
A: Check the ILM explain API output for the index. It will indicate if a rollover was skipped because the index was empty when this setting is true.