The data_streams.lifecycle.target.merge.policy.floor_segment setting in Elasticsearch controls the minimum segment size that should be considered for merging during the data stream lifecycle management process.
- Default value: 2mb
- Possible values: Any byte size value (e.g., 1mb, 5mb, 10mb)
- Recommendations: The default value is suitable for most use cases. Adjust only if you have specific performance requirements or storage constraints.
This setting determines the smallest segment size that the merge policy will consider for merging. Segments smaller than this value will be merged more aggressively, which can help reduce the total number of segments and improve search performance.
This setting is available in Elasticsearch 7.10 and later versions.
Example
To change the data_streams.lifecycle.target.merge.policy.floor_segment setting using the cluster settings API:
PUT _cluster/settings
{
"persistent": {
"data_streams.lifecycle.target.merge.policy.floor_segment": "5mb"
}
}
Reason for change: You might want to increase this value if you have many small segments and want to reduce the frequency of merges, potentially improving indexing performance at the cost of slightly slower searches.
Effects of change: Increasing this value will result in more small segments being retained, which can lead to slower searches but potentially faster indexing. Decreasing the value will cause more aggressive merging of small segments, potentially improving search performance but increasing the indexing overhead.
Common Issues or Misuses
- Setting the value too high can lead to a large number of small segments, which may negatively impact search performance.
- Setting the value too low can cause excessive merging, which may impact indexing performance and increase I/O operations.
Do's and Don'ts
Do's:
- Monitor your segment sizes and merge times to determine if adjustments are necessary.
- Consider your specific use case and data patterns when adjusting this setting.
- Test changes in a non-production environment before applying them to production.
Don'ts:
- Don't set this value extremely low (e.g., below 1mb) as it can lead to excessive merging.
- Avoid frequent changes to this setting, as it may disrupt ongoing merge processes.
- Don't ignore other related merge policy settings when tuning this one.
Frequently Asked Questions
Q: How does this setting affect index performance?
A: This setting influences the balance between indexing and search performance. A lower value can improve search performance by reducing the number of segments, while a higher value may improve indexing speed by reducing merge frequency.
Q: Can I change this setting for individual data streams?
A: No, this is a cluster-level setting that applies to all data streams. For index-specific control, you would need to use index-level merge policy settings.
Q: How does this setting interact with other merge policy settings?
A: This setting works in conjunction with other merge policy settings like max_merged_segment and segments_per_tier. It's important to consider all these settings together when optimizing merge behavior.
Q: Will changing this setting affect existing segments?
A: Changing this setting will not immediately affect existing segments, but it will influence future merge operations as they occur.
Q: How can I monitor the impact of changing this setting?
A: You can use Elasticsearch's index stats API to monitor segment counts and sizes, and the _cat/segments API to view detailed segment information. Additionally, keep an eye on indexing and search latencies to assess the overall impact.
