What is a refresh interval?
Refresh interval in Elasticsearch is the time period between index refreshes. A refresh makes recent changes to an index visible to search. By default, Elasticsearch refreshes indices every second, which is why it's often described as providing near real-time search capabilities. The refresh interval can be configured to balance between search performance and indexing throughput.
The refresh operation is relatively lightweight but still has a cost. It creates a new segment in memory, which later gets merged and committed to disk. While a shorter refresh interval provides more up-to-date search results, it can impact indexing performance and increase segment counts.
Elasticsearch also provides the index.refresh_interval
setting at the index level, allowing for fine-grained control over different indices based on their specific requirements.
Best practices
Adjust refresh interval based on your use case:
- For write-heavy workloads, increase the refresh interval to reduce indexing overhead.
- For search-heavy applications requiring up-to-date results, keep the interval short.
Use
?refresh=wait_for
on indexing requests when you need immediate visibility of new documents.Consider setting
refresh_interval: -1
for bulk indexing operations to disable automatic refreshes.Monitor refresh times using the
_cat/indices
API to ensure they're not impacting overall cluster performance.Use time-based indices with different refresh intervals for hot and cold data.
Common issues or misuses
Setting refresh interval too low, causing unnecessary load on the cluster.
Forgetting to re-enable refreshes after bulk indexing operations.
Misunderstanding the relationship between refresh interval and transaction log flushes.
Expecting real-time consistency with default settings in distributed environments.
Overlooking the impact of refresh interval on segment merging and overall indexing performance.
Frequently Asked Questions
Q: How does refresh interval affect search latency?
A: A shorter refresh interval reduces search latency by making new documents visible more quickly, but it can increase overall system load due to more frequent refresh operations.
Q: Can I change the refresh interval dynamically?
A: Yes, you can update the refresh interval dynamically using the Update Index Settings API without closing the index.
Q: What's the difference between refresh and flush operations?
A: Refresh makes recent changes searchable without writing to disk, while flush writes the transaction log and in-memory segments to disk, clearing the transaction log.
Q: How does refresh interval impact indexing throughput?
A: A longer refresh interval generally improves indexing throughput by reducing the frequency of refresh operations, allowing more documents to be indexed between refreshes.
Q: Is it possible to trigger a manual refresh?
A: Yes, you can trigger a manual refresh using the Refresh API by sending a POST request to /_refresh
or /{index}/_refresh
for specific indices.