The thread_pool.search.queue_size
setting in Elasticsearch controls the size of the queue for search operations. This queue holds pending search requests when all search threads are busy.
- Default value: 1000
- Possible values: Any positive integer
- Recommendations: The optimal value depends on your specific use case and hardware resources. Increasing this value allows more search requests to be queued, potentially reducing rejected requests during high load periods.
Example
To change the search queue size to 2000:
thread_pool:
search:
queue_size: 2000
Reasons for changing:
- High number of rejected search requests
- Desire to handle more concurrent search operations
Effects of the change:
- Increased memory usage
- Potentially longer wait times for individual requests during high load
Common Issues
- Setting the value too high can lead to excessive memory usage and longer wait times for requests
- Setting the value too low may result in too many rejected requests during peak loads
Do's and Don'ts
Do:
- Monitor rejected requests and adjust the queue size accordingly
- Consider your available memory when increasing this value
- Balance this setting with other performance tuning parameters
Don't:
- Set an extremely high value without considering the memory implications
- Ignore other factors that might be causing performance issues, such as inadequate hardware resources
Frequently Asked Questions
Q: How does thread_pool.search.queue_size affect search performance?
A: It determines how many search requests can be queued when all search threads are busy. A larger queue can reduce rejected requests but may increase overall latency during high load.
Q: Can increasing thread_pool.search.queue_size solve all my search performance issues?
A: No, it's just one factor. You should also consider hardware resources, index design, and query optimization.
Q: What happens if the search queue is full?
A: Elasticsearch will reject new search requests with a "429 Too Many Requests" error until space becomes available in the queue.
Q: How can I monitor if my search queue is filling up?
A: Use Elasticsearch's Cat Thread Pool API or monitor the thread_pool.search.queue
metric in your monitoring system.
Q: Should I set different queue sizes for different nodes in my cluster?
A: Generally, it's best to keep this setting consistent across all nodes. However, if you have dedicated search nodes with more resources, you might consider larger queue sizes for those specific nodes.