Elasticsearch thread_pool.search.queue_size Setting

The thread_pool.search.queue_size setting in Elasticsearch controls the size of the queue for search operations. This queue holds pending search requests when all search threads are busy.

  • Default value: 1000
  • Possible values: Any positive integer
  • Recommendations: The optimal value depends on your specific use case and hardware resources. Increasing this value allows more search requests to be queued, potentially reducing rejected requests during high load periods.

Example

To change the search queue size to 2000:

thread_pool:
  search:
    queue_size: 2000

Reasons for changing:

  • High number of rejected search requests
  • Desire to handle more concurrent search operations

Effects of the change:

  • Increased memory usage
  • Potentially longer wait times for individual requests during high load

Common Issues

  • Setting the value too high can lead to excessive memory usage and longer wait times for requests
  • Setting the value too low may result in too many rejected requests during peak loads

Do's and Don'ts

Do:

  • Monitor rejected requests and adjust the queue size accordingly
  • Consider your available memory when increasing this value
  • Balance this setting with other performance tuning parameters

Don't:

  • Set an extremely high value without considering the memory implications
  • Ignore other factors that might be causing performance issues, such as inadequate hardware resources

Frequently Asked Questions

Q: How does thread_pool.search.queue_size affect search performance?
A: It determines how many search requests can be queued when all search threads are busy. A larger queue can reduce rejected requests but may increase overall latency during high load.

Q: Can increasing thread_pool.search.queue_size solve all my search performance issues?
A: No, it's just one factor. You should also consider hardware resources, index design, and query optimization.

Q: What happens if the search queue is full?
A: Elasticsearch will reject new search requests with a "429 Too Many Requests" error until space becomes available in the queue.

Q: How can I monitor if my search queue is filling up?
A: Use Elasticsearch's Cat Thread Pool API or monitor the thread_pool.search.queue metric in your monitoring system.

Q: Should I set different queue sizes for different nodes in my cluster?
A: Generally, it's best to keep this setting consistent across all nodes. However, if you have dedicated search nodes with more resources, you might consider larger queue sizes for those specific nodes.

Pulse - Elasticsearch Operations Done Right

Stop googling errors and staring at dashboards.

Free Trial

Subscribe to the Pulse Newsletter

Get early access to new Pulse features, insightful blogs & exclusive events , webinars, and workshops.