Disable Caching in Elasticsearch and OpenSearch: Complete Guide

Caching in Elasticsearch and OpenSearch dramatically improves query performance by storing frequently accessed data in memory. However, there are scenarios where disabling caching is necessary or beneficial:

1. Benchmarking and performance testing to measure true query execution time
2. Troubleshooting cache-related issues or memory pressure
3. Queries on constantly changing data where cache hits provide no benefit
4. Debugging inconsistent search results that may be cached
5. Testing new queries or index configurations before enabling caching

Understanding Cache Types

Elasticsearch and OpenSearch maintain several distinct caches, each serving different purposes:

Request Cache (Shard Request Cache)

Caches the results of search requests at the shard level. When enabled, identical search requests against the same shard return cached results instantly without re-executing the query.

What gets cached:

• Query results for size=0 searches (aggregations, suggestions)
• Results when request_cache=true is specified
• Only caches if size=0 or the query is marked as cacheable

Cache invalidation:

• Automatically cleared when the index is refreshed
• Invalidated when any data changes occur in the shard

Query Cache (Node Query Cache)

Caches the results of filter clauses and frequently used query segments. This cache operates at the segment level and is shared across all indices on a node.

What gets cached:

• Filter contexts (must, must_not, filter clauses)
• Results of bool query filter clauses
• Frequently executed query segments

Cache invalidation:

• Uses LRU (Least Recently Used) eviction policy
• Cleared when segments are merged
• Automatically managed based on memory pressure

Field Data Cache

Stores field values in memory for sorting, aggregations, and scripting on text fields. Unlike doc values, field data is built on-demand and consumes heap memory.

What gets cached:

• Text field values for sorting and aggregations
• Script field access
• Parent-child relationships (in older versions)

Cache invalidation:

• Persists until the segment is deleted or the circuit breaker triggers
• Can grow unbounded if not monitored

Shard Query Cache

Caches the local results of the query phase before they're combined across shards. This is distinct from the request cache and operates during query execution.

Request-Level Cache Control

The most common and safest way to disable caching is at the request level, affecting only specific queries without impacting cluster-wide performance.

Disable Request Cache for Specific Queries

Use the request_cache parameter in your search requests:

GET /my-index/_search?request_cache=false
{
  "size": 0,
  "aggs": {
    "popular_tags": {
      "terms": {
        "field": "tags"
      }
    }
  }
}

This disables the request cache for this specific query, forcing fresh execution even if identical requests were cached.

Disable Query Cache for Specific Filters

Wrap filters that shouldn't be cached in a bool query with caching disabled:

GET /my-index/_search
{
  "query": {
    "bool": {
      "filter": {
        "bool": {
          "_cache": false,
          "must": [
            { "term": { "status": "active" } }
          ]
        }
      }
    }
  }
}

Note: The _cache parameter is deprecated in newer versions. Instead, use request-level caching controls.

Using the now Date Math to Prevent Caching

Queries using now date math are not cached by default since results change over time:

GET /logs-*/_search
{
  "query": {
    "range": {
      "timestamp": {
        "gte": "now-1h"
      }
    }
  }
}

Elasticsearch automatically detects now usage and prevents caching, ensuring fresh results.

Search Preference to Bypass Cache

Use the preference parameter with a unique value to bypass cached results:

GET /my-index/_search?preference=_local&request_cache=false
{
  "query": {
    "match_all": {}
  }
}

The preference parameter controls which shard copies handle the request, and different preferences may have separate cache entries.

Index-Level Cache Settings

Configure caching behavior for specific indices, affecting all queries against those indices unless overridden at request level.

Disable Request Cache at Index Level

PUT /my-index/_settings
{
  "index.requests.cache.enable": false
}

This disables the request cache for all queries against this index. Queries can still explicitly enable caching with request_cache=true.

Configure Request Cache During Index Creation

PUT /my-index
{
  "settings": {
    "index": {
      "requests.cache.enable": false,
      "number_of_shards": 1,
      "number_of_replicas": 1
    }
  }
}

Clear Specific Index Cache

Clear the cache for specific indices without disabling it:

POST /my-index/_cache/clear?request=true

Clear all cache types for an index:

POST /my-index/_cache/clear

Clear only specific cache types:

# Clear request cache only
POST /my-index/_cache/clear?request=true

# Clear query cache only
POST /my-index/_cache/clear?query=true

# Clear field data cache only
POST /my-index/_cache/clear?fielddata=true

Disable Field Data for Text Fields

Prevent field data caching on text fields by using doc_values or keyword fields instead:

PUT /my-index
{
  "mappings": {
    "properties": {
      "description": {
        "type": "text",
        "fielddata": false
      }
    }
  }
}

When disabled, attempts to sort or aggregate on this field will fail, preventing memory consumption.

Cluster-Level Cache Settings

Configure caching behavior cluster-wide, affecting all indices and queries unless overridden at lower levels.

Configure Request Cache Size

Set the maximum size of the request cache as a percentage of heap:

PUT /_cluster/settings
{
  "persistent": {
    "indices.requests.cache.size": "1%"
  }
}

Default: 1% of heap memory. Setting to 0% effectively disables request caching cluster-wide.

Configure Query Cache Size

Set the maximum size of the query cache:

PUT /_cluster/settings
{
  "persistent": {
    "indices.queries.cache.size": "10%"
  }
}

Default: 10% of heap memory. This cache is shared across all indices on a node.

Configure Field Data Cache Circuit Breaker

Limit field data cache to prevent OutOfMemory errors:

PUT /_cluster/settings
{
  "persistent": {
    "indices.breaker.fielddata.limit": "40%"
  }
}

Default: 40% of heap. When exceeded, queries requiring field data will fail rather than consume more memory.

Disable Caches in elasticsearch.yml

Configure cache settings permanently in the Elasticsearch configuration file:

# Reduce or disable request cache
indices.requests.cache.size: 0%

# Reduce query cache
indices.queries.cache.size: 5%

# Lower field data circuit breaker
indices.breaker.fielddata.limit: 30%

Note: Changes to elasticsearch.yml require a node restart to take effect.

View Current Cache Settings

Check current cache configuration:

GET /_cluster/settings?include_defaults=true&filter_path=**.cache,**.breaker

View cache statistics:

GET /_nodes/stats/indices/request_cache,query_cache,fielddata

When to Disable Caching

Recommended Scenarios

1. Performance Testing and Benchmarking

Disable caching to measure true query execution time without cache interference:

# Benchmark query without cache
GET /my-index/_search?request_cache=false
{
  "query": { "match_all": {} }
}

2. Rapidly Changing Data

For indices with very high write rates where cache hit rates are minimal:

PUT /real-time-events/_settings
{
  "index.requests.cache.enable": false
}

3. Debugging and Troubleshooting

Force fresh query execution to diagnose caching-related issues:

POST /my-index/_cache/clear
GET /my-index/_search?request_cache=false
{
  "query": { "term": { "status": "pending" } }
}

4. Memory-Constrained Environments

Reduce cache sizes in environments with limited heap:

PUT /_cluster/settings
{
  "persistent": {
    "indices.requests.cache.size": "0.5%",
    "indices.queries.cache.size": "5%"
  }
}

5. Unique or One-Time Queries

For queries that are never repeated, disable caching to avoid polluting the cache:

GET /logs-*/_search?request_cache=false
{
  "query": {
    "bool": {
      "filter": [
        { "term": { "user_id": "unique_value_12345" } }
      ]
    }
  }
}

Scenarios to Avoid Disabling

Dashboard and Reporting Queries

Frequently executed queries benefit significantly from caching. Keep caching enabled for:

• Kibana dashboards that refresh regularly
• Scheduled reports running identical queries
• Application queries with high repetition rates

Aggregation-Heavy Workloads

Aggregations on large datasets see massive performance improvements from caching:

• Analytics queries calculating metrics
• Time-series aggregations over historical data
• Cardinality and statistical aggregations

Production Search Applications

User-facing search applications where response time is critical should leverage caching for:

• Common search queries and filters
• Auto-complete and suggestion queries
• Filter facets and navigation queries

Performance Impact of Disabling Caches

Request Cache Disabled

Performance Impact:

• Queries execute from scratch every time, even if identical
• Aggregation queries (size=0) run significantly slower
• CPU and I/O usage increases for repeated queries
• Response times can increase 10-100x for cached queries

Memory Impact:

• Reduces heap usage by 1% (default cache size)
• Frees memory for other operations
• May improve garbage collection performance

When Acceptable:

• Low query repetition rate
• Fast-executing queries where cache overhead isn't beneficial
• Memory is more critical than response time

Query Cache Disabled

Performance Impact:

• Filter clauses re-execute on every query
• Bool query filters don't benefit from caching
• Segment-level query results aren't reused
• Complex filter chains execute slower

Memory Impact:

• Reduces heap usage by 10% (default cache size)
• Significant memory savings in multi-tenant environments
• May reduce garbage collection pressure

When Acceptable:

• Unique filter patterns with no repetition
• Indices with very high refresh rates
• Testing filter performance optimization

Field Data Cache Disabled

Performance Impact:

• Queries requiring field data will fail
• Forces use of doc_values (stored on disk)
• Slightly slower sorting and aggregations on text fields

Memory Impact:

• Eliminates unbounded heap growth from text field aggregations
• Prevents OutOfMemory errors on large text fields
• Strongly recommended for production stability

When Acceptable:

• Always prefer doc_values over field data when possible
• Only enable field data on small, known-cardinality text fields

Monitoring Cache Performance

Using Standard APIs

Monitor cache hit rates and effectiveness:

# Overall cache statistics
GET /_nodes/stats/indices?filter_path=**.request_cache,**.query_cache,**.fielddata

# Specific index cache stats
GET /my-index/_stats/request_cache,query_cache

# Per-node cache statistics
GET /_nodes/stats/indices/request_cache

Key metrics to monitor:

• hit_count vs miss_count: Cache effectiveness ratio
• evictions: Cache pressure indicator
• memory_size_in_bytes: Current memory consumption
• cache_size: Number of entries cached

Cache Performance Indicators

Good Cache Performance:

• Hit rate above 80% for request cache
• Low eviction rates (< 5% of total entries)
• Memory usage stable and below configured limits
• Query response times consistently fast

Poor Cache Performance:

• Hit rate below 50%
• High eviction rates indicating cache thrashing
• Memory usage at or near limits
• Frequent cache clears needed

Monitoring with Pulse

Pulse provides comprehensive cache monitoring and optimization insights:

Real-Time Cache Analysis:

• Cache hit/miss ratios across all cache types
• Eviction rate trends and patterns
• Memory consumption by cache type
• Per-index cache effectiveness

Performance Impact Tracking:

• Query performance with and without caching
• Cache contribution to response time improvements
• Memory pressure from cache usage
• Recommendations for cache size tuning

Proactive Alerts:

• Low cache hit rates indicating ineffective caching
• High eviction rates suggesting undersized caches
• Cache-related memory pressure
• Opportunities to disable caching on specific indices

Pulse helps you make data-driven decisions about cache configuration, identifying where caching provides value and where it's consuming resources without benefit.

Best Practices

Request-Level Control First

Prefer granular control:

• Use request_cache=false for specific queries rather than disabling globally
• Test cache impact query-by-query
• Keep beneficial caching enabled while disabling problematic cases

Monitor Before Disabling

Gather evidence:

• Check cache hit rates using stats APIs
• Measure query performance with and without caching
• Analyze memory consumption patterns
• Identify which cache types are actually being used

Gradual Changes

Make incremental adjustments:

• Disable caching on one index or query type at a time
• Monitor performance impact before expanding changes
• Document baselines before and after modifications
• Be prepared to roll back if performance degrades

Use Appropriate Field Types

Prevent field data usage:

• Use keyword type instead of text for aggregations and sorting
• Enable doc_values on numeric and keyword fields (enabled by default)
• Never enable field data on high-cardinality text fields
• Use fielddata: false explicitly to prevent accidents

Clear Caches Strategically

When to clear caches:

• After major index configuration changes
• When diagnosing performance issues
• Before accurate benchmarking sessions
• Not routinely in production

How to clear selectively:

# Clear only what you need
POST /specific-index/_cache/clear?request=true

# Avoid clearing all caches cluster-wide
# POST /_cache/clear  # Too aggressive for production

Configure Circuit Breakers

Protect against memory issues:

PUT /_cluster/settings
{
  "persistent": {
    "indices.breaker.fielddata.limit": "40%",
    "indices.breaker.request.limit": "60%",
    "indices.breaker.total.limit": "95%"
  }
}

Circuit breakers prevent runaway cache growth from causing OutOfMemory errors.

Frequently Asked Questions

Q: Will disabling the request cache improve indexing performance?
A: No, the request cache only affects search queries, not indexing operations. Disabling it won't improve indexing speed but may free up a small amount of heap memory that could indirectly help in memory-constrained environments.

Q: How do I know if my cache hit rate is good enough?
A: A good cache hit rate depends on your workload. For dashboard and reporting queries, aim for 80%+ hit rates. For user-generated search queries, 30-50% may be reasonable. Low hit rates (< 20%) indicate caching may not be beneficial for that workload.

Q: Does disabling caching affect all query types equally?
A: No. Aggregation queries (size=0) see the most dramatic performance impact from request cache. Filter-heavy queries are most affected by query cache. Full-text search queries may see minimal impact if they're rarely repeated.

Q: Can I disable caching for specific users or applications?
A: Not directly, but you can add request_cache=false to queries from specific applications. Some organizations use separate indices for different applications and configure caching per index based on usage patterns.

Q: What's the difference between clearing the cache and disabling it?
A: Clearing the cache removes current entries but allows new entries to be cached. Disabling the cache prevents new entries from being stored. Clear caches for troubleshooting; disable them when caching provides no benefit.

Q: How often does Elasticsearch automatically clear caches?
A: The request cache is cleared automatically when an index is refreshed (default: every 1 second). The query cache uses LRU eviction when memory limits are reached. Field data persists until segments are deleted or circuit breakers trigger.

Q: Will disabling caching reduce memory pressure during heap issues?
A: It can help, but the impact depends on cache sizes. Request cache (1% of heap) and query cache (10% of heap) can free significant memory. However, heap issues often stem from field data or other sources, so investigate the root cause first.

Q: Can I enable caching for some queries on an index where caching is disabled?
A: Yes. If you set index.requests.cache.enable: false, individual queries can still override with request_cache=true. Index settings provide the default, but request-level parameters take precedence.

Q: Does the cache work across shards and replicas?
A: The request cache is per-shard, meaning each shard (including replicas) maintains its own cache. The query cache is shared across all indices on a node. This means identical queries to different shards may not benefit from each other's cache.

Q: How do I benchmark query performance without cache interference?
A: Clear the cache first (POST /index/_cache/clear), then run your query with request_cache=false. Run the query multiple times and take the average, as the first execution may include JVM warm-up overhead. Consider using the Profile API for detailed execution analysis.

Q: Are there OpenSearch-specific differences in cache behavior?
A: OpenSearch maintains compatibility with Elasticsearch caching mechanisms. The APIs and settings are identical for most use cases. Some advanced features and default values may differ in newer versions, so consult the OpenSearch documentation for version-specific details.