An Elasticsearch index is a logical grouping of related JSON documents that share a mapping and settings. Physically, each index is split into one or more primary shards, each of which is a self-contained Lucene index, and zero or more replica shards for redundancy and read scaling. Indices are the unit of mapping, settings, lifecycle policy, and (loosely) access control - and the unit you pick when you query or reindex.
How an Elasticsearch Index Is Structured
An index has three things that matter: a mapping, settings, and a shard topology.
- Mapping defines field types and analyzers. See Elasticsearch mapping.
- Settings include
number_of_shards,number_of_replicas,refresh_interval, analyzer definitions, and dozens of others. - Shard topology is the distribution of primary and replica shards across nodes in the cluster.
PUT /events-2026.05
{
"settings": {
"number_of_shards": 3,
"number_of_replicas": 1,
"refresh_interval": "30s"
},
"mappings": {
"properties": {
"@timestamp": { "type": "date" },
"service": { "type": "keyword" },
"message": { "type": "text" }
}
}
}
number_of_shards is fixed at index creation. You cannot resize primaries on an existing index without reindexing (or using the Shrink/Split APIs, which have their own constraints). number_of_replicas is dynamic and can change at any time.
Shard Sizing Guidelines
| Workload | Target primary shard size | Notes |
|---|---|---|
| Time-series / logs | 10-50 GB | Smaller = faster recovery, more overhead |
| General search | 20-40 GB | Balanced |
| Frequently updated | 5-20 GB | Smaller for faster merges |
| Cold/frozen tier | 30-50+ GB | Read-rarely, optimize for storage |
Elastic's general rule: aim for shards between 10 GB and 50 GB, and target around 20 shards per GB of JVM heap on a node. Both are guidelines, not hard limits - measure on your workload.
The classic mistake is over-sharding small indices. A 1 GB index with 10 shards has 10x the cluster-state overhead of a 1 GB index with 1 shard, with no recovery or query benefit. Conversely, a single 200 GB shard means slow recovery and limited parallelism.
Index Aliases
An alias is a named pointer to one or more indices. Aliases let you swap indices behind a stable name, which is foundational for zero-downtime reindexing:
POST /_aliases
{
"actions": [
{ "add": { "index": "events-v2", "alias": "events" } },
{ "remove": { "index": "events-v1", "alias": "events" } }
]
}
Applications write to and query events; the actual backing index changes underneath. Aliases also support filters (read-only data slices) and routing.
For time-series data, data streams are the modern abstraction: they manage a backing series of hidden indices (.ds-events-2026.05.17-000001, ...) with rollover and lifecycle policies wired in. Data streams replace the older write-alias pattern for most log/metric workloads.
Index Lifecycle Management (ILM)
ILM automates how an index moves through tiers as it ages:
| Phase | Typical action |
|---|---|
| Hot | Active writes and queries, primary shards on fast disks |
| Warm | Read-only, force-merged, on slower nodes |
| Cold | Searchable but rarely queried, possibly searchable snapshots |
| Frozen | Searchable snapshots, very cheap storage |
| Delete | Removed |
A typical log ILM policy: rollover at 50 GB or 30 days, move to warm after 7 days, cold after 30 days, delete after 90. Configure once via ILM policy and Elasticsearch handles the transitions.
Common Index Mistakes
- Too many small indices. Each index has overhead in cluster state. A pattern of 10,000 daily indices each holding a few MB will destabilize the cluster long before disk runs out.
- Too many shards per node. The unofficial limit is around 1000 shards per node before performance degrades materially. Plan for it.
- Forgetting to set replicas to 0 on dev/single-node clusters - yellow status alarms for no reason.
- Reindexing without using an alias - clients have to be redeployed when the index name changes.
- Not using data streams for time-series workloads, then hand-rolling rollover logic with cron.
Operating Indices in Production
Watch:
- Shard count per node (
_cat/allocation) - rising shard count is a leading indicator of cluster instability. - Primary shard sizes (
_cat/shards) - shards >50 GB slow recovery and snapshots; shards <1 GB usually mean over-sharding. - Index health (
_cat/indices?h=index,health,status,docs.count) - red indices have unassigned primaries (data loss risk); yellow means replicas are missing. - Hot shards - one shard handling disproportionate traffic, often a routing or sharding-key issue.
Pulse tracks shard count, shard size distribution, index health, and routing imbalance across your Elasticsearch cluster. Its Shard Heatmap surfaces hot nodes and unbalanced shard placement at a glance, and its agentic root-cause analysis traces a slow query back to the specific shard or node where it stalled - without the manual hot-threads + indices-stats sweep.
Frequently Asked Questions
Q: How is an Elasticsearch index different from a database table?
A: Loosely, an index is like a table: it holds documents (rows) with a shared mapping (schema). But indices are distributed across shards, can be queried alongside other indices in one request, and support full-text features (analyzers, scoring, aggregations) that relational tables do not.
Q: How many shards should an Elasticsearch index have?
A: Size each primary shard between 10-50 GB depending on workload. Calculate by estimating total index size at peak retention divided by target shard size. For log workloads, a data stream with rollover at 50 GB usually gets you to a sensible number automatically.
Q: Can I change the number of shards in an Elasticsearch index?
A: Not directly on an existing index. To change the primary shard count, reindex into a new index with the new shard configuration, or use the Shrink API (reduces shards by factors of the original count) or Split API (increases shards). Replicas can be changed at any time without reindexing.
Q: What is the difference between an index and a shard?
A: An index is a logical container. A shard is a physical Lucene index that holds a subset of the index's documents. An Elasticsearch index with number_of_shards: 3 and number_of_replicas: 1 has 3 primary shards and 3 replica shards = 6 shards total distributed across the cluster.
Q: What happened to types in Elasticsearch indices?
A: Types were deprecated in Elasticsearch 6.x, removed from the API in 7.x, and completely removed in 8.x. One index = one document type. To group different document shapes, use separate indices.
Q: What is the maximum size of an Elasticsearch index?
A: An index can hold up to 2,147,483,519 documents per shard (Lucene's internal limit). Practically, you'll hit shard-size operational limits long before that - aim for primaries under 50 GB and rollover via ILM or data streams.
Related Reading
- What is Elasticsearch Mapping: the schema attached to each index
- What is Elasticsearch Node: where shards live
- What is Elasticsearch Refresh Interval: per-index visibility tuning
- Elasticsearch Dynamic Mapping: how new fields enter an index
- Elasticsearch Too Many Fields - Mapping Explosion: the field-count failure mode
- Elasticsearch Reindex: moving data between indices
