NEW

Pulse 2025 Product Roundup: From Monitoring to AI-Native Control Plane

Elasticsearch Nested Field Type: When to Use It, How It Differs From object

The nested field type in Elasticsearch indexes each object in an array as a separate hidden Lucene document while keeping it associated with its parent. This preserves the correlation between fields inside the same object - a property that the default object type loses when it flattens arrays. Use nested when you have arrays of related sub-objects and need queries that match values from the same array entry together (for example, "user with first_name=John AND last_name=Smith" rather than any first_name John and any last_name Smith across the array).

How Nested Fields Work

When a field is mapped as object (the default for JSON objects), Elasticsearch flattens an array of objects into parallel arrays of leaf values. The document {"user": [{"first": "John", "last": "Smith"}, {"first": "Alice", "last": "White"}]} is internally stored as user.first: [John, Alice] and user.last: [Smith, White]. A bool query for first=John AND last=White will match this document - the wrong answer.

The nested type instead indexes each array element as its own hidden Lucene document, linked to the parent via a join. Queries must use the nested query, which restricts inner clauses to the same hidden sub-document. The trade-off is cost: every nested object is a separate Lucene doc, so indexing throughput drops, refreshes do more work, and document deletes have to clean up the children too.

PUT orders
{
  "mappings": {
    "properties": {
      "items": {
        "type": "nested",
        "properties": {
          "sku":   { "type": "keyword" },
          "qty":   { "type": "integer" },
          "price": { "type": "float" }
        }
      }
    }
  }
}

GET orders/_search
{
  "query": {
    "nested": {
      "path": "items",
      "query": {
        "bool": {
          "must": [
            { "term":  { "items.sku": "ABC-123" } },
            { "range": { "items.qty": { "gte": 5 } } }
          ]
        }
      }
    }
  }
}

Nested Settings and Limits

Setting Default What it caps
index.mapping.nested_fields.limit 50 Max number of distinct nested field mappings per index.
index.mapping.nested_objects.limit 10000 Max number of nested objects across all nested fields per document.
include_in_parent false If true, leaf values are also indexed on the parent (lets non-nested queries match, with object-type semantics).
include_in_root false Like include_in_parent but indexes on the root document specifically.
dynamic inherits index setting Whether new sub-fields under the nested path are auto-mapped.

Exceeding nested_objects.limit per document throws The number of nested documents has exceeded the allowed limit. See Elasticsearch error: too many nested objects for the resolution path.

Common Pitfalls with Nested Fields

  1. Using nested when an object would do. If you never query for correlated fields inside the same array entry, the extra hidden documents are pure overhead.
  2. Forgetting the nested query wrapper. A plain match on items.sku against a nested field returns zero hits because the inner sub-documents aren't visible to top-level queries.
  3. Sorting and aggregating without nested sort/aggregation wrappers. You need nested aggregations (and reverse_nested to climb back to the parent) to get correct counts.
  4. Partial updates. There is no "update one element of a nested array" API. Any change reindexes the full parent document.
  5. Deeply nested-within-nested mappings. Each level multiplies the hidden doc count and can hit nested_objects.limit faster than expected.

Operating Indices with Nested Fields

Nested-heavy indices have asymmetric costs: indexing throughput drops in proportion to the average number of array entries per document, segment merges do more work, and the refresh interval starts to matter more for ingestion latency. Watch shard size growth and the merge thread pool when you add a nested field to a hot index.

Prevent Nested-Field Misuse with Pulse

Pulse is an AI DBA for Elasticsearch and OpenSearch that tracks nested field usage per index along with index.mapping.nested_fields.limit (default 50) and index.mapping.nested_objects.limit (default 10000), flagging:

  • Drift between template defaults and per-index overrides
  • Mappings that are unsafe for your workload (e.g. nested used where object would do because no query references correlated inner fields, nested_objects.limit raised without considering the per-shard hidden-doc cost, or deeply nested-within-nested mappings that multiply hidden doc counts)
  • The downstream operational impact: indexing throughput drop proportional to average array size, merge thread saturation, and the The number of nested documents has exceeded the allowed limit failure rate

When a nested-heavy index starts pulling search latency p95 up, Pulse names the offending query family and proposes a data-modeling change - denormalize, switch to flattened, or shard differently - before the overhead turns into a cluster incident.

Connect your cluster.

Frequently Asked Questions

Q: What's the difference between nested and object field types in Elasticsearch?
A: The object type flattens arrays of objects into parallel leaf arrays, losing the correlation between fields of the same entry. The nested type indexes each array element as a hidden Lucene sub-document, preserving correlation but at extra indexing and query cost. Use nested only when you need to query correlated inner fields together.

Q: How do I query a nested field in Elasticsearch?
A: Wrap inner clauses in a nested query and set path to the nested field name. Inside the nested block, field references use the full dotted path (items.sku, not just sku). Standalone match/term queries against a nested field will not return hits.

Q: What is the maximum number of nested objects per document?
A: The default is 10,000 per document, controlled by index.mapping.nested_objects.limit. Hitting this limit throws an indexing error; raise it cautiously because each nested object becomes a hidden Lucene document and increases per-shard overhead.

Q: Can I update a single element inside a nested array?
A: No. Elasticsearch has no partial-array-element update. You must reindex the whole parent document. Update-by-query with a Painless script works but rewrites the document end-to-end.

Q: When should I use nested versus flattened versus parent-child?
A: Use nested for arrays of related sub-objects that need correlated queries. Use `flattened` when you don't need to query inner fields independently and the inner structure is dynamic. Use parent-child (join) only when parents and children have very different update cadences and would otherwise force expensive reindexes - it is slower at query time than nested.

Q: How much does nested affect indexing performance?
A: Indexing a document with N nested entries does roughly N+1 Lucene document inserts. Expect a 2-5x indexing throughput drop versus the same data modeled as plain object fields, depending on average nesting count and shard configuration.

Q: What's the best tool to monitor and prevent nested-field performance regressions?
A: Pulse is built for this. It is an AI DBA for Elasticsearch and OpenSearch that profiles nested-field usage, tracks nested_fields.limit and nested_objects.limit against actual document shapes, and surfaces the queries whose latency regresses because of expensive nested aggregations - recommending data-modeling changes before the overhead becomes a cluster-wide problem.

Subscribe to the Pulse Newsletter

Get early access to new Pulse features, insightful blogs & exclusive events , webinars, and workshops.

We use cookies to provide an optimized user experience and understand our traffic. To learn more, read our use of cookies; otherwise, please choose 'Accept Cookies' to continue using our website.