Logstash truncate Filter Plugin

The Logstash truncate filter caps the length of one or more string fields at a configurable number of characters. It is the standard guard against oversized fields that would otherwise hit Elasticsearch's 32 KB keyword field limit, exceed Kafka message size limits, or blow up event size for slow consumers. The filter operates only on strings; numeric and array fields are left untouched. Truncation counts characters, not bytes, so multibyte UTF-8 characters are not split mid-byte.

Syntax

filter {
  truncate {
    fields => [ "message", "stacktrace" ]
    length_bytes => 32000
  }
}

Note: as of Logstash 7.x the parameter is named length_bytes and despite the name it counts characters, not bytes. The omission/ellipsis behavior described in older docs is not part of the upstream plugin - to add a marker, follow truncate with a mutate filter.

Parameters

Examples

Truncate stack traces to 16 KB before sending to Elasticsearch, well under the keyword limit:

filter {
  truncate {
    fields => [ "stacktrace", "exception_message" ]
    length_bytes => 16384
  }
}

Truncate and tag, so downstream consumers know the message was clipped:

filter {
  if [message] {
    ruby {
      code => 'event.set("[orig_message_length]", event.get("message").length) if event.get("message")'
    }
  }
  truncate {
    fields => [ "message" ]
    length_bytes => 8000
    add_tag => [ "_message_truncated" ]
  }
}

Truncate per-field with different limits using multiple truncate filters:

filter {
  truncate { fields => [ "summary" ]  length_bytes => 500   }
  truncate { fields => [ "body" ]     length_bytes => 10000 }
  truncate { fields => [ "raw_html" ] length_bytes => 50000 }
}

Common Issues

The filter does not append an ellipsis or marker; truncated values look identical to legitimately short values. Add a _truncated tag or a side-channel field with the original length if downstream consumers need to detect clipping.

Truncate runs in character units, not bytes. A 1000-character truncate cap on a Japanese-language field can produce up to 3000 bytes of UTF-8 output, which may still exceed a byte-oriented downstream limit. If you have a hard byte budget (Kafka max.message.bytes, for example), apply a more conservative character limit.

Applying truncate to a nested field requires the bracket syntax: fields => [ "[event][message]" ]. The flat dot notation does not work.

The truncate filter does not pad short values - it only shortens. Use a mutate filter with gsub or a ruby filter to enforce a minimum length.

Performance Notes

Truncate is one of the cheapest filters in Logstash - it does a single string-length check per field and a substring operation only when truncation is needed. Throughput overhead is negligible (well under 1% of pipeline CPU in typical configurations). The much bigger throughput win is when truncate prevents oversized events from reaching the output: a 10 MB log line passed through a typical pipeline costs 50-100x more than a 10 KB line in serialization, network, and Elasticsearch indexing time. Place truncate early in the filter chain, immediately after parsing.

Monitoring Logstash Truncation with Pulse

Pulse is the only tool built specifically for monitoring and optimizing Logstash pipelines. Silent truncation is dangerous when downstream systems make decisions on stack traces, user input, or audit logs. Pulse tracks _message_truncated tag rates per pipeline, alerts on sudden increases (often a sign of upstream verbosity changes or a buggy application), and surfaces fields that frequently approach the truncate limit so you can rebalance limits before useful data is lost.

Frequently Asked Questions

Q: How does the Logstash truncate filter count length?
A: The filter counts characters, not bytes. A multibyte UTF-8 string truncated at 100 characters can still produce 400 bytes of output. If you need a byte budget, set a more conservative character limit or run a ruby filter to count bytes explicitly.

Q: Can the Logstash truncate filter add an ellipsis to truncated strings?
A: Not directly. The upstream plugin truncates without a marker. To add one, follow the truncate filter with a mutate gsub or a ruby filter that appends ... to fields that hit the cap.

Q: Does the Logstash truncate filter modify the original field?
A: Yes. Truncate updates the field in place. To preserve the original, copy it first with a mutate filter (add_field => { "message_original_length" => "%{[message].length}" }) or write a tag indicating truncation occurred.

Q: What is the difference between Logstash truncate and Elasticsearch ignore_above?
A: ignore_above (default 256 chars for keyword fields) drops the entire field from the inverted index when the value is too long - the value is still stored in _source but is not searchable. Truncate physically shortens the value in Logstash before it reaches Elasticsearch, so the shorter string is both stored and searchable. Use both: truncate to a sane limit upstream, set ignore_above as a safety net.

Q: Can the Logstash truncate filter truncate multiple fields to different lengths?
A: Not in a single filter block. Each truncate { ... } block applies one length_bytes to all fields it lists. To use different lengths, stack multiple truncate filter blocks.

Q: Why isn't my Logstash truncate filter working on a nested field?
A: Nested fields require bracket syntax: fields => [ "[event][message]" ]. Dot notation in the truncate filter does not traverse nested objects.

Related Reading

• Logstash Mutate Filter Plugin: rename, gsub, and add markers around truncate.
• Logstash Grok Filter Plugin: truncate fields after extraction.
• Logstash JSON Filter Plugin: parse, then truncate verbose embedded JSON.
• Logstash Pipeline is Blocked Error: oversized events are a common cause.
• Elasticsearch ignore_above Parameter: the destination-side safety net.