Logstash JSON Filter Plugin

The Logstash JSON filter plugin takes a JSON-encoded string from a named source field and expands it into structured event fields. When no target is set, parsed keys are merged into the event root, which can overwrite reserved fields like @timestamp. Set target to namespace the result under a sub-field. The plugin is the standard way to handle application logs that emit a JSON message inside a larger envelope.

Syntax

filter {
  json {
    source                => "message"
    target                => "parsed"
    skip_on_invalid_json  => true
    tag_on_failure        => [ "_jsonparsefailure" ]
  }
}

The source field must contain a valid JSON string (object or array). If the source field is missing, the filter is a no-op. If parsing fails, the event is tagged with _jsonparsefailure unless skip_on_invalid_json is true.

Parameters

Examples

Parse a JSON envelope from a file input and route the inner fields to a sub-namespace:

filter {
  json {
    source => "message"
    target => "app"
  }
}

After parsing, event.app.user_id and event.app.level are available without polluting the root.

Parse a JSON payload pulled from an HTTP webhook input and merge into the root, but tag and drop malformed events:

filter {
  json {
    source => "payload"
    skip_on_invalid_json => false
    tag_on_failure => [ "_jsonparsefailure", "webhook_bad_payload" ]
  }
  if "_jsonparsefailure" in [tags] {
    drop { }
  }
}

Parse a nested JSON-in-JSON message - common when an application logs message: "{\"inner\":{...}}":

filter {
  json { source => "message" target => "outer" }
  json { source => "[outer][inner_json]" target => "inner" }
}

Common Issues

Merging into the root with no target is the most frequent source of broken pipelines. If the JSON payload contains @timestamp, tags, or host, those keys overwrite the event's existing values silently. Always set target unless you have audited the payload's top-level keys.

Unicode escapes are decoded transparently, but unescaped control characters (raw newlines inside string values) are invalid JSON and produce parse failures. Logs that pretty-print JSON across multiple lines need to be re-joined with a multiline codec on the input before reaching the json filter.

Numeric values larger than Long.MAX_VALUE (2^63 - 1) lose precision when parsed into the JVM. If you need exact representation of large integer IDs (Twitter-style snowflake IDs, for example), keep them as strings in the upstream system.

Performance Notes

The JSON filter uses Jackson under the hood and is fast - typically 5-10x faster than equivalent grok parsing for the same payload. The cost grows linearly with document size, so events larger than a few KB benefit from filtering or trimming before parsing. When you only need a handful of fields from a large payload, consider using a dissect or grok pattern to extract just those fields rather than parsing the entire object.

Monitoring Logstash JSON Parsing with Pulse

Pulse is the only tool built specifically for monitoring and optimizing Logstash pipelines. When upstream services change their log schema, the json filter starts emitting _jsonparsefailure tags and downstream consumers quietly lose fields. Pulse continuously tracks per-pipeline parse failure rates, surfaces schema drift the moment it starts, and traces the failure back to the input plugin and source so you can fix the broken upstream before it corrupts a day of analytics.

Frequently Asked Questions

Q: What does the Logstash JSON filter do when no target field is specified?
A: The json filter merges all parsed keys directly into the event root. This is convenient but risky: a payload key named @timestamp, host, or tags will overwrite the event's existing field. Setting target to a sub-field is the safer default.

Q: How does the Logstash JSON filter handle invalid JSON?
A: By default the json filter tags the event with _jsonparsefailure and leaves the original field untouched. Set skip_on_invalid_json => true to silently skip without tagging, which is useful when the source field sometimes contains plain text and sometimes JSON.

Q: Can the Logstash JSON filter parse nested JSON inside a JSON field?
A: Yes, but you have to run the filter twice. First parse the outer JSON with source => "message", then run a second json filter with source pointing at the nested string field, for example source => "[outer][inner_json]".

Q: Does the Logstash JSON filter handle JSON arrays?
A: Yes, top-level JSON arrays are parsed into a Ruby array. To produce one Logstash event per array element, follow the json filter with a split filter on the target field.

Q: How is the Logstash JSON filter different from the JSON codec?
A: The JSON codec runs at the input or output stage and applies to the entire stream. The JSON filter runs per-event on a specific field and is the right choice when only part of the message is JSON (typical for syslog-wrapped application logs).

Q: What is the performance overhead of the Logstash JSON filter?
A: The json filter uses Jackson and is usually 5-10x faster than the equivalent grok parsing. Cost scales linearly with payload size, so the bottleneck for most pipelines is the size of the JSON, not the filter itself.

Related Reading

• Logstash Grok Filter Plugin: parse non-JSON text before the json filter, or extract a JSON substring.
• Logstash Split Filter Plugin: one event per array element after JSON parsing.
• Logstash KV Filter Plugin: alternative for key=value payloads.
• Logstash Date Filter Plugin: convert a timestamp string in the parsed JSON into @timestamp.
• Logstash Cannot Decode Event: related parse failure errors.