Logstash metricize Filter Plugin

The metricize filter plugin in Logstash is used to convert complex events with multiple values into multiple metrics-friendly events. This is particularly useful when you want to transform data into a format suitable for time series databases or monitoring systems.

Syntax

filter {
  metricize {
    metric_name => "metric_name"
    value_field => "field_name"
    metrics => [ "field1", "field2", ... ]
  }
}

For detailed configuration options, refer to the official Logstash metricize filter plugin documentation.

Example Use Case

Suppose you have an event with multiple metrics:

{
  "cpu_usage": 80,
  "memory_usage": 60,
  "disk_usage": 50
}

You can use the metricize filter to split this into separate events:

filter {
  metricize {
    metric_name => "system_metric"
    value_field => "value"
    metrics => [ "cpu_usage", "memory_usage", "disk_usage" ]
  }
}

This will produce three separate events:

{ "system_metric": "cpu_usage", "value": 80 }
{ "system_metric": "memory_usage", "value": 60 }
{ "system_metric": "disk_usage", "value": 50 }

Common Issues and Best Practices

1. Ensure that the fields specified in the metrics array exist in your events.
2. Be cautious when using this filter on high-volume data streams, as it can significantly increase the number of events.
3. Consider using this filter in combination with the aggregate filter if you need to perform calculations on the metrics before sending them to the output.

Frequently Asked Questions

Q: Can I use the metricize filter with nested fields?
A: Yes, you can use dot notation to access nested fields. For example, metrics => [ "system.cpu.usage", "system.memory.usage" ].

Q: How does the metricize filter handle non-numeric values?
A: The metricize filter is designed to work with numeric values. Non-numeric values will be converted to 0 or ignored, depending on the configuration.

Q: Can I customize the names of the output fields?
A: Yes, you can use the metric_name_field and value_field options to specify custom field names for the metric name and value in the output events.

Q: Is it possible to add additional fields to the output events?
A: Yes, you can use the add_field option to include additional fields in each output event.

Q: How does the metricize filter affect event timestamps?
A: The metricize filter preserves the original event's timestamp for all generated events. If you need different timestamps, you may need to use additional filters or modify your pipeline configuration.