Logstash translate Filter Plugin

The translate filter plugin in Logstash is used for data enrichment by performing key-value lookups. It allows you to add additional information to your events based on the content of a specific field. This plugin is particularly useful for tasks such as converting error codes to human-readable messages, mapping IP addresses to geographic locations, or translating abbreviations to full terms.

Syntax

The basic syntax for the translate filter is:

translate {
  field => "[field_name]"
  destination => "[new_field]"
  dictionary => {
    "key1" => "value1"
    "key2" => "value2"
  }
}

For more detailed information, refer to the official Logstash translate filter documentation.

Example Use Case

Suppose you have log entries with country codes, and you want to add the full country names to your events. Here's an example configuration:

filter {
  translate {
    field => "[country_code]"
    destination => "[country_name]"
    dictionary => {
      "US" => "United States"
      "UK" => "United Kingdom"
      "FR" => "France"
      "DE" => "Germany"
    }
    fallback => "Unknown"
  }
}

In this example, if an event has a country_code field with the value "US", a new field country_name will be added with the value "United States".

Common Issues and Best Practices

1. Large dictionaries can impact performance. Consider using an external file for large datasets.
2. The plugin is case-sensitive by default. Use the ignore_case => true option if needed.
3. Use the fallback option to handle cases where the key is not found in the dictionary.
4. For dynamic updates, use a dictionary file and enable refresh_interval for periodic reloading.

Frequently Asked Questions

Q: Can I use an external file for the dictionary instead of defining it inline?
A: Yes, you can use an external file by specifying the dictionary_path option instead of dictionary. This is useful for large dictionaries or when you need to update the dictionary without changing the Logstash configuration.

Q: How can I make the translate filter case-insensitive?
A: You can add the ignore_case => true option to your translate filter configuration to make it case-insensitive.

Q: What happens if a key is not found in the dictionary?
A: By default, if a key is not found, no action is taken. You can use the fallback option to specify a default value to be used when a key is not found in the dictionary.

Q: Can the translate filter handle multiple fields at once?
A: No, the translate filter processes one field at a time. If you need to translate multiple fields, you'll need to use multiple translate filter instances.

Q: Is it possible to use regular expressions in the dictionary keys?
A: Yes, you can use regular expressions in the dictionary keys by setting regex => true in the filter configuration. This allows for more flexible matching of field values.