Skip to main content

Integration

Pulse for Kafka​

If you haven't done so yet, follow the instructions under the Pulse Agent integration section for installation and basic configuration instructions.

The clusters section needs to contain the configurations for the Kafka clusters to monitor. The configurations are the same as for any Kafka Consumer configuration, and all connectivity options are supported. No data is being read, only cluster metadata and metrics. For Kafka hosted on AWS MSK or Confluent Cloud, see adjustments below.

clusters:
  - type: kafka
    bootstrap.servers: PLAINTEXT://example-broker-1:9092,PLAINTEXT://example-broker-2:9092
    # the recommendation is to use the JMX port for monitoring, but the Prometheus port can be used instead
    jmx.port: 5555
    #prometheus.port: 9999
    # all Kafka client configurations are supported:
    request.timeout.ms: 5000
    ssl: false

  • bootstrap-servers

    Comma-separated list of Kafka brokers hosts.

  • jmx-port

    The port, on which kafka nodes expose the JMX metrics. It is usually configured on your Kafka nodes via JMX_PORT env variable, but it might vary depending on your Kafka deployment.

  • prometheus-port

    If the direct access to JMX is not available, as it is in case of using Amazon MSK, you can instead specify the port on which is running the Prometheus JMX exposter.

    Note that Kafkana expects the metrics in the format that Amazon MSK is publishing them via it's Open Monitoring option. So not every JMX exporter configuration will work.