Buf Docs

Appearance

Observability – Overview ​

Bufstream is instrumented for monitoring cluster health, performance, and errors. Metrics are instrumented using OpenTelemetry, with support for standard exporters like OTLP and Prometheus. Specific configuration recommendations including dashboards and monitors/alerts for Datadog and Grafana are available.

Configuring metrics ​

OTLP ​

To configure Bufstream to report to an OpenTelemetry collector, configure Bufstream as follows:

yaml
observability:
  exporter:
    address: "<collector-hostname>:4318"
    # Set to true to report over HTTP, false to report over TLS.
    insecure: true
  metrics:
    exporter_type: "OTLP_HTTP"

Prometheus ​

To enable the Prometheus endpoint for collecting metrics, configure Bufstream as follows:

yaml
observability:
  metrics:
    exporter_type: "PROMETHEUS"

The Prometheus endpoint (/metrics) will be available on the Bufstream broker on port 9090.

Logging ​

Logging is output to standard error in the Bufstream brokers in JSON format. Use the following to configure the log level:

yaml
observability:
  # One of "DEBUG", "INFO", "WARN", or "ERROR".
  log_level: "INFO"

Tracing ​

Use the following to configure tracing:

yaml
observability:
  exporter:
    address: "<collector-hostname>:4318"
    # Set to true to report over HTTP, false to report over TLS.
    insecure: true
  traces:
    exporter_type: "OTLP_HTTP"
    # Optional, trace sampling ratio, defaults to 0.1
    # trace_ratio: 0.1

Redacting sensitive information ​

Because consumer group IDs and topic names are user-provided and may contain information that users don't wish to report to monitoring systems, Bufstream can redact that information before reporting via metrics, logs, or traces. When redaction is set to OPAQUE, topic names are replaced with topic IDs (UUIDs) and consumer group ids are replaced with hashed values. To configure sensitive information redaction, configure Bufstream as follows:

yaml
observability:
  # -- Redact sensitive information, such as topic names, before adding to to metrics, traces, and logs.
  # Supports [NONE, OPAQUE].
  # Default: NONE.
  sensitive_information_redaction: "OPAQUE"