Buf Docs

Appearance

Observability – Overview ​

Bufstream provides comprehensive observability through metrics, logging, and distributed tracing, giving you full visibility into cluster health, throughput, and error rates.

  • Metrics: Track broker performance, topic throughput, consumer lag, and more. Bufstream exports metrics via OpenTelemetry, with support for OTLP and Prometheus exporters. For a full list of available metrics, see the metrics reference.
  • Logging: JSON-formatted logs to standard error with configurable log levels.
  • Tracing: Distributed tracing via OpenTelemetry for debugging request flows across your cluster.

For platform-specific setup guides, see the Datadog and Grafana integration pages.

The sections below cover how to configure each of these capabilities.

Configuring metrics ​

OTLP ​

To configure Bufstream to report to an OpenTelemetry collector, configure Bufstream as follows:

yaml
metrics:
  otlp:
    # Required: Type of transport to use for OTLP. Must be "http" or "grpc".
    type: http
    # Required: URL of OTLP endpoint to export metrics to.
    url: <string>

Prometheus ​

Prometheus metrics are exposed at the debug address. To enable the Prometheus endpoint for collecting metrics, configure Bufstream as follows:

yaml
debug:
  # Address to listen for connections for debug information. If configured,
  # pprof and Prometheus exported metrics will be exposed on this address.
  listen_address: <hostport>

Logging ​

Logging is output to standard error in the Bufstream brokers in JSON format. Use the following to configure the log level:

yaml
logging:
  # Log level, defaults to info.
  level: debug|info|warn|error

Tracing ​

Use the following to configure tracing:

yaml
traces:
  # Configuration for exporting OpenTelemetry-based traces.
  otlp:
    # Required: Type of transport to use for OTLP. Must be "http" or "grpc".
    type: http
    # Required: URL of OTLP endpoint to export traces to.
    url: <string>
    # OpenTelemetry trace sample ratio, defaults to 0.1.
    trace_ratio: 0.1