Tracing
The tracing system allows developers to visualize call flows in their infrastructure. Traefik Hub API Gateway uses OpenTelemetry, an open standard designed for distributed tracing.
Helm Chart Configuration Options
Traefik Hub API Gateway can be configured to provide traces in the OpenTelemetry format using the Helm Chart values.
To know more about the Helm Chart options, refer to the Helm Chart reference page (Options tracing.otlp).
Configuration Options
The section below describes how to configure Traefik Hub API Gateway to provide tracing using the static configuration.
| Field | Description | Default | Required |
|---|---|---|---|
tracing.addInternals | Enables tracing for internal resources (e.g.: ping@internals). | false | No |
tracing.serviceName | Service name used in selected backend. | "traefik" | No |
tracing.sampleRate | The proportion of requests to trace, specified between 0.0 and 1.0. | 1.0 | No |
tracing.globalAttributes | Applies a list of shared key:value attributes on all spans. | No | |
tracing.capturedRequestHeaders | Defines the list of request headers to add as attributes. It applies to client and server kind spans. | No | |
tracing.capturedResponseHeaders | Defines the list of response headers to add as attributes. It applies to client and server kind spans. | False | |
tracing.safeQueryParams | By default, all query parameters are redacted. Defines the list of query parameters to not redact. | No | |
tracing.additionalTraceHeaders.traceContext | Configuration object to duplicate Trace Context headers into dedicated headers. | No | |
tracing.additionalTraceHeaders.traceContext.traceParent | Name of the header to replicate the entire traceparent value. | "" | No |
tracing.additionalTraceHeaders.traceContext.traceState | Name of the header to replicate the tracestate value. | "" | No |
tracing.additionalTraceHeaders.traceContext.traceId | Extracts the trace-id from the traceparent header and duplicates it into a dedicated header. | "" | No |
tracing.additionalTraceHeaders.traceContext.parentId | Extracts the parent-id from the traceparent header and duplicates it into a dedicated header. | "" | No |
tracing.otlp.http | This instructs the exporter to send the tracing to the OpenTelemetry Collector using HTTP. Setting the sub-options with their default values. | null/false | No |
tracing.otlp.http.endpoint | URL of the OpenTelemetry Collector to send tracing to. Format=" <scheme>://<host>:<port><path>" | "http://localhost:4318/v1/tracing" | Yes |
tracing.otlp.http.headers | Additional headers sent with tracing by the exporter to the OpenTelemetry Collector. | No | |
tracing.otlp.http.tls.ca | Path to the certificate authority used for the secure connection to the OpenTelemetry Collector, it defaults to the system bundle. | "" | No |
tracing.otlp.http.tls.cert | Path to the public certificate used for the secure connection to the OpenTelemetry Collector. When using this option, setting the key option is required. | "" | No |
tracing.otlp.http.tls.key | This instructs the exporter to send the tracing to the OpenTelemetry Collector using HTTP. Setting the sub-options with their default values. | ""null/false "" | No |
tracing.otlp.http.tls.insecureskipverify | If insecureSkipVerify is true, the TLS connection to the OpenTelemetry Collector accepts any certificate presented by the server regardless of the hostnames it covers. | false | Yes |
tracing.otlp.grpc | This instructs the exporter to send tracing to the OpenTelemetry Collector using gRPC. | false | No |
tracing.otlp.grpc.endpoint | Address of the OpenTelemetry Collector to send tracing to. Format=" <host>:<port>" | "localhost:4317" | Yes |
tracing.otlp.grpc.headers | Additional headers sent with tracing by the exporter to the OpenTelemetry Collector. | No | |
tracing.otlp.grpc.insecure | Allows exporter to send tracing to the OpenTelemetry Collector without using a secured protocol. | false | Yes |
tracing.otlp.grpc.tls.ca | Path to the certificate authority used for the secure connection to the OpenTelemetry Collector, it defaults to the system bundle. | "" | No |
tracing.otlp.grpc.tls.cert | Path to the public certificate used for the secure connection to the OpenTelemetry Collector. When using this option, setting the key option is required. | "" | No |
tracing.otlp.grpc.tls.key | This instructs the exporter to send the tracing to the OpenTelemetry Collector using HTTP. Setting the sub-options with their default values. | ""null/false "" | No |
tracing.otlp.grpc.tls.insecureskipverify | If insecureSkipVerify is true, the TLS connection to the OpenTelemetry Collector accepts any certificate presented by the server regardless of the hostnames it covers. | false | Yes |
Additional Information
Default protocol
The OpenTelemetry trace exporter will export traces to the collector to https://localhost:4318/v1/traces using HTTP by default,
use the tracing.otlp.grpc option explicitly to enable GRPC instead.
Trace sampling
By default, the OpenTelemetry trace exporter will sample 100% of traces. See the OpenTelemetry's SDK configuration to customize the sampling strategy.
Propagation
Traefik Hub API Gateway supports the OTEL_PROPAGATORS env variable to set up the propagators. The supported propagators are:
- tracecontext (default)
- baggage (default)
- b3
- b3multi
- jaeger
- xray
- ottrace
OTEL_PROPAGATORS=b3,jaeger
Forward OTel trace-id using a Header
To facilitate the propagation of OpenTelemetry (OTel) trace identifiers through dedicated headers, Traefik Hub API Gateway offers the tracing.additionalTraceHeaders.traceContext configuration.
This feature allows the duplication of Trace Context headers into custom headers, enabling enhanced traceability across services.
Example
When migrating from a legacy tracing solution to OpenTelemetry, you may need to continue supplying a legacy tracing header.
By using the duplicate headers option, you can automatically replicate the traceparent and tracestate values, as well as extract the trace-id and parent-id into custom headers.
For example, to replicate the trace-id in a header called E2ETrackingID, configure:
- Static Configuration
hub:
tracing:
additionalTraceHeaders:
traceContext:
traceId: E2ETrackingID
# ...
If the request arrives with an E2ETrackingID header, it will be overwritten by the trace-id extracted from the OpenTelemetry traceparent.
This helps maintain backward compatibility with existing applications that rely on a legacy tracing header, while still adopting the standard W3C trace context format for OTel-based tracing.