Jaeger - a Distributed Tracing System

💥💥💥 Jaeger v2 is coming! Read the blog post and try it out.

mermaid graph TD SDK["OpenTelemetry SDK"] --> |HTTP or gRPC| COLLECTOR COLLECTOR["Jaeger Collector"] --> STORE[Storage] COLLECTOR --> |gRPC| PLUGIN[Storage Plugin] COLLECTOR --> |gRPC/sampling| SDK PLUGIN --> STORE QUERY[Jaeger Query Service] --> STORE QUERY --> |gRPC| PLUGIN UI[Jaeger UI] --> |HTTP| QUERY subgraph Application Host subgraph User Application SDK end end

Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing platform created by Uber Technologies and donated to Cloud Native Computing Foundation. It can be used for monitoring microservices-based distributed systems:

• Distributed context propagation
• Distributed transaction monitoring
• Root cause analysis
• Service dependency analysis
• Performance / latency optimization

See also:

• Jaeger documentation for getting started, operational details, and other information.
• Blog post Evolving Distributed Tracing at Uber.
• Tutorial / walkthrough Take Jaeger for a HotROD ride.

Jaeger is hosted by the Cloud Native Computing Foundation (CNCF) as the 7th top-level project (graduated in October 2019). If you are a company that wants to help shape the evolution of technologies that are container-packaged, dynamically-scheduled and microservices-oriented, consider joining the CNCF. For details about who's involved and how Jaeger plays a role, read the CNCF Jaeger incubation announcement and Jaeger graduation announcement.

Get Involved

Jaeger is an open source project with open governance. We welcome contributions from the community, and we would love your help to improve and extend the project. Here are some ideas for how to get involved. Many of them do not even require any coding.

Features

High Scalability

Jaeger backend is designed to have no single points of failure and to scale with the business needs. For example, any given Jaeger installation at Uber is typically processing several billions of spans per day.

Relationship with OpenTelemetry

The Jaeger and OpenTelemetry projects have different goals. OpenTelemetry aims to provide APIs and SDKs in multiple languages to allow applications to export various telemetry data out of the process, to any number of metrics and tracing backends. The Jaeger project is primarily the tracing backend that receives tracing telemetry data and provides processing, aggregation, data mining, and visualizations of that data. For more information please refer to a blog post Jaeger and OpenTelemetry.

Jaeger was originally designed to support the OpenTracing standard. The terminology is still used in Jaeger UI, but the concepts have direct mapping to the OpenTelemetry data model of traces.

| Capability | OpenTracing concept | OpenTelemetry concept | | ------------- | ------------------- | --------------------- | | Represent traces as directed acyclic graphs (not just trees) | span references | span links | | Strongly typed span attributes | span tags | span attributes | | Strongly typed events/logs | span logs | span events |

Jaeger project recommends OpenTelemetry SDKs for instrumentation, instead of now-deprecated Jaeger SDKs.

Multiple storage backends

Jaeger can be used with a growing a number of storage backends: * It natively supports two popular open source NoSQL databases as trace storage backends: Cassandra and Elasticsearch. * It integrates via a gRPC API with other well known databases that have been certified to be Jaeger compliant: TimescaleDB via Promscale, ClickHouse. * There is embedded database support using Badger and simple in-memory storage for testing setups. * ScyllaDB can be used as a drop-in replacement for Cassandra since it uses the same data model and query language. * There are ongoing community experiments using other databases, such as InfluxDB, Amazon DynamoDB, YugabyteDB(YCQL).

Modern Web UI

Jaeger Web UI is implemented in Javascript using popular open source frameworks like React. Several performance improvements have been released in v1.0 to allow the UI to efficiently deal with large volumes of data and to display traces with tens of thousands of spans (e.g. we tried a trace with 80,000 spans).

Cloud Native Deployment

Jaeger backend is distributed as a collection of Docker images. The binaries support various configuration methods, including command line options, environment variables, and configuration files in multiple formats (yaml, toml, etc.).

The recommended way to deploy Jaeger in a production Kubernetes cluster is via the Jaeger Operator.

The Jaeger Operator provides a CLI to generate Kubernetes manifests from the Jaeger CR. This can be considered as an alternative source over plain Kubernetes manifest files.

The Jaeger ecosystem also provides a Helm chart as an alternative way to deploy Jaeger.

Observability

All Jaeger backend components expose Prometheus metrics by default (other metrics backends are also supported). Logs are written to standard out using the structured logging library zap.

Security

Third-party security audits of Jaeger are available in https://github.com/jaegertracing/security-audits. Please see Issue #1718 for the summary of available security mechanisms in Jaeger.

Backwards compatibility with Zipkin

Although we recommend instrumenting applications with OpenTelemetry, if your organization has already invested in the instrumentation using Zipkin libraries, you do not have to rewrite all that code. Jaeger provides backwards compatibility with Zipkin by accepting spans in Zipkin formats (Thrift or JSON v1/v2) over HTTP. Switching from Zipkin backend is just a matter of routing the traffic from Zipkin libraries to the Jaeger backend.

Version Compatibility Guarantees

Occasionally, CLI flags can be deprecated due to, for example, usability improvements or new functionality. In such situations, developers introducing the deprecation are required to follow these guidelines.

In short, for a deprecated CLI flag, you should expect to see the following message in the --help documentation: (deprecated, will be removed after yyyy-mm-dd or in release vX.Y.Z, whichever is later)

A grace period of at least 3 months or two minor version bumps (whichever is later) from the first release containing the deprecation notice will be provided before the deprecated CLI flag can be deleted.

For example, consider a scenario where v1.28.0 is released on 01-Jun-2021 containing a deprecation notice for a CLI flag. This flag will remain in a deprecated state until the later of 01-Sep-2021 or v1.30.0 where it can be removed on or after either of those events. It may remain deprecated for longer than the aforementioned grace period.

Go Version Compatibility Guarantees

The Jaeger project attempts to track the currently supported versions of Go, as defined by the Go team. Removing support for an unsupported Go version is not considered a breaking change.

Starting with the release of Go 1.21, support for Go versions will be updated as follows:

1. Soon after the release of a new Go minor version N, updates will be made to the build and tests steps to accommodate the latest Go minor version.
2. Soon after the release of a new Go minor version N, support for Go version N-2 will be removed and version N-1 will become the minimum required version.

Related Repositories

Documentation

• Published: https://www.jaegertracing.io/docs/
• Source: https://github.com/jaegertracing/documentation

Instrumentation Libraries

Jaeger project recommends OpenTelemetry SDKs for instrumentation, instead of Jaeger's native SDKs that are now deprecated.

Deployment

• Jaeger Operator for Kubernetes

Components

• UI
• Data model

Building From Source

See CONTRIBUTING.

Contributing

See CONTRIBUTING.

Thanks to all the people who already contributed!

Maintainers

Rules for becoming a maintainer are defined in the GOVERNANCE document. Below are the official maintainers of the Jaeger project. Please use @jaegertracing/jaeger-maintainers to tag them on issues / PRs.

• @albertteoh
• @jkowall
• @joe-elliott
• @pavolloffay
• @yurishkuro

Some repositories under jaegertracing org have additional maintainers.

Emeritus Maintainers

We are grateful to our former maintainers for their contributions to the Jaeger project.

• @black-adder
• @jpkrohling
• @objectiser
• @tiffon
• @vprithvi

Project Status Meetings

The Jaeger maintainers and contributors meet regularly on a video call. Everyone is welcome to join, including end users. For meeting details, see https://www.jaegertracing.io/get-in-touch/.

Roadmap

See https://www.jaegertracing.io/docs/roadmap/

Get in Touch

Have questions, suggestions, bug reports? Reach the project community via these channels:

• Slack chat room #jaeger (need to join CNCF Slack for the first time)
• jaeger-tracing mail group
• GitHub issues and discussions

Adopters

Jaeger as a product consists of multiple components. We want to support different types of users, whether they are only using our instrumentation libraries or full end to end Jaeger installation, whether it runs in production or you use it to troubleshoot issues in development.

Please see ADOPTERS.md for some of the organizations using Jaeger today. If you would like to add your organization to the list, please comment on our survey issue.

License

Copyright (c) The Jaeger Authors. Apache 2.0 License.