Data

Tools

Indexes

Applications

Experiments

Open Source Science

oteldb

OpenTelemetry signal storage

https://github.com/go-faster/oteldb

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.7%) to scientific vocabulary

Keywords

clickhouse clickhouse-server grafana grafana-datasource logging logql logs loki monitoring observability open-telemetry opentelemetry otel prometheus promql tempo traceql tracing

Keywords from Contributors

mesh interpretability sequences projection interactive hacking network-simulation
Last synced: 6 months ago · JSON representation ·

Repository

OpenTelemetry signal storage

Basic Info
  • Host: GitHub
  • Owner: go-faster
  • License: apache-2.0
  • Language: Go
  • Default Branch: main
  • Homepage:
  • Size: 7.64 MB
Statistics
  • Stars: 57
  • Watchers: 3
  • Forks: 4
  • Open Issues: 15
  • Releases: 39
Topics
clickhouse clickhouse-server grafana grafana-datasource logging logql logs loki monitoring observability open-telemetry opentelemetry otel prometheus promql tempo traceql tracing
Created almost 3 years ago · Last pushed 6 months ago
Metadata Files
Readme Contributing License Code of conduct Citation Security Support Roadmap Maintainers

README.md

oteldb svg logo

oteldb codecov experimental

The next generation, OpenTelemetry-first aggregation system for metrics, traces and logs.

Compatible with PromQL, TraceQL and LogQL.

Based on ClickHouse, fastest open-source (Apache 2.0) column-oriented database.

[!WARNING] Work in progress. Not ready for production use.

Supported query languages: - PromQL (Prometheus) for metrics, >99% compatibility - TraceQL (Grafana Tempo) for traces - LogQL (Grafana Loki) for logs

Supported ingestion protocols: - Prometheus remote write, including exemplars - OpenTelemetry protocol (gRPC) for metrics, traces and logs

Ingestion is possible with OpenTelemetry collector, supporting over 90 protocols.

Demo

https://github.com/user-attachments/assets/647d220c-052a-40d4-9358-7d6039a0f198

Prometheus Compatibility

See ch-compliance for Prometheus compatibility tests.

console $ promql-compliance-tester -config-file promql-test-queries.yml -config-file test.oteldb.yml Total: 547 / 548 (99.82%) passed, 0 unsupported

Quick Start

Setup oteldb, ClickHouse, Grafana, and telemetry generators:

shell docker compose -f dev/local/ch/docker-compose.yml up -d

You can open Grafana dashboard at http://localhost:3000/d/oteldb/oteldb

License

Apache License 2.0, see LICENSE.

Owner

  • Name: go faster
  • Login: go-faster
  • Kind: organization

causing performance

Citation (CITATION.cff) 

---
cff-version: 1.2.0
version: 0.6.1
title: "oteldb: OpenTelemetry-first signal aggregation system"
license: Apache 2.0
abstract: >
   The next generation, OpenTelemetry-first aggregation system for metrics, traces and logs.
   Compatible with PromQL, TraceQL and LogQL.
   Based on ClickHouse, fastest open-source (Apache 2.0) column-oriented database.
authors:
  - family-names: Aleksandr
    given-names: Razumov
    affiliation: go-faster.org
    orcid: "https://orcid.org/0009-0009-8907-3954"
date-released: "2024-01-08"
keywords:
  - observability
  - opentelemetry
  - golang
repository-code: "https://github.com/go-faster/oteldb"
type: software
message: "Available at https://github.com/go-faster/oteldb"

GitHub Events

Total
  • Create event: 196
  • Issues event: 5
  • Release event: 11
  • Watch event: 21
  • Delete event: 178
  • Issue comment event: 169
  • Push event: 362
  • Pull request review event: 2
  • Pull request event: 377
Last Year
  • Create event: 196
  • Issues event: 5
  • Release event: 11
  • Watch event: 21
  • Delete event: 178
  • Issue comment event: 169
  • Push event: 362
  • Pull request review event: 2
  • Pull request event: 377

Committers

Last synced: over 1 year ago

All Time
  • Total Commits: 2,207
  • Total Committers: 4
  • Avg Commits per committer: 551.75
  • Development Distribution Score (DDS): 0.488
Past Year
  • Commits: 1,467
  • Committers: 4
  • Avg Commits per committer: 366.75
  • Development Distribution Score (DDS): 0.543
Top Committers
Name Email Commits
tdakkota t****3@y****u 1,129
Aleksandr Razumov e****o@g****g 709
dependabot[bot] 4****] 365
Raven 3****n 4
Committer Domains (Top 20 + Academic)
go-faster.org: 1 yandex.ru: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 29
  • Total pull requests: 528
  • Average time to close issues: about 1 month
  • Average time to close pull requests: about 22 hours
  • Total issue authors: 6
  • Total pull request authors: 3
  • Average comments per issue: 0.66
  • Average comments per pull request: 0.46
  • Merged pull requests: 431
  • Bot issues: 4
  • Bot pull requests: 467
Past Year
  • Issues: 5
  • Pull requests: 270
  • Average time to close issues: 1 day
  • Average time to close pull requests: 1 day
  • Issue authors: 4
  • Pull request authors: 3
  • Average comments per issue: 0.2
  • Average comments per pull request: 0.52
  • Merged pull requests: 216
  • Bot issues: 2
  • Bot pull requests: 257
View more stats
Top Authors
Issue Authors
  • ernado (20)
  • dependabot[bot] (4)
  • tdakkota (2)
  • lmangani (1)
  • jordanfung (1)
  • orloffv (1)
Pull Request Authors
  • dependabot[bot] (467)
  • tdakkota (51)
  • ernado (10)
Top Labels
Issue Labels
enhancement (8) perf (5) openapi (4) dependencies (4) go (4) traces (3) bug (2) logs (1) fuzzing (1) test (1)
Pull Request Labels
dependencies (467) go (451) github_actions (16)

Packages

  • Total packages: 1
  • Total downloads: unknown
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 43
proxy.golang.org: github.com/go-faster/oteldb

Package oteldb is OpenTelemetry-compatible storage.

  • Versions: 43
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 9.6%
Stargazers count: 9.9%
Dependent repos count: 10.8%
Average: 12.3%
Forks count: 18.7%
Last synced: 6 months ago

Dependencies

.github/workflows/codeql.yml actions
  • actions/cache v3 composite
  • actions/checkout v4 composite
  • actions/setup-go v4 composite
  • github/codeql-action/analyze v2 composite
  • github/codeql-action/init v2 composite
.github/workflows/dep-review.yml actions
  • actions/checkout v4 composite
  • actions/dependency-review-action v3 composite
.github/workflows/dep.yml actions
  • actions/checkout v4 composite
  • actions/dependency-review-action v3 composite
.github/workflows/deploy.yml actions
  • actions/cache v3 composite
  • actions/checkout v4 composite
  • actions/setup-go v4 composite
  • andstor/file-existence-action v2 composite
  • docker/build-push-action v5 composite
  • docker/login-action v3 composite
  • docker/setup-buildx-action v3 composite
.github/workflows/e2e.yml actions
  • actions/cache v3 composite
  • actions/checkout v4 composite
  • actions/setup-go v4 composite
.github/workflows/release.yml actions
  • actions/cache v3 composite
  • actions/checkout v4 composite
  • actions/setup-go v4 composite
  • anchore/sbom-action/download-syft v0 composite
  • docker/login-action v3 composite
  • docker/setup-buildx-action v3 composite
  • goreleaser/goreleaser-action v5 composite
  • sigstore/cosign-installer v3.1.2 composite
.github/workflows/test.main.yml actions
.github/workflows/test.yml actions
Dockerfile docker
  • alpine latest build
  • golang latest build
dev/local/clickhouse/docker-compose.yml docker
  • clickhouse/clickhouse-server 23.4
  • ghcr.io/grafana/xk6-client-tracing v0.0.2
  • go-faster/oteldb latest
  • grafana/grafana 10.0.0
dev/local/ytsaurus/docker-compose.yml docker
  • ghcr.io/grafana/xk6-client-tracing v0.0.2
  • go-faster/oteldb latest
  • grafana/grafana 10.0.0
  • ytsaurus/local stable
  • ytsaurus/ui stable
dev/otelproxy/docker-compose.yml docker
  • ghcr.io/grafana/xk6-client-tracing v0.0.2
  • go-faster/otelproxy latest
  • grafana/grafana 10.0.0
  • grafana/loki 2.8.2
  • grafana/promtail 2.8.2
  • grafana/tempo latest
  • prom/prometheus latest
  • pyroscope/pyroscope latest
internal/ytlocal/Dockerfile docker
  • ubuntu 20.04 build
go.mod go
  • contrib.go.opencensus.io/exporter/prometheus v0.4.2
  • dario.cat/mergo v1.0.0
  • github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1
  • github.com/ClickHouse/ch-go v0.58.2
  • github.com/Masterminds/goutils v1.1.1
  • github.com/Masterminds/semver/v3 v3.2.0
  • github.com/Masterminds/sprig/v3 v3.2.3
  • github.com/Microsoft/go-winio v0.6.1
  • github.com/Microsoft/hcsshim v0.11.0
  • github.com/andybalholm/brotli v1.0.5
  • github.com/beorn7/perks v1.0.1
  • github.com/cenkalti/backoff/v4 v4.2.1
  • github.com/cespare/xxhash/v2 v2.2.0
  • github.com/containerd/containerd v1.7.6
  • github.com/cpuguy83/dockercfg v0.3.1
  • github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
  • github.com/dlclark/regexp2 v1.10.0
  • github.com/dmarkham/enumer v1.5.8
  • github.com/docker/distribution v2.8.2+incompatible
  • github.com/docker/docker v24.0.6+incompatible
  • github.com/docker/go-connections v0.4.0
  • github.com/docker/go-units v0.5.0
  • github.com/dustin/go-humanize v1.0.1
  • github.com/fatih/color v1.15.0
  • github.com/felixge/httpsnoop v1.0.3
  • github.com/fsnotify/fsnotify v1.6.0
  • github.com/ghodss/yaml v1.0.0
  • github.com/go-faster/city v1.0.1
  • github.com/go-faster/errors v0.6.1
  • github.com/go-faster/jx v1.1.0
  • github.com/go-faster/sdk v0.8.2
  • github.com/go-faster/tcpproxy v0.1.0
  • github.com/go-faster/yaml v0.4.6
  • github.com/go-kit/log v0.2.1
  • github.com/go-logfmt/logfmt v0.6.0
  • github.com/go-logr/logr v1.2.4
  • github.com/go-logr/stdr v1.2.2
  • github.com/go-logr/zapr v1.2.4
  • github.com/go-ole/go-ole v1.2.6
  • github.com/gofrs/uuid v4.2.0+incompatible
  • github.com/gogo/protobuf v1.3.2
  • github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da
  • github.com/golang/protobuf v1.5.3
  • github.com/golang/snappy v0.0.4
  • github.com/google/tink/go v1.7.0
  • github.com/google/uuid v1.3.1
  • github.com/grafana/regexp v0.0.0-20221122212121-6b5c0a4cb7fd
  • github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0
  • github.com/hashicorp/go-version v1.6.0
  • github.com/huandu/xstrings v1.3.3
  • github.com/imdario/mergo v0.3.16
  • github.com/inconshreveable/mousetrap v1.1.0
  • github.com/jackc/puddle/v2 v2.2.1
  • github.com/json-iterator/go v1.1.12
  • github.com/klauspost/compress v1.16.7
  • github.com/knadh/koanf v1.5.0
  • github.com/knadh/koanf/v2 v2.0.1
  • github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0
  • github.com/magiconair/properties v1.8.7
  • github.com/mattn/go-colorable v0.1.13
  • github.com/mattn/go-isatty v0.0.19
  • github.com/matttproud/golang_protobuf_extensions v1.0.4
  • github.com/mitchellh/copystructure v1.2.0
  • github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4
  • github.com/mitchellh/reflectwalk v1.0.2
  • github.com/moby/patternmatcher v0.5.0
  • github.com/moby/sys/sequential v0.5.0
  • github.com/moby/term v0.5.0
  • github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
  • github.com/modern-go/reflect2 v1.0.2
  • github.com/morikuni/aec v1.0.0
  • github.com/mostynb/go-grpc-compression v1.2.0
  • github.com/ogen-go/ogen v0.75.0
  • github.com/opencontainers/go-digest v1.0.0
  • github.com/opencontainers/image-spec v1.1.0-rc4
  • github.com/opencontainers/runc v1.1.5
  • github.com/opentracing/opentracing-go v1.2.0
  • github.com/pascaldekloe/name v1.0.1
  • github.com/pierrec/lz4 v2.6.1+incompatible
  • github.com/pierrec/lz4/v4 v4.1.18
  • github.com/pkg/errors v0.9.1
  • github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2
  • github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c
  • github.com/prometheus/client_golang v1.16.0
  • github.com/prometheus/client_model v0.4.0
  • github.com/prometheus/common v0.44.0
  • github.com/prometheus/procfs v0.11.0
  • github.com/prometheus/prometheus v0.47.0
  • github.com/prometheus/statsd_exporter v0.22.7
  • github.com/pyroscope-io/client v0.7.2
  • github.com/rs/cors v1.10.0
  • github.com/segmentio/asm v1.2.0
  • github.com/shirou/gopsutil/v3 v3.23.7
  • github.com/shoenig/go-m1cpu v0.1.6
  • github.com/shopspring/decimal v1.2.0
  • github.com/sirupsen/logrus v1.9.0
  • github.com/spf13/cast v1.3.1
  • github.com/spf13/cobra v1.7.0
  • github.com/spf13/pflag v1.0.5
  • github.com/stretchr/testify v1.8.4
  • github.com/testcontainers/testcontainers-go v0.24.1
  • github.com/tklauser/go-sysconf v0.3.11
  • github.com/tklauser/numcpus v0.6.0
  • github.com/yusufpapurcu/wmi v1.2.3
  • go.opencensus.io v0.24.0
  • go.opentelemetry.io/collector v0.85.0
  • go.opentelemetry.io/collector/component v0.85.0
  • go.opentelemetry.io/collector/config/configauth v0.85.0
  • go.opentelemetry.io/collector/config/configcompression v0.85.0
  • go.opentelemetry.io/collector/config/configgrpc v0.85.0
  • go.opentelemetry.io/collector/config/confighttp v0.85.0
  • go.opentelemetry.io/collector/config/confignet v0.85.0
  • go.opentelemetry.io/collector/config/configopaque v0.85.0
  • go.opentelemetry.io/collector/config/configtelemetry v0.85.0
  • go.opentelemetry.io/collector/config/configtls v0.85.0
  • go.opentelemetry.io/collector/config/internal v0.85.0
  • go.opentelemetry.io/collector/confmap v0.85.0
  • go.opentelemetry.io/collector/connector v0.85.0
  • go.opentelemetry.io/collector/consumer v0.85.0
  • go.opentelemetry.io/collector/exporter v0.85.0
  • go.opentelemetry.io/collector/extension v0.85.0
  • go.opentelemetry.io/collector/extension/auth v0.85.0
  • go.opentelemetry.io/collector/featuregate v1.0.0-rcv0014
  • go.opentelemetry.io/collector/pdata v1.0.0-rcv0014
  • go.opentelemetry.io/collector/processor v0.85.0
  • go.opentelemetry.io/collector/receiver v0.85.0
  • go.opentelemetry.io/collector/receiver/otlpreceiver v0.85.0
  • go.opentelemetry.io/collector/semconv v0.85.0
  • go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.44.0
  • go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.44.0
  • go.opentelemetry.io/contrib/instrumentation/runtime v0.42.0
  • go.opentelemetry.io/contrib/propagators/autoprop v0.42.0
  • go.opentelemetry.io/contrib/propagators/aws v1.17.0
  • go.opentelemetry.io/contrib/propagators/b3 v1.17.0
  • go.opentelemetry.io/contrib/propagators/jaeger v1.17.0
  • go.opentelemetry.io/contrib/propagators/ot v1.17.0
  • go.opentelemetry.io/contrib/zpages v0.44.0
  • go.opentelemetry.io/otel v1.18.0
  • go.opentelemetry.io/otel/bridge/opencensus v0.40.0
  • go.opentelemetry.io/otel/bridge/opentracing v1.18.0
  • go.opentelemetry.io/otel/exporters/jaeger v1.16.0
  • go.opentelemetry.io/otel/exporters/otlp/otlpmetric v0.40.0
  • go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v0.40.0
  • go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v0.40.0
  • go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.17.0
  • go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.17.0
  • go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.17.0
  • go.opentelemetry.io/otel/exporters/prometheus v0.40.1-0.20230831181707-02616a25c68e
  • go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v0.40.0
  • go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.17.0
  • go.opentelemetry.io/otel/metric v1.18.0
  • go.opentelemetry.io/otel/sdk v1.18.0
  • go.opentelemetry.io/otel/sdk/metric v0.40.0
  • go.opentelemetry.io/otel/trace v1.18.0
  • go.opentelemetry.io/proto/otlp v1.0.0
  • go.uber.org/atomic v1.11.0
  • go.uber.org/multierr v1.11.0
  • go.uber.org/zap v1.26.0
  • go.ytsaurus.tech/library/go/blockcodecs v0.0.2
  • go.ytsaurus.tech/library/go/core/log v0.0.3
  • go.ytsaurus.tech/library/go/core/xerrors v0.0.3
  • go.ytsaurus.tech/library/go/ptr v0.0.1
  • go.ytsaurus.tech/library/go/x/xreflect v0.0.2
  • go.ytsaurus.tech/library/go/x/xruntime v0.0.3
  • go.ytsaurus.tech/yt/go v0.0.9
  • go4.org/netipx v0.0.0-20230303233057-f1b76eb4bb35
  • golang.org/x/crypto v0.13.0
  • golang.org/x/exp v0.0.0-20230725093048-515e97ebf090
  • golang.org/x/mod v0.12.0
  • golang.org/x/net v0.15.0
  • golang.org/x/sync v0.3.0
  • golang.org/x/sys v0.12.0
  • golang.org/x/text v0.13.0
  • golang.org/x/tools v0.13.0
  • golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2
  • gonum.org/v1/gonum v0.14.0
  • google.golang.org/genproto/googleapis/api v0.0.0-20230717213848-3f92550aa753
  • google.golang.org/genproto/googleapis/rpc v0.0.0-20230717213848-3f92550aa753
  • google.golang.org/grpc v1.58.1
  • google.golang.org/protobuf v1.31.0
  • gopkg.in/yaml.v2 v2.4.0
  • gopkg.in/yaml.v3 v3.0.1
go.sum go
  • 702 dependencies
cmd/ytlocal/requirements.txt pypi
  • ytsaurus-client *
  • ytsaurus-local *
  • ytsaurus-yson *