Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of https://github.com/madesroches/micromegas

https://github.com/madesroches/micromegas - Micromegas v0.12.0 - Major Async Span Tracing Release

🎉 Micromegas v0.12.0 - Major Async Span Tracing Release

We're excited to announce Micromegas v0.12.0, featuring comprehensive async span tracing capabilities, new integrations, and significant infrastructure improvements!

🚀 Major Features

Async Span Tracing Revolution

  • Comprehensive async span tracing with micromegas_main proc macro (#451)
  • Named async span event tracking with improved API ergonomics (#475)
  • Async span depth tracking for performance analysis (#474)
  • Async trait tracing support in span_fn macro (#469)
  • Perfetto async spans support with trace generation (#485)

New Integrations & Gateways

  • HTTP gateway for easier interoperability (#433, #435, #436)
  • JSONB support for flexible data structures (#409)
  • Web UI Perfetto export capabilities (#482)

⚡ Infrastructure & Performance

  • SQL-powered Perfetto trace generation consolidation (#489)
  • Query latency tracking and async span instrumentation optimization (#468)
  • Internment crate integration replacing custom interning logic (#430)
  • Optimized view_instance metadata requests (#450)
  • In-memory recording for all unit tests (#472)

📚 Documentation & Developer Experience

  • Complete Python API documentation with comprehensive docstrings (#491)
  • Complete SQL functions documentation with all missing UDFs/UDAFs/UDTFs (#470)
  • Visual architecture diagrams in documentation (#462)
  • Unreal instrumentation documentation (#492)
  • Automated documentation publishing workflow (#444)

🔒 Security & Dependencies

  • CVE-2025-58160 fix: Updated tracing-subscriber to 0.3.20 (#490)
  • DataFusion, tokio and dependency updates (#429, #476)
  • Rust edition 2024 upgrade with unsafe operations fixes (#408)

☁️ Cloud & Deployment

  • Docker deployment scripts (#422)
  • Amazon Linux setup script (#423)
  • Cloud environment configuration support (#426)
  • Configurable PostgreSQL port via MICROMEGASDBPORT (#425)

📦 What's Included

🦀 Rust Crates (crates.io)

All core crates published to v0.12.0: - micromegas - Main public crate - micromegas-analytics - DataFusion analytics engine - micromegas-tracing - High-performance instrumentation - micromegas-telemetry - Core data structures - micromegas-ingestion - Data ingestion utilities - micromegas-telemetry-sink - Event sinks - micromegas-perfetto - Perfetto trace generation - micromegas-transit - Serialization framework - micromegas-derive-transit - Transit macros - micromegas-tracing-proc-macros - Tracing macros - micromegas-proc-macros - Top-level macros

🐍 Python Package (PyPI)

🛠️ Installation

Rust

toml [dependencies] micromegas = "0.12.0" micromegas-tracing = "0.12.0"

Python

```bash pip install micromegas==0.12.0

or

poetry add micromegas==0.12.0 ```

📖 Resources

🎯 Migration & Breaking Changes

This release focuses on expanding async capabilities and improving performance. Most existing APIs remain compatible, with new async tracing features being additive.

Full Changelog: https://github.com/madesroches/micromegas/compare/micromegas-v0.11.0...v0.12.0

- Rust
Published by madesroches 6 months ago

https://github.com/madesroches/micromegas - micromegas 0.11.0

What's Changed

  • datetimetoscalar by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/392
  • allow custom merge operations to use joins by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/393
  • full query context for merges in SqlBatchView by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/394
  • Batch partition merger by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/395
  • Prepared statements by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/398
  • daemon: specify which views to update by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/399
  • update datafusion by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/400
  • feat: Update Rust edition to 2024 and fix unsafe operations by @madesroches in https://github.com/madesroches/micromegas/pull/408
  • JSONB support by @madesroches in https://github.com/madesroches/micromegas/pull/409
  • document cost modeling by @madesroches in https://github.com/madesroches/micromegas/pull/411
  • tolerate but warn when invalid json is given by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/415
  • ExportLogView: materializing the log of an export function by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/418

Full Changelog: https://github.com/madesroches/micromegas/compare/micromegas-v0.10.0...micromegas-v0.11.0

- Rust
Published by madesroches 7 months ago

https://github.com/madesroches/micromegas - micromegas-v0.10.0

What's Changed

  • Process properties in log_entries & measures by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/378
  • ignore partitions with wrong schema by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/379
  • add support for nulls in histogram source data by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/382
  • histogram accessors by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/383
  • new processes view based on SqlBatchView by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/384
  • new streams view based on SqlBatchView by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/385
  • update datafusion & other dependencies by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/388

Full Changelog: https://github.com/madesroches/micromegas/compare/micromegas-v0.9.0...micromegas-v0.10.0

- Rust
Published by madesroches 8 months ago

https://github.com/madesroches/micromegas - micromegas-v0.9.0

What's Changed

  • parallel minutes task by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/366
  • GetPayload user defined function by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/368
  • replication by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/369
  • Flight message size by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/370
  • Python FlightSQL refac by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/372

Full Changelog: https://github.com/madesroches/micromegas/compare/micromegas-v0.8.0...micromegas-v0.9.0

- Rust
Published by madesroches 9 months ago

https://github.com/madesroches/micromegas - 0.8.0

What's Changed

  • update datafusion by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/355
  • use target property in local sink by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/356
  • add frame budget reporting code, json logging of record batches and m… by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/357
  • Query processes, frame stats by @madesroches in https://github.com/madesroches/micromegas/pull/360
  • Add histogram support by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/361
  • Histogram support in custom views by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/362
  • run seconds tasks in parallel by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/363

Full Changelog: https://github.com/madesroches/micromegas/compare/micromegas-v0.7.0...micromegas-v0.8.0

- Rust
Published by madesroches 10 months ago

https://github.com/madesroches/micromegas - micromegas-0.7.0

What's Changed

  • clippy warnings by @madesroches in https://github.com/madesroches/micromegas/pull/339
  • do not skip tasks - allow them to be late by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/341
  • leaner schedule by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/342
  • update dependencies by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/343
  • Fixed macros needlessly required literals by @jkauffmann-ubi in https://github.com/madesroches/micromegas/pull/344
  • fixed empty jit partition error by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/345
  • Exposed process properties in telemetry guards by @jkauffmann-ubi in https://github.com/madesroches/micromegas/pull/347
  • query blocks using time instead of ticks by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/348
  • fix crash because property set were moving around in memory by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/349
  • simpler query interface by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/350
  • Implemented retry for blocks by @jkauffmann-ubi in https://github.com/madesroches/micromegas/pull/351

Full Changelog: https://github.com/madesroches/micromegas/compare/micromegas-v0.6.0...micromegas-v0.7.0

- Rust
Published by madesroches 10 months ago

https://github.com/madesroches/micromegas - micromegas-v0.6.0

What's Changed

  • perfetto trace server by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/327
  • update datafusion by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/328
  • divide jit partitions into segments by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/330
  • allow custom merge operation on sql-defined view by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/331
  • better error reporting when merging partitions by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/332
  • Memoptim by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/333
  • make all SessionContext share a runtime with an optional memory budget by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/334
  • don't execute low frequency tasks when booting the daemon, run tasks … by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/335
  • optimize memory use of fetchpartitionsource_data by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/337

Full Changelog: https://github.com/madesroches/micromegas/compare/micromegas-v0.5.0...micromegas-v0.6.0

- Rust
Published by madesroches 11 months ago

https://github.com/madesroches/micromegas - micromegas-v0.5.0

What's Changed

  • Perfetto trace client by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/325
  • Removed dependency on openssl by @ereOn in https://github.com/madesroches/micromegas/pull/317
  • Added a method to manually set the telemetry sink url by @jkauffmann-ubi in https://github.com/madesroches/micromegas/pull/318
  • unreal crash reporting by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/320
  • update deps, including ring by @madesroches in https://github.com/madesroches/micromegas/pull/321
  • flightsql rust client auth by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/323

New Contributors

  • @jkauffmann-ubi made their first contribution in https://github.com/madesroches/micromegas/pull/318

- Rust
Published by madesroches 11 months ago

https://github.com/madesroches/micromegas - micromegas-v0.5.0

What's Changed

  • Perfetto trace client by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/325
  • Removed dependency on openssl by @ereOn in https://github.com/madesroches/micromegas/pull/317
  • Added a method to manually set the telemetry sink url by @jkauffmann-ubi in https://github.com/madesroches/micromegas/pull/318
  • unreal crash reporting by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/320
  • update deps, including ring by @madesroches in https://github.com/madesroches/micromegas/pull/321
  • flightsql rust client auth by @madesroches-ubi in https://github.com/madesroches/micromegas/pull/323

New Contributors

  • @jkauffmann-ubi made their first contribution in https://github.com/madesroches/micromegas/pull/318

- Rust
Published by madesroches 11 months ago