dfir_datalog
A high-level distributed programming framework for Rust
Science Score: 54.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.5%) to scientific vocabulary
Keywords
Repository
A high-level distributed programming framework for Rust
Basic Info
- Host: GitHub
- Owner: hydro-project
- License: apache-2.0
- Language: Rust
- Default Branch: main
- Homepage: https://hydro.run/
- Size: 316 MB
Statistics
- Stars: 986
- Watchers: 14
- Forks: 58
- Open Issues: 129
- Releases: 0
Topics
Metadata Files
README.md
Hydro is a high-level distributed programming framework for Rust. Hydro can help you quickly write scalable distributed services that are correct by construction. Much like Rust helps with memory safety, Hydro helps with distributed safety.
Hydro integrates naturally with standard Rust constructs and IDEs, providing types and programming constructs for ensuring distributed safety. Under the covers, Hydro is powered by the Dataflow Intermediate Representation (DFIR), a compiler and low-level runtime for stream processing. DFIR enables automatic vectorization and efficient scheduling without restricting your application logic.
Get started today at hydro.run!
Learn More
Docs: There are docs for the high-level Hydro framework and the low-level dataflow IR, DFIR, as well as the Hydro Deploy framework for launching Hydro programs.
Research Papers: Our research publications are available on the project website. Some notable selections:
- The original Hydro vision paper from CIDR 2021: New Directions in Cloud Programming
- The first paper on optimizations from SIGMOD 2024: Optimizing Distributed Protocols with Query Rewrites
- The first paper on Hydro's formal semantics to appear in POPL 2025: Flo: a Semantic Foundation for Progressive Stream Processing
Contributing
For Hydro development setup and contribution info, see CONTRIBUTING.md.
Owner
- Name: Hydro
- Login: hydro-project
- Kind: organization
- Location: Berkeley, CA
- Website: https://hydro.run
- Repositories: 13
- Profile: https://github.com/hydro-project
Citation (CITATION.cff)
# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!
cff-version: 1.2.0
title: Hydro
message: >-
If you use this software, please cite it using the
metadata from this file.
type: software
authors:
- given-names: Mingwei
family-names: Samuel
email: mingwei@shv.com
affiliation: Sutter Hill Ventures
orcid: 'https://orcid.org/0009-0004-9873-6266'
- given-names: Shadaj
family-names: Laddad
email: shadaj@berkeley.edu
orcid: 'https://orcid.org/0000-0002-6658-6548'
affiliation: UC Berkeley
- given-names: Joseph M.
family-names: Hellerstein
email: hellerstein@berkeley.edu
orcid: 'https://orcid.org/0000-0002-7712-4306'
affiliation: UC Berkeley
- given-names: Rohit
family-names: Kulshreshtha
email: rohit@shv.com
affiliation: Sutter Hill Ventures
- given-names: Lucky
family-names: Katahanas
email: lucky@shv.com
affiliation: Sutter Hill Ventures
- given-names: Justin
family-names: Jaffray
email: justin@shv.com
affiliation: Sutter Hill Ventures
- given-names: David
family-names: Chu
affiliation: UC Berkeley
email: thedavidchu@berkeley.edu
- given-names: Conor
email: conorpower@berkeley.edu
family-names: Power
orcid: 'https://orcid.org/0000-0002-0660-5110'
affiliation: UC Berkeley
repository-code: 'https://github.com/hydro-project/hydro/'
url: 'https://hydro.run/'
abstract: >-
Hydro is a framework for programming distributed systems,
built on top of DFIR, a lattice-aware dataflow system.
keywords:
- dataflow
- distributed systems
- cloud computing
- query optimization
- programming languages
- rust
license: Apache-2.0
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 103
- Total pull requests: 255
- Average time to close issues: about 2 months
- Average time to close pull requests: 7 days
- Total issue authors: 8
- Total pull request authors: 15
- Average comments per issue: 0.51
- Average comments per pull request: 0.68
- Merged pull requests: 164
- Bot issues: 0
- Bot pull requests: 2
Past Year
- Issues: 101
- Pull requests: 254
- Average time to close issues: about 1 month
- Average time to close pull requests: 3 days
- Issue authors: 8
- Pull request authors: 15
- Average comments per issue: 0.5
- Average comments per pull request: 0.68
- Merged pull requests: 164
- Bot issues: 0
- Bot pull requests: 2
Top Authors
Issue Authors
- shadaj (43)
- MingweiSamuel (38)
- jhellerstein (12)
- rohitkulshreshtha (4)
- davidchuyaya (3)
- rbalicki2 (1)
- ntrp (1)
- chenyh1990 (1)
Pull Request Authors
- shadaj (115)
- MingweiSamuel (74)
- davidchuyaya (21)
- jhellerstein (15)
- Copilot (10)
- luckyworkama (9)
- Honeycomb (2)
- Flamenco (2)
- rohitkulshreshtha (1)
- github-actions[bot] (1)
- rupert648 (1)
- runllm-pr-agent[bot] (1)
- FHaggs (1)
- ydewit (1)
- RyanAlameddine (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 20
-
Total downloads:
- cargo 258,429 total
-
Total dependent packages: 7
(may contain duplicates) -
Total dependent repositories: 4
(may contain duplicates) - Total versions: 131
- Total maintainers: 2
crates.io: variadics
Variadic generics on stable Rust using tuple lists
- Documentation: https://docs.rs/variadics/
- License: Apache-2.0
-
Latest release: 0.0.9
published 12 months ago
Rankings
Maintainers (2)
crates.io: lattices
Lattice data types for simplifying distributed state by providing associativity, commutativity, and idempotence.
- Documentation: https://docs.rs/lattices/
- License: Apache-2.0
-
Latest release: 0.6.1
published 11 months ago
Rankings
Maintainers (2)
crates.io: pusherator
Push-based version of Rust iterators
- Documentation: https://docs.rs/pusherator/
- License: Apache-2.0
-
Latest release: 0.0.12
published 11 months ago
Rankings
Maintainers (2)
crates.io: hydro_optimize
Automatic decoupling and partitioning
- Documentation: https://docs.rs/hydro_optimize/
- License: Apache-2.0
-
Latest release: 0.13.0
published 7 months ago
Rankings
Maintainers (1)
crates.io: hydro_deploy_integration
`hydro_deploy` integration for deploying Rust crates.
- Documentation: https://docs.rs/hydro_deploy_integration/
- License: Apache-2.0
-
Latest release: 0.14.0
published 7 months ago
Rankings
Maintainers (1)
crates.io: dfir_lang
Hydro's Dataflow Intermediate Representation (DFIR) implementation
- Documentation: https://docs.rs/dfir_lang/
- License: Apache-2.0
-
Latest release: 0.14.0
published 7 months ago
Rankings
Maintainers (2)
crates.io: dfir_datalog_core
Datalog implementation for DFIR
- Documentation: https://docs.rs/dfir_datalog_core/
- License: Apache-2.0
-
Latest release: 0.13.0
published 11 months ago
Rankings
Maintainers (2)
crates.io: dfir_rs
DFIR runtime for Rust, used by Hydro.
- Documentation: https://docs.rs/dfir_rs/
- License: Apache-2.0
-
Latest release: 0.14.0
published 7 months ago
Rankings
Maintainers (2)
crates.io: dfir_macro
Macro for using Hydro's Data Flow Intermediate Representation (DFIR).
- Documentation: https://docs.rs/dfir_macro/
- License: Apache-2.0
-
Latest release: 0.14.0
published 7 months ago
Rankings
Maintainers (2)
crates.io: dfir_datalog
Datalog proc-macro for DFIR
- Documentation: https://docs.rs/dfir_datalog/
- License: Apache-2.0
-
Latest release: 0.13.0
published 11 months ago
Rankings
Maintainers (2)
crates.io: multiplatform_test
A simple attribute macro to combine `#[test]` and `#[wasm_bindgen_test]`
- Documentation: https://docs.rs/multiplatform_test/
- License: Apache-2.0
-
Latest release: 0.5.0
published 12 months ago
Rankings
Maintainers (2)
crates.io: example_test
Test `cargo example` code.
- Documentation: https://docs.rs/example_test/
- License: Apache-2.0
-
Latest release: 0.0.0
published 7 months ago
Rankings
Maintainers (1)
crates.io: include_mdtests
Include markdown for testing, via doctests.
- Documentation: https://docs.rs/include_mdtests/
- License: Apache-2.0
-
Latest release: 0.0.0
published 11 months ago
Rankings
Maintainers (1)
crates.io: hydro_lang
Choreographic programming for Hydro, built on DFIR
- Documentation: https://docs.rs/hydro_lang/
- License: Apache-2.0
-
Latest release: 0.14.0
published 7 months ago
Rankings
Maintainers (2)
crates.io: hydro_std
Standard library of distributed systems building blocks for Hydro
- Documentation: https://docs.rs/hydro_std/
- License: Apache-2.0
-
Latest release: 0.14.0
published 7 months ago
Rankings
Maintainers (2)
crates.io: variadics_macro
Procedural macros for the `variadics` crate.
- Documentation: https://docs.rs/variadics_macro/
- License: Apache-2.0
-
Latest release: 0.6.1
published 7 months ago
Rankings
Maintainers (1)
crates.io: hydroflow_deploy_integration
`hydro_deploy` integration for Hydroflow
- Documentation: https://docs.rs/hydroflow_deploy_integration/
- License: Apache-2.0
-
Latest release: 0.12.0
published 12 months ago
Rankings
Maintainers (1)
crates.io: lattices_macro
Procedural macros for the `lattices` crate.
- Documentation: https://docs.rs/lattices_macro/
- License: Apache-2.0
-
Latest release: 0.5.10
published 7 months ago
Rankings
Maintainers (1)
crates.io: hydro_cli
Hydro Deploy Command Line Interface
- Documentation: https://docs.rs/hydro_cli/
- License: Apache-2.0
-
Latest release: 0.13.0
published 11 months ago
Rankings
Maintainers (2)
crates.io: hydro_deploy
Hydro Deploy
- Documentation: https://docs.rs/hydro_deploy/
- License: Apache-2.0
-
Latest release: 0.14.0
published 7 months ago
Rankings
Maintainers (2)
Dependencies
- 164 dependencies
- anyhow 1.0.69
- async-channel 1.8.0
- async-process 1.6.0
- async-recursion 1
- async-ssh2-lite 0.4.2
- async-trait 0.1.64
- bytes 1.1.0
- cargo_metadata 0.15.4
- clap 4.1.8
- ctrlc 3.2.5
- dyn-clone 1
- futures 0.3.26
- futures-core 0.3.26
- hydroflow_cli_integration ^0.3.0
- indicatif 0.17.3
- nanoid 0.4.0
- nix 0.26.2
- once_cell 1.17
- pyo3 0.18
- pyo3-asyncio 0.18
- pythonize 0.18
- serde 1
- serde_json 1
- shell-escape 0.1.5
- tempfile 3.3.0
- tokio 1.16
- tokio-util 0.7.7
- chrono 0.4.20 development
- clap 4.1.8 development
- colored 2.0 development
- core_affinity 0.5.10 development
- ctor 0.2 development
- futures 0.3 development
- hdrhistogram 7 development
- insta 1.7.1 development
- multiplatform_test 0.0.0 development
- rand 0.8.4 development
- rand_distr 0.4.3 development
- regex 1 development
- static_assertions 1.1.0 development
- textnonce 1.0.0 development
- time 0.3 development
- tracing-subscriber 0.3 development
- trybuild 1.0.80 development
- wasm-bindgen-test 0.3.34 development
- zipf 7.0.0 development
- bincode 1.3
- byteorder 1.4.3
- bytes 1.1.0
- futures 0.3
- hydroflow_cli_integration ^0.3.0
- hydroflow_datalog ^0.5.0
- hydroflow_lang ^0.5.0
- hydroflow_macro ^0.5.0
- instant 0.1.12
- itertools 0.10
- lattices ^0.5.0
- pusherator ^0.0.3
- pyo3 0.18
- ref-cast 1.0
- regex 1.8.4
- rustc-hash 1.1.0
- sealed 0.5
- serde 1
- serde_json 1
- slotmap 1.0
- smallvec 1.10.0
- tokio-stream 0.1.10
- tracing 0.1
- variadics ^0.0.2
- insta 1.7.1 development
- prettyplease 0.2.0 development
- tempfile 3.2.0 development
- hydroflow_lang ^0.5.0
- proc-macro-crate 1.1.0
- proc-macro2 1.0.63
- quote 1.0.0
- rust-sitter 0.3.2
- slotmap 1.0.6
- syn 2.0.0
- auto_impl 1.0.1
- clap 4.1.8
- data-encoding 2.4.0
- itertools 0.10
- prettyplease 0.2.0
- proc-macro2 1.0.63
- quote 1.0.0
- regex 1.7.0
- serde 1.0.1
- serde_json 1.0.8
- slotmap 1.0.6
- syn 2.0.0
- webbrowser 0.8.11
- env_logger 0.10 development
- log 0.4 development
- tracing 0.1 development
- tracing-subscriber 0.3 development
- wasm-bindgen-test 0.3 development
- proc-macro2 1.0.63
- quote 1.0
- wasm-bindgen-test 0.3.13 development
- console_error_panic_hook 0.1.6
- futures 0.3
- prettyplease 0.2.0
- proc-macro2 1.0.63
- quote 1.0
- serde 1.0
- serde-wasm-bindgen 0.4
- syn 2.0.0
- tokio 1.16
- wasm-bindgen 0.2
- web-sys 0.3.61
- wee_alloc 0.4.5
- pytest * test
- pytest-asyncio * test
- actions-rs/audit-check v1 composite
- actions/checkout v3 composite
- PyO3/maturin-action v1 composite
- actions/checkout v3 composite
- actions/download-artifact v3 composite
- actions/setup-python v4 composite
- actions/upload-artifact v3 composite
- fkirc/skip-duplicate-actions v5.3.0 composite
- actions-rs/cargo v1 composite
- actions-rs/toolchain v1 composite
- actions/checkout v3 composite
- actions/setup-python v4 composite
- actions/upload-artifact master composite
- benchmark-action/github-action-benchmark v1 composite
- fkirc/skip-duplicate-actions v5.3.0 composite
- actions/checkout v2 composite
- webiny/action-conventional-commits v1.1.0 composite
- actions/checkout v3 composite
- docker/build-push-action v3 composite
- docker/login-action v2 composite
- docker/setup-buildx-action v2 composite
- docker/setup-qemu-action v2 composite
- fkirc/skip-duplicate-actions v5.3.0 composite
- actions-rs/cargo v1 composite
- actions-rs/toolchain v1 composite
- actions/checkout v3 composite
- tibdex/github-app-token v1 composite
- criterion 0.4 development
- differential-dataflow * development
- lazy_static 1.4.0 development
- rand 0.8.4 development
- rand_distr 0.4.3 development
- seq-macro 0.2 development
- static_assertions 1.1.0 development
- timely * development
- tokio 1.0 development
- rust slim-buster build
- 1131 dependencies
- @docusaurus/module-type-aliases 2.4.1 development
- @docusaurus/core 2.4.1
- @docusaurus/plugin-ideal-image ^2.4.1
- @docusaurus/preset-classic 2.4.1
- @docusaurus/theme-mermaid ^2.4.1
- @mdx-js/react ^1.6.22
- @monaco-editor/react ^4.4.6
- clsx ^1.2.1
- hast-util-is-element ^1.1.0
- mermaid ^9.3.0
- prism-react-renderer ^1.3.5
- raw-loader ^4.0.2
- react ^17.0.2
- react-dom ^17.0.2
- react-katex ^3.0.1
- rehype-katex ^5.0.0
- remark-math ^3.0.1
- website_playground file:../website_playground/pkg