Data

Tools

Indexes

Applications

Experiments

Open Source Science

DeepEquilibriumNetworks

Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence.

https://github.com/sciml/deepequilibriumnetworks.jl

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 10 committers (10.0%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.9%) to scientific vocabulary

Keywords

deep-equilibrium-models deep-learning implicit-deep-learning julia machine-learning neural-networks nonlinear-equations nonlinear-solve

Keywords from Contributors

ode matrix-exponential neural-sde hybrid-differential-equations sciml differential-equations sde pde dae stochastic-differential-equations
Last synced: 4 months ago · JSON representation ·

Repository

Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence.

Basic Info
Statistics
  • Stars: 57
  • Watchers: 4
  • Forks: 6
  • Open Issues: 1
  • Releases: 26
Topics
deep-equilibrium-models deep-learning implicit-deep-learning julia machine-learning neural-networks nonlinear-equations nonlinear-solve
Created over 4 years ago · Last pushed 5 months ago
Metadata Files
Readme License Citation

README.md

DeepEquilibriumNetworks

Join the chat at https://julialang.zulipchat.com #sciml-bridged Global Docs

codecov Build Status Build status

ColPrac: Contributor's Guide on Collaborative Practices for Community Packages SciML Code Style

DeepEquilibriumNetworks.jl is a framework built on top of DifferentialEquations.jl and Lux.jl enabling the efficient training and inference for Deep Equilibrium Networks (Infinitely Deep Neural Networks).

Installation

julia using Pkg Pkg.add("DeepEquilibriumNetworks")

Quickstart

```julia using DeepEquilibriumNetworks, Lux, Random, NonlinearSolve, Zygote, SciMLSensitivity

using LuxCUDA, LuxAMDGPU ## Install and Load for GPU Support. See https://lux.csail.mit.edu/dev/manual/gpu_management

seed = 0 rng = Random.default_rng() Random.seed!(rng, seed)

model = Chain(Dense(2 => 2), DeepEquilibriumNetwork( Parallel(+, Dense(2 => 2; usebias=false), Dense(2 => 2; usebias=false)), NewtonRaphson()))

gdev = gpudevice() cdev = cpudevice()

ps, st = Lux.setup(rng, model) |> gdev x = rand(rng, Float32, 2, 3) |> gdev y = rand(rng, Float32, 2, 3) |> gdev

model(x, ps, st)

gs = only(Zygote.gradient(p -> sum(abs2, first(model(x, p, st)) .- y), ps)) ```

Citation

If you are using this project for research or other academic purposes consider citing our paper:

bibtex @article{pal2022continuous, title={Continuous Deep Equilibrium Models: Training Neural ODEs Faster by Integrating Them to Infinity}, author={Pal, Avik and Edelman, Alan and Rackauckas, Christopher}, booktitle={2023 IEEE High Performance Extreme Computing Conference (HPEC)}, year={2023} }

For specific algorithms, check the respective documentations and cite the corresponding papers.

Owner

  • Name: SciML Open Source Scientific Machine Learning
  • Login: SciML
  • Kind: organization
  • Email: contact@chrisrackauckas.com

Open source software for scientific machine learning

Citation (CITATION.bib) 

@article{pal2022continuous,
  title={Continuous Deep Equilibrium Models: Training Neural ODEs Faster by Integrating Them to Infinity},
  author={Pal, Avik and Edelman, Alan and Rackauckas, Christopher},
  booktitle={2023 IEEE High Performance Extreme Computing Conference (HPEC)}, 
  year={2023}
}

GitHub Events

Total
  • Release event: 1
  • Watch event: 6
  • Delete event: 7
  • Issue comment event: 3
  • Push event: 15
  • Pull request review event: 1
  • Pull request review comment event: 1
  • Pull request event: 21
  • Fork event: 1
  • Create event: 10
Last Year
  • Release event: 1
  • Watch event: 6
  • Delete event: 7
  • Issue comment event: 3
  • Push event: 15
  • Pull request review event: 1
  • Pull request review comment event: 1
  • Pull request event: 21
  • Fork event: 1
  • Create event: 10

Committers

Last synced: 6 months ago

All Time
  • Total Commits: 517
  • Total Committers: 10
  • Avg Commits per committer: 51.7
  • Development Distribution Score (DDS): 0.184
Past Year
  • Commits: 36
  • Committers: 4
  • Avg Commits per committer: 9.0
  • Development Distribution Score (DDS): 0.583
Top Committers
Name Email Commits
Avik Pal a****l@i****n 422
Christopher Rackauckas a****s@c****m 36
CompatHelper Julia c****y@j****g 23
Arno Strouwen a****n@t****e 13
dependabot[bot] 4****] 13
Anant Thazhemadam a****m@g****m 5
Krishna Bhogaonker c****q@g****m 2
Hendrik Ranocha m****l@r****e 1
David Widmann d****n@i****e 1
Anas Abdelrehim 7****R 1
Committer Domains (Top 20 + Academic)
it.uu.se: 1 ranocha.de: 1 telenet.be: 1 julialang.org: 1 chrisrackauckas.com: 1 iitk.ac.in: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time
  • Total issues: 11
  • Total pull requests: 169
  • Average time to close issues: 5 months
  • Average time to close pull requests: 14 days
  • Total issue authors: 6
  • Total pull request authors: 10
  • Average comments per issue: 6.27
  • Average comments per pull request: 0.5
  • Merged pull requests: 90
  • Bot issues: 0
  • Bot pull requests: 111
Past Year
  • Issues: 0
  • Pull requests: 23
  • Average time to close issues: N/A
  • Average time to close pull requests: 2 days
  • Issue authors: 0
  • Pull request authors: 6
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 16
  • Bot issues: 0
  • Bot pull requests: 11
View more stats
Top Authors
Issue Authors
  • avik-pal (6)
  • yadmtr (1)
  • ChrisRackauckas (1)
  • AnasAbdelR (1)
  • EthanDecleyn (1)
  • JuliaTagBot (1)
Pull Request Authors
  • github-actions[bot] (104)
  • avik-pal (28)
  • ChrisRackauckas (19)
  • dependabot[bot] (19)
  • ArnoStrouwen (13)
  • thazhemadam (3)
  • 00krishna (2)
  • ranocha (1)
  • devmotion (1)
  • AnasAbdelR (1)
Top Labels
Issue Labels
Pull Request Labels
dependencies (19) formatting (8) automated pr (8) no changelog (8) github_actions (1)

Packages

  • Total packages: 1
  • Total downloads:
    • julia 3 total
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 26
juliahub.com: DeepEquilibriumNetworks

Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence.

  • Versions: 26
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 3 Total
Rankings
Dependent repos count: 9.9%
Stargazers count: 15.0%
Average: 22.1%
Forks count: 24.5%
Dependent packages count: 38.9%
Last synced: 5 months ago

Dependencies

.github/workflows/CI.yml actions
  • actions/cache v1 composite
  • actions/checkout v2 composite
  • codecov/codecov-action v2 composite
  • julia-actions/julia-buildpkg v1 composite
  • julia-actions/julia-processcoverage v1 composite
  • julia-actions/julia-runtest v1 composite
  • julia-actions/setup-julia v1 composite
.github/workflows/CompatHelper.yml actions
  • julia-actions/setup-julia v1 composite
.github/workflows/Documentation.yml actions
  • actions/checkout v2 composite
  • codecov/codecov-action v2 composite
  • julia-actions/julia-processcoverage v1 composite
  • julia-actions/setup-julia v1 composite
.github/workflows/FormatCheck.yml actions
  • actions/checkout v1 composite
  • julia-actions/setup-julia latest composite
.github/workflows/Invalidations.yml actions
  • actions/checkout v3 composite
  • julia-actions/julia-buildpkg v1 composite
  • julia-actions/julia-invalidations v1 composite
  • julia-actions/setup-julia v1 composite
.github/workflows/TagBot.yml actions
  • JuliaRegistries/TagBot v1 composite