Science Score: 65.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 5 DOI reference(s) in README -
○Academic publication links
-
○Academic email domains
-
✓Institutional organization owner
Organization luxdl has institutional domain (lux.csail.mit.edu) -
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.8%) to scientific vocabulary
Keywords
Repository
Elegant and Performant Deep Learning
Basic Info
- Host: GitHub
- Owner: LuxDL
- License: mit
- Language: Julia
- Default Branch: main
- Homepage: https://lux.csail.mit.edu/
- Size: 294 MB
Statistics
- Stars: 609
- Watchers: 11
- Forks: 74
- Open Issues: 73
- Releases: 241
Topics
Metadata Files
README.md
Elegant & Performant Deep Learning in JuliaLang
Model with the elegance of Julia, and the performance of XLA.
💻 Installation
julia
import Pkg
Pkg.add("Lux")
[!TIP] To use Lux online, use Google Colab. The Julia Runtime comes pre-installed with Lux and Reactant!
🤸 Quickstart
Reactant & Enzyme
```julia using Lux, Random, Optimisers, Reactant, Enzyme
rng = Random.default_rng() Random.seed!(rng, 0)
model = Chain(Dense(128, 256, tanh), Chain(Dense(256, 1, tanh), Dense(1, 10)))
dev = reactant_device()
ps, st = Lux.setup(rng, model) |> dev
x = rand(rng, Float32, 128, 2) |> dev
We need to compile the model before we can use it.
modelforward = @compile model(x, ps, Lux.testmode(st)) modelforward(x, ps, Lux.testmode(st))
Gradients can be computed using Enzyme
@jit Enzyme.gradient(Reverse, sum ∘ first ∘ Lux.apply, Const(model), x, ps, Const(st))
All of this can be automated using the TrainState API
train_state = Training.TrainState(model, ps, st, Adam(0.001f0))
gs, loss, stats, trainstate = Training.singletrainstep!( AutoEnzyme(), MSELoss(), (x, dev(rand(rng, Float32, 10, 2))), trainstate ) ```
Native Julia & Zygote
```julia using Lux, Random, Optimisers, Zygote
using LuxCUDA, AMDGPU, Metal, oneAPI # Optional packages for GPU support
Seeding
rng = Random.default_rng() Random.seed!(rng, 0)
Construct the layer
model = Chain(Dense(128, 256, tanh), Chain(Dense(256, 1, tanh), Dense(1, 10)))
Get the device determined by Lux
dev = gpu_device()
Parameter and State Variables
ps, st = Lux.setup(rng, model) |> dev
Dummy Input
x = rand(rng, Float32, 128, 2) |> dev
Run the model
y, st = Lux.apply(model, x, ps, st)
Gradients
First construct a TrainState
train_state = Lux.Training.TrainState(model, ps, st, Adam(0.0001f0))
We can compute the gradients using Training.compute_gradients
gs, loss, stats, trainstate = Lux.Training.computegradients(AutoZygote(), MSELoss(), (x, dev(rand(rng, Float32, 10, 2))), train_state)
Optimization
trainstate = Training.applygradients!(trainstate, gs) # or Training.applygradients (no ! at the end)
Both these steps can be combined into a single call
gs, loss, stats, trainstate = Training.singletrainstep!(AutoZygote(), MSELoss(), (x, dev(rand(rng, Float32, 10, 2))), trainstate) ```
📚 Examples
Look in the examples directory for self-contained usage examples. The documentation has examples sorted into proper categories.
🆘 Getting Help
For usage related questions, please use Github Discussions which allows questions and answers to be indexed. To report bugs use github issues or even better send in a pull request.
🧑🔬 Citation
If you found this library to be useful in academic work, then please cite:
```bibtex @software{pal2023lux, author = {Pal, Avik}, title = {{Lux: Explicit Parameterization of Deep Neural Networks in Julia}}, month = apr, year = 2023, note = {If you use this software, please cite it as below.}, publisher = {Zenodo}, version = {v1.4.2}, doi = {10.5281/zenodo.7808903}, url = {https://doi.org/10.5281/zenodo.7808903}, swhid = {swh:1:dir:1a304ec3243961314a1cc7c1481a31c4386c4a34;origin=https://doi.org/10.5281/zenodo.7808903;visit=swh:1:snp:e2bbe43b14bde47c4ddf7e637eb7fc7bd10db8c7;anchor=swh:1:rel:2c0c0ff927e7bfe8fc8bc43fd553ab392a6eb403;path=/} }
@thesis{pal2023efficient, title = {{On Efficient Training & Inference of Neural Differential Equations}}, author = {Pal, Avik}, year = {2023}, school = {Massachusetts Institute of Technology} } ```
Also consider starring our github repo.
🧑💻 Contributing
This section is somewhat incomplete. You can contribute by contributing to finishing this section 😜.
💎 Formatting (JuliaFormatter)
[!NOTE] Pin JuliaFormatter to v1 until upstream issues with v2 are resolved.
julia
using JuliaFormatter
format(".")
🧪 Testing
The full test of Lux.jl takes a long time, here's how to test a portion of the code.
For each @testitem, there are corresponding tags, for example:
julia
@testitem "SkipConnection" setup=[SharedTestSetup] tags=[:core_layers]
For example, let's consider the tests for SkipConnection:
julia
@testitem "SkipConnection" setup=[SharedTestSetup] tags=[:core_layers] begin
...
end
We can test the group to which SkipConnection belongs by testing core_layers.
To do so set the LUX_TEST_GROUP environment variable, or rename the tag to
further narrow the test scope:
shell
export LUX_TEST_GROUP="core_layers"
Or directly modify the default test tag in runtests.jl:
```julia
const LUXTESTGROUP = lowercase(get(ENV, "LUXTESTGROUP", "all"))
const LUXTESTGROUP = lowercase(get(ENV, "LUXTESTGROUP", "core_layers")) ```
But be sure to restore the default value "all" before submitting the code.
Furthermore if you want to run a specific test based on the name of the testset, you can use TestEnv.jl as follows. Start with activating the Lux environment and then run the following:
```julia using TestEnv; TestEnv.activate(); using ReTestItems;
Assuming you are in the main directory of Lux
ReTestItems.runtests("tests/"; name = "NAME OF THE TEST") ```
For the SkipConnection tests that would be:
julia
ReTestItems.runtests("tests/"; name = "SkipConnection")
Owner
- Name: LuxDL
- Login: LuxDL
- Kind: organization
- Location: United States of America
- Website: https://lux.csail.mit.edu/stable/
- Repositories: 1
- Profile: https://github.com/LuxDL
Citation (CITATION.cff)
cff-version: 1.2.0 message: "If you use this software, please cite it as below." authors: - family-names: "Pal" given-names: "Avik" orcid: "https://orcid.org/0000-0002-3938-7375" title: "Lux: Explicit Parameterization of Deep Neural Networks in Julia" version: 1.4.2 date-released: 2022-05-10 url: "https://github.com/LuxDL/Lux.jl"
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 237
- Total pull requests: 669
- Average time to close issues: about 2 months
- Average time to close pull requests: 3 days
- Total issue authors: 73
- Total pull request authors: 42
- Average comments per issue: 3.52
- Average comments per pull request: 0.75
- Merged pull requests: 438
- Bot issues: 4
- Bot pull requests: 266
Past Year
- Issues: 113
- Pull requests: 380
- Average time to close issues: 10 days
- Average time to close pull requests: 2 days
- Issue authors: 45
- Pull request authors: 30
- Average comments per issue: 1.08
- Average comments per pull request: 0.6
- Merged pull requests: 221
- Bot issues: 3
- Bot pull requests: 171
Top Authors
Issue Authors
- avik-pal (106)
- schlichtanders (9)
- NeroBlackstone (7)
- prbzrg (6)
- ExpandingMan (5)
- bertini97 (5)
- gdalle (5)
- CarloLucibello (4)
- github-actions[bot] (4)
- vpuri3 (4)
- chooron (4)
- CatYukino (2)
- arthur-bizzi (2)
- ablaom (2)
- MartinuzziFrancesco (2)
Pull Request Authors
- avik-pal (321)
- github-actions[bot] (213)
- dependabot[bot] (53)
- abhro (11)
- NeroBlackstone (11)
- Sleort (4)
- wsmoses (4)
- MartinuzziFrancesco (4)
- ldeso (3)
- CarloLucibello (3)
- Copilot (3)
- agdestein (3)
- asinghvi17 (3)
- prbzrg (2)
- simeonschaub (2)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 6
-
Total downloads:
- julia 9,059 total
-
Total dependent packages: 31
(may contain duplicates) -
Total dependent repositories: 0
(may contain duplicates) - Total versions: 432
juliahub.com: Lux
- Documentation: https://docs.juliahub.com/General/Lux/stable/
- License: MIT
-
Latest release: 1.21.0
published 6 months ago
Rankings
juliahub.com: MLDataDevices
- Documentation: https://docs.juliahub.com/General/MLDataDevices/stable/
- License: MIT
-
Latest release: 1.11.2
published 6 months ago
Rankings
juliahub.com: LuxCore
Elegant and Performant Deep Learning
- Homepage: https://lux.csail.mit.edu/
- Documentation: https://docs.juliahub.com/General/LuxCore/stable/
- License: MIT
-
Latest release: 1.4.0
published 7 months ago
Rankings
juliahub.com: LuxLib
- Documentation: https://docs.juliahub.com/General/LuxLib/stable/
- License: MIT
-
Latest release: 1.11.0
published 6 months ago
Rankings
juliahub.com: WeightInitializers
Elegant and Performant Deep Learning
- Homepage: https://lux.csail.mit.edu/
- Documentation: https://docs.juliahub.com/General/WeightInitializers/stable/
- License: MIT
-
Latest release: 1.2.0
published 7 months ago
Rankings
juliahub.com: LuxTestUtils
Elegant and Performant Deep Learning
- Homepage: https://lux.csail.mit.edu/
- Documentation: https://docs.juliahub.com/General/LuxTestUtils/stable/
- License: MIT
-
Latest release: 2.0.1
published 6 months ago
Rankings
Dependencies
- actions/cache v1 composite
- actions/checkout v2 composite
- codecov/codecov-action v3 composite
- julia-actions/julia-buildpkg v1 composite
- julia-actions/julia-processcoverage v1 composite
- julia-actions/julia-runtest v1 composite
- julia-actions/setup-julia v1 composite
- actions/cache v1 composite
- actions/checkout v2 composite
- codecov/codecov-action v3 composite
- julia-actions/julia-buildpkg v1 composite
- julia-actions/julia-processcoverage v1 composite
- julia-actions/julia-runtest v1 composite
- julia-actions/setup-julia v1 composite
- actions/checkout v2 composite
- actions/cache v1 composite
- actions/checkout v2 composite
- codecov/codecov-action v3 composite
- julia-actions/julia-processcoverage v1 composite
- julia-actions/setup-julia v1 composite
- actions/checkout v2 composite
- codecov/codecov-action v3 composite
- julia-actions/julia-buildpkg v1 composite
- julia-actions/julia-processcoverage v1 composite
- julia-actions/setup-julia v1 composite
- actions/checkout v1 composite
- julia-actions/setup-julia latest composite
- actions/checkout v2 composite
- peter-evans/create-pull-request v3 composite
- actions/checkout v3 composite
- julia-actions/julia-buildpkg v1 composite
- julia-actions/julia-invalidations v1 composite
- julia-actions/setup-julia v1 composite
- JuliaRegistries/TagBot v1 composite