Lux

Elegant and Performant Deep Learning

https://github.com/luxdl/lux.jl

Science Score: 65.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
    Found 5 DOI reference(s) in README
  • Academic publication links
  • Academic email domains
  • Institutional organization owner
    Organization luxdl has institutional domain (lux.csail.mit.edu)
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (11.8%) to scientific vocabulary

Keywords

deep-learning gpu machine-learning neural-networks scientific-machine-learning tpu xla
Last synced: 6 months ago · JSON representation ·

Repository

Elegant and Performant Deep Learning

Basic Info
Statistics
  • Stars: 609
  • Watchers: 11
  • Forks: 74
  • Open Issues: 73
  • Releases: 241
Topics
deep-learning gpu machine-learning neural-networks scientific-machine-learning tpu xla
Created almost 4 years ago · Last pushed 6 months ago
Metadata Files
Readme License Citation

README.md

[![GitHub Discussions](https://img.shields.io/github/discussions/LuxDL/Lux.jl?color=white&logo=github&label=Discussions)](https://github.com/LuxDL/Lux.jl/discussions) [![Latest Docs](https://img.shields.io/badge/docs-latest-blue.svg)](http://lux.csail.mit.edu/dev/) [![Stable Docs](https://img.shields.io/badge/docs-stable-blue.svg)](http://lux.csail.mit.edu/stable/) [![CI](https://github.com/LuxDL/Lux.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/LuxDL/Lux.jl/actions/workflows/CI.yml) [![CI (pre-release)]()](https://github.com/LuxDL/Lux.jl/actions/workflows/CIPreRelease.yml) [![Build status](https://img.shields.io/buildkite/ba1f9622add5978c2d7b194563fd9327113c9c21e5734be20e/main.svg?label=gpu&branch=main&logo=buildkite)](https://buildkite.com/julialang/lux-dot-jl) [![codecov](https://codecov.io/gh/LuxDL/Lux.jl/branch/main/graph/badge.svg?token=IMqBM1e3hz)](https://codecov.io/gh/LuxDL/Lux.jl) [![Benchmarks](https://github.com/LuxDL/Lux.jl/actions/workflows/Benchmark.yml/badge.svg?branch=main)](https://lux.csail.mit.edu/benchmarks/) [![Downloads](https://img.shields.io/badge/dynamic/json?url=http%3A%2F%2Fjuliapkgstats.com%2Fapi%2Fv1%2Fmonthly_downloads%2FLux&query=total_requests&suffix=%2Fmonth&label=Downloads)](https://juliapkgstats.com/pkg/Lux) [![Downloads](https://img.shields.io/badge/dynamic/json?url=http%3A%2F%2Fjuliapkgstats.com%2Fapi%2Fv1%2Ftotal_downloads%2FLux&query=total_requests&&label=Total%20Downloads)](https://juliapkgstats.com/pkg/Lux) [![JET Testing](https://img.shields.io/badge/%F0%9F%9B%A9%EF%B8%8F_tested_with-JET.jl-233f9a)](https://github.com/aviatesk/JET.jl) [![Aqua QA](https://raw.githubusercontent.com/JuliaTesting/Aqua.jl/master/badge.svg)](https://github.com/JuliaTesting/Aqua.jl) [![ColPrac: Contributor's Guide on Collaborative Practices for Community Packages](https://img.shields.io/badge/ColPrac-Contributor's%20Guide-blueviolet)](https://github.com/SciML/ColPrac) [![Code Style: Blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/JuliaDiff/BlueStyle)

Elegant & Performant Deep Learning in JuliaLang

Model with the elegance of Julia, and the performance of XLA.

💻 Installation

julia import Pkg Pkg.add("Lux")

[!TIP] To use Lux online, use Google Colab. The Julia Runtime comes pre-installed with Lux and Reactant!

| **Packages** | **Stable Version** | **Monthly Downloads** | **Total Downloads** | **Build Status** | | :----------------------------------------------------- | :------------------------------------------------------------- | :-------------------------------------------------------------------- | :-------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------- | | 📦 [Lux.jl](./src) | [![][lux-version]][lux-juliahub] | [![][downloads-lux]][downloads-lux-url] | [![][total-downloads-lux]][downloads-lux-url] | [![][gh-actions-lux]][gh-actions-lux-url] [![][gh-actions-lux-prerelease]][gh-actions-lux-prerelease-url] [![][buildkite-badge]][buildkite-url] | | └ 📦 [LuxLib.jl](./lib/LuxLib) | [![][luxlib-version]][luxlib-juliahub] | [![][downloads-luxlib]][downloads-luxlib-url] | [![][total-downloads-luxlib]][downloads-luxlib-url] | [![][gh-actions-luxlib]][gh-actions-luxlib-url] | | └ 📦 [LuxCore.jl](./lib/LuxCore) | [![][luxcore-version]][luxcore-juliahub] | [![][downloads-luxcore]][downloads-luxcore-url] | [![][total-downloads-luxcore]][downloads-luxcore-url] | [![][gh-actions-luxcore]][gh-actions-luxcore-url] | | └ 📦 [MLDataDevices.jl](./lib/MLDataDevices) | [![][mldatadevices-version]][mldatadevices-juliahub] | [![][downloads-mldatadevices]][downloads-mldatadevices-url] | [![][total-downloads-mldatadevices]][downloads-mldatadevices-url] | [![][gh-actions-mldatadevices]][gh-actions-mldatadevices-url] | | └ 📦 [WeightInitializers.jl](./lib/WeightInitializers) | [![][weightinitializers-version]][weightinitializers-juliahub] | [![][downloads-weightinitializers]][downloads-weightinitializers-url] | [![][total-downloads-weightinitializers]][downloads-weightinitializers-url] | [![][gh-actions-weightinitializers]][gh-actions-weightinitializers-url] | | └ 📦 [LuxTestUtils.jl](./lib/LuxTestUtils) | [![][luxtestutils-version]][luxtestutils-juliahub] | [![][downloads-luxtestutils]][downloads-luxtestutils-url] | [![][total-downloads-luxtestutils]][downloads-luxtestutils-url] | [![][gh-actions-luxtestutils]][gh-actions-luxtestutils-url] | | └ 📦 [LuxCUDA.jl](./lib/LuxCUDA) | [![][luxcuda-version]][luxcuda-juliahub] | [![][downloads-luxcuda]][downloads-luxcuda-url] | [![][total-downloads-luxcuda]][downloads-luxcuda-url] | [![][gh-actions-luxcuda]][gh-actions-luxcuda-url] |

🤸 Quickstart

Reactant & Enzyme

```julia using Lux, Random, Optimisers, Reactant, Enzyme

rng = Random.default_rng() Random.seed!(rng, 0)

model = Chain(Dense(128, 256, tanh), Chain(Dense(256, 1, tanh), Dense(1, 10)))

dev = reactant_device()

ps, st = Lux.setup(rng, model) |> dev

x = rand(rng, Float32, 128, 2) |> dev

We need to compile the model before we can use it.

modelforward = @compile model(x, ps, Lux.testmode(st)) modelforward(x, ps, Lux.testmode(st))

Gradients can be computed using Enzyme

@jit Enzyme.gradient(Reverse, sum ∘ first ∘ Lux.apply, Const(model), x, ps, Const(st))

All of this can be automated using the TrainState API

train_state = Training.TrainState(model, ps, st, Adam(0.001f0))

gs, loss, stats, trainstate = Training.singletrainstep!( AutoEnzyme(), MSELoss(), (x, dev(rand(rng, Float32, 10, 2))), trainstate ) ```

Native Julia & Zygote

```julia using Lux, Random, Optimisers, Zygote

using LuxCUDA, AMDGPU, Metal, oneAPI # Optional packages for GPU support

Seeding

rng = Random.default_rng() Random.seed!(rng, 0)

Construct the layer

model = Chain(Dense(128, 256, tanh), Chain(Dense(256, 1, tanh), Dense(1, 10)))

Get the device determined by Lux

dev = gpu_device()

Parameter and State Variables

ps, st = Lux.setup(rng, model) |> dev

Dummy Input

x = rand(rng, Float32, 128, 2) |> dev

Run the model

y, st = Lux.apply(model, x, ps, st)

Gradients

First construct a TrainState

train_state = Lux.Training.TrainState(model, ps, st, Adam(0.0001f0))

We can compute the gradients using Training.compute_gradients

gs, loss, stats, trainstate = Lux.Training.computegradients(AutoZygote(), MSELoss(), (x, dev(rand(rng, Float32, 10, 2))), train_state)

Optimization

trainstate = Training.applygradients!(trainstate, gs) # or Training.applygradients (no ! at the end)

Both these steps can be combined into a single call

gs, loss, stats, trainstate = Training.singletrainstep!(AutoZygote(), MSELoss(), (x, dev(rand(rng, Float32, 10, 2))), trainstate) ```

📚 Examples

Look in the examples directory for self-contained usage examples. The documentation has examples sorted into proper categories.

🆘 Getting Help

For usage related questions, please use Github Discussions which allows questions and answers to be indexed. To report bugs use github issues or even better send in a pull request.

🧑‍🔬 Citation

If you found this library to be useful in academic work, then please cite:

```bibtex @software{pal2023lux, author = {Pal, Avik}, title = {{Lux: Explicit Parameterization of Deep Neural Networks in Julia}}, month = apr, year = 2023, note = {If you use this software, please cite it as below.}, publisher = {Zenodo}, version = {v1.4.2}, doi = {10.5281/zenodo.7808903}, url = {https://doi.org/10.5281/zenodo.7808903}, swhid = {swh:1:dir:1a304ec3243961314a1cc7c1481a31c4386c4a34;origin=https://doi.org/10.5281/zenodo.7808903;visit=swh:1:snp:e2bbe43b14bde47c4ddf7e637eb7fc7bd10db8c7;anchor=swh:1:rel:2c0c0ff927e7bfe8fc8bc43fd553ab392a6eb403;path=/} }

@thesis{pal2023efficient, title = {{On Efficient Training & Inference of Neural Differential Equations}}, author = {Pal, Avik}, year = {2023}, school = {Massachusetts Institute of Technology} } ```

Also consider starring our github repo.

🧑‍💻 Contributing

This section is somewhat incomplete. You can contribute by contributing to finishing this section 😜.

💎 Formatting (JuliaFormatter)

[!NOTE] Pin JuliaFormatter to v1 until upstream issues with v2 are resolved.

julia using JuliaFormatter format(".")

🧪 Testing

The full test of Lux.jl takes a long time, here's how to test a portion of the code.

For each @testitem, there are corresponding tags, for example:

julia @testitem "SkipConnection" setup=[SharedTestSetup] tags=[:core_layers]

For example, let's consider the tests for SkipConnection:

julia @testitem "SkipConnection" setup=[SharedTestSetup] tags=[:core_layers] begin ... end

We can test the group to which SkipConnection belongs by testing core_layers. To do so set the LUX_TEST_GROUP environment variable, or rename the tag to further narrow the test scope:

shell export LUX_TEST_GROUP="core_layers"

Or directly modify the default test tag in runtests.jl:

```julia

const LUXTESTGROUP = lowercase(get(ENV, "LUXTESTGROUP", "all"))

const LUXTESTGROUP = lowercase(get(ENV, "LUXTESTGROUP", "core_layers")) ```

But be sure to restore the default value "all" before submitting the code.

Furthermore if you want to run a specific test based on the name of the testset, you can use TestEnv.jl as follows. Start with activating the Lux environment and then run the following:

```julia using TestEnv; TestEnv.activate(); using ReTestItems;

Assuming you are in the main directory of Lux

ReTestItems.runtests("tests/"; name = "NAME OF THE TEST") ```

For the SkipConnection tests that would be:

julia ReTestItems.runtests("tests/"; name = "SkipConnection")

Owner

  • Name: LuxDL
  • Login: LuxDL
  • Kind: organization
  • Location: United States of America

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Pal"
  given-names: "Avik"
  orcid: "https://orcid.org/0000-0002-3938-7375"
title: "Lux: Explicit Parameterization of Deep Neural Networks in Julia"
version: 1.4.2
date-released: 2022-05-10
url: "https://github.com/LuxDL/Lux.jl"

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 237
  • Total pull requests: 669
  • Average time to close issues: about 2 months
  • Average time to close pull requests: 3 days
  • Total issue authors: 73
  • Total pull request authors: 42
  • Average comments per issue: 3.52
  • Average comments per pull request: 0.75
  • Merged pull requests: 438
  • Bot issues: 4
  • Bot pull requests: 266
Past Year
  • Issues: 113
  • Pull requests: 380
  • Average time to close issues: 10 days
  • Average time to close pull requests: 2 days
  • Issue authors: 45
  • Pull request authors: 30
  • Average comments per issue: 1.08
  • Average comments per pull request: 0.6
  • Merged pull requests: 221
  • Bot issues: 3
  • Bot pull requests: 171
Top Authors
Issue Authors
  • avik-pal (106)
  • schlichtanders (9)
  • NeroBlackstone (7)
  • prbzrg (6)
  • ExpandingMan (5)
  • bertini97 (5)
  • gdalle (5)
  • CarloLucibello (4)
  • github-actions[bot] (4)
  • vpuri3 (4)
  • chooron (4)
  • CatYukino (2)
  • arthur-bizzi (2)
  • ablaom (2)
  • MartinuzziFrancesco (2)
Pull Request Authors
  • avik-pal (321)
  • github-actions[bot] (213)
  • dependabot[bot] (53)
  • abhro (11)
  • NeroBlackstone (11)
  • Sleort (4)
  • wsmoses (4)
  • MartinuzziFrancesco (4)
  • ldeso (3)
  • CarloLucibello (3)
  • Copilot (3)
  • agdestein (3)
  • asinghvi17 (3)
  • prbzrg (2)
  • simeonschaub (2)
Top Labels
Issue Labels
reactant (24) bug (21) enhancement (18) documentation (13) good first issue (9) autodiff (8) high-priority (6) performance (6) breaking (6) enzyme (6) nested-ad (4) upstream (3) mldatadevices (2) weightinitializers (1) luxlib (1) speculative (1) metal (1) oneapi (1) amdgpu (1) xla (1) tracker (1) type-stability (1) component-arrays (1) cuda (1) help wanted (1)
Pull Request Labels
dependencies (53) formatting (11) automated pr (11) no changelog (11) github_actions (10) reactant (8) enzyme (3) xla (3) autodiff (2) mldatadevices (2) needs more information (1) enhancement (1) upstream (1) run benchmarks (1) run downstream test (1) needs test (1) documentation (1)

Packages

  • Total packages: 6
  • Total downloads:
    • julia 9,059 total
  • Total dependent packages: 31
    (may contain duplicates)
  • Total dependent repositories: 0
    (may contain duplicates)
  • Total versions: 432
juliahub.com: Lux
  • Versions: 186
  • Dependent Packages: 18
  • Dependent Repositories: 0
  • Downloads: 1,567 Total
Rankings
Stargazers count: 2.5%
Dependent packages count: 5.8%
Average: 6.4%
Forks count: 7.5%
Dependent repos count: 9.9%
Last synced: 6 months ago
juliahub.com: MLDataDevices
  • Versions: 39
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 3,105 Total
Rankings
Downloads: 2.0%
Dependent repos count: 3.2%
Average: 7.2%
Dependent packages count: 16.3%
Last synced: 6 months ago
juliahub.com: LuxCore

Elegant and Performant Deep Learning

  • Versions: 38
  • Dependent Packages: 10
  • Dependent Repositories: 0
  • Downloads: 1,509 Total
Rankings
Stargazers count: 2.5%
Forks count: 7.5%
Dependent repos count: 9.9%
Average: 10.7%
Dependent packages count: 23.0%
Last synced: 6 months ago
juliahub.com: LuxLib
  • Versions: 108
  • Dependent Packages: 1
  • Dependent Repositories: 0
  • Downloads: 1,478 Total
Rankings
Stargazers count: 2.5%
Forks count: 7.5%
Dependent repos count: 9.9%
Average: 10.7%
Dependent packages count: 23.0%
Last synced: 6 months ago
juliahub.com: WeightInitializers

Elegant and Performant Deep Learning

  • Versions: 23
  • Dependent Packages: 2
  • Dependent Repositories: 0
  • Downloads: 1,399 Total
Rankings
Dependent repos count: 9.8%
Average: 36.5%
Dependent packages count: 38.3%
Stargazers count: 43.4%
Forks count: 54.6%
Last synced: 6 months ago
juliahub.com: LuxTestUtils

Elegant and Performant Deep Learning

  • Versions: 38
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 1 Total
Rankings
Dependent repos count: 9.9%
Dependent packages count: 39.2%
Average: 45.3%
Forks count: 55.8%
Stargazers count: 76.1%
Last synced: 6 months ago

Dependencies

.github/workflows/CI.yml actions
  • actions/cache v1 composite
  • actions/checkout v2 composite
  • codecov/codecov-action v3 composite
  • julia-actions/julia-buildpkg v1 composite
  • julia-actions/julia-processcoverage v1 composite
  • julia-actions/julia-runtest v1 composite
  • julia-actions/setup-julia v1 composite
.github/workflows/CINightly.yml actions
  • actions/cache v1 composite
  • actions/checkout v2 composite
  • codecov/codecov-action v3 composite
  • julia-actions/julia-buildpkg v1 composite
  • julia-actions/julia-processcoverage v1 composite
  • julia-actions/julia-runtest v1 composite
  • julia-actions/setup-julia v1 composite
.github/workflows/DocCleanUp.yml actions
  • actions/checkout v2 composite
.github/workflows/Documentation.yml actions
  • actions/cache v1 composite
  • actions/checkout v2 composite
  • codecov/codecov-action v3 composite
  • julia-actions/julia-processcoverage v1 composite
  • julia-actions/setup-julia v1 composite
.github/workflows/Downstream.yml actions
  • actions/checkout v2 composite
  • codecov/codecov-action v3 composite
  • julia-actions/julia-buildpkg v1 composite
  • julia-actions/julia-processcoverage v1 composite
  • julia-actions/setup-julia v1 composite
.github/workflows/FormatCheck.yml actions
  • actions/checkout v1 composite
  • julia-actions/setup-julia latest composite
.github/workflows/FormatPR.yml actions
  • actions/checkout v2 composite
  • peter-evans/create-pull-request v3 composite
.github/workflows/Invalidations.yml actions
  • actions/checkout v3 composite
  • julia-actions/julia-buildpkg v1 composite
  • julia-actions/julia-invalidations v1 composite
  • julia-actions/setup-julia v1 composite
.github/workflows/TagBot.yml actions
  • JuliaRegistries/TagBot v1 composite