https://github.com/darsnack/zygote.jl
Intimate Affection Auditor
Science Score: 28.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (12.2%) to scientific vocabulary
Repository
Intimate Affection Auditor
Basic Info
- Host: GitHub
- Owner: darsnack
- License: other
- Language: Julia
- Default Branch: master
- Homepage: https://fluxml.ai/Zygote.jl/
- Size: 2.99 MB
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
] add Zygote
Zygote provides source-to-source automatic differentiation (AD) in Julia, and is the next-gen AD system for the Flux differentiable programming framework. For more details and benchmarks of Zygote's technique, see our paper. You may want to check out Flux for more interesting examples of Zygote usage; the documentation here focuses on internals and advanced AD usage.
Zygote supports Julia 1.0 onwards, but we highly recommend using Julia 1.3 or later.
```julia julia> using Zygote
julia> f(x) = 5x + 3
julia> f(10), f'(10) (53, 5.0)
julia> @codellvm f'(10) define i64 @"julia#625_38792"(i64) { top: ret i64 5 } ```
"Source-to-source" means that Zygote hooks into Julia's compiler, and generates the backwards pass for you – as if you had written it by hand.
Zygote supports the flexibility and dynamism of the Julia language, including control flow, recursion, closures, structs, dictionaries, and more. Mutation and exception handling are currently not supported.
```julia julia> fs = Dict("sin" => sin, "cos" => cos, "tan" => tan);
julia> gradient(x -> fsreadline(), 1) sin 0.5403023058681398 ```
Zygote benefits from using the ChainRules.jl ruleset.
Custom gradients can be defined by extending the ChainRulesCore.jl's rrule:
```julia julia> using ChainRulesCore
julia> add(a, b) = a + b
julia> function ChainRulesCore.rrule(::typeof(add), a, b) addpb(dy) = (NoTangent(), dy, dy) return add(a, b), addpb end ```
To support large machine learning models with many parameters, Zygote can differentiate implicitly-used parameters, as opposed to just function arguments.
```julia julia> W, b = rand(2, 3), rand(2);
julia> predict(x) = W*x .+ b;
julia> g = gradient(Params([W, b])) do sum(predict([1,2,3])) end Grads(...)
julia> g[W], gb ```
Owner
- Name: Kyle Daruwalla
- Login: darsnack
- Kind: user
- Location: Cold Spring Harbor Lab, NY
- Website: darsnack.github.io
- Repositories: 67
- Profile: https://github.com/darsnack
NeuroAI scholar at CSHL
Citation (CITATION.bib)
@article{Zygote.jl-2018,
author = {Michael Innes},
title = {Don't Unroll Adjoint: Differentiating SSA-Form Programs},
journal = {CoRR},
volume = {abs/1810.07951},
year = {2018},
url = {http://arxiv.org/abs/1810.07951},
archivePrefix = {arXiv},
eprint = {1810.07951},
timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1810-07951},
bibsource = {dblp computer science bibliography, https://dblp.org}
}