Recent Releases of Flux
Flux - v0.16.5
Flux v0.16.5
Merged pull requests: - Fix Typos in Old Tutorials Documentation (#2610) (@leopardracer) - CompatHelper: bump compat for AMDGPU in [weakdeps] to 2, (keep existing compat) (#2613) (@github-actions[bot]) - Bump to 0.16.5 (#2614) (@pxl-th)
Closed issues:
- unsafe_free! from MLDataDevices fails for CuArray{CartesianIndex{4}, 1, CUDA.DeviceMemory}) (#2612)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] 11 months ago
Flux - v0.16.4
Flux v0.16.4
Merged pull requests: - Fix missing imports in FluxMPIExt (#2589) (@Alexander-Barth) - Added shape validation for Conv weight tensor (#2590) (@mikymatt01) - Disable broken Reactant tests for now (#2595) (@ToucheSir) - Test re enable reactant test to identify failures (#2596) (@wsmoses) - fix recurrent docstrings (#2597) (@CarloLucibello) - Fix loading of pooling layers (#2598) (@adrhill) - Fix typos (#2601) (@omahs) - small updates to tests (#2602) (@CarloLucibello) - reinstate enzyme tests (#2603) (@CarloLucibello) - fix docs CI (#2604) (@CarloLucibello) - fix piracy with DataLoader (#2608) (@CarloLucibello)
Closed issues:
- Reduce hcat creates dense matrix (#1596)
- Update GSoC 2025 Idea List (#2586)
- Type piracy breaks (dev::AbstractDevice)(d::DataLoader) (#2592)
- Dropout erroring on CUDA, when using cu but not gpu (#2594)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 1 year ago
Flux - v0.16.2
Flux v0.16.2
Merged pull requests: - Update deps & bump to 0.16.1 (#2574) (@pxl-th)
Closed issues: - New Gradients ruin everything (#2580) - Failure to precompile on 1.12: cannot declare Flux.destructure public; it is already declared exported (#2583)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 1 year ago
Flux - v0.16.1
Flux v0.16.1
Merged pull requests: - Adding RecurrentLayers to ecosystem.md (#2555) (@MartinuzziFrancesco) - Fixed typo in recurrence documentation (#2556) (@MartinuzziFrancesco) - Adding return state option to recurrent layers (#2557) (@MartinuzziFrancesco) - update Schedulers docs (#2560) (@CarloLucibello) - collapse doc string in layers docs (#2562) (@CarloLucibello) - fix test enzyme (#2563) (@CarloLucibello) - Remove 2 items from public, to fix 1.12 (#2569) (@mcabbott) - Add reactant forward and reverse pass tests (#2576) (@wsmoses) - cleanup Reactant and Enzyme tests (#2578) (@CarloLucibello)
Closed issues:
- cell output is not clearly distinguishable from the state (#2548)
- Flux.cpu and Flux.gpu no longer move data on views (#2553)
- remove usage example of old optimiser (#2558)
- Optimizing over AbstractMatrix subtypes (#2559)
- introduce a FlattenLayer (#2561)
- [enzyme] broken MeanPool gradient (#2564)
- [enzyme] broken BatchNorm gradient (#2566)
- [enyzme] broken recurrent cell loss (#2568)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 1 year ago
Flux - v0.16.0
Flux v0.16.0
Highlights
This release has a single breaking change:
- The recurrent cells RNNCell, LSTMCell, and GRUCell forward has been changed to
$yt, statet = cell(xt, state{t-1})$ in (#2551). Previously, it was $statet = cell(xt, state_{t-1})$.
Other highlights include:
* Added WeightNorm normalization layer.
* Added Recurrence layer, turning a recurrent layer into a layer processing the entire sequence at once.
Merged pull requests:
- Recurrence layer (#2549) (@CarloLucibello)
- Add WeightNorm reparametrization (#2550) (@pxl-th)
- Change cells' return to out, state (#2551) (@CarloLucibello)
- fix: gpu_device not defined in Flux.DistributedUtils (#2552) (@AntonOresten)
Closed issues: - Feature request: Weight normalization (#942) - recurrent dropout (#1040) - Stacked RNN in Flux.jl? (#2452)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 1 year ago
Flux - v0.15.1
Flux v0.15.1
Merged pull requests:
- Re-write "basics" page of docs (#2535) (@mcabbott)
- Adding initialstates function to RNNs (#2541) (@MartinuzziFrancesco)
- Update NEWS.md highlighting breaking changes (#2542) (@CarloLucibello)
- relax identity test for devices (#2544) (@CarloLucibello)
- fix Flux.@functor (#2546) (@CarloLucibello)
Closed issues:
- Flux.@functor is broken on 0.15 (#2545)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 1 year ago
Flux - v0.15.0
Flux v0.15.0
Highlights
This release includes two breaking changes:
- The recurrent layers have been thoroughly revised. See below and read the documentation for details.
- Flux now defines and exports its own gradient function. Consequently, using gradient in an unqualified manner (e.g., after using Flux, Zygote) could result in an ambiguity error.
The most significant updates and deprecations are as follows:
* Recurrent layers have undergone a complete redesign in PR 2500.
* RNNCell, LSTMCell, and GRUCell are now exported and provide functionality for single time-step processing: rnncell(x_t, h_t) -> h_{t+1}.
* RNN, LSTM, and GRU no longer store the hidden state internally, it has to be explicitely passed to the layer. Moreover, they now process entire sequences at once, rather than one element at a time: rnn(x, h) -> h′.
* The Recur wrapper has been deprecated and removed.
* The reset! function has also been removed; state management is now entirely up to the user.
* The Flux.Optimise module has been deprecated in favor of the Optimisers.jl package.
Now Flux re-exports the optimisers from Optimisers.jl. Most users will be uneffected by this change.
The module is still available for now, but will be removed in a future release.
* Most Flux layers will re-use memory via NNlib.bias_act!, when possible.
* Further support for Enzyme.jl, via methods of Flux.gradient(loss, Duplicated(model)).
Flux now owns & exports gradient and withgradient, but without Duplicated this still defaults to calling Zygote.jl.
* Flux.params has been deprecated. Use Zygote's explicit differentiation instead,
gradient(m -> loss(m, x, y), model), or use Flux.trainables(model) to get the trainable parameters.
* Flux now requires Functors.jl v0.5. This new release of Functors assumes all types to be functors by default. Therefore, applying Flux.@layer or Functors.@functor to a type is no longer strictly necessary for Flux's models. However, it is still recommended to use @layer Model for additional functionality like pretty printing.
* @layer Modelnow behaves the same as @layer :expand Model, which means that the model is expanded into its sublayers (if there are any) when printed. To force compact printing, use @layer :noexpand Model.
Merged pull requests:
- Use NNlib.bias_act! (#2327) (@mcabbott)
- Allow Parallel(+, f)(x, y, z) to work like broadcasting, and enable Chain(identity, Parallel(+, f))(x, y, z) (#2393) (@mcabbott)
- Epsilon change in normalise for stability (#2421) (@billera)
- Add more Duplicated methods for Enzyme.jl support (#2471) (@mcabbott)
- Export Optimisers and remove params and Optimise from tests (#2495) (@CarloLucibello)
- RNNs redesign (#2500) (@CarloLucibello)
- Adjust docs & Flux.@functor for Functors.jl v0.5, plus misc. depwarns (#2509) (@mcabbott)
- GPU docs (#2510) (@mcabbott)
- CompatHelper: bump compat for Optimisers to 0.4, (keep existing compat) (#2520) (@github-actions[bot])
- Distinct init for kernel and recurrent (#2522) (@MartinuzziFrancesco)
- Functors v0.5 + tighter version bounds (#2525) (@CarloLucibello)
- deprecation of params and Optimise (continued) (#2526) (@CarloLucibello)
- Bump codecov/codecov-action from 4 to 5 (#2527) (@dependabot[bot])
- updates for Functors v0.5 (#2528) (@CarloLucibello)
- fix comment (#2529) (@oscardssmith)
- set expand option as default for @layer (#2532) (@CarloLucibello)
- misc stuff for v0.15 release (#2534) (@CarloLucibello)
- Tweak quickstart.md (#2536) (@mcabbott)
- Remove usage of global variables in linear and logistic regression tutorial training functions (#2537) (@christiangnrd)
- Fix linear regression example (#2538) (@christiangnrd)
- Update gpu.md (#2539) (@AdamWysokinski)
Closed issues:
- RNN layer to skip certain time steps (like Masking layer in keras) (#644)
- Backprop through time (#648)
- Initial state in RNNs should not be learnable by default (#807)
- Bad recurrent layers training performance (#980)
- flip function assumes the input sequence is a Vector or List, it can be Matrix as well. (#1042)
- Regression in package load time (#1155)
- Recurrent layers can't use Zeros() as bias (#1279)
- Flux.destructure doesn't preserve RNN state (#1329)
- RNN design for efficient CUDNN usage (#1365)
- Strange result with gradient (#1547)
- Call of Flux.stack results in StackOverfloxError for approx. 6000 sequence elements of a model output of a LSTM (#1585)
- Gradient dimension mismatch error when training rnns (#1891)
- Deprecate Flux.Optimisers and implicit parameters in favour of Optimisers.jl and explicit parameters (#1986)
- Pull request #2007 causes Flux.params() calls to not get cached (#2040)
- gradient of Flux.normalise return NaN when std is zero (#2096)
- explicit differentiation for RNN gives wrong results (#2185)
- Make RNNs blocked (and maybe fixing gradients along the way) (#2258)
- Should everything be a functor by default? (#2269)
- Flux new explicit API does not work but old implicit API works for a simple RNN (#2341)
- Adding Simple Recurrent Unit as a recurrent layer (#2408)
- deprecate Flux.params (#2413)
- Implementation of AdamW differs from PyTorch (#2433)
- gpu should warn if cuDNN is not installed (#2440)
- device movement behavior inconsistent (#2513)
- mark as public any non-exported but documented interface (#2518)
- broken image in the quickstart (#2530)
- Consider making the :expand option the default in @layer (#2531)
- Flux.params is broken (#2533)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 1 year ago
Flux - v0.14.23
Flux v0.14.23
Merged pull requests:
- Support for lecun normal weight initialization (#2311) (@RohitRathore1)
- Some small printing upgrades (#2344) (@mcabbott)
- simplify test machinery (#2498) (@CarloLucibello)
- Correct dead link for "quickstart page" in README.md (#2499) (@zengmao)
- make gpu(x) = gpu_device()(x) (#2502) (@CarloLucibello)
- some cleanup (#2503) (@CarloLucibello)
- unbreak some data movement cuda tests (#2504) (@CarloLucibello)
Closed issues:
- Add support for lecun normal weight initialization (#2290)
- using Flux, cuDNN freezes, but using Flux, CUDA, cuDNN works (#2346)
- Problem with RNN and CUDA. (#2352)
- since new version: Flux throws error when for train! / update! even on quick start problem (#2358)
- Cannot take gradient of L2 regularization loss (#2441)
- Potential bug of RNN training flow (#2455)
- Problem with documentation (#2485)
- Flux has no Lecun Normalization weight init function? (#2491)
- Zygote fails to differentiate through Flux.params on julia v0.11 (#2497)
- ERROR: UndefVarError: ADAM not defined in Main in flux (#2507)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 1 year ago
Flux - v0.14.22
Flux v0.14.22
Merged pull requests: - Bump actions/checkout from 4.2.0 to 4.2.1 (#2489) (@dependabot[bot]) - handle data movement with MLDataDevices.jl (#2492) (@CarloLucibello) - remove some v0.13 deprecations (#2493) (@CarloLucibello)
Closed issues:
- use MLDataDevices.jl? (#2482)
- The dependency error about Flux->FluxMPIExt occurs when updating to Julia 1.11 (#2490)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 1 year ago
Flux - v0.14.21
Flux v0.14.21
Merged pull requests: - Update ci.yml for macos-latest to use aarch64 (#2481) (@ViralBShah) - Remove leading empty line in example (#2486) (@blegat) - Bump actions/checkout from 4.1.7 to 4.2.0 (#2487) (@dependabot[bot]) - fix: CUDA package optional for FluxMPIExt (#2488) (@askorupka)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 1 year ago
Flux - v0.14.20
Flux v0.14.20
Merged pull requests: - feat: Distributed data parallel training support (#2464) (@askorupka) - Run Enzyme tests only on CUDA CI machine (#2478) (@pxl-th) - Adapt to pending Enzyme breaking change (#2479) (@wsmoses) - Update TagBot.yml (#2480) (@ViralBShah) - Bump patch version (#2483) (@wsmoses)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 1 year ago
Flux - v0.14.19
Flux v0.14.19
Merged pull requests:
- Allow loading of ConvTranspose state without .outpad field (#2477) (@mcabbott)
Closed issues: - Model saved under Flux v0.14.16 does not load on v0.14.17 (#2476)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 2 years ago
Flux - v0.14.17
Flux v0.14.17
Merged pull requests:
- Add Enzyme train function (#2446) (@wsmoses)
- Bump actions/checkout from 4.1.5 to 4.1.7 (#2460) (@dependabot[bot])
- Add output padding for ConvTranspose (#2462) (@guiyrt)
- Fix ConvTranspose symmetric non-constant padding (#2463) (@paulnovo)
- CompatHelper: add new compat entry for Enzyme at version 0.12, (keep existing compat) (#2466) (@github-actions[bot])
- move enzyme to extension (#2467) (@CarloLucibello)
- Fix function _size_check() (#2472) (@gruberchr)
- Fix ConvTranspose output padding on AMDGPU (#2473) (@paulnovo)
Closed issues: - Hoping to offer a version without cuda (#2155) - ConvTranspose errors with symmetric non-constant pad (#2424) - Create a flag to use Enzyme as the AD in training/etc. (#2443) - Can't load a Fluxml trained & saved model. Getting ERROR: CUDA error: invalid device context (code 201, ERRORINVALIDCONTEXT) (#2461) - Requires deprecated cuNN.jl package (#2470)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 2 years ago
Flux - v0.14.16
Flux v0.14.16
Merged pull requests:
- Make sure first example in Custom Layers docs uses type parameter (#2415) (@BioTurboNick)
- Add GPU GC comment to Performance Tips (#2416) (@BioTurboNick)
- Fix some typos in docs (#2418) (@JoshuaLampert)
- fix component arrays test (#2419) (@CarloLucibello)
- Bump julia-actions/setup-julia from 1 to 2 (#2420) (@dependabot[bot])
- documentation update (#2422) (@CarloLucibello)
- remove public dropout (#2423) (@mcabbott)
- Allow BatchNorm on CUDA with trackstats=False (#2427) (@paulnovo)
- Bump actions/checkout from 4.1.2 to 4.1.3 (#2428) (@dependabot[bot])
- Add working downloads badge (#2429) (@pricklypointer)
- Bump actions/checkout from 4.1.3 to 4.1.4 (#2430) (@dependabot[bot])
- Add tip for non-CUDA users (#2434) (@micahscopes)
- Add hint for choosing a different GPU backend (#2435) (@micahscopes)
- Patch `Flux.isleaf` for abstract arrays with bitstype elements (#2436) (@jondeuce)
- Bump julia-actions/cache from 1 to 2 (#2437) (@dependabot[bot])
- Bump actions/checkout from 4.1.4 to 4.1.5 (#2438) (@dependabot[bot])
- Enzyme: bump version and mark models as working test (@wsmoses)
- Enable remaining enzyme test (#2442) (@wsmoses)
- Bump AMDGPU to 0.9 (#2449) (@pxl-th)
- Do not install all GPU backends at once (#2453) (@pxl-th)
- CompatHelper: add new compat entry for BSON at version 0.3, (keep existing compat) (#2457) (@github-actions[bot])
- remove BSON dependence (#2458) (@CarloLucibello)
Closed issues:
- How to have a stable GPU memory while being performant? (#780)
- Why is Flux.destructure type unstable? (#2405)
- tests are failing due to ComponentArrays (#2411)
- Significant time spent moving medium-size arrays to GPU, type instability (#2414)
- Dense layers with shared parameters (#2432)
- why is my withgradient type unstable ? (#2456)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 2 years ago
Flux - v0.14.15
Flux v0.14.15
Merged pull requests: - Restore some support for Tracker.jl (#2387) (@mcabbott) - start testing Enzyme (#2392) (@CarloLucibello) - Add Ignite.jl to ecosystem.md (#2395) (@mcabbott) - Bump actions/checkout from 4.1.1 to 4.1.2 (#2401) (@dependabot[bot]) - More lazy strings (#2402) (@lassepe) - Fix dead link in docs (#2403) (@BioTurboNick) - Improve errors for conv layers (#2404) (@mcabbott)
Closed issues:
- Given that DataLoader implements length shouldn't it also be able to provide size? (#2372)
- Dimensions check for Conv is incomplete, leading to confusing error (#2398)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 2 years ago
Flux - v0.14.14
Flux v0.14.14
Merged pull requests: - Bump actions/cache from 3 to 4 (#2371) (@dependabot[bot]) - Use LazyString in depwarn (#2400) (@mcabbott)
Closed issues: - precompilation issue on Julia 1.10 (#2354) - Flux installation error under Julia 1.10 on Apple Silicon (#2366) - Compilation time of Flux models (#2391) - Flux.setup buggy and broken in latest v.0.13.17 (#2394) - 2x performance regression due to 5e80211c3302b5e7b79b4f670498f5a68af6659b (#2399)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 2 years ago
Flux - v0.14.13
Flux v0.14.13
Merged pull requests:
- Add a macro to opt-in to fancy printing, and to everything else (#1932) (@mcabbott)
- Small upgrades to training docs (#2331) (@mcabbott)
- Bump codecov/codecov-action from 3 to 4 (#2376) (@dependabot[bot])
- Bump dorny/paths-filter from 3.0.0 to 3.0.1 (#2381) (@dependabot[bot])
- Bump thollander/actions-comment-pull-request from 2.4.3 to 2.5.0 (#2382) (@dependabot[bot])
- Fix https://github.com/FluxML/Flux.jl/issues/2380 (#2384) (@diegozea)
- Allow cpu(::DataLoader) (#2388) (@mcabbott)
- Bump dorny/paths-filter from 3.0.1 to 3.0.2 (#2389) (@dependabot[bot])
- doc changes re at-functor and at-layer (#2390) (@mcabbott)
Closed issues: - Macro to display model struct the way Flux does (#2044) - Update GH Actions across all repos (#2170) - Flux.jl Documentation (Training API Reference) (#2303) - Flux docs missing withgradient() call for multi-objective loss functions (#2325)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 2 years ago
Flux - v0.14.12
Flux v0.14.12
Merged pull requests: - Add SignDecay for L1 norm (#2377) (@mcabbott) - Update mlutils.md (#2379) (@diegozea)
Closed issues: - The dedicated tutorial on DataLoader is missing (#2378)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 2 years ago
Flux - v0.14.8
Flux v0.14.8
Merged pull requests: - Use stable API for AMDGPU RNG conversion (#2360) (@ToucheSir) - Non-diff shape handling in norm layers (#2363) (@ToucheSir)
Closed issues: - 0.2 tutorial at the top of google for "flux mnist" (#2163) - Illegal Memory Access Error During Gradient Calculation of predefined losses on GPU RTX 4050 (#2361) - Unnecessarily using shared GPU memory (#2364)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 2 years ago
Flux - v0.14.7
Flux v0.14.7
Merged pull requests:
- Bump actions/checkout from 4.0.0 to 4.1.0 (#2340) (@dependabot[bot])
- Use new public feature (#2342) (@mcabbott)
- Bump thollander/actions-comment-pull-request from 2.4.2 to 2.4.3 (#2347) (@dependabot[bot])
- Bump actions/checkout from 4.1.0 to 4.1.1 (#2348) (@dependabot[bot])
- Fix test that is not broken anymore (#2349) (@devmotion)
- CompatHelper: add new compat entry for Statistics at version 1, (keep existing compat) (#2351) (@github-actions[bot])
- Fix typo in docs/src/tutorials/2021-01-26-mlp.md (#2353) (@poludmik)
- Fixing typos in documentation (#2355) (@poludmik)
- Bump AMDGPU.jl compat to 0.7 (#2356) (@pxl-th)
- Bump AMDGPU compat to 0.8 (#2359) (@pxl-th)
Closed issues: - Android/iOS support (#2357)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 2 years ago
Flux - v0.14.6
Flux v0.14.6
Merged pull requests:
- Adding tooling for JuliaFormatter. (#2323) (@codetalker7)
- Fix typo (#2329) (@christiangnrd)
- Bump AMDGPU compat to 0.6 (#2332) (@pxl-th)
- Bump dorny/paths-filter from 2.9.1 to 2.11.1 (#2333) (@dependabot[bot])
- Bump actions/checkout from 2.2.0 to 4.0.0 (#2334) (@dependabot[bot])
- update CUDA compat (#2338) (@CarloLucibello)
Closed issues: - move the codebase to 4 spaces indentation (#2298) - No cudnn implementation of Conv((1,) N=>M) (#2322) - can't use masks in multi-head-attention layer (#2336) - Adapt saving & loading example to CuArrays (#2337) - Segmentation fault when doing a forward pass with a model saved with BSON (#2339)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 2 years ago
Flux - v0.14.5
Flux v0.14.5
Merged pull requests: - Bump actions/checkout from 1.0.0 to 3.6.0 (#2324) (@dependabot[bot]) - Bump actions/checkout from 3.6.0 to 4.0.0 (#2326) (@dependabot[bot]) - rename "AMD" backend to "AMDGPU" (#2328) (@CarloLucibello)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 3 years ago
Flux - v0.14.3
Flux v0.14.3
Closed issues:
- No error from negative learning rates (#1982)
- Implement data movement across GPU devices. (#2302)
- train! using Metal and stateful optimizers fails (#2310)
- Warning: sort(d::Dict; args...) is deprecated, use sort!(OrderedDict(d); args...) instead. (#2312)
- Does withgradient have lower precicision than simply calling the function? (#2315)
- Warning: sort(d::Dict; args...) is deprecated, use sort!(OrderedDict(d); args...) instead. (#2320)
Merged pull requests:
- Bump thollander/actions-comment-pull-request from 2.4.0 to 2.4.2 (#2307) (@dependabot[bot])
- Implement interface for data transfer across GPU devices. (#2308) (@codetalker7)
- Removing deprecated method call in GPU_BACKEND_ORDER. (#2314) (@codetalker7)
- Added entry for RobustNeuralNetworks.jl in ecosystem.md (#2317) (@nic-barbara)
- Allow Optimisers.jl v0.3 (#2318) (@CarloLucibello)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 3 years ago
Flux - v0.14.2
Flux v0.14.2
Closed issues:
- Mixed precision training. (#543)
- have buildkite run GPU tests only (#2271)
- Allow old silent behavior for gpu (#2293)
- @autosize macro is not working (#2296)
- Why Flux is Significantly Slower than Pytorch? (#2300)
- Huber Loss Fails with Metal GPU (#2305)
Merged pull requests: - Adding device objects for selecting GPU backends (and defaulting to CPU if none exists). (#2297) (@codetalker7) - Run only GPU tests on buildkite. (#2301) (@codetalker7) - Avoid broadcast-related type instabilities with huber_loss (#2306) (@jeremiahpslewis)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 3 years ago
Flux - v0.14.1
Flux v0.14.1
Closed issues:
- gpu() returns nothing (#2294)
Merged pull requests:
- Update NEWS.md for 0.14 release (#2288) (@mcabbott)
- Update description of trainable in "advanced.md" (#2289) (@mcabbott)
- fix: make gpu(x) return unmodified x when GPU backends aren't loaded (#2295) (@IanButterworth)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 3 years ago
Flux - v0.14.0
Flux v0.14.0
Flux now requires Julia 1.9, to take advantage of package extensions. CUDA is no longer loaded automatically, to speed up loading when not using a GPU / using a non-Nvidia one.
Previously deprecated functions removed are:
* Flux.stop and Flux.skip, in favour of break / continue
* The macro @epochs, in favour of a for loop
* Flux.zeros and Flux.ones, in favour of zeros32 and ones32.
Closed issues: - Choosing model serialization format(s) for cross-framework support (like HuggingFace) (#1907) - Very slow "using Flux" (#1961) - SSIM loss (#2165) - buildkite failure on julia v1.6 (#2214) - Sysimage compilaiton failed (#2242) - PackageCompiler fails with Flux on embedded ARM/no GPU (#2262) - how to make CUDA functionalities an extension (#2265) - Recurrent layers can't be applied to views of OneHotArrays (#2279) - Question about using loop in loss function (#2280)
Merged pull requests:
- add CUDA extension (#2268) (@CarloLucibello)
- fix doc of PairwiseFusion (#2281) (@ctarn)
- Update AMDGPU & Metal compat & add CI job (#2282) (@pxl-th)
- Cleanup for v0.14 release (#2283) (@CarloLucibello)
- tag v0.14 (#2284) (@CarloLucibello)
- Update training.md (#2286) (@dreivmeister)
- Fix typos in 0.14 docs (#2287) (@christiangnrd)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 3 years ago
Flux - v0.13.17
Flux v0.13.17
Closed issues: - Metal GPU acceleration on Apple Silicon (#1304) - Docs Revamp (#2174) - Add Tutorial Image Segmentation using Metalhead's UNet (#2192) - Flux.state binding does not exist (#2256) - "failed to start primary task" with Julia 1.9 and nthreads(:interactive) > 0 (#2257) - Error message (#2259) - GPU/CUDA memory leak (#2261)
Merged pull requests: - [CI] Separate julia-nightly from common matrix generator (#2254) (@skyleaworlder) - Bump thollander/actions-comment-pull-request from 2.3.1 to 2.4.0 (#2264) (@dependabot[bot]) - add outdated version banner to the docs (#2267) (@CarloLucibello) - initial Metal support (#2270) (@CarloLucibello)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 3 years ago
Flux - v0.13.16
Flux v0.13.16
Closed issues: - Failed to precompile Flux (#2231) - need help from an expert (#2233) - Conv layer throws TaskFailedException when internal parameters are BigFloat (#2243) - Code block in documentation not rendering properly (#2248) - Regularisation looks to slow down gradient function by factor 500 (#2253)
Merged pull requests:
- Add EmbeddingBag (#2031) (@mcognetta)
- Remove greek-letter keyword arguments (#2139) (@mcabbott)
- Speed-up normalization layers (#2220) (@pxl-th)
- Print the state of Dropout etc. (#2222) (@mcabbott)
- Float32 warning bug fix (#2226) (@ashwani-rathee)
- Remove type restrictions for recurrent cells (#2234) (@darsnack)
- Fix Conv transfer to AMDGPU (#2235) (@pxl-th)
- add Flux.state(x) (#2239) (@CarloLucibello)
- fix perf issue loadmodel! (#2241) (@CarloLucibello)
- gpu(::DataLoader), take III (#2245) (@mcabbott)
- Update NEWS (#2247) (@mcabbott)
- fix indent in recurrent.jl (#2249) (@mcabbott)
- rm StatsBase (#2251) (@mcabbott)
- Remove greek-letter keyword from normalise (#2252) (@mcabbott)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 3 years ago
Flux - v0.13.15
Flux v0.13.15
Closed issues:
- MethodError from show on custom layer, using @functor with no fields (#2208)
- Simple L2 regularisation with Flux.params excruciatingly slow (#2211)
- Quick start tutorial doesn't work; Flux.setup not defined (#2217)
- Migrating to new explicit-style Flux (#2221)
- should f16,f32, ... convert integer or boolean arrays? (#2225)
- Creation of Adversarial Examples (#2229)
Merged pull requests: - Add a logistic regression example to the Getting Started section (#2021) (@Saransh-cpp) - MultiHeadAttention implementation (#2146) (@CarloLucibello) - Use consistent spelling for optimise (#2203) (@jeremiahpslewis) - Fix a bug in show (#2210) (@mcabbott) - Create dependabot.yml (#2212) (@CarloLucibello) - Bump thollander/actions-comment-pull-request from 1.0.1 to 2.3.1 (#2213) (@dependabot[bot]) - manual gradient checks for RNN - implicit and explicit gradients (#2215) (@jeremiedb) - doc entry for MultiHeadAttention (#2218) (@CarloLucibello) - fix some documenter warnings (#2219) (@CarloLucibello) - example code have a little mistake (#2223) (@jaco267) - Update 2020-09-15-deep-learning-flux.md (#2224) (@natema) - Fix link to ParameterSchedulers.jl docs (#2227) (@white-alistair) - f16,f32,.. don't convert int arrays + handle complex (#2228) (@CarloLucibello) - doc: Syntax in GAN tutorial (#2230) (@musoke)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 3 years ago
Flux - v0.13.14
Flux v0.13.14
Closed issues:
- loadmodel! example in the docs doesn't work (#2191)
- ConvTranspose can cause Julia crash on GPU (#2193)
- Flattern removal (#2195)
- Flux v0.13.13 gpu crashes (#2199)
- FLux and Lux upon using: ERROR: LoadError: ArgumentError: invalid version string: local (#2202)
- UndefVarError: #flatten not defined with existing model (#2204)
Merged pull requests: - Add AMDGPU extension (#2189) (@pxl-th) - fixed BSON loadmodel! documentation and added a test case (#2194) (@jonathanBieler) - reintroduce flatten (#2196) (@CarloLucibello) - fix various deprecation warnings (#2197) (@ExpandingMan) - CompatHelper: add new compat entry for Preferences at version 1, (keep existing compat) (#2198) (@github-actions[bot]) - reintegrate has_cudnn check (#2200) (@CarloLucibello) - remove comment on not importing losses in Flux's namespace (#2201) (@CarloLucibello) - dummy implementation of flatten (#2205) (@CarloLucibello) - tiny change to docstring of gpu function (#2206) (@GSmithApps)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 3 years ago
Flux - v0.13.13
Flux v0.13.13
Closed issues:
- Normalization layers promote eltype (#1562)
- Recurrent cell eltype restriction breaks outputsize (#1565)
- Performance regression with graph neural networks (#1577)
- Opaque error caused by Float64 input to RNN (#1972)
- Binding Flux.setup does not exist (#2169)
- Un-intended behaviour? Should Flux be able to reduce StaticArrays? (#2180)
- Custom model can not be trained (#2187)
Merged pull requests:
- Match layer output to weights (#2156) (@mcabbott)
- Add friendly size check (#2176) (@mcabbott)
- Add f16 (#2184) (@mcabbott)
- remove Flux.flatten in favor of MLUtils.flatten (#2188) (@CarloLucibello)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 3 years ago
Flux - v0.13.12
Flux v0.13.12
Closed issues: - Delta neural networks inference (#2129) - [Bug] Embedding forward pass breaks for onehotbatch with multiple batch dimensions (#2160) - MethodError: no method matching when training LSTMs even when loss function is working corrently (#2168) - Type instability with Flux.update! when loss function involves extra arguments (#2175)
Merged pull requests:
- Un-deprecate track_stats for InstanceNorm (#2149) (@ToucheSir)
- Move dropout to NNlib (#2150) (@mcabbott)
- Use NNlib's within_gradient (#2152) (@mcabbott)
- Export rand32 and friends (#2157) (@mcabbott)
- Remove piratical array conversion rule (#2167) (@ToucheSir)
- update: actions node 12 => node 16 (#2173) (@skyleaworlder)
- cuda 4.0 compat (#2177) (@CarloLucibello)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 3 years ago
Flux - v0.13.11
Flux v0.13.11
Closed issues:
- Deprecate track_stats=true for GroupNorm and InstanceNorm (#2006)
- cpu(x) errors for x isa CuArray{<:CartesianIndex} (#2116)
- Constructing a Chain from a dictionary (#2142)
- Method error when using Flux.setup with Embedding layer (#2144)
- Method Error when using Flux.withgradient (#2148)
Merged pull requests:
- fix cpu(x) for immutable arrays (#2117) (@CarloLucibello)
- Fix two bugs re setup (#2145) (@mcabbott)
- CompatHelper: bump compat for MLUtils to 0.4, (keep existing compat) (#2147) (@github-actions[bot])
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 3 years ago
Flux - v0.13.10
Flux v0.13.10
Closed issues: - remove Bors (#1843) - Only generate and upload coverage for one matrix entry (#1939) - [Discussion]: Revamped Getting Started guide (#2012) - Using users provided weight matrix to build LSTM layers (#2130)
Merged pull requests:
- Re-write training docs (#2114) (@mcabbott)
- Move doc sections to "guide" + "reference" (#2115) (@mcabbott)
- Allow ForwardDiff in BatchNorm's track_stats (#2127) (@mcabbott)
- Fix last block in quickstart.md (#2131) (@simonschnake)
- Delete bors.toml (#2133) (@CarloLucibello)
- Docs for onecold (#2134) (@nathanielvirgo)
- [ISSUE 1939] Update workflow, to only generate coverage for a specific entry (#2136) (@skyleaworlder)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 3 years ago
Flux - v0.13.9
Flux v0.13.9
Closed issues:
- Iteration over params(m) in explicit mode gives no gradient (#2091)
- Flux.Optimise.update! updating grads instead of params? (#2121)
- Flux.reset! triggers a BoundsError (#2124)
Merged pull requests:
- Remove train! from quickstart example (#2110) (@mcabbott)
- Re-organise "built-in layers" section (#2112) (@mcabbott)
- Narrower version of @non_differentiable params (#2118) (@mcabbott)
- allow non-tuple data in the new train! (#2119) (@CarloLucibello)
- fix train! test (#2123) (@CarloLucibello)
- Move 5 tutorials from fluxml.github.io (#2125) (@mcabbott)
- Remove Flux.Data module (#2126) (@mcabbott)
- CompatHelper: bump compat for Functors to 0.4, (keep existing compat) (#2128) (@github-actions[bot])
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 3 years ago
Flux - v0.13.8
Flux v0.13.8
Closed issues:
- using Flux is broken on the Julia nightly (#2097)
- Chain(Parallel(...), ...) (#2100)
- Apparent memory leak when using Distributed? (#2102)
- [API] Preventing errors from misplaced optimizer objects (#2106)
Merged pull requests: - Safer gradients (by copying before mutating) & less piracy (by removing ArrayInterface) (#2098) (@mcabbott) - Allow OneHotArrays.jl v0.2 (#2109) (@mcabbott)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 3 years ago
Flux - v0.13.7
Flux v0.13.7
Closed issues:
- DimensionMismatch("array could not be broadcast to match destination") (#1457)
- Warn on NaN loss (#1981)
- Make create_bias a public API? (#2049)
- Make rng_from_array non-differentiable (#2062)
- @autosize does not work with semi-colon separated kwargs (#2086)
- early_stopping does not work as expected (#2089)
Merged pull requests:
- Documentation headings & sections (#2056) (@mcabbott)
- Add a dark mode version of logo (#2063) (@Saransh-cpp)
- Fix a few crossrefs + update Zygote's page (#2064) (@Saransh-cpp)
- Make rng_from_array non differentiable (#2065) (@Saransh-cpp)
- Add an example to the readme? (#2067) (@mcabbott)
- Add a quick start example, and change some headings (#2069) (@mcabbott)
- Stop training on Inf/NaN loss (#2070) (@mcabbott)
- Export Embedding (#2072) (@mcognetta)
- Relax RNN/LSTM/GRUCell internal matrix type restrictions (#2073) (@mcognetta)
- Finish docs for #2073 (#2075) (@mcognetta)
- Add @autosize (#2078) (@mcabbott)
- Back to create_bias (#2081) (@Saransh-cpp)
- Simplify Embedding (#2084) (@mcabbott)
- Fix |> gpu bug in @autosize (#2085) (@mcabbott)
- Fix #2086 re @autosize (#2087) (@mcabbott)
- Use the standard Documenter.jl local redirect (#2093) (@ChrisRackauckas)
- CompatHelper: bump compat for MLUtils to 0.3, (keep existing compat) (#2095) (@github-actions[bot])
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 3 years ago
Flux - v0.13.6
Flux v0.13.6
Closed issues:
- OneHotArrays.jl? (#1544)
- [Discussion]: doctests, docstrings, documentation manual, and unclear internal API (for newcomers) (#1990)
- [Bug]: Swapped alpha and beta in tversky loss? (#1993)
- [Discussion]: documentation for @reexported and imported (or using) packages (#2038)
- Pull request #2007 causes Flux.params() calls to not get cached (#2040)
- v0.13.5 breaks Flux.train! on a custom type (#2045)
- Bounds erro for Flux.reset! in loss function (#2057)
Merged pull requests:
- Miscellaneous docstring additions and fixes (#1998) (@Saransh-cpp)
- Use muladd for LSTM cell matmuls (#2023) (@ToucheSir)
- using OneHotArrays (#2025) (@mcabbott)
- mark stop, skip, @epochs as deprecated (#2027) (@mcabbott)
- Fix the last remaining 404 errors (#2035) (@Saransh-cpp)
- Add ability to filter loadmodel! recursion (#2041) (@darsnack)
- Mark track_stats=true as deprecated (#2042) (@akahard2dj)
- Better docs for reexported packages (#2046) (@Saransh-cpp)
- Typo in BatchNorm number of channels assertion (#2047) (@MarcoVela)
- Add extra test for params (#2051) (@christiangnrd)
- Restore some private functions (#2052) (@ToucheSir)
- Make params non-differentiable (Closes #2040 & #2048) (#2054) (@christiangnrd)
- Leftover changes from #2046 (#2055) (@Saransh-cpp)
- unthunk in some rules (#2058) (@mcabbott)
- Fix the failing CI build (#2059) (@christiangnrd)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 4 years ago
Flux - v0.13.5
Flux v0.13.5
Closed issues:
- PINN loss doesn't converge to 0? (#1966)
- Simple chaining compatibility check (#2017)
- v0.12.10 => v0.13.4 breaks Dropout on CUDA (#2018)
- Wrong rrule dispatch for Array constructor (#2033)
Merged pull requests:
- Get rid of documentation warnings and 404 pages (#1987) (@Saransh-cpp)
- use Functors 0.3 in Flux (#2007) (@mcabbott)
- Typo (#2020) (@trigaten)
- Add NNlib.grid_sample (#2022) (@scheidan)
- Remove CTC loss (moved to NNlib) (#2024) (@mcabbott)
- Fix typo in docs (#2030) (@svilupp)
- fix array constructor rrule (#2034) (@chengchingwen)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 4 years ago
Flux - v0.13.4
Flux v0.13.4
Closed issues:
- Repository: on the addition of loss/distance functions and other niceties to Flux (#826)
- trainable for BatchNorm stops parameters from being saved and loaded (#1027)
- Non-descriptive arg in Conv: why filter intead of size? (#1212)
- Ada or ADA (#1949)
- Make gpu(::DataLoader) work or error loudly if it doesn't (#1974)
- Conversion error when loading a model with v0.13+ with BSON (#1984)
- GPU broadcasting error when using softmax on GPU (#1994)
- Error when using CUDA (#1997)
- type cannot been referred with structured model function (#2000)
- [Broken Documentation] Dense(1 => 1) (#2001)
Merged pull requests:
- Fix slight typos in LayerNorm docs (#1975) (@theabhirath)
- Piratical errors for two mistakes (#1976) (@mcabbott)
- Show using Flux before BSON @load (#1977) (@JeffFessler)
- Update docstrings of basic.jl and conv.jl (#1978) (@Saransh-cpp)
- Added Common GPU Workflows in Docs (#1980) (@lfenzo)
- PairwiseFusion layer, take 2 (#1983) (@theabhirath)
- deprecations.jl: depwarn -> Base.depwarn (#1985) (@skleinbo)
- Update docstrings in upsample.jl, recurrent.jl, and normalise.jl (#1995) (@Saransh-cpp)
- replace ADAM with Adam and its variants thereof (#1996) (@Karthik-d-k)
- Make Dropout docs a little more user friendly (#2014) (@theabhirath)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 4 years ago
Flux - v0.13.2
Flux v0.13.2
Closed issues: - Inconsistent "Julia ecosystem" docs (#1922) - sigmoid_fast in GRU? (#1967)
Merged pull requests:
- Unify ecosystem.md (#1923) (@Saransh-cpp)
- Updated path to DiffImages.jl (#1964) (@arcAman07)
- Explain stride≠1 case for SamePad (#1965) (@KronosTheLate)
- fast sigmoid (#1968) (@oysteinsolheim)
- CompatHelper: bump compat for ArrayInterface to 6, (keep existing compat) (#1969) (@github-actions[bot])
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 4 years ago
Flux - v0.13.1
Flux v0.13.1
Closed issues:
- Batchnorm on GPU for Float64 values (#1897)
- Tag? (#1924)
- DataLoader causes scalar indexing on GPU in Flux v0.13.0 (regression) (#1935)
- Flux.flip with broadcasting warning (#1936)
- Add a workflow to clean-up gh-pages branch? (#1940)
- DimensionMismatch: All data containers must have the same number of observations. (#1941)
- Type instability in Recur for 3 dimensional arrays (#1947)
- What is the idiomatic way to get training loss from gradient()? (#1950)
- Dropout erroring on latest CUDA (#1960)
- AdaBelief issues (#1962)
Merged pull requests:
- Add a ton of doctests + fix outdated documentation in .md files (#1916) (@Saransh-cpp)
- Get the DocBot up again! (#1937) (@Saransh-cpp)
- Broadcasting replaced with comprehension in the Flux.flip function. (#1938) (@fpartl)
- Fix type instabilities in apply!(optimizer, ...) (#1942) (@ancapdev)
- Add a workflow to delete PR previews (#1943) (@Saransh-cpp)
- Fix for progress logging to non-VS Code loggers (#1944) (@darsnack)
- Add Base.firstindex(c::Chain) = 1 (#1945) (@KronosTheLate)
- Recur type stability for 3d arrays (#1948) (@MarcoVela)
- Resolve two warnings in the test suite (#1951) (@mcognetta)
- Update documentation on Split layer (#1953) (@JLDC)
- [docs] suggest using ADAM with LR=1 when combined with ExpDecay (#1955) (@ericphanson)
- Type stable conv_reshape_bias and AD-friendly ConvDims helpers (#1956) (@ToucheSir)
- onehotbatch with CuArray (#1959) (@CarloLucibello)
- AdaBelief bias correction (#1963) (@cossio)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 4 years ago
Flux - v0.13.0
Flux v0.13.0
Changes in NEWS.md
Closed issues:
- DepthwiseConv does not run on GPU (#459)
- Flux type piracy breaks REPL completions (#629)
- Cannot do double iteration of DataLoader (#1227)
- elu activation fails on nested pullbacks on GPU (#1383)
- Training not working for 1D types (#1479)
- adjoint of conv adjoint. (#1665)
- pullback's back returns unexpected size if some parameters are not used (#1601)
- Allow specification of RNG in Dropout (#1617)
- deprecate DepthwiseConv once we have groups in standard conv (#1667)
- Parallel edge-cases (#1685)
- Layer printing interferes with different element types (#1690)
- Normalization Layers not interating well with destructure/restructure (#1727)
- missing docstring for Flux.params and trainable (#1732)
- inconsistency between params and destructure (#1733)
- Parameter Sharing breaks destructure (#1767)
- Remove Juno.jl dependency (#1779)
- Flux.destructure's restructure fails in the gradient if loss does not use all parameters (#1826)
- Flux.chunk for multi-dimensional arrays (#1841)
- onehotbatch performance (#1844)
- Issue taking gradients of Chains on GPU (#1853)
- Chain forgets names under fmap (#1857)
- Recurrent 3d interface uses a lot of memory (#1872)
- Gradient incorrect for Conv-layer and complex numbers (#1876)
- Add Siamese Contrastive Loss function (#1880)
- Urgent GSoC revisions are needed. (#1890)
- Flux v0.12.9 and the Flux.Tracker.gradient is wrong, why? (#1898)
- LoadError UnderVarError: flatten not defined (#1899)
- Proposal: Move params to Zygote (#1900)
- This one is not in use, which one should I use instead in Flux? (#1903)
- ERROR: LoadError: Can't differentiate foreigncall expression (#1904)
- Missing docstring for Flux.Data.Dataloader (#1909)
- Different Julia versions at different places for doctests (#1914)
- Parallel layer behaves diffferently in a Chain than on its own (#1919)
- ADAMW not stable (#1920)
- Chain ignores Base.show function of custom layer (#1929)
Merged pull requests:
- v0.13 deprecations (#1751) (@CarloLucibello)
- Print channel dimensions of Dense like those of Conv (#1658) (@mcabbott)
- Replace unrolled foldl used to evaluate Chain with a better one (#1809) (@mcabbott)
- Zero is a real number (Flux.Nil) (#1830) (@mcabbott)
- Use faster activation functions (#1837) (@mcabbott)
- Add RNG support for Dropout/AlphaDropout (#1849) (@darsnack)
- Fix CI to run on LTS + latest + nightly (#1852) (@darsnack)
- Fix type-stability for normalization layers (#1856) (@pxl-th)
- Use ProgressLogging instead of Juno (#1859) (@darsnack)
- Speed up onehotbatch (#1861) (@mcabbott)
- Simplify trainable, functor and Parallel (#1862) (@mcabbott)
- Replace @adjoint with rrule (#1863) (@mcabbott)
- Depend on Optimisers.jl (#1864) (@mcabbott)
- rationalize CI (#1865) (@CarloLucibello)
- Updated Dropout for more input types. (#1867) (@ShoofLLC)
- fix adamw (#1868) (@CarloLucibello)
- Add OperatorLearning.jl to Flux downstream tests (#1869) (@ChrisRackauckas)
- Mark dropout_mask as non-differentiable (#1870) (@ToucheSir)
- Recurrent benchmarks (#1871) (@mkschleg)
- Changed view to eachslice for folding in recurrent (#1873) (@mkschleg)
- use MLUtils (#1874) (@CarloLucibello)
- Add a structural loadparams! (#1875) (@darsnack)
- Truncated normal initialisation for weights (#1877) (@theabhirath)
- Extending Diagonal (#1881) (@theabhirath)
- rm Flux.Zeros (#1882) (@mcabbott)
- CompatHelper: add new compat entry for SpecialFunctions at version 2, (keep existing compat) (#1883) (@github-actions[bot])
- Make RNN layers accept in => out (#1886) (@mcabbott)
- Speeding up onehotbatch by creating OneHotArray directly (#1888) (@TLipede)
- CompatHelper: bump compat for MLUtils to 0.2, (keep existing compat) (#1889) (@github-actions[bot])
- Addition of Siamese Contrastive Loss function ( Updated ) (#1892) (@arcAman07)
- Buildkite: don't persist registry across runs (#1893) (@ToucheSir)
- Use destructure from Optimisers.jl (#1901) (@mcabbott)
- RFC: Restrict train! to AbstractOptimiser (#1902) (@mcabbott)
- Add dims keywords to some tests (#1906) (@mcabbott)
- Mark initialisations nograd, restrict signatures (#1908) (@mcabbott)
- Add MLUtils's docs and fix some missing docstrings (#1910) (@Saransh-cpp)
- Improvements for LayerNorm (#1911) (@theabhirath)
- Improve docs for initialisation (#1912) (@mcabbott)
- Turn off doctests while building docs (#1915) (@Saransh-cpp)
- dampening -> damping (#1918) (@alhirzel)
- remove DepthwiseConv type in favor of Conv (#1921) (@CarloLucibello)
- Allow activation function for Diagonal (#1925) (@theabhirath)
- Upgrade warnings for v0.13 (#1926) (@mcabbott)
- Rename Diagonal to Scale (#1927) (@mcabbott)
- Fix a code block (#1933) (@prbzrg)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 4 years ago
Flux - v0.12.10
Flux v0.12.10
Closed issues: - ADAMW not stable (#1920)
Merged pull requests: - CompatHelper: bump compat for ArrayInterface to 5, (keep existing compat) (#1895) (@github-actions[bot]) - fix adamw (#1868) (@CarloLucibello)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 4 years ago
Flux - v0.12.9
Flux v0.12.9
Closed issues:
- Coverage (#89)
- Support for grouped convolutions (#330)
- onehot.md in docs should not have subtitle "Batches" (#510)
- Repo tagged with the "the-human-brian" (potential typo) (#512)
- RNNs, batching and sequences (#705)
- Model Zoo Housing.jl Example functionality not clear (#769)
- Asymmetric padding fails on gpu models (#775)
- Can't get user defined model to work (#812)
- Cryptic error in Flux#zygote "Can't differentiate foreigncall expression" (#817)
- Passing transposed matrix to softmax causes scalar indexing on GPU, which is very slow (#888)
- Does it support training on multiple GPUs? (#910)
- batchedmul causes a 'cannot take the CPU address of a CuArray' error on GPU (#1090)
- CTC loss (#1135)
- Inconsistent behavior of gradient of empty matrices (#1151)
- Flux.Conv type instability (#1178)
- CUDA.jl (#1194)
- Incorrect types following CUDA.jl refactoring (#1200)
- Got an error, while trying to implement softplus with beta (#1216)
- test regression with recurrent neural networks (#1245)
- regression in RNN with OneHotMatrix and CUDA (#1262)
- Gradient calculation bug re-introduced in Flux v0.10.4 and Zygote v0.4.22 (#1269)
- LSTM "succeeds" on data with incompatible dimensions (#1396)
- Document conv data handling, especially for 1d (#1465)
- Flux.destructure gives DimensionMismatch error in backward pass due to Chain of mutable struct(s) (#1502)
- Adjoints for regularizers? (#1575)
- Zygote error: UndefVarError: S not defined (#1578)
- Warning using Flux on Linux device without CUDA or Nvidia card (#1581)
- Flux downloads CUDA110 Artifacts every time I precompile on Ubuntu (#1600)
- Why does calling the gpu function not return an error when CUDA is unavailable (#1634)
- Flux errors on Julia 1.7 Beta 2 (#1652)
- LLVM 4.x.x compatibility (#1669)
- Add better docs for the LSTM function (#1696)
- Recurrent docs out of sync (#1714)
- Docs haven't built since Aug. 3 (#1723)
- Investigate nightly CI build issues (#1724)
- unsqueeze is not type stable (#1737)
- failing doc tests (#1739)
- Link to "train!" gives 404 page not found error on the website. (#1745)
- Issues model with custom gradient (w.r.t. input variable) layer (#1760)
- Flux.loadparams! is slow. (#1764)
- world age issues when loading a bson file containing a model with flux utility functions (#1769)
- How to fast find source code of function, like Dense() Chain() (#1770)
- How to get the mathematical expression of Neural Network. (#1771)
- How to write a seq of wi: w1, w2, ... , w_1000 (#1773)
- Error when training simple Flux model (#1777)
- Differentiating through my custom struct its restructuring throws an error (#1796)
- Incompatibility with SpecialFunctions 2.0 (#1802)
- Buildkite CI failures with grad test of ConvTranspose + selu (#1804)
- Slowdown when running multiple large models in parallel (#1806)
- ERROR: LoadError: Some tests did not pass: 252 passed, 1 failed, 0 errored, 21 broken. in expression starting at /home/ian/.julia/packages/Flux/BPPNj/test/runtests.jl:11 ERROR: Package Flux errored during testing (#1814)
- Can ExpDecay of learning rate start at some intermediate step? (#1815)
- Optimisers epsilon (#1818)
- Zygote Flux and custom adjoints on GPU (#1828)
- TypeErro in DEQ example: non-boolean (Nothing) used in boolean context #677 (#1846)
Merged pull requests:
- Clarify that params updates (#1752) (@KronosTheLate)
- Add custom model example to docs. (#1758) (@Gregliest)
- Make unsqueeze type stable (#1759) (@cossio)
- Use view for RNN gate slice extraction (#1761) (@ToucheSir)
- Doc update (saving.md): removed outdated info; Typo fix. (#1762) (@NightMachinary)
- Doc update (recurrence.md): fixed incorrect output dimensions, clarified batching. (#1763) (@NightMachinary)
- Expand RNN/LSTM/GRU docs (#1772) (@mcognetta)
- Fix a doctest failure (#1775) (@mcognetta)
- Use conjugates in optimizers to better learn on complex-valued inputs (#1776) (@staticfloat)
- Fix AlphaDropout implementation and add tests (#1781) (@ToucheSir)
- add logo to documentation (#1782) (@kwehmeyer)
- Doc update (training.md): fix DataLoader example in Training section (#1783) (@eliascarv)
- Fix link to train in the docs (#1784) (@logankilpatrick)
- Update train.jl to add a more detailed train! docstring (#1785) (@logankilpatrick)
- Add docstring for params (#1786) (@logankilpatrick)
- Create a PR comment with docs preview link (#1788) (@logankilpatrick)
- Add trilinear Upsample layer (#1792) (@tknopp)
- Tidy up Maxout (#1794) (@mcabbott)
- Simplify mse() to use abs2() (#1795) (@staticfloat)
- Mark destructure gradient test as broken (#1797) (@ToucheSir)
- Fix failing params doctests (#1798) (@ToucheSir)
- Only add PR comment with docs build if the docs label is added (#1799) (@logankilpatrick)
- Add more context on the behavior of the GPU function (#1800) (@logankilpatrick)
- Add warning if the GPU function is called and CUDA is not available (#1801) (@logankilpatrick)
- Add buildkite step to run on Julia LTS (#1805) (@DhairyaLGandhi)
- ExpDecay start step (#1816) (@cossio)
- make eps a parameter of optimisers (#1819) (@cossio)
- Contributor's Guide draft (#1824) (@lilianabs)
- Update conv.jl (#1825) (@rkube)
- CompatHelper: bump compat for ArrayInterface to 4, (keep existing compat) (#1827) (@github-actions[bot])
- Remove "Batches" from one hot section header in docs (#1831) (@darsnack)
- Document disabling GPUs (#1835) (@DhairyaLGandhi)
- Try using latest cu(DNN) binaries (#1836) (@ToucheSir)
- Add news for bump version (#1838) (@DhairyaLGandhi)
- move eps to the end (#1840) (@cossio)
- Add codecov on CI (#1842) (@ToucheSir)
- add token secret for codecov (#1845) (@ToucheSir)
- CompatHelper: bump compat for NNlib to 0.8, NNlibCUDA to 0.2 ~(keep existing compat)~ (#1847) (@github-actions[bot])
- Tweak docs about disabling CUDA devices (#1850) (@IanButterworth)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 4 years ago
Flux - v0.12.8
Flux v0.12.8
Closed issues:
- Coverage (#89)
- Flux.train! stops working after the first iteration without an error. (#1692)
- Update Zygote (#1728)
- additional arguments to loss function? (#1730)
- The Purpose and Goals of Flux.jl (#1734)
- FluxML's NumFOCUS Affiliate project application (#1740)
- ConvTranspose does not support groups (#1743)
- deepcopy(nn::Chain) does not deep copy with CuArray weights! (#1747)
- InvalidIRError when putting a model on the GPU (#1754)
Merged pull requests:
- remove Manifest (#1725) (@CarloLucibello)
- add unbatch (#1726) (@CarloLucibello)
- Adds affine and track_stats params to BatchNorm docstring (#1729) (@Mottl)
- add some changes to the beginning of docs (#1736) (@DhairyaLGandhi)
- Fix doc string of Upsample (#1738) (@chunjiw)
- allow groups in ConvTranspose (#1744) (@jw3126)
- Fix Saving and loading model output example (#1746) (@logankilpatrick)
- Fix train! doc string 404 (#1748) (@logankilpatrick)
- Fix @ Functors 404's (#1749) (@logankilpatrick)
- fix CI build (#1750) (@DhairyaLGandhi)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 4 years ago
Flux - v0.12.7
Flux v0.12.7
Closed issues:
- Poor performance relative to PyTorch (#886)
- Recur struct's fields are not type annotated, which is causing run–time dispatch and a significant slowdowns (#1092)
- Bug: lower degree polynomial substitute in gradient chain! (#1188)
- Very slow precompile (>50min) on julia 1.6.0 on Windows (#1554)
- Do not initialize CUDA during precompilation (#1597)
- GRU implementation details (#1671)
- Parallel layer doesn't need to be tied to array input (#1673)
- update! a scalar parameter (#1677)
- Support NamedTuples for Container Layers (#1680)
- Freezing layer parameters still computes all gradients (#1688)
- A demo is 1.5x faster in Flux than tensorflow, both use cpu; while 3.0x slower during using CUDA (#1694)
- Problems with a mixed CPU/GPU model (#1695)
- Flux tests with master fail with signal 11 (#1697)
- [Q] How does Flux.jl work on Apple Silicon (M1)? (#1701)
- Typos in documents (#1706)
- Fresh install of Flux giving errors in precompile (#1710)
- Flux.gradient returns dict of params and nothing (#1713)
- Weight matrix not updating with a user defined initial weight matrix (#1717)
- [Documentation] No logsumexp in NNlib page (#1718)
- Flattened data vs Flux.flatten layer in MNIST MLP in the model zoo (#1722)
Merged pull requests:
- Add WIP docstrings to CPU and GPU (#1632) (@logankilpatrick)
- Add section on Checking GPU Availability (#1633) (@logankilpatrick)
- fix README (#1668) (@DhairyaLGandhi)
- Generalise Parallel forwards pass (#1674) (@DhairyaLGandhi)
- Adding GRUv3 support. (#1675) (@mkschleg)
- Support NamedTuples for Chain + Parallel (#1681) (@mcabbott)
- Adding support for folding RNNs over 3d arrays (#1686) (@mkschleg)
- Update nnlib.md (#1689) (@CarloLucibello)
- fix typo (#1691) (@foldfelis)
- Typo fix (#1693) (@lukemerrick)
- Remove out of date dead code in Conv layers (#1702) (@ToucheSir)
- Gradient definitions for cpu & gpu (#1704) (@mcabbott)
- Fix #1706 (#1707) (@rongcuid)
- Add GPU Adaptor (#1708) (@DhairyaLGandhi)
- Initialize CUDA lazily. (#1711) (@maleadt)
- Update community.md to reflect help wanted != good first issue (#1712) (@logankilpatrick)
- Fix link in README (#1716) (@nilsmartel)
- Add logsumexp to docs (#1719) (@DhairyaLGandhi)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 4 years ago
Flux - v0.12.6
Flux v0.12.6
Merged pull requests: - Add grouped convolution (#1531) (@DhairyaLGandhi) - fix deprecations of zeros (#1670) (@DhairyaLGandhi) - Add GPU activation tests for grouped conv (#1672) (@DhairyaLGandhi)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 5 years ago
Flux - v0.12.5
Flux v0.12.5
Closed issues:
- Hessian vector products (#129)
- Stopping criteria (#227)
- Flux + Julia ecosystem docs (#251)
- RNN unbroadcast on GPU not working (#421)
- Shouldn't gradcheck compares Jacobian? (#462)
- Transition examples in docs to doctests (#561)
- Batch-axis thread parallelism (#568)
- Add tests of ExpDecay (#684)
- Sudden memory leak when training on GPU over many epochs (#736)
- performance variance between macOS / Linux ? (#749)
- onehot ambiguous method (#777)
- Killed while training the model (#779)
- type Method has no field sparamsyms, while @save model (#783)
- Flux#zygote Error in phenomes... Mutating arrays is not supported (#819)
- Custom serialization pass for intermediate states (#845)
- OneHotMatrix does not support map (#958)
- CuArrays + huberloss iterate(::nothing) error (#1128)
- Can't get Flux (v0.10.3) working for Custom Loss function (#1153)
- Custom loss function on subset of parameters fails (#1371)
- Minimizing sum fails (#1510)
- gpu behaves differently from cu on a Char array (#1517)
- Warn different size inputs in loss functions (#1522)
- Recurrent docs need to be update for v0.12 (#1564)
- Computation of higher order derivatives for recurrent models results in strange errors (#1593)
- Why does DataLoader not throw an error when fed with a 1D vector for the target? (#1599)
- a small error in the documentation... (#1609)
- Slow unnecessary GPU copy of output of gpu(::OffsetArray) (#1610)
- "using Flux" makes type inference fail when there is a Ref{} (#1611)
- @epochs is missing a bracket (#1615)
- Flux Overview Documentation Out of Date (#1621)
- missing kernel for Base.unique (#1622)
- Compilation error on PPC (#1623)
- _restructure as part of the public API? (#1624)
- ERROR: setindex! not defined for Zygote.OneElement{...} (#1626)
- MethodError: Cannot convert an object of type Params to an object of type Float64 (#1629)
- MethodError: no method matching flatten(::Array{Float32,4}) (#1630)
- Where are the cpu() and gpu() functions? (#1631)
- bug in RNN docs (#1638)
- Bug in the current overview documentation (#1642)
- How to tell Flux.jl not to use the GPU? (#1644)
- Missing docs for @functor (#1653)
- typo in the docs/overview section right at the beginning (#1663)
Merged pull requests:
- multiplication of {Transpose, Adjoint} of Array and OneHotVector (#1424) (@gxyd)
- show(::Chain) (#1467) (@mcabbott)
- Add test for show(io, ::OneHotArray) on GPU (#1550) (@darsnack)
- document Join and Split error (#1607) (@magicly)
- fix typo in models overview document (#1608) (@teamclouday)
- fix AdamW and improve decays docs (#1612) (@CarloLucibello)
- use ArrayInterface.restructure in update! (#1613) (@CarloLucibello)
- Warn on reconstruct length mismatch (#1616) (@ToucheSir)
- Forward map(f, ::OneHotLike) to broadcast (#1619) (@darsnack)
- Properly move isbits and numeric arrays to GPU (#1620) (@ToucheSir)
- Update "Composing Optimisers" docs (#1628) (@StevenWhitaker)
- Fixup Dataloader's docstring (#1635) (@mcabbott)
- Add warnings for mismatched sizes in losses (#1636) (@mcabbott)
- updated recurrence.md which fixes #1564 (#1637) (@aditkumar72)
- fix recurrence docs (#1639) (@CarloLucibello)
- Update docstring for Conv to clarify feature dimensions (#1646) (@vivekkumar7089)
- Use correct eltype and rtol in CrossCor tests (#1650) (@ToucheSir)
- add Functors docs (#1654) (@DhairyaLGandhi)
- remove Manifest (#1657) (@CarloLucibello)
- Printing & docstrings for onehot / onehotbatch (#1660) (@mcabbott)
- Deprecate Flux.zeros (#1661) (@mcabbott)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 5 years ago
Flux - v0.12.4
Flux v0.12.4
Closed issues: - Unable to get gradients of "Dense" models when sparse arrays are involved (#965) - Pullback within pullback throws error when using swish activation function (#1500) - Stable docs are stuck on v0.11.2 (#1580) - LSTM gradient calculation fails on GPU, works on CPU (#1586) - BSON.@save model_path * ".bson" model ERROR: type Method has no field ambig (#1591) - Too slow hcat of OneHotMatrix. (#1594) - Fallback implementation convolution when using Duals (#1598) - Bad printing for OneHot* (#1603) - SamePad() with even kernel dimensions does not work (only in CUDA) (#1605)
Merged pull requests: - Add AbstractOptimiser type (#1325) (@DhairyaLGandhi) - Add early stopping utils (#1545) (@queensferryme) - Add Flux Overview to basics.md (#1579) (@batate) - [doc] fix Upsample docstring code block (#1587) (@johnnychen94) - fix DataFrames.jl link (#1589) (@achuchmala) - optimized hcat of onehot vectors and matrices (#1595) (@racinmat) - Use limited array printing for OneHotArrays (#1604) (@darsnack)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 5 years ago
Flux - v0.12.2
Flux v0.12.2
Closed issues: - Cosineembeddingloss could be added to Flux.jl (#1094) - Char RNN errors (#1215) - Colab - MethodError: no method matching (::Flux.LSTMCell{... (#1563) - Issue with Flux.jl installation (#1567) - Issue with Flux.jl installation (#1568) - Model no longer type stable when using destructure and restructure (#1569)
Merged pull requests: - Cuda 3.0 support (#1571) (@DhairyaLGandhi)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 5 years ago
Flux - v0.12.1
Flux v0.12.1
Closed issues: - Helper functions for choosing data types for bias and weight in Flux chains? (#1548) - LSTM failed to return gradient (#1551) - Flux.destructure gives MethodError when used with non-trainable parameters (#1553) - Restructure on Dense no longer plays nicely with alternative types (#1556)
Merged pull requests: - Add Julia 1.6 doc changes to CI (#1503) (@DhairyaLGandhi) - Fix #1556 (#1557) (@DhairyaLGandhi) - Minimal fix of #1556, remove eltype checks (#1558) (@mcabbott)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 5 years ago
Flux - v0.12.0
Flux v0.12.0
Closed issues:
- RNN state dimension with batches (#121)
- Support for additional dimensions in Dense layer (#282)
- Error messages when CUDNN is not loaded. (#287)
- Easier way of switching models from cpu to gpu? (#298)
- how would I implement an echo state network in flux ? (#336)
- Pkg.update() in Julia 0.6.x gets you an incompatible version of Flux (#341)
- Indices not defined (#368)
- Regression with Flux (#386)
- LSTM sequence processing (#393)
- Checkpointing (#402)
- Allowing users to specify their default data folder (#436)
- elu not working with GPU (#477)
- Tied Weights (#488)
- rethinking Conv, and layer granularity in general (#502)
- σ.() on GPU not using CUDAnative (#519)
- Using tensorflow and pytorch layers (#521)
- Abstract layers (#525)
- Max norm regularisation (#541)
- Typical accuracy function using onecold with a OneHotMatrix fails to compile on GPU (#582)
- Export apply!, etc (#588)
- Better initialization support (#670)
- Deprecate initialiser keyword arguments (#671)
- backprop fails on min.(x1,x2) (#673)
- Adaptive pooling layers in Flux. (#677)
- CUDAnative (#682)
- accumulate gradient with the new gradient API? (#707)
- sigmoid: multiplicative identity only defined for non-square matrices (#730)
- 1D Conv Broken (#740)
- Layers and Params should support equality (#1012)
- InstanceNorm throws a scalar getindex disallowed error on GPU (#1195)
- Error with GroupNorm on GPU (#1247)
- Error with BatchNorm/InstanceNorm after Conv1D on GPU (#1280)
- How to apply L2 regularization to a subset of parameters? (#1284)
- define modules function (#1294)
- Misleading InstanceNorm documentation? (#1308)
- ConvTranspose on GPU fails with certain activation functions (#1350)
- Conv with non homogenous array eltypes gives confusing error message (#1421)
- Layers' docstrings and constructors inconsistencies (#1422)
- BatchNorm alters its sliding mean/standard deviation parameters even in testmode if Zygote is called (#1429)
- BatchNorm on CUDA accepts improper channel size argument and "works" in a possibly ill-defined way. Proper errors on CPU (#1430)
- Better handling for layers with multiple inputs w/ outputsize (#1466)
- Dense function does not support tensor? (#1480)
- Cannot load model saved with JLD (#1482)
- RNN and GRU give mutation error; LSTM gives ArgumentError about number of fields (#1483)
- Moving OneHotMatrix to GPU triggers the slow scalar operations (#1494)
- Does gain do anything in kaiming_uniform? (#1498)
- Zeros has old behaviour on releases up to 0.11.6 (#1507)
- getting this -> ERROR: Mutating arrays is not supported (solved) (#1512)
- Moving multihead attention from transformers.jl into Flux.jl (#1514)
- Gradient cannot be got under testmode gpu net with Batchnorm (#1520)
- Development version document example on Dense layer's bias not working (#1523)
- how to use flatten layer? (it does not flatten arrays) (#1525)
- Ambiguity in recurrent neural network training (#1528)
- scalar indexing when showing OneHot gpu (#1532)
- Acitvation function relu terrible performance (#1537)
- Error on precompile (#1539)
- Flux.normalise vs standardise (#1541)
- Cudnn batchnorm causes errors when I disable BatchNorm when training (#1542)
- DimensionMismatch("All data should contain same number of observations") (#1543)
- Softmax stucks the network (#1546)
Merged pull requests:
- Added Bilinear layer (#1009) (@bhvieira)
- Rework normalization layers (#1397) (@CarloLucibello)
- Dense keyword handling, and docstring (#1440) (@mcabbott)
- define modules function (#1444) (@CarloLucibello)
- Use fallback for reshape/cat OneHotArray (#1459) (@darsnack)
- add Upsample and PixelShuffle layers (#1468) (@CarloLucibello)
- Add activation tests for GPU layers (#1472) (@DhairyaLGandhi)
- CompatHelper: bump compat for "Functors" to "0.2" (#1474) (@github-actions[bot])
- reexport compat (#1478) (@DhairyaLGandhi)
- add FluxBot (#1484) (@DhairyaLGandhi)
- Make outputsize understand multiple inputs (#1486) (@mcabbott)
- Add training loop to docs (#1488) (@DhairyaLGandhi)
- Implementation of Focal loss (#1489) (@shikhargoswami)
- Make Dropout docstring clear w.r.t. N-D dropout (#1490) (@darsnack)
- Update ecosystem.md (#1491) (@churchofthought)
- Add Orthogonal initialization feature. (#1496) (@SomTambe)
- Fix docs syntax for Join/ Split layers (#1497) (@DhairyaLGandhi)
- Fix layer init functions kwargs getting overwritten (#1499) (@DevJac)
- updated DataLoader, added a optional keyword argument rng. (#1501) (@norci)
- Add Parallel GPU tests (#1505) (@darsnack)
- Add ParameterSchedulers.jl to docs (#1511) (@darsnack)
- Update for latest ParameterSchedulers.jl release (#1513) (@darsnack)
- Fixes to Recurrent models for informative type mismatch error & output Vector for Vector input (#1521) (@jeremiedb)
- Add identity_init (#1524) (@DrChainsaw)
- fix print layernorm (#1526) (@CarloLucibello)
- preparatory work for v0.12 (#1527) (@CarloLucibello)
- small refactor of Dense (#1533) (@CarloLucibello)
- Fix printing of OneHotArray in REPL when CUDA.allowscalar(false) (#1534) (@darsnack)
- add vgg16 performance script (#1535) (@CarloLucibello)
- fx norm deprecation (#1538) (@CarloLucibello)
- add news entry for end of deprecation cycle (#1540) (@CarloLucibello)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 5 years ago
Flux - v0.11.2
Flux v0.11.2
Closed issues: - Error with Flux.crossentropy (#435) - Unnecessary typeasserts in Flux.Optimise.apply! cause training to fail (#816) - OneHotMatrix causes a 'scalar getindex disallowed' error on GPU (#1006) - Higher order derivative products? (#1102) - Gradient of Chain with respect to input on gpu (#1132) - Backprop through time is truncated to only 1 time step (#1209) - Failed to load Flux 1.11.0 and 1.11.1 with Julia 1.4.2 and 1.5.0 on a windows machine (#1313) - ADAMW Optimise has no field eta (#1316) - LayerNorm only operates on 2D tensors (also Diagonal) (#1321) - NNlib not defined error when loading model saved with BSON (#1322) - Map and broadcast on LSTM layers give different gradients (#1324) - zygote (#1327) - Error while pre-compIling Flux in Julia v1.4.2 on windows 10 (#1328) - DepthwiseConv gives incorrect channel sizes when initialized from array (#1331) - Flux.params return extra parameter (#1348) - XOR Error not converging to 0 (#1352) - Broken methods(Base.show) (#1354) - Applying Dense layer on OneHotMatrix is very slow and can be optimized. (#1356) - Unable to obtain gradient after flattened pooling layer. (#1359) - "incremental compilation may be fatally broken for this module" when using Flux (#1370)
Merged pull requests: - add Flux.skip() (#1232) (@Moelf) - Add ColPrac badge (#1317) (@oxinabox) - Change ConvTranspose with SamePad to have outsize = stride * insize (#1320) (@DrChainsaw) - change nadam cite (#1333) (@JeffFessler) - params([W, b]) to params(W, b) (#1334) (@paulxshen) - export OADAM (#1336) (@cossio) - update for Cuda 2 (#1345) (@CarloLucibello) - Fix BPTT by overriding stateful broadcast adjoint (#1358) (@DhairyaLGandhi) - Implement AdaBelief (#1362) (@willtebbutt) - Update functions.jl (#1366) (@okaerin) - Fixes #1354 (#1368) (@racinmat) - Trailing spaces (#1369) (@racinmat) - Update Slack URL (#1373) (@logankilpatrick)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 5 years ago
Flux - v0.11.1
Flux v0.11.1
Closed issues: - ADADelta not training parameters (#1158) - Improve repository's tags (#1181) - CONTRIBUTING.md missing (#1182) - Matrix times OneHotVector product does not check dimensions (#1223) - Performance issue when calculating loss (#1255) - Expose the RNGs used in initialization to the user (#1274) - DataLoader fails on tuple input (#1285) - Unnecessarily slow normalisation, twice calculating mean (#1295) - Basic example in docs fails (#1311)
Merged pull requests: - Fixed Dimension Mismatch - AbtractMatrix and OneHotVector (#1242) (@maerory) - Updated onehot.jl (#1256) (@Dsantra92) - Update links and use main page of papers instead of their PDFs (#1276) (@Hieronimo) - Corrections in the Optimisers section of documents (#1290) (@coldinjection) - Expose RNG in initializers (#1292) (@findmyway) - Change CuArrays to CUDA on docs homepage (#1297) (@scimas) - Fix ADADelta calculations and broken tests not catching the problems (#1299) (@scimas)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 6 years ago
Flux - v0.11.0
Flux v0.11.0
Closed issues: - Support for asymmetric padding (#258) - Support for Kaiming Initialization (#424) - trained recurrent model can't be saved in BSON (#531) - saving ADAM optimizer is broken [@save] BSON - BatchNorm gradients return Float64 instead of Float32 (#757) - ERROR: UndefVarError: derivative not defined (#768) - "Same" padding for conv layers? (#813) - Strange bug with Adjoint (#866) - Convolution without bias (#868) - REST API for real-time prediction (#911) - Zygote errors building bidirectional RNN (#962) - Batch aware binarycrossentropy and logitbinarycrossentropy (#1024) - Ways to freeze some part of a functor during training (#1034) - dropout function is implemented as just an identity (#1084) - revisit DataLoader api (#1088) - Dead link in documentation (#1097) - Orthogonal Initialization for RNN (#1107) - no method matching apply! (#1111) - DOC. typo in section of DataLoader (#1112) - InitError: could not load library "cudnn64_7.dll" (#1116) - How to downloading only one artifact of CUDA (#1117) - gpu function does not fully work on structs within structs (#1118) - SGD exported but not defined (#1121) - outdim not defined&dont know how to update flux from 0.90 to 0.10 (#1154) - Simple regularisation fails for Flux 0.10.4 (#1157) - DataLoader type instability (#1159) - Remove Manifest from master (#1164) - LSTM cannot be trained successfully with the latest release version (#1168) - BatchNorm failed on GPU (#1172) - ExpDecay does not decay according to the description (#1176) - Repeating crashes of NVIDIA GPU/CUDA drivers while training on basic model zoo (#1183) - Can't use Flux (#1193) - Gradient Does not work on parameterized Variable (#1196) - Wrong MaxPool gradient? (#1197) - Apply boolean mask in loss function (#1198) - Passing Number of hidden units as a float has unexpected behaviour (#1199) - Error in displying example for Flux.Dense (#1203) - Error running Flux on Jupyter (#1205) - MethodError: no method matching apply! in custom loss function (#1210) - Setting input or output layer size to a float in the Dense constructor should error (#1217) - MethodError: no method matching apply!(::Type{ADAM}, ::Array{Float64,2}, ::Array{Float64,2}) for simple example (#1219) - Incorrect gradients LSTM (#1222) - Create additional pooling layers (#1224) - ANN Forecasting with Flux (#1225) - Neural Networks for Image Segmentation (#1228) - Got an error while training on GPU with Mish activation function (#1235) - Gradient for BatchNorm no longer works (#1244) - how to restrain each element of weights to be nonnegative？ (#1250) - Retrieving weights (#1251) - Adding regularisation causes NaNs on first Epoch (#1254) - ERROR: Can't differentiate foreigncall expression (#1257) - Get wrong third order derivative of Morse potential (#1267) - ERROR: LoadError: Need an adjoint for constructor EnsembleSolution (#1270)
Merged pull requests:
- Fix for onecold broadcast bug (#764) (@DhairyaLGandhi)
- Make bias optional (#873) (@DhairyaLGandhi)
- Add option for "Same" padding to conv and pooling layers (#901) (@DrChainsaw)
- Add some gradient checking tests on GPUs (#957) (@DhairyaLGandhi)
- docstring for pad, stride, dilation (#1093) (@saswatpp)
- Explicitly import Flux.Optimiser.apply! in optimiser docs (#1113) (@SebastianCallh)
- Fix doc indent (#1123) (@matsueushi)
- Removed deprecated SGD exports (#1127) (@bhvieira)
- Added dropgrad in huberloss (#1129) (@HenriDeh)
- Update glorotnormal doc (#1131) (@AdarshKumar712)
- add ClipValue and ClipNorm (#1133) (@AStupidBear)
- Add functor Cholesky. (#1138) (@aterenin)
- Speedup matmul of CuMatrix and OneHotMatrix (#1141) (@AStupidBear)
- Cleaner training loop (#1149) (@DhairyaLGandhi)
- generalize and homogenize losses (#1150) (@CarloLucibello)
- extend dataloader (#1152) (@CarloLucibello)
- Add correct overload for apply! in docs (#1156) (@DhairyaLGandhi)
- Build docs on Julia 1.3 (#1160) (@DhairyaLGandhi)
- Update CompatHelper.yml (#1162) (@aminya)
- Fix docstring of logitcrossentropy (#1165) (@cossio)
- Fix crossentropy when some probabilities are zero (#1166) (@cossio)
- Update basics.md (#1167) (@mipals)
- Functors (#1174) (@MikeInnes)
- xlogy broadcast adjoint (#1175) (@MikeInnes)
- Align ExpDecay implementation with documentation (#1177) (@DrChainsaw)
- CompatHelper: add new compat entry for "Functors" at version "0.1" (#1179) (@github-actions[bot])
- Add some functions to docs (#1184) (@DhairyaLGandhi)
- Add some news (#1185) (@DhairyaLGandhi)
- LayerNorm regularization (#1187) (@sdobber)
- Correcting advanced.md (#1190) (@Sleort)
- Pull Request Template (#1191) (@MikeInnes)
- Improve restructure performance (#1192) (@MikeInnes)
- Fixing ambiguous remark in Preserve inputs' types (#1206) (@natema)
- Fixing typo in docs (#1207) (@natema)
- Fixing output format for onehot (#1208) (@natema)
- Fixing syntax in onehot docstring (#1211) (@natema)
- Fixing indentation in train! docstring (#1213) (@natema)
- Require weight and bias to be AbstractArrays (#1218) (@oxinabox)
- CompatHelper: bump compat for "Adapt" to "2.0" (#1220) (@github-actions[bot])
- DataLoader with NamedTuple (#1221) (@cossio)
- use ntuple in conv (#1231) (@MikeInnes)
- Fix jldoctest for Flux.Dense (#1236) (@lassepe)
- Fix inline code block (#1238) (@harryscholes)
- add adaptive pool (#1239) (@dnabanita7)
- Documentation: Move logging example outside gradient block (#1240) (@contradict)
- add kaiming initialization and relevant docstrings (#1243) (@johnnychen94)
- Optimistic ADAM (#1246) (@cossio)
- outdims: revise implementation for Chain, dimension check for Dense (#1252) (@hhaensel)
- move to CUDA.jl (#1258) (@CarloLucibello)
- improve regularisation docs (#1260) (@CarloLucibello)
- dropout function always active (#1263) (@CarloLucibello)
- create Losses module (#1264) (@CarloLucibello)
- fix a link typo in NEWS (#1265) (@johnnychen94)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] almost 6 years ago
Flux - v0.10.4
Flux v0.10.4
Closed issues: - Binary cross entropy does not work on GPUs (#464) - Cost functions don't show up in documentation (#1003) - freeze parameters (#1022) - a Tracked Array mention (#1071) - Setup BlackBoxOptim.jl and Evolutionary.jl with sciml_train (#1075) - Using Flux.train! with train and test DataLoaders? (#1081) - Function "DataLoader()" does not exist! (#1109)
Merged pull requests: - added GlobalMaxPool, GlobalMeanPool, and flatten layers (#950) (@gartangh) - Adapt to CuArrays ArrayStyle changes. (#1050) (@maleadt) - update freeze docs (#1072) (@CarloLucibello) - fix typo in the Dropout docs (#1076) (@AzamatB) - CompatHelper: bump compat for "CodecZlib" to "0.7" (#1078) (@github-actions[bot]) - CompatHelper: bump compat for "Colors" to "0.12" (#1080) (@github-actions[bot]) - Fix typo in the docstrings of AlphaDropout (#1083) (@AzamatB) - fix doc typos (#1096) (@wenjie-p) - Allow CuArrays v2.x (#1098) (@ararslan) - fix tests and new version (#1110) (@CarloLucibello)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] about 6 years ago
Flux - v0.10.3
Flux v0.10.3
Closed issues: - Conv layer don't work on with CuArray (#809)
Merged pull requests:
- Common questions answered in docs (#1028) (@dhairyagandhi96)
- Added Some Loss functions with some doc improvements (#1053) (@AdarshKumar712)
- Make really good clear examples and explination of @functor in docs (#1059) (@findmyway)
- update documenter (#1065) (@CarloLucibello)
- Updated activation functions in NNlib doc (#1069) (@AdarshKumar712)
- Prevent breakage due to new active field in normalise layers (#1070) (@ianshmean)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 6 years ago
Flux - v0.10.2
Flux v0.10.2
Closed issues:
- Training pipeline inappropriate for large datasets (#278)
- Iterators for batches and epochs (#317)
- Implicit to Explicit Parameterization of Flux Models (#742)
- crossentropy is broken with CUDA due to log (#889)
- MethodError: no method matching CuArrays.CuArray{Float32,N} where N(::Float32) (#908)
- Limitation of Flux.istraining() (#909)
- Error with regularization using norm() and Zygote (#930)
- Zygote error on moving array to GPU (#947)
- update! not working (#951)
- Gradients of Chain including leakyrelu function (#963)
- Is there a way to have layers (esp. a Conv) without biases? (#966)
- Zygote error (#967)
- Why run MNIST example is vary slow? (#968)
- model-zoo Cifar10.jl is generating "Loss is NaN" (#970)
- Handling imbalanced data (#972)
- BatchNorm is broken (#976)
- Some activation functions change type when backpropagating and pooling layers doesn't like it (#979)
- Conv layers with CPU backend randomly mixes up batch dimensions (#982)
- destructure/restructure is doing scalar indexing on GPU in back pass (#989)
- Flux pins down Colors (#995)
- Suggestion: Bounds for stochastic gradient descent loss fluctuations (#1000)
- How to keep weights of parts of a model fixed under Flux.train! (#1001)
- Support Colors.jl v0.10 and v0.11 (#1002)
- Typo in Flux home page description? Gougle? (#1004)
- Taking the package description (not too) seriously (#1007)
- le_float not differentiable: implementing reverse huber loss (#1011)
- Do you support or have any materials about optimizing with nonlinear constraints? (#1014)
- Loopinfo expression error with onecold (#1020)
- Type Promotion often Unwieldy and day Ruining (#1026)
- LoadError: MethodError: no method matching softmax(::Float32; dims=1) (#1029)
- Flux compat with Juno? (#1036)
- Failed to precompile Flux (#1045)
- train!() hasn't been export in Flux.jl (#1048)
- error with Conv (#1055)
- The most basic Conv layer fails to compute gradients (#1060)
Merged pull requests: - Added new loss functions. (#680) (@thebhatman) - Added utility function outdims to compute output dimensions of a layer (#960) (@darsnack) - Adding CompatHelper (#984) (@aminya) - Add custom training loops to docs (#994) (@oxinabox) - test restructure on the GPU (#998) (@ChrisRackauckas) - Remove unused imports. (#1008) (@maleadt) - Adapt to GPUArrays/CuArrays changes (#1013) (@maleadt) - nograd for onecold, onehot, onehotbatch (#1021) (@CarloLucibello) - Feature: Added Boston Housing Dataset (#1023) (@pranjaldatta) - Install TagBot as a GitHub Action (#1030) (@JuliaTagBot) - Remove outdated reference to truncate! (#1032) (@mcognetta) - Remove get! macro (#1035) (@matsueushi) - update compat to Juno 0.8 (#1037) (@heliosdrm) - add NNlib docs + misc docs improvements (#1041) (@CarloLucibello) - Add testmode! back for normalization layers (#1044) (@darsnack) - Bump Colors compat to include 0.10, 0.11 (#1046) (@ianshmean) - Edit description of convolutional layer (#1047) (@MotJuMi) - add DataLoader (#1051) (@CarloLucibello) - update docs and export update! (#1052) (@CarloLucibello) - add Julia ecosystem doc section (#1057) (@CarloLucibello) - fix a few typos in docstrings (#1061) (@visr) - docstring ensure signature code formatting (#1062) (@visr) - Include cuda/cuda.jl during precompilation? (#1064) (@ianshmean) - fix travis for documentation build (#1066) (@johnnychen94)
Scientific Software - Peer-reviewed
- Julia
Published by github-actions[bot] over 6 years ago
Flux - v0.10.1
v0.10.1 (2020-01-13)
Closed issues:
- destructure then restructure fails (#988)
- ConvTranspose produces nonsense results for 3D (aka 5D) inputs (#978)
- Missing Tracker dependency causes DiffEqFlux v0.4.0 and DifferentialEquations v6.6.0 fail to pre-compile in (J V1.3) (#975)
- UndefVarError: @functor not defined (#974)
- A problem with the first example (#959)
- Error when compiling model in GPU (#956)
- Gradient on a Neural Network (#955)
- Why the result of Flux.jl is totally different from tf.Keras (with the same simple MLP) (#953)
- CuArrays failed to initialize (#949)
Merged pull requests:
- Compat bounds for a couple more packages (#992) (dhairyagandhi96)
- Update CuArrays + Zygote deps (#991) (dhairyagandhi96)
- Destructure/restructure for models (#986) (MikeInnes)
- Give
NNPACKa bit of numerical leeway (#973) (staticfloat) - [WIP] Decaydocs (#954) (baggepinnen)
Scientific Software - Peer-reviewed
- Julia
Published by julia-tagbot[bot] over 6 years ago
Flux - v0.10.0
v0.10.0 (2019-11-29)
Closed issues:
- filtering a subset of parameters (#939)
- import Flux fails with errors about libcublas (#933)
- UndefVarError: project not defined (#931)
- Warning on fresh Flux install (#929)
- Looking for libcuda and failing on a device without CUDA (#928)
- Spurious RNN failure with CUDNN (#923)
train!method forChain(#921)- LoadError: LoadError: UndefVarError: libcudnn not defined (#918)
- Failed to precompile Flux (#917)
- Zygote: add adjoint for onehotbatch (#912)
- How to convert Tracked Number in Number? (#905)
- ConvTranspose gradient error: no method matching iterate(::Nothing) (#900)
- Gradient Through Trebuchet.jl Error (#897)
- Jacobian for Conv layers (#896)
- t (#895)
- can't find MNIST (#894)
- Regularisation docs missing norm() definition (#892)
- Flux master does not precompile (#881)
- circshift breaks Tracker.gradient, but manual shifting works (#872)
- Convolution without bias (#868)
- Github stars in pkg.julialang.org (#867)
- Typo in Optimisers Doc (#862)
- Can't use Flux.Tracker (#859)
- CUBLASError(code 13, the GPU program failed to execute) on seemingly simple applychain() (#858)
- Should one(::TrackedArray) and zero(::TrackedArray) return TrackedArrays, instead of normal arrays? (#855)
- Flux fails its own tests when run with CuArrays (#854)
- Gradient does not converge on simple dataset (#849)
- MethodError when taking gradient of Flux.mse on GPU (#848)
- RAdam Optimizer (#841)
- Optimiser interface is not documented (#836)
- Flux#zygote: conv_transpose_dims mutates arrays. (#820)
- LinearAlgebra.Transpose{Float64,Array{Float64,2}} multiplied by Tracked 2-element Array{Float64,1} doesn't work (#743)
- Glorot initialization not correctly implemented (#442)
- RNN test failures with CUDNN (#267)
- OpenCL support (#173)
Merged pull requests:
- compat, pkg up (#946) (MikeInnes)
- RNN failure hackaround (#944) (MikeInnes)
- Fixes #900 (#943) (dhairyagandhi96)
- Don't include the CUDA module during precompilation. (#941) (maleadt)
- Fix logitbinarycrossentropy on CuArrays (#940) (matsueushi)
- Fix Glorot initialization, add He initialization (#937) (Sleort)
- Avoid unnecessary conversion (#936) (baggepinnen)
- Fix AMSGrad on GPU (#935) (baggepinnen)
- Travis: test on 1.0 (#932) (MikeInnes)
- Extend docs about
train!(#927) (heliosdrm) - Fix binarycrossentropy on CuArrays (#926) (janEbert)
- CUDA package initialization improvements (#924) (maleadt)
- Restore Julia 1.0 compatibility. (#922) (maleadt)
- use release versions of packages (#920) (MikeInnes)
- Check for CUDA availability at run time. (#916) (maleadt)
- Change
gatefunction toviewinstead of copy (#907) (janEbert) - Backticks and examples for normalise (#902) (kshyatt)
- Fix problem in crossentropy breaking GPU compilation (#898) (kshyatt)
- Check if CUDA availability changed during init. (#882) (maleadt)
- Fix functor's
params!to work with complex numbers (#877) (PhilipVinc) - Move CUDNN wrappers to CuArrays (#874) (MikeInnes)
- Fix printing of SkipConnection (#870) (mcabbott)
- Functor (#865) (MikeInnes)
- removed extra parenthesis (#863) (Naba7)
- GPU CI maintainance (#861) (dhairyagandhi96)
- Activations (#860) (dsweber2)
- Restore purity (#857) (giordano)
- Add RADAM optimizer (#842) (baggepinnen)
- using Zygote (#669) (MikeInnes)
Scientific Software - Peer-reviewed
- Julia
Published by julia-tagbot[bot] over 6 years ago
Flux - v0.9.0
v0.9.0 (2019-08-29)
Closed issues:
- CuArrays.CUDNN.libcudnn not defined (#846)
- Flux.onecold gives incorrect result with CuArrays. (#839)
- Forward differentiation causes stackoverflow (#831)
- Layer initialization with predefined weights (#830)
- RNN behavior under different input shapes (#811)
- prefor does not update seen (#803)
- Flux is not training model (#798)
- How to Detrack the variables (#792)
- Optimisers not work for real number (#790)
- apply! in NADAM not correct (#789)
- How to onehot encode a vector (#788)
- ERROR: UndefVarError: Conv2D not defined (#786)
- Difference between params() and Params() (#781)
- Slowdown with Conv on CPU in newer versions of Flux+NNlib: forward pass doesn't use all available threads anymore (#771)
- CUDNN BatchNorm with relu doesn't apply relu (when using CuArrays) (#760)
- back! calculates gradient incorrectly for two layer perceptron with relu (#744)
- Split out Tracker and create a Zygote branch (#664)
- ForwardDiff derivative on chain gives stackoverflow (#645)
- Calling a gpu model with a cpu array crashes julia (#581)
Merged pull requests:
- RFC: Replace Requires with direct CuArrays dependency. (#852) (maleadt)
- Fix CuArrays.libcudnn imports (#847) (janEbert)
- Use
CuArrays.onesinsteadcuoneswhich is deprecated (#837) (mimame) - Fix cuzeros deprecation (#835) (Moelf)
- Momentum doesn't need params (#827) (ChrisRackauckas)
- Fix for #803 (#805) (DrChainsaw)
- Fix lack of x (#801) (quatrejuin)
- Pick beta from the state - NADAM (#796) (dhairyagandhi96)
- Two minor typos in docs (#793) (amellnik)
- typo of comvolutional in NEWS.md (#774) (zsz00)
- delete redundant section (#772) (johnnychen94)
- Some cleanup on performance tips docs (#767) (oxinabox)
- CrossCor layer (#762) (ayush-1506)
- Fixes #760 (#761) (avik-pal)
- bump version to v0.8.3 (#759) (dhairyagandhi96)
- Change
DepthwiseConv\(\)to usein=\>outinstead ofin=\>mult. (#756) (staticfloat) - Fixed ExpDecay (#733) (thebhatman)
- add broken test for #700 (#701) (jw3126)
- Fixes OneHotMatrix/Vector GPU Performance (#612) (dhairyagandhi96)
- noise shape for dropout (#563) (chengchingwen)
- Added the SkipConnection layer and constructor (#446) (bhvieira)
Scientific Software - Peer-reviewed
- Julia
Published by julia-tagbot[bot] almost 7 years ago