norse - v1.1 Support for NIR and torch.compile

This release features a lot of quality-of-life improvements. Most notably, we started testing for torch.compile support, which gives us significant speedup. The improvements gained by moving to torch.compile meant that we could safely remove the C++ code such that Norse now is a Python-only module. That means the installations should be significantly faster.

We also added initialization methods for spatial and temporal receptive fields, added support for NIR, cleaned up the docs, restructured the imports, removed unnecessary (and slow) try-catch clauses, and cleaned up dependencies.

New state tuples

We also added tentative support for a new StateTuple implementation based on PyTorch's pytrees, which makes it easier to operate on parameters. This allows us to cast parameters to devices or

python p = LIFParameters(...) # Create a parameter p.to("cuda:0") # Cast the parameters to a device p.float() # Cast the parameters to floats. Note that this is currently only implemented for LIFParameters and LIFBoxParameters. Let us know how it works!

What's Changed

• Feature li box by @Jegp in https://github.com/norse/norse/pull/367
• Feature rf by @Jegp in https://github.com/norse/norse/pull/368
• Added pytree state and support for torch.compile by @Jegp in https://github.com/norse/norse/pull/349
• Adds NIR export by @Jegp in https://github.com/norse/norse/pull/371
• Constrain pytorch-lightning version in example tasks README by @4iar in https://github.com/norse/norse/pull/372
• Feature rf by @Jegp in https://github.com/norse/norse/pull/376
• Feature docs by @Jegp in https://github.com/norse/norse/pull/380
• Feature nir by @Jegp in https://github.com/norse/norse/pull/379
• Fix typos by @omahs in https://github.com/norse/norse/pull/385
• Added tests for torch.compile by @Jegp in https://github.com/norse/norse/pull/386
• Improves spatial receptive field API by @Jegp in https://github.com/norse/norse/pull/387
• Remove broken workflows by @Jegp in https://github.com/norse/norse/pull/391

New Contributors

• @4iar made their first contribution in https://github.com/norse/norse/pull/372
• @omahs made their first contribution in https://github.com/norse/norse/pull/385

Full Changelog: https://github.com/norse/norse/compare/v1.0.0...v1.1.0

- Python
Published by Jegp almost 2 years ago

norse - v1.0.0 - First Stable Release

This is the first stable release of Norse. We feel like that after close to almost 4 years of development it is time to take this step. The API has stabilised somewhat and while we anticipate some changes in the future, we will try to do them in ways that are easy to accommodate for users. Since the last release we mostly focussed on bugs and worked on performance. We also got some nice additions

• Feature spikes to times decoder (#321): A differentiable way of encoding spikes to times.
• Feature no delay (#326): We changed the integration order of most models
• Feature AdEx refractory (#291): Adaptive Exponential Fire Neuron with refractory state

We also worked on improvements to documentation, our continuous integration and build tooling.

- Python
Published by cpehle about 3 years ago

norse - v0.0.7 Neuron models and stability improvements

This release includes prototypical sparse and adjoint equations, neuron models, utilities, and various stability fixes.

Specifically, we included * Izhikevich and IAF neuron models (thanks to @adelpierre) * Discrete adjoint sensitivity and sparse equations for LIF dynamics * Efficient convolutions in time (LConv2d) * Triangular surrogate gradients (thanks to @Huizerd) * Preliminary plotting primitives * Usage improvements for tensor datatypes (from, for instance, Tonic) and parameter settings for JIT optimized code * Increased code coverage * Documentation improvements * Stability improvements around C++ code, sequential state models, and tasks * Numerous bug fixes, e.g. around recurrent autapses * Nix support

- Python
Published by Jegp over 4 years ago

norse - Sparse and adjoint code

- Python
Published by Jegp over 4 years ago

norse - Sparse and adjoint code

RC3 for the sparse and adjoint code. Aims to resolve builds for Windows and Linux

- Python
Published by Jegp over 4 years ago

norse - Sparse and adjoint code

RC2 for the sparse and adjoint code. Aims to resolve builds for Windows and Linux

- Python
Published by Jegp over 4 years ago

norse - Sparse and adjoint code

This release candidate drafts code for sparse activations and adjoint-based optimizations as described in https://arxiv.org/abs/2009.08378

- Python
Published by Jegp over 4 years ago

norse - Streamlined Module API

This release features our shiny and new module API, it unifies all Spiking Neuron modules under one common base class thereby eliminating redundant code.

From a user perspective it also means that the API is now consistent across all Neuron types.

- Python
Published by cpehle about 5 years ago

norse - Optimizations, deep-learning models, and plasticity

This release brings numerous improvements in terms of speed, usability, specializations, documentation and more. In general, we tried to make Norse more user-friendly and applicable for both the die-hard deep-learning expert and neuroscience enthusiasts new to Python. Specifically, this release includes: * Compatibility with the PyTorch Lightning library, which means that Norse now scales to multiple GPUs and even supercomputing clusters with SLURM. As an example, see our MNIST task. * The SequentialState module, which works similar to PyTorch's Sequential layers in that it allows for seamless composition of PyTorch and Norse modules. Together with the Lift module, this is an important step towards powerful and simple tools for developing spiking neural networks. * As Norse becomes faster to work with, it is also easier to implement more complex models. Norse now features spiking convolutions, MobileNet and VGG networks which can be used out-of-the box. See the norse.torch.models package for more information. * Improved performance. We implemented the LIF neuron equations and the SuperSpike synthetic gradient in C++. All in all, Norse is roughly twice as fast as it was before. * Improved documentation. The main pages and the introductory pages were edited and cleaned up. This is an area we will be improving much more in the future. * Various bugfixes. Norse is now more stable and useable than before.

As always, we welcome feedback and are looking forward to hearing how you are using Norse! Happy hacking :partying_face:

- Python
Published by Jegp about 5 years ago

norse - STDP, neuron models, and PyTorch compatibility

This release contains a number of functionality and model additions, as well as improved PyTorch compatibility through the Lift module. Most notably, we

• Added spike-time plasticity
• Added regularization for spiking cells/layers
• Added a layer for Lifting regular PyTorch layers to work with temporal data
• Improved usability by
  • cleaning up neuron model parameters and
  • inferring initial neuron state
  • inferring device parameter

- Python
Published by cpehle over 5 years ago

norse - Models, performance testing and logo

This release includes biologically plausible neuron parameters, performance testing (including comparison to like-minded frameworks), and a new shiny logo!

- Python
Published by Jegp over 5 years ago

norse - Stability changes

This release includes a number of stability changes and documentation additions. Most significantly, some inconsistencies in neuron model flags have been fixed, and the MNIST, CIFAR, and cartpole tasks, have been tested on both CPU and GPU backends.

- Python
Published by Jegp about 6 years ago