https://github.com/bgavran/compositional_deep_learning
Deep learning via category theory and functional programming
Science Score: 36.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.4%) to scientific vocabulary
Repository
Deep learning via category theory and functional programming
Basic Info
Statistics
- Stars: 145
- Watchers: 14
- Forks: 8
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
Compositional Deep Learning
Update May 2020: It's important to note that while eventual implementation of neural networks from first principles is my main goal, I've currently shifted to studying them from a category-theoretic perspective. This is being done in collaboration with some other people and is mostly theory right now. This is why for a while there hasn't been any developments in this repo, although I plan on getting back to it.
This is a long-term project of understanding and reimplementing neural networks from first principles using the language of category theory. It's a compositional approach to organizing and layering different abstractions found in neural networks: * Differentiation of composition of functions (forward mode, reverse mode) * Notion of composition of parametrized functions * Notion of a cost function (fixed or adversarial) * Notion of an update rule * Notion of meta learning * Notion of multi-agent neural networks
As more and more components of our deep learning systems stop being fixed throughout training, there is an increasingly larger need for more precise formal specification of the things that do stay fixed. Standard methods don't seem to be as effective: the invariants across all these networks seem to be rather abstract and hard to describe. This repository explores the speculation that the language of category theory could be well suited to describe and quantify these structures.
Focus is currently not on reimplementation of popular new network architectures, but rather on principled, structured design of both neural networks and the way they are trained.
This is research level stuff and at the moment not really usable for anything other than playing around.
Related work:
A tiny example of simple linear regression can be found in Examples.hs
Progress so far: * Main parts of SimpleAD paper are fully implemented * Generalized, mode-independent automatic differentiation with rudimentary functions * Backpropagation (which is just a specialization of GAD to the dual category of additive functions) * Implemented category Para from BackpropFunctor (in a slightly different way than in the paper, such that the request function in the specified form isn't really needed)
TODO: * Find a suitable tensor library with the following features: * Static tensor shapes, known and type-checked at compile time * Some variant of Einstein summation notation for handling of tensors of arbitrary rank I'm working on one such library in Idris, which implements Dependently Typed Einstein Summation. * Provide working examples of training simple neural networks * Find a way to graphically show composition of GAD, Para and Learners * Explore using effects for data loading
This is a heavy work in progress, but feel free to drop me a message with any questions!
Owner
- Name: Bruno Gavranović
- Login: bgavran
- Kind: user
- Location: London, United Kingdom
- Company: Symbolica
- Website: www.brunogavranovic.com
- Repositories: 26
- Profile: https://github.com/bgavran
Principal Scientist - Categorical Deep Learning @symbolica-ai
GitHub Events
Total
- Watch event: 3
Last Year
- Watch event: 3
Committers
Last synced: 12 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| bgavran3 | b****c@f****r | 89 |
| Bruno Gavranović | b****o@b****m | 3 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 12 months ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- CompositionalDeepLearning *
- NumInstances *
- base >=4.7 && <5
- hTensor *
- lens *
- random *
- text *
- CompositionalDeepLearning * test
- base >=4.7 && <5 test