learn2learn

A PyTorch Library for Meta-learning Research

https://github.com/learnables/learn2learn

Keywords

few-shot finetuning learn2learn learning2learn maml meta-descent meta-learning meta-optimization meta-rl metalearning pytorch

Keywords from Contributors

jax interpretability distribution interactive medical-image-processing healthcare-imaging monai kedro deep-neural-networks network-simulation

Last synced: 6 months ago · JSON representation ·

Repository

A PyTorch Library for Meta-learning Research

Basic Info

Host: GitHub
Owner: learnables
License: mit
Language: Python
Default Branch: master
Homepage: http://learn2learn.net
Size: 9.52 MB

Statistics

Stars: 2,825
Watchers: 30
Forks: 362
Open Issues: 32
Releases: 11

Topics

few-shot finetuning learn2learn learning2learn maml meta-descent meta-learning meta-optimization meta-rl metalearning pytorch

Created over 6 years ago · Last pushed over 1 year ago

Metadata Files

Readme Changelog License Citation

README.md

learn2learn is a software library for meta-learning research.

learn2learn builds on top of PyTorch to accelerate two aspects of the meta-learning research cycle:

fast prototyping, essential in letting researchers quickly try new ideas, and
correct reproducibility, ensuring that these ideas are evaluated fairly.

learn2learn provides low-level utilities and unified interface to create new algorithms and domains, together with high-quality implementations of existing algorithms and standardized benchmarks. It retains compatibility with torchvision, torchaudio, torchtext, cherry, and any other PyTorch-based library you might be using.

To learn more, see our whitepaper: arXiv:2008.12284

Overview

learn2learn.data: Taskset and transforms to create few-shot tasks from any PyTorch dataset.
learn2learn.vision: Models, datasets, and benchmarks for computer vision and few-shot learning.
learn2learn.gym: Environment and utilities for meta-reinforcement learning.
learn2learn.algorithms: High-level wrappers for existing meta-learning algorithms.
learn2learn.optim: Utilities and algorithms for differentiable optimization and meta-descent.

Resources

Website: http://learn2learn.net/
Documentation: http://learn2learn.net/docs/learn2learn
Tutorials: http://learn2learn.net/tutorials/getting_started/
Examples: https://github.com/learnables/learn2learn/tree/master/examples
GitHub: https://github.com/learnables/learn2learn/
Slack: http://slack.learn2learn.net/

Installation

~~~bash pip install learn2learn ~~~

Snippets & Examples

The following snippets provide a sneak peek at the functionalities of learn2learn.

High-level Wrappers

Few-Shot Learning with MAML

For more algorithms (ProtoNets, ANIL, Meta-SGD, Reptile, Meta-Curvature, KFO) refer to the examples folder. Most of them can be implemented with with the `GBML` wrapper. (documentation). ~~~python maml = l2l.algorithms.MAML(model, lr=0.1) opt = torch.optim.SGD(maml.parameters(), lr=0.001) for iteration in range(10): opt.zero_grad() task_model = maml.clone() # torch.clone() for nn.Modules adaptation_loss = compute_loss(task_model) task_model.adapt(adaptation_loss) # computes gradient, update task_model in-place evaluation_loss = compute_loss(task_model) evaluation_loss.backward() # gradients w.r.t. maml.parameters() opt.step() ~~~

Meta-Descent with Hypergradient

Learn any kind of optimization algorithm with the `LearnableOptimizer`. (example and documentation) ~~~python linear = nn.Linear(784, 10) transform = l2l.optim.ModuleTransform(l2l.nn.Scale) metaopt = l2l.optim.LearnableOptimizer(linear, transform, lr=0.01) # metaopt has .step() opt = torch.optim.SGD(metaopt.parameters(), lr=0.001) # metaopt also has .parameters() metaopt.zero_grad() opt.zero_grad() error = loss(linear(X), y) error.backward() opt.step() # update metaopt metaopt.step() # update linear ~~~

Learning Domains

Custom Few-Shot Dataset

Many standardized datasets (Omniglot, mini-/tiered-ImageNet, FC100, CIFAR-FS) are readily available in `learn2learn.vision.datasets`. (documentation) ~~~python dataset = l2l.data.MetaDataset(MyDataset()) # any PyTorch dataset transforms = [ # Easy to define your own transform l2l.data.transforms.NWays(dataset, n=5), l2l.data.transforms.KShots(dataset, k=1), l2l.data.transforms.LoadData(dataset), ] taskset = Taskset(dataset, transforms, num_tasks=20000) for task in taskset: X, y = task # Meta-train on the task ~~~

Environments and Utilities for Meta-RL

Parallelize your own meta-environments with `AsyncVectorEnv`, or use the standardized ones. (documentation) ~~~python def make_env(): env = l2l.gym.HalfCheetahForwardBackwardEnv() env = cherry.envs.ActionSpaceScaler(env) return env env = l2l.gym.AsyncVectorEnv([make_env for _ in range(16)]) # uses 16 threads for task_config in env.sample_tasks(20): env.set_task(task) # all threads receive the same task state = env.reset() # use standard Gym API action = my_policy(env) env.step(action) ~~~

Low-Level Utilities

Differentiable Optimization

Learn and differentiate through updates of PyTorch Modules. (documentation) ~~~python model = MyModel() transform = l2l.optim.KroneckerTransform(l2l.nn.KroneckerLinear) learned_update = l2l.optim.ParameterUpdate( # learnable update function model.parameters(), transform) clone = l2l.clone_module(model) # torch.clone() for nn.Modules error = loss(clone(X), y) updates = learned_update( # similar API as torch.autograd.grad error, clone.parameters(), create_graph=True, ) l2l.update_module(clone, updates=updates) loss(clone(X), y).backward() # Gradients w.r.t model.parameters() and learned_update.parameters() ~~~

Changelog

A human-readable changelog is available in the CHANGELOG.md file.

Citation

To cite the learn2learn repository in your academic publications, please use the following reference.

Arnold, Sebastien M. R., Praateek Mahajan, Debajyoti Datta, Ian Bunner, and Konstantinos Saitas Zarkias. 2020. “learn2learn: A Library for Meta-Learning Research.” arXiv [cs.LG]. http://arxiv.org/abs/2008.12284.

You can also use the following Bibtex entry.

~~~bib @article{Arnold2020-ss, title = "learn2learn: A Library for {Meta-Learning} Research", author = "Arnold, S{\'e}bastien M R and Mahajan, Praateek and Datta, Debajyoti and Bunner, Ian and Zarkias, Konstantinos Saitas", month = aug, year = 2020, url = "http://arxiv.org/abs/2008.12284", archivePrefix = "arXiv", primaryClass = "cs.LG", eprint = "2008.12284" }

~~~

Acknowledgements & Friends

TorchMeta is similar library, with a focus on datasets for supervised meta-learning.
higher is a PyTorch library that enables differentiating through optimization inner-loops. While they monkey-patch nn.Module to be stateless, learn2learn retains the stateful PyTorch look-and-feel. For more information, refer to their ArXiv paper.
We are thankful to the following open-source implementations which helped guide the design of learn2learn:
- Tristan Deleu's pytorch-maml-rl
- Jonas Rothfuss' ProMP
- Kwonjoon Lee's MetaOptNet
- Han-Jia Ye's and Hexiang Hu's FEAT

Owner

Name: learnables
Login: learnables
Kind: organization

Repositories: 2
Profile: https://github.com/learnables

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Arnold"
  given-names: "Sebastien M. R."
- family-names: "Mahajan"
  given-names: "Praateek"
- family-names: "Datta"
  given-names: "Debajyoti"
- family-names: "Bunner"
  given-names: "Ian"
- family-names: "Saitas Zarkias"
  given-names: "Konstantinos"
title: "learn2learn: A Library for Meta-Learning Research"
version: 0.1.5
date-released: 2020-08-27
url: "https://github.com/learnables/learn2learn"

GitHub Events

Total

Issues event: 2
Watch event: 175
Issue comment event: 5
Fork event: 19

Last Year

Issues event: 2
Watch event: 175
Issue comment event: 5
Fork event: 19

Committers

Last synced: 9 months ago

All Time

Total Commits: 302
Total Committers: 31
Avg Commits per committer: 9.742
Development Distribution Score (DDS): 0.414

Past Year

Commits: 0
Committers: 0
Avg Commits per committer: 0.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
Séb Arnold	s**1@h**m	177
Praateek Mahajan	p**m@g**m	55
debajyotidatta	d**2@g**m	18
Varad Pimpalkhute	3****n	8
Ian Bunner	b**r@u**u	6
Debajyoti Datta	d**o@D**l	4
Kostis-S-Z	K****Z	4
8bitmp3	1****3	4
Jan Bollenbacher	j**n@b**o	3
Théo Morales	t**r@g**m	2
Joel Joseph	3****n	1
Jorge	4****f	1
Kristian Georgiev	k**g@m**u	1
Mayug Maniparambil	3****g	1
Michael Moor	m**r@g**m	1
Nimish	n**i@g**m	1
EVKrikunov	4****V	1
Ethan Harris	e**3@s**k	1
Farzam khodajoo	a**o@g**m	1
Isak Falk	i**k@l**e	1
vfdev	v**5@g**m	1
kzhang2	k**g@g**m	1
joemzhao	j****o	1
dependabot[bot]	4****]	1
Zhaofeng Wu	w**7@g**m	1
Tsam Kiu Pun	e**n@g**m	1
TrellixVulnTeam	1****m	1
Tim	t**m@w**e	1
Tianye Shu	4****y	1
Stergiadis Manos	s**7@g**m	1
and 1 more...

Committer Domains (Top 20 + Academic)

weiland-lahnstein.de: 1 live.se: 1 soton.ac.uk: 1 mit.edu: 1 bollenbacher.info: 1 usc.edu: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 168
Total pull requests: 50
Average time to close issues: 4 months
Average time to close pull requests: 2 months
Total issue authors: 76
Total pull request authors: 14
Average comments per issue: 3.58
Average comments per pull request: 1.58
Merged pull requests: 37
Bot issues: 0
Bot pull requests: 1

Past Year

Issues: 2
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 2
Pull request authors: 0
Average comments per issue: 0.0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

brando90 (27)
jkang1640 (5)
farzam-khodajoo (3)
patricks-lab (3)
AntreasAntoniou (2)
DubiousCactus (2)
joshuasv (2)
swaggyP9527 (2)
seba-1511 (2)
CorleoneJW (1)
furkanpala (1)
Jeong-Bin (1)
fankaisheng (1)
xuliwalker (1)
gsygsy96 (1)

Pull Request Authors

seba-1511 (8)
DubiousCactus (5)
nightlessbaron (5)
TroddenSpade (2)
qthequartermasterman (1)
SliverySky (1)
mi92 (1)
steremma (1)
jorgectf (1)
KrikunovEV (1)
TrellixVulnTeam (1)
farzam-khodajoo (1)

Top Labels

Issue Labels

help wanted (1) good first issue (1)

Pull Request Labels

Packages

Total packages: 3
Total downloads:
- pypi 1,682 last-month
Total docker downloads: 62

Total dependent packages: 1
(may contain duplicates)
Total dependent repositories: 55
(may contain duplicates)
Total versions: 35
Total maintainers: 1

pypi.org: learn2learn

PyTorch Library for Meta-Learning Research

Homepage: https://github.com/learnables/learn2learn
Documentation: https://learn2learn.readthedocs.io/
License: MIT
Latest release: 0.2.0
published over 2 years ago

Versions: 19
Dependent Packages: 1
Dependent Repositories: 54
Downloads: 1,671 Last month
Docker Downloads: 62

Rankings

Stargazers count: 1.5%

Dependent repos count: 2.0%

Forks count: 2.9%

Docker downloads count: 3.0%

Average: 3.5%

Dependent packages count: 4.8%

Downloads: 6.7%

Maintainers (1)

seba-1511

Last synced: 6 months ago

proxy.golang.org: github.com/learnables/learn2learn

Documentation: https://pkg.go.dev/github.com/learnables/learn2learn#section-documentation
License: mit
Latest release: v0.2.1
published over 2 years ago

Versions: 10
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 5.6%

Average: 5.8%

Dependent repos count: 5.9%

Last synced: 6 months ago

pypi.org: learn2learn-dev

PyTorch Meta-Learning Framework for Researchers

Homepage: https://github.com/learnables/learn2learn
Documentation: https://learn2learn-dev.readthedocs.io/
License: MIT
Latest release: 0.1.0
published almost 6 years ago

Versions: 6
Dependent Packages: 0
Dependent Repositories: 1
Downloads: 11 Last month

Rankings

Stargazers count: 1.5%

Forks count: 2.9%

Dependent packages count: 10.1%

Average: 14.2%

Dependent repos count: 21.5%

Downloads: 35.0%

Maintainers (1)

seba-1511

Last synced: 6 months ago

learn2learn

Science Score: 64.0%

Keywords

Keywords from Contributors

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

Installation

Snippets & Examples

High-level Wrappers

Learning Domains

Low-Level Utilities

Changelog

Citation

Acknowledgements & Friends

Owner

Citation (CITATION.cff)

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: learn2learn

Rankings

Maintainers (1)

proxy.golang.org: github.com/learnables/learn2learn

Rankings

pypi.org: learn2learn-dev

Rankings

Maintainers (1)