https://github.com/cedrickchee/ranger-optimizer
Ranger - a synergistic optimizer using RAdam (Rectified Adam) and Lookahead in one codebase
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (5.3%) to scientific vocabulary
Keywords
deep-neural-network
lookahead-methods
optimization-algorithms
radam
ranger-optimizer
Last synced: 5 months ago
·
JSON representation
Repository
Ranger - a synergistic optimizer using RAdam (Rectified Adam) and Lookahead in one codebase
Basic Info
Statistics
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
- Releases: 0
Fork of lessw2020/Ranger-Deep-Learning-Optimizer
Topics
deep-neural-network
lookahead-methods
optimization-algorithms
radam
ranger-optimizer
Created over 6 years ago
· Last pushed over 6 years ago
https://github.com/cedrickchee/ranger-optimizer/blob/master/
# [Ranger Neural Network Optimizer](https://forums.fast.ai/t/meet-ranger-radam-lookahead-optimizer/52886?u=cedric) Ranger a synergistic optimizer combining RAdam (Rectified Adam) and Lookahead in one codebase. > - RAdam stabilizes training at the start. (reference: ["On the Variance of the Adaptive Learning Rate and Beyond"](https://arxiv.org/abs/1908.03265v1) paper by Liyuan Liu et al.) > > - Lookahead stabilizes training and convergence during the rest of training. (reference: ["Lookahead Optimizer: k steps forward, 1 step back"](https://arxiv.org/abs/1907.08610v1) paper by Michael Zhang, Geoffrey Hinton, et al.) ## Examples _Usage and notebook to test with coming shortly._ ## Software Requirements - Python 3.x - PyTorch 1.x ## License [Apache License 2.0](./LICENSE) ## Credits Based on original work done by [Less Wright (lessw2020)](https://github.com/lessw2020). He built on LonePatient's implementation for the Lookahead part, and the official RAdam code for RAdam: - Lookahead implementation from LonePatient - https://github.com/lonePatient/lookahead_pytorch/blob/master/optimizer.py - RAdam code by https://github.com/LiyuanLucasLiu/RAdam/blob/master/radam.py
Owner
- Name: Cedric Chee
- Login: cedrickchee
- Kind: user
- Location: PID 1
- Company: InvictusByte
- Website: https://cedricchee.com
- Twitter: cedric_chee
- Repositories: 227
- Profile: https://github.com/cedrickchee
Lead Software Engineer | LLMs | full stack Go/JS dev, backend | product dev @ startups | 🧑🎓 CompSci | alumni: fast.ai, Antler.co