Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/codingfisch/flashrl

Fast reinforcement learning 💨

https://github.com/codingfisch/flashrl

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.4%) to scientific vocabulary

Keywords

multi-agent-reinforcement-learning reinforcement-learning reinforcement-learning-environments
Last synced: 10 months ago · JSON representation

Repository

Fast reinforcement learning 💨

Basic Info
  • Host: GitHub
  • Owner: codingfisch
  • License: mit
  • Language: Cython
  • Default Branch: main
  • Homepage:
  • Size: 74.2 KB
Statistics
  • Stars: 25
  • Watchers: 1
  • Forks: 1
  • Open Issues: 0
  • Releases: 0
Topics
multi-agent-reinforcement-learning reinforcement-learning reinforcement-learning-environments
Created over 1 year ago · Last pushed 12 months ago
Metadata Files
Readme License

README.md

flashrl

flashrl does RL with millions of steps/second 💨 while being tiny: ~200 lines of code

🛠ī¸ pip install flashrl or clone the repo & pip install -r requirements.txt - If cloned (or if envs changed), compile: python setup.py build_ext --inplace

💡 flashrl will always be tiny: Read the code (+paste into LLM) to understand it!

Quick Start 🚀

flashrl uses a Learner that holds an env and a model (default: Policy with LSTM)

```python import flashrl as frl

learn = frl.Learner(frl.envs.Pong(nagents=2**14)) curves = learn.fit(40, steps=16, desc='done') frl.printcurve(curves['loss'], label='loss') frl.play(learn.env, learn.model, fps=8) learn.env.close() `` .fitdoes RL with ~**10 million steps**:40iterations ×16steps ×2**14` agents!

Run it yourself via python train.py and play against the AI đŸĒ„

Click here, to read a tiny doc 📑 `Learner` takes the arguments - `env`: RL environment - `model`: A `Policy` model - `device`: Per default picks `mps` or `cuda` if available else `cpu` - `dtype`: Per default `torch.bfloat16` if device is `cuda` else `torch.float32` - `compile_no_lstm`: Speedup via `torch.compile` if `model` has no `lstm` - `**kwargs`: Passed to the `Policy`, e.g. `hidden_size` or `lstm` `Learner.fit` takes the arguments - `iters`: Number of iterations - `steps`: Number of steps in `rollout` - `desc`: Progress bar description (e.g. `'reward'`) - `log`: If `True`, `tensorboard` logging is enabled - run `tensorboard --logdir=runs`and visit `http://localhost:6006` in the browser! - `stop_func`: Function that stops training if it returns `True` e.g. ```python ... def stop(kl, **kwargs): return kl > .1 curves = learn.fit(40, steps=16, stop_func=stop) ... ``` - `lr`, `anneal_lr` & args of `ppo` after `bs`: Hyperparameters The most important functions in `flashrl/utils.py` are - `print_curve`: Visualizes the loss across the `iters` - `play`: Plays the environment in the terminal and takes - `model`: A `Policy` model - `playable`: If `True`, allows you to act (or decide to let the model act) - `steps`: Number of steps - `fps`: Frames per second - `obs`: Argument of the env that should be rendered as observations - `dump`: If `True`, no frame refresh -> Frames accumulate in the terminal - `idx`: Agent index between `0` and `n_agents` (default: `0`)

Environments 🕹ī¸

Each env is one Cython(=.pyx) file in flashrl/envs. That's it!

To add custom envs, use grid.pyx, pong.pyx or multigrid.pyx as a template: - grid.pyx for single-agent envs (~110 LOC) - pong.pyx for 1 vs 1 agent envs (~150 LOC) - multigrid.pyx for multi-agent envs (~190 LOC)

| Grid | Pong | MultiGrid | |-----------------------|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------| | Agent must reach goal | Agent must score | Agent must reach goal first | |grid| pong| multigrid|

Acknowledgements 🙌

I want to thank - Joseph Suarez for open sourcing RL envs in C(ython)! Star PufferLib ⭐ - Costa Huang for open sourcing high-quality single-file RL code! Star cleanrl ⭐

and last but not least...

Owner

  • Login: codingfisch
  • Kind: user

GitHub Events

Total
  • Watch event: 22
  • Delete event: 2
  • Push event: 96
  • Fork event: 1
  • Create event: 10
Last Year
  • Watch event: 22
  • Delete event: 2
  • Push event: 96
  • Fork event: 1
  • Create event: 10

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

pyproject.toml pypi
  • python ^3.9
setup.py pypi
.github/workflows/publish.yml actions
  • actions/checkout v4 composite
  • actions/download-artifact v4 composite
  • actions/setup-python v5 composite
  • actions/upload-artifact v4 composite
requirements.txt pypi
  • Cython *
  • plotille *
  • tensorboard *
  • torch *
  • tqdm *