dualcrl
This repository provides the necessary code to reproduce all experiments of the paper "A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints" [1].
Science Score: 67.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 3 DOI reference(s) in README -
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.2%) to scientific vocabulary
Keywords
Repository
This repository provides the necessary code to reproduce all experiments of the paper "A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints" [1].
Basic Info
Statistics
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Topics
Metadata Files
README.md
Dual Constrained Reinforcement Learning
This repository provides the necessary code to reproduce all experiments of the paper "A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints" [1].
An example Pytorch implementation of the DualCRL algorithm is provided, with support for any combination of entropy regularization, value constraints, density constraints and transition constraints.
Custom constrained setups are defined for the CliffWalking-v0 and Pendulum-v1 Gymnasium environments.
Further details are provided in the paper's Experiments section.
| https://github.com/user-attachments/assets/c3c32c2c-3ba0-4098-9580-9a8216c3246e | https://github.com/user-attachments/assets/94a3130d-e4f7-4458-91b1-43e7442f4226 |
Installation
- Clone this repository.
git clone https://github.com/dcbr/dualcrl
cd dualcrl
- Install the required packages, as specified in
environment.yml. This can be easily done by creating a virtual environment (using e.g. conda or venv).
conda env create -f environment.yml
Usage
Activate the virtual environment, using e.g. conda activate dualcrl. Afterwards, you can simply run the main script with suitable arguments to train the models or analyze their performance.
For example
python main.py --mode train --job cliffwalk
to train on the cliff walking environment (with additional policy constraints).
To reproduce all results of Section VI, first train on all jobs with python main.py --mode train --job [JOB] --uid paper, followed by the analysis python main.py --mode analyze --job [JOB] --uid paper. Beware that this might take a while to complete, depending on your hardware!
A summary of the most relevant parameters to this script is provided below.
Check python main.py --help for a full overview of supported parameters.
| Parameter | Supported values | Description |
|:----------|:------------------------|:------------------------------------------------------------------------------|
| --mode | train, analyze | Run mode. Either train models, or analyze (and summarize) the results. |
| --job | cliffwalk, pendulum | Job to run. The job file defines the environment and constraints to train on. |
| --uid | Any value | Unique identifier for a job run. |
References
[1] De Cooman, B., Suykens, J.: A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints. Accepted for publication in IEEE Transactions on Artificial Intelligence. DOI: 10.1109/TAI.2025.3564898, ArXiv: 2404.16468
Owner
- Login: dcbr
- Kind: user
- Repositories: 2
- Profile: https://github.com/dcbr
Citation (CITATION.bib)
@article{DeCooman2024,
title = {A {{Dual Perspective}} of {{Reinforcement Learning}} for {{Imposing Policy Constraints}}},
author = {De Cooman, Bram and Suykens, Johan},
date = {2024-04},
journal = {IEEE Transactions on Artificial Intelligence},
eprint = {2404.16468},
eprinttype = {arXiv},
doi = {10.1109/TAI.2025.3564898}
}
GitHub Events
Total
- Push event: 3
- Create event: 2
Last Year
- Push event: 3
- Create event: 2