CompressedBeliefMDPs.jl

CompressedBeliefMDPs.jl: A Julia Package for Solving Large POMDPs with Belief Compression - Published in JOSS (2025)

https://github.com/juliapomdp/compressedbeliefmdps.jl

Science Score: 93.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 1 DOI reference(s) in JOSS metadata
✓
Academic publication links
Links to: joss.theoj.org
○
Academic email domains
○
Institutional organization owner
✓
JOSS paper metadata
Published in Journal of Open Source Software

Keywords

artificial-intelligence compression dimensionality-reduction julia markov-decision-processes mdps pomdps reinforcement-learning

Scientific Fields

Engineering Computer Science - 40% confidence

Last synced: 6 months ago · JSON representation

Repository

A Julia package for solving POMDPs with belief compression. Part of the POMDPs.jl community.

Basic Info

Host: GitHub
Owner: JuliaPOMDP
License: mit
Language: Julia
Default Branch: main
Homepage: https://juliapomdp.github.io/CompressedBeliefMDPs.jl/
Size: 719 KB

Statistics

Stars: 6
Watchers: 3
Forks: 0
Open Issues: 1
Releases: 6

Topics

artificial-intelligence compression dimensionality-reduction julia markov-decision-processes mdps pomdps reinforcement-learning

Created almost 2 years ago · Last pushed 8 months ago

Metadata Files

Readme License

CompressedBeliefMDPs

Introduction

Welcome to CompressedBeliefMDPs.jl! This package is part of the POMDPs.jl ecosystem and takes inspiration from Exponential Family PCA for Belief Compression in POMDPs.

This package provides a general framework for applying belief compression in large POMDPs with generic compression, sampling, and planning algorithms.

Installation

You can install CompressedBeliefMDPs.jl using Julia's package manager. Open the Julia REPL (press ] to enter the package manager mode) and run the following command:

julia-repl pkg> add CompressedBeliefMDPs

Quickstart

Using belief compression is easy. Simplify pick a Sampler, Compressor, and a base Policy and then use the standard POMDPs.jl interface.

```julia using POMDPs, POMDPTools, POMDPModels using CompressedBeliefMDPs

pomdp = BabyPOMDP() compressor = PCACompressor(1) updater = DiscreteUpdater(pomdp) sampler = BeliefExpansionSampler(pomdp) solver = CompressedBeliefSolver( pomdp; compressor=compressor, sampler=sampler, updater=updater, verbose=true, maxiterations=100, ngenerative_samples=50, k=2 ) policy = solve(solver, pomdp) ```

Continuous Example

This example demonstrates using CompressedBeliefMDP in a continuous setting with the LightDark1D POMDP. It combines particle filters for belief updating and Monte Carlo Tree Search (MCTS) as the solver. While compressing a 1D space is trivial toy problem, this architecture can be easily scaled to larger POMDPs with continuous state and action spaces.

```julia using POMDPs, POMDPModels, POMDPTools using ParticleFilters using MCTS using CompressedBeliefMDPs

pomdp = LightDark1D() pomdp.movementcost = 1 basesolver = MCTSSolver(niterations=10, depth=50, explorationconstant=5.0) updater = BootstrapFilter(pomdp, 100) solver = CompressedBeliefSolver( pomdp, basesolver; updater=updater, sampler=PolicySampler(pomdp; updater=updater) ) policy = solve(solver, pomdp) rs = RolloutSimulator(maxsteps=50) r = simulate(rs, pomdp, policy) ```

Large Example

In this example, we tackle a more realistic scenario with the TMaze POMDP, which has 123 states. To handle the larger state space efficiently, we employ a variational auto-encoder (VAE) to compress the belief simplex. By leveraging the VAE's ability to learn a compact representation of the belief state, we focus computational power on the relevant (compressed) belief states during each Bellman update.

```julia using POMDPs, POMDPModels, POMDPTools using CompressedBeliefMDPs

pomdp = TMaze(60, 0.9) solver = CompressedBeliefSolver( pomdp; compressor=VAECompressor(123, 6; hiddendim=10, verbose=true, epochs=2), sampler=PolicySampler(pomdp, n=500), verbose=true, maxiterations=1000, ngenerativesamples=30, k=2 ) policy = solve(solver, pomdp) rs = RolloutSimulator(max_steps=50) r = simulate(rs, pomdp, policy) ```

Owner

Name: JuliaPOMDP
Login: JuliaPOMDP
Kind: organization
Location: Stanford University, University of Colorado Boulder

Website: http://juliapomdp.github.io/POMDPs.jl/latest/
Repositories: 61
Profile: https://github.com/JuliaPOMDP

POMDP packages for Julia

JOSS Publication

CompressedBeliefMDPs.jl: A Julia Package for Solving Large POMDPs with Belief Compression

Published

June 20, 2025

DOI

10.21105/joss.07346

Volume 10, Issue 110, Page 7346

Authors

Logan Mondal Bhamidipaty

Stanford University

Mykel J. Kochenderfer

Stanford University

Editor

Chris Vernon

GitHub Events

Total

Release event: 2
Delete event: 1
Issue comment event: 5
Push event: 22
Pull request event: 2
Create event: 3

Last Year

Release event: 2
Delete event: 1
Issue comment event: 5
Push event: 22
Pull request event: 2
Create event: 3

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 3
Total pull requests: 66
Average time to close issues: 1 day
Average time to close pull requests: 4 days
Total issue authors: 3
Total pull request authors: 3
Average comments per issue: 0.67
Average comments per pull request: 0.17
Merged pull requests: 28
Bot issues: 0
Bot pull requests: 64

Past Year

Issues: 0
Pull requests: 4
Average time to close issues: N/A
Average time to close pull requests: about 2 hours
Issue authors: 0
Pull request authors: 1
Average comments per issue: 0
Average comments per pull request: 0.25
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 4

View more stats

Top Authors

Issue Authors

FlyingWorkshop (1)
mykelk (1)
JuliaTagBot (1)

Pull Request Authors

github-actions[bot] (50)
dependabot[bot] (14)
dylan-asmar (2)

Top Labels

Issue Labels

Pull Request Labels

dependencies (14) github_actions (2)

Packages

Total packages: 1
Total downloads:
- julia 1 total

Total dependent packages: 0
Total dependent repositories: 0
Total versions: 6

juliahub.com: CompressedBeliefMDPs