pymaro - maro-0.3.2a4 🚀

Add DDQN
Add prioritized sampling
Improve explore strategy
Fix minor issues

- Python
Published by Jinyu-W over 2 years ago

pymaro - maro-0.3.2a3 🚀

Fix DiscreteRLPolicy set_state bug (https://github.com/microsoft/maro/commit/a9a638134bef84176a0623fdadc5b789c2d7c3cf, https://github.com/microsoft/maro/commit/b3c6a589ad9036b03221e776a6929b2bc1eb4680)

- Python
Published by Jinyu-W about 3 years ago

pymaro - maro-0.3.2a1 🚀

Refine RL workflow
- Add **kwargs to support more problem setting (e.g., Graph based ones) (#589)
  - add **kwargs to RL models' forward funcs and _shape_check()
  - add **kwargs to RL policies' get_action related funcs and _post_check()
  - add **kwargs to choose_actions of AbsEnvSampler; remain it None in current sample() and eval()
- Add detached loss to the return value of update_critic() and update_actor() of current TrainOps; add default False early_stop to update_actor() of current TrainOps (#589)
- Refine random seed setting logic in RL workflow (#584)
- Refine rollout workflow (#577) to support:
- Run a specific number of steps in rollout
- Run a specific number of episodes during evaluation with num_eval_episodes
- Flexible metrics management during rollout with AbsEnvSampler.metrics
- Add AbsEnvSampler.metrics to support flexible metrics management during roullout (#577)
- Add Callback as a general interface to support customized operations in each phase of the workflow.
  - Two instances Checkpoint and MetricsRecorder are added. (#577)
  - Add customized_callbacks to RLComponentBundle. (#589)
- Re-organize RL job's output paths. (#577)
- Fix several RL algorithm bugs. (#577, #589)
Replace the numpy data type with python common data type in whole project (#571)
Add RL benchmark on Mujoco as a module to tests/, compared with spinning up benchmark, performance results can be found in tests/rl/performance.md (#575, #577, #583, #584)
Other minor code refinements

- Python
Published by Jinyu-W about 3 years ago

pymaro - maro-0.3.1a2 🚀

Update build workflow to fix pymaro package failed in manylinux issue

- Python
Published by Jinyu-W about 3 years ago

pymaro - maro-0.3.1a1 🚀

Refine decision event logic (#559)
- Add DecisionEventPayload and ActionPayload as the base class for payloads of decision events & action events.
- Add related type check during running.
- Rename related variables under simulator/.
- Refine the action handling logic. Modify Env and related BEs.
Refine RL component bundle logic (#549)
- Refine rl_component_bundle. Use a more straightforward & easier way to organize. In this new version, we do not need easyrl anymore.
- Update related examples.
- Fix bugs in distributed training. Add config YAML file for distributed training in CIM scenario.
- Add rl_formulation.ipynb as an example.
Fix CITI BIKE hello world dashboard (#555)
Update data model doc (#554)
Update requirements (#552, #553)

- Python
Published by Jinyu-W over 3 years ago

pymaro - maro-0.3.0a1 🚀

RL Toolkit: ✨ A whole new design of RL Toolkit (#539)
CLI: refactorization (#539)
Backend: data frame/snapshot precision issue fix (#544)
MISC: Whole repository code formating (#538, #547)
Citi bike visualization tool hot fix (#543)

- Python
Published by Jinyu-W almost 4 years ago

pymaro - maro-0.2.4a1 🚀

Add higher Python version 3.8, 3.9 (#398)
Refinement for some core modules, including more type hints, better coding style, small interface adjustment, potential bug fix
- Core & Abstract Business Engine refinement (#392)
- Event Buffer refinement (#389)
- Interface updates for SimRandom to better support scenario buildings (#401, #400)
CIM scenario refinement (#400)
- Disable auto action type detection, i.e., requires for explicit ActionType (#399)
- Data processing updates, including data generation part and real data loader part (#395)
- Some testing program updates, and enlarge the test coverage (#395)
Examples update
- OnlineLP for Citi Bike: change the default solver from GLPK to CBC (#391)

- Python
Published by Jinyu-W over 4 years ago

pymaro - maro-0.2.3a4 🚀

Scenario/CIM: - Fix the potential bug of vessel plan for some specific topologies after calling Env.reset() (Related issue: #385, Related PR: #387, #388)

- Python
Published by Jinyu-W almost 5 years ago

pymaro - maro-0.2.3a3 🚀

Simulator/CIM Scenario: - Fix the environment reset issue #385, and update the simulator random seed interfaces (#387) - Fix the "Wrong future stop tick predictions" issue #384 (#386)

RL Toolkit: - Add the arguments used by Actor to the Trajectory.on_env_feedback() interface (#373, #374)

Visualization Tool/CIM: - Fix Geo vis IP address & SQL logic bugs (#352, #383)

Others: - update dataclasses requirement to setup - fix some typo in codes comments and online document

- Python
Published by Jinyu-W almost 5 years ago

pymaro - maro-0.2.3a2 🚀

CIM data container/data loader interface for real data mode updated (#372)
README for pymaro description updated

- Python
Published by Jinyu-W almost 5 years ago

CIM Scenario: * Add the data container and data loader for real data schema (#361) * Fix the vessel plan, vessel past/future stops info issues (#363) * Add attributes is_parking and loc_port_idx for vessels (#366)

Simulator: * Add support to dump finished events (#365) * Add property business_engine to AbsEnv (#368)

RL Toolkit: * Fixed scheduler init bug (#362)

Tutorials docker images: * Fix bugs in examples and notebooks caused by interface changes. (#367) * Update simple strategies used in examples (E.g. random instead of dumpy in CIM hello series). (#367) * Update docker images and related documents. (#367)

- Python
Published by Jinyu-W almost 5 years ago

pymaro - maro-0.2.2a3 🚀

Package dependency: - Update package dependency (#345 #346)

Simulator: - Fixed the vessel plan issue for CIM scenario (#348)

Examples: - Update hello world examples for CIM scenario (#338)

- Python
Published by Jinyu-W about 5 years ago

pymaro - maro-0.2.2a2 🚀

Package dependency: - Loosen package dependency (#328) to be more flexible: #331

CLI related: - Fixed issues (#317, #314) for geographic visualization tool: #325 - Fixed Redis database data not cleaned issue (#327) by adding a close function for Proxy: #335

Examples: - Add offline ILP example for VM scheduling scenario: #275

- Python
Published by Jinyu-W about 5 years ago

pymaro - maro-0.2.2a1 🚀

Backend related: - bug fix: dynamic backend slot number not correct after reset (#321) - bug fix: value is interrupt if one node contains multiple list attribute (#321) - changes: snapshot query result default value set to 0 instead nan, make it align with static backend (#321) - bug fix: on windows compiling fail due to cannot find max function. (#321)

RL Toolkit related: - Fix bug of dqn.chooseaction and ddpg.chooseaction (#307) - Double DQN: loss function bug fixed (#315) - Fix bug caused by invalid snapshot index (#318, #319)

CLI related: - Add MARO admin tool (#288) - Add master/node vm join deployment file for grass/on-premises. (#316) - Visualization tool: Add support of dump data path creation, default sampling ratio to 1 (#323) - Visualization tool: Bug occurs in Linux fixed (#312, #313) - Update online doc Installation part (#316); Visualization part (#301)

- Python
Published by Jinyu-W about 5 years ago

pymaro - maro-0.2.1a1 🚀

Backend related:
- Vector env support (#266)
- Add slot filter functions for node attribute (#273)
- Refine joint decision sequential action mode (#219)
- Add dynamic node support (#172)
VM scheduling scenario related:
- Add oversubscription feature (#246), (#256)
- Add hierarchy vm region architecture support (#258)
- Fix bug of auto-downloading data (#257)
- Pricing model and energy model update (#286)
- Rule-based algorithm examples (#255), (#282)
RL toolkit related:
- Add DDPG algorithm (#252)
- Merge algorithm with agent (#259)
- Distributed framework update (#206)
CLI related:
- CLI refactoring (#227)
- Feature: Add a cli command to support create new project (#279)
- Add Env-Geographic visualization tool, CIM hello as example (#291), (#294), (#295), (#298)
- CLI visualization support and maro grass local mode (#277)

- Python
Published by Jinyu-W about 5 years ago