simglucose

A Type-1 Diabetes simulator implemented in Python for Reinforcement Learning purpose

This simulator is a python implementation of the FDA-approved UVa/Padova Simulator (2008 version) for research purpose only. The simulator includes 30 virtual patients, 10 adolescents, 10 adults, 10 children. There is documentation of the virtual patient's parameters.

HOW TO CITE: Jinyu Xie. Simglucose v0.2.1 (2018) [Online]. Available: https://github.com/jxx123/simglucose. Accessed on: Month-Date-Year.

Notice: simglucose no longer supports python 3.7 and 3.8, please update to >=3.9 verison. Thanks!

Announcement (08/20/2023): simglucose now supports gymnasium! Check examples/run_gymnasium.py for usage.

| Animation | CVGA Plot | BG Trace Plot | Risk Index Stats | | ------------------------------------------------ | :---------------------------- | ----------------------------------------------- | ----------------------------------------------- | | | | | |

<!-- -->

Main Features

• Simulation environment follows OpenAI gym and rllab APIs. It returns observation, reward, done, info at each step, which means the simulator is "reinforcement-learning-ready".
• Supports customized reward function. The reward function is a function of blood glucose measurements in the last hour. By default, the reward at each step is risk[t-1] - risk[t]. risk[t] is the risk index at time t defined in this paper.
• Supports parallel computing. The simulator simulates multiple patients in parallel using pathos multiprocessing package (you are free to turn parallel off by setting parallel=False).
• The simulator provides a random scenario generator (from simglucose.simulation.scenario_gen import RandomScenario) and a customized scenario generator (from simglucose.simulation.scenario import CustomScenario). Commandline user-interface will guide you through the scenario settings.
• The simulator provides the most basic basal-bolus controller for now. It provides very simple syntax to implement your own controller, like Model Predictive Control, PID control, reinforcement learning control, etc.
• You can specify random seed in case you want to repeat your experiments.
• The simulator will generate several plots for performance analysis after simulation. The plots include blood glucose trace plot, Control Variability Grid Analysis (CVGA) plot, statistics plot of blood glucose in different zones, risk indices statistics plot.
• NOTE: animate and parallel cannot be set to True at the same time in macOS. Most backends of matplotlib in macOS is not thread-safe. Windows has not been tested. Let me know the results if anybody has tested it out.

Installation

It is highly recommended using pip to install simglucose, follow this link to install pip.

Auto installation:

bash pip install simglucose

Manual installation:

bash git clone https://github.com/jxx123/simglucose.git cd simglucose

If you have pip installed, then

bash pip install -e .

If you do not have pip, then

bash python setup.py install

If rllab (optional) is installed, the package will utilize some functionalities in rllab.

Note: there might be some minor differences between auto install version and manual install version. Use git clone and manual installation to get the latest version.

Quick Start

Use simglucose as a simulator and test controllers

Run the simulator user interface

python from simglucose.simulation.user_interface import simulate simulate()

You are free to implement your own controller, and test it in the simulator. For example,

```python from simglucose.simulation.user_interface import simulate from simglucose.controller.base import Controller, Action

class MyController(Controller): def init(self, initstate): self.initstate = initstate self.state = initstate

def policy(self, observation, reward, done, **info):
    '''
    Every controller must have this implementation!
    ----
    Inputs:
    observation - a namedtuple defined in simglucose.simulation.env. For
                  now, it only has one entry: blood glucose level measured
                  by CGM sensor.
    reward      - current reward returned by environment
    done        - True, game over. False, game continues
    info        - additional information as key word arguments,
                  simglucose.simulation.env.T1DSimEnv returns patient_name
                  and sample_time
    ----
    Output:
    action - a namedtuple defined at the beginning of this file. The
             controller action contains two entries: basal, bolus
    '''
    self.state = observation
    action = Action(basal=0, bolus=0)
    return action

def reset(self):
    '''
    Reset the controller state to inital state, must be implemented
    '''
    self.state = self.init_state

ctrller = MyController(0) simulate(controller=ctrller) ```

These two examples can also be found in examples\ folder.

In fact, you can specify a lot more simulation parameters through simulation:

python simulate(sim_time=my_sim_time, scenario=my_scenario, controller=my_controller, start_time=my_start_time, save_path=my_save_path, animate=False, parallel=True)

OpenAI Gym usage

• Using default reward

```python import gym

Register gym environment. By specifying kwargs,

you are able to choose which patient or patients to simulate.

patient_name must be 'adolescent#001' to 'adolescent#010',

or 'adult#001' to 'adult#010', or 'child#001' to 'child#010'

It can also be a list of patient names

You can also specify a custom scenario or a list of custom scenarios

If you chose a list of patient names or a list of custom scenarios,

every time the environment is reset, a random patient and scenario will be

chosen from the list

from gym.envs.registration import register from simglucose.simulation.scenario import CustomScenario from datetime import datetime

starttime = datetime(2018, 1, 1, 0, 0, 0) mealscenario = CustomScenario(starttime=starttime, scenario=[(1,20)])

register( id='simglucose-adolescent2-v0', entrypoint='simglucose.envs:T1DSimEnv', kwargs={'patientname': 'adolescent#002', 'customscenario': mealscenario} )

env = gym.make('simglucose-adolescent2-v0')

observation = env.reset() for t in range(100): env.render(mode='human') print(observation) # Action in the gym environment is a scalar # representing the basal insulin, which differs from # the regular controller action outside the gym # environment (a tuple (basal, bolus)). # In the perfect situation, the agent should be able # to control the glucose only through basal instead # of asking patient to take bolus action = env.action_space.sample() observation, reward, done, info = env.step(action) if done: print("Episode finished after {} timesteps".format(t + 1)) break ```

• Customized reward function

```python import gym from gym.envs.registration import register

def customreward(BGlasthour): if BGlasthour[-1] > 180: return -1 elif BGlast_hour[-1] < 70: return -2 else: return 1

register( id='simglucose-adolescent2-v0', entrypoint='simglucose.envs:T1DSimEnv', kwargs={'patientname': 'adolescent#002', 'rewardfun': customreward} )

env = gym.make('simglucose-adolescent2-v0')

reward = 1 done = False

observation = env.reset() for t in range(200): env.render(mode='human') action = env.action_space.sample() observation, reward, done, info = env.step(action) print(observation) print("Reward = {}".format(reward)) if done: print("Episode finished after {} timesteps".format(t + 1)) break ```

rllab usage

```python from rllab.algos.ddpg import DDPG from rllab.envs.normalizedenv import normalize from rllab.explorationstrategies.oustrategy import OUStrategy from rllab.policies.deterministicmlppolicy import DeterministicMLPPolicy from rllab.qfunctions.continuousmlpqfunction import ContinuousMLPQFunction from rllab.envs.gymenv import GymEnv from gym.envs.registration import register

register( id='simglucose-adolescent2-v0', entrypoint='simglucose.envs:T1DSimEnv', kwargs={'patientname': 'adolescent#002'} )

env = GymEnv('simglucose-adolescent2-v0') env = normalize(env)

policy = DeterministicMLPPolicy( envspec=env.spec, # The neural network policy should have two hidden layers, each with 32 hidden units. hiddensizes=(32, 32) )

es = OUStrategy(env_spec=env.spec)

qf = ContinuousMLPQFunction(env_spec=env.spec)

algo = DDPG( env=env, policy=policy, es=es, qf=qf, batchsize=32, maxpathlength=100, epochlength=1000, minpoolsize=10000, nepochs=1000, discount=0.99, scalereward=0.01, qflearningrate=1e-3, policylearningrate=1e-4 ) algo.train() ```

Advanced Usage

You can create the simulation objects, and run batch simulation. For example,

```python from simglucose.simulation.env import T1DSimEnv from simglucose.controller.basalbolusctrller import BBController from simglucose.sensor.cgm import CGMSensor from simglucose.actuator.pump import InsulinPump from simglucose.patient.t1dpatient import T1DPatient from simglucose.simulation.scenariogen import RandomScenario from simglucose.simulation.scenario import CustomScenario from simglucose.simulation.simengine import SimObj, sim, batch_sim from datetime import timedelta from datetime import datetime

specify start_time as the beginning of today

now = datetime.now() start_time = datetime.combine(now.date(), datetime.min.time())

--------- Create Random Scenario --------------

Specify results saving path

path = './results'

Create a simulation environment

patient = T1DPatient.withName('adolescent#001') sensor = CGMSensor.withName('Dexcom', seed=1) pump = InsulinPump.withName('Insulet') scenario = RandomScenario(starttime=starttime, seed=1) env = T1DSimEnv(patient, sensor, pump, scenario)

Create a controller

controller = BBController()

Put them together to create a simulation object

s1 = SimObj(env, controller, timedelta(days=1), animate=False, path=path) results1 = sim(s1) print(results1)

--------- Create Custom Scenario --------------

Create a simulation environment

patient = T1DPatient.withName('adolescent#001') sensor = CGMSensor.withName('Dexcom', seed=1) pump = InsulinPump.withName('Insulet')

custom scenario is a list of tuples (time, meal_size)

scen = [(7, 45), (12, 70), (16, 15), (18, 80), (23, 10)] scenario = CustomScenario(starttime=starttime, scenario=scen) env = T1DSimEnv(patient, sensor, pump, scenario)

Create a controller

controller = BBController()

Put them together to create a simulation object

s2 = SimObj(env, controller, timedelta(days=1), animate=False, path=path) results2 = sim(s2) print(results2)

--------- batch simulation --------------

Re-initialize simulation objects

s1.reset() s2.reset()

create a list of SimObj, and call batch_sim

s = [s1, s2] results = batch_sim(s, parallel=True) print(results) ```

Run analysis offline (example/offline_analysis.py):

```python from simglucose.analysis.report import report import pandas as pd from pathlib import Path

get the path to the example folder

exmaplepth = Path(file_).parent

find all csv with pattern #.csv, e.g. adolescent#001.csv

resultfilenames = list(exmaplepth.glob( 'results/2017-12-3117-46-32/#.csv')) patientnames = [f.stem for f in resultfilenames] df = pd.concat( [pd.readcsv(str(f), indexcol=0) for f in resultfilenames], keys=patient_names) report(df) ```

Release Notes

08/20/2023

• Fixed numpy compatibility issues for risk index computation (thanks to @yihuicai)
• Support Gymnasium.
  • NOTE: the observation in gymnasium version is no longer a namedtuple with a CGM field. It is a numpy array instead (to be consistent with its space definition).
• NOTE: Python 3.7 and 3.8 are no longer supported. Please update to >3.9 version.

03/10/2021

• Fixed some random seed issues.

5/27/2020

• Add PIDController at simglucose/controller/pidctrller. There is an example at examples/runpid_controller.py showing how to use it.

9/10/2018

• Controller policy method gets access to all the current patient state through info['patient_state'].

2/26/2018

• Support customized reward function.

1/10/2018

• Added workaround to select patient when make gym environment: register gym environment by passing kwargs of patient_name.

1/7/2018

• Added OpenAI gym support, use gym.make('simglucose-v0') to make the environment.
• Noticed issue: the patient name selection is not available in gym.make for now. The patient name has to be hard-coded in the constructor of simglucose.envs.T1DSimEnv.

Reporting issues

Shoot me any bugs, enhancements or even discussion by creating issues.

How to contribute

The following instruction is originally from the contribution instructions of sklearn.

The preferred workflow for contributing to simglucose is to fork the main repository on GitHub, clone, and develop on a branch. Steps:

1. Fork the project repository by clicking on the 'Fork' button near the top right of the page. This creates a copy of the code under your GitHub user account. For more details on how to fork a repository see this guide.

2. Clone your fork of the simglucose repo from your GitHub account to your local disk:

bash $ git clone git@github.com:YourLogin/simglucose.git $ cd simglucose

1. Create a feature branch to hold your development changes:

bash $ git checkout -b my-feature

Always use a feature branch. It's good practice to never work on the master branch!

1. Develop the feature on your feature branch. Add changed files using git add and then git commit files:

bash $ git add modified_files $ git commit

to record your changes in Git, then push the changes to your GitHub account with:

bash $ git push -u origin my-feature

1. Follow these instructions to create a pull request from your fork. This will email the committers.

(If any of the above seems like magic to you, please look up the Git documentation on the web, or ask a friend or another contributor for help.)