PyDTMC

PyDTMC is a full-featured and lightweight library for discrete-time Markov chains analysis. It provides classes and functions for creating, manipulating, simulating and visualizing Markov processes.

Status:
Info:
PyPI:
Conda:
Donation:

Requirements

The Python environment must include the following packages:

• Matplotlib
• NetworkX
• NumPy
• SciPy

Notes:

• It's recommended to install Graphviz and pydot before using the plot_graph function.
• The packages pytest and pytest-benchmark are required for performing unit tests.
• The package Sphinx is required for building the package documentation.

Installation & Upgrade

PyPI:

sh $ pip install PyDTMC $ pip install --upgrade PyDTMC

Git:

```sh $ pip install https://github.com/TommasoBelluzzo/PyDTMC/tarball/master $ pip install --upgrade https://github.com/TommasoBelluzzo/PyDTMC/tarball/master

$ pip install git+https://github.com/TommasoBelluzzo/PyDTMC.git#egg=PyDTMC $ pip install --upgrade git+https://github.com/TommasoBelluzzo/PyDTMC.git#egg=PyDTMC ```

Conda:

```sh $ conda install -c conda-forge pydtmc $ conda update -c conda-forge pydtmc

$ conda install -c tommasobelluzzo pydtmc $ conda update -c tommasobelluzzo pydtmc ```

Usage: MarkovChain Class

The MarkovChain class can be instantiated as follows:

```console

p = [[0.2, 0.7, 0.0, 0.1], [0.0, 0.6, 0.3, 0.1], [0.0, 0.0, 1.0, 0.0], [0.5, 0.0, 0.5, 0.0]] mc = MarkovChain(p, ['A', 'B', 'C', 'D']) print(mc)

DISCRETE-TIME MARKOV CHAIN SIZE: 4 RANK: 4 CLASSES: 2

RECURRENT: 1 TRANSIENT: 1 ERGODIC: NO APERIODIC: YES IRREDUCIBLE: NO ABSORBING: YES MONOTONE: NO REGULAR: NO REVERSIBLE: YES SYMMETRIC: NO ```

Below a few examples of MarkovChain properties:

```console

print(mc.is_ergodic) False

print(mc.recurrent_states) ['C']

print(mc.transient_states) ['A', 'B', 'D']

print(mc.steady_states) [array([0.0, 0.0, 1.0, 0.0])]

print(mc.is_absorbing) True

print(mc.fundamental_matrix) [[1.50943396, 2.64150943, 0.41509434] [0.18867925, 2.83018868, 0.30188679] [0.75471698, 1.32075472, 1.20754717]]

print(mc.kemeny_constant) 5.547169811320755

print(mc.entropy_rate) 0.0 ```

Below a few examples of MarkovChain methods:

```console

print(mc.absorption_probabilities()) [1.0 1.0 1.0]

print(mc.expected_rewards(10, [2, -3, 8, -7])) [44.96611926, 52.03057032, 88.00000000, 51.74779651]

print(mc.expected_transitions(2)) [[0.0850, 0.2975, 0.0000, 0.0425] [0.0000, 0.3450, 0.1725, 0.0575] [0.0000, 0.0000, 0.7000, 0.0000] [0.1500, 0.0000, 0.1500, 0.0000]]

print(mc.firstpassageprobabilities(5, 3)) [[0.5000, 0.0000, 0.5000, 0.0000] [0.0000, 0.3500, 0.0000, 0.0500] [0.0000, 0.0700, 0.1300, 0.0450] [0.0000, 0.0315, 0.1065, 0.0300] [0.0000, 0.0098, 0.0761, 0.0186]]

print(mc.hitting_probabilities([0, 1])) [1.0, 1.0, 0.0, 0.5]

print(mc.meanabsorptiontimes()) [4.56603774, 3.32075472, 3.28301887]

print(mc.meannumbervisits()) [[0.50943396, 2.64150943, INF, 0.41509434] [0.18867925, 1.83018868, INF, 0.30188679] [0.00000000, 0.00000000, INF, 0.00000000] [0.75471698, 1.32075472, INF, 0.20754717]]

print(mc.simulate(10, seed=32)) ['D', 'A', 'B', 'B', 'C', 'C', 'C', 'C', 'C', 'C', 'C'] ```

```console

sequence = ["A"] for i in range(1, 11): ... currentstate = sequence[-1] ... nextstate = mc.next(currentstate, seed=32) ... print((' ' if i < 10 else '') + f'{i}) {currentstate} -> {nextstate}') ... sequence.append(nextstate) 1) A -> B 2) B -> C 3) C -> C 4) C -> C 5) C -> C 6) C -> C 7) C -> C 8) C -> C 9) C -> C 10) C -> C ```

Below a few examples of MarkovChain plotting functions; in order to display the output of plots immediately, the interactive mode of Matplotlib must be turned on:

```console

ploteigenvalues(mc, dpi=300) plotgraph(mc, dpi=300) plotsequence(mc, 10, plottype='histogram', dpi=300) plotsequence(mc, 10, plottype='heatmap', dpi=300) plotsequence(mc, 10, plottype='matrix', dpi=300) plotredistributions(mc, 10, plottype='heatmap', dpi=300) plotredistributions(mc, 10, plottype='projection', dpi=300) ```

Usage: HiddenMarkovModel Class

The HiddenMarkovModel class can be instantiated as follows:

```console

p = [[0.4, 0.6], [0.8, 0.2]] states = ['A', 'B'] e = [[0.5, 0.0, 0.0, 0.5], [0.2, 0.2, 0.2, 0.4]] symbols = ['H1', 'H2', 'H3', 'H4'] hmm = HiddenMarkovModel(p, e, states, symbols) print(hmm)

HIDDEN MARKOV MODEL STATES: 2 SYMBOLS: 4 ERGODIC: NO REGULAR: NO ```

Below a few examples of HiddenMarkovModel methods:

```console

simstates, simsymbols = hmm.simulate(12, seed=1488) print(simstates) ['B', 'A', 'A', 'A', 'B', 'A', 'A'] print(simsymbols) ['H2', 'H4', 'H4', 'H4', 'H3', 'H4', 'H4']

esthmm = hmm.estimate(states, symbols, simstates, simsymbols) print(esthmm.p) [[0.75, 0.25] [1.00, 0.00]] print(est_hmm.e) [[0.0, 0.0, 0.0, 1.0] [0.0, 0.5, 0.5, 0.0]]

declp, decposterior, decbackward, decforward, _ = hmm.decode(simsymbols) print(declp) -8.77549587 print(decposterior) [[0.00000000, 0.84422968, 0.41785105, 0.84422968, 0.00000000, 0.82089552, 0.52238806] [1.00000000, 0.15577032, 0.58214895, 0.15577032, 1.00000000, 0.17910448, 0.47761194]] print(decbackward) [[1.50000000, 0.88942581, 1.01307561, 0.79988630, 1.31154065, 0.94776119, 0.98507463, 1.00000000] [0.50000000, 1.00000000, 0.93462194, 1.21887436, 0.43718022, 1.00000000, 1.07462687, 1.00000000]] print(dec_forward) [[0.50000000, 0.00000000, 0.83333333, 0.52238806, 0.64369311, 0.00000000, 0.83333333 0.52238806] [0.50000000, 1.00000000, 0.16666667, 0.47761194, 0.35630689, 1.00000000, 0.16666667 0.47761194]]

prelp, prestates = hmm.predict('viterbi', simsymbols) print(prelp) -13.24482936 print(pre_states) ['B', 'A', 'B', 'A', 'B', 'A', 'B'] ```

Below a few examples of HiddenMarkovModel plotting functions; in order to display the output of plots immediately, the interactive mode of Matplotlib must be turned on:

```console

plotgraph(hmm, dpi=300) plotsequence(hmm, 10, plottype='histogram', dpi=300) plotsequence(hmm, 10, plottype='heatmap', dpi=300) plotsequence(hmm, 10, plottype='matrix', dpi=300) plottrellis(hmm, 10, dpi=300) ```