Change Point Detection

Change point refers to the phenomenon where the statistical properties of a target variable or data distribution change over time. Detecting change point is crucial in various domains such as financial markets, healthcare, and online advertising to adapt models and decision-making processes to changing environments.

Example: Monitoring learning model performance

Model drift refers to the degradation of machine learning model performance due to changes in data or in the relationships between input and output variables.

Generate date

Apply data generator to create time-series data with abrupt mean-shift

Simulating streaming data

Simulate a streaming data scenario by iterating over a sequence of observations. During each iteration: 1. Make prediction with recursive-least-squares model 2. Retrieve the true value 3. Update model's parameters 4. Compute residual 5. Use the CUSUM detector on the residuals

Change Point Detectors

Change point detectors are algorithms designed to detect change points in streaming data or sequential observations. These detectors analyze the data stream and identify points where the underlying data distribution has changed significantly.

Generate Data with Mean Shift

```python import numpy as np from source.generator.changepointgenerator import ChangePointGenerator from source.detector.cusum import CUSUMDetector, ProbCUSUMDetector, ChartCUSUM_Detector

set seed

np.random.seed(12)

Generate time series data with change points

generator = ChangePointGenerator(numsegments=3, segmentlength=1000, changepointtype='suddenshift') generator.generatedata()

Plot the generated data

generator.plot_data() ```

Three commonly used drift detectors are:

1. CUSUM Detector (The PageHinkley Algorithm)

The CUSUM detector monitors the cumulative sum of deviations between observed data points and a reference value. When the cumulative sum exceeds a predefined threshold, it signals the presence of a change point.

```python

Detect change points using CUSUM Detector

cusumdetector = CUSUMDetector(warmupperiod=500, delta=3, threshold=10) cusumposchanges, cusumnegchanges, cusumchangepoints = cusumdetector.detectchangepoints(np.array(generator.data))

Plot the detected change points using CUSUM Detector

cusumdetector.plotchangepoints(generator.data, cusumchangepoints, cusumposchanges, cusumneg_changes) ```

2. Probabilistic CUSUM Detector

The Probabilistic CUSUM detector extends the CUSUM method by incorporating statistical probability measures. It evaluates the probability of observing deviations between data points, making it more robust to variations in data distribution.

```python

Detect change points using Probabilistic CUSUM Detector

probcusumdetector = ProbCUSUMDetector(warmupperiod=500, thresholdprobability=0.01) probprobabilities, probchangepoints = probcusumdetector.detectchangepoints(np.array(generator.data))

Plot the detected change points using Probabilistic CUSUM Detector

probcusumdetector.plotchangepoints(generator.data, probchangepoints, prob_probabilities) ```

3. CUSUM Control Chart Detector

The Control Chart CUSUM detector is a specialized form of CUSUM change point detection algorithm commonly used in quality control and process monitoring applications.

3.1 CUSUM of Deviations

```python

Detect change points using Control Chart CUSUM Detector

chartcusumdetector = ChartCUSUMDetector(warmupperiod=500, level=3, deviationtype='dev') upperlimits, lowerlimits, cusums, changepoints = chartcusumdetector.detectchangepoints(np.array(generator.data))

Plot the detected change points using Control Chart CUSUM Detector

chartcusumdetector.plotchangepoints(np.array(generator.data), changepoints, cusums, upperlimits, lower_limits) ```

3.2 CUSUM of Squares

```python

Detect change points using Control Chart CUSUM Detector

chartcusumdetector = ChartCUSUMDetector(warmupperiod=500, level=3, deviationtype='sqr-dev') upperlimits, lowerlimits, cusums, changepoints = chartcusumdetector.detectchangepoints(np.array(generator.data))

Plot the detected change points using Control Chart CUSUM Detector

chartcusumdetector.plotchangepoints(np.array(generator.data), changepoints, cusums, upperlimits, lower_limits) ```

Extensions: KS-CUM Detector (Kolmogorov-Smirnov Test)

```python

import numpy as np from source.generator.changepointgenerator import ChangePointGenerator from source.detector.cusum import KSCUMDetector

Set seed

np.random.seed(11)

Generate time series data with change points

generator = ChangePointGenerator(numsegments=3, segmentlength=1000, changepointtype='suddenshift') generator.generatedata()

Plot the generated data

generator.plot_data()

Kolmogorov-Smirnov Test

ksdetector = KSCUMDetector(windowpre=600, windowpost=300, alpha=0.001) ksstatistics , pvalues, changepoints = ksdetector.detectchangepoints(np.array(generator.data)) ksdetector.plotchangepoints(generator.data, changepoints, pvalues) ```