Surface Similarity Parameter machine learning loss metric for oscillatory spatio-temporal data

Introduction

This repository contains the code for the Surface Similarity Parameter (SSP) loss metric proposed in

Wedler, M., Stender, M., Klein, M., Ehlers, S. and Hoffmann, N., 2022, "Surface Similarity Parameter: A new machine learning loss metric for oscillatory spatio-temporal data". Neural Networks 156, 123--134. DOI: https://doi.org/10.1016/j.neunet.2022.09.023

The Surface Similarity Parameter

$$\mathrm{SSP}(\mathbf{y},\hat{\mathbf{y}})=\frac{\sqrt{\int|Y - \hat{Y}|^2df}}{\sqrt{\int|Y|^2df} + \sqrt{\int|\hat{Y}|^2df}}\in[0,1]$$

with $Y$ the Fourier transform of $y$, is a normalized error metric originally introduced by Perlin and Bustamante (2016). The SSP quantifies the difference between two signals in the complex Fourier space, and thus inherently penalizes deviations in magnitude and phase in a single metric.

For discrete signals, the SSP Equation collapses to

$$\mathrm{SSP} = \frac{||Y-\hat{Y}||}{||Y|| + ||\hat{Y}||}\in[0, 1].$$

Being a normalized error, the SSP is defined in the range [0, 1], where - $\mathrm{SSP}=0$ indicates perfect agreement, and - $\mathrm{SSP}=1$ indicates perfect disagreement among the signals.

Perfect disagreement means that either $\hat{y}=-y$, or $\hat{y}=0$ while $y\neq 0$

Using the SSP as a machine learning loss function forces the model to improve the prediction in terms of magnitude and phase in order to reduce the loss (cf. Wedler et al. (2022)). This sets the SSP apart from established Euclidean distance-based loss functions like the MSE and MAE, when training models on oscillatory spatio-temporal data. The stricter error penalization of the SSP leads to a more refined and optimizer-friendly loss surface, where local minima are sparse but meaningful. This allows the optimizer to take more confident steps, leading to faster convergence to better local minima.

This package contains - a very basic numpy implementation, and - a more complex Keras3 SSP loss function.

Optional lowpass filter in the loss function

The Keras3 loss function offers two different optional lowpass filters, a static and an adaptive one. The lowpass filter is applied to the ground truth only within the loss function. This way, the ML model is forced to suppress the high-frequency range in order to reduce the loss, and effectively learns a lowpass filter behavior. This is beneficial when - the relevant dynamics are within a certain frequency band, or - the model is trained on noisy (measurement) data.

The static lowpass simply cuts of the frequency range at a static value f_filter that is passed to the constructor of the SSP loss function.

The adaptive filter is more advanced and allows to automatically derive the peak frequency of the ground truth data and adjust the cut-off frequency based on that. With lowpass="adaptive", the argument f_filter becomes a multiplier for the internally derived peak frequency f_p, and the cut-off frequency becomes f_p * f_filter.

NOTE To use either of these options, the instance of the SSP loss function has to know the frequency range the data lives on. The frequencies have to be passed to the constructor of the loss function as the argument f. The package offers the helper function ssp.keras.ops.fftfreq, which generates the FFT frequency range based on the number of grid points and the discretization of the data. The additional argument rad can be set to get angular frequencies, cf. docs for fftfreq.

Installation

The package is hosted on PyPI.org and can be installed via pip $ pip install surface-similarity-parameter Optionally, the most recent version of Keras3 can be automatically installed alongside using the option [keras] $ pip install surface-similarity-parameter[keras] Note that the Keras backend - Tensorflow, or - JAX

has to be manually installed. At the moment, there is no implementation for the 'torch' backend.

Documentation

The full documentation is available at Read the Docs.

Usage examples

Please note that more elaborate examples for each class or method can be found in the docstring of the respective class or method.

Numpy example (metric only)

Let's start with a very basic call of the numpy implementation on two random arrays. ```

from ssp.numpy import ssp import numpy as np np.random.seed(0) # for deterministic results y1 = np.random.random((2, 32)) y2 = np.random.random((2, 32)) ssp(y1, y2) # some value between 0.0 and 1.0 np.float64(0.35119514129237195) ssp(y1, y1) # should be 0.0 np.float64(0.0) ssp(y1, -y1) # should be 1.0 np.float64(1.0) ssp(y1, np.zeros_like(y1)) # should be 1.0 np.float64(1.0) ```

With the option batched=True, the SSP operates batch-wise and returns a result for each signal in a batch: ```

from ssp.numpy import ssp import numpy as np np.random.seed(0) # for deterministic results y1 = np.random.random((2, 32)) y2 = np.random.random((2, 32)) ssp(y1, y2, batched=True) array([0.34864963, 0.35827101]) ssp(y1, y1, batched=True) array([0., 0.]) ssp(y1, -y1, batched=True) array([1., 1.]) ssp(y1, np.zeros_like(y1), batched=True) array([1., 1.]) ```

Keras example (SSP as an ML loss function)

Again, let's start with a very basic usage of the SSP as a Keras loss function. In the following example, a Sequential model is defined, build, compiled, and trained with the SSP as the loss function. ```

from ssp.keras import SSP1D from keras import ops, Sequential, layers from math import pi t = ops.arange(0, 2 * pi, 2 * pi / 512) y = ops.expanddims(ops.sin(t), axis=0) # shape (1, 512) x = ops.ones((1, 32)) # some input data with shape (1, 32) model = Sequential([layers.Dense(64), layers.Dense(512)]) model.build(inputshape=x.shape) model.compile(optimizer="adam", loss=SSP1D(lowpass="static", f=f, f_filter=2.0)) model.fit(x=x, y=y, epochs=1) ```

The optional lowpass filter can help the model to focus on the relevant frequency range of the data, or omit high-frequency noise (like, e.g., in raw measurement data). In the following example, a static lowpass filter with a threshold at f_filter=2.0 is used. When using a lowpass filter, the SSP loss function requires the FFT frequencies f, which can be generated using the ssp.keras.ops.fftshift method, which strongly aligns with numpy.fft.fftshift, but offers the optional argument rad to return the FFT frequencies in terms of rad instead of Hz (multiplication by $2\pi$). ```

from ssp.keras import SSP1D from ssp.keras.ops import fftfreq from keras import ops, Sequential, layers from math import pi t = ops.arange(0, 2 * pi, 2 * pi / 512) f = fftfreq(n=512, d=2 * pi / 512) y = ops.expanddims(ops.sin(t), axis=0) # shape (1, 512) x = ops.ones((1, 32)) # some input data with shape (1, 32) model = Sequential([layers.Dense(64), layers.Dense(512)]) model.build(inputshape=x.shape) model.compile(optimizer="adam", loss=SSP1D(lowpass="static", f=f, f_filter=2.0)) model.fit(x=x, y=y, epochs=1) ```

Citation for original paper

@article{10.1016/j.neunet.2022.09.023, title = {Surface similarity parameter: {A} new machine learning loss metric for oscillatory spatio-temporal data}, volume = {156}, issn = {0893-6080}, shorttitle = {Surface similarity parameter}, url = {https://www.sciencedirect.com/science/article/pii/S0893608022003732}, doi = {10.1016/j.neunet.2022.09.023}, abstract = {Supervised machine learning approaches require the formulation of a loss functional to be minimized in the training phase. Sequential data are ubiquitous across many fields of research, and are often treated with Euclidean distance-based loss functions that were designed for tabular data. For smooth oscillatory data, those conventional approaches lack the ability to penalize amplitude, frequency and phase prediction errors at the same time, and tend to be biased towards amplitude errors. We introduce the surface similarity parameter (SSP) as a novel loss function that is especially useful for training machine learning models on smooth oscillatory sequences. Our extensive experiments on chaotic spatio-temporal dynamical systems indicate that the SSP is beneficial for shaping gradients, thereby accelerating the training process, reducing the final prediction error, increasing weight initialization robustness, and implementing a stronger regularization effect compared to using classical loss functions. The results indicate the potential of the novel loss metric particularly for highly complex and chaotic data, such as data stemming from the nonlinear two-dimensional Kuramoto–Sivashinsky equation and the linear propagation of dispersive surface gravity waves in fluids.}, language = {en}, urldate = {2022-12-22}, journal = {Neural Networks}, author = {Wedler, Mathies and Stender, Merten and Klein, Marco and Ehlers, Svenja and Hoffmann, Norbert}, month = dec, year = {2022}, keywords = {Deep learning, Error metric, Loss function, Nonlinear dynamics, Similarity, Spatio-temporal dynamics}, pages = {123--134}, }