PyTurbo

---

License: MIT Python Version Documentation

Overview

---

PyTurboSF is a Python package for efficient structure function calculations in 1D, 2D, and 3D data. The package provides optimized implementations for analyzing turbulent flows and other spatially or temporally varying fields. With advanced bootstrapping techniques and adaptive binning, PyTurboSF can handle large datasets while maintaining statistical accuracy.

Features

---

• Fast structure function calculations in 1D, 2D, and 3D
• Optimized memory usage for large datasets
• Advanced bootstrapping with adaptive sampling indices
• Multiple structure function types: longitudinal, transverse, scalar, and combined
• Isotropic averaging for 2D and 3D data
• Parallel processing for improved performance
• Automatic convergence detection based on a standard error threshold (in physical units)
• Comprehensive statistical analysis

For detailed documentation and examples, see the PyTurbo_SF documentation.

Installation

---

The easiest method to install PyTurbo_SF is with pip:

console $ pip install pyturbo_sf

You can also fork/clone this repository to your local machine and install it locally with pip as well:

console $ pip install .

Dimension Names and Coordinates

---

PyTurbo_SF processes data stored in xarray Datasets, which should contain both the variables and the coordinate information. Below is a guide for data preparation in different dimensions:

1D Data

For 1D data, PyTurbo_SF expects a single dimension which can be either: - time: For time-series data - x, y, or z: For spatial data along a single axis

Coordinate Types and Units: - Time coordinates: Can be provided as: - Numeric values: seconds - Datetime objects: pandas.DatetimeIndex or numpy.datetime64 objects - Spatial coordinates: Typically in meters - All coordinates should use consistent units within a dataset

Structure for 1D Dataset: ```python import xarray as xr import numpy as np

Example 1: 1D spatial data along x-axis

nx = 1024 x = np.linspace(0, 100000, nx) # Spatial coordinate in meters (100 km domain) scalar_var = 5 + 2*np.sin(x/10000) # Example: temperature in °C

ds1dspatial = xr.Dataset( datavars={"temperature": ("x", scalarvar)}, coords={"x": x}, attrs={ "description": "Temperature variation along a horizontal transect", "unitsx": "meters", "unitstemperature": "celsius" } )

Example 2: Time series with datetime coordinates

import pandas as pd from datetime import datetime, timedelta

Create a time series with datetime objects

startdate = datetime(2023, 1, 1, 0, 0, 0) # January 1, 2023 nhours = 720 # 30 days of hourly data dates = [startdate + timedelta(hours=h) for h in range(nhours)] time_index = pd.DatetimeIndex(dates) # Creates pandas DatetimeIndex

Generate sample meteorological data

temperature = 15 + 5np.sin(np.arange(n_hours)2np.pi/24) + np.random.normal(0, 1, n_hours) # Daily cycle + noise humidity = 70 + 10np.sin(np.arange(nhours)2np.pi/24 + np.pi) + np.random.normal(0, 3, nhours) # Out of phase with temp

dsdatetime = xr.Dataset( datavars={ "temperature": ("time", temperature), # Temperature in °C "humidity": ("time", humidity) # Relative humidity in % }, coords={"time": timeindex}, attrs={ "description": "Hourly meteorological data with datetime coordinates", "unitstemperature": "celsius", "unitshumidity": "percent", "samplingfrequency": "hourly" } )

Example 3: Oceanographic vertical profile

nz = 500 z = np.linspace(0, 1000, nz) # Depth in meters (0-1000m) temperature = 20 * np.exp(-z/200) + 5 # Temperature decreasing with depth salinity = 35 + 0.01 * z # Salinity increasing with depth

dsprofile = xr.Dataset( datavars={ "temperature": ("z", temperature), # Temperature in °C "salinity": ("z", salinity) # Salinity in PSU }, coords={"z": z}, attrs={ "description": "Oceanographic vertical profile", "unitsz": "meters", "unitstemperature": "celsius", "unitssalinity": "PSU", "verticalresolution": f"{z[1]-z[0]} meters" } ) ```

2D Data

For 2D data, PyTurbo_SF supports the following dimension combinations: - (y, x): Common for 2D planar data (horizontal plane) - (z, x): Vertical slice in x-direction - (z, y): Vertical slice in y-direction - (space,time): space can be x,y or z

The order of dimensions is important as it affects how PyTurbo_SF interprets your data and calculates structure functions.

Coordinate Types and Units: - For physical domains, coordinates should typically be in physical units: - x, y: Spatial horizontal coordinates in consistent units (e.g., meters, kilometers) - z: Vertical (stretched) coordinate in consistent units (e.g., meters, kilometers)

Structure for 2D Dataset: ```python

Example: 2D data in (y, x) plane with physical units

nx, ny = 256, 256 x = np.linspace(0, 10000, nx) # x-coordinates in meters (0-10 km domain) y = np.linspace(0, 10000, ny) # y-coordinates in meters (0-10 km domain) X, Y = np.meshgrid(x, y) # 2D coordinate grids

Create velocity components (e.g., in m/s)

u = 5 * np.sin(X/1000) * np.cos(Y/1000) # u-velocity in m/s v = -5 * np.cos(X/1000) * np.sin(Y/1000) # v-velocity in m/s T = 20 + 2 * np.sin((X+Y)/2000) # Temperature in °C

For structured grid (regularly spaced coordinates)

ds2dstructured = xr.Dataset( datavars={ "u": (["y", "x"], u), # Note the order of dimensions "v": (["y", "x"], v), # Must match the coords definition "temperature": (["y", "x"], T), # Example scalar field }, coords={ "x": (["y", "x"], X),
"y": (["y", "x"], Y)
}, attrs={ "description": "2D flow field in horizontal plane", "unitsx": "meters", "unitsy": "meters", "unitsu": "m/s", "unitsv": "m/s", "unitstemperature": "celsius" } )

Example: 2D data in (z, x) plane (vertical section)

nz, nx = 100, 256 x = np.linspace(0, 100000, nx) # Horizontal distance (100 km) z = np.linspace(0, 10000, nz) # Depth/height (0-10 km) X, Z = np.meshgrid(x, z)

Create velocity components

u = 5 * np.sin(X/10000) * np.exp(-Z/1000) # Horizontal velocity (m/s) w = 0.1 * np.cos(X/10000) * np.exp(-Z/2000) # Vertical velocity (m/s)

dszxplane = xr.Dataset( datavars={ "u": (["z", "x"], u), # Note the dimension order here! "w": (["z", "x"], w), # Using u and w for horiz/vert velocities }, coords={ "x": (["z", "x"], X), # Horizontal coordinate (meters) "z": (["z", "x"], Z) # Vertical coordinate (meters) }, attrs={ "description": "Vertical slice in x-direction", "unitsx": "meters", "unitsz": "meters", "unitsu": "m/s", "units_w": "m/s" } ) ```

3D Data

For 3D data, PyTurbo_SF expects dimensions in the order (z, y, x) or (time, y, x) or (time, y, z) or (time, z, x):

Important Note for Geophysical Flows: Since in most geophysical fluid flows the vertical length scale is much smaller than the horizontal scale, we advise users to use a stretched coordinate system:

$z^* = \int \frac{N}{f} dz'$

where N is the buoyancy frequency and f is the Coriolis parameter. This transformation helps normalize the structure function calculations across different scales.

Structure for 3D Dataset: ```python

Example: 3D data in (z, y, x) space

nx, ny, nz = 64, 64, 64 x = np.linspace(0, 2np.pi, nx) y = np.linspace(0, 2np.pi, ny) z = np.linspace(0, 2*np.pi, nz)

For a regular grid, use meshgrid

X, Y, Z = np.meshgrid(x, y, z, indexing='ij')

Example velocity field (simple vortex)

u = np.sin(X) * np.cos(Y) * np.cos(Z) v = -np.cos(X) * np.sin(Y) * np.cos(Z) w = 0.3 * np.sin(Z)

For structured 3D grid, include full coordinate arrays

ds3dstructured = xr.Dataset( data_vars={ "u": (["z", "y", "x"], u), "v": (["z", "y", "x"], v), "w": (["z", "y", "x"], w), }, coords={ "x": (["z", "y", "x"], X), "y": (["z", "y", "x"], Y), "z": (["z", "y", "x"], Z), } ) ```

Structure Functions and Their Mathematical Definitions

---

Structure functions quantify the statistical relationships between points separated by a given distance. PyTurbo_SF implements various structure functions for different dimensions and data types.

1D Structure Functions

For 1D data, PyTurbo_SF supports:

Scalar Structure Function

For a scalar field $f(x)$, the scalar structure function of order $n$ at separation $r$ is:

$$Sn(r) = \langle |f(x + r) - f(x)|^n \ranglex$$

where $\langle \cdot \rangle_x$ denotes averaging over all positions $x$.

Scalar-Scalar Structure Function

For two scalar fields $f(x)$ and $g(x)$, the cross-scalar structure function of orders $n$ and $k$ is:

$$S{n,k}(r) = \langle |f(x + r) - f(x)|^n \cdot |g(x + r) - g(x)|^k \ranglex$$

2D Structure Functions

For 2D fields, PyTurbo_SF provides multiple types of structure functions:

Longitudinal Structure Function

For velocity components $(u, v)$ in a 2D field, the longitudinal structure function measures the component of velocity difference parallel to the separation vector:

$$S{\parallel,n}(r) = \langle ((\vec{u}(\vec{x} + \vec{r}) - \vec{u}(\vec{x})) \cdot \frac{\vec{r}}{|\vec{r}|})^n \rangle{\vec{x}}$$

where $\vec{u} = (u, v)$ and $\vec{r}$ is the separation vector.

Transverse Structure Function

The transverse structure function measures the component of velocity difference perpendicular to the separation vector:

$$S{\perp,n}(r) = \langle ((\vec{u}(\vec{x} + \vec{r}) - \vec{u}(\vec{x})) \times \frac{\vec{r}}{|\vec{r}|})^n \rangle{\vec{x}}$$

Default Velocity Structure Function

The default velocity structure function computes the sum of the structure functions for each velocity component's differences:

$S{n}(r) = \langle |u(\vec{x} + \vec{r}) - u(\vec{x})|^n + |v(\vec{x} + \vec{r}) - v(\vec{x})|^n \rangle{\vec{x}}$

where $\vec{x}$ represents the position vector and $\vec{r}$ is the separation vector.

Scalar Structure Function (2D)

Similar to 1D but for 2D scalar field:

$$Sn(r) = \langle |f(\vec{x} + \vec{r}) - f(\vec{x})|^n \rangle{\vec{x}}$$

Scalar-Scalar Structure Function (2D)

For two scalar fields in 2D:

$$S{n,k}(r) = \langle |f(\vec{x} + \vec{r}) - f(\vec{x})|^n \cdot |g(\vec{x} + \vec{r}) - g(\vec{x})|^k \rangle{\vec{x}}$$

Longitudinal-Transverse Structure Function

Combines longitudinal and transverse components:

$$S{\parallel\perp,n,k}(r) = \langle ((\vec{u}(\vec{x} + \vec{r}) - \vec{u}(\vec{x})) \cdot \frac{\vec{r}}{|\vec{r}|})^n \cdot ((\vec{u}(\vec{x} + \vec{r}) - \vec{u}(\vec{x})) \times \frac{\vec{r}}{|\vec{r}|})^k \rangle{\vec{x}}$$

Longitudinal-Scalar Structure Function

Combines longitudinal velocity with scalar field:

$$S{\parallel S,n,k}(r) = \langle ((\vec{u}(\vec{x} + \vec{r}) - \vec{u}(\vec{x})) \cdot \frac{\vec{r}}{|\vec{r}|})^n \cdot |f(\vec{x} + \vec{r}) - f(\vec{x})|^k \rangle{\vec{x}}$$

Transverse-Scalar Structure Function

Combines transverse velocity with scalar field:

$$S{\perp S,n,k}(r) = \langle ((\vec{u}(\vec{x} + \vec{r}) - \vec{u}(\vec{x})) \times \frac{\vec{r}}{|\vec{r}|})^n \cdot |f(\vec{x} + \vec{r}) - f(\vec{x})|^k \rangle{\vec{x}}$$

Advective Structure Function

Combines velocity differences with advective velocity differences:

$$S{adv,n}(r) = \langle ((\vec{u}(\vec{x} + \vec{r}) - \vec{u}(\vec{x})) \cdot (\vec{u}{adv}(\vec{x} + \vec{r}) - \vec{u}{adv}(\vec{x})))^n \rangle{\vec{x}}$$

Pressure Work Structure Function (2D)

Measures the structure function of pressure-velocity work term:

$$S{pw,n}(r) = \langle (\nabla \cdot (δP \cdot δ\vec{u}))^n \rangle{\vec{x}}$$

where $δP = P(\vec{x} + \vec{r}) - P(\vec{x})$ is the pressure increment and $δ\vec{u}$ is the velocity increment.

3D Structure Functions

PyTurbo_SF extends the structure functions to 3D fields with velocity components $(u, v, w)$:

Longitudinal Structure Function (3D)

$$S{\parallel,n}(r) = \langle ((\vec{u}(\vec{x} + \vec{r}) - \vec{u}(\vec{x})) \cdot \frac{\vec{r}}{|\vec{r}|})^n \rangle{\vec{x}}$$

where $\vec{u} = (u, v, w)$ and $\vec{r}$ is the 3D separation vector.

Default Velocity Structure Function (3D)

Sums the structure functions for each velocity component:

$$S{n}(r) = \langle |u(\vec{x} + \vec{r}) - u(\vec{x})|^n + |v(\vec{x} + \vec{r}) - v(\vec{x})|^n + |w(\vec{x} + \vec{r}) - w(\vec{x})|^n \rangle{\vec{x}}$$

Transverse Structure Functions in Different Planes

PyTurbo_SF offers transverse structure functions in specific planes:

• Transverse_ij: In the xy-plane
• Transverse_ik: In the xz-plane
• Transverse_jk: In the yz-plane

Scalar and Combined Structure Functions

Similar to 2D, PyTurbo_SF offers scalar, scalar-scalar, longitudinal-scalar, and transverse-scalar structure functions in 3D.

3D Advective Structure Function

$$S{adv,n}(r) = \langle ((\vec{u}(\vec{x} + \vec{r}) - \vec{u}(\vec{x})) \cdot (\vec{u}{adv}(\vec{x} + \vec{r}) - \vec{u}{adv}(\vec{x})))^n \rangle{\vec{x}}$$

3D Pressure Work Structure Function

Measures the structure function of pressure-velocity work term in 3D:

$$S{pw,n}(r) = \langle (\nabla \cdot (δP \cdot δ\vec{u}))^n \rangle{\vec{x}}$$

where $δP = P(\vec{x} + \vec{r}) - P(\vec{x})$ is the pressure increment and $δ\vec{u}$ is the velocity increment vector.

Structure Function Keys

1D Structure Functions

• Scalar structure function: fun='scalar'
• Scalar-scalar structure function: fun='scalar_scalar'

2D Structure Functions

• Longitudinal structure function: fun='longitudinal'
• Transverse structure function: fun='transverse'
• Default velocity structure function: fun='default_vel'
• Scalar structure function: fun='scalar'
• Scalar-scalar structure function: fun='scalar_scalar'
• Longitudinal-transverse structure function: fun='longitudinal_transverse'
• Longitudinal-scalar structure function: fun='longitudinal_scalar'
• Transverse-scalar structure function: fun='transverse_scalar'
• Advective structure function: fun='advective'
• Pressure work structure function: fun='pressure_work'

3D Structure Functions

• Longitudinal structure function: fun='longitudinal'
• Default velocity structure function: fun='default_vel'
• Transverse in xy-plane: fun='transverse_ij'
• Transverse in xz-plane: fun='transverse_ik'
• Transverse in yz-plane: fun='transverse_jk'
• Scalar structure function: fun='scalar'
• Scalar-scalar structure function: fun='scalar_scalar'
• Longitudinal-scalar structure function: fun='longitudinal_scalar'
• Transverse-scalar structure function (xy-plane): fun='transverse_ij_scalar'
• Transverse-scalar structure function (xz-plane): fun='transverse_ik_scalar'
• Transverse-scalar structure function (yz-plane): fun='transverse_jk_scalar'
• Longitudinal-transverse structure function (xy-plane): fun='longitudinal_transverse_ij'
• Longitudinal-transverse structure function (xz-plane): fun='longitudinal_transverse_ik'
• Longitudinal-transverse structure function (yz-plane): fun='longitudinal_transverse_jk'
• Advective structure function: fun='advective'
• Pressure work structure function: fun='pressure_work'

Quickstart

---

Once PyTurbo_SF is installed, you can perform structure function calculations on your data:

1D Example

```python import numpy as np import xarray as xr import pyturbo_sf

Create sample 1D dataset (time series)

n = 10000 time = np.arange(n) * 0.01 # Time in seconds signal = np.sin(2np.pi0.1time) + 0.2np.random.randn(n) # Noisy sine wave

Create xarray Dataset

ds = xr.Dataset( data_vars={"signal": ("time", signal)}, coords={"time": time} )

Define logarithmic bins

bins = {'time': np.logspace(-2, 1, 20)}

Calculate 1D structure function

sfresult = pyturbosf.binsf1d( ds=ds, variablesnames=["signal"], order=2, bins=bins, fun='scalar', bootsize=100, initialnbootstrap=50, maxnbootstrap=200, convergenceeps=0.1, backend='loky' )

Wall time: 18.9 s

```

2D Example

```python import numpy as np import xarray as xr import pyturbo_sf

Create sample 2D dataset

nx, ny = 256, 256 x = np.linspace(0, 2np.pi, nx) y = np.linspace(0, 2np.pi, ny) X, Y = np.meshgrid(x, y)

Create velocity components

u = np.sin(X) * np.cos(Y) v = -np.cos(X) * np.sin(Y)

Create xarray Dataset with 2D coordinates

ds = xr.Dataset( data_vars={ "u": (["y", "x"], u), "v": (["y", "x"], v), }, coords={ "x": (["y", "x"], X), "y": (["y", "x"], Y) } )

Define logarithmic bins

bins = { 'x': np.logspace(-2, 0, 20), 'y': np.logspace(-2, 0, 20) }

Calculate 2D structure function

sfresult = pyturbosf.binsf2d( ds=ds, variablesnames=["u", "v"], order=2, bins=bins, fun='longitudinal', bootsize={'x':32,'y':32}, initialnbootstrap=50, maxnbootstrap=200, convergenceeps=0.1, backend='loky' )

Wall time: 2.88 s

Calculate isotropic structure function

sfiso = pyturbosf.getisotropicsf2d( ds=ds, variablesnames=["u", "v"], order=2, bins={'r': np.logspace(-2, 0, 20)}, fun='longitudinal', backend='loky' )

Wall time: 12.2 s

```

3D Example

```python import numpy as np import xarray as xr import pyturbo_sf

Create sample 3D dataset

nx, ny, nz = 64, 64, 64 x = np.linspace(0, 2np.pi, nx) y = np.linspace(0, 2np.pi, ny) z = np.linspace(0, 2*np.pi, nz)

For a regular grid, create 3D arrays

Z,Y,X = np.meshgrid(z, y, x, indexing='ij')

Example velocity field (simple vortex)

u = np.sin(X) * np.cos(Y) * np.cos(Z) v = -np.cos(X) * np.sin(Y) * np.cos(Z) w = 0.3 * np.sin(Z)

Create the dataset

ds = xr.Dataset( data_vars={ "u": (["z", "y", "x"], u), "v": (["z", "y", "x"], v), "w": (["z", "y", "x"], w), }, coords={ "x": (["z", "y", "x"], X), "y": (["z", "y", "x"], Y), "z": (["z", "y", "x"], Z) } )

Define logarithmic bins

bins = { 'x': np.logspace(-2, 0, 15), 'y': np.logspace(-2, 0, 15), 'z': np.logspace(-2, 0, 15) }

Calculate 3D structure function

sfresult = pyturbosf.binsf3d( ds=ds, variablesnames=["u", "v", "w"], order=2, bins=bins, fun='longitudinal', bootsize={'x':16,'y':16,'z':16}, initialnbootstrap=30, maxnbootstrap=100, convergenceeps=0.15, backend='threading' )

Wall time: 1min 47s

Calculate isotropic 3D structure function

sfiso = pyturbosf.getisotropicsf3d( ds=ds, variablesnames=["u", "v", "w"], order=2, bins={'r': np.logspace(-2, 0, 15)}, fun='longitudinal', bootsize={'x':16,'y':16,'z':16}, initialnbootstrap=30, maxnbootstrap=100, convergence_eps=0.15, backend='threading'

)

Wall time: 24.5 s ```

"Can I use PyTurbo_SF with my data?"

---

PyTurbo_SF is designed to work with various types of data, provided they can be organized into an xarray Dataset with proper dimensional information. It supports:

• Turbulence simulation data
• Fluid flow measurements
• Meteorological and oceanographic data
• Time series
• Any structured (uniform or non-uniform grid) 1D, 2D, or 3D data in cartesian Grid

If you are uncertain about using PyTurbo_SF with your specific dataset, please open an issue or start a discussion in the GitHub repository.

Performance Benchmarks

---

PyTurbo_SF is optimized for both speed and memory efficiency. Our benchmarks show computational complexity scaling as O(N log N), making it suitable for large datasets.

2D DYCOMS Benchmarking

We conducted comprehensive performance benchmarks using 2D DYCOMS (DYnamics and Chemistry of Marine Stratocumulus) turbulence data to evaluate PyTurbo_SF's scalability and convergence behavior. The benchmarks tested various grid sizes while maintaining consistent bootstrap parameters.

Benchmark Setup: - Grid Sizes: 512×512, 1024×1024, 2048×2048, 4096×4096 - Bootstrap Size: Fixed at 16×16 for all tests - Convergence Threshold (ε): Varied to study convergence behavior - Structure Function: 2nd-order longitudinal structure function - Data: 2D DYCOMS marine stratocumulus turbulence simulation

Key Findings: - Excellent scalability across all tested grid sizes - Consistent convergence behavior independent of domain size - Memory-efficient processing even for the largest 4096×4096 grids - Adaptive bootstrap algorithm effectively balances accuracy and computational cost

The benchmark results demonstrate PyTurbo_SF's capability to handle large-scale turbulence datasets efficiently while maintaining statistical accuracy through adaptive convergence criteria.

Contributing

---

This project welcomes contributions and suggestions. Feel free to open an issue, submit a pull request, or contact the maintainers directly.

Funding Acknowledgement

---

This software package is based upon work supported by the US Department of Energy grant DE-SC0024572.

Any opinions, findings, and conclusions or recommendations expressed in this package are those of the authors and do not necessarily reflect the views of the US Department of Energy.

References

---

• Pearson, B. et al., 2021: Advective structure functions in anisotropic two-dimensional turbulence. Journal of Fluid Mechanics.
• Lindborg, E. 2008: Third-order structure function relations for quasi-geostrophic turbulence. Journal of Fluid Mechanics.
• Kolmogorov, A.N. 1941: The local structure of turbulence in incompressible viscous fluid for very large Reynolds numbers. Proc. R. Soc. Lond. A.
• Frisch, U. 1995: Turbulence: The Legacy of A.N. Kolmogorov. Cambridge University Press.
• Pope, S.B. 2000: Turbulent Flows. Cambridge University Press.