ndx-patterned-ogen Extension for NWB

We release ten PyNWB containers as part of this extension (we currently only have a Python implementation, rather than both Python and a MATLAB ones -- this is why the matnwb directory is empty):

• The SpatialLightModulator2D/SpatialLightModulator3D and LightSource containers store metadata about the spatial light modulator (either bidimensional or threedimensional) and light source used in the photostimulation, respectively. These containers are then linked to the PatternedOptogeneticStimulusSite parent container, which stores the remaining photostimulation method-specifici metadata.
• OptogeneticStimulusPattern stores parameters for a generic photostimulation pattern.
  • TemporalFocusing stores parameters associated with the temporal focusing pattern.
  • SpiralScanning stores parameters associated with the spiral scanning pattern.
• OptogeneticStimulusTarget container stored a subset of targeted ROIs targeted_rois, that is a DynamicTableRegion referencing the rows of a PlaneSegmentation. Optionally, we can store the corresponding segmented ROIs that have been successfully photostimulated. segmented_rois is also a DynamicTableRegion referencing the rows of a PlaneSegmentation. Since not all targeted ROIs may result in an actual photostimulation a global_roi_ids column should be added to both PlaneSegmentation object to express the correspondence between the targeted and segmented ROIs.
• PhotostimulationTable is an TimeIntervals table. Each row stores a stimulus onset - defined by start, stop, power (optionally frequency and pulse_width). Each stimulus onset reference a specific OptogeneticStimulusTarget and PhotostimulationPattern NB: When power(frequency and pulse_width) is defined, its value apply to all the ROIs in targets. When power_per_roi(frequency_per_roi and pulse_width_per_roi) id defined, the length must equal to the number of ROIs in targets, and each element refers to a specific targeted ROI.

Background

State-of-the-art patterned photostimulation methods, used in concert with two-photon imaging, allow unprecedented control and measurement of cell activity in the living brain. Methods for managing data for two-photon imaging experiments are improving, but there is little to no standardization of data for these stimulation methods. Stimulation in vivo depends on fine-tuning many experimental variables, which poses a challenge for reproducibility and data sharing between researchers. To improve standardization of photostimulation data storage and processing, we release this extension as a generic data format for simultaneous patterned optogenetic stimulation experiments, using the NWB format to store experimental details and data relating to both acquisition and photostimulation.

Installation

To install the extension, first clone the ndx-patterned-ogen repository to the desired folder using the command angular2svg git clone https://github.com/https://github.com/catalystneuro/ndx-patterned-ogen.git Then, to install the requisite python packages and extension, run: angular2svg python -m pip install -r requirements.txt -r requirements-dev.txt python setup.py install The extension can then be imported into python scripts via import ndx_patterned_ogen.

Usage

For full example usage, see tutorial.ipynb

Example Usage for the ndx-patterne-ogen-stimulation for 2D stimulus

In the following tutorial, we demonstrate use of the ndx-patterned-ogen extension to the NWB data standard. Specifically we: 1. Create SpatialLightModulator2D and LightSource containers, representing the devices used in the paradigm. 2. Use the PatternedOptogeneticStimulusSite container to store information about location, the opsin and excitation wavelength used in the paradigm 3. Use the OptogeneticStimulus2DPattern (or SpiralScanning or TemporalFocusing) container to store the pattern-specific parameters of the stimulus onset. 4. Record the stimulus presentation within the PatternedOptogeneticStimulusTable container 6. Write all devices, stimuli, and presentation tables to an .nwb file and confirm it can be read back

```python

First, we import then necessary files and create an empty NWBFile.

import datetime import numpy as np from pynwb import NWBFile, NWBHDF5IO from ndxpatternedogen import ( SpatialLightModulator2D, LightSource, PatternedOptogeneticStimulusSite, PatternedOptogeneticStimulusTable, OptogeneticStimulus2DPattern, OptogeneticStimulusTarget, SpiralScanning, TemporalFocusing, ) from hdmf.common.table import DynamicTableRegion

from pynwb.ophys import PlaneSegmentation, ImageSegmentation, OpticalChannel

nwbfile = NWBFile( sessiondescription="patterned optogenetic synthetic experiment (all optical system)", identifier="identifier", sessionstart_time=datetime.datetime.now(datetime.timezone.utc), )

metadata for spiatial light modulator

spatiallightmodulator = SpatialLightModulator2D( name="SpatialLightModulator2D", description="Generic description for the slm", model="slm model", manufacturer="slm manufacturer", spatialresolution=[512, 512], ) nwbfile.adddevice(spatiallightmodulator)

metadata for the light source

lightsource = LightSource( name="Laser", model="laser model", manufacturer="laser manufacturer", stimulationwavelength=1035.0, # nm filterdescription="Short pass at 1040 nm", description="Generic description for the laser", peakpower=70e-3, # the peak power of stimulation in Watts intensity=0.005, # the intensity of excitation in W/mm^2 exposuretime=2.51e-13, # the exposure time of the sample in seconds pulserate=1 / 2.51e-13, # the pulse rate of the laser is in Hz ) nwbfile.adddevice(lightsource)

metadata for the microscope

microscope = nwbfile.createdevice( name="2Pmicroscope", description="My two-photon microscope", manufacturer="The best microscope manufacturer", )

metadata for the stimulus methods

site = PatternedOptogeneticStimulusSite( name="PatternedOptogeneticStimulusSite", description="Scanning", # Scanning or scanless method for shaping optogenetic light (e.g., diffraction limited points, 3D shot, disks, etc.). excitationlambda=600.0, # nm effector="ChR2", location="VISrl", device=microscope, spatiallightmodulator=spatiallightmodulator, lightsource=lightsource, ) nwbfile.addogen_site(site)

For demonstrative purpose, we define here fout different stimulation pattern:

1. two generic where the sweep_size and the sweep_mask can be defined to describe the spatial pattern. If sweep_size is a scalar, the sweep pattern is assumed to be a circle with diameter sweep_size. If sweep_size is a two or three dimensional array, the the sweep pattern is assumed to be a rectangle, with dimensions [width, height]. If the shape is neither a circle or a rectangular, the shape can be save in sweep_mask.

2. one spiral pattern

3. one temporal focusing beam pattern

metadata for a generic stimulus pattern

import numpy as np import matplotlib.pyplot as plt

auxiliary function to generate the sweep shape, either circular or rectangular

def generateimagemasknp(width, height, sweepsizeinpizels): # Create a black image mask image_mask = np.zeros((height, width), dtype=np.uint8)

# Calculate the position for the center of the white spot
center_x = width // 2
center_y = height // 2

if isinstance(sweep_size_in_pizels, int):
    # Circular spot
    Y, X = np.ogrid[:height, :width]
    dist_from_center = np.sqrt((X - center_x) ** 2 + (Y - center_y) ** 2)
    image_mask[dist_from_center <= sweep_size_in_pizels / 2] = 255

elif len(sweep_size_in_pizels) == 2:
    # Rectangular spot
    half_width = sweep_size_in_pizels[0] // 2
    half_height = sweep_size_in_pizels[1] // 2
    top_left = (center_x - half_width, center_y - half_height)
    bottom_right = (center_x + half_width, center_y + half_height)
    image_mask[top_left[1] : bottom_right[1], top_left[0] : bottom_right[0]] = 255
else:
    raise ValueError("Invalid sweep_size_in_pizels. Should be a scalar or a 2-element array.")
return image_mask

sweepsize = 8 circularimagemasknp = generateimagemasknp( width=sweepsize * 2, height=sweepsize * 2, sweepsizeinpizels=sweepsize ) # assuming 1 pixel=1 um genericcircularpattern = OptogeneticStimulus2DPattern( name="CircularOptogeneticStimulusPattern", description="circular pattern", sweepsize=sweepsize, # um # sweepmask=circularimagemasknp, ) nwbfile.addlabmetadata(genericcircularpattern)

Display the image mask using matplotlib

plt.imshow(circularimagemask_np, cmap="gray") plt.show()

sweepsize = [5, 10] rectangularimagemasknp = generateimagemasknp(width=20, height=20, sweepsizeinpizels=sweepsize) genericrectangularpattern = OptogeneticStimulus2DPattern( name="RectangularOptogeneticStimulusPattern", description="rectangular pattern", sweepsize=sweepsize, # um sweepmask=rectangularimagemasknp, ) nwbfile.addlabmetadata(genericrectangularpattern)

Display the image mask using matplotlib

plt.imshow(rectangularimagemask_np, cmap="gray") plt.show()

metadata for spiral scanning pattern

spiralscanning = SpiralScanning( name="SpiralScanning", diameter=15, # um height=10, # um numberofrevolutions=5, description="scanning beam pattern", ) nwbfile.addlabmetadata(spiral_scanning)

metadata for temporal focusing pattern

temporalfocusing = TemporalFocusing( name="TemporalFocusing", description="scanless beam pattern", lateralpointspreadfunction="9 um ± 0.7 um", axialpointspreadfunction="32 um ± 1.6 um", ) nwbfile.addlabmetadata(temporal_focusing)

Define two PlaneSegmentation tables; one for post-hoc ROI (possibly cell) identification; the other for targeted ROIs. Additional columns on both tables can indicate if the ROI is a cell, and the two tables can be harmonized with the use of a globalroiid field that matches ROI IDs from one table to the other.

To do so, we need to define an ImagingPlane and an OpticalChannel first.

opticalchannel = OpticalChannel( name="OpticalChannel", description="an optical channel", emissionlambda=500.0, ) imagingplane = nwbfile.createimagingplane( name="ImagingPlane", opticalchannel=opticalchannel, imagingrate=30.0, description="a very interesting part of the brain", device=microscope, excitationlambda=600.0, indicator="GFP", location="V1", gridspacing=[0.01, 0.01], gridspacingunit="meters", origincoords=[1.0, 2.0, 3.0], origincoords_unit="meters", )

All the ROIs simultaneously illuminated are stored in targeted_rois in an OptogeneticStimulusTarget container, as a table region referencing the TargetPlaneSegmentation.

In this example, the targeted ROIs are 45 in total, divided in 3 groups of 15 ROIs that will be simultaneously illuminated with the same stimulus pattern. Only 30 of them, 10 for each group, results in a successful photostimulation.

Therefore, we define a PlaneSegmentation containing 30 ROIs in total and 3 roi_table_region containing 10 ROIs each that would be segmented after being stimulated, and stored in three separate OptogeneticStimulusTarget containers.

ntargetedrois = 45 ntargetedroispergroup = ntargetedrois // 3

targetedroiscentroids = np.array([[i, i, 0] for i in np.arange(ntargetedrois, dtype=int)])

targetedplanesegmentation = PlaneSegmentation( name="TargetPlaneSegmentation", description="Table for storing the targeted roi centroids, defined by a one-pixel mask", imagingplane=imagingplane, )

for roicentroid in targetedroiscentroids: targetedplanesegmentation.addroi(pixelmask=[roicentroid])

if nwbfile is not None: if "ophys" not in nwbfile.processing: nwbfile.createprocessingmodule("ophys", "ophys") nwbfile.processing["ophys"].add(targetedplanesegmentation)

targetedrois1 = targetedplanesegmentation.createroitableregion( name="targetedrois", # it must be called "segmentedrois" description="targeted rois", region=list(np.arange(ntargetedroisper_group, dtype=int)), )

targetedrois2 = targetedplanesegmentation.createroitableregion( name="targetedrois", # it must be called "segmentedrois" description="targeted rois", region=list(np.arange(ntargetedroispergroup, 2 * ntargetedroisper_group, dtype=int)), )

targetedrois3 = targetedplanesegmentation.createroitableregion( name="targetedrois", # it must be called "segmentedrois" description="targeted rois", region=list(np.arange(2 * ntargetedroispergroup, ntargeted_rois, dtype=int)), )

nsegmentedrois = 30 nsegmentedroispergroup = nsegmentedrois // 3

planesegmentation = PlaneSegmentation( name="PlaneSegmentation", description="output from segmenting my favorite imaging plane", imagingplane=imaging_plane, )

TODO add globalroiid

for _ in range(nsegmentedrois): planesegmentation.addroi(image_mask=np.zeros((512, 512)))

if nwbfile is not None: if "ophys" not in nwbfile.processing: nwbfile.createprocessingmodule("ophys", "ophys") nwbfile.processing["ophys"].add(plane_segmentation)

segmentedrois1 = planesegmentation.createroitableregion( name="segmentedrois", # it must be called "segmentedrois" description="segmented rois", region=list(np.arange(nsegmentedroispergroup, dtype=int)), )

segmentedrois2 = planesegmentation.createroitableregion( name="segmentedrois", description="segmented rois", region=list(np.arange(nsegmentedroispergroup, 2 * nsegmentedroisper_group, dtype=int)), )

segmentedrois3 = planesegmentation.createroitableregion( name="segmentedrois", description="segmented rois", region=list(np.arange(2 * nsegmentedroispergroup, nsegmented_rois, dtype=int)), )

hologram1 = OptogeneticStimulusTarget(name="Hologram1", segmentedrois=segmentedrois1, targetedrois=targetedrois1) nwbfile.addlabmetadata(hologram_1)

hologram2 = OptogeneticStimulusTarget(name="Hologram2", segmentedrois=segmentedrois2, targetedrois=targetedrois_2)

nwbfile.addlabmetadata(hologram2)

hologram3 = OptogeneticStimulusTarget(name="Hologram3", targetedrois=targetedrois3)

nwbfile.addlabmetadata(hologram3)

Define the stimulus sequences on the targeted ROIs previously defined in the imaging frame coordinates

If power,frequency and pulse_width are defined as a scalar it is assumed that all the ROIs defined in targets receive the same stimulus power,frequency and pulse_width. However, we can also define power,frequency and pulse_width as 1D arrays of dimension equal to the number of ROIs in targets, so we can define different power,frequency and pulse_width for each target.

stimulustable = PatternedOptogeneticStimulusTable( name="PatternedOptogeneticStimulusTable", description="Patterned stimulus" ) stimulustable.addinterval( starttime=0.0, stoptime=1.0, power=70e-3, frequency=20.0, pulsewidth=0.1, stimuluspattern=temporalfocusing, targets=nwbfile.labmetadata["Hologram1"], stimulussite=site, ) stimulustable.addinterval( starttime=0.5, stoptime=1.0, power=50e-3, stimuluspattern=spiralscanning, targets=hologram2, stimulussite=site, ) stimulustable.addinterval( starttime=0.8, stoptime=1.7, power=40e-3, frequency=20.0, pulsewidth=0.1, stimuluspattern=genericcircularpattern, targets=hologram3, stimulussite=site, ) nwbfile.addtimeintervals(stimulustable)

hologram3.addsegmentedrois(segmentedrois_3)

Write and read the NWB File

nwbfilepath = "basicstutorialpatternedogen.nwb" with NWBHDF5IO(nwbfile_path, mode="w") as io: io.write(nwbfile)

with NWBHDF5IO(nwbfilepath, mode="r") as io: nwbfilein = io.read()

```

Documentation

Specification

Documentation for the extension's specification, which is based on the YAML files, is generated and stored in the ./docs folder. To create it, run the following from the home directory: angular2svg cd docs make fulldoc This will save documentation to the ./docs/build folder, and can be accessed via the ./docs/build/html/index.html file.

API

To generate documentation for the Python API (stores in ./api_docs), we use Sphinx and a template from ReadTheDocs. API documentation can be created by running angular2svg sphinx-build -b html api_docs/source/ api_docs/build/ from the home folder. Similar to the specification docs, API documentation is stored in ./api_docs/build. Select ./api_docs/build/index.html to access the API documentation in a website format.

Credit

Code by Alessandra Trapani. Collaboration between the CatalystNeuro Team and Histed Lab.

This extension was created using ndx-template.