ndx-miniscope Extension for NWB

This is a Neurodata Extension (NDX) for Neurodata Without Borders (NWB) 2.0 for Miniscope acquisition data.

Miniscope extends the Device core NWB neurodata_type by including additional metadata for the Miniscope. Depending on the version of the acquisition software the data structure can be quite different.

Miniscope V4 format

The data recorded by the software is saved in a folder structure similar to this:

    C6-J588_Disc5/ (main folder)
    ├── 15_03_28/ (subfolder corresponding to the recording time)
    │   ├── Miniscope/ (subfolder containing the microscope video stream)
    │   │   ├── 0.avi (microscope video)
    │   │   ├── metaData.json (metadata for the microscope device)
    │   │   └── timeStamps.csv (timing of this video stream)
    │   ├── BehavCam_2/ (subfolder containing the behavioral video stream)
    │   │   ├── 0.avi (bevavioral video)
    │   │   ├── metaData.json (metadata for the behavioral camera)
    │   │   └── timeStamps.csv (timing of this video stream)
    │   └── metaData.json (metadata for the recording, such as the start time)
    ├── 15_06_28/
    │   ├── Miniscope/
    │   ├── BehavCam_2/
    │   └── metaData.json
    └── 15_12_28/

Miniscope V3 format

The Miniscope V3 acquisition software generally outputs the following files:

• msCam[##].avi
• behavCam[##].avi
• timestamp.dat
• settingsandnotes.dat

python

Convert to NWB using NeuroConv

Use the MiniscopeConverter from NeuroConv to easily convert Miniscope acquisition data to NWB.

Install NeuroConv with the additional dependencies necessary for reading Miniscope data. bash pip install neuroconv[miniscope]

The MiniscopeConverter combines the imaging and behavior data streams into a single conversion.

```python from dateutil import tz from neuroconv.converters import MiniscopeConverter

The 'folder_path' is the path to the main Miniscope folder containing both the recording and behavioral data streams in separate subfolders.

folderpath = "C6-J588Disc5/" converter = MiniscopeConverter(folderpath=folderpath, verbose=False)

metadata = converter.get_metadata()

For data provenance we can add the time zone information to the conversion if missing

sessionstarttime = metadata["NWBFile"]["sessionstarttime"] tzinfo = tz.gettz("US/Pacific") metadata["NWBFile"].update(sessionstarttime=sessionstarttime.replace(tzinfo=tzinfo))

Choose a path for saving the nwb file and run the conversion

nwbfilepath = "miniscope.nwb" converter.runconversion(nwbfilepath=nwbfilepath, metadata=metadata) ```

Access the data from NWB

Access the Miniscope devices from the in-memory NWBFile. ```python from pynwb import NWBHDF5IO

nwbfilepath = "miniscope.nwb" with NWBHDF5IO(nwbfilepath, "r") as io: nwbfilein = io.read() # Access the device with the microscope metadata nwbfilein.devices["Miniscope"] # Access the device that holds the metadata for the behavior camera nwbfilein.devices["BehavCam2"] Miniscope abc.Miniscope at 0x5775754960 Fields: compression: FFV1 deviceType: MiniscopeV3 frameRate: 15FPS framesPerFile: 1000 gain: High led0: 47

BehavCam2 abc.Miniscope at 0x5775972816 Fields: ROI: compression: MJPG deviceType: WebCam-1920x1080 framesPerFile: 1000 The imaging data was added to the `NWBFile` as `OnePhotonSeries` which can be accessed from the file as the follows: python from pynwb import NWBHDF5IO

nwbfilepath = "miniscope.nwb" with NWBHDF5IO(nwbfilepath, "r") as io: nwbfilein = io.read() # Access the OnePhotonSeries that holds the imaging data from the microscope. nwbfile.acquisition["OnePhotonSeries"] OnePhotonSeries pynwb.ophys.OnePhotonSeries at 0x5775755728 Fields: comments: no comments conversion: 1.0 data: description: Imaging data from one-photon excitation microscopy. dimension: imagingplane: ImagingPlane pynwb.ophys.ImagingPlane at 0x5775755200 Fields: conversion: 1.0 description: The plane or volume being imaged by the microscope. device: Miniscope abc.Miniscope at 0x5775754960 Fields: compression: FFV1 deviceType: MiniscopeV3 frameRate: 15FPS framesPerFile: 1000 gain: High led0: 47 excitationlambda: nan imagingrate: 15.0 indicator: unknown location: unknown opticalchannel: ( OpticalChannel ) unit: meters interval: 1 offset: 0.0 resolution: -1.0 timestamps: timestamps_unit: seconds unit: px The behavior camera data was added to the `NWBFile` as `ImageSeries` which can be accessed from the file as the follows: python from pynwb import NWBHDF5IO

nwbfilepath = "miniscope.nwb" with NWBHDF5IO(nwbfilepath, "r") as io: nwbfilein = io.read() # Access the ImageSeries that holds the behavior data. nwbfile.acquisition["BehavCamImageSeries"] BehavCamImageSeries pynwb.image.ImageSeries at 0x5775971616 Fields: comments: no comments conversion: 1.0 data: description: no description device: BehavCam2 abc.Miniscope at 0x5775972816 Fields: ROI: compression: MJPG deviceType: WebCam-1920x1080 framesPerFile: 1000 dimension: externalfile: format: external interval: 1 offset: 0.0 resolution: -1.0 startingframe: [ 0 690 1383 2073 2763] timestamps: timestampsunit: seconds unit: px ```

For more information about accessing data in NWB, visit the File Basics tutorial. To learn more about NeuroConv, visit this documentation page.

Installing ndx-miniscope

Get most recent release: bash pip install ndx-miniscope

Install latest: bash git clone https://github.com/catalystneuro/ndx-miniscope.git cd ndx-miniscope pip install -e .

The following code demonstrates the usage of this extension to convert Miniscope acquisition data into NWB.

Usage

```python from datetime import datetime from dateutil.tz import tzlocal import glob import os from pynwb import NWBFile, NWBHDF5IO from pynwb.image import ImageSeries from natsort import natsorted

from ndxminiscope.utils import ( addminiscopedevice, getstartingframes, gettimestamps, readminiscopeconfig, read_notes, )

The main folder that contains subfolders with the Miniscope data

folderpath = "C6-J588Disc5/"

Create the NWBFile

sessionstarttime = datetime(2017, 4, 15, 12, tzinfo=tzlocal()) nwbfile = NWBFile( sessiondescription="sessiondescription", identifier="identifier", sessionstarttime=sessionstarttime, )

Load the miscroscope settings

miniscopefolderpath = "C6-J588Disc5/150328/Miniscope/" miniscopemetadata = readminiscopeconfig(folderpath=miniscopefolder_path)

Create the Miniscope device with the microscope metadata and add it to NWB

addminiscopedevice(nwbfile=nwbfile, devicemetadata=miniscopemetadata)

Load the behavioral camera settings

behavcamfolderpath = "C6-J588Disc5/150328/BehavCam2/" behavcammetadata = readminiscopeconfig(folderpath=behavcamfolderpath)

Create the Miniscope device with the behavioral camera metadata and add it to NWB

addminiscopedevice(nwbfile=nwbfile, devicemetadata=behavcammetadata)

Loading the timestamps

behavcamtimestamps = gettimestamps(folderpath=folderpath, file_pattern="BehavCam*/timeStamps.csv")

Load the starting frames of the video files

Note this function requires to have cv2 installed

startingframes = getstartingframes(folderpath=folderpath, videofile_pattern="/BehavCam/*.avi")

Legacy usage for Miniscope V3

msfiles = natsorted(glob(os.path.join(folderpath, 'msCam.avi'))) nwbfile.addacquisition( ImageSeries( name='OnePhotonSeries', # this is not recommended since pynwb has native OnePhotonSeries format='external', externalfile=[os.path.split(x)[1] for x in msfiles], timestamps=gettimestamps(folderpath=folderpath, camnum=1), startingframe=getstartingframes(folderpath=folderpath, videofilepattern="msCam.avi"), ) )

behavfiles = natsorted(glob(os.path.join(folderpath, 'behavCam.avi'))) nwbfile.addacquisition( ImageSeries( name='behaviorCam', format='external', externalfile=[os.path.split(x)[1] for x in behavfiles], timestamps=gettimestamps(folderpath=folderpath, camnum=2), startingframe=getstartingframes(folderpath=folderpath, videofilepattern="behavCam.avi"), ) )

annotations = readnotes(folderpath=folderpath) if annotations is not None: nwbfile.addacquisition(annotations)

savepath = os.path.join(folderpath, "testout.nwb") with NWBHDF5IO(savepath, "w") as io: io.write(nwbfile)

test read

with NWBHDF5IO(savepath, "r") as io: nwbfilein = io.read()

```

MATLAB:

Installation

bash git clone https://github.com/bendichter/ndx-miniscope.git matlab generateExtension('path/to/ndx-miniscope/spec');

Usage

under construction...