neuroconv - v0.8.0

v0.8.0 (August 21, 2025)

Removals, Deprecations and Changes

• Segmentation writing pipeline no longer supports writing segmentation data without image or pixel masks PR #1400
• Removed deprecated arguments: load_sync_channel in SpikeGLXNIDQInterface initialization and start_time, write_as and write_electrical_series in SpikeGLXNIDQInterface.add_to_nwbfile(). PR #1378
• Removed starting_time as an argument from the recording interfaces add_to_nwbfile method and the stand alone add_recording_segment utility PR #1378
• Deprecated the container_name parameter in ImageInterface.add_to_nwbfile() method. Use metadata_key in __init__ instead. This parameter will be removed on or after February 2026. PR #1439
• Removed deprecated type aliases FolderPathType, FilePath, OptionalFilePath, and OptionalFolderPathType from utils. Use pydantic.DirectoryPath, pydantic.FilePath, or their optional variants directly. PR #1442

Bug Fixes

• Fixed SpikeInterface physical unit properties being incorrectly included in electrodes table PR #1406
• Fixed deprecated ROI extractor method calls: replaced get_image_size() with get_frame_shape(), get_num_frames() with get_num_samples(), and frame_slice() with slice_samples() in ophys interfaces PR #1443
• Fixed logic bug in get_package function where boolean check was incorrectly compared to None PR #1477
• Fixed docstring typos: corrected "default: Falsee" to "default: False" in multiple datainterface files PR #1472

Features

• Segmentation interfaces now support roi ids that are strings PR #1390
• Added InscopixImagingInterface for converting Inscopix imaging data. PR #1361
• Added InscopixSegmentationInterface for converting Inscopix segmentation data. PR #1364
• Added AxonRecordingInterface for converting extracellular electrophysiology data from Axon Binary Format (ABF) files with automatic session start time extraction PR #1413
• Added FemtonicsImagingInterfacefor converting Femtonics imaging data. PR #1408
• Added get_available_subjects static method to DeepLabCutInterface for extracting subject names from DeepLabCut output files PR #1425
• Added MockPoseEstimationInterface for testing pose estimation workflows with deterministic Lissajous figure motion patterns PR #1435
• Added IntanAnalogInterface for converting non-amplifier analog streams from Intan data files, supporting RHD2000 auxiliary input channels, RHD2000 supply voltage channels, USB board ADC input channels, and DC amplifier channels (RHS system only) PR #1440
• Added metadata_key parameter to ImageInterface to allow custom naming and organization of image containers in NWB files. This enables multiple image interfaces to coexist with distinct container names. PR #1439
• Added per-image metadata support to ImageInterface allowing users to specify individual resolution (pixels/cm), name and description for each image through metadata structure. PR #1441
• Added rename_unit_ids() method to BaseSortingExtractorInterface for dictionary-based unit ID renaming, enabling clean handling of multiple sorting interfaces with overlapping unit IDs PR #1451
• Added support for setting ProbeGroup objects in BaseRecordingExtractorInterface.set_probe() methodPR #1464
• Added comprehensive tests for set_probe method in BaseRecordingExtractorInterface to verify probe and probe group functionality with proper electrode group organization in NWB files PR #1464
• Added PyData Sphinx Theme version switcher to documentation navbar, enabling users to switch between stable (latest release) and main (development) versions PR #1478

Improvements

• Added comprehensive FFmpeg video conversion how-to guide for converting bespoke video formats to DANDI-compatible formats PR #1426
• Refactored Femtonics Imaging Interface session, munit and channel selection logic. PR #1433
• Implemented PEP 735 dependency groups for test, docs, and dev dependencies PR #1434
• Expanded test coverage for CaimanSegmentationInterface to include all stub files and added quality metrics properties (rvalues, SNRcomp, cnnpreds) to the PlaneSegmentation table as segmentationextractor_properties PR #1436
• Added comprehensive how-to guide "Adding Multiple Sorting Interfaces" documenting approaches for handling multiple spike sorting outputs, including unit renaming strategies, separate processing tables, and adding custom properties for provenance tracking PR #1451 PR #1473
• The copy button no longer copies the prompt (>>>) in the conversion gallery PR 1467

- Python
Published by h-mayorquin 7 months ago

neuroconv - v0.7.5

v0.7.5 (June 11, 2025)

Removals, Deprecations and Changes

• Removed automatic subject addition for DeepLabCutInterface. A link is now created only if the skeleton metadata matches the subject ID. PR #1362

Bug Fixes

• Fix a bug for avoiding loading the sync stream in SpikeGLXConverterPipe PR #1373
• Fixed a bug in the BrukerTiffSinglePlaneImagingInterface where the criteria to identify frames belonging to a specific stream relied on the file name instead of the stream name. PR #1375
• Fixed a bug with the Docker dev build PR #1376

Features

• Added apply_global_compression method to BackendConfiguration classes to apply compression settings globally across all datasets in a backend configuration. This method allows users to easily configure compression options for all datasets at once rather than setting them individually. PR #1379
• Extra optional kwargs to BlackrockRecordingInterface and BlackrockSortingInterface for finer control of the neo reader PR #1290

Improvements

• Add a how to documentation for adding extracellular electrophysiology metadata PR #1311
• Improved the docker dailies PR #1372
• Re-enable and improve conversion gallery testing PR #1380
• Implemented cross-OS caches in GitHub Actions load-data action to enable cache sharing between Ubuntu, Windows, and macOS runners, reducing cache storage usage and improving CI efficiency PR #1385
• MedPC format is now tested on the conversion gallery PR #1382
• Added conversion gallery testing to daily workflows PR #1387
• Added full metadata support for PoseEstimation container in DeepLabCutInterface PR #1392.

- Python
Published by h-mayorquin 9 months ago

neuroconv - v0.7.4

v0.7.4 (May 23, 2025)

Removals, Deprecations and Changes

• Drop support for python 3.9 PR #1313
• Updated type hints to take advantage of the | operator PR #1316
• Deprecated the following ScanImage interfaces: ScanImageMultiFileImagingInterface, ScanImageMultiPlaneImagingInterface, ScanImageMultiPlaneMultiFileImagingInterface, ScanImageSinglePlaneImagingInterface, and ScanImageSinglePlaneMultiFileImagingInterface. These interfaces will be removed in or after October 2025. Use ScanImageImagingInterface for all those cases instead. PR #1330 PR #1331
• Set minimum version requirement for ndx-pose to 0.2.0 PR #1322
• Set minimum version for roiextractors as 0.5.13 PR #1339
• ndx-events is now a required dependency by spikeglx PR #1353

Bug Fixes

• Fix AudioInterface to correctly handle WAV filenames with multiple dots by validating only the last suffix. PR #1327
• Fix a stubbing bug in SpikeGLXNIDQInterface and OpenEphysBinaryAnalogInterface PR #1360

Features

• Add metadata support for DeepLabCutInterface. PR #1319
• AudioInterface Adding support for IEEE float in WAV format PR #1325
• Added a RecordingInterface for WhiteMatter ephys data PR #1297 PR #1333
• Improved ScanImageInteface to support both single and multi-file data PR #1330
• DeepDict now behaves as a python dict when printed in notebooks PR #1351
• Enable chunking for InternalVideoInterface PR #1338
• ImageSeries and TwoPhotonSeries now are chunked by default even if the data is passed as a plain array PR #1338
• Added support for 'I;16' mode in ImageInterface. This mode is mapped to GrayscaleImage in NWB PR #1365

Improvements

• Make metadata optional in NWBConverter.add_to_nwbfile PR #1309
• Add installation instructions on the documentation for neuroconv PR #1344
• Separate dailies and dev-dailies workflows PR #1343
• Added support for renaming Skeletons with DeepLabCutInterface PR #1359
• Updated default PoseEstimationSeries names in DeepLabCutInterface PR #1363
• Testing dependencies include only testing packages (.e.g pytest, pytest-cov) PR #1357
• Testing modalities now run in their separated environment to avoid sequence contamination of dependencies PR #1357

- Python
Published by h-mayorquin 10 months ago

neuroconv - v0.7.3

v0.7.3 (April 25, 2025)

Deprecations and Changes

• Release pydantic ceiling PR #1273
• write_scaled behavior on add_electrical_series_to_nwbfile is deprecated and will be removed in or after October 2025 PR #1292
• add_electrical_series_to_nwbfile now requires both gain and offsets to write scaling factor for voltage conversion when writing to NWB PR #1292
• add_electrical_series_to_nwbfile, add_units_table_to_nwbfile and add_electrodes_to_nwbfile and add_electrode_groups_to_nwbfile are becoming private methods. Use add_recording_to_nwbfile, add_sorting_to_nwbfile and add_recording_metadata_to_nwbfile instead PR #1298
• Set a new minimal dependency for hdmf to 4.0.0, pynwb to 3.0.0 and hdmf-zarr 0.11 PR #1303

Bug Fixes

• Fixed import errors in main modules caused by non-lazy dependencies. Added tests to prevent regressions. PR #1305

Features

• Added a new add_recording_as_time_series_to_nwbfile function to add recording extractors from SpikeInterface as recording extractors to an nwbfile as time series PR #1296
• Added OpenEphysBinaryAnalogInterface for converting OpenEphys analog channels data similar to the SpikeGLX NIDQ interface PR #1237
• Expose iterative write options on BaseImagingExtractorInterface PR #1307

Improvements

• Add documentation for conversion options with NWBConverter PR #1301
• Support roiextractors 0.5.12 PR #1306
• configure_backend is now exposed to be imported as from neuroconv.tools import configure_and_write_nwbfile PR #1287
• Added metadata section to video conversion gallery PR #1276
• DeepLabCutInterface now calculates whether the timestamps come from a constant sampling rate and adds that instead if detected PR #1293
• Fixed a bug in the extractor interfaces where segmentation and sorting interfaces were initialized twice PR #1288
• Support python 3.13 PR #1117
• Added how to documentation on how to set a probe to a recording interfaces PR #1300
• Fix API docs for OpenEphysRecordingInterface PR #1302

- Python
Published by h-mayorquin 11 months ago

neuroconv - v0.7.2

v0.7.2 (April 4, 2025)

Deprecations and Changes

• Split VideoInterface (now deprecated) into ExternalVideoInterface and InternalVideoInterface PR #1251 PR #1256 #1278
• output_filepath deprecated on configure_and_write_nwbfile use nwbfile_path instead PR #1270
• Temporary set a ceiling on pydantic <2.11 PR #1275

Bug Fixes

• Fixed a check in _configure_backend on neurodataobject ndxevents.Events to work only when ndx-events==0.2.0 is used. PR #998
• Added an append_on_disk_nwbfile argumento to run_conversion. This changes the semantics of the overwrite parameter from assuming append mode when a file exists to a more conventional safe writing mode where confirmation is required to overwrite an existing file. Append mode now is controlled with the append_on_disk_nwbfile. #1256

Features

• Added SortedRecordingConverter to convert sorted recordings to NWB with correct metadata mapping between units and electrodes PR #1132
• Support roiextractors 0.5.11 PR #1236
• Added stub_test option to TDTFiberPhotometryInterface PR #1242
• Added ThorImagingInterface for Thor TIFF files with OME metadata PR #1238
• Added always_write_timestamps parameter to ExternalVideoInterface and InternalVideoInterface to force writing timestamps even when they are regular #1279

Improvements

• Filter out warnings for missing timezone information in continuous integration PR #1240
• FilePathType is deprecated, use FilePath from pydantic instead PR #1239
• Change np.NAN to np.nan to support numpy 2.0 PR #1245
• Re activate Plexon tests on Mac. Testing this for a while as they are unreliable tests #1195
• Testing: only run tests for oldest and newest versions of python #1249
• Improve error display on scan image interfaces PR #1246
• Added concurrency to live-service-testing GitHub Actions workflow to prevent simultaneous write to the dandiset. #1252
• Updated GitHub Actions workflows to use Environment Files instead of the deprecated set-output command #1259
• Propagate verbose parameter from Converters to Interfaces #1253
• Replace uses of scipy loadmat and h5storage loadmat with pymatreader read_mat in CellExplorerSortingInterface #1254
• Added camera device support for ExternalVideoInterface and InternalVideoInterface: PR #1282

- Python
Published by h-mayorquin 11 months ago

neuroconv - v0.7.1

v0.7.1 (March 5, 2025)

Deprecations and Changes

Bug Fixes

• Fix parsing of group_names in tools.spikeinterface PR #1234

Features

Improvements

• Testing suite now supports numpy 2.0. PR #1235

- Python
Published by h-mayorquin about 1 year ago

neuroconv - v0.7.0

v0.7.0 (March 3, 2025)

Deprecations and Changes

• Interfaces and converters now have verbose=False by default PR #1153
• Added metadata and conversion_options as arguments to NWBConverter.temporally_align_data_interfaces PR #1162
• Deprecations in the ecephys pipeline: compression options, old iterator options, methods that did not end up in *tonwbfile and the `getschemafrommethod_signature` function PR #1207
• Removed all deprecated functions from the roiextractors module: add_fluorescence_traces, add_background_fluorescence_traces, add_summary_images, add_segmentation, and write_segmentation #1233

Bug Fixes

• run_conversion does not longer trigger append mode when nwbfile_path points to a faulty file PR #1180
• DatasetIOConfiguration now recommends chunk_shape = (len(candidate_dataset),) for datasets with compound dtypes as used by hdmf >= 3.14.6.PR #1146
• OpenEphysBinaryRecordingInterface no longer stores analog data as an ElectricalSeries PR #1179

Features

• Added PlexonLFPInterface for converting Plexon FPl-Low Pass Filtered stream data PR #1209
• Added ImageInterface for writing large collection of images to NWB and automatically map the images to the correct NWB data types PR #1190
• Fixed AudioInterface to properly handle 24-bit WAV files by disabling memory mapping for 24-bit files PR #1226
• Use the latest version of ndx-pose for DeepLabCutInterface and LightningPoseDataInterface PR #1128
• Added a first draft of .clinerules PR #1229
• Support for pynwb 3.0 PR #1231
• Support for hdmf 4.0 PR #1204
• Support for numpy 2.0 PR #1206
• Support Spikeinterface 0.102 PR #1194

Improvements

• Simple writing no longer uses a context manager PR #1180
• Added Returns section to all getter docstrings PR #1185
• ElectricalSeries have better chunking defaults when data is passed as plain array PR #1184
• Ophys interfaces now call get_metadata by default when no metadata is passed PR #1200 and PR #1232

- Python
Published by h-mayorquin about 1 year ago

neuroconv - v0.6.7

v0.6.7 (January 20, 2025)

Deprecations

Bug Fixes

• Temporary set a ceiling for hdmf to avoid a chunking bug PR #1175

Features

• Add description to inter-sample-shift for SpikeGLXRecordingInterface PR #1177

Improvements

• get_json_schema_from_method_signature now throws a more informative error when an untyped parameter is passed #1157
• Improve the naming of ElectrodeGroups in the SpikeGLXRecordingInterface when multi probes are present PR #1177
• Detect mismatch errors between group and group names when writing ElectrodeGroups PR #1165
• Fix metadata bug in IntanRecordingInterface where extra devices were added incorrectly if the recording contained multiple electrode groups or names #1166
• Source validation is no longer performed when initializing interfaces or converters PR #1168

- Python
Published by h-mayorquin about 1 year ago

neuroconv - v0.6.6

v0.6.6 (December 20, 2024)

Deprecations

• Removed use of jsonschema.RefResolver as it will be deprecated from the jsonschema library PR #1133
• Completely removed compression settings from most places PR #1126
• Soft deprecation for file_path as an argument of SpikeGLXNIDQInterface and SpikeGLXRecordingInterface PR #1155
• starting_time in RecordingInterfaces has given a soft deprecation in favor of time alignment methods PR #1158

Bug Fixes

• datetime objects now can be validated as conversion options #1139
• Make NWBMetaDataEncoder public again PR #1142
• Fix a bug where data in DeepLabCutInterface failed to write when ndx-pose was not imported. #1144
• SpikeGLXConverterPipe converter now accepts multi-probe structures with multi-trigger and does not assume a specific folder structure #1150
• SpikeGLXNIDQInterface is no longer written as an ElectricalSeries #1152
• Fix a bug on ecephys interfaces where extra electrode group and devices were written if the property of the "group_name" was set in the recording extractor #1164

Features

• Propagate the unit_electrode_indices argument from the spikeinterface tools to BaseSortingExtractorInterface. This allows users to map units to the electrode table when adding sorting data PR #1124
• Imaging interfaces have a new conversion option always_write_timestamps that can be used to force writing timestamps even if neuroconv's heuristics indicates regular sampling rate PR #1125
• Added .csv support to DeepLabCutInterface PR #1140
• SpikeGLXRecordingInterface now also accepts folder_path making its behavior equivalent to SpikeInterface #1150
• Added the rclone_transfer_batch_job helper function for executing Rclone data transfers in AWS Batch jobs. PR #1085
• Added the deploy_neuroconv_batch_job helper function for deploying NeuroConv AWS Batch jobs. PR #1086
• YAML specification files now accepts an outer keyword upload_to_dandiset="< six-digit ID >" to automatically upload the produced NWB files to the DANDI archive PR #1089 *SpikeGLXNIDQInterface now handdles digital demuxed channels (XD0) #1152

Improvements

• Use mixing tests for ecephy's mocks PR #1136
• Use pytest format for dandi tests to avoid window permission error on teardown PR #1151
• Added many docstrings for public functions PR #1063
• Clean up with warnings and deprecations in the testing framework PR #1158
• Enhance the typing of the signature on the NWBConverter by adding zarr as a literal option on the backend and backend configuration PR #1160

- Python
Published by h-mayorquin about 1 year ago

neuroconv - v0.6.5

v0.6.5 (November 1, 2024)

Deprecations

Bug Fixes

• Fixed formatwise installation from pipy PR #1118
• Fixed dailies PR #1113

Features

• Using in-house GenericDataChunkIterator PR #1068
• Data interfaces now perform source (argument inputs) validation with the json schema PR #1020
• Improve the error message when writing a recording extractor with multiple offsets PR #1111
• Added channels_to_skip to EDFRecordingInterface so the user can skip non-neural channels PR #1110

Improvements

• Remove dev test from PR PR #1092
• Run only the most basic testing while a PR is on draft PR #1082
• Test that zarr backend_configuration works in gin data tests PR #1094
• Consolidated weekly workflows into one workflow and added email notifications PR #1088
• Avoid running link test when the PR is on draft PR #1093
• Centralize gin data preparation in a github action PR #1095

- Python
Published by h-mayorquin over 1 year ago

neuroconv - v0.6.4

v0.6.4 (September 17, 2024)

Bug Fixes

• Fixed a setup bug introduced in v0.6.2 where installation process created a directory instead of a file for test configuration file PR #1070
• The method get_extractor now works for MockImagingInterface PR #1076
• Updated opencv version for security PR #1087
• Solved a bug of PlexonRecordingInterface where data with multiple streams could not be opened PR #989

Deprecations

Features

• Added chunking/compression for string-only compound objects: PR #1042
• Added automated EFS volume creation and mounting to the submit_aws_job helper function. PR #1018
• Added a mock for segmentation extractors interfaces in ophys: MockSegmentationInterface PR #1067
• Added a MockSortingInterface for testing purposes. PR #1065

Improvements

• Testing on mac sillicon PR #1061
• Add writing to zarr test for to the test on data PR #1056
• Modified the CI to avoid running doctests twice PR #1077
• Consolidated daily workflows into one workflow and added email notifications PR #1081
• Added zarr tests for the test on data with checking equivalent backends PR #1083

- Python
Published by h-mayorquin over 1 year ago

neuroconv - v0.6.3

Same as v0.6.2. Testing automatic upload

- Python
Published by h-mayorquin over 1 year ago

neuroconv - v0.6.2

v0.6.2 (September 10, 2024)

Deprecations

• The following classes and objects are now private NWBMetaDataEncoder, NWBMetaDataEncoder, check_if_imaging_fits_into_memory, NoDatesSafeLoader PR #1050

Bug Fixes

• Fixed a bug where IntanRecordingInterface added two devices PR #1059
• Fix a bug in add_sorting_to_nwbfile where unit_electrode_indices was only propagated if waveform_means was passed PR #1057

Features

• Make config_file_path optional in DeepLabCutInterfacePR #1031
• Added get_stream_names to OpenEphysRecordingInterface: PR #1039
• Most data interfaces and converters now use Pydantic to validate their inputs, including existence of file and folder paths. PR #1022
• All remaining data interfaces and converters now use Pydantic to validate their inputs, including existence of file and folder paths. PR #1055

Improvements

• Using ruff to enforce existence of public classes' docstrings PR #1034
• Separated tests that use external data by modality PR #1049
• Added Unit Table descriptions for phy and kilosort: PR #1053
• Using ruff to enforce existence of public functions's docstrings PR #1062
• Improved device metadata of IntanRecordingInterface by adding the type of controller used PR #1059

- Python
Published by h-mayorquin over 1 year ago

neuroconv - v0.6.1

v0.6.1 (August 30, 2024)

Bug fixes

• Fixed the JSON schema inference warning on excluded fields; also improved error message reporting of which method triggered the error. PR #1037

- Python
Published by h-mayorquin over 1 year ago

neuroconv - v0.6.0

v0.6.0 (August 27, 2024)

Deprecations

• Deprecated WaveformExtractor usage. PR #821
• Changed the spikeinterface.tool functions (e.g. add_recording, add_sorting) to have _to_nwbfile as suffix PR #1015
• Deprecated use of compression and compression_options in VideoInterface PR #1005
• get_schema_from_method_signature has been deprecated; please use get_json_schema_from_method_signature instead. PR #1016
• neuroconv.utils.FilePathType and neuroconv.utils.FolderPathType have been deprecated; please use pydantic.FilePath and pydantic.DirectoryPath instead. PR #1017
• Changed the roiextractors.tool function (e.g. add_imaging and add_segmentation) to have the _to_nwbfile suffix [PR #1027]PR #1017

Features

• Added MedPCInterface for operant behavioral output files. PR #883
• Support SortingAnalyzer in the SpikeGLXConverterPipe. PR #821
• Added TDTFiberPhotometryInterface data interface, for converting fiber photometry data from TDT file formats. PR #920
• Add argument to add_electrodes that grants fine control of what to do with the missing values. As a side effect this drops the implicit casting to int when writing int properties to the electrodes table PR #985
• Add Plexon2 support PR #918
• Converter working with multiple VideoInterface instances PR #914
• Added helper function neuroconv.tools.data_transfers.submit_aws_batch_job for basic automated submission of AWS batch jobs. PR #384
• Data interfaces run_conversion method now performs metadata validation before running the conversion. PR #949
• Introduced null_values_for_properties to add_units_table to give user control over null values behavior PR #989

Bug fixes

• Fixed the default naming of multiple electrical series in the SpikeGLXConverterPipe. PR #957
• Write new properties to the electrode table use the global identifier channel_name, group PR #984
• Removed a bug where int64 was casted lossy to float PR #989

Improvements

• The OpenEphysBinaryRecordingInterface now uses lxml for extracting the session start time from the settings.xml file and does not depend on pyopenephys anymore. PR #971
• Swap the majority of package setup and build steps to pyproject.toml instead of setup.py. PR #955
• The DeeplabcutInterface now skips inferring timestamps from movie when timestamps are specified, running faster. PR #967
• Improve metadata writing for SpikeGLX data interface. Added contact ids, shank ids and, remove references to shanks for neuropixels 1.0. Also deprecated the previous neuroconv exclusive property "electrodeshanknumber` PR #986
• Add tqdm with warning to DeepLabCut interface PR #1006
• BaseRecordingInterface now calls default metadata when metadata is not passing mimicking run_conversion behavior. PR #1012
• Added get_json_schema_from_method_signature which constructs Pydantic models automatically from the signature of any function with typical annotation types used throughout NeuroConv. PR #1016
• Replaced all interface annotations with Pydantic types. PR #1017
• Changed typehint collections (e.g. List) to standard collections (e.g. list). PR #1021
• Testing now is only one dataset per test PR #1026

- Python
Published by h-mayorquin over 1 year ago

neuroconv - v0.5.0

v0.5.0 (July 17, 2024)

Deprecations

• The usage of compression_options directly through the neuroconv.tools.audio submodule is now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #939
• The usage of compression and compression_opts directly through the FicTracDataInterface is now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #941
• The usage of compression directly through the neuroconv.tools.neo submodule is now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #943
• The usage of compression_options directly through the neuroconv.tools.ophys submodule is now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #940
• Removed the option of running interface.run_conversion without nwbfile_path argument . PR #951

Features

• Added docker image and tests for an automated Rclone configuration (with file stream passed via an environment variable). PR #902

Bug fixes

• Fixed the conversion option schema of a SpikeGLXConverter when used inside another NWBConverter. PR #922
• Fixed a case of the NeuroScopeSortingExtractor when the optional xml_file_path is not specified. PR #926
• Fixed Can't specify experiment type when converting .abf to .nwb with Neuroconv. PR #609
• Remove assumption that the ports of the Intan acquisition system correspond to electrode groupings in IntanRecordingInterface PR #933
• Add ValueError for empty metadata in make_or_load_nwbfile when an nwbfile needs to be created PR #948

Improvements

• Make annotations from the raw format available on IntanRecordingInterface. PR #934
• Add an option to suppress display the progress bar (tqdm) in VideoContext PR #937
• Automatic compression of data in the LightnignPoseDataInterface has been disabled - users should refer to the new configure_backend method for a general approach for setting compression. PR #942
• Port over dlc2nwb utility functions for ease of maintenance. PR #946

- Python
Published by CodyCBakerPhD over 1 year ago

neuroconv - v0.4.11

v0.4.11 (June 14, 2024)

Bug fixes

• Added a skip condition in get_default_dataset_io_configurations for datasets with any zero-length axis in their full_shape. PR #894
• Added packaging explicitly to minimal requirements. PR #904
• Fixed bug when using make_or_load_nwbfile with overwrite=True on an existing (but corrupt) HDF5 file. PR #911
• Change error trigger with warning trigger when adding both OnePhotonSeries and TwoPhotonSeries to the same file (Issue #906). PR #907

Improvements

• Propagated photon_series_type to BaseImagingExtractorInterface init instead of passing it as an argument of get_metadata() and get_metadata_schema(). PR #847

- Python
Published by CodyCBakerPhD over 1 year ago

neuroconv - v0.4.10

v0.4.10 (June 6, 2024)

Bug fixes

• Fixed bug causing overwrite of NWB GUIDE watermark. PR #890

- Python
Published by CodyCBakerPhD almost 2 years ago

neuroconv - v0.4.9

v0.4.9 (June 5, 2024)

Deprecations

• Removed stream_id as an argument from IntanRecordingInterface. PR #794
• The usage of compression and compression_opts directly through the neuroconv.tools.spikeinterface submodule are now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #805
• Dropped the testing of Python 3.8 on the CI. Dropped support for Python 3.8 in setup. PR #853
• Deprecated skipfeatures argument in `addsorting`. PR #872
• Deprecate old (v1) iterator from the ecephys pipeline. PR #876

Features

• Added backend control to the make_or_load_nwbfile helper method in neuroconv.tools.nwb_helpers. PR #800
• Released the first official Docker images for the package on the GitHub Container Repository (GHCR). PR #383
• Support "one-file-per-signal" and "one-file-per-channel" mode with IntanRecordingInterface. PR #791
• Added get_default_backend_configuration method to all DataInterface classes. Also added HDF5 backend control to all standalone .run_conversion(...) methods for those interfaces. PR #801
• Added get_default_backend_configuration method to all NWBConverter classes. Also added HDF5 backend control to .run_conversion(...). PR #804
• Released the first official Docker images for the package on the GitHub Container Repository (GHCR). PR #383
• Added ScanImageMultiFileImagingInterface for multi-file (buffered) ScanImage format and changed ScanImageImagingInterface to be routing classes for single and multi-plane imaging. PR #809
• Added a function to generate ogen timestamps and data from onset times and parameters to tools.optogenetics. PR #832
• Added configure_and_write_nwbfile and optimized imports in tools.nwb_helpers module. PR #848
• configure_backend may now apply a BackendConfiguration to equivalent in-memory pynwb.NWBFile objects that have different address in RAM. PR #848
• Add support for doubled ragged arrays in add_units_table. PR #879
• Add support for doubled ragged arrays in add_electrodes. PR #881

Bug fixes

• Remove JSON Schema definitions from the properties field. PR #818
• Fixed writing waveforms directly to file. PR #799
• Avoid in-place modification of the metadata in the VideoInterface and on neo tools. PR #814
• Replaced waveform_extractor.is_extension with waveform_extractor.has_extension. PR #799
• Fixed an issue with set_aligned_starting_time for all SortingInterface's that did not have an initial segment start set (and no recording attached). PR #823
• Fixed a bug with parameterized and pytest-xdist==3.6.1 in the ScanImageImagingInterface tests. PR #829
• Added XX and XO to the base metadata schema. PR #833
• BaseImagingExtractor.add_to_nwbfile() is fixed in the case where metadata is not supplied. PR #849
• Prevent SpikeGLXConverterPipe from setting false properties on the sub-SpikeGLXNIDQInterface. PR #860
• Fixed a bug when adding ragged arrays to the electrode and units table. PR #870
• Fixed a bug where write_recording will call an empty nwbfile when passing a path. PR #877
• Fixed a bug that failed to properly include time alignment information in the output NWB file for objects added from any RecordingInterface in combination with stub_test=True. PR #884
• Fixed a bug that prevented passing nwbfile=None and a backend_configuration to NWBConverter.run_conversion. PR #885

Improvements

• Fixed docstrings related to backend configurations for various methods. PR #822
• Propagated photon_series_type to BaseImagingExtractorInterface init instead of passing it as an argument of get_metadata() and get_metadata_schema(). PR #847
• Added automatic backend detection when a backend_configuration is passed to an interface or converter. PR #840
• Improve printing of bytes. PR #831
• Support for pathlib in source data schema validation. PR #854
• Use ZoneInfo instead of dateutil.tz in the conversion gallery. PR #858
• Exposed progress_bar_class to ecephys and ophys data iterators. PR #861
• Unified the signatures between add_units, add_sorting and write_sorting. PR #875
• Improved descriptions of all folder and file paths in the source schema, useful for rendering in the GUIDE. PR #886
• Added watermark via source_script field of NWBFile metadata. source_script_file_name is also required to be specified in this case to avoid invalidation. PR #888

Testing

• Add general test for metadata in-place modification by interfaces. PR #815

- Python
Published by CodyCBakerPhD almost 2 years ago

neuroconv - v0.4.8

v0.4.8 (March 20, 2024)

Bug fixes

• Fixed writing the electrodes field in add_electrical_series when multiple groups are present. PR #784

Improvements

• Upgraded Pydantic support to >v2.0.0. PR #767
• Absorbed the DatasetInfo model into the DatasetIOConfiguration model. PR #767
• Keyword argument field_name of the DatasetIOConfiguration.from_neurodata_object method has been renamed to dataset_name to be more consistent with its usage. This only affects direct initialization of the model; usage via the BackendConfiguration constructor and its associated helper functions in neuroconv.tools.nwb_helpers is unaffected. PR #767
• Manual construction of a DatasetIOConfiguration now requires the field dataset_name, and will be validated to match the final path of location_in_file. Usage via the automated constructors is unchanged. PR #767
• Enhance get_schema_from_method_signature to extract descriptions from the method docval. PR #771

- Python
Published by CodyCBakerPhD almost 2 years ago

neuroconv - v0.4.7

v0.4.7 (February 21, 2024)

Deprecation

• Removed .get_electrode_table_json() on the BaseRecordingExtractorInterface in favor of GUIDE specific interactions. PR #431
• Removed the SIPickleRecordingInterface and SIPickleSortingInterface interfaces. PR #757
• Removed the SpikeGLXLFPInterface interface. PR #757

Bug fixes

• LocalPathExpander matches only folder_paths or file_paths if that is indicated in the passed specification. PR #679 and [PR #675](https://github.com/catalystneuro/neuroconv/pull/679
• Fixed depth consideration in partial chunking pattern for the ROI data buffer. PR #677
• Fix mapping between channel names and the electrode table when writing more than one ElectricalSeries to the NWBFile. This fixes an issue when the converter pipeline of SpikeGLXConverterPipe was writing the electrode table region of the NIDQ stream incorrectly. PR #678
• Fix configure_backend when applied to TimeSeries contents that leverage internal links for data or timestamps. PR #732

Features

• Changed the Suite2pSegmentationInterface to support multiple plane segmentation outputs. The interface now has a plane_name and channel_name arguments to determine which plane output and channel trace add to the NWBFile. PR #601
• Added create_path_template and corresponding tests PR #680
• Added tool function configure_datasets for configuring all datasets of an in-memory NWBFile to be backend specific. PR #571
• Added LightningPoseConverter to add pose estimation data and the original and the optional labeled video added as ImageSeries to NWB. PR #633
• Added gain as a required __init__ argument for TdtRecordingInterface. PR #704
• Extract sessionstarttime from Plexon plx recording file. PR #723

Improvements

• nwbinspector has been removed as a minimal dependency. It becomes an extra (optional) dependency with neuroconv[dandi]. PR #672
• Added a from_nwbfile class method constructor to all BackendConfiguration models. PR #673
• Added compression to FicTracDataInterface. PR #678
• Exposed block_index to all OpenEphys interfaces. PR #695
• Added support for DynamicTable columns in the configure_backend tool function. PR #700
• Refactored ScanImagingInterface to reference ROIExtractors' version of extract_extra_metadata. PR #731
• Added support for Long NHP probe types for the SpikeGLXRecorddingInterfacce. PR #701
• Remove unnecessary duplication of probe setting in SpikeGLXRecordingInterface. PR #696
• Added associated suffixes to all interfaces and converters. PR #734
• Added convenience function get_format_summaries to tools.importing (and exposed at highest level). PR #734

Testing

• RecordingExtractorInterfaceTestMixin now compares either group_name, group or a default value of ElectrodeGroup to the group property in the NWBRecordingExtractor instead of comparing group to group as it was done before PR #736
• TestScanImageImagingInterfaceRecent now checks metadata against new roiextractors implementation PR #741.

- Python
Published by CodyCBakerPhD about 2 years ago

neuroconv - v0.4.6

v0.4.6

Back-compatability break

• Changed the metadata schema for Fluorescence and DfOverF where the traces metadata should be provided as a dict instead of a list of dicts. The name of the plane segmentation is used to determine which traces to add to the Fluorescence and DfOverF containers. PR #632

Features

• Added Pydantic data models of BackendConfiguration for both HDF5 and Zarr datasets (container/mapper of all the DatasetConfigurations for a particular file). PR #568
• Modify the filtering of traces to also filter out traces with empty values. PR #649
• Added tool function get_default_dataset_configurations for identifying and collecting all fields of an in-memory NWBFile that could become datasets on disk; and return instances of the Pydantic dataset models filled with default values for chunking/buffering/compression. PR #569
• Added tool function get_default_backend_configuration for conveniently packaging the results of get_default_dataset_configurations into an easy-to-modify mapping from locations of objects within the file to their correseponding dataset configuration options, as well as linking to a specific backend DataIO. PR #570
• Added set_probe() method to BaseRecordingExtractorInterface. PR #639

Fixes

• Fixed GenericDataChunkIterator (in hdmf.py) in the case where the number of dimensions is 1 and the size in bytes is greater than the threshold of 1 GB. PR #638
• Changed np.floor and np.prod usage to math.floor and math.prod in various files. PR #638
• Updated minimal required version of DANDI CLI; updated run_conversion_from_yaml API function and tests to be compatible with naming changes. PR #664

Improvements

• Change metadata extraction library from fparse to parse. PR #654
• The dandi CLI/API is now an optional dependency; it is still required to use the tool function for automated upload as well as the YAML-based NeuroConv CLI. PR #655

- Python
Published by CodyCBakerPhD over 2 years ago

neuroconv - v0.4.5

v0.4.5

Back-compatibility break

• The CEDRecordingInterface has now been removed; use the Spike2RecordingInterface instead. PR #602

Features

• Added session_start_time extraction to FicTracDataInterface. PR #598
• Added imaging_plane_name keyword argument to add_imaging_plane function to determine which imaging plane to add from the metadata by name instead of imaging_plane_index.
• Added reference for imaging_plane to default plane segmentation metadata. PR #594
• Changed Compass container for Position container in the FicTracDataInterface. PR #606
• Added option to write units in meters by providing a radius in FicTracDataInterface. PR #606
• Added parent_container keyword argument to add_photon_series that defines whether to add the photon series to acquisition or 'ophys' processing module. PR #587
• Added Pydantic data models of DatasetInfo (immutable summary of core dataset values such as maximum shape and dtype) and DatasetConfiguration for both HDF5 and Zarr datasets (the optional layer that specifies chunk/buffering/compression). PR #567
• Added alignment methods to FicTracDataInterface. PR #607
• Added alignment methods support to MockRecordingInterface PR #611
• Added NeuralynxNvtInterface, which can read position tracking NVT files. PR #580
• Adding radius as a conversion factor in FicTracDataInterface. PR #619
• Coerce FicTracDataInterface original timestamps to start from 0. PR #619
• Added configuration metadata to FicTracDataInterface. PR #618
• Expose number of jobs to automatic_dandi_upload. PR #624
• Added plane_segmentation_name keyword argument to determine which plane segmentation to add from the metadata by name instead of plane_segmentation_index. plane_segmentation_name is exposed at BaseSegmentationExtractorInterface.add_to_nwbfile() function to support adding segmentation output from multiple planes. PR #623
• Added SegmentationImages to metadata_schema in BaseSegmentationExtractorInterface to allow for the modification of the name and description of Images container and description of the summary images. PR #622
• Default chunking pattern of RecordingInterfaces now attempts to use as many channels as possible up to 64 total, and fill with as much time as possible up to the chunk_mb. This also required raising the lower HDMF version to 3.11.0 (which introduced 10 MB default chunk sizes). PR #630

Fixes

• Remove starting_time reset to default value (0.0) when adding the rate and updating the photon_series_kwargs or roi_response_series_kwargs, in add_photon_series or add_fluorescence_traces. PR #595
• Changed the date parsing in OpenEphysLegacyRecordingInterface to datetime.strptime with the expected date format explicitly set to "%d-%b-%Y %H%M%S". PR #577
• Pin lower bound HDMF version to 3.10.0. PR #586

Deprecation

• Removed use_times and buffer_size from add_photon_series. PR #600

Testing

• Adds MockImagingInterface as a general testing mechanism for ophys imaging interfaces PR #604.

- Python
Published by CodyCBakerPhD over 2 years ago

neuroconv - v0.4.4

v0.4.4

Features

• DeepLabCutInterface now allows using custom timestamps via set_aligned_timestamps method before running conversion. PR #531

Fixes

• Reorganize timeintervals schema to reside in schemas/ dir to ensure its inclusion in package build. PR #573

- Python
Published by CodyCBakerPhD over 2 years ago

neuroconv - v0.4.3

v0.4.3

Fixes

• The sonpy package for the Spike2 interface no longer attempts installation on M1 Macs. PR #563
• Fixed subset_sorting to explicitly cast end_frame to int to avoid SpikeInterface frame slicing edge case. PR #565

- Python
Published by CodyCBakerPhD over 2 years ago

neuroconv - v0.4.2

v0.4.2

Fixes

• Exposed es_key argument to users where it was previously omitted on MaxOneRecordingInterface, OpenEphysLegacyRecordingInterface, and OpenEphysRecordingInterface. PR #542
• Added deepcopy for metadata in make_nwbfile_from_metadata. PR #545
• Fixed edge case in subset_sorting where end_frame could exceed recording length. PR #551
• Alter add_electrodes behavior, no error is thrown if a property is present in the metadata but not in the recording extractors. This allows the combination of recording objects that have different properties. PR #558

Features

• Added converters for Bruker TIF format to support multiple streams of imaging data. Added BrukerTiffSinglePlaneConverter for single plane imaging data which initializes a BrukerTiffSinglePlaneImagingInterface for each data stream. The available data streams can be checked by BrukerTiffSinglePlaneImagingInterface.get_streams(folder_path) method. Added BrukerTiffMultiPlaneConverter for volumetric imaging data with plane_separation_type argument that defines whether to load the imaging planes as a volume ("contiguous") or separately ("disjoint"). The available data streams for the defined plane_separation_type can be checked by BrukerTiffMultiPlaneImagingInterface.get_streams(folder_path, plane_separation_type) method.
• Added FicTrac data interface. PR #517

Documentation and tutorial enhancements

• Added FicTrac to the conversion gallery and docs API. PR #560

- Python
Published by CodyCBakerPhD over 2 years ago

neuroconv - v0.4.1

v0.4.1

Fixes

• Propagated additional arguments, such as cell_id, from the metadata["Icephys"]["Electrodes"] dictionary used in tools.neo.add_icephys_electrode. PR #538
• Fixed mismatch between expected Electrodes key in tools.neo.add_icephys_electrode and the metadata automatically generated by the AbfInterface. PR #538

- Python
Published by CodyCBakerPhD over 2 years ago

neuroconv - v0.4.0

v0.4.0

Back-compatibility break

• Created separate .add_to_nwbfile method for all DataInterfaces. This is effectively the previous .run_conversion method but limited to operations on an in-memory nwbfile: pynwb.NWBFile` object and does not handle any I/O. PR #455

Fixes

• Set gzip compression by default on spikeinterface based interfaces run_conversion. PR #499

Features

• Added stream control with the stream_name argument to the NeuralynxRecordingExtractor. PR #369

• Added a common .temporally_align_data_interfaces method to the NWBConverter class to use as a specification of the protocol for temporally aligning the data interfaces of the converter. PR #362

• Added CellExplorerRecordingInterface for adding data raw and lfp data from the CellExplorer format. CellExplorer's new format contains a basename.session.mat file containing rich metadata about the session which can be used to extract the recording information such as sampling frequency and type and channel metadata such as groups, location and brain area #488

• CellExplorerSortingInterface now supports extracting sampling frequency from the new data format. CellExplorer's new format contains a basename.session.mat file containing rich metadata including the sorting sampling frequency PR #491 and PR #502

• Added MiniscopeBehaviorInterface for Miniscope behavioral data. The interface uses ndx-miniscope extension to add a Miniscope device with the behavioral camera metadata, and an ImageSeries in external mode that is linked to the device. PR #482

  • CellExplorerSortingInterface now supports adding channel metadata to the nwbfile with write_ecephys_metadata=True as a conversion option PR #494

• Added MiniscopeImagingInterface for Miniscope imaging data stream. The interface uses ndx-miniscope extension to add a Miniscope device with the microscope device metadata, and the imaging data as OnePhotonSeries. PR #468

• Added MiniscopeConverter for combining the conversion of Miniscope imaging and behavioral data streams. PR #498

Improvements

• Avoid redundant timestamp creation in add_eletrical_series for recording objects without time vector. PR #495

• Avoid modifying the passed metadata structure via deep_dict_update in make_nwbfile_from_metadata. PR #476

Testing

• Added gin test for CellExplorerRecordingInterface. CellExplorer's new format contains a basename.session.mat file containing rich metadata about the session which can be used to extract the recording information such as sampling frequency and type and channel metadata such as groups, location and brain area #488.
  • Added gin test for CellExplorerSortingInterface. CellExplorer's new format contains a basename.session.mat file containing rich metadata about the session which can be used to extract the recording information such as sampling frequency and type and channel metadata such as groups, location and brain area PR #494.

- Python
Published by CodyCBakerPhD over 2 years ago

neuroconv - v0.3.0

v0.3.0

Back-compatibility break

• ExtractorInterface classes now access their extractor with the classmethod cls.get_extractor() instead of the attribute self.Extractor. PR #324
• The spikeextractor_backend option was removed for all RecordingExtractorInterface classes. (PR #324, PR #309]
• The NeuroScopeMultiRecordingExtractor has been removed. If your conversion required this, please submit an issue requesting instructions for how to implement it. PR #309
• The SIPickle interfaces have been removed. PR #309
• The previous conversion option es_key has been moved to the __init__ of all BaseRecordingExtractorInterface classes. It is no longer possible to use this argument in the run_conversion method. PR #318
• Change BaseDataInterface.get_conversion_options_schema from classmethod to object method. PR #353
• Removed utils.json_schema.get_schema_for_NWBFile and moved base metadata schema to external json file. Added constraints to Subject metadata to match DANDI. PR #376
• Duplicate video file paths in the VideoInterface and AudioInterface are no longer silently resolved; please explicitly remove duplicates when initializing the interfaces. PR #403
• Duplicate audio file paths in the AudioInterface are no longer silently resolved; please explicitly remove duplicates when initializing the interfaces. PR #402

Features

• The OpenEphysRecordingInterface is now a wrapper for OpenEphysBinaryRecordingInterface. PR #294
• Swapped the backend for CellExplorerSortingInterface from spikeextactors to spikeinterface. PR #267
• In the conversion YAML, DataInterface classes must now be specified as a dictionary instead of a list. PR #311
• In the conversion YAML, conversion_options can be specified on the global level. PR #312
• The OpenEphysRecordingInterface now redirects to legacy or binary interface depending on the file format. It raises NotImplementedError until the interface for legacy format is added. PR #296
• Added the OpenEphysLegacyRecordingInterface to support Open Ephys legacy format (.continuous files). PR #295
• Added PlexonSortingInterface to support plexon spiking data. PR #316
• Changed SpikeGLXRecordingInterface to accept either the AP or LF bands as file paths. Each will automatically set the correseponding es_key and corresponding metadata for each band or probe. PR #298
• The OpenEphysRecordingInterface redirects to OpenEphysLegacyRecordingInterface for legacy format files instead of raising NotImplementedError. PR #349
• Added a SpikeGLXConverter for easy combination of multiple IMEC and NIDQ data streams. PR #292
• Added an interfaces_by_category lookup table to neuroconv.datainterfaces to make searching for interfaces by modality and format easier. PR #352
• neuroconv.utils.jsonschema.get_schema_from_method_signature can now support the Dict[str, str] typehint, which allows DataInterface.__init__ and .run_conversion to handle dictionary arguments. PR #360
• Added neuroconv.tools.testing.data_interface_mixins module, which contains test suites for different types of DataInterfaces PR #357
• Added keywords to DataInterface classes. PR #375
• Uses open-cv-headless instead of open-cv, making the package lighter PR #387.
• Adds MockRecordingInterface as a general testing mechanism for ecephys interfaces PR #395.
• metadata returned by DataInterface.get_metadata() is now a DeepDict object, making it easier to add and adjust metadata. PR #404.
• The OpenEphysLegacyRecordingInterface is now extracts the session_start_time in get_metadata() from Neo (OpenEphysRawIO) and does not depend on pyopenephys anymore. PR #410
• Added expand_paths. PR #377
• Added basic temporal alignment methods to ecephys, ophys, and icephys DataInterfaces. These are get_timestamps, align_starting_time, align_timestamps, and align_by_interpolation. Added tests that serve as a first demonstration of the intended uses in a variety of cases. PR #237 PR #283 PR #400
• Added basic temporal alignment methods to the SLEAPInterface. Added holistic per-interface, per-method unit testing for ecephys and ophys interfaces. PR #401
• Added expand_paths. PR #377, PR #448
• Added .get_electrode_table_json() to the BaseRecordingExtractorInterface as a convenience helper for the GUIDE project. PR #431
• Added BrukerTiffImagingInterface to support Bruker TIF imaging data. This format consists of individual TIFFs (each file contains a single frame) in OME-TIF format (.ome.tif files) and metadata in XML format (.xml file). PR #390
• Added MicroManagerTiffImagingInterface to support Micro-Manager TIF imaging data. This format consists of multipage TIFFs in OME-TIF format (.ome.tif files) and configuration settings in JSON format ('DisplaySettings.json' file). PR #423
• Added a TemporallyAlignedDataInterface definition for convenience when creating a custom interface for pre-aligned data. PR #434
• Added write_as, units_name, units_description to BaseSortingExtractorInterface run_conversion method to be able to modify them in conversion options. PR #438
• Added basic temporal alignment methods to the VideoInterface. These are align_starting_time is split into align_starting_times (list of times, one per video file) and align_global_starting_time (shift all by a scalar amount). align_by_interpolation is not yet implemented for this interface. PR #283
• Added stream control for the OpenEphysBinaryRecordingInterface. PR #445
• Added the BaseTemporalAlignmentInterface to serve as the new base class for all new temporal alignment methods. PR #442
• Added direct imports for all base classes from the outer level; you may now call from neuroconv import BaseDataInterface, BaseTemporalAlignmentInterface, BaseExtractorInterface. PR #442
• Added basic temporal alignment methods to the AudioInterface. align_starting_time is split into align_starting_times (list of times, one per audio file) and align_global_starting_time (shift all by a scalar amount). align_by_interpolation and other timestamp-based approaches is not yet implemented for this interface. PR #402
• Changed the order of recording properties extraction in NeuroscopeRecordingInterface and NeuroScopeLFPInterface to make them consistent with each other PR #466
• The ScanImageImagingInterface has been updated to read metadata from more recent versions of ScanImage PR #457
• Refactored add_two_photon_series() to add_photon_series() and added photon_series_type optional argument which can be either "OnePhotonSeries" or "TwoPhotonSeries". Changed get_default_ophys_metadata() to add Device and ImagingPlane metadata which are both used by imaging and segmentation. Added photon_series_type to get_nwb_imaging_metadata() to fill metadata for OnePhotonSeries or TwoPhotonSeries. PR #462
• Split align_timestamps and align_starting_times into align_segment_timestamps and align_segment_starting_times for API consistency for multi-segment RecordingInterfaces. PR #463
• Rename align_timestamps and align_segmentt_timestamps into set_aligned_timestamps and set_aligned_segment_timestamps to more clearly indicate their usage and behavior. PR #470

Testing

• The tests for automatic_dandi_upload now follow up-to-date DANDI validation rules for file name conventions. PR #310
• Deactivate MaxOneRecordingInterface metadata tests [PR #371]((https://github.com/catalystneuro/neuroconv/pull/371)
• Integrated the DataInterface testing mixin to the SLEAP Interface. PR #401
• Added holistic per-interface, per-method unit testing for ecephys and ophys interfaces. PR #283
• Live service tests now run in a separate non-required GitHub action. [PR #420]((https://github.com/catalystneuro/neuroconv/pull/420)
• Integrated the DataInterfaceMixin class of tests to the VideoInterface. PR #403
• Add generate_path_expander_demo_ibl and associated test for LocalPathExpander PR #456
• Improved testing of all interface alignment methods via the new TemporalAlignmentMixin class. PR #459

Fixes

• BlackrockRecordingInterface now writes all ElectricalSeries to "acquisition" unless changed using the write_as flag in run_conversion. PR #315
• Excluding Python versions 3.8 and 3.9 for the EdfRecordingInterface on M1 macs due to installation problems. PR #319
• Extend type array condition in get_schema_from_hdmf_class for dataset types (excludes that are DataIO). PR #418
• The base_directory argument to all PathExpander classes can now accept string inputs as well as Path inputs. PR #427
• Fixed the temporal alignment methods for the RecordingInterfaces which has multiple segments. PR #411
• Fixes to the temporal alignment methods for the SortingInterface, both single and multi-segment and recordingless. PR #413
• Fixes to the temporal alignment methods for the certain formats of the RecordingInterface. PR #459
• Fixes the naming of LFP interfaces to be ElectricalSeriesLFP instead of ElectricalSeriesLF. PR #467
• Fixed an issue with incorrect modality-specific extra requirements being associated with certain behavioral formats. PR #469

Documentation and tutorial enhancements

• The instructions to build the documentation were moved to ReadTheDocs. PR #323
• Move testing instructions to ReadTheDocs. PR #320
• Moved NeuroConv catalogue from ReadMe.md to ReadTheDocs. PR #322
• Moved instructions to build the documentation from README.md to ReadTheDocs. PR #323
• Add Spike2RecordingInterface to conversion gallery. PR #338
• Remove authors from module docstrings PR #354
• Add examples for LocalPathExpander usage PR #456

Pending deprecation

• Change name from CedRecordingInterface to Spike2RecordingInterface. PR #338

Improvements

• Use Literal in typehints (incompatible with Python<=3.8). PR #340
• BaseDataInterface.get_source_schema modified so it works for .__init__ and .__new__. PR #374

- Python
Published by CodyCBakerPhD almost 3 years ago

neuroconv - v0.2.4

v0.2.4

Deprecation

• All usages of use_times have been removed from spikeinterface tools and interfaces. The function add_electrical_series now determines whether the timestamps of the spikeinterface recording extractor are uniform or not and automatically stores the data according to best practices PR #40

Features

• Added a tool for determining rising and falling frames from TTL signals (parse_rising_frames_from_ttl and get_falling_frames_from_ttl). PR #244
• Added the SpikeGLXNIDQInterface for reading data from .nidq.bin files, as well as the ability to parse event times from specific channels via the get_event_starting_times_from_ttl method. Also included a neuroconv.tools.testing.MockSpikeGLXNIDQInterface for testing purposes. PR #247
• Improved handling of writing multiple probes to the same NWB file PR #255

Pending deprecation

• Added DeprecationWarnings to all spikeextractors backends. PR #265
• Added DeprecationWarnings for spikeextractors objects in neuroconv.tools.spikeinterface. PR #266

Fixes

• Temporarily hotfixed the tensorflow dependency after the release of deeplabcut==2.3.0. PR #268
• Fixed cleanup of waveform tests in SI tools. PR #277
• Added propagation of the load_sync_channel argument for the SpikeGLXNIDQInterface. PR #282
• Fixed the default es_key used by stand-alone write using any RecordingExtractorInterface or LFPExtractorInterface. PR #288
• Fixed the default ExtractorName used to load the spikeinterface extractor of the SpikeGLXLFPInterface. PR #288

Testing

• Re-organized the test_gin_ecephys file by splitting into each sub-modality. PR #282

- Python
Published by CodyCBakerPhD about 3 years ago

neuroconv - v0.2.3

v0.2.3

Documentation and tutorial enhancements

• Remove Path(path_to_save_nwbfile).is_file() from each of the gallery pages. PR #177
• Improve docstring for SpikeGLXRecordingInterface. PR #226
• Correct typing of SpikeGLX in conversion gallery. PR #223
• Added tutorial for utilizing YAML metadata in a conversion pipeline. PR #240
• Added page in User Guide for how to use CSVs to specifiy metadata. PR #241
• Added the BaseDataInterface in the API docs. PR #242
• Fixed type in styling section. PR #253

Features

• Added AudioInterface for files in WAV format using the add_acoustic_waveform_series utility function from tools/audio to write audio data to NWB. PR #196
• Added the MCSRaw for writing data stored in MCSRaw (.raw) format. PR #220
• Added the MEArecRecordingInterface for writing data stored in MEArec (structured .h5) format. PR #218
• Added the AlphaOmegaRecordingInterface for writing data stored in AlphaOmega (folder of .mrx) format. PR #212
• Added the PlexonRecordingInterface for writing data stored in Plexon (.plx) format. PR #206
• Added the BiocamRecordingInterface for writing data stored in Biocam (.bwr) format. PR #210
• Added function to add acoustic series as AcousticWaveformSeries object as acquisition or stimulus to NWB. PR #201
• Added new form to the GitHub repo for requesting support for new formats. PR #207
• Simplified the writing of channel_conversion during add_electrical_series if the vector of gains is uniform; in this case, they are now combined into the scalar conversion value. PR #218
• Implement timestamp extraction from videos for the SLEAPInterface PR #238
• Prevented writing of default values for optional columns on the ElectrodeTable. PR #219
• Add interfaces for Excel and Csv time intervals tables. PR #252

Testing

• Added a session_id to the test file for the automatic_dandi_upload helper function. PR #199
• pre-commit version bump. PR #235

Fixes

• VideoInterface. Only raise a warning if the difference between the rate estimated from timestamps and the fps (frames per seconds) is larger than two decimals. PR #200
• Fixed the bug in a VideoInterface where it would use DataChunkIterator even if the conversion options indicated that it should not. PR #200
• Update usage requirements for HDMF to prevent a buffer overflow isssue fixed in hdmf-dev/hdmf#780. PR #195
• Remove the deprecated distutils.version in favor of packaging.version PR #233

- Python
Published by CodyCBakerPhD about 3 years ago

neuroconv - v0.2.2

v0.2.2

Testing

• Added a set of dev branch gallery tests for PyNWB, HDMF, SI, and NEO. PR #113
• Added tests for the TypeError and ValueError raising for the new starting_frames argument of MovieDataInterface.run_conversion(). PR #113
• Added workflow for automatic detection of CHANGELOG.md updates for PRs. PR #187
• Added support for python 3.10 PR #229

Fixes

• Fixed a new docval typing error that arose in hdmf>3.4.6 versions. PR #113
• Fixed a new input argument issue for starting_frames when using external_file for an ImageSeries in pynwb>2.1.0 versions. PR #113
• Fixed issues regarding interaction between metadata rate values and extractor rate values in tools.roiextractors. PR #159
• Fixed sampling frequency resolution issue when detecting this from timestamps in roiextractors.write_imaging and roiextractors.write_segmentation. PR #159

Documentation and tutorial enhancements

• Added a note in User Guide/DataInterfaces to help installing custom dependencies for users who use Z-shell (zsh). PR #180
• Added MovieInterface example in the conversion gallery. PR #183

Features

• Added ConverterPipe, a class that allows chaining previously intialized interfaces for batch conversion and corresponding tests PR #169
• Added automatic extraction of metadata for NeuralynxRecordingInterface including filtering information for channels, device and recording time information PR #170
• Added stubbing capabilities to timestamp extraction in the MovieInterface avoiding scanning through the whole file when stub_test=True PR #181
• Added a flag include_roi_acceptance to tools.roiextractors.write_segmentation and corresponding interfaces to allow disabling the addition of boolean columns indicating ROI acceptance. PR #193

Pending deprecation

• Replaced the MovieInterface with VideoInterface and introduced deprecation warnings for the former. PR #74

- Python
Published by CodyCBakerPhD over 3 years ago

neuroconv - v0.2.1

v0.2.1

Fixes

• Updated BlackrockRecordingInterface to support multi stream file and added gin corresponding gin tests PR #176. PR #132

- Python
Published by CodyCBakerPhD over 3 years ago

neuroconv - v0.2.0

v0.2.0

Back-compatability break

• All built-in DataInterfaces are now nested under the neuroconv.datainterfaces import structure - they are no longer available from the outer level. To import a data interface, use the syntax from neuroconv.datainterfaces import <name of interface>. PR #74
• The AxonaRecordingExtractorInterface has been renamed to AxonaRecordingInterface. PR #74
• The AxonaUnitRecordingExtractorInterface has been renamed to AxonaUnitRecordingInterface. PR #74
• The BlackrockRecordingExtractorInterface has been renamed to BlackrockRecordingInterface. PR #74
• The BlackrockSortingExtractorInterface has been renamed to BlackrockSortingInterface. PR #74
• The OpenEphysRecordingExtractorInterface has been renamed to OpenEphysRecordingInterface. PR #74
• The OpenEphysSortingExtractorInterface has been renamed to OpenEphysSortingInterface. PR #74
• The KilosortSortingInterface has been renamed to KiloSortSortingInterface to be more consistent with SpikeInterface. PR #107
• The Neuroscope interfaces have been renamed to NeuroScope to be more consistent with SpikeInterface. PR #107
• The tools.roiextractors.add_epoch functionality has been retired in the newest versions of ROIExtractors. PR #112
• Removed deprecation warnings for save_path argument (which is now nwbfile_path everywhere in the package). PR #124
• Changed default device name for the ecephys pipeline. Device_ecephys -> DeviceEcephys PR #154
• Change names of written electrical series on the ecephys pipeline. ElectricalSeriesraw -> ElectricalSeriesRaw, ElectricalSeriesprocessed -> ElectricalSeriesProcessed, ElectricalSeries_lfp -> ElectricalSeriesLFP PR #153

Fixes

• Prevented the CEDRecordingInterface from writing non-ecephys channel data. PR #37
• Fixed description in write_sorting and in add_units_table to have "neuroconv" in the description. PR #104
• Updated spikeinterface version number to 0.95.1 to fix issue with SpikeGLXInterface probe annotations. The issue is described here. PR #132

Improvements

• Unified the run_conversion method of BaseSegmentationExtractorInterface with that of all the other base interfaces. The method write_segmentation now uses the common make_or_load_nwbfile context manager PR #29
• Coerced the recording extractors with spikeextractors_backend=True to BaseRecording objects for Axona, Blackrock, Openephys, and SpikeGadgets. PR #38
• Added function to add PlaneSegmentation objects to an nwbfile in roiextractors and corresponding unit tests. PR #23
• use_times argument to be deprecated on the ecephys pipeline. The function add_electrical_series now determines whether the timestamps of the spikeinterface recording extractor are uniform or not and automatically stores the data according to best practices PR #40
• Add NWBFile metadata key at the level of the base data interface so it can always be inherited to be available. PR #51.
• Added spikeinterface support to Axona LFP and coerece gin tests for LFP to be spikeinterface objects PR #85
• Added function to add fluorescence traces to an nwbfile in roiextractors and corresponding unit tests. The df over f traces are now added to a DfOverF container instead of the Fluorescence container. The metadata schema has been changed for the BaseSegmentationExtractorInterface to allow metadata for DfOverF, and Flurorescence is now not required in the metadata schema. PR #41
• Improved default values of OpticalChannel object names and other descriptions for Imaging data. PR #88
• Extended the ImagingDataChunkIterator to be compatible with volumetric data. PR #90
• Integrated the ImagingDataChunkIterator with the write_imaging methods. PR #90
• Began work towards making SpikeInterface, SpikeExtractors, and ROIExtractors all non-minimal dependencies. PR #74
• Implemented format-wise and modality-wise extra installation requirements. If there are any requirements to use a module or data interface, these are defined in individual requirements files at the corresponding level of the package. These are in turn easily accessible from the commands pip install neuroconv[format_name]. pip install neuroconv[modality_name] will also install all dependencies necessary to make full use of any interfaces from that modality. PR #100
• Added frame stubbing to the BaseSegmentationExtractorInterface. PR #116
• Added mask_type: str and include_roi_centroids: bool to the add_plane_segmentation helper and write_segmentation functions for the tools.roiextractors submodule. PR #117
• Propagate output_struct_name argument to ExtractSegmentationInterface to match its extractor arguments. PR #128
• Added compression and iteration (with options control) to all Fluorescence traces in write_segmentation. PR #120
• For irregular recordings, timestamps can now be saved along with all traces in write_segmentation. PR #130
• Added mask_type argument to tools.roiextractors.add_plane_segmentation function and all upstream calls. This allows users to request writing not just the image_masks (still the default) but also pixels, voxels or None of the above. PR #119
• utils.json_schema.get_schema_from_method_signature now allows Optional[...] annotation typing and subsequent None values during validation as long as it is still only applied to a simple non-conflicting type (no Optional[Union[..., ...]]). PR #119

Documentation and tutorial enhancements:

• Unified the documentation of NeuroConv structure in the User Guide readthedocs. PR #39
• Added package for viewing source code in the neuroconv documentation PR #62
• Added Contributing guide for the Developer section of readthedocs. PR #73
• Added style guide to the readthedocs PR #28
• Added ABF data conversion tutorial @luiztauffer PR #89
• Added Icephys API documentation @luiztauffer PR #103
• Added Blackrock sorting conversion gallery example PR #134
• Extended the User Guide Get metadata section in DataInterfaces with a demonstration for loading metadata from YAML. PR #144
• Fixed a redundancy in PR #144 and API links. PR #154
• Added SLEAP conversion gallery example PR #161

Features

• Added conversion interface for Neuralynx sorting data together with gin data test and a conversion example in the gallery. PR #58
• Added conversion interface for DeepLabCut data together with gin data test and a conversion example in the gallery. PR #24
• Allow writing of offsets to ElectricalSeries objects from SpikeInterface (requires PyNWB>=2.1.0). PR #37
• Added conversion interface for EDF (European Data Format) data together with corresponding unit tests and a conversion example in the gallery. PR #45
• Created ImagingExtractorDataChunkIterator, a data chunk iterator for ImagingExtractor objects. PR #54
• Added support for writing spikeinterface recording extractor with multiple segments and corresponding unit test PR #67
• Added spikeinterface support to the Axona data interface PR #61
• Added new util function get_package for safely attempting to attempt a package import and informatively notifying the user of how to perform the installation otherwise. PR #74
• All built-in DataInterfaces now load their external dependencies on-demand at time of object initialization instead of on package or interface import. PR #74
• Adde spikeinterface support for Blackrock sorting interfacePR #134
• Added conversion interface for TDT recording data together with gin data test. PR #135
• Added conversion interface for SLEAP pose estimation data together with gin test for data. PR #160

Testing

• Added unittests for correctly writing the scaling factors to the nwbfile in the add_electrical_series function of the spikeinterface module. PR #37
• Added unittest for compresion options in the add_electrical_series function of the spikeinterface module. PR #64
• Added unittests for chunking in the add_electrical_series function of the spikeinterface module. PR #84
• Tests are now organized according to modality-wise lazy installations. PR #100

- Python
Published by CodyCBakerPhD over 3 years ago

neuroconv - v0.1.1

v0.1.1

Fixes

• Fixed the behavior of the file_paths usage in the MovieInterface when run via the YAML conversion specification. PR #33

Improvements

• Added function to add ImagingPlane objects to an nwbfile in roiextractors and corresponding unit tests. PR #19

Features

• Add non-iterative writing capabilities to add_electrical_series. PR #32

Testing

• Added unittests for the write_as functionality in the add_electrical_series of the spikeinterface module. PR #32

- Python
Published by CodyCBakerPhD over 3 years ago

neuroconv - v0.1.0

v0.1.0

The first release of NeuroConv.

- Python
Published by CodyCBakerPhD over 3 years ago