Cologne-Hackathon-2023

This is the repository for documentation during the first NFDI4Bioimage - TA3-Hackathon - UoC-2023 (in short: Cologne Hackathon).

TOC - In this README:
- Cologne-Hackathon-2023 - Context of proposed topics - Brief report about the event - Setting the Stage - Potential topics - Spot on the Topics - Let the show(s) begin - Workflow and roadmap session(s) - Topic selection on-site - Hacking - Documentation - Use Cases - Shine! - 42? - Closing the Hackathon - Overview of the content in this repository - Contributions - Contributors in alphabetical order - Zenodo submission - Zenodo sandbox test submission - Organizers - Acknowledgement - Funding - for this hackathon - NFDI4Bioimage - Other Organisations - CEPLAS - CECAD - CRC TRR341

Context of proposed topics

Open Microscopy Environment Remote Objects (OMERO) is a data management platform used in experimental biology and is in particular specialized for the needs of imaging data.

Imaging data can be very complex. An image acquisition usually comprises multiple files of different types covering the data, and respective metadata in one directory or condensed to one object in a vendor-specific file format. If not managed accordingly in an Research Data Management (RDM) compliant way, this can cause all kinds of problems for downstream analysis, linking it with data across disciplines and preventing tools from being interoperable. OME-Zarr is one such format which tries to avoid such obstacles and minimize data loss upon conversion.

OMERO supports with handling this specific type of data. It extracts, maps and stores metadata in a relational database and enriches it with proprietary-file-format-specific metadata if required while converting the proprietary file formats using Bio-Formats.

Moreover, using OMERO, researchers can visualize, analyze and process their imaging data. One example is a volume viewer (VolViewer). However, other solutions for more specific 3D volume visualization accessible from OMERO are desired. Two options for this are 3D-Slicer and neuroglancer. While 3D-Slicer will be covered elsewhere, this hackathon explores neuroglancer as a possible extension to the current functionality accessible via OMERO.

On the one hand, there is OMERO which is central to data management of different types of microscopic data as e.g. outlined above. On the other hand, ARCs (Annotated Research Contexts), offer - as indicated by its name - an annotated research context for multi-modal research projects conducted using methods across disciplines. However, imaging data and its metadata has not been covered well, so far.
Here, we intend to bridge between OMERO and ARC and make use of the interoperability of both to enrich metadata and the research context on both sides.

ARCs can open the door to the world of FAIR digital objects (FDOs). Essentially, an ARC has according to its specification a defined directory structure, adheres to several standards, is based on the ISA model, can be versioned and enables workflows e.g. using Common Workflow Language (CWL) and RO-Crate conversion. ARCs and respective supportive tools are developed by DataPLANT.

The Brain Imagin Data structure (BIDS) provides some of the aspects mentioned above in order to handle complex and multi-modal neuroimaging and behavioural data. Key components are a formalized file and directory structure, use of controlled vocabulary, required metadata and BIDS Extension Proposals (BEPs).

In order to properly manage and use imaging data, the availability of appropriate metadata is of utmost importance.

While some metadata can be recorded by devices and automatically be extracted from the respective files or acquisition-specific file collections, other metadata need to be provided by the researcher to enable setting the imaging data in the appropriate research context. In particular, a well-defined set of metadata should be deposited when imaging data is submitted to a repository in order to preserve and to share it in a way following the principles of FAIR (Findability, Accessibility, Interoperability, and Reusability). REMBI (Recommended Metadata for Biological Images) is providing guidance on the content of such a well-defined set of metadata. Therefore, it is intended to explore REMBI with examples in the context of OMERO and ARC during this hackathon and to identify requirements obstacles for mapping metadata when transferring data between OMERO and ARC.

The NFDI4Bioimage Task Area 3 (TA3) works on multimodal data linking and integration. It also prepares with this hackathon an upcoming project on OMERO-ARC interoperability at the 2nd de.NBI BioHackathon. (For more details on all contributors, please see the end of this file.)

Brief report about the event

In the following, a brief report about the event following the schedule is provided.

For in depth content, please see the individual talks, topics and follow-up work on OMERO-ARC interoperability (e.g. here) and subsequent publications. In the end of this README, the contributors, respective links, organizers and funding statements can be founds. In between, you can find an overview of the repository content (file tree).

Prior to the hackathon, some supportive installation informations including links targeted to the participants were shared for OMERO (here only for Linux) and ARC-related installations.

Setting the Stage

The first NFDI4Bioimage - TA3-Hackathon at CECAD at the University of Cologne was opened by Astrid Schauss, moderated by Andrea Schrader and the TA3 opening presentation was provided by Torsten Stter.

Potential topics

While preparing this Hackathon, we identified these potential topics for the Cologne Hackathon from a preparatory meeting with several participants:

• Interoperability OMERO-ARC (OMERO-BIDS, ARC-BIDS)
• REMBI / Mapping
• OMERO / zarr
• 3D Slicer, Neuroglancer

Spot on the Topics

These and related topics were covered with introductory talks:

• OMERO and zarr (Joshua A. Moore)
• Annotated Research Context (ARC) (H. Lukas Weil)
• Brain Imaging Data Structure (BIDS) (Julia Thnnien)
• 3D Slicer, Neuroglancer (Torsten Stter)
• REMBI (Vanessa A. F. Fuchs, Tom Boissonnet)

Let the show(s) begin

Workflow and roadmap session(s)

In a workflow and roadmap session, graphical workflows were created, tasks were defined and the topics were selected.

Beforehand, a task list was created used as an inspiration and optionally to be adjusted for the different topics at the Hackathon (see markdown version in the 'Topics' directory).

Topic selection on-site

Following the initial intense discussions and task identification, some adjustments to the potential topics were made and the final topics on-site were selected:

The interoperability topic focussed on OMERO-ARC Interoperability.

3D Slicer and Neuroglancer are both covered in NFDI4Bioimage TA3 but only the Neuroglancer topic was eventually covered in this hackathon.

The OMERO / zarr topic was represented as a topic in the Neuroglancer topic but not followed as an individual topic at this hackathon.

The REMBI topic focusses on OMERO and ARC implementation concepts and on creating an example on-site.

Hacking - Documentation

Eventually, intensive work on the topics 'Interoperability', 'REMBI / Mapping', '3D Slicer / Neuroglancer (+OMEROzarr)' was conducted in groups.

From the beginning, there was a focus on documentation and including use cases. The interoperability and REMBI topic served as preparatory work for a project at the 2nd de.NBI BioHackathon in Bielefeld: 'Towards OMERO and ARC interoperability for RDM-compliant bio-image data'.

Use Cases

A file for collecting Use Cases was created and filled.

Shine!

The final reports summarized the outcome of the three groups.
(Those presenting the work of the respective groups are provided in brackets.)

• Interoperability (H. Lukas Weil)
• REMBI / Mapping (Andrea Schrader, Tom Boissonnet)
• 3D Slicer / Neuroglancer (+OMEROzarr) (Torsten Stter)

42?

We concluded with a discussion of all participants on where we found already answers and what needs to be solved in the future.

Closing the Hackathon

After closing the Hackathon from the organizational side (Andrea Schrader), Torsten Stter summarized its content and Astrid Schauss, who hosted the hackathon at CECAD with the Cologne team, closed this hackathon that also bridged from NFDI4Bioimage to DataPLANT on the topic of OMERO-ARC interoperability.

The interoperability and REMBI topic served as preparatory work for a project at the 2nd de.NBI BioHackathon Germany in Bielefeld: 'Towards OMERO and ARC interoperability for RDM-compliant bio-image data'. The respective repository for this hackathon project is growing here.

---

---

Overview of the content in this repository

. CITATION.cff CONTRIBUTING.md Installation ARC README.md OMERO README.md Introduction_Talks 2023-11-29_TA3-Hackathon_REMBI.pdf BIDS.pdf BioimagingHackathon_2023_ARC.pdf BioimagingHackathon_2023_ARC.pptx README.md intro-3d-slicer-neuroglancer.pdf intro-OMERO-zarr.pdf intro-OMERO-zarr.pptx opening-ta3-cologne-hackathon.pdf LICENSE README.md Topics 3DSlicer_Neuroglancer README.md Report FinalReport_3DSlicer_Neuroglancer.pdf images 3Dslicer_Neuroglancer.jpeg Screenshot 2023-12-01 115118.png Screenshot at 2023-11-30 17-24-16.png Screenshot_2023-11-30_17-15-43.png Cologne_Hackathon_Tasks.md Interoperability OMERO-ARC-Exporter-WorkFlow-Introduction.pdf Omero-ARC-topic-'use-cases'.pdf README.md Report FinalReport_Interoperability.pdf images Interoperability.jpeg Interoperability_progress.jpg omero_arc_connection.md omero_arc_exporter.md omero_arc_mapping.md OMERO-zarr README.md images OMERO_zarr.jpeg REMBI-Mapping README.md REMBI-template_CAi_VF.xlsx REMBI-template_CAi_VF_AS.xlsx Report FinalReport_REMBI_Mapping.pdf images REMBI_Mapping.jpeg Use_Cases.md shared_pdfs Cologne_Hackathon_Schedule.pdf Cologne_Hackathon_Tasks.pdf 16-01-2024 (created using tree v2.1.1 1996 - 2023 by Steve Baker, Thomas Moore, Francesc Rocher, Florian Sesser, Kyosuke Tokoro)

---

---

Contributions

Optimal contribution during the Hackathon was suggested here.

Thank you for any contribution to the NFDI4Bioimage - TA3-Hackathon - UoC-2023 repository!!!

When contributing, participants were asked to consider that (parts of) this repository might be used by people with various background and intentions including teaching in the future. Therefore, it is appreciated if a brief non-specialist targeted introduction / summary was provided preceding the respective content or is added after the hackathon to complete these sections.

Contributors in alphabetical order

Participants contributing to the NFDI4Bioimage - TA3-Hackathon - UoC-2023 could provide their name, GitHub handle, ORCID iD and ROR right here in the README - if they liked.

|name|GitHub|ORCID|ROR|
|---|---|---|---|
|Eik Dahms| @eik-dahms|https://orcid.org/0000-0002-7452-1146|https://ror.org/00rcxh774| |Niraj Kandpal|@Nirkan|https://orcid.org/0009-0007-5101-4786|https://ror.org/00rcxh774
https://ror.org/04c4bwh63| |Christoph Mhl|@cmohl2013|https://orcid.org/0000-0002-0829-5101|| |Josh Moore|@joshmoore|https://orcid.org/0000-0003-4028-811X|https://ror.org/05tpnw772| |Astrid Schauss|@AstridSchauss|https://orcid.org/0000-0002-6658-2192|https://ror.org/00rcxh774
https://ror.org/04c4bwh63 | |Andrea Schrader|@andreaschrader|https://orcid.org/0000-0002-3879-7057|https://ror.org/034waa237

https://ror.org/00rcxh774| |Torsten Stter|@tstoeter||https://ror.org/01zwmgk08| |Julia Thnnien|@JThoennissen|https://orcid.org/0000-0002-5467-871X|| |Mnica E. Valencia-Schneider|@mvals|https://orcid.org/0000-0003-3430-2683|https://ror.org/00rcxh774| |Heinrich Lukas Weil|@HLWeil|https://orcid.org/0000-0003-1945-6342|https://ror.org/01qrts582| |Peter Zentis|@pzentis|https://orcid.org/0000-0002-6999-132X|https://ror.org/00rcxh774
https://ror.org/04c4bwh63 |

Zenodo submission

The full list of participants - some only contributing to conceptual work on site and not directly to this repo - is provided in the CITATION.cff and .zenodo.json files.

CITATION.cff and .zenodo.json: - CITATION.cff is used as a citation being directly in the repository and being accessible via the GitHub sidebar. - Citation.cff was tested with cffconvert prior to adding the references to be valid.
- The repo is prepared to be submitted as described here or here. .zenodo.json is used for transferring the contained metadata to Zenodo.

Zenodo sandbox test submission

Organizers

Eik Dahms, Niraj Kandpal, Astrid Schauss, Andrea Schrader, Torsten Stter, Mnica E. Valencia-Schneider, Peter Zentis (alphabetical order)

Acknowledgement

We would like to acknowledge the support from the NFDI4Bioimage, in particular TA3, TA1 and TA5 and in particular all participants to make this hackathon happen.

For the OMERO-ARC topic, we are thankful for the collaboration with DataPLANT. We acknowledge Timo Mhlhaus in discussions during planning for the OMERO-ARC project and Kevin Schneider in preparatory meetings on the OMERO-ARC project (both: DataPLANT).

We thank Heinrich Lukas Weil (DataPLANT) for participating and contributing as a guest, also providing the introductory talk on ARC.

Moreover, we thank the RRZK (Daniel Wickeroth) for presenting and explaining the CAVE to the hackathon participants!

Funding - for this hackathon

(funding for the event and enabling for organization and participation)

NFDI4Bioimage

Funded by the German Research Foundation (DFG) within the framework of the NFDI project numbers: 501864659.

Other Organisations

CEPLAS

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) within Germanys Excellence Strategy EXC-2048/1 project number 390686111.

CECAD

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy - EXC 2030 - 390661388.

CRC TRR341

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) Project-ID 456082119.