datalab-server
datalab is a place to store experimental data and the connections between them.
Science Score: 57.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 3 DOI reference(s) in README -
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.2%) to scientific vocabulary
Keywords
Repository
datalab is a place to store experimental data and the connections between them.
Basic Info
- Host: GitHub
- Owner: datalab-org
- License: mit
- Language: Python
- Default Branch: main
- Homepage: https://docs.datalab-org.io
- Size: 31.8 MB
Statistics
- Stars: 67
- Watchers: 4
- Forks: 22
- Open Issues: 266
- Releases: 45
Topics
Metadata Files
README.md
datalab
datalab is a user-friendly, open-source platform that can capture all the experimental data and metadata produced in a scientific lab, targeted (broadly) at materials chemistry but with customisability and extensability in mind. datalab records data and metadata securely and makes it accessible and reusable by both humans and machines via the web UI and API, respectively. datalab can be self-hosted and managed deployments are also available.
You can try the demo deployment at demo.datalab-org.io and read the online documentation at docs.datalab-org.io with release notes and changelog available on GitHub and online.
Features:
- Capture and store sample and device metadata
- Connect and sync raw data directly and from laboratory instruments
- Built-in support for multiple characterisation techniques (XRD, NMR, echem, TEM, TGA, Mass Spec, Raman and more).
- Capture scientific context: store the graph of relationships between research objects.
- Python API for programmatic access to your lab's data enabling custom analysis and automation.
- Join the datalab federation: you can add your datalab to the federation for additional shared features.
- Plugin ecosystem allowing for custom data blocks, AI integration and other instance-specific code.
- Deployment and infrastructure automation via Ansible playbooks.
[!NOTE] You may be looking for the identically named project DataLab for signal processing, which also has plugins, clients and other similar concepts!
Getting started
To set up your own datalab instance or to get started with development, you can follow the installation and deployment instructions in the online documentation.
We can also provide paid managed deployments via datalab industries ltd.: contact us at hello@datalab.industries.
Design philosophy and architecture
The datalab architecture is shown below:
annotations, connections"] class UmbrellaDesc umbrellaLabel; subgraph ClientInterfaces [Client interfaces] direction TB BrowserApp[_datalab_
Browser app] PythonAPI[_datalab_
Python API] end class BrowserApp,PythonAPI clientInterface; class ClientInterfaces subgraphStyle; subgraph Backend direction TB RESTAPI[_datalab_
REST API] MongoDB[MongoDB Database] DataLake[Data Lake] end class RESTAPI,MongoDB,DataLake coreComponent; class Backend subgraphStyle; User <-- "User data I/O" --> UmbrellaDesc; Machine <-- "Machine data I/O" --> UmbrellaDesc; UmbrellaDesc <-- "_via_ GUI" --> BrowserApp; UmbrellaDesc <-- "_via_ scripts" --> PythonAPI; BrowserApp <-- "HTTP (Data exchange)" --> RESTAPI; PythonAPI <-- "API calls (Data exchange)" --> RESTAPI; RESTAPI <-- "Annotations, connections" --> MongoDB; RESTAPI <-- "Raw and structured characterisation data" --> DataLake; linkStyle 0 stroke:#666,stroke-width:3px linkStyle 1 stroke:#666,stroke-width:3px linkStyle 2 stroke:#666,stroke-width:3px linkStyle 3 stroke:#666,stroke-width:3px linkStyle 4 stroke:#666,stroke-width:3px linkStyle 5 stroke:#666,stroke-width:3px linkStyle 6 stroke:#666,stroke-width:3px linkStyle 7 stroke:#666,stroke-width:3px click PythonAPI "https://github.com/datalab-org/datalab-api" "datalab Python API on GitHub" _blank click BrowserApp "https://github.com/datalab-org/datalab/tree/main/webapp" "datalab Browser App on GitHub" _blank click RESTAPI "https://github.com/datalab-org/datalab/tree/main/pydatalab" "pydatalab REST API on GitHub" _blank ```
The main aim of datalab is to provide a platform for capturing the significant amounts of long-tail experimental data and metadata produced in a typical lab, and enable storage, filtering and future data re-use by humans and machines. datalab is targeted (broadly) at materials chemistry labs but with customisability and extensability in mind.
The platform provides researchers with a way to record sample- and cell-specific metadata, attach and sync raw data from instruments, and perform analysis and visualisation of many characterisation techniques in the browser (XRD, NMR, electrochemical cycling, TEM, TGA, Mass Spec, Raman).
Importantly, datalab stores a network of interconnected research objects in the lab, such that individual pieces of data are stored with the context needed to make them scientifically useful.
License
This software is released under the conditions of the MIT license. Please see LICENSE for the full text of the license.
Contact
We are available for consultations on setting up and managing datalab deployments, as well as collaborating on or sponsoring additions of new features and techniques. Please contact Josh or Matthew on their academic emails, or join the public datalab Slack workspace.
Contributions
This software was conceived and developed by:
- Prof Joshua Bocarsly (Department of Chemistry, University of Houston, previously Department of Chemistry, University of Cambridge)
- Dr Matthew Evans (MODL-IMCN, UCLouvain & Matgenix)
with support from the group of Professor Clare Grey (University of Cambridge), and major contributions from:
plus many contributions, feedback and testing performed by other members of the community, in particular, the groups of Prof Matt Cliffe (University of Nottingham) and Dr Peter Kraus (TUBerlin).
A full list of code contributions can be found on GitHub.
Funding
Contributions to datalab have been supported by a mixture of academic funding and consultancy work through datalab industries ltd.
In particular, the developers thank:
- Initial proof-of-concept funding from the European Union's Horizon 2020 research and innovation programme under grant agreement 957189 (DOI: 10.3030/957189), the Battery Interface Genome - Materials Acceleration Platform (BIG-MAP), as an external stakeholder project.
- The BEWARE fellowship scheme of the Wallonie recherche SPW for funding under the European Commission's Marie Curie-Skłodowska Action (COFUND 847587).
Owner
- Name: datalab-org
- Login: datalab-org
- Kind: organization
- Repositories: 1
- Profile: https://github.com/datalab-org
Citation (CITATION.cff)
# YAML 1.2
# Metadata for citation of this software according to the CFF format (https://citation-file-format.github.io/)
cff-version: 1.2.0
message: If you use this software in a publication, please cite the source code archive on Zenodo.
title: datalab
abstract: datalab is a place to store experimental data and the connections between them.
repository-code: https://github.com/datalab-org/datalab
url: https://github.com/datalab-org/datalab
type: software
license: MIT
identifiers:
- type: doi
value: 10.5281/zenodo.14719467
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 184
- Total pull requests: 296
- Average time to close issues: 4 months
- Average time to close pull requests: 12 days
- Total issue authors: 11
- Total pull request authors: 11
- Average comments per issue: 0.44
- Average comments per pull request: 1.45
- Merged pull requests: 185
- Bot issues: 1
- Bot pull requests: 43
Past Year
- Issues: 151
- Pull requests: 244
- Average time to close issues: 13 days
- Average time to close pull requests: 9 days
- Issue authors: 11
- Pull request authors: 9
- Average comments per issue: 0.3
- Average comments per pull request: 1.47
- Merged pull requests: 149
- Bot issues: 1
- Bot pull requests: 37
Top Authors
Issue Authors
- ml-evs (144)
- MJCliffe (12)
- BenjaminCharmes (12)
- jdbocarsly (4)
- KubaLemanski (3)
- yue-here (2)
- KusalvinD (2)
- PeterKraus (2)
- isabelcooley (1)
- SteffenBrinckmann (1)
- dependabot[bot] (1)
Pull Request Authors
- ml-evs (180)
- BenjaminCharmes (50)
- dependabot[bot] (32)
- be-smith (10)
- github-actions[bot] (7)
- jdbocarsly (7)
- pre-commit-ci[bot] (4)
- yue-here (3)
- elbee99 (1)
- MJCliffe (1)
- KusalvinD (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 1
-
Total downloads:
- pypi 596 last-month
- Total dependent packages: 0
- Total dependent repositories: 0
- Total versions: 10
- Total maintainers: 1
pypi.org: datalab-server
datalab is a research data management platform for materials science and chemistry.
- Documentation: https://datalab-server.readthedocs.io/
- License: mit
-
Latest release: 0.6.3
published 6 months ago
Rankings
Maintainers (1)
Dependencies
- actions/checkout v3 composite
- actions/setup-node v3 composite
- actions/setup-python v4 composite
- actions/upload-artifact v3 composite
- codecov/codecov-action v3 composite
- mongo 3 docker
- @testing-library/cypress ^8.0.0 development
- @testing-library/jest-dom ^5.14.1 development
- @testing-library/vue ^6.4.2 development
- @vue/cli-plugin-babel ~4.5.0 development
- @vue/cli-plugin-e2e-cypress 5.0.8 development
- @vue/cli-plugin-eslint ~4.5.0 development
- @vue/cli-plugin-router ~4.5.0 development
- @vue/cli-plugin-unit-jest ~4.5.0 development
- @vue/cli-plugin-vuex ~4.5.0 development
- @vue/cli-service ~4.5.0 development
- @vue/compiler-sfc ^3.2.4 development
- @vue/eslint-config-prettier ^6.0.0 development
- @vue/test-utils ^2.0.0-0 development
- babel-core 7.0.0-bridge.0 development
- babel-eslint ^10.1.0 development
- cypress 12.3.0 development
- eslint ^6.7.2 development
- eslint-plugin-prettier ^3.3.1 development
- eslint-plugin-vue ^7.0.0 development
- jest-fetch-mock ^3.0.3 development
- prettier ^2.2.1 development
- typescript ~3.9.3 development
- vue-jest ^5.0.0-0 development
- web-worker ^1.2.0 development
- @fortawesome/fontawesome-svg-core ^1.2.26-2
- @fortawesome/free-brands-svg-icons ^5.15.4
- @fortawesome/free-regular-svg-icons ^5.15.2
- @fortawesome/free-solid-svg-icons ^5.12.0-2
- @fortawesome/vue-fontawesome ^3.0.0-3
- @tinymce/tinymce-vue ^4.0.0
- @uppy/core ^1.16.0
- @uppy/dashboard ^1.16.0
- @uppy/webcam ^1.8.4
- @uppy/xhr-upload ^1.6.10
- @vueuse/components ^9.12.0
- @vueuse/core ^9.12.0
- bootstrap ^4.5.3
- core-js ^3.6.5
- cytoscape ^3.23.0
- cytoscape-cola ^2.5.1
- cytoscape-dagre ^2.4.0
- cytoscape-elk ^2.1.0
- d3 ^7.6.1
- tinymce ^5.6.2
- vue ^3.2.4
- vue-router ^4.0.0-0
- vue-select ^4.0.0-beta.6
- vuex ^4.0.0-0
- webcola ^3.4.0
- 1640 dependencies
- 106 dependencies
- CasperWA/push-protected v2 composite
- actions/checkout v3 composite
- actions/setup-python v4 composite