EspressoDB
EspressoDB: A scientific database for managing high-performance computing workflows - Published in JOSS (2020)
Science Score: 95.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 1 DOI reference(s) in JOSS metadata -
✓Academic publication links
Links to: joss.theoj.org -
✓Committers with academic emails
2 of 8 committers (25.0%) from academic institutions -
○Institutional organization owner
-
✓JOSS paper metadata
Published in Journal of Open Source Software
Scientific Fields
Repository
Science database interface using Django as the content manager.
Basic Info
- Host: GitHub
- Owner: callat-qcd
- License: bsd-3-clause
- Language: Python
- Default Branch: master
- Homepage: https://espressodb.readthedocs.io
- Size: 23.3 MB
Statistics
- Stars: 8
- Watchers: 6
- Forks: 3
- Open Issues: 7
- Releases: 3
Metadata Files
README.md
EspressoDB
EspressoDB is a Python framework designed to organize (relational) data without losing flexibility. Its objective is to be intuitive and fast.
More specifically, EspressoDB is built on top of the Object-Relational Mapping web framework Django and adds additional convenience functionalities to easily set up your project. Additionally, EspressoDB provides an extended framework of data consistency checks, giving users the freedom to define data tables and their relationships which uniquely mirror the underlying computation.
What does EspressoDB provide?
EspressoDB provides an easy to use database interface which helps you make educated decisions fast.
Once you have created your Python project (e.g., my_project) with EspressoDB
you can use it in all your Python apps to query your data. For example, ```python import numpy as np from my_project.hamiltonian.models import Contact as ContactHamiltonian
Ask the database for specific entries
hamiltonian = ContactHamiltonian.objects.filter(n_sites=20).first()
Use class methods for an intuitive interface
Print a formatted summary of the table entry
print(hamiltonian)
Allocate an actual matrix for the given entry and use it in computations
eigs, vecs = np.linalg.eigh(hamiltonian.matrix) ``
you can generate web views which summarize your tables and data.
Because the web pages use a Python API as well, you can completely customize views with code you have already developed.
E.g., you can automate plots and display summaries in your browser.
If you want to, you can also make your web app public (with different layers of accessibility) and share results with others.
See also the Documentation for more detailed usage instructions.
How to install it?
EspressoDB can be installed via pip:
bash
pip install [--user] espressodb
Run the Tests
Since EspressoDB is about creating projects, the tests are implemented for the example project.
To run the tests, clone this repo, install the dependencies:
bash
pip install .
pip install -r requirements-dev.txt
pip install -r example/my_project/requirements.txt
and run pytest (or the regular test) in example/my_project:
bash
cd example/my_project
pytest [--cov=espressodb]
What's the story behind it?
EspressoDB was developed when we created LatteDB -- a database for organizing Lattice Quantum Chromodynamics research. We intended to create a database for several purposes, e.g. to optimize the scheduling of architecture-dependent many-node jobs and to help in the eventual analysis process. For this reason, we started to abstract our thinking of how to organize physics objects.
It was the goal to have easily shareable and completely reproducible snapshots of our workflow while being flexible and not restricting ourselves too much -- in the end science is full of surprises. The challenges we encountered were: 1. How can we program a table structure which can be easily extended in the future? 2. How do we write a database such that users not familiar with the database concept can start using this tool with minimal effort?
The core module of LatteDB, EspressoDB, is trying to address those challenges.
Who is responsible for it?
Comparison to related technologies
- SQLAlchemy - SQLAlchemy is a minimal low-level Python-SQL interface. While it also provides an ORM (Data Mapper implementation), setting up projects, establishing access to the database and migrating changes are not automated and thus interfacing to an existing project is less beginner-friendly than, e.g., interfacing with an existing Django project.
- Django - Django is the parent of EspressoDB. It uses an active record implementation (direct correspondence between table rows and Python objects). While Django is intended to serve as a web framework, EspressoDB reinterprets and expands Django to be directly used in computational projects. To do so, EspressoDB streamlines the project creation, automates the setup components and provides additional cross-checks needed for multi-user scenarios.
Contributing
Thanks for your interest in contributing! There are many ways to contribute to this project. Get started here.
License
BSD 3-Clause License. See also the LICENSE file.
Owner
- Name: California Lattice Collaboration
- Login: callat-qcd
- Kind: organization
- Location: Not all at California
- Website: https://callat-qcd.github.io
- Repositories: 14
- Profile: https://github.com/callat-qcd
JOSS Publication
EspressoDB: A scientific database for managing high-performance computing workflows
Authors
iTHEMS RIKEN, Wako, Saitama 351-0198, Department of Physics, University of California, Berkeley, California 94720, Nuclear Science Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720
Department of Physics, University of California, Berkeley, California 94720, Nuclear Science Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720
Nuclear Science Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, Department of Physics, University of California, Berkeley, California 94720
Editor
George K. Thiruvathukal
Tags
Django High-performance computing Lattice QCDGitHub Events
Total
Last Year
Committers
Last synced: 5 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| Christopher Koerber | c****r@c****m | 826 |
| Chris | c****i@g****m | 90 |
| Jason Chang | p****0@g****m | 62 |
| walkloud | w****d@g****m | 28 |
| ithems | i****s@i****v | 4 |
| Remi Rampin | r@r****m | 2 |
| Kenneth Lyons | i****s@g****m | 1 |
| Christopher Körber | c****r@b****u | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 4 months ago
All Time
- Total issues: 48
- Total pull requests: 36
- Average time to close issues: 19 days
- Average time to close pull requests: 2 days
- Total issue authors: 4
- Total pull request authors: 4
- Average comments per issue: 1.23
- Average comments per pull request: 0.11
- Merged pull requests: 35
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- ckoerber (40)
- cchang5 (5)
- kenmcelvain (2)
- ixjlyons (1)
Pull Request Authors
- ckoerber (31)
- cchang5 (2)
- remram44 (2)
- ixjlyons (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 1
-
Total downloads:
- pypi 129 last-month
- Total dependent packages: 0
- Total dependent repositories: 3
- Total versions: 6
- Total maintainers: 1
pypi.org: espressodb
Science database interface using Django as the content manager.
- Homepage: https://github.com/callat-qcd/espressodb
- Documentation: https://espressodb.readthedocs.io
- License: BSD License
-
Latest release: 1.2.1
published over 5 years ago
Rankings
Maintainers (1)
Dependencies
- Django *
- bokeh *
- espressodb *
- pandas *
- beautifulsoup4 >=4.8.1 development
- pytest >=4.5.0 development
- pytest-django >=3.6.0 development
- pytest-env >=0.6.2 development
- recommonmark >=0.6.0 development
- sphinx >=2.2.0 development
- sphinx-autodoc-typehints >=1.8.0 development
- sphinx-markdown-tables >=0.0.9 development
- sphinx_rtd_theme >=0.4.3 development
- Django >=2.2.2
- PyYAML >=3.13
- django-bootstrap4 >=0.0.6
- django-extensions >=2.2.1
- django-pandas >=0.6.1
- django-widget-tweaks >=1.4.5
- recommonmark >=0.6.0
- sphinx >=2.2.0
- sphinx-autodoc-typehints >=1.8.0
- sphinx-markdown-tables >=0.0.9
- sphinx_rtd_theme >=0.4.3
- Django >=2.2.2
- PyYAML >=3.13
- django-bootstrap4 >=0.0.6
- django-extensions >=2.2.1
- django-pandas >=0.6.1
- django-widget-tweaks >=1.4.5
- Django * test
- django-bootstrap4 * test
- django-widget-tweaks * test
- espressodb * test
- Django * test
- django-bootstrap4 * test
- django-widget-tweaks * test
- espressodb * test