jobman
A command line job manager with flexible support for retries, timeouts, logging, notifications, and more.
Science Score: 31.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (16.1%) to scientific vocabulary
Keywords
Repository
A command line job manager with flexible support for retries, timeouts, logging, notifications, and more.
Basic Info
Statistics
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 1
Topics
Metadata Files
README.md
Jobman automates the process of running and monitoring jobs on the command line. Jobman supports * running commands in the background immune to hangups * logging command output * retrying commands * aborting commands after timeout * delaying command execution for a specified time or event * sending notifications on command success or failure
Documentation
Visit the :book: jobman documentation site :book: for complete information on using jobman.
Requirements
Jobman runs on UNIX-like operating systems including Linux and MacOS. Windows is not supported.
Jobman requires Python3.9+.
Installation
Install or upgrade from PyPI with
$ pip install -U jobman
Example
The example below uses jobman to run a Python script train.py in the background and immune to hangups (e.g., a SIGHUP from an SSH timeout).
Jobman will ensure 60 seconds have passed and that the file data.csv exists before starting the program. If those conditions haven't been met by 5:00PM on March 5, 2032, jobman will abort the job.
Jobman will retry the program up to five times until there's one successful run, defined as an exit code of 0 or 42, waiting ten seconds between retries.
If the job succeeds, jobman will send a notification email. If the job fails, jobman will send an SMS message.
bash
$ jobman \
--wait-duration 60s --wait-for-file data.csv \
--abort-time "2032-03-05T17:00:00" \
--retry-attempts 5 --retry-delay 10s -c 0 -c 42 \
--notify-on-job-success my-email --notify-on-job-failure my-cell \
train.py
12e4b604
After submitting the train.py job above, use jobman show to display details on job progress:
bash
jobman show 12e4b604
To view a running log of the consolidated stdout and stderr streams of the latest run of the job, use jobman logs:
bash
jobman logs 12e4b604 --follow
Alternatives
Jobman aims to be reliable and fully-featured. It operates without requiring a system service/daemon for orchestration.
Alternative tools for similar use cases include: * cron: for scheduling repeated executions of a job * Airflow, Prefect, and Dagster: for managing dependencies between multiple jobs * supervisord: for daemon-based job management
Developing
Jobman uses pyenv for Python version management and poetry for builds. Before working on Jobman, ensure you have pyenv and poetry installed.
The Makefile defines targets for common operations during development, including the following:
* make setup: set up and install the package
* make fmt: run the autoformatters
* make test: run the type tests and unit test suite
* make build: build the jobman wheel
To release a new version of the package, use the bumpver.sh script. For example, to update to version 1.2.3, run ./bumpver.sh 1.2.3. Before committing, be sure to update the CHANGELOG.md with user-facing changes described in the recent commit log printed out by bumpver.sh.
Contributing
Feature requests, bug reports, and pull requests are welcome! See CONTRIBUTING.md for details on how to contribute to jobman.
Owner
- Name: Ryan Wallace
- Login: ryancswallace
- Kind: user
- Location: Boston, MA
- Repositories: 17
- Profile: https://github.com/ryancswallace
Data Science @ FRBB | CS @ Harvard
Citation (CITATION.ccf)
cff-version: 1.2.0 message: "If you use this software, please cite it as below." authors: - family-names: "Wallace" given-names: "Ryan" title: "jobman" date-released: 2022-08-29 url: "https://github.com/ryancswallace/jobman"
GitHub Events
Total
Last Year
Committers
Last synced: 7 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| Ryan C Wallace | R****e@b****g | 105 |
| Ryan Wallace | r****w@R****l | 3 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 5 months ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 2
-
Total downloads:
- pypi 15 last-month
-
Total dependent packages: 0
(may contain duplicates) -
Total dependent repositories: 0
(may contain duplicates) - Total versions: 23
- Total maintainers: 1
proxy.golang.org: github.com/ryancswallace/jobman
Copyright © 2021-2022 Ryan Wallace Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- Homepage: https://github.com/ryancswallace/jobman
- Documentation: https://pkg.go.dev/github.com/ryancswallace/jobman#section-documentation
- License: MIT
-
Latest release: v0.4.0
published over 2 years ago
Rankings
pypi.org: jobman
A command line job manager with flexible support for retries, timeouts, logging, notifications, and more
- Homepage: https://ryancswallace.github.io/jobman
- Documentation: https://ryancswallace.github.io/jobman
- License: MIT
-
Latest release: 0.4.0
published over 2 years ago
Rankings
Maintainers (1)
Dependencies
- actions/checkout v3 composite
- actions/configure-pages v2 composite
- actions/deploy-pages v1 composite
- actions/jekyll-build-pages v1 composite
- actions/upload-pages-artifact v1 composite
- actions/checkout v3 composite
- actions/setup-python v4 composite
- ncipollo/release-action v1 composite
- snok/install-poetry v1 composite
- actions/checkout v3 composite
- actions/setup-python v4 composite
- snok/install-poetry v1 composite
- altgraph 0.17.3
- annotated-types 0.5.0
- autoflake 2.2.1
- black 23.9.1
- cachetools 5.3.1
- chardet 5.2.0
- click 8.1.7
- colorama 0.4.6
- coverage 7.3.1
- distlib 0.3.7
- exceptiongroup 1.1.3
- filelock 3.12.3
- iniconfig 2.0.0
- isort 5.12.0
- macholib 1.16.2
- markdown-it-py 3.0.0
- mdurl 0.1.2
- mypy 1.5.1
- mypy-extensions 1.0.0
- packaging 23.1
- pathspec 0.11.2
- peewee 3.16.3
- pefile 2023.2.7
- platformdirs 3.10.0
- pluggy 1.3.0
- psutil 5.9.5
- pydantic 2.3.0
- pydantic-core 2.6.3
- pyflakes 3.1.0
- pygments 2.16.1
- pyinstaller 5.13.2
- pyinstaller-hooks-contrib 2023.8
- pyproject-api 1.6.1
- pytest 7.4.2
- pytest-cov 4.1.0
- pywin32-ctypes 0.2.2
- rich 13.5.2
- ruamel-yaml 0.17.32
- ruamel-yaml-clib 0.2.7
- setuptools 68.2.0
- tomli 2.0.1
- tox 4.11.3
- types-peewee 3.16.0.2
- types-psutil 5.9.5.16
- typing-extensions 4.7.1
- virtualenv 20.24.5
- autoflake ^2.2.1 develop
- black ^23.7.0 develop
- isort ^5.12.0 develop
- pyinstaller ^5.13.2 develop
- click ^8.1.7
- peewee ^3.16.3
- psutil ^5.9.5
- pydantic ^2.3.0
- python >=3.9,<3.13
- rich ^13.5.2
- ruamel-yaml ^0.17.32
- mypy ^1.5.1 test
- pytest ^7.4.1 test
- pytest-cov ^4.1.0 test
- tox ^4.11.1 test
- types-peewee ^3.16.0.2 test
- types-psutil ^5.9.5.16 test