cromwell

Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments

https://github.com/broadinstitute/cromwell

Science Score: 62.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
✓
Committers with academic emails
46 of 156 committers (29.5%) from academic institutions
✓
Institutional organization owner
Organization broadinstitute has institutional domain (www.broadinstitute.org)
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (14.0%) to scientific vocabulary

Keywords

application bioinformatics cloud containers docker executor ga4gh hpc scala wdl workflow workflow-description-language workflow-execution

Keywords from Contributors

cwl common-workflow-language workflows commonwl sciworkflows genomics pypi workflow-engine dataflow nextflow

Last synced: 6 months ago · JSON representation ·

Repository

Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments

Basic Info

Host: GitHub
Owner: broadinstitute
License: bsd-3-clause
Language: Scala
Default Branch: develop
Homepage: http://cromwell.readthedocs.io/
Size: 48.2 MB

Statistics

Stars: 1,032
Watchers: 112
Forks: 373
Open Issues: 811
Releases: 95

Topics

application bioinformatics cloud containers docker executor ga4gh hpc scala wdl workflow workflow-description-language workflow-execution

Created almost 11 years ago · Last pushed 6 months ago

Metadata Files

Readme Changelog Contributing License Citation Codeowners Support Authors

Welcome to Cromwell

Cromwell is an open-source Workflow Management System for bioinformatics. Licensing is BSD 3-Clause.

The Cromwell documentation has a dedicated site.

First time to Cromwell? Get started with Tutorials.

Google LifeSciences is shutting down in July - migrate to GCP Batch

Google is replacing Cloud Life Sciences (sometimes referred to as PAPI) with a next-generation API named GCP Batch https://cloud.google.com/batch/docs/migrate-to-batch-from-cloud-life-sciences.

See documentation on how to migrate to GCP Batch

Community

Thinking about contributing to Cromwell? Get started by reading our Contributor Guide.

Cromwell has a growing ecosystem of community-backed projects to make your experience even better! Check out our Ecosystem page to learn more.

Talk to us: - Join the Cromwell Slack workspace to discuss the Cromwell workflow engine. - Join the OpenWDL Slack workspace to discuss the evolution of the WDL language itself. - More information about WDL is available in that project's repository.

Capabilities and roadmap

Many users today run their WDL workflows in Terra, a managed cloud bioinformatics platform with built-in WDL support provided by Cromwell. See here for a quick-start guide.

Users with specialized needs who wish to install and maintain their own Cromwell instances can download a JAR or Docker image. The development team accepts reproducible bug reports from self-managed instances, but cannot feasibly provide direct support.

Cromwell's backends receive development resources proportional to user demand. The team is actively developing for Google Cloud and Microsoft Azure (see Cromwell on Azure). Maintenance of other backends is primarily community-based.

Cromwell supports the WDL workflow language. Cromwell version 80 and above no longer support CWL.

CWL will be re-introduced at a later date in the Terra platform, using a solution other than Cromwell. See the blog post "Terra’s roadmap to supporting more workflow languages" for details.

Security reports

If you believe you have found a security issue please contact infosec@broadinstitute.org.

Issue tracking

Need to file an issue? Head over to Github Issues.

If you previously filed an issue in JIRA, the link is here. New signups are no longer accepted.

Jamie, the Cromwell pig

Owner

Name: Broad Institute
Login: broadinstitute
Kind: organization
Location: Cambridge, MA

Website: http://www.broadinstitute.org/
Twitter: broadinstitute
Repositories: 1,083
Profile: https://github.com/broadinstitute

Broad Institute of MIT and Harvard

Citation (CITATION.md)

# Citing `Cromwell`

If you use `Cromwell` in your work we would prefer it if you would use the following reference in your work.

## BibTeX

```bibtex
@misc{https://doi.org/10.7490/f1000research.1114634.1,
  doi = {10.7490/f1000research.1114634.1},
  url = {https://f1000research.com/slides/6-1381},
  author = {Voss,  Kate and Auwera,  Geraldine Van Der and Gentry,  Jeff},
  title = {Full-stack genomics pipelining with GATK4 + WDL + Cromwell [version 1; not peer reviewed]},
  journal = {ISCB Comm J},
  publisher = {F1000Research},
  type = {slides},
  volume = {6},
  number = {1381},
  year = {2017}
}
```

## Textual

Voss K, Van der Auwera G and Gentry J. Full-stack genomics pipelining with GATK4 + WDL + Cromwell
[version 1; not peer reviewed]. _F1000Research_ 2017, **6**(ISCB Comm J):1381 (slides)
(https://doi.org/10.7490/f1000research.1114634.1)

Committers

Last synced: about 2 years ago

All Time

Total Commits: 4,120
Total Committers: 156
Avg Commits per committer: 26.41
Development Distribution Score (DDS): 0.876

Past Year

Commits: 133
Committers: 16
Avg Commits per committer: 8.313
Development Distribution Score (DDS): 0.737

Top Committers

Name	Email	Commits
Khalid Shakir	k**r@b**g	509
Miguel Covarrubias	m**r@b**g	399
Chris Llanwarne	c**e@g**m	343
Chris Llanwarne	c****e	340
Thib	H****h	325
Jeff Gentry	g**y@g**m	270
Miguel Covarrubias	m****r	217
Scott Frazer	s**r@g**m	158
Grigoriy Sterin	g**n@b**g	149
Ruchi	r**i@b**g	135
Thibault Jeandet	t**t@b**g	127
Saloni Shah	s**1@g**m	96
Adam Nichols	a**s@g**m	91
Dan Billings	d****s	91
Adam Nichols	a**s@b**g	77
Janet Gainer-Dewar	j**r@b**g	76
Dan Billings	d**b@b**g	48
Kristian Cibulskis	k**l@b**g	44
Rebecca Asch	r**h@b**g	42
Tom Wiseman	t**n@b**g	32
Scala Steward	me@s****g	28
Scott Frazer	s**r@b**g	23
Ruben Vorderman	r**n@l**l	23
Francisco M. Casares	f**s@g**m	19
Christian Freitas	c**s@b**g	18
Chris Llanwarne	c**l@b**g	18
Brian Reilly	b**y@b**g	17
Christian Freitas	c**s@g**m	17
Christina Ahrens Roberts	r**s@b**g	17
Katrina P	6****3	15
and 126 more...

Committer Domains (Top 20 + Academic)

broadinstitute.org: 38 intel.com: 3 google.com: 2 epam.com: 2 broadinstitute.com: 2 prinsesmaximacentrum.nl: 1 illusional.net: 1 petermac.org: 1 patto.net: 1 burwood.com: 1 kshakir.org: 1 list.ru: 1 ohsu.edu: 1 wm963-226.broadinstitute.org: 1 lerch.org: 1 pacificbiosciences.com: 1 lumc.nl: 1 scala-steward.org: 1 fhcrc.org: 1 masella.name: 1 ucsc.edu: 1 stanford.edu: 1 wustl.edu: 1 uw.edu: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 163
Total pull requests: 978
Average time to close issues: over 1 year
Average time to close pull requests: 22 days
Total issue authors: 115
Total pull request authors: 56
Average comments per issue: 2.22
Average comments per pull request: 0.45
Merged pull requests: 540
Bot issues: 0
Bot pull requests: 3

Past Year

Issues: 34
Pull requests: 435
Average time to close issues: 4 days
Average time to close pull requests: 4 days
Issue authors: 28
Pull request authors: 27
Average comments per issue: 0.15
Average comments per pull request: 0.29
Merged pull requests: 213
Bot issues: 0
Bot pull requests: 2

View more stats

Top Authors

Issue Authors

aofarrel (7)
danbills (5)
mcovarr (5)
yihming (5)
aednichols (4)
GregoryDougherty (4)
freeseek (4)
LiterallyUniqueLogin (3)
jeremylp2 (2)
karlkashofer (2)
EugeneEA (2)
jbakerpmc (2)
Lipastomies (2)
elerch (2)
davidlougheed (2)

Pull Request Authors

scala-steward (252)
aednichols (173)
jgainerdewar (154)
mcovarr (74)
THWiseman (70)
salonishah11 (33)
lucymcnatt (28)
kshakir (25)
AlexITC (23)
rsaperst (14)
dspeck1 (14)
JVThomas (13)
cjllanwarne (10)
kraefrei (9)
LizBaldo (8)

Top Labels

Issue Labels

GCP Batch (13) Needs Triage (6) AWS Backend (3) User Requested Improvement (2) Epic (2) 💚GreenFriends (1) 🌿Mint Friends (1) 📞💵Call Caching (1) s3 (1)

Pull Request Labels

GCP Batch (78) ⛔ DO NOT MERGE ⛔ (22) AWS Backend (5) Community Contribution (5) Documentation (2) CI Clone :woman: :woman: (2) After GCP Batch in Terra (2) Don't Look At Me 🙈 (2) dependencies (1) java (1) Please re-review now (1) Docs (1)

Packages

Total packages: 3
Total downloads:
- homebrew 24 last-month
- pypi 23 last-month

Total dependent packages: 0
(may contain duplicates)
Total dependent repositories: 7
(may contain duplicates)
Total versions: 59
Total maintainers: 1

pypi.org: cromtool

Development tool for Cromwell

Homepage: http://github.com/broadinstitute/cromwell
Documentation: https://cromtool.readthedocs.io/
License: MIT
Latest release: 1.4
published almost 10 years ago

Versions: 5
Dependent Packages: 0
Dependent Repositories: 2
Downloads: 23 Last month

Rankings

Stargazers count: 2.1%

Forks count: 2.8%

Dependent packages count: 10.1%

Dependent repos count: 11.5%

Average: 11.6%

Downloads: 31.4%

Maintainers (1)

scottfrazer

Last synced: 6 months ago

conda-forge.org: cromwell

Homepage: https://github.com/broadinstitute/cromwell
License: BSD-3-Clause
Latest release: 83
published over 3 years ago

Versions: 41
Dependent Packages: 0
Dependent Repositories: 5

Rankings

Forks count: 9.7%

Stargazers count: 13.6%

Dependent repos count: 14.8%

Average: 22.4%

Dependent packages count: 51.6%

Last synced: 6 months ago

formulae.brew.sh: cromwell

Workflow Execution Engine using Workflow Description Language

Homepage: https://github.com/broadinstitute/cromwell
License: BSD-3-Clause
Latest release: 90
published 9 months ago

Versions: 13
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 24 Last month

Rankings

Forks count: 8.0%

Stargazers count: 17.4%

Dependent packages count: 19.0%

Average: 32.4%

Dependent repos count: 50.7%

Downloads: 67.0%

Last synced: 6 months ago

Dependencies

CromwellRefdiskManifestCreator/pom.xml maven

com.fasterxml.jackson.core:jackson-databind 2.13.2.2
org.apache.logging.log4j:log4j-api 2.17.1
org.apache.logging.log4j:log4j-core 2.17.1
junit:junit 4.13.1 test
org.hamcrest:java-hamcrest 2.0.0.0 test

scripts/backpressure_report/setup.py pypi

python-dateutil *

supportedBackends/google/pipelines/v2beta/src/main/resources/cromwell-monitor/requirements.txt pypi

google-cloud-monitoring *
psutil *
requests *

.github/workflows/chart_update_on_merge.yml actions

actions/checkout v2 composite
broadinstitute/repository-dispatch master composite
docker/login-action v1 composite
olafurpg/setup-scala v10 composite

.github/workflows/combine_scalasteward_prs.yml actions

actions/checkout v2 composite

.github/workflows/make_publish_prs.yml actions

actions/checkout v2 composite
actions/github-script v6 composite

.github/workflows/trivy.yml actions

actions/cache v2 composite
actions/checkout v2 composite
broadinstitute/dsp-appsec-trivy-action v1 composite
olafurpg/setup-scala v10 composite

centaur/src/main/resources/standardTestCases/non_root_default_user/Dockerfile docker

ubuntu latest build

publish/Dockerfile docker

linuxbrew/brew latest build

scripts/docker-compose-mysql/compose/cromwell/Dockerfile docker

broadinstitute/cromwell develop build

scripts/docker-compose-mysql/docker-compose.yml docker

mysql 5.7

scripts/docker-develop/Dockerfile docker

ubuntu focal build

scripts/perf/vm_scripts/docker-compose.yml docker

${CROMWELL_DOCKER_IMAGE} latest
gcr.io/cloudsql-docker/gce-proxy 1.11

src/ci/docker-compose/cromwell-test/Dockerfile docker

ubuntu focal build

src/ci/docker-compose/docker-compose-deadlock.yml docker

broadinstitute/cromwell ${CROMWELL_BUILD_DOCKER_TAG}
dockercloud/haproxy latest
mysql 5.7
python 3.7

src/ci/docker-compose/docker-compose-horicromtal.yml docker

broadinstitute/cromwell ${CROMWELL_BUILD_CENTAUR_MANAGED_TAG}

src/ci/docker-compose/docker-compose.yml docker

mariadb 10.3
mysql 5.7
postgres 11.3

supportedBackends/google/pipelines/v2beta/src/main/resources/cromwell-monitor/Dockerfile docker

python alpine build

.github/set_up_cromwell_action/action.yml actions

actions/checkout v3 composite
actions/setup-java v3 composite
coursier/cache-action v6 composite

.github/workflows/cromwell_unit_tests.yml actions

./.github/set_up_cromwell_action * composite
actions/checkout v3 composite

.github/workflows/consumer_contract_tests.yml actions

actions/checkout v3 composite
broadinstitute/workflow-dispatch v3 composite

.github/workflows/docker_build_test.yml actions

actions/checkout v2 composite
olafurpg/setup-scala v10 composite

.github/workflows/integration_tests.yml actions

./.github/set_up_cromwell_action * composite
actions/checkout v3 composite
ravsamhq/notify-slack-action v2 composite

.github/workflows/validate_pr_name.yml actions

actions/github-script v3 composite

scripts/docker-compose-mysql/jes-cromwell/docker-compose.yml docker

.github/workflows/azure_e2e_run_workflow.yml actions

broadinstitute/workflow-dispatch v3 composite

.github/workflows/scalafmt-check.yml actions

./.github/set_up_cromwell_action * composite
actions/checkout v3 composite

.github/workflows/scalafmt-fix.yml actions

./.github/set_up_cromwell_action * composite
actions/checkout v3 composite

cromwell

Science Score: 62.0%

Keywords

Keywords from Contributors

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

Welcome to Cromwell

Google LifeSciences is shutting down in July - migrate to GCP Batch

Community

Capabilities and roadmap

Security reports

Issue tracking

Owner

Citation (CITATION.md)

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: cromtool

Rankings

Maintainers (1)

conda-forge.org: cromwell

Rankings

formulae.brew.sh: cromwell

Rankings

Dependencies