metagenomics

A Nextflow workflow for QC, evaluation, and profiling of metagenomic samples using short- and long-read technologies

https://github.com/arcadia-science/metagenomics

Science Score: 57.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 4 DOI reference(s) in README
○
Academic publication links
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (13.1%) to scientific vocabulary

Keywords

illumina metagenomics nanopore

Last synced: 10 months ago · JSON representation ·

Repository

A Nextflow workflow for QC, evaluation, and profiling of metagenomic samples using short- and long-read technologies

Basic Info

Host: GitHub
Owner: Arcadia-Science
License: mit
Language: Nextflow
Default Branch: main
Homepage:
Size: 2.41 MB

Statistics

Stars: 36
Watchers: 7
Forks: 3
Open Issues: 7
Releases: 1

Topics

illumina metagenomics nanopore

Created over 3 years ago · Last pushed about 3 years ago

Metadata Files

Readme Changelog License Citation

Arcadia-Science/metagenomics

Introduction

Arcadia-Science/metagenomics is a pipeline for profiling metagenomes obtained through either Illumina or Nanopore technologies.

The pipeline is built using Nextflow, a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The Nextflow DSL2 implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from nf-core/modules in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!

Pipeline Summary

This pipeline performs common QC, processing, and profiling steps of metagenomes obtained through either Illumina or Nanopore technologies. The pipeline consists of two separate workflows for processing the Illumina and Nanopore data and producing assemblies. Therefore Illumina or Nanopore samples are processed separately, as this pipeline does not handle hybrid assembly or polishing with short reads. Downstream steps for summarizing the composition of metagenomes are mostly identical for the two technologies. You can find more information about how the pipeline operates in the docs.

Quick Start

Install Nextflow (>=22.10.1)
Install Docker, Singularity (you can follow this tutorial), or Conda. You can use conda to install Nextflow itself but use it for managing software within pipelines as a last resort. We recommend using Docker if possible as this has been tested most frequently. See the nf-core docs) for more information.
Download the pipeline and test it on the minimal datasets for Illumina and Nanopore:

bash nextflow run Arcadia-Science/metagenomics -profile test_illumina,YOURPROFILE --outdir <OUTDIR>

bash nextflow run Arcadia-Science/metagenomics -profile test_nanopore,YOURPROFILE --outdir <OUTDIR>

Note that some form of configuration will be needed so that Nextflow knows how to fetch the required software. This is usually done in the form of a config profile (YOURPROFILE in the example command above). You can chain multiple config profiles in a comma-separated string.

The pipeline comes with several config profiles, but we recommend using docker when possible, such as -profile test,docker.

Please check nf-core/configs to see if a custom config file to run nf-core pipelines already exists for your institute. If so, you can simply use -profile <institute> in your command. This will enable either docker or singularity and set the appropriate execution settings for your local compute environment.

If you are using singularity, please use the nf-core download command to download images first, before running the pipeline. Setting the NXF_SINGULARITY_CACHEDIR or singularity.cacheDir Nextflow options enables you to store and re-use the images from a central location for future pipeline runs.

If you are using conda, it is highly recommended to use the NXF_CONDA_CACHEDIR or conda.cacheDir settings to store the environments in a central location for future pipeline runs.

Start running your own analysis!

Prior to running the workflow on your samples, you will need to prepare and download databases for sourmash and DIAMOND. There are a number of already prepared sourmash databases available, and you can checkout the DIAMOND documentation for creating a database compatible with diamond blastp used in the workflow. See the usage documentation for more information on how to prepare these databases.

bash nextflow run Arcadia-Science/metagenomics --input samplesheet.csv --outdir <OUTDIR> --platform <illumina|nanopore> --sourmash_dbs sourmash_dbs_paths.csv --diamond_db prepared_diamond_db.dmnd -profile <docker/singularity/conda/institute>

You can find more information about how to format your input samplesheet CSV and providing the paths to pre-downloaded sourmash and DIAMOND database files in docs/usage.md.

Citations

The nf-core template was used as a guideline for putting this workflow together. You can cite the nf-core publication as follows:

The nf-core framework for community-curated bioinformatics pipelines.

Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.

Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x. An extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.

Owner

Name: Arcadia Science
Login: Arcadia-Science
Kind: organization
Location: United States of America

Website: https://www.arcadiascience.com/
Twitter: ArcadiaScience
Repositories: 16
Profile: https://github.com/Arcadia-Science

Citation (CITATIONS.md)

# Arcadia-Science/metagenomics Citations

Below are citations of tools used in the pipeline that you should cite in your work when using this pipeline.

## Nextflow and Reporting Tools

- [nf-core](https://pubmed.ncbi.nlm.nih.gov/32055031/)

> Ewels PA, Peltzer A, Fillinger S, Patel H, Alneberg J, Wilm A, Garcia MU, Di Tommaso P, Nahnsen S. (2020). The nf-core framework for community-curated bioinformatics pipelines. [https://doi.org/10.1038/s41587-020-0439-x](https://www.nature.com/articles/s41587-020-0439-x)

- [Nextflow](https://pubmed.ncbi.nlm.nih.gov/28398311/)

> Di Tommaso P, Chatzou M, Floden EW, Barja PP, Palumbo E, Notredame C. Nextflow enables reproducible computational workflows. (2017). [doi: 10.1038/nbt.3820](https://www.nature.com/articles/nbt.3820)

- [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)
  > Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. (2016). [doi: 10.1093/bioinformatics/btw354.](https://academic.oup.com/bioinformatics/article/32/19/3047/2196507)

## Pipeline tools

Below are pipeline-specific tools, categorized by tools for Illumina preprocessing, Nanopore preprocessing, and general software tools.

### Illumina Pipeline Tools

- [Fastp](https://doi.org/10.1093/bioinformatics/bty560)
  > Shifu Chen, Yanqing Zhou, Yaru Chen, Jia Gu; fastp: an ultra-fast all-in-one FASTQ preprocessor. (2018). [https://doi.org/10.1093/bioinformatics/bty560](https://doi.org/10.1093/bioinformatics/bty560)
- [SPAdes](https://currentprotocols.onlinelibrary.wiley.com/doi/abs/10.1002/cpbi.102)
  > Andrey Prjibelski,Dmitry Antipov,Dmitry Meleshko,Alla Lapidus,Anton Korobeynikov. Using SPAdes De Novo Assembler. (2020). [https://doi.org/10.1002/cpbi.102](https://currentprotocols.onlinelibrary.wiley.com/doi/abs/10.1002/cpbi.102)
- [Bowtie2](https://www.nature.com/articles/nmeth.1923)
  > Ben Langmead and Steven L Salzberg. Fast-gapped read alignment with bowtie2. (2012). [https://doi.org/10.1038/nmeth.1923](https://www.nature.com/articles/nmeth.1923)

### Nanopore Pipeline Tools

- [NanoPlot](https://academic.oup.com/bioinformatics/article/34/15/2666/4934939)
  > De Coster W, D'Hert S, Schultz DT, Cruts M, Van Broeckhoven C. NanoPack: visualizing and processing long-read sequencing data. (2018). [https://doi.org/10.1093/bioinformatics/bty149](https://academic.oup.com/bioinformatics/article/34/15/2666/4934939)
- [Porechop_ABI](https://www.biorxiv.org/content/10.1101/2022.07.07.499093v1)
  > Bonenfant Q, Noe L, Touzet H. Porechop_ABI: discovering unknown adapters in ONT sequencing reads for downstream trimming. (2022). [doi:10.1101/2022.07.07.499093](https://academic.oup.com/bioinformatics/article/34/15/2666/4934939)
- [Flye](https://www.nature.com/articles/s41587-019-0072-8)
  > Kolmogorov M, Yuan J, Lin Y, Pevzner P. Assembly of long, error-prone reads using repeat graphs. (2019). [doi:10.1038/s41587-019-0072-8](https://www.nature.com/articles/s41587-019-0072-8)
- [minimap2](https://academic.oup.com/bioinformatics/article/34/18/3094/4994778)
  > Li H. Minimap2: pairwise alignment for nucleotide sequences. (2018). [https://doi.org/10.1093/bioinformatics/bty191](https://academic.oup.com/bioinformatics/article/34/18/3094/4994778)
- [Medaka](https://github.com/nanoporetech/medaka)
  > ONT Research. (2018). [https://github.com/nanoporetech/medaka](https://github.com/nanoporetech/medaka)

### General Pipeline Tools

- [QUAST](https://academic.oup.com/bioinformatics/article/29/8/1072/228832?login=false)
  > Gurevich A, Saveliev V, Vyahhi N, Tesler G. QUAST: quality assessment tool for genome assemblies. (2013). [doi:10.1093/bioinformatics/btt086](https://academic.oup.com/bioinformatics/article/29/8/1072/228832?login=false)
- [metabat2 JGI Summarize Contigs](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6662567/)
  > Dongwan D. Kang,1 Feng Li,2 Edward Kirton,1 Ashleigh Thomas,1 Rob Egan,1 Hong An,2 and Zhong Wang. MetaBAT 2: an adaptive binning algorithm for robust and efficient genome reconstruction from metagenome assemblies. (2019). [https://doi.org/10.7717/peerj.7359](https://peerj.com/articles/7359/)
- [Samtools](https://academic.oup.com/gigascience/article/10/2/giab008/6137722?login=false)
  > Petr Danecek, James K Bonfield, Jennifer Liddle, John Marshall, Valeriu Ohan, Martin O Pollard, Andrew Whitwham, Thomas Keane, Shane A McCarthy, Robert M Davies, Heng Li. Twelve years of SAMtools and BCFtools. (2021). [https://doi.org/10.1093/gigascience/giab008](https://academic.oup.com/gigascience/article/10/2/giab008/6137722?login=false)
- [Sourmash](https://joss.theoj.org/papers/10.21105/joss.00027)
  > Brown CT, Irber L. (2016). Sourmash: a library for MinHash sketching of DNA. [https://doi.org/10.21105/joss.00027](https://joss.theoj.org/papers/10.21105/joss.00027)
- [Biopyton](https://biopython.org/)
  > Cock PJA, Antao T, Chang JT, Chapman BA, Cox CJ, Dalke A, Friedberg I, Hamelryck T, Kauff F, Wilcynzski B, de Hoon MJL. Biopython: freely available Python tools for computational molecular biology and bioinformatics. (2009). [https://doi.org/10.1093/bioinformatics/btp163](https://academic.oup.com/bioinformatics/article/25/11/1422/330687?login=false)
- [Prodigal](https://github.com/hyattpd/Prodigal)
  > Hyatt D, Chen GL, LoCascio PF, Land ML, Larimer FW, Hauser LJ. Prodigal: prokaryotic gene recognition and translation initiation site identification. (2010). [https://doi.org/10.1186/1471-2105-11-119](https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-11-119)
- [DIAMOND](https://github.com/bbuchfink/diamond)
  > Buchfink B, Xie C, Huson DH. Fast and sensitive protein alignment using DIAMOND. (2014). [https://doi.org/10.1038/nmeth.3176](https://www.nature.com/articles/nmeth.3176)

## Software packaging/containerisation tools

- [Anaconda](https://anaconda.com)

  > Anaconda Software Distribution. Computer software. Vers. 2-2.4.0. Anaconda, Nov. 2016. Web.

- [Bioconda](https://pubmed.ncbi.nlm.nih.gov/29967506/)

  > Grüning B, Dale R, Sjödin A, Chapman BA, Rowe J, Tomkins-Tinch CH, Valieris R, Köster J; Bioconda Team. Bioconda: sustainable and comprehensive software distribution for the life sciences. Nat Methods. 2018 Jul;15(7):475-476. doi: 10.1038/s41592-018-0046-7. PubMed PMID: 29967506.

- [BioContainers](https://pubmed.ncbi.nlm.nih.gov/28379341/)

  > da Veiga Leprevost F, Grüning B, Aflitos SA, Röst HL, Uszkoreit J, Barsnes H, Vaudel M, Moreno P, Gatto L, Weber J, Bai M, Jimenez RC, Sachsenberg T, Pfeuffer J, Alvarez RV, Griss J, Nesvizhskii AI, Perez-Riverol Y. BioContainers: an open-source and community-driven framework for software standardization. Bioinformatics. 2017 Aug 15;33(16):2580-2582. doi: 10.1093/bioinformatics/btx192. PubMed PMID: 28379341; PubMed Central PMCID: PMC5870671.

- [Docker](https://dl.acm.org/doi/10.5555/2600239.2600241)

- [Singularity](https://pubmed.ncbi.nlm.nih.gov/28494014/)
  > Kurtzer GM, Sochat V, Bauer MW. Singularity: Scientific containers for mobility of compute. PLoS One. 2017 May 11;12(5):e0177459. doi: 10.1371/journal.pone.0177459. eCollection 2017. PubMed PMID: 28494014; PubMed Central PMCID: PMC5426675.

GitHub Events

Total

Watch event: 2
Fork event: 1

Last Year

Watch event: 2
Fork event: 1

Dependencies

.github/workflows/awstest.yml actions

actions/upload-artifact v3 composite
nf-core/tower-action v3 composite

.github/workflows/ci.yml actions

actions/checkout v3 composite
nf-core/setup-nextflow v1 composite

.github/workflows/linting.yml actions

actions/checkout v3 composite
actions/setup-node v3 composite
mshick/add-pr-comment v1 composite
psf/black stable composite

modules/local/nf-core-modified/medaka/meta.yml cpan

modules/local/nf-core-modified/metabat2/jgisummarizebamcontigdepths/meta.yml cpan

modules/local/nf-core-modified/minimap2/align/meta.yml cpan

modules/local/nf-core-modified/nanoplot/meta.yml cpan

modules/local/nf-core-modified/prodigal/meta.yml cpan

modules/local/nf-core-modified/quast/meta.yml cpan

modules/local/nf-core-modified/sourmash/compare/meta.yml cpan

modules/local/nf-core-modified/sourmash/gather/meta.yml cpan

modules/local/nf-core-modified/sourmash/sketch/meta.yml cpan

modules/local/nf-core-modified/sourmash/taxannotate/meta.yml cpan

modules/nf-core/custom/dumpsoftwareversions/meta.yml cpan

modules/nf-core/diamond/blastp/meta.yml cpan

modules/nf-core/fastp/meta.yml cpan

modules/nf-core/flye/meta.yml cpan

modules/nf-core/megan/daa2info/meta.yml cpan

modules/nf-core/minimap2/index/meta.yml cpan

modules/nf-core/multiqc/meta.yml cpan

modules/nf-core/porechop/abi/meta.yml cpan

modules/nf-core/samtools/stats/meta.yml cpan

pyproject.toml pypi

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science