Data

Tools

Indexes

Applications

Experiments

Open Source Science

preprocessing

Preprocessing workflow for sequencing data at CMGG

https://github.com/nf-cmgg/preprocessing

Science Score: 57.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 4 DOI reference(s) in README
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (7.2%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

Preprocessing workflow for sequencing data at CMGG

Basic Info
  • Host: GitHub
  • Owner: nf-cmgg
  • License: mit
  • Language: Nextflow
  • Default Branch: main
  • Size: 19.5 MB
Statistics
  • Stars: 5
  • Watchers: 2
  • Forks: 4
  • Open Issues: 6
  • Releases: 13
Created almost 4 years ago · Last pushed 8 months ago
Metadata Files
Readme Changelog Contributing License Citation

README.md

nf-cmgg/preprocessing nf-cmgg/preprocessing

GitHub Actions CI Status GitHub Actions Linting Status Nextflow run with conda run with docker run with singularity

Introduction

nf-cmgg/preprocessing is a bioinformatics pipeline that demultiplexes and aligns raw sequencing data. It also performs basic QC and coverage analysis.

The pipeline is built using Nextflow, a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker containers making installation trivial and results highly reproducible.

Steps inlcude:

  1. Demultiplexing using BCLconvert
  2. Read QC and trimming using fastp
  3. Alignment using either bwa, bwa-mem2, bowtie2, dragmap or snap for DNA-seq and STAR for RNA-seq
  4. Duplicate marking using bamsormadup or samtools markdup
  5. Coverage analysis using mosdepth and samtools coverage
  6. Alignment QC using samtools flagstat, samtools stats, samtools idxstats and picard CollecHsMetrics, picard CollectWgsMetrics, picard CollectMultipleMetrics
  7. QC aggregation using multiqc

metro map

Usage

[!NOTE] If you are new to Nextflow and nf-core, please refer to this page on how to set-up Nextflow. Make sure to test your setup with -profile test before running the workflow on actual data.

The full documentation can be found here

First, prepare a samplesheet with your input data that looks as follows:

samplesheet.csv for fastq inputs:

csv id,samplename,organism,library,fastq_1,fastq_2 sample1,sample1,Homo sapiens,Library_Name,reads1.fq.gz,reads2.fq.gz

samplesheet.csv for flowcell inputs:

csv id,samplesheet,lane,flowcell,sample_info flowcell_id,/path/to/illumina_samplesheet.csv,1,/path/to/sequencer_uploaddir,/path/to/sampleinfo.csv

sampleinfo.csv for use with flowcell inputs:

csv samplename,library,organism,tag fc_sample1,test,Homo sapiens,WES

Now, you can run the pipeline using:

bash nextflow run nf-cmgg/preprocessing \ -profile <docker/singularity/.../institute> \ --igenomes_base /path/to/genomes \ --input samplesheet.csv \ --outdir <OUTDIR>

[!WARNING] Please provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.

Credits

nf-cmgg/preprocessing was originally written by the CMGG ICT team.

Support

This pipeline uses code and infrastructure developed and maintained by the nf-core community, reused here under the MIT license.

The nf-core framework for community-curated bioinformatics pipelines.

Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.

Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.

Owner

  • Name: Center For Medical Genetics Ghent - Nextflow workflows
  • Login: nf-cmgg
  • Kind: organization
  • Email: ict@cmgg.be
  • Location: Belgium

Center For Medical Genetics Ghent - Ghent University Hospital - Nextflow workflows

Citation (CITATIONS.md) 

# nf-cmgg/preprocessing: Citations

## [nf-core](https://pubmed.ncbi.nlm.nih.gov/32055031/)

> Ewels PA, Peltzer A, Fillinger S, Patel H, Alneberg J, Wilm A, Garcia MU, Di Tommaso P, Nahnsen S. The nf-core framework for community-curated bioinformatics pipelines. Nat Biotechnol. 2020 Mar;38(3):276-278. doi: 10.1038/s41587-020-0439-x. PubMed PMID: 32055031.

## [Nextflow](https://pubmed.ncbi.nlm.nih.gov/28398311/)

> Di Tommaso P, Chatzou M, Floden EW, Barja PP, Palumbo E, Notredame C. Nextflow enables reproducible computational workflows. Nat Biotechnol. 2017 Apr 11;35(4):316-319. doi: 10.1038/nbt.3820. PubMed PMID: 28398311.

## Pipeline tools

- [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/)

> Andrews, S. (2010). FastQC: A Quality Control Tool for High Throughput Sequence Data [Online].

- [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)

> Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924.

## Software packaging/containerisation tools

- [Anaconda](https://anaconda.com)

  > Anaconda Software Distribution. Computer software. Vers. 2-2.4.0. Anaconda, Nov. 2016. Web.

- [Bioconda](https://pubmed.ncbi.nlm.nih.gov/29967506/)

  > Grüning B, Dale R, Sjödin A, Chapman BA, Rowe J, Tomkins-Tinch CH, Valieris R, Köster J; Bioconda Team. Bioconda: sustainable and comprehensive software distribution for the life sciences. Nat Methods. 2018 Jul;15(7):475-476. doi: 10.1038/s41592-018-0046-7. PubMed PMID: 29967506.

- [BioContainers](https://pubmed.ncbi.nlm.nih.gov/28379341/)

  > da Veiga Leprevost F, Grüning B, Aflitos SA, Röst HL, Uszkoreit J, Barsnes H, Vaudel M, Moreno P, Gatto L, Weber J, Bai M, Jimenez RC, Sachsenberg T, Pfeuffer J, Alvarez RV, Griss J, Nesvizhskii AI, Perez-Riverol Y. BioContainers: an open-source and community-driven framework for software standardization. Bioinformatics. 2017 Aug 15;33(16):2580-2582. doi: 10.1093/bioinformatics/btx192. PubMed PMID: 28379341; PubMed Central PMCID: PMC5870671.

- [Docker](https://dl.acm.org/doi/10.5555/2600239.2600241)

  > Merkel, D. (2014). Docker: lightweight linux containers for consistent development and deployment. Linux Journal, 2014(239), 2. doi: 10.5555/2600239.2600241.

- [Singularity](https://pubmed.ncbi.nlm.nih.gov/28494014/)

  > Kurtzer GM, Sochat V, Bauer MW. Singularity: Scientific containers for mobility of compute. PLoS One. 2017 May 11;12(5):e0177459. doi: 10.1371/journal.pone.0177459. eCollection 2017. PubMed PMID: 28494014; PubMed Central PMCID: PMC5426675.

GitHub Events

Total
  • Create event: 16
  • Release event: 3
  • Issues event: 11
  • Watch event: 2
  • Delete event: 14
  • Issue comment event: 9
  • Push event: 60
  • Pull request review event: 40
  • Pull request review comment event: 28
  • Pull request event: 31
  • Fork event: 3
Last Year
  • Create event: 16
  • Release event: 3
  • Issues event: 11
  • Watch event: 2
  • Delete event: 14
  • Issue comment event: 9
  • Push event: 60
  • Pull request review event: 40
  • Pull request review comment event: 28
  • Pull request event: 31
  • Fork event: 3

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 7
  • Total pull requests: 14
  • Average time to close issues: 3 months
  • Average time to close pull requests: 7 days
  • Total issue authors: 4
  • Total pull request authors: 3
  • Average comments per issue: 0.57
  • Average comments per pull request: 0.14
  • Merged pull requests: 8
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 6
  • Pull requests: 14
  • Average time to close issues: 2 months
  • Average time to close pull requests: 7 days
  • Issue authors: 3
  • Pull request authors: 3
  • Average comments per issue: 0.5
  • Average comments per pull request: 0.14
  • Merged pull requests: 8
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • matthdsm (7)
  • nvnieuwk (4)
  • mvheetve (2)
  • ToonRosseel (1)
Pull Request Authors
  • matthdsm (31)
  • nvnieuwk (11)
  • Hwanseok-Jeong (1)
Top Labels
Issue Labels
enhancement (9) bug (2)
Pull Request Labels
enhancement (1)

Dependencies

.github/workflows/branch.yml actions
  • mshick/add-pr-comment v1 composite
.github/workflows/ci.yml actions
  • actions/checkout v2 composite
  • nf-core/setup-nextflow v1 composite
.github/workflows/fix-linting.yml actions
  • actions/checkout v3 composite
  • actions/setup-node v2 composite
.github/workflows/linting.yml actions
  • actions/checkout v3 composite
  • actions/setup-node v3 composite
  • actions/setup-python v4 composite
  • actions/upload-artifact v3 composite
  • mshick/add-pr-comment v1 composite
  • nf-core/setup-nextflow v1 composite
  • psf/black stable composite
.github/workflows/linting_comment.yml actions
  • dawidd6/action-download-artifact v2 composite
  • marocchino/sticky-pull-request-comment v2 composite
modules/nf-core/bcl2fastq/Dockerfile docker
  • debian bullseye-slim build
modules/nf-core/bclconvert/Dockerfile docker
  • debian bullseye-slim build
modules/nf-core/bcl2fastq/meta.yml cpan
modules/nf-core/bclconvert/meta.yml cpan
modules/nf-core/biobambam/bamsormadup/meta.yml cpan
modules/nf-core/bowtie2/align/meta.yml cpan
modules/nf-core/bwa/mem/meta.yml cpan
modules/nf-core/bwamem2/mem/meta.yml cpan
modules/nf-core/cat/fastq/meta.yml cpan
modules/nf-core/custom/dumpsoftwareversions/meta.yml cpan
modules/nf-core/dragmap/align/meta.yml cpan
modules/nf-core/fastp/meta.yml cpan
modules/nf-core/fgbio/fastqtobam/meta.yml cpan
modules/nf-core/md5sum/meta.yml cpan
modules/nf-core/mosdepth/meta.yml cpan
modules/nf-core/multiqc/meta.yml cpan
modules/nf-core/picard/collecthsmetrics/meta.yml cpan
modules/nf-core/picard/collectmultiplemetrics/meta.yml cpan
modules/nf-core/picard/collectwgsmetrics/meta.yml cpan
modules/nf-core/samtools/collatefastq/meta.yml cpan
modules/nf-core/samtools/convert/meta.yml cpan
modules/nf-core/samtools/flagstat/meta.yml cpan
modules/nf-core/samtools/getrg/meta.yml cpan
modules/nf-core/samtools/idxstats/meta.yml cpan
modules/nf-core/samtools/index/meta.yml cpan
modules/nf-core/samtools/stats/meta.yml cpan
modules/nf-core/snapaligner/align/meta.yml cpan
subworkflows/nf-core/bam_stats_samtools/meta.yml cpan
subworkflows/nf-core/bcl_demultiplex/meta.yml cpan
subworkflows/nf-core/fastq_align_dna/meta.yml cpan
pyproject.toml pypi
.github/workflows/clean-up.yml actions
  • actions/stale v7 composite
modules/nf-core/samtools/cat/meta.yml cpan
modules/nf-core/bcl2fastq/environment.yml conda
modules/nf-core/biobambam/bamsormadup/environment.yml conda
  • biobambam 2.0.183.*
modules/nf-core/bowtie2/align/environment.yml conda
  • bowtie2 2.5.2.*
  • pigz 2.6.*
  • samtools 1.18.*
modules/nf-core/bwa/mem/environment.yml conda
  • bwa 0.7.17.*
  • samtools 1.18.*
modules/nf-core/bwamem2/mem/environment.yml conda
  • bwa-mem2 2.2.1.*
  • samtools 1.18.*
modules/nf-core/cat/fastq/environment.yml conda
  • sed 4.7.*
modules/nf-core/custom/dumpsoftwareversions/environment.yml conda
  • multiqc 1.17.*
modules/nf-core/dragmap/align/environment.yml conda
  • dragmap 1.2.1.*
  • pigz 2.3.4.*
  • samtools 1.15.1.*
modules/nf-core/fastp/environment.yml conda
  • fastp 0.23.4.*
modules/nf-core/fgbio/fastqtobam/environment.yml conda
  • fgbio 2.1.0.*
modules/nf-core/md5sum/environment.yml conda
  • coreutils 9.1.*
modules/nf-core/mosdepth/environment.yml conda
  • mosdepth 0.3.6.*
modules/nf-core/multiqc/environment.yml conda
  • multiqc 1.18.*
modules/nf-core/picard/collecthsmetrics/environment.yml conda
  • picard 3.1.1.*
modules/nf-core/picard/collectmultiplemetrics/environment.yml conda
  • picard 3.1.1.*
modules/nf-core/picard/collectwgsmetrics/environment.yml conda
  • picard 3.1.1.*
  • r-base
modules/nf-core/samtools/cat/environment.yml conda
  • samtools 1.18.*
modules/nf-core/samtools/collatefastq/environment.yml conda
  • samtools 1.18.*
modules/nf-core/samtools/convert/environment.yml conda
  • samtools 1.18.*
modules/nf-core/samtools/flagstat/environment.yml conda
  • samtools 1.18.*
modules/nf-core/samtools/getrg/environment.yml conda
  • samtools 1.18.*
modules/nf-core/samtools/idxstats/environment.yml conda
  • samtools 1.18.*
modules/nf-core/samtools/index/environment.yml conda
  • samtools 1.18.*
modules/nf-core/samtools/stats/environment.yml conda
  • samtools 1.18.*
modules/nf-core/snapaligner/align/environment.yml conda
  • snap-aligner 2.0.3.*