Data

Tools

Indexes

Applications

Experiments

Open Source Science

fastqrepair

A pipeline that can be used to recover corrupted FASTQ.gz files, drop or fix uncompliant reads, remove unpaired reads, and settles reads that became disordered

https://github.com/nf-core/fastqrepair

Science Score: 57.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 10 DOI reference(s) in README
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.6%) to scientific vocabulary

Keywords

corruption data-cleaning data-recovery fastq fastq-corrupted fastq-format nextflow nf-core pipeline reads-interleaving recovery-tool unpaired-reads well-formed workflow
Last synced: 6 months ago · JSON representation ·

Repository

A pipeline that can be used to recover corrupted FASTQ.gz files, drop or fix uncompliant reads, remove unpaired reads, and settles reads that became disordered

Basic Info
Statistics
  • Stars: 5
  • Watchers: 189
  • Forks: 5
  • Open Issues: 3
  • Releases: 1
Topics
corruption data-cleaning data-recovery fastq fastq-corrupted fastq-format nextflow nf-core pipeline reads-interleaving recovery-tool unpaired-reads well-formed workflow
Created over 1 year ago · Last pushed 6 months ago
Metadata Files
Readme Changelog Contributing License Code of conduct Citation

README.md

nf-core/fastqrepair

GitHub Actions CI Status GitHub Actions Linting StatusAWS CICite with Zenodo nf-test

Nextflow run with conda run with docker run with singularity Launch on Seqera Platform

Get help on SlackFollow on TwitterFollow on MastodonWatch on YouTube

Introduction

nf-core/fastqrepair is a bioinformatics pipeline that can be used to recover corrupted FASTQ.gz files, drop or fix uncompliant reads, remove unpaired reads, and settles reads that became disordered. It takes a samplesheet with FASTQ/FASTQ.gz files as input (both single-end and paired-end) and produces clean FASTQ files and QC reports.

pipeline_diagram

  1. Recover reads from corrupted fastq.gz file (gzrt)
  2. Make recovered reads well-formed (wipertools)
  3. Re-pair reads (bbmap/repair.sh)
  4. Check QC of recovered reads (FastQC)
  5. Aggregate and report QC (MultiQC)

Usage

[!NOTE] If you are new to Nextflow and nf-core, please refer to this page on how to set-up Nextflow. Make sure to test your setup with -profile test before running the workflow on actual data.

First, prepare a samplesheet with your input data that looks as follows:

samplesheet.csv:

csv title="samplesheet.csv" sample,fastq_1,fastq_2 mysampleA,sample_R1.fastq.gz,sample_R2.fastq.gz mysampleB,sample_R3.fq,sample_R4.fq mysampleC,sample_R5.fq.gz

Each row represents a FASTQ file (single-end) or a pair of FASTQ files (paired-end).

[!WARNING] Rows with different file extensions (e.g., mysampleA,sample_R1.fastq.gz,sample_R2.fastq) are not allowed.

Now, you can run the pipeline using:

bash nextflow run nf-core/fastqrepair \ -profile <test/docker/singularity/.../institute> \ --input samplesheet.csv \ --outdir <OUTDIR>

[!WARNING] Please provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.

For more details and further functionality, please refer to the usage documentation and the parameter documentation.

Pipeline output

This pipeline produces clean and well-formed FASTQ files together with short textual reports of the cleaning actions.

To see the results of an example test run with a full size dataset refer to the results tab on the nf-core website pipeline page. For more details about the output files and reports, please refer to the output documentation.

Credits

nf-core/fastqrepair was designed and written by Tommaso Mazza.

Contributions and Support

If you would like to contribute to this pipeline, please see the contributing guidelines.

For further information or help, don't hesitate to get in touch on the Slack #fastqrepair channel (you can join with this invite).

Citations

If you use nf-core/fastqrepair for your analysis, please cite it using the following doi: 10.5281/zenodo.14796348

An extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.

You can cite the nf-core publication as follows:

The nf-core framework for community-curated bioinformatics pipelines.

Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.

Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.

Owner

  • Name: nf-core
  • Login: nf-core
  • Kind: organization
  • Email: core@nf-co.re

A community effort to collect a curated set of analysis pipelines built using Nextflow.

Citation (CITATIONS.md) 

# nf-core/fastqrepair: Citations

## [nf-core](https://pubmed.ncbi.nlm.nih.gov/32055031/)

> Ewels PA, Peltzer A, Fillinger S, Patel H, Alneberg J, Wilm A, Garcia MU, Di Tommaso P, Nahnsen S. The nf-core framework for community-curated bioinformatics pipelines. Nat Biotechnol. 2020 Mar;38(3):276-278. doi: 10.1038/s41587-020-0439-x. PubMed PMID: 32055031.

## [Nextflow](https://pubmed.ncbi.nlm.nih.gov/28398311/)

> Di Tommaso P, Chatzou M, Floden EW, Barja PP, Palumbo E, Notredame C. Nextflow enables reproducible computational workflows. Nat Biotechnol. 2017 Apr 11;35(4):316-319. doi: 10.1038/nbt.3820. PubMed PMID: 28398311.

## Pipeline tools

- [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/)

> Andrews, S. (2010). FastQC: A Quality Control Tool for High Throughput Sequence Data [Online].

- [gzrt](https://www.urbanophile.com/arenn/hacking/gzrt/gzrt.html)

> Renn, A. M. (2013). The gzip Recovery Toolkit [Online].

- [Wipertools](https://github.com/mazzalab/fastqwiper)

> MazzaLab, Tommaso Mazza, & Jose F Oviedo. (2025). mazzalab/fastqwiper: v1.1.5 (v1.1.5). Zenodo. [https://doi.org/10.5281/zenodo.14774039](https://doi.org/10.5281/zenodo.14774039)

- [BBMap](http://sourceforge.net/projects/bbmap/)

> Bushnell B. (2024). BBMap [Online]

- [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)

> Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924.

## Software packaging/containerisation tools

- [Anaconda](https://anaconda.com)

> Anaconda Software Distribution. Computer software. Vers. 2-2.4.0. Anaconda, Nov. 2016. Web.

- [Bioconda](https://pubmed.ncbi.nlm.nih.gov/29967506/)

> Grüning B, Dale R, Sjödin A, Chapman BA, Rowe J, Tomkins-Tinch CH, Valieris R, Köster J; Bioconda Team. Bioconda: sustainable and comprehensive software distribution for the life sciences. Nat Methods. 2018 Jul;15(7):475-476. doi: 10.1038/s41592-018-0046-7. PubMed PMID: 29967506.

- [BioContainers](https://pubmed.ncbi.nlm.nih.gov/28379341/)

> da Veiga Leprevost F, Grüning B, Aflitos SA, Röst HL, Uszkoreit J, Barsnes H, Vaudel M, Moreno P, Gatto L, Weber J, Bai M, Jimenez RC, Sachsenberg T, Pfeuffer J, Alvarez RV, Griss J, Nesvizhskii AI, Perez-Riverol Y. BioContainers: an open-source and community-driven framework for software standardization. Bioinformatics. 2017 Aug 15;33(16):2580-2582. doi: 10.1093/bioinformatics/btx192. PubMed PMID: 28379341; PubMed Central PMCID: PMC5870671.

- [Docker](https://dl.acm.org/doi/10.5555/2600239.2600241)

> Merkel, D. (2014). Docker: lightweight linux containers for consistent development and deployment. Linux Journal, 2014(239), 2. doi: 10.5555/2600239.2600241.

- [Singularity](https://pubmed.ncbi.nlm.nih.gov/28494014/)

> Kurtzer GM, Sochat V, Bauer MW. Singularity: Scientific containers for mobility of compute. PLoS One. 2017 May 11;12(5):e0177459. doi: 10.1371/journal.pone.0177459. eCollection 2017. PubMed PMID: 28494014; PubMed Central PMCID: PMC5426675.

GitHub Events

Total
  • Create event: 10
  • Release event: 1
  • Issues event: 15
  • Watch event: 4
  • Delete event: 6
  • Member event: 1
  • Issue comment event: 22
  • Push event: 78
  • Pull request review comment event: 155
  • Pull request review event: 99
  • Pull request event: 22
  • Fork event: 3
Last Year
  • Create event: 10
  • Release event: 1
  • Issues event: 15
  • Watch event: 4
  • Delete event: 6
  • Member event: 1
  • Issue comment event: 22
  • Push event: 78
  • Pull request review comment event: 155
  • Pull request review event: 99
  • Pull request event: 22
  • Fork event: 3

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 6
  • Total pull requests: 8
  • Average time to close issues: 3 months
  • Average time to close pull requests: 17 days
  • Total issue authors: 3
  • Total pull request authors: 4
  • Average comments per issue: 0.0
  • Average comments per pull request: 1.88
  • Merged pull requests: 5
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 6
  • Pull requests: 8
  • Average time to close issues: 3 months
  • Average time to close pull requests: 17 days
  • Issue authors: 3
  • Pull request authors: 4
  • Average comments per issue: 0.0
  • Average comments per pull request: 1.88
  • Merged pull requests: 5
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • mazzalab (6)
  • jen-reeve (1)
  • irongraft (1)
Pull Request Authors
  • mazzalab (5)
  • nf-core-bot (4)
  • mirpedrol (2)
  • ShimiraOleg (1)
  • lvclark (1)
Top Labels
Issue Labels
enhancement (5) bug (2) documentation (1)
Pull Request Labels

Dependencies

.github/workflows/awsfulltest.yml actions
  • actions/upload-artifact v4 composite
  • seqeralabs/action-tower-launch v2 composite
.github/workflows/awstest.yml actions
  • actions/upload-artifact v4 composite
  • seqeralabs/action-tower-launch v2 composite
.github/workflows/branch.yml actions
  • mshick/add-pr-comment b8f338c590a895d50bcbfa6c5859251edc8952fc composite
.github/workflows/ci.yml actions
  • actions/checkout 0ad4b8fadaa221de15dcec353f45205ec38ea70b composite
  • jlumbroso/free-disk-space 54081f138730dfa15788a46383842cd2f914a1be composite
  • nf-core/setup-nextflow v2 composite
.github/workflows/clean-up.yml actions
  • actions/stale 28ca1036281a5e5922ead5184a1bbf96e5fc984e composite
.github/workflows/download_pipeline.yml actions
  • actions/setup-python 82c7e631bb3cdc910f68e0081d67478d79c6982d composite
  • eWaterCycle/setup-singularity 931d4e31109e875b13309ae1d07c70ca8fbc8537 composite
  • jlumbroso/free-disk-space 54081f138730dfa15788a46383842cd2f914a1be composite
  • nf-core/setup-nextflow v2 composite
.github/workflows/fix-linting.yml actions
  • actions/checkout 0ad4b8fadaa221de15dcec353f45205ec38ea70b composite
  • actions/setup-python 82c7e631bb3cdc910f68e0081d67478d79c6982d composite
  • peter-evans/create-or-update-comment 71345be0265236311c031f5c7866368bd1eff043 composite
.github/workflows/linting.yml actions
  • actions/checkout 0ad4b8fadaa221de15dcec353f45205ec38ea70b composite
  • actions/setup-python 82c7e631bb3cdc910f68e0081d67478d79c6982d composite
  • actions/upload-artifact 65462800fd760344b1a7b4382951275a0abb4808 composite
  • nf-core/setup-nextflow v2 composite
.github/workflows/linting_comment.yml actions
  • dawidd6/action-download-artifact 09f2f74827fd3a8607589e5ad7f9398816f540fe composite
  • marocchino/sticky-pull-request-comment 331f8f5b4215f0445d3c07b4967662a32a2d3e31 composite
.github/workflows/release-announcements.yml actions
  • actions/setup-python 82c7e631bb3cdc910f68e0081d67478d79c6982d composite
  • rzr/fediverse-action master composite
  • zentered/bluesky-post-action 80dbe0a7697de18c15ad22f4619919ceb5ccf597 composite
modules/nf-core/fastqc/meta.yml cpan
subworkflows/nf-core/utils_nextflow_pipeline/meta.yml cpan
subworkflows/nf-core/utils_nfcore_pipeline/meta.yml cpan
.github/workflows/template_version_comment.yml actions
  • actions/checkout 0ad4b8fadaa221de15dcec353f45205ec38ea70b composite
  • mshick/add-pr-comment b8f338c590a895d50bcbfa6c5859251edc8952fc composite
  • nichmor/minimal-read-yaml v0.0.2 composite
modules/nf-core/gzrt/meta.yml cpan
modules/nf-core/trimmomatic/meta.yml cpan
subworkflows/nf-core/utils_nfschema_plugin/meta.yml cpan
modules/local/wipertools/environment.yml pypi
modules/nf-core/fastqc/environment.yml pypi
modules/nf-core/gzrt/environment.yml pypi
modules/nf-core/trimmomatic/environment.yml pypi