NF-CaseReports (Nextflow - Case Reports Pipeline)

Introduction

mskilab-org/nf-casereports is a bioinformatics pipeline from mskilab-org for running JaBbA, our algorithm for MIP based joint inference of copy number and rearrangement state in cancer whole genome sequence data. This pipeline runs all the pre-requisite tools (among others) and generates the necessary inputs for running JaBbA and loading into gOS, our clinical front-end. It is designed to take paired tumor-normal samples or tumor-only samples as input.

Workflow Summary:

1. Align to Reference Genome (currently supports BWA-MEM, BWA-MEM2, and GPU accelerated fq2bam).
2. Quality Control (using FastQC, Picard CollectWGSMetrics, Picard CollectMultipleMetrics, and GATK4 EstimateLibraryComplexity)
3. Mark Duplicates (using GATK MarkDuplicates)
4. Base recalibration (using GATK BaseRecalibrator)
5. Apply BQSR (using GATK ApplyBQSR)
6. Perform structural variant calling (using GRIDSS)
7. Perform pileups (using AMBER)
8. Generate raw coverages and correct for GC & Mappability bias (using fragCounter)
9. Remove biological and technical noise from coverage data. (using Dryclean)
10. Perform segmentation using tumor/normal ratios of corrected read counts, (using the CBS (circular binary segmentation) algorithm)
11. Purity & ploidy estimation (using PURPLE)
12. Junction copy number estimation and event calling (using JaBbA
13. Call SNVs and indels (using SAGE)
14. Annotate variants (using Snpeff)
15. Assign mutational signatures (using SigProfiler)
16. Detect HRD (Homologous Recombination Deficiency) (using HRDetect)

Usage

Note If you are new to Nextflow and nf-core, please refer to this page on how to set-up Nextflow. Make sure to test your setup with -profile test before running the workflow on actual data.

Setting up the samplesheet.csv file for input:

You need to create a samplesheet with information regarding the samples you want to run the pipeline on. You need to specify the path of your samplesheet using the --input flag to specify the location. Make sure the input samplesheet.csv file is a comma-separated file and contains the headers discussed below. It is highly recommended to provide the *absolute path** for inputs inside the samplesheet rather than relative paths.*

For paired tumor-normal samples, use the same patient ID, but different sample names. Indicate their respective tumor/normal status, where 1 in the status field indicates a tumor sample, and 0 indicates a normal sample. You may pass multiple sample IDs per patient, nf-casereports will consider them as separate samples belonging to the same patient and output the results accordingly.

Specify the desired output root directory using the --outdir flag.

The input samplesheet should look like this:

csv patient,sex,status,sample,lane,fastq_1,fastq_2 TCXX49,XX,0,TCXX49_N,lane_1,/path/to/fastq_1.fq.gz,/path/to/fastq_2.gz

Each row represents a pair of fastq files (paired end) for a single sample (in this case a normal sample, status: 0).

After the input file is ready, you can run the pipeline using our gOSh CLI. For gOSh installation instructions, see https://github.com/mskilab-org/gosh.

The recommended way to run nf-gos is through the gOSh CLI.

With a samplesheet.csv in the run directory, nf-gOS is ready to be run through the gOSh wrapper. bash gosh run pipeline

The first time the pipeline is executed in a run directory, a wizard will prompt the user through several essential configuration parameters (like reference assembly to use, and pipeline presets).

You can also run the pipeline directly using the following command:

bash nextflow run mskilab-org/nf-casereports \ -profile <docker|singularity|institute> \ --input samplesheet.csv \ --outdir <OUTDIR> \ --genome <GATK.GRCh37/GATK.GRCh38>

Warning: Please provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.

JaBbA Pipeline

If you want to run only the JaBbA pipeline, you can use the following command:

bash gosh run pipeline --preset jabba

HR Deficiency (HRD) Pipeline

If you want to run only the HRD pipeline, you can use the following command:

bash gosh run pipeline --preset hrd

Discussion of expected fields in input file and expected inputs for each --step

A typical sample sheet can populate with all or some of the column names as shown below. The pipeline will use the information provided in the samplesheet to parsimoniously run the steps of the pipeline to generate all remaining outputs.

N.B You do not need to supply all the columns in the table below. The table represents all the possible inputs that can be passed. If you are starting from BAMs just pass bam and bai columns. If you are starting from FASTQs, pass fastq1 (and fastq2 for paired reads). If you have already generated other outputs, you may pass them as well to prevent the pipeline from running tools for which you already have outputs.

| Column Name | Description | |---------------------|----------------------------------------------------------------------------------------------------------------------------------------------| | patient | (required) Patient or Sample ID. This should differentiate each patient/sample. Note: Each patient can have multiple sample names. | | sample | (required) Sample ID for each Patient. Should differentiate between tumor and normal (e.g sample1_t vs. sample1_n). Sample IDs should be unique to Patient IDs | | lane | If starting with FASTQ files, and if there are multiple lanes for each sample for each patient, mention lane name. | | sex | If known, please provide the sex for the patient. For instance if Male type XY, else if Female type XX, otherwise put NA. | | status | (required) This should indicate if your sample is tumor or normal. For normal, write 0, and for tumor, write 1. | | fastq1 | Full path to FASTQ file read 1. The extension should be .fastq.gz or .fq.gz. | | fastq2 | Full path to FASTQ file read 2 (if paired reads). The extension should be .fastq.gz or .fq.gz. | | bam | Full path to BAM file. The extension should be .bam. | | bai | Full path to BAM index file. The extension should be .bam.bai. | | hets | Full path to sites.txt file. | | amberdir | Full path to AMBER output directory. | | fragcov | Full path to the fragCounter coverage file. | | drycleancov | Full path to the Dryclean corrected coverage file. | | ploidy | Ploidies for each sample. | | seg | Full path to the CBS segmented file. | | nseg | Full path to the CBS segmented file for normal samples. | | vcf | Full path to the GRIDSS VCF file. | | vcftbi | Full path to the GRIDSS VCF index file. | | jabbards | Full path to the JaBbA RDS (jabba.simple.rds) file. | | jabbagg | Full path to the JaBbA gGraph (jabba.gg.rds) file. | | nibalancedgg | Full path to the non-integer balanced gGraph (non_integer.balanced.gg.rds) file. | | lpphasedgg | Full path to the LP phased gGraph (lp_phased.balanced.gg.rds) file. | | events | Full path to the events file. | | fusions | Full path to the fusions file. | | snvsomaticvcf | Full path to the somatic SNV VCF file. | | snvsomatictbi | Full path to the somatic SNV VCF index file. | | snvgermlinevcf | Full path to the germline SNV VCF file. | | snvgermlinetbi | Full path to the germline SNV VCF index file. | | variantsomaticann | Full path to the somatic SNV annotated VCF file. | | variantsomaticbcf | Full path to the somatic SNV BCF file. | | variantgermlineann| Full path to the germline SNV annotated VCF file. | | variantgermlinebcf| Full path to the germline SNV BCF file. | | snvmultiplicity | Full path to the SNV multiplicity file. | | sbssignatures | Full path to the SBS signatures file. | | indelsignatures | Full path to the indel signatures file. | | signaturesmatrix | Full path to the signatures matrix file. | | hrdetect | Full path to the HRDetect file. |

Tumor-Only Samples

For tumor-only samples, simply add the flag --tumor_only true to the nextflow command. The pipeline will then run in tumor-only mode.

For more information regarding the pipeline usage and the inputs necesaary for each step, please follow the Usage documentation.

Helpful Core Nextflow Commands:

-resume

If a process of the pipeline fails or is interrupted at some point, Nextflow can resume from that point without having to start over from the beginning. You must specify this in the CLI or on the command-line when restarting a pipeline. You can also supply a run name to resume a specific run using: -resume [run-name]. Use the nextflow log command to show previous run names.

-profile

Use this parameter for choosing a configuration profile. Profiles contain configuration presets for different computing environments.

Several generic profiles have been provided by default which instruct the pipeline to use software packaged using different methods. You can use this option to run the pipeline via containers (singularity/Docker) (highly recommended)

-c

You can mention custom configuration scripts to run the pipeline with using the -c flag and providing a path to the .config file. This is advised when you want to submit processes into an executor like slurm or LSF.

-bg

The Nextflow -bg flag launches the Nextflow pipeline as a background process. This allows you to detach or exit your terminal without interrupting the run. A log of the run will be saved inside a file upon completion. You can also use screen or tmux sessions to persist runs.

Containers:

Every module in the pipeline has been containerized. Some modules are partially modified versions of nf-core/modules, these modules use nf-core containers. Modules that use our lab packages and scripts were containerized into Docker images. These images can be found on our DockerHub.

Warning: JaBbA depends on CPLEX MIP Optimizer to work. Because CPLEX is a proprietary software, it isn't included in the image and needs to be installed by the user. To add CPLEX: 1. Download CPLEX (Linux x86-64). (You may need to use the HTTP method.) 2. Pull image and run the container using: docker pull mskilab/jabba:0.0.1 docker run -it --rm --platform linux/amd64 mskilab/jabba:0.0.1 3. Copy CPLEX binary into the container: docker cp /PATH/TO/DOWNLOADEDCPLEX.bin CONTAINERID:/opt/cplexstudio 4. Install CPLEX: /opt/cplexstudio/DOWNLOADEDCPLEX.bin (If you get a Permission denied error, run chmod 777 /PATH/TO/DOWNLOADEDCPLEX.bin before copying it into the container.) 5. When prompted for an installation path, type /opt/cplex. This is what the CPLEXDIR environmental variable is set to. 6. Save changes to a new image for future use: Exit container (type exit or press Ctrl-D) Run docker commit CONTAINERID NEWIMAGEID

Debugging any step/process:

To debug any step or process that failed, first check your current execution_trace*.txt file inside the <outdir>/pipeline_info/ folder. There you'll find a hash number for that process. You can use that hash number to locate that process's working directory. This directory will contain multiple .command.* files that correspond to your run and contain valuable information that can help you debug your error. You can also run the .command.sh script to do a manual, isolated execution of the offending process for quick testing.

Credits

nf-gos was written by Shihab Dider and Tanubrata Dey and at the Perlmutter Cancer Center and the New York Genome Center.

We thank the following people for their extensive guidance in the development of this pipeline: - Marcin Imielinski - Joel Rosiene

Citations

An extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.

This pipeline uses code and infrastructure developed and maintained by the nf-core community, reused here under the MIT license.

Most large structural variants in cancer genomes can be detected without long reads. Choo, ZN., Behr, J.M., Deshpande, A. et al.

Nat Genet 2023 Nov 09. doi: https://doi.org/10.1038/s41588-023-01540-6

The nf-core framework for community-curated bioinformatics pipelines.

Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.

Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.