https://github.com/cancerit/cgppindel - 3.9.0 - Masking invalid complex INDEL calls

3.9.0

• Masking implemented to remove calls from pindel output where 'ref == alt'. These complex indel calls are invalid and were not present prior to 3.6.0 changes
  • Core pindel fix required to resolve the underlying problem

- Perl
Published by ThomasSClarke over 4 years ago

https://github.com/cancerit/cgppindel - 3.8.0 - New normal panel flag method (optional)

• Adds -noflag option to pindel.pl for use cases where flagging needs to be executed separately or not at all.
• Add -range option to pindel_np_from_vcf.pl
  • Creates bed.gz using pindel call range instead of legacy gff3.gz
• Adds new flag FF021, normal panel filtering using pindel call range
  • Legacy normal panel filtering remove variants with any overlap with normal panel, looses larger events that we have more confidence in with longer reads.
  • This is not included by default in any flag set
  • A different normal panel needs to be constructed to use this (bed.gz)
• Nextflow DSL2 worflows
  • pindel_pl - wraps pindel.pl, subworkflow included for later use.
  • np_generation - Generate a normal panel from a list of input BAMs
  • Handles the complete data generation and conversion to bed/gff3.

- Perl
Published by keiranmraine over 4 years ago

https://github.com/cancerit/cgppindel - 3.6.0 - new FFPE flags

New flag rule set pulldownFfpeRulesFragment.lst including FF019 and FF020 made

- Perl
Published by ThomasSClarke over 4 years ago

https://github.com/cancerit/cgppindel - v3.5.0 - change to pindel core

Fix in underlying pindel algorithm to allow complex events with inserted sequence longer than the deleted element.

Causes a minor shuffle DI events which are known to be subjective.

- Perl
Published by keiranmraine about 5 years ago

https://github.com/cancerit/cgppindel - v3.3.0 - IO hardening

See milestone 3

- Perl
Published by keiranmraine almost 7 years ago

https://github.com/cancerit/cgppindel - v3.2.0 - speedup

The input step (which runs pindel_input_gen.pl) is now significantly faster (~50% shorter run time).

There is no scientific impact from this change.

- Perl
Published by keiranmraine about 7 years ago

https://github.com/cancerit/cgppindel - v3.1.2 - Species name command line fix

Resolves bug introduced in 3.0.5

- Perl
Published by keiranmraine over 7 years ago

https://github.com/cancerit/cgppindel - v3.1.1 - Processing for mmQC processed BAMs

• Stable release for working with BAMs passed through mismatch QC
  • Works with data not handled in this way also
• Fix regression in 3.x branch affecting results with no events on individual chromosomes.

- Perl
Published by keiranmraine over 7 years ago

https://github.com/cancerit/cgppindel - Hotfix v3.0.6

• Handles species names with spaces
• modified checks for species, assembly and checksum

- Perl
Published by sb43 over 7 years ago

https://github.com/cancerit/cgppindel - v3.0.4 - bugfix - affects previous v3.0.x releases

A silent failure in the construction of pindel alignment BAM files was introduced in v3.0.0 resulting in file containing only a header.

This release fixes this bug.

We have updated our testing strategy to ensure this doesn't occur in future.

- Perl
Published by keiranmraine over 7 years ago

https://github.com/cancerit/cgppindel - v3.0.2: fix to example filters

Correct example rule files for *Fragment.lst files to use FF[[:digit:]]+ filter types

- Perl
Published by keiranmraine almost 8 years ago

https://github.com/cancerit/cgppindel - v3.0.1 - tabix call change

3.0.1

• Update tabix calls to directly use query_full (solves GRCh38 contig name issues).

- Perl
Published by over 8 years ago

https://github.com/cancerit/cgppindel - v3.0.0, reduced i/o

3.0.0

• Germline bed file is now merged for adjacent regions (#31)
• Ability to use fragment based counts for filters (#56)
• More compressed intermediate files (#55)
• Change to Const::Fast where appropriate (#41)
• Removed TG VG from genotype.
  • Readgroups are always variable, often 1 in data from last few years
  • Not used by our filters.
• Supports BAM/CRAM inputs
• Output will be aligned with inputs
  • bam vs cram
  • bai vs csi
• Although ground work for csi input/output has been done Bio::DB::HTS doesn't support csi indexed input yet.
  • Created our own fork at cancerit/Bio::DB::HTS so that this could be enabled.
  • You will need to install this manually or use one of our images for this functionallity.
  • dockstore-cgpwxs
  • dockstore-cgpwxs

- Perl
Published by keiranmraine over 8 years ago

https://github.com/cancerit/cgppindel - v2.2.4 - Internal record sorting

Sorting within a records fields cleaned up, primarily ensures filter and info column is consistently ordered.

- Perl
Published by keiranmraine over 8 years ago

https://github.com/cancerit/cgppindel - v2.2.3 - Fix to DI event collation in pindel core

Correct read sorting during collection of DI events. Caused some events to be split into many and others to be missed (Thanks to @liangkaiye for patch)

Testing details:

For passed variants:

Comparing sites in VCF files... Found 277 sites common to both files. Found 0 sites only in main file. Found 0 sites only in second file. Found 0 non-matching overlapping sites. After filtering, kept 277 out of a possible 912840 Sites

For all variants:

Comparing sites in VCF files... Found 907254 sites common to both files. Found 2294 sites only in main file. Found 3698 sites only in second file. Found 3292 non-matching overlapping sites.

If you then investigate the individual classes unfiltered:

Deletions:

$ zgrep -c 'PC=D;' pre-fix/TUMOUR_vs_NORMAL.flagged.vcf.gz postfix/TUMOUR_vs_NORMAL.flagged.vcf.gz pre-fix/TUMOUR_vs_NORMAL.flagged.vcf.gz:463740 postfix/TUMOUR_vs_NORMAL.flagged.vcf.gz:463740

Insertions:

$ zgrep -c 'PC=I;' pre-fix/TUMOUR_vs_NORMAL.flagged.vcf.gz postfix/TUMOUR_vs_NORMAL.flagged.vcf.gz pre-fix/TUMOUR_vs_NORMAL.flagged.vcf.gz:423638 postfix/TUMOUR_vs_NORMAL.flagged.vcf.gz:423638

Complex:

$ zgrep -c 'PC=DI;' pre-fix/TUMOUR_vs_NORMAL.flagged.vcf.gz postfix/TUMOUR_vs_NORMAL.flagged.vcf.gz pre-fix/TUMOUR_vs_NORMAL.flagged.vcf.gz:25462 postfix/TUMOUR_vs_NORMAL.flagged.vcf.gz:26866

There is actually overall an increase in total complex events. This is actually not that surprising. The bad sorting of the reads could just as easily prevent an event from reaching the required threshold for reporting.

- Perl
Published by keiranmraine about 9 years ago

https://github.com/cancerit/cgppindel - Handle LD_PRELOAD problem

• Fixed an incorrect loop in file handling for filter_pindel_reads.cpp (#49).
• Moved some messages in cpp code to stderr.

No change to usage or output, release specifically to fix problem encountered during profiling by Ellexus

- Perl
Published by keiranmraine over 9 years ago

https://github.com/cancerit/cgppindel - Reduces the amount of temporary space required and overall I/O

To process 40 million readpairs (40x Tumour + 40x Normal, chr21, 100bp reads):

Original time: User time (seconds): 3553.88 System time (seconds): 63.92 Percent of CPU this job got: 159% Elapsed (wall clock) time (h:mm:ss or m:ss): 37:51.63 File system inputs: 64 File system outputs: 1782080

New time: User time (seconds): 3572.21 System time (seconds): 74.06 Percent of CPU this job got: 167% Elapsed (wall clock) time (h:mm:ss or m:ss): 36:15.01 File system inputs: 0 File system outputs: 1139128

Original peak size: 650MB New peak size: 291MB

~55% reduction in working space and about 40% fewer writes to the file system.

Exactly the same results:

```bash $ diff old/f9c3bc8e-dbc4-1ed0-e040-11ac0d4803a9vsf9c3bc8e-dbc1-1ed0-e040-11ac0d4803a9.germline.bed new/f9c3bc8e-dbc4-1ed0-e040-11ac0d4803a9vsf9c3bc8e-dbc1-1ed0-e040-11ac0d4803a9.germline.bed

$ diffbams -a old/f9c3bc8e-dbc4-1ed0-e040-11ac0d4803a9vsf9c3bc8e-dbc1-1ed0-e040-11ac0d4803a9wt.bam -b new/f9c3bc8e-dbc4-1ed0-e040-11ac0d4803a9vsf9c3bc8e-dbc1-1ed0-e040-11ac0d4803a9_wt.bam Reference sequence count passed Reference sequence order passed Matching records: 194543

$ diffbams -a old/f9c3bc8e-dbc4-1ed0-e040-11ac0d4803a9vsf9c3bc8e-dbc1-1ed0-e040-11ac0d4803a9mt.bam -b new/f9c3bc8e-dbc4-1ed0-e040-11ac0d4803a9vsf9c3bc8e-dbc1-1ed0-e040-11ac0d4803a9_mt.bam Reference sequence count passed Reference sequence order passed Matching records: 239737

$ /software/CGP/canpipe/live/bin/canpipelive vcftools --gzvcf old/f9c3bc8e-dbc4-1ed0-e040-11ac0d4803a9vsf9c3bc8e-dbc1-1ed0-e040-11ac0d4803a9.flagged.vcf.gz --gzdiff new/f9c3bc8e-dbc4-1ed0-e040-11ac0d4803a9vsf9c3bc8e-dbc1-1ed0-e040-11ac0d4803a9.flagged.vcf.gz ... Comparing individuals in VCF files... Ncombinedindividuals: 2 Nindividualscommontobothfiles: 2 Nindividualsuniquetofile1: 0 Nindividualsuniquetofile2: 0 Comparing sites in VCF files... Found 15321 SNPs common to both files. Found 0 SNPs only in main file. Found 0 SNPs only in second file. After filtering, kept 16309 out of a possible 16309 Sites Run Time = 6.00 seconds ```

- Perl
Published by keiranmraine over 9 years ago

https://github.com/cancerit/cgppindel - Legacy v1 support fix: v1.5.7

Pulls this fix back into the legacy 1.x codebase.

- Perl
Published by keiranmraine over 9 years ago

https://github.com/cancerit/cgppindel - v2.0.8 - Bugfix for WXS filter F009

The F009 filter was always passing. Only applied on WXS data.

- Perl
Published by keiranmraine almost 10 years ago

https://github.com/cancerit/cgppindel - v2.0.7: bugfix for newer perl versions

Fixes #46, apparent use of experimental features and/or typos causing issues in perl-5.20

- Perl
Published by keiranmraine about 10 years ago

https://github.com/cancerit/cgppindel -

- Perl
Published by about 10 years ago

https://github.com/cancerit/cgppindel - v2.0.0 - legacy software cleanup

• Migrates all Tabix and Bio::DB::Sam to Bio::DB::HTS::Tabix and Bio::DB::Sam
• Cleans up install

note: you will need PCAP-core v2.0+

- Perl
Published by keiranmraine over 10 years ago

https://github.com/cancerit/cgppindel - v1.5.6 - Added fake line printed to bed file if no germline results found

• Added fake line printed to bed file if no germline results found
• Uses the first contig found and prints contig\t0\t1 as a fake entry to the germline bed file

- Perl
Published by over 10 years ago

https://github.com/cancerit/cgppindel - v1.5.5 - Prevent accidental overwrite of completed work

User will encounter successful job with warning if the logs directory has already been moved to the output location.

- Perl
Published by keiranmraine over 10 years ago

https://github.com/cancerit/cgppindel - v1.5.4 - workaround for bug in underlying code

Workaround fix for sam files with no reads causing failure in bamsort

- Perl
Published by keiranmraine over 10 years ago

https://github.com/cancerit/cgppindel - v1.5.2 - Allow readgroups with ID=0

Fixes #39.

- Perl
Published by keiranmraine almost 11 years ago

https://github.com/cancerit/cgppindel - v1.5.1 - Correct non-absolute paths

Some paths were not being expanded which resulted in failures in later steps (#38).

- Perl
Published by keiranmraine about 11 years ago

https://github.com/cancerit/cgppindel - v1.4.0 - Handle species with large number of contigs

This release primarily handles species where very large numbers of contigs are being analysed. If more than 100 contigs are being processed (after exclusion criteria applied) then a slower merging process is used to ensure reliable processing.

- Perl
Published by keiranmraine about 11 years ago

https://github.com/cancerit/cgppindel - v1.3.2 - correct limit based processing

Number of required threads when using limit was not being set correctly. Generally results in needless thrashing of disk doing file checks but in some cases corrupt data or race conditions were occurring.

- Perl
Published by keiranmraine over 11 years ago

https://github.com/cancerit/cgppindel - v1.3.1 - as v1.3.0 with correct version reported

As per v1.3.0 but version is correct (was reporting 1.2.0).

- Perl
Published by keiranmraine over 11 years ago

https://github.com/cancerit/cgppindel - v1.3.0 - Modified behaviour for species and assembly

Species and assembly default to BAM contents unless not present. If commandline provided and don't match the BAM file a warning is emitted and defaults to BAM contents.

WARNING - reported version not updated, subsequent release corrects

- Perl
Published by over 11 years ago

https://github.com/cancerit/cgppindel - v1.2.0 - add limit option for pindel/pin2vcf steps

Adds the ability to run pindel and pin2vcf steps without defining an '-i' option, allows parallel processing in a more granular way

- Perl
Published by keiranmraine over 11 years ago

https://github.com/cancerit/cgppindel - v1.1.3 - Correct path problem

An error in path manipulation caused paths containing '.' to be handled incorrectly during the flagging phase.

- Perl
Published by keiranmraine over 11 years ago

https://github.com/cancerit/cgppindel - v1.1.2 - Vcf REF column fix

Corrects bug in REF column when the event is a large scale deletion.

Also corrected some errors in tests.

- Perl
Published by keiranmraine over 11 years ago

https://github.com/cancerit/cgppindel - v1.1.1 - minor patch

-c was not being honored when running with -p and -i defined for 'input' step.

- Perl
Published by keiranmraine over 11 years ago

https://github.com/cancerit/cgppindel - v1.1.0 - ready for PanCancer

Added facility to exclude reads anchored in regions of high-depth (or anything) via bed file.

Extended reference implementation to incorporate this, and also added generation of a 'likely' germline bed file from the flagged data.

- Perl
Published by keiranmraine almost 12 years ago

https://github.com/cancerit/cgppindel - v1.0.3 - maintenance release

- Perl
Published by keiranmraine about 12 years ago

https://github.com/cancerit/cgppindel - cgpPindel-v1.0.2

Minor fix to readgroup id parsing

- Perl
Published by keiranmraine about 12 years ago

https://github.com/cancerit/cgppindel - cgpPindel-v1.0.1

Fixes missing executables on install.

- Perl
Published by keiranmraine about 12 years ago

https://github.com/cancerit/cgppindel - cgpPindel-v1.0

- Perl
Published by keiranmraine about 12 years ago