treeval - 1.4.0 - Ancient Hippaforalkus

Enhancements & Fixes

• NF-Core template upgrade to 3.3.1.
  • Pipeline now uses NF-Schema for input validation.
  • Pipeline now uses nf-test for local and CI testing.
• Updating the structure of modules and subworkflows to follow new standards.
• Updating Channel names to follow new standards.
• Addition of run_hires flag (boolean) to control use of the hires pretext modules.
• Removal of unused modules such as avgcov, the average coverage module.
• Addition of the mode parameter, to replace the depreciating -entry flag.
  • Mode lists are 'include' lists, listing the processes needed per run.
• Removing entry points has significantly simplified the pipeline; mode now controls a set of steps for pipeline execution.
• Addition of specified reads.
  • Longreads and HiC are now specified in the input yaml file. Details in the usage document.
  • Longread and HiC data can optionally take a fofn (file of file names) where each line contains one file, see YAML_INPUT function fn_get_validated_channel.
  • NOTE: in cases where the same file is in both the fofn and a supplied list, the pipeline will exit with an error which will be detailed in the nextflow.log file.
• Corrected input values.
• SummaryStats Code has been removed, this will be replaced by a plugin in the future!
• Replace 5 modules with GAWK instead of cat | sed pattern modules.
• Simplification of the YAML_INPUT subworkflow.
• Removal of GrabFiles and replacement with the newer .resolve().
• Cleaning up of it -> it[0] into the more correct _meta, file -> file pattern.
• Moved GAWK commands into their own awk script files, this cleans up modules.config.
  • Added my_abs function to the GAWK_REFORMAT_INTERSECT to caluclate difference between start and end.
• Added [TreeVal:Error] and [TreeVal:Info] into intentional print statements to make it easier to see on the CLI and to search for in logs.

New Contributors

• @mahesh-panchal made their first contribution in https://github.com/sanger-tol/treeval/pull/412

Full Changelog: https://github.com/sanger-tol/treeval/compare/1.3.0...1.4.0

- Nextflow
Published by DLBPointon 7 months ago

treeval - 1.3.0 - Ancient Tria

What's Changed

• Selfcomp update by @yumisims in https://github.com/sanger-tol/treeval/pull/379
• Input update by @DLBPointon in https://github.com/sanger-tol/treeval/pull/381
• Optimise memory for large genome by @yumisims in https://github.com/sanger-tol/treeval/pull/392
• Merge dev into main for release by @weaglesBio in https://github.com/sanger-tol/treeval/pull/393
• Update naming of module in config by @DLBPointon in https://github.com/sanger-tol/treeval/pull/394
• Update base.config by @DLBPointon in https://github.com/sanger-tol/treeval/pull/396
• Softmasked telo fix by @DLBPointon in https://github.com/sanger-tol/treeval/pull/398
• Fixed threshold by @yumisims in https://github.com/sanger-tol/treeval/pull/400
• Fix telomere no strong words by @yumisims in https://github.com/sanger-tol/treeval/pull/401
• Pretextgraph container + bam2bed_sort memory fix by @yumisims in https://github.com/sanger-tol/treeval/pull/403
• Changelog update by @DLBPointon in https://github.com/sanger-tol/treeval/pull/404
• 1.3.0 release by @DLBPointon in https://github.com/sanger-tol/treeval/pull/395

Full Changelog: https://github.com/sanger-tol/treeval/compare/1.2.2...1.3.0

- Nextflow
Published by DLBPointon 10 months ago

treeval - 1.2.2 - Ancient Destiny (H2)

What's Changed

• Pretext graph 7 by @yumisims in https://github.com/sanger-tol/treeval/pull/360
• Selfcomp edit by @yumisims in https://github.com/sanger-tol/treeval/pull/358
• Dp24shellto_script by @DLBPointon in https://github.com/sanger-tol/treeval/pull/362
• Update pretext version by @weaglesBio in https://github.com/sanger-tol/treeval/pull/368
• Artifact update by @DLBPointon in https://github.com/sanger-tol/treeval/pull/375
• Dev by @DLBPointon in https://github.com/sanger-tol/treeval/pull/374

Full Changelog: https://github.com/sanger-tol/treeval/compare/1.2.1...1.2.2

- Nextflow
Published by DLBPointon 12 months ago

treeval - 1.2.1 - Ancient Destiny (H1)

What's Changed

• Steps flag spelling by @DLBPointon in https://github.com/sanger-tol/treeval/pull/350
• Update pretext_graph.nf by @DLBPointon in https://github.com/sanger-tol/treeval/pull/351
• Update CHANGELOG.md by @DLBPointon in https://github.com/sanger-tol/treeval/pull/352
• Update branch.yml by @DLBPointon in https://github.com/sanger-tol/treeval/pull/355

Full Changelog: https://github.com/sanger-tol/treeval/compare/1.2.0...1.2.1

- Nextflow
Published by DLBPointon about 1 year ago

treeval - 1.2.0 - Ancient Destiny

[1.2.0] - Ancient Destiny [14/01/2025]

• Added Steps - subworkflows are now split into configurable steps.
• Added a .bin output - this will later be used to create modified pretext maps in downstream processes.
• Added .ktab as an output.
• All modules have been updated to remove any Anaconda dependency e.g. the defaults channel.
• Some module versions have been updated too.
• Replaces PyFasta split with seqtk split - fasta split is severely out of date now.
• CICD has been reverted to near the original template - this was changed, originally, to mitigate an error with nextflow being unable to download the large number of containers TreeVal has.
• MerquryFK Module patch to remove memory arg in module as it is unnessecary.
• Modules updated to ensure that version information is being saved.
• BUSCO module now includes the arg --metaeuk by default.
• Changing GENESET to be more userfriendly.
• Adds a JBrowse Only workflow (this will lead to an update to the FULL workflow which can now call JBROWSE_ONLY and RAPID).
• Removing dp24's liberal use of spaces.
• Resource review.

Full Changelog: https://github.com/sanger-tol/treeval/compare/1.1.1...1.2.0

- Nextflow
Published by DLBPointon about 1 year ago

treeval - v1.1.1 - Ancient Aurora (H1)

[1.1.1] - Ancient Aurora H1 - [2024-07-04]

Our first hotfix for TreeVal, including changing the name of some outputs, correcting some code and fixing an issue with cram files containing multiple RG lines. This will require further fixes in the future, this will include the splitting of the cram file depending on the number of RG lines.

What's Changed

• Reacov bedname by @yumisims in https://github.com/sanger-tol/treeval/pull/295
• Generate csv fix by @yumisims in https://github.com/sanger-tol/treeval/pull/302
• Dp24 hotfix by @DLBPointon in https://github.com/sanger-tol/treeval/pull/298

Full Changelog: https://github.com/sanger-tol/treeval/compare/1.1.0...1.1.1

- Nextflow
Published by DLBPointon over 1 year ago

treeval - v1.1.0 - Ancient Aurora

[1.1.0] - Ancient Aurora - [2024-04-26]

The second release for sanger-tol, created with the nf-core template.

This builds on the initial release by adding subworkflows which generate Kmer-based coverage tracks and a Kmer spectra graph. There are also a number of updates to the logic used throughout the pipeline, as well as to the resources required by a significant number of modules.

What's Changed

• Updates to the resource allocation methods used by a number of modules in the base.config.
• Added a flag to stop the usage of Juicer.
• Subworkflow to generate a kmer-based coverage track.
• Subworkflow to generate/update a kmer spectra graph.
• Subworkflow to use minimap2 for HiC mapping, if selected.
• Subworkflow to use BWAmem2 for HiC mapping, if selected.
• Subworkflow to ingest Pretext accessory files into the Pretext file, simplifying post-TreeVal data manipulation.
• Updated the logic in use throughout the pipeline.
• Updated the modules.config to include some of the logic, cleaning the code.
• Updated the HiC subworkflow to include subsampling the HiC data for Juicer due to resource requirements with large amounts of data.
• Updated the YAML_INPUT subworkflow, this now contains "flags" to change some software options.
• Updated the data names in the input YAML to reduce confusion.
• Updated software (Pretext{View, Snapshot, Graph}) to allow for use on large genomes with big data.
  • Added associated patch files and CPU architecture files.
• Updated the minimap2 align module to remove samtools view in preference of paftools for our use case.
• Updated the test.yml in line with the above changes.
• Updated the SELFCOMP subworkflow to allow for the parallelisation of the work on large genomes.
• Updated the READ_COVERAGE subworkflow to produce the scaffold-based AVG coverage and STND coverage
• Updated Modules from NF-Core - mostly relates to module structure rather than software.
• Updated the SummaryStats output to include HiC container counts.
• Added -T / -t flags where possible to minimise the use of the /tmp directory.
• Replaced CONCAT_MUMMER with CATCAT for simplicity.
• Removed JUICER from the RAPID entry point.
• Removed the CSI or TBI logic. CSI is now used by default, this simplifies the workflow and enlarges the capacity to handle much larger genomes. The logic block previously required was then moved.
• Added NF-DOWNLOAD to the CI-CD due to an error that causes incomplete downloaded when downloading a number of images at the same time.
• Added the RAPID_TOL entry point which is more geared towards the requirements of Sanger.
• Fix a bug in buildalignmentblocks.py to avoid indexing errors happening in large genomes.
• Change output BEDGRAPH from EXTRACT_TELO module.
• Fix to the telomere output.
• Fix to the SelfComp scripts.

New Contributors

• @gq1 made their first contribution in https://github.com/sanger-tol/treeval/pull/186

Contributors

• @DLBPointon
• @yumisims
• @weaglesBio

Full Changelog: https://github.com/sanger-tol/treeval/compare/1.0.0...v1.1.0

- Nextflow
Published by DLBPointon almost 2 years ago

treeval - Ancient Atlantis

[1.0.0] - Ancient Atlantis - [2023-09-27]

Initial release of sanger-tol/treeval, created with the nf-core template.

This pipeline allows for the generation of data needed for the high-quality manual curation of genomic assemblies.

What's Changed

• Removing unnecessary code by @DLBPointon in https://github.com/sanger-tol/treeval/pull/7
• Generate genome by @DLBPointon in https://github.com/sanger-tol/treeval/pull/21
• Digest by @yumisims in https://github.com/sanger-tol/treeval/pull/10
• Input refactor by @DLBPointon in https://github.com/sanger-tol/treeval/pull/26
• Digest fix by @DLBPointon in https://github.com/sanger-tol/treeval/pull/27
• Gene alignment by @DLBPointon in https://github.com/sanger-tol/treeval/pull/24
• Changes to conform to linting by @DLBPointon in https://github.com/sanger-tol/treeval/pull/29
• Changed input static file system by @DLBPointon in https://github.com/sanger-tol/treeval/pull/30
• Self comp merge test by @DLBPointon in https://github.com/sanger-tol/treeval/pull/31
• Reorg merge by @DLBPointon in https://github.com/sanger-tol/treeval/pull/32
• Main synteny branch by @DLBPointon in https://github.com/sanger-tol/treeval/pull/33
• Miniprot p2 by @DLBPointon in https://github.com/sanger-tol/treeval/pull/34
• Issue 44 by @weaglesBio in https://github.com/sanger-tol/treeval/pull/50
• Issue 43 by @weaglesBio in https://github.com/sanger-tol/treeval/pull/49
• Issue 45 by @weaglesBio in https://github.com/sanger-tol/treeval/pull/48
• Second round fixes by @DLBPointon in https://github.com/sanger-tol/treeval/pull/51
• To fix bug #55 by @DLBPointon in https://github.com/sanger-tol/treeval/pull/57
• Create a file object to download it transparently when splitCsv is called by @muffato in https://github.com/sanger-tol/treeval/pull/67
• Hap by @yumisims in https://github.com/sanger-tol/treeval/pull/68
• Outdir correction by @DLBPointon in https://github.com/sanger-tol/treeval/pull/70
• Save file corrections by @DLBPointon in https://github.com/sanger-tol/treeval/pull/72
• amended selfcomp process, shrunk the size/number of intermediate files by @yumisims in https://github.com/sanger-tol/treeval/pull/73
• Minor fixes, update miniprot params and synteny path by @weaglesBio in https://github.com/sanger-tol/treeval/pull/74
• Minor fixes by @DLBPointon in https://github.com/sanger-tol/treeval/pull/78
• Longread coverage workflow by @weaglesBio in https://github.com/sanger-tol/treeval/pull/82
• Repeat density subworkflow by @DLBPointon in https://github.com/sanger-tol/treeval/pull/92
• Local testing by @DLBPointon in https://github.com/sanger-tol/treeval/pull/94
• Anc by @DLBPointon in https://github.com/sanger-tol/treeval/pull/97
• Anc fixed by @DLBPointon in https://github.com/sanger-tol/treeval/pull/100
• Notes for GAP_FINDER by @DLBPointon in https://github.com/sanger-tol/treeval/pull/108
• Hic mapping doc by @yumisims in https://github.com/sanger-tol/treeval/pull/111
• Busco gene by @yumisims in https://github.com/sanger-tol/treeval/pull/118
• Template update by @DLBPointon in https://github.com/sanger-tol/treeval/pull/102
• Fixes comma, image links and explainer by @DLBPointon in https://github.com/sanger-tol/treeval/pull/120
• Fixes - Modules updated to latest by @DLBPointon in https://github.com/sanger-tol/treeval/pull/124
• sanger-tol logo by @muffato in https://github.com/sanger-tol/treeval/pull/131
• fixed findHalfcoverage by @yumisims in https://github.com/sanger-tol/treeval/pull/130
• Pre tag mm49 by @muffato in https://github.com/sanger-tol/treeval/pull/133
• Code suggestions by @muffato in https://github.com/sanger-tol/treeval/pull/140
• Pre tag by @DLBPointon in https://github.com/sanger-tol/treeval/pull/132
• Tower by @DLBPointon in https://github.com/sanger-tol/treeval/pull/142
• Mm49 docs reformat by @DLBPointon in https://github.com/sanger-tol/treeval/pull/149
• Adding pretext ingestion docs by @DLBPointon in https://github.com/sanger-tol/treeval/pull/145
• Fix genome size by @yumisims in https://github.com/sanger-tol/treeval/pull/146
• Refactoring of the test and test_github profiles by @muffato in https://github.com/sanger-tol/treeval/pull/152

New Contributors

• @yumisims made their first contribution in https://github.com/sanger-tol/treeval/pull/10
• @weaglesBio made their first contribution in https://github.com/sanger-tol/treeval/pull/50
• @muffato made their first contribution in https://github.com/sanger-tol/treeval/pull/67

Full Changelog: https://github.com/sanger-tol/treeval/commits/v1.0.0

- Nextflow
Published by DLBPointon over 2 years ago