NGLess - Version 1.5.0

The two big changes are:

1. the ability to use Yaml files to specify samples,
2. the introduction of run_for_all (and run_for_all_samples) functions to simplify the usage of the parallel module.

Several of the other changes were then to support these two features. Additionally, some minor fixes and improvements were made.

Full ChangeLog:

• Add load_sample_list function to load samples in YAML format.
• Add compress_level argument to write function to specify the compression level.

• Added name() method to ReadSet objects, so you can do:

  input = loadfastqdirectory("my-sample") print(input.name())

which will print my-sample. - Added println function which works like print but prints a newline after the output. - Make print() accept ints and doubles as well as strings. - Added run_for_all function to parallel module, simplifying its API. - When using the parallel module and a job fails, writes the log to the corresponding .failed file. - External modules can now use the sequenceset type to represent a FASTA file. - The load_fastq_directory function now supports .xz compressed files. - The parallel module now checks for stale locks before re-trying failed tasks. The former model could lead to a situation where a particular sample failed deterministically and then blocked progress even when some locks were stale.

Bugfixes

• The parallel module should generate a .failed file for each failed job, but this was not happening in every case.
• Fixed parsing of GFF files to support negative values (reported by Josh Sekela on the mailing-list).

- Haskell
Published by luispedro almost 4 years ago

NGLess - Version 1.4.2

Fix issue parsing GFF with negative score values

- Haskell
Published by luispedro almost 4 years ago

NGLess - Version 1.4.1

Bugfix release

Fixes issue in low memory mapping mode

- Haskell
Published by luispedro about 4 years ago

NGLess - v1.4.0

User-visible Improvements

• write() now returns the filename used
• write() can use multiple threads
• Better error messages in multiple situations
• Add a module for GMGC — Global Microbial Gene Catalogue
• Old motus (version 1) module deprecated
• Update smoothtrim: even window size by @mkuhn in https://github.com/ngless-toolkit/ngless/pull/149

Bugfixes

• Update –install-reference-data mode to newer URLs, see #107
• Update –create-reference-pack mode to newer format (where indices are versioned), see #108
• Do not fail when merging empty files, see #113

Internal improvements

• Better building infrastructure
• Switched to the tasty testing framework
• assemble() is now using a more up to date version of megahit, which means that the older versions cannot be run.

New Contributors

• @mkuhn made their first contribution in https://github.com/ngless-toolkit/ngless/pull/149

What's new docs: http://ngless.embl.de/whatsnew.html#version-1-4-0

Full Changelog: https://github.com/ngless-toolkit/ngless/compare/v1.3.0...v1.4.0

- Haskell
Published by luispedro about 4 years ago

NGLess - Version 1.4.0-beta2

Version 1.4.0 (beta2)

Compared to beta1:

• Do not fail when merging empty files (#113)
• Fix version in install-reference mode (#107)
• Update --create-reference-pack mode (#108)
• Reset terminal colors after error for --download-demo mode
• Add --trace argument to all modes

Full Changelog: https://github.com/ngless-toolkit/ngless/compare/v1.4.0-beta1...v1.4.0-beta2

- Haskell
Published by luispedro about 4 years ago

NGLess - Version 1.4.0-beta

What's Changed

• Update internal version of megahit
• Faster code
• Added module for GMGC
• Build static Linux image with GitHub Actions by @unode in https://github.com/ngless-toolkit/ngless/pull/146
• update smoothtrim: even window size by @mkuhn in https://github.com/ngless-toolkit/ngless/pull/149

New Contributors

• @mkuhn made their first contribution in https://github.com/ngless-toolkit/ngless/pull/149

Full Changelog: https://github.com/ngless-toolkit/ngless/compare/v1.3.0...v1.4.0-beta1

- Haskell
Published by luispedro about 4 years ago

NGLess - Version 1.3.0

An accumulation of small improvements rather than a big new feature:

• Validate count() headers on --validate-only
• Better error message if the user attempts to use the non-existent <\> operator (suggest </>)
• Add `min-ngless-version field for modules
• Add early check that block assignments are always to block variables
• Use ZStd compression for temporary files from preprocess()
• Correctly handle subpaths in samples for collect (fixes #141)
• Add to_string() to int and double types (partially fixes #78 & fixes #81)
• Add read_int() and read_double() functions (fixes #78)

- Haskell
Published by luispedro over 5 years ago

NGLess - Release 1.2.0

Big changes

• Add load_fastq_directory to builtin functions
• Enable specifying all module resources by URL with download on first use

Other

• Better messages when using lock1
• No longer ship JS libraries (also do not expand them in-place)
• Remove old motus/specI moduels
• Fix CIGAR reinjection bug (already fixed in v`1.1.1)

- Haskell
Published by luispedro almost 6 years ago

NGLess - Version 1.1.1

Bugfix release, fixing a sequence reinjection bug (when filtering MappedRead objects using select)

It should introduce no changes compared to v1.1.0.

- Haskell
Published by luispedro about 6 years ago

NGLess - Version 1.1.0

Many changes to the language and the internal code warrant a new release:

• Fix CIGAR interpretation (#109) occurring when I is present
• Call bwa mem so that it behaves in a deterministic way (independently of the number of threads used)
• Add include_fragments option to orf_find()
• Add early check for column headers in count()
• Add sense argument to count()
• Add line numbers to FastQ parsing errors
• Fix __extra_args argument in map()
• Add discard_singles function
• Add interleaved option to fastq()
• load_mocat_sample now fails if pair.2 exists but pair.1 doesn't
• Reintroduce zstd compression (after fixes upstream)

- Haskell
Published by luispedro over 6 years ago

NGLess - Release 1.0.1

This is a bugfix release for version 1.0: upgrading is recommended and no changes in behaviour are expected (except in situations where NGLess used to fail and will now produce the correct result).

- Haskell
Published by luispedro almost 7 years ago

NGLess - Release 1.0.0

User-visible improvements

The handling of multiple annotations in count (i.e., when the user requests multiple features and/or subfeatures) has changed. The previous model caused a few issues (#63, but also mixing with collect(). Unfortunately, this means that scripts asking for the old behaviour in their version declaration are no longer supported if they use multiple features.

- Haskell
Published by luispedro about 7 years ago

NGLess - Release 0.11.1

Release 0.11.1 fixes issues in 0.11.0 (which should not be used).

Release 0.11

User-visible improvements since version 0.10

• Module samtools (version 0.1) now includes samtools_view
• Add --verbose flag to check-install mode (ngless --check-install --verbose)
• Add early checks for input files in more situations (#33)
• Support compression in collect() output (#42)
• Add smoothtrim() function

Bugfixes since version 0.10

• Fix bug with orf_find & prots_out argument
• Fix bug in garbage collection where intermediate files were often left on disk for far longer than necessary.
• Fix CIGAR (#92) for select() blocks

Internal improvements since version 0.10

• Switched to diagrams package for plotting. This should make building easier as cairo was often a complicated dependency.
• Update to LTS-13 (GHC 8.6)
• Update minimap2 version to 2.14
• Call bwa/minimap2 with interleaved fastq files. This avoids calling it twice (which would mean that the indices were read twice).
• Avoid leaving open file descriptors after FastQ encoding detection
• Tar extraction uses much less memory now (#77)

- Haskell
Published by luispedro over 7 years ago

NGLess - NGLess 0.11.0

Release 0.11.0

Please use version 0.11.1 as this release introduced an important bug due to the unreliability of zstandard compression (as it manifests itself only in a small subset of inputs, it it hard to test for). The bug manifests itself in ngless exiting with an internal error. Thus, if you got results with v0.11.0, they can be trusted.

A collection of several bugfixes and performance improvements over the last few months.

User-visible improvements

• Module samtools (version 0.1) now includes samtools_view
• Add --verbose flag to check-install mode (ngless --check-install --verbose)
• Add early checks for input files in more situations (#33)
• Support compression in collect() output (#42)
• Add smoothtrim() function

Bugfixes

• Fix bug with orf_find & prots_out argument
• Fix bug in garbage collection where intermediate files were often left on disk for far longer than necessary.
• Fix CIGAR (#92) for select() blocks

Internal improvements

• Switched to diagrams package for plotting. This should make building easier as cairo was often a complicated dependency.
• Update to LTS-13 (GHC 8.6)
• Update minimap2 version to 2.14
• Call bwa/minimap2 with interleaved fastq files. This avoids calling it twice (which would mean that the indices were read twice).
• Avoid leaving open file descriptors after FastQ encoding detection
• Compress intermediate SAM files. In some situations this should even speed up the process as Zstandard compression is used and the process was often IO bound (#22)
• Tar extraction uses much less memory now (#77)

- Haskell
Published by luispedro over 7 years ago

NGLess - Release 0.9.1

Adds biorxiv preprint link.

- Haskell
Published by luispedro almost 8 years ago

NGLess - 0.6.1

Bug fixes: - make install works now - index splitting works with --index-path

Full ChangeLog

Version 0.6.1 2017-12-10 by luispedro
    * Add --check-install mode
    * Fix streaming short read QC (performance regression)
    * Better error message in readlines() when file opening fails
    * Compute statistics after select()

- Haskell
Published by luispedro almost 8 years ago

NGLess - Release 0.9.0

Many small improvements:

• Add allbest() method to mappedreads
• Output FastQ quality graphs as PNGs
• Added MouseGutCatalog module
• Added DogGutGeneCatalog module
• Added PigGeneCatalog module
• Added reference genome for Sus scrofa (pig)
• Update IGC module to 0.9
• Continuously update mtime on all lock files
• Warn when overwriting files
• Version automatically downloaded reference URLs

- Haskell
Published by luispedro almost 8 years ago

NGLess - 0.6.0

Big new changes are orf_find(), qcstats() [and generally much better statistics], and the low memory mode.

Full ChangeLog:

Version 0.6.0 2017-11-29 by luispedro
* Add `orf_find` function (implemented through Prodigal)
* Add qcstats() function
* Output preprocessed FQ statistics (had been erroneously removed)
* Fix --strict-threads command-line option spelling
* Use multiple threads in SAM->BAM conversion
* Change include_m1 default in count() function to True
* Add --index-path functionality to define where to write indices (issue #47)
* Allow `citations` as key in external modules
* Better citations information
* Better error checking/script validation
* Added reference alias for a more human readable name
* Version embedded megahit binary
* Updated builtin referenced to include latest releases of assemblies
* Fixed inconsistency between reference identifiers and underlying files

See the release notes for more information.

- Haskell
Published by luispedro over 8 years ago

NGLess - 0.5.1

Fixes compilation issues in version 0.5.0 (no functional changes).

- Haskell
Published by luispedro over 8 years ago

NGLess - Release 0.5.0

First tagged release.

This version supports all basic NGLess functionality.

- Haskell
Published by luispedro over 8 years ago

NGLess - Version 0.5

First tagged release.

This version supports all basic NGLess functionality.

- Haskell
Published by luispedro over 8 years ago