xpresspipe - XPRESSpipe-v0.6.4

What's Changed

• Update test cases
• Add frozen requirements yml to aid in installation
• Add more explicit R installs
• Fixed compiled plotting - more graceful errors
• Fixed GTF partitioning for edits for more graceful checks

Full Changelog: https://github.com/XPRESSyourself/XPRESSpipe/compare/v0.6.3...XPRESSpipe-v0.6.4

- Python
Published by j-berg almost 2 years ago

xpresspipe - XPRESSpipe-v0.6.3

Major - New installation method: Instead of installing the software with the pip install . method, a new install script is provided that handles installation of some dependencies better. The new method now only requires the user to run bash install.sh. See the Installation page for an updated walkthrough on this update.

Minor - Updated docs and examples - Removed unused variable in args_dict

- Python
Published by j-berg about 5 years ago

xpresspipe - XPRESSpipe-v0.6.2

• Added --suppress_version_check flag to enable use of XPRESSpipe without internet access
• Added --smoothen flag to any module that uses the geneCoverage sub-module. By default, a sliding window will not be used to smoothen the geneCoverage plots. If provided, a rolling window set at 20 will be used to smoothen the plots.

conda-freeze-v0.6.2.txt r-freeze-v0.6.2.txt

- Python
Published by j-berg over 5 years ago

xpresspipe - XPRESSpipe-v0.6.1

• Added flag during curation steps to allow of UCSC/refseq GTFs during GTF modification steps (truncation, etc.)
  Usage: Provide the --ucsc_format flag to the curateReference or modifyGTF sub-modules. These modifications in format only apply to XPRESSpipe GTF truncation features. Any formatting errors with the GTF file that pertain to alignment, counting, etc. dependencies will need to be addressed by the user.
• Fixed error in XPRESSpipe interface with XPRESSplot’s convert_names function where XPRESSpipe did not read in first column of table as index

freeze_v0.6.1.txt

- Python
Published by j-berg about 6 years ago

xpresspipe - XPRESSpipe-v0.6.0

• Minor modification to instructions on how to install XPRESSpipe and use its conda environment on a supercomputing node.
• Removed version specifications for conda environment setup to ease install (fixes issues in a better way than the solution from v0.5.0)
• Moved required riboWaltz functions to XPRESSpipe as installation has been recurrently problematic

Working conda environment: freeze_v0.6.0.txt

- Python
Published by j-berg about 6 years ago

xpresspipe - XPRESSpipe-v0.5.0

• Fixed issue where genome size calculation would round up and miscalculate genome_size parameter for STAR.
• Added fastp_lite for removal of 3' internal UMIs (generally takes ~1 min per RNA-seq sample with about 30 million reads)
• For example: 5'-read-spacer-UMI-adapter-3'
  • Integrating this addition into options and trimming
• Updated MANIFEST file to reliably copy R and Julia scripts to executing folder for XPRESSpipe
• Updated command builder (xpresspipe build) to include recent additions
• Updated requirements to prevent issue where solved environment required to install R 3.5.1 or greater would create an error where samtools markdup would freeze
• Frequently, R 3.5.1 or greater would create library linking error to stringi, causing GenomicFeatures to not function. Added to RbuildIndex.r to reinstall stringi, which appears to clear up the issue.

Most recent working conda environment dependencies: freeze_v0.5.0.txt

- Python
Published by j-berg about 6 years ago

xpresspipe - XPRESSpipe-v0.4.4

Fixed critical error with string catenation during fastp call with UMIs (see #38).

- Python
Published by j-berg over 6 years ago

xpresspipe - XPRESSpipe-v0.4.3

• Fixed issue with metagene where parallelization overloaded memory and resulted in OOM errors. Fixed by making memory thresholding slightly more strict. See #37

- Python
Published by j-berg over 6 years ago

xpresspipe - XPRESSpipe-v0.4.2

• Slight modification to convert_names module

- Python
Published by j-berg over 6 years ago

xpresspipe - XPRESSpipe-v0.4.1

Minor release - Introduced some restrictions to dependency versions. Some newer versions of dependencies were acting problematic. Will try to figure out how to allow for current versions of these dependencies to be used - Fixed plotting issue with periodicity plots

- Python
Published by j-berg over 6 years ago

xpresspipe - XPRESSpipe-v0.4.0

• Introduced rRNA depletion during alignment step (previously could only do so during the quantification step)
• Expanded periodicity analysis to cover more holistic P-site analysis (report codon usage stats). The submodule previously called periodicity is now called by p_sites
• Allow for setting upper limit threshold for read length during pre-processing reads and P-site analysis (previously only a lower limit was available)
• All documentation associated with this changes has been updated.

- Python
Published by j-berg over 6 years ago

xpresspipe - XPRESSpipe-v0.3.1

• Fix BAM file threshold for metagene and geneCoverage to avoid OOM errors
• Turn off BAM file threshold for counting (low memory footprint, so can use all cores available)
• Import openssl library manually in Rperiodicity -- occasionally had trouble finding the library on its own and would error

- Python
Published by j-berg over 6 years ago

xpresspipe - XPRESSpipe-v0.3.0

• Transfers R dependency installs to Anaconda environment load
• Modified fastq and bam memory factor to optimize resources
• Rebuilt read distribution module with JuliaLang for super memory efficiency during parallelization
• Fixed issue where one-exon genes would not display feature annotations in geneCoverage modules
• Made matplotlib backend calls flexible for HPC usage
• Made directory checks more thorough
• Fixed a potential off-by-one issue with GTF truncator
• Updated appropriate tests
• Updates to documentation
• Added code of conduct and contributions information

- Python
Published by j-berg over 6 years ago

xpresspipe - XPRESSpipe-v0.2.4b0

Updated journal release version

• Fixed issues with using polyX adaptors
• Allowed more multi-threading during post-processing of aligned reads to use resources more efficiently
• Added integrated pipeline tests for Travis CI build to assess pipeline integrity each push
• Updated install walkthrough video

- Python
Published by j-berg almost 7 years ago

xpresspipe - XPRESSpipe-v0.2.3b0

• Implemented minor changes to let code continue to run if hits error (for example, continue QC tests if generic gene name doesn't have a match in organism of interest)
• Updated some tests and docs

- Python
Published by j-berg almost 7 years ago

xpresspipe - XPRESSpipe-v0.2.2b0

Journal Submission Release

Changes to metagene profile setup

• Before there were big spikes due to skewing from super-expressors (i.e., 10x reads as transcripts with next most reads)
• a4b93881f57c1ddbf9cfcb5282794f875492b8af and 289d0565ad0f32af07a853930c5a04c0d0219a7c summarize logic behind methodology

Other changes

• More personable command builder guide
• Updates to documentation
• Looser parallelism threshold for more BAM files at once since it appears to be handling the memory pressure fine right now

- Python
Published by j-berg almost 7 years ago

xpresspipe - XPRESSpipe-v0.2.1b0

• Fixes an issue where gene name search for geneCoverage module was not strict enough and would fit if string was found anywhere in gene name. Now requires a strict match for carry through in the module

- Python
Published by j-berg almost 7 years ago

xpresspipe - XPRESSpipe-v0.2.0b0

bioRXiv release

• doi: https://doi.org/10.1101/704320

Notes:

• Most upcoming changes will be documentation and developing more tests
• readDistribution is RAM hungry and unstable. Removed from pipeline runs for now, but still available as a separate module. Users are encouraged to refer to MultiQC output in the meantime to determine read distributions
• Metagene appears to be stable but will look into the spikes that consistently appear in plots

- Python
Published by j-berg almost 7 years ago

xpresspipe - XPRESSpipe-v0.1.4b2

• Updated modifyGTF to parse for Ensembl canonical transcripts and all riboseq-related modules look for CDS instead of exon (i.e. truncator, periodicity plotting)
• Implemented riboseq pipeline test with checking

- Python
Published by j-berg about 7 years ago

xpresspipe - XPRESSpipe-v0.1.3b2

• Configured setup to install cufflinks before setup execution and to port cufflinks binary to site packages to be accessible during run-time
• Will automatically check for conda, install if not available, and then install conda dependencies

- Python
Published by j-berg about 7 years ago

xpresspipe - XPRESSpipe-v0.1.3b1

• Finished testing on GTF modification and truncation (travis says otherwise, but it should be stable)
• Implemented masking reference creation
• Implemented cufflinks read quantification

- Python
Published by j-berg about 7 years ago

xpresspipe - XPRESSpipe-v0.1.2b1

• Implemented single-pass GTF-guided alignment to quicker alignment. Set as default option, where a two-pass can be used in cases of organisms not characterized genomically
• Updated some conda requirements info

- Python
Published by j-berg about 7 years ago

xpresspipe - XPRESSpipe-v0.1.1b1

Minor fixes to create a stable(ish) release of the pipeline for preliminary testing

- Python
Published by j-berg about 7 years ago

xpresspipe - XPRESSpipe-v0.1.0b1

Beta version for first round user testing Most aspects functional, still waiting on thorough test cases and edge testing Everything involved directly in pipeline is working, but in some cases, running steps alone can be buggy still

Major Updates: - No SAM output, unsorted BAM gets parsed for unique maps, sorted, and indexed - Outputs deduplicated BAM files for downstream processing - Counts to BAM file where duplicates were removed - Implemented complexity via dupRadar and plots summaries - Implemented homebrew metagene and periodicity analysis - Implemented modify GTF features

Several other minor updates

- Python
Published by j-berg about 7 years ago

xpresspipe - XPRESSpipe-v0.0.1b1

Most aspects have been preliminarily tested Waiting to build better codecov that will be python specific and not just bash scripted Need tests for rRNA prober -- currently not working

- Python
Published by j-berg about 7 years ago