macs2 - v3.0.3

Changes for MACS (3.0.3)

Features added

1) Now support FRAG format for single-cell ATAC-seq in callpeak and pileup. FRAG format is used by 10x Genomics to store alignments from the single-cell ATAC-seq pipeline cellranger-atac or the multi-omics pipeline cellranger-arc. The format is essentially BEDPE with two additional columns: the barcode and the count of fragments aligned to the same location with the same barcode. Support for FRAG in other tools is coming soon, as well as for hmmratac calls.

If you specify FRAG as your input format:

• You can use a barcode list for a subset of cells with --barcodes, then callpeak will identify peaks and pileup will build pileup track for the fragments of this subset of cells.
• Duplicates will not get removed as we'll assume all fragments are valid. Optionally, an option, --max-count, can be applied to set the maximum count.

2) We transitioned our pyx codes to py codes, adopting a 'pure Python style' with PEP-484 type annotations. This change has made our source codes more compatible with Python programming tools such as flake8. During this process, we performed further code cleaning and eliminated unnecessary dependencies. We intend to continue improving our code quality in the future.

3) We have modified the handling of 'blacklist' regions in the hmmratac tool. This change impacts both the Expectation-Maximization (EM) step that estimates fragment length distributions, and the Hidden Markov Model (HMM) step that learns and predicts nucleosome states. We now exclude aligned fragments located in the 'blocklist' regions before both steps. We implemented the exclude functions in both PETrackI and PETrackII to support this feature. For more detailed information and the reasoning behind it, refer to issue #680.

4) We have tested Numpy>=2. Now MACS3 can be run on Numpy version 1 and version 2.

Bugs fixed

1) The hmmratagc option --keep-duplicate previously had the opposite effect of what its name and description suggested. Therefore, it was renamed to --remove-dup to more accurately describe the actual behavior. Duplicate fragments will not be removed by hmmratac unless this option is explicitly set up.

2) hmmratac: wrong class name was used while saving digested signals in BedGraph files. Fixed multiple other issues related to output filenames. #682

3) Fix issues in big-endian system in Parser.py codes. Enable big-endian support in BAM.py codes for accessig certain alignment records that overlap with given genomic coordinates using BAM/BAI files.

4) predictd and filterdup: wrong variable name used while reading multiple pe/frag files.

Doc

1) Explanation on the filtering criteria on SAM/BAM/BAMPE files.

PRs

• Feat/macs3/reformat pyproject by @taoliu in https://github.com/macs3-project/MACS/pull/662
• Feat/macs3/python style cython (1st) by @taoliu in https://github.com/macs3-project/MACS/pull/664
• Feat/macs3/fragmentfile by @taoliu in https://github.com/macs3-project/MACS/pull/668
• Expose the "peaks" field in BroadPeakIO by @kaizhang in https://github.com/macs3-project/MACS/pull/678
• FRAG format support and bdg filename type fixed by @taoliu in https://github.com/macs3-project/MACS/pull/685
• Change the way to exclude regions in hmmratac and fix the incorrect keep-duplicate option by @taoliu in https://github.com/macs3-project/MACS/pull/689
• Feat/macs3/fragsupport by @taoliu in https://github.com/macs3-project/MACS/pull/690
• numpy 2 support/prep for macs3.0.3 by @taoliu in https://github.com/macs3-project/MACS/pull/691

New Contributors

• @kaizhang made their first contribution in https://github.com/macs3-project/MACS/pull/678

Full Changelog: https://github.com/macs3-project/MACS/compare/v3.0.2...v3.0.3

- Python
Published by taoliu about 1 year ago

macs2 - MACS3 v3.0.2

What's New

• hmmratac poison emission added. #635
• hmmratac output in narrowPeak format
• cutoff-analysis feature enhanced. #636 #642
• hmmratac memory usage optimized #628 #640
• file format to be more compatible with UCSC browser #653
• Use -O3 instead of -Ofast for compatibility. #637
• Update instruction to install macs3 through conda/bioconda
• Reorganize MACS3 docs and publish through https://macs3-project.github.io/MACS
• Add documents for various file formats used in MACS3.

What's Changed (automatically generated log)

• explictly use Python version 3 by @mr-c in https://github.com/macs3-project/MACS/pull/623
• fix SyntaxWarning by marking a regex as such by @mr-c in https://github.com/macs3-project/MACS/pull/624
• Feat/macs3/output vers libs ga by @taoliu in https://github.com/macs3-project/MACS/pull/625
• Feat/macs3/unittest hmm by @taoliu in https://github.com/macs3-project/MACS/pull/627
• Fix spelling typos found by lintian by @mr-c in https://github.com/macs3-project/MACS/pull/629
• use CC instead of gcc by @bgruening in https://github.com/macs3-project/MACS/pull/630
• Use trusted publishing to upload package to pypi by @taoliu in https://github.com/macs3-project/MACS/pull/632
• Feat/macs3/hmmlearn0.3.2 by @taoliu in https://github.com/macs3-project/MACS/pull/631
• Feat/macs3/hmm poisson by @philippadoherty in https://github.com/macs3-project/MACS/pull/635
• [Feat/macs3/memory consumption] To address the memory usage issue of hmmratac by @taoliu in https://github.com/macs3-project/MACS/pull/640
• Documentation update and refine cutoff-analysis for hmmratac by @taoliu in https://github.com/macs3-project/MACS/pull/642
• Feat/macs3/narrowpeak by @taoliu in https://github.com/macs3-project/MACS/pull/645
• Doc/macs3/3.0.2 for v3.0.2 release by @taoliu in https://github.com/macs3-project/MACS/pull/657

Full Changelog: https://github.com/macs3-project/MACS/compare/v3.0.1...v3.0.2

- Python
Published by taoliu over 1 year ago

macs2 - MACS3 v3.0.1

What's Changed

• fix labels for signal files from hmmratac by @philippadoherty in https://github.com/macs3-project/MACS/pull/611
• remove cython from setup.py requirements by @taoliu in https://github.com/macs3-project/MACS/pull/612
• documentation with Sphinx by @taoliu in https://github.com/macs3-project/MACS/pull/618
• Jaccard Index method to test similarity of outputs by @taoliu in https://github.com/macs3-project/MACS/pull/619
• patch hmmlearn, fix dependencies by @taoliu in https://github.com/macs3-project/MACS/pull/620

Full Changelog: https://github.com/macs3-project/MACS/blob/release/macs3/3.0.1/ChangeLog

- Python
Published by taoliu about 2 years ago

macs2 - MACS3 v3.0.0

First release of MACS3 (v3.0.0). Compared with the previous v3.0.0b3, changes are:

1. Cython 3 support #579
2. BEDPE support in hmmratac #580
3. correct libm support #587
4. removal of the requirement of pip and wheel #589
5. rewritten and reorganized documentation #597

Check other updates in MACS3 so far in previous beta/alpha releases and the ChangeLog file.

- Python
Published by taoliu over 2 years ago

macs2 - v3.0.0 beta 3 release

The third beta version of MACS3, addressing the Cython issue and adding more HMMRATAC options.

• New features from beta2:

1) HMMRATAC module

--modelonly option: only generate HMM model and quit

-t or --training: customized training regions can be provided through this option.

--min-frag-p: exclude fragments with abnormal fragment length while generating four signal tracks. #577 Check macs3 hmmratac -h.

2) testing for Mac OS12 is added.

3) We require Cython 0.29.*. The new Cython3 will break our codes. We will adopt Cython3 later. #574

- Python
Published by taoliu over 2 years ago

macs2 - MACS2 v2.2.9.1

Since Cython has a major upgrade to 3.0, previous MACS2 can't be automatically installed through pip. Now we changed some requirements for dependencies: Python: 3.7 to 3.11, Cython 0.29.*, and Numpy >=0.19. The MACS3 setup.py script has been borrowed into MACS2 codes. Also, we tested MACS2 on Mac OS12 through Github Actions as well.

Thank @jemajet! #569

(overwrite the previous v2.2.9 since I forgot to include some changes to the documentation.)

- Python
Published by taoliu over 2 years ago

macs2 - v3.0.0 beta 2 release

MACS3 beta 2 release includes some improvements on hmmratac command. We've added the cutoff-analysis feature as in callpeak subcommand to hmmratac. It will provide help for users to decide the correct cutoff for training HMM. We also add the memory usage display to the runtime message. Also, we brought back GitHub Action testing for s390x, power64le, and armv7. We will work on rest of the documentation for the next release.

- Python
Published by taoliu over 2 years ago

macs2 - MACS2 v2.2.8

Now support Python 3.6-3.11, and newer numpy and cython, tested on x86 and ARM. Please note that .c files from cython pyx files have been removed from the package. They should be regenerated during installation so Cython is required.

- Python
Published by taoliu almost 3 years ago

macs2 - v3.0.0 beta 1 release

MACS3 beta 1 release includes the new hmmratac command. We rewrote the HMMRATAC (Tarbell 2019) idea in Python. Now all the functions for MACS3 release is ready. We plan to do some optimization and tweaking on hmmratac feature in the next beta release and finish the documentation.

- Python
Published by taoliu over 3 years ago

macs2 - v3.0.0 Alpha 7 release

Alpha 7 provides support for BAI index of BAM file so that we can use the original BAM file instead of a subset BAM file for callvar command. Also, we fixed bugs related to #497 #498 (python 3.10 support) and #501 #502 (missing header line).

- Python
Published by taoliu about 4 years ago

macs2 - v3.0.0 Alpha 6 release

Alpha 6 release fixes an issue during building the paired-peak model #442. Also, now MACS3 uses SIMde v0.7.0. #437

- Python
Published by taoliu about 5 years ago

macs2 - v3.0.0 Alpha 5 release

The fifth alpha release fixed an issue to correctly set CFLAG for older gcc compiler. Also, now it will use simde-no-test instead of simde as sub-module. The setup.py script has been cleaned up.

- Python
Published by taoliu about 5 years ago

macs2 - v3.0.0 Alpha 4 release

The alpha 4 release fixed an issue in 3.0.0a3 where the MANIFEST.in file doesn't include correct source code files, and fixed an issue so that --std=gnu99 cflag will be added to CFLAG while compiling fermi-lite codes.

- Python
Published by taoliu about 5 years ago

macs2 - v3.0.0 Alpha 3 release

The alpha 3 release fixed an issue in 3.0.0a2 where simde was not correctly included while uploading the package to PyPI through github action.

- Python
Published by taoliu about 5 years ago

macs2 - v3.0.0 Alpha 2 release

The second alpha release contains a newly added a callvar subcommand which can be used to call variants in given peak regions directly from ChIP-seq/ATAC-seq bam files with or without control. It also contains optimization in the MACS3.IO.Parser module to make reading gzipped BED files faster (20% less time).

- Python
Published by taoliu about 5 years ago

macs2 - v3.0.0 Alpha 1 release

v3.0.0 Alpha 1 release is to test the new reorganized codes and the new CI through GitHub Action which replaces TravisCI. The package should be uploaded to PyPI with twine.

- Python
Published by taoliu about 5 years ago

macs2 - MACS v2.2.7.1

hotfix

Add 'wheel' and 'pip' to pyproject.toml so that pip install can work.

- Python
Published by taoliu almost 6 years ago

macs2 - MACS v2.2.7 (broken pip pkg)

Bugs fixed

1) MACS2 has been tested on multiple architectures to make sure it can successfully generate consistent results. Currently the supported architectures are: AMD64, ARM64, i386, PPC64LE, and S390X. Thanks to @mr-c, @junaruga, and @tillea! Related to issue

340, #349, #351, and #359; to PR #348, #350, #360, #361, #367,

and #370. The lesson is that if the project is built on Cython and is aimed at memory efficiency, we should specifically define all int/float types in pyx files such as int8t or uint32t using either libc or numpy (c version) instead of relying on Cython types such as short, long, double.

2) MACS2 setup script will check numpy and install numpy if necessary. PR #378, issue #364

3) bdgbroadcall command will correctly add the score column (5th column). The score (5th) column contains 10 times of the average score in the broad region. PR #373, issue #362

4) The missing test on bdgopt subcommand has been added. PR #363

5) The obsolete option --ratio from callpeak subcommand has been removed. PR #369, issue #366

6) Fixed the incorrect description in README on the 'maximum length of broad region is 4 times of d' to 'maximum gap for merging broad regions is 4 times of tag size by default'. PR #380, issue #365.

• Other

1) CODE OF CONDUCT document has been added to MACS2 github repository. PR #358

- Python
Published by taoliu almost 6 years ago

macs2 - MACS v2.2.6

Speed up and other fixes

New Features

1) Speed up MACS2 with some programming tricks and code cleanup.

The `filter_dup` function replaces `separate_dups`. The later one was

implemented for potentially putting back duplicate reads in certain downstream analysis. However such analysis hasn't been implemented. Optimize the speed of writing bedGraph files. Optimize BAM and BAMPE parsing with pointer casting instead of python unpack.

2) The comment lines in the headers of BED or SAM files will be correctly skipped. However, MACS2 won't check comment lines in the middle of the file.

Bugs fixed

1) Cutoff-analysis in callpeak command. #341

2) Issues related to SAMParser and three ELAND Parsers are fixed. #347

Other

1) cmdlinetest script in test/ folder has been updated to: 1. test cutoff-analysis with callpeak cmd; 2. output the two lines before and after the error or warning message during testing; 3. output only the first 10 lines if the difference between test result and standard result can be found; 4. prockreport script monitor CPU time and memory usage in 1 second interval, providing a bit more accuracy.

2) Python3.5 support is removed. Now MACS2 requires Python>=3.6.

3) In the next release, we will remove the support for ELAND and BOWTIE format file, since SAM/BAM and BED format are more standard and flexible.

- Python
Published by taoliu about 6 years ago

macs2 - MACS v2.2.5

Python3 speed up

Features added

1) Github code only and Not included in MACS2 PyPI release New testing data for performance test. An subsampled ENCODE2 CTCF ChIP-seq dataset, including 5million ChIP reads and 5 million control reads, has been included in the test folder for testing CPU and memory usage (i.e. 5M test). Several related scripts , including prockreport for output cpu memory usage, pyprofile and pyprofile_stat for debuging and profiling MACS2 codes, have been included.

2) Speed up pvalue-qvalue checkup (pqtable checkup) #335 #338. The old hashtable.pyx implementation copied from Pandas (very old version) doesn't work well in Python3+Cython. It slows down the pqtable checkup using the identical Cython codes as in v2.1.4. While running 5M test, the __getitem__ function in the hashtable.pyx took 3.5s with 37,382,037 calls in MACS2 v2.1.4, but 148.6s with the same number of calls in MACS2 v2.2.4. As a consequence, the standard python dictionary implementation has replaced hashtable.pyx for pqtable checkup. Now MACS2 runs a bit faster than py2 version, but uses a bit more memory. In general, v2.2.5 can finish 5M reads test in 20% less time than MACS2 v2.1.4, but use 15% more memory.

Bug fixed

1) More Python3 related fixes, e.g. the return value of keys from py3 dict. #333 #337

Note: The Github release package grows bigger since more testing data have been included.

- Python
Published by taoliu over 6 years ago

macs2 - MACS v2.2.4

The first release that is compatible with Python3

Features added

1) First Python3 version MACS2 released.

2) Version number 2.2.X will be used for MACS2 in Python3, in parallel to 2.1.X.

3) More comprehensive test.sh script to check the consistency of results from Python2 version and Python3 version.

4) Simplify setup.py script since the newest version transparently supports cython. And when cython is not installed by the user, setup.py can still compile using only C codes.

5) Fix Signal.pyx to use np.array instead of np.mat.

- Python
Published by taoliu over 6 years ago

macs2 - MACS v2.1.4

MACS version 2.1.4

• Features added

Github Actions is used together with Travis CI for testing and deployment.

• Bugs fixed

PR #322:

1) #318 Random score in bdgdiff output. It turns out the sum_v is not initialized as 0 before adding. Potential bugs are fixed in other functions in ScoreTrack and CallPeakUnit codes.

2) #321 Cython dependency in setup.py script is removed. And place 'cythonzie' call to the correct position.

3) A typo is fixed in Github Actions script.

- Python
Published by taoliu over 6 years ago

macs2 - MACS v2.1.3.3

Redeploy to PyPI with only source code distribution. Plan to figure out how to build wheel for manylinux later.

- Python
Published by taoliu over 6 years ago

macs2 - MACS v2.1.3.2

Retry deployment. Keep only one setup script. Refer to v2.1.3 for ChangLog.

- Python
Published by taoliu over 6 years ago

macs2 - MACS v2.1.3

2019-09-19 Tao Liu vladimir.liu@gmail.com

* Features added

1) Support Docker auto-deploy. PR #309

2) Support Travis CI auto-testing, update unit-testing
scripts, and enable subcommand testing on small datasets.

3) Update README documents. #297 PR #306

4) `cmbreps` supports more than 2 replicates. Merged from PR #304
@Maarten-vd-Sande and PR #307 (our own chi-sq test code)

5) `--d-min` option is added in `callpeak` and `predictd`, to
exclude predictions of fragment size smaller than the given
value. Merged from PR #267 @shouldsee.

6) `--buffer-size` option is added in `predictd`, `filterdup`,
`pileup` and `refinepeak` subcommands. Users can use this option
to decrease memory usage while there are a large number of contigs
in the data. Also, now `callpeak`, `predictd`, `filterdup`,
`pileup` and `refinepeak` will suggest users to tweak
`--buffer-size` while catching a MemoryError. #313 PR #314

* Bugs fixed

1) #265 Fixed a bug where the pseudocount hasn't been applied
while calculating p-value score in ScoreTrack object.

2) Fixed bdgbroadcall so that it will report those broad peaks
without strong peak inside, a consistent behavior as `callpeak
--broad`.

3) Rename COPYING to LICENSE.

- Python
Published by taoliu over 6 years ago

macs2 - MACS v2.1.2.1

Note: this is a retagging of MACS v2.1.2 back on Oct 18, 2018. Due to a wrong upload to PyPI, then the re-uploaded package to PyPI for MACS v2.1.2 has been renamed as MACS2-2.1.2.1.tar.gz.

- Python
Published by taoliu over 6 years ago

macs2 - MACS2 v2.1.2

Please download from PyPI: https://pypi.org/project/MACS2/

- Python
Published by taoliu over 6 years ago

macs2 - MACS2 2.1.0 20150420 rc

Speed up and dispel some bugs. Note: this is ONLY a TAG for 2.1.0 release. The official release can be found at PyPI: https://pypi.python.org/pypi/MACS2/2.1.0.20150420

- Python
Published by taoliu almost 11 years ago

macs2 - MACS2 2.1.0 20140616 rc

Note: this is ONLY a TAG for 2.1.0 release. The official release can be found at PyPI: https://pypi.python.org/pypi/MACS2/2.1.0.20140616

- Python
Published by taoliu over 11 years ago