mtools

A collection of scripts to set up MongoDB test environments and parse and visualize MongoDB log files.

https://github.com/rueckstiess/mtools

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
○
.zenodo.json file
○
DOI references
○
Academic publication links
✓
Committers with academic emails
2 of 68 committers (2.9%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (12.9%) to scientific vocabulary

Last synced: 10 months ago · JSON representation

Repository

A collection of scripts to set up MongoDB test environments and parse and visualize MongoDB log files.

Basic Info

Host: GitHub
Owner: rueckstiess
License: apache-2.0
Language: Python
Default Branch: develop
Homepage:
Size: 20.5 MB

Statistics

Stars: 1,922
Watchers: 113
Forks: 406
Open Issues: 77
Releases: 23

Created almost 14 years ago · Last pushed about 2 years ago

Metadata Files

Readme Changelog Contributing License

README.rst

======
mtools
======

|PyPI version| |PyPI pyversions| |PyPI license|

**mtools** is a collection of helper scripts to parse, filter, and visualize
MongoDB log files (``mongod``, ``mongos``). mtools also includes ``mlaunch``, a
utility to quickly set up complex MongoDB test environments on a local machine,
and ``mtransfer``, a tool for transferring databases between MongoDB instances.

.. figure:: https://raw.githubusercontent.com/rueckstiess/mtools/develop/mtools.png
   :alt: mtools box

What's in the box?
------------------

The following tools are in the mtools collection:

`mlogfilter `__
   slices log files by time, merges log files, filters slow queries, finds
   table scans, shortens log lines, filters by other attributes, convert to
   JSON

`mloginfo `__
   returns info about log file, like start and end time, version, binary,
   special sections like restarts, connections, distinct view
   (requires ``numpy``)

`mplotqueries `__
   visualize log files with different types of plots (requires ``matplotlib``)

`mlaunch `__
   a script to quickly spin up local test environments, including replica sets
   and sharded systems (requires ``pymongo``, ``psutil``, ``packaging``)

`mtransfer `__
   an experimental script to transfer databases between MongoDB instances by
   copying WiredTiger data files (requires ``pymongo`` and ``wiredtiger``)

For more information, see the `mtools documentation
`__.

Requirements and Installation Instructions
------------------------------------------

The mtools collection is written in Python, and most of the tools only use the
standard packages shipped with Python. The tools are currently tested with
Python 3.8, 3.9, 3.10, and 3.11.

Some of the tools have additional dependencies, which are listed under the
specific tool's section. See the `installation instructions
`__ for more information.

The mtools suite is only tested with
`actively supported `
(non End-of-Life) versions of the MongoDB server. As of September 2023,
that includes MongoDB 4.4 or newer.

Recent Changes
--------------

See `Changes to mtools `__
for a list of changes from previous versions of mtools.

Contribute to mtools
--------------------

If you'd like to contribute to mtools, please read the `contributor page
`__ for instructions.

Disclaimer
----------

This software is not supported by `MongoDB, Inc. `__
under any of their commercial support subscriptions or otherwise. Any usage of
mtools is at your own risk. Bug reports, feature requests and questions can be
posted in the `Issues
`__ section on GitHub.

.. |PyPI version| image:: https://img.shields.io/pypi/v/mtools.svg
   :target: https://pypi.python.org/pypi/mtools/
.. |PyPI pyversions| image:: https://img.shields.io/pypi/pyversions/mtools.svg
   :target: https://pypi.python.org/pypi/mtools/
.. |PyPI license| image:: https://img.shields.io/pypi/l/mtools.svg
   :target: https://pypi.python.org/pypi/mtools/

Owner

Name: Thomas Rückstieß
Login: rueckstiess
Kind: user
Location: Sydney, Australia
Company: MongoDB @mongodb-js

Repositories: 42
Profile: https://github.com/rueckstiess

GitHub Events

Total

Issues event: 2
Watch event: 40
Issue comment event: 1
Pull request event: 1
Fork event: 7

Last Year

Issues event: 2
Watch event: 40
Issue comment event: 1
Pull request event: 1
Fork event: 7

Committers

Last synced: about 1 year ago

All Time

Total Commits: 1,328
Total Committers: 68
Avg Commits per committer: 19.529
Development Distribution Score (DDS): 0.668

Past Year

Commits: 0
Committers: 0
Avg Commits per committer: 0.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
Thomas Rueckstiess	t**s@r**t	441
Thomas Rueckstiess	t**s@1**m	338
Stephen Steneker	s**e@c**g	225
Thomas Rueckstiess	tr@c****l	51
Kevin Adistambha	k**i@g**m	41
garycahill	4****l	21
Gianfranco Palumbo	g**a@g**m	21
Shraya Ramani	s**7@c**u	20
Thomas Rueckstiess	tr@T****l	17
Jim OLeary	j**y@g**m	17
Brian Moss	k**s@g**m	17
Kevin Pulo	k**o@m**m	14
A. Jesse Jiryu Davis	j**e@m**m	10
Thomas Rueckstiess	t**s@m**m	8
Oleg Pudeyev	c**e@o**e	8
mitesh-gosavi	5****i	5
Steve Hand	s**d@m**m	4
Oleg Pudeyev	3****o	4
James Broadhead	j****d	3
Pash10g	p**g@g**m	3
Bernard Gorman	b**n@m**m	3
corymintz	c**z@1**m	2
Steven Silvester	s**r@i**g	2
Savinay Vijay	5****y	2
Max Oberberger	m**x@o**e	2
Jason R. Coombs	j**o@j**m	2
Dave Rolsky	a**h@u**g	2
Aaron Heckmann	a**b@g**m	2
Pash10g	p**y@P**l	2
Norberto	n@n****m	2
and 38 more...

Committer Domains (Top 20 + Academic)

mongodb.com: 11 10gen.com: 4 rueckstiess.net: 1 cpan.org: 1 cornell.edu: 1 olegp.name: 1 ieee.org: 1 oberbergers.de: 1 jaraco.com: 1 urth.org: 1 norbertoleite.com: 1 amsa.info: 1 heymrtough.com: 1 micron.com: 1 allegrogroup.com: 1 peterjs.com: 1 borsuk.sk: 1 xk72.com: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time

Total issues: 30
Total pull requests: 78
Average time to close issues: 3 months
Average time to close pull requests: 5 days
Total issue authors: 15
Total pull request authors: 7
Average comments per issue: 1.03
Average comments per pull request: 0.14
Merged pull requests: 74
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 2
Pull requests: 1
Average time to close issues: 11 minutes
Average time to close pull requests: N/A
Issue authors: 2
Pull request authors: 1
Average comments per issue: 0.5
Average comments per pull request: 0.0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

stennie (14)
FGasper (2)
akung0324 (2)
ivansbishop (1)
Jolexer (1)
p-mongo (1)
dcoupal (1)
nahoj (1)
dunckerr (1)
danielsen870 (1)
rueckstiess (1)
thommy66 (1)
mzraly (1)
autarch (1)
njcosta (1)

Pull Request Authors

stennie (72)
xuclachina (2)
renatoriccio (2)
akung0324 (1)
josh-allan (1)
autarch (1)
echoturing (1)

Top Labels

Issue Labels

improvement (18) bug (7) mlaunch (5) util/other (3) mlogfilter (2) question (2) mplotqueries (2) tests (1) task (1) duplicate (1)

Pull Request Labels

mlaunch (9) util/other (3) task (3) improvement (2) mloginfo (1) docs (1) mtransfer (1) mlogvis (1) tests (1) bug (1)

Packages

Total packages: 3
Total downloads:
- pypi 44,206 last-month
Total docker downloads: 23

Total dependent packages: 0
(may contain duplicates)
Total dependent repositories: 22
(may contain duplicates)
Total versions: 78
Total maintainers: 3

pypi.org: mtools

Useful scripts to parse and visualize MongoDB log files, launch test environments, and reproduce issues.

Homepage: https://github.com/rueckstiess/mtools
Documentation: https://mtools.readthedocs.io/
License: Apache 2.0
Latest release: 1.7.2
published over 3 years ago

Versions: 37
Dependent Packages: 0
Dependent Repositories: 21
Downloads: 40,435 Last month
Docker Downloads: 23

Rankings

Stargazers count: 1.6%

Forks count: 2.7%

Dependent repos count: 3.2%

Average: 4.5%

Downloads: 4.8%

Dependent packages count: 10.0%

Maintainers (2)

rueckstiess stennie

Last synced: 10 months ago

pypi.org: mtools-legacy

Useful scripts to parse and visualize MongoDB log files, launch test environments, and reproduce issues.

Homepage: https://github.com/rueckstiess/mtools
Documentation: https://mtools-legacy.readthedocs.io/
License: Apache 2.0
Latest release: 1.5.7
published about 4 years ago

Versions: 9
Dependent Packages: 0
Dependent Repositories: 1
Downloads: 3,771 Last month

Rankings

Stargazers count: 1.6%

Forks count: 2.7%

Downloads: 3.0%

Average: 7.8%

Dependent packages count: 10.0%

Dependent repos count: 21.7%

Maintainers (1)

olegp

Last synced: 10 months ago

proxy.golang.org: github.com/rueckstiess/mtools

Documentation: https://pkg.go.dev/github.com/rueckstiess/mtools#section-documentation
License: apache-2.0
Latest release: v1.7.2
published over 3 years ago

Versions: 32
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 7.0%

Average: 8.2%

Dependent repos count: 9.3%

Last synced: 10 months ago

Dependencies

doc/requirements.txt pypi

sphinx *
sphinx-rtd-theme *

requirements.txt pypi

matplotlib >=3.5.0
numpy >=1.21.4
ordereddict >=1.1
psutil >=5.8.0
pymongo >=3.12.2,<5.0.0
python-dateutil >=2.8.2

test-requirements.txt pypi

pytest *
pytest-cov *
tox *

.github/workflows/test.yml actions

actions/cache v2 composite
actions/checkout v2 composite
actions/setup-python v2 composite
supercharge/mongodb-github-action 1.7.0 composite

setup.py pypi