gitmc
concurrent, pure Python asyncio + Git: manage large (100+) numbers of Git repos
Science Score: 49.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 3 DOI reference(s) in README -
✓Academic publication links
Links to: zenodo.org -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.5%) to scientific vocabulary
Keywords
Repository
concurrent, pure Python asyncio + Git: manage large (100+) numbers of Git repos
Basic Info
Statistics
- Stars: 11
- Watchers: 2
- Forks: 3
- Open Issues: 0
- Releases: 39
Topics
Metadata Files
README.md
GitMC -- concurrent asynchronous Git Utilities for operations on massive numbers of Git repos
Platform-independent (Linux/Mac/Windows) Git utilities, useful for managing large (100+) numbers of Git repos. Speed is an emphasis throughout, with concurrency via Python stdlib asyncio via asyncio.createsubprocessexec and pipelining makes operations in effect 100x faster overall as the coroutines simultaneously wait for Git operations (particularly remote operations like "fetch" and "pull"). We have implemented individual concurrent subprocess timeout using asyncio.wait_for so that one Git operation hanging doesn't cause other Git operations to fail--this is good for when a Git login popup may go unnoticed by the human.
GitMC uses command-line Git because PyGit also requires command-line Git installed, and we don't need the advanced functionality.
Also see PyGit-bulk for managing large (100+) numbers of users / teams.
This repo contains a Git pre-commit script with explanation.
Install
Install Git in a way accessible from the command line line
- Mac:
brew install git - Linux:
apt install git - Windows: command line Git.
sh
python -m pip install -e .
Usage
gitbranch
: Tells of any non-master branches under directory ~/code
python -m gitutils.git_email
: list all contributor email addresses. To fix unwanted emails use Git-filter-repo
find_missing_file
: find directories missing exact fullpath to file
find_matching_file
: find directories matching exact fullpath to file
Sync large number of git repos
These assume numerous subdirectories under ~/code.
They work very quickly for large numbers (100+) of repos.
gitstatcheck if any local repos have pending changesgitpullGit pulls all repos (suggestgitfetchinstead)gitfetchGit fetches all repos, printing a summary of files changed on remote
Place an empty file .nogit in a subdirectory to skip it.
[optional] speedup with https pull
For public repos, to make the Git remote checking go at least twice as fast, and significantly reduce the computational burden when SSH is used for git push (as is recommended), consider the "pushInsteadOf" global Git config.
To do this, when cloning a public repo (including ones you're a collaborator on), use git clone https://.
This global SSH push config one-time does SSH push for HTTPS-cloned repos:
sh
git config --global url."ssh://github.com/".pushInsteadOf https://github.com/
The pattern matching can be made for all sites by omitting github.com from the command above, or you can refine it for each site, or even for specific usernames by editing the command above.
For private repos, simply clone with SSH as usual
Preview all changed Jekyll files
This is for a website made using Jekyll or Hugo:
sh
ActOnChanged . -p
It shows web page previews of all pages changed locally--start the Jekyll or Hugo debug server first e.g. hugo serve
Owner
- Login: scivision
- Kind: user
- Repositories: 353
- Profile: https://github.com/scivision
CodeMeta (codemeta.json)
{
"@context": "https://doi.org/10.5063/schema/codemeta-2.0",
"@type": "SoftwareSourceCode",
"license": "https://spdx.org/licenses/MIT",
"codeRepository": "https://github.com/scivision/gitmc",
"contIntegration": "https://github.com/scivision/gitmc/actions",
"dateCreated": "2016-07-21",
"issueTracker": "https://github.com/scivision/gitmc/issues",
"name": "GitMC",
"identifier": "10.5281/zenodo.3885497",
"description": "concurrent, pure Python asyncio + Git: manage large (100+) numbers of Git repos",
"applicationCategory": "git",
"developmentStatus": "active",
"keywords": [
"git"
],
"programmingLanguage": [
"Python"
],
"author": [
{
"@type": "Person",
"@id": "https://orcid.org/0000-0002-1637-6526",
"givenName": "Michael",
"familyName": "Hirsch"
}
]
}GitHub Events
Total
Last Year
Committers
Last synced: 11 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| Michael Hirsch, Ph.D | s****n | 268 |
| scivision | s****n@n****m | 7 |
| John Vandenberg | j****b@g****m | 1 |
| thin | u****m | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 9 months ago
All Time
- Total issues: 0
- Total pull requests: 1
- Average time to close issues: N/A
- Average time to close pull requests: 2 days
- Total issue authors: 0
- Total pull request authors: 1
- Average comments per issue: 0
- Average comments per pull request: 0.0
- Merged pull requests: 1
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
Pull Request Authors
- jayvdb (1)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- actions/checkout v3 composite
- actions/setup-python v4 composite
- actions/checkout v4 composite
- actions/setup-python v5 composite
- pypa/gh-action-pypi-publish release/v1 composite
- pygit2 *