Data

Tools

Indexes

Applications

Experiments

Open Source Science

ml-swe-analysis

Quantify the usage of software engineering best practices in open source code associated with machine learning papers in high-profile conferences and journals.

https://github.com/BonnBytes/ml-swe-analysis

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.7%) to scientific vocabulary

Keywords

reproducibility
Last synced: 10 months ago · JSON representation

Repository

Quantify the usage of software engineering best practices in open source code associated with machine learning papers in high-profile conferences and journals.

Basic Info
  • Host: GitHub
  • Owner: BonnBytes
  • License: eupl-1.2
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 242 KB
Statistics
  • Stars: 2
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Topics
reproducibility
Created over 1 year ago · Last pushed 11 months ago
Metadata Files
Readme Citation

README.md

More Rigorous Software Engineering Would Improve Reproducibility in Machine Learning Research

Source code for our position paper on software engineering in machine learning. An example template repository for most concepts discussed in the paper is available here.

Getting Started

First of all, we have to clone this repository, bash git clone git@github.com:BonnBytes/position_we_need_more_tests_in_ml.git In the next step, you need to configure an environment to use the code in this project. To do that, create a .env-file with the following content.

bash PYTHONPATH=. OPENREVIEW_USERNAME=YOUR_OPENREVIEW_ACCOUNT_NAME OPENREVIEW_PASSWORD=YOUR_PASSWORD

This crawler utilizes the Selenium package, which in turn requires an installed version of the Chrome browser.

Reusability

After cloning and navigating into this repository, you can install the code in this repository via pip.

bash pip install .

Reproduction

To aggregate the statistical data we used for the paper, run the command below.

bash ./run_all.sh

Run the tests

Set up a dotenv with your OpenReview account credentials. Make sure you set the OPENREVIEW_USERNAME and OPENREVIEW_PASSWORD variables are set correctly. To run the tests, type bash nox -s test into the console.

Funding

The Bundesministerium für Bildung und Forschung (BMBF) supported research through its "BNTrAInee" (16DHBK1022) and "WestAI" (01IS22094A) projects. The sole responsibility for the content of the paper and this corresponding code lies with the authors.

Owner

  • Name: BonnBytes
  • Login: BonnBytes
  • Kind: organization
  • Location: Germany

GitHub Events

Total
Last Year

Dependencies

.github/workflows/tests.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
pyproject.toml pypi
  • beautifulsoup4 *
  • openreview-py *
  • pdfx *
  • python-dotenv *
requirements.txt pypi
  • Deprecated ==1.2.15
  • PyJWT ==2.10.1
  • argcomplete ==3.5.3
  • beautifulsoup4 ==4.12.3
  • certifi ==2024.12.14
  • cffi ==1.17.1
  • chardet ==4.0.0
  • charset-normalizer ==3.4.1
  • cli_exit_tools ==1.2.7
  • click ==8.1.8
  • colorlog ==6.9.0
  • contourpy ==1.3.1
  • cryptography ==44.0.0
  • cycler ==0.12.1
  • dill ==0.3.9
  • distlib ==0.3.9
  • filelock ==3.16.1
  • fonttools ==4.55.6
  • future ==1.0.0
  • idna ==3.10
  • jax ==0.5.0
  • jaxlib ==0.5.0
  • kiwisolver ==1.4.8
  • lib-detect-testenv ==2.0.8
  • matplotlib ==3.5.2
  • ml_dtypes ==0.5.1
  • multiprocess ==0.70.17
  • nox ==2024.10.9
  • numpy ==2.2.2
  • openreview-py ==1.46.0
  • opt_einsum ==3.4.0
  • pdfminer.six ==20201018
  • pdfx ==1.4.1
  • pillow ==11.1.0
  • platformdirs ==4.3.6
  • psutil ==6.1.1
  • pycparser ==2.22
  • pycryptodome ==3.21.0
  • pylatexenc ==2.10
  • pyparsing ==3.2.1
  • pypdf ==5.1.0
  • python-dateutil ==2.9.0.post0
  • python-dotenv ==1.0.1
  • requests ==2.32.3
  • scipy ==1.15.1
  • setuptools ==75.8.0
  • six ==1.17.0
  • sortedcontainers ==2.4.0
  • soupsieve ==2.6
  • tikzplotlib ==0.10.1
  • tld ==0.13
  • tqdm ==4.67.1
  • uritools ==4.0.3
  • urlextract ==1.9.0
  • urllib3 ==2.3.0
  • virtualenv ==20.29.1
  • webcolors ==1.12
  • wrapt ==1.17.2
  • wrapt_timeout_decorator ==1.5.1
environment.yml conda
  • _libgcc_mutex 0.1
  • _openmp_mutex 4.5
  • bzip2 1.0.8
  • ca-certificates 2024.12.14
  • colorama 0.4.6
  • exceptiongroup 1.2.2
  • iniconfig 2.0.0
  • ld_impl_linux-64 2.43
  • libexpat 2.6.4
  • libffi 3.4.2
  • libgcc 14.2.0
  • libgcc-ng 14.2.0
  • libgomp 14.2.0
  • liblzma 5.6.3
  • libmpdec 4.0.0
  • libsqlite 3.48.0
  • libuuid 2.38.1
  • libzlib 1.3.1
  • ncurses 6.5
  • openssl 3.4.0
  • packaging 24.2
  • pip 24.3.1
  • pluggy 1.5.0
  • pytest 8.3.4
  • python 3.13.1
  • python_abi 3.13
  • readline 8.2
  • tk 8.6.13
  • tomli 2.2.1
  • tzdata 2025a