Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/data-science-in-mechanical-engineering/conditional-test

Code for the paper "A Conditional Two-sample Test"

https://github.com/data-science-in-mechanical-engineering/conditional-test

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (11.0%) to scientific vocabulary
Last synced: 9 months ago · JSON representation

Repository

Code for the paper "A Conditional Two-sample Test"

Basic Info
  • Host: GitHub
  • Owner: Data-Science-in-Mechanical-Engineering
  • Language: Python
  • Default Branch: main
  • Size: 88.9 KB
Statistics
  • Stars: 0
  • Watchers: 2
  • Forks: 1
  • Open Issues: 0
  • Releases: 0
Created over 1 year ago · Last pushed 12 months ago
Metadata Files
Readme

README.md

A kernel conditional two-sample test

The structure of this project is the following:

.
├── figures                     # where figures will be stored (not part of the repo)
├── results                     # where results will be stored (not part of the repo)
├── src                         # Source files
│   ├── experiments             # Main scripts for configuring and running our numerical experiments
│   ├── expyro                  # Utility package for experiment management
│   ├── figures                 # Scripts for generating figures
│   ├── rkhs                    # Core implementation of kernel methods and statistical test
├── requirements.txt            # .txt-file with package specifications
└── README.md

Installation

Create an environment called conditional-test with Python 3.12.6, pull the content from this repo into the environment, and install all needed packages with:

bash cd conditional-test source <path/to/venv>/bin/activate pip install -r requirements.txt

Reproducing numerical results

All of our numerical experiments can be reproduced from the command line. By running the following commands, you can reproduce our results using the configurations used in the paper.

The outcome of every run is saved to ./results from where they can be used to reproduce our figures stored under ./figures.

Illustrative Example

bash python -m src.experiments.example_1d python -m src.figures.example_1d

Empirical error rates

We repeat all experiments on empirical error rates for 100 different random seeds. You can adjust this as needed. ```bash for i in $(seq 0 99); do # for Figure 2 (left and middle) python -m src.experiments.errorrates disturbance --test-name="hu-leihg" --relative-norm=0.05 --seed=$i python -m src.experiments.errorrates disturbance --test-name="hu-leihg" --relative-norm=0.1 --seed=$i python -m src.experiments.error_rates disturbance --test-name="hu-leihg" --relative-norm=0.25 --seed=$i python -m src.experiments.errorrates disturbance --test-name="hu-leihg" --relative-norm=0.5 --seed=$i python -m src.experiments.errorrates disturbance --test-name="hu-lei_hg" --relative-norm=1 --seed=$i python -m src.experiments.errorrates disturbance --test-name="bootstrap" --relative-norm=0.05 --seed=$i python -m src.experiments.errorrates disturbance --test-name="bootstrap" --relative-norm=0.1 --seed=$i python -m src.experiments.errorrates disturbance --test-name="bootstrap" --relative-norm=0.25 --seed=$i python -m src.experiments.errorrates disturbance --test-name="bootstrap" --relative-norm=0.5 --seed=$i python -m src.experiments.errorrates disturbance --test-name="bootstrap" --relative-norm=1 --seed=$i

# for Figure 2 (right)
python -m src.experiments.error_rates local-disturbance --test-name="hu-lei__gt" --relative-norm=1 --weight=0.005 --tolerance=0.01 --seed=$i
python -m src.experiments.error_rates local-disturbance --test-name="hu-lei__gt" --relative-norm=1 --weight=0.01 --tolerance=0.01 --seed=$i
python -m src.experiments.error_rates local-disturbance --test-name="hu-lei__gt" --relative-norm=1 --weight=0.02 --tolerance=0.01 --seed=$i
python -m src.experiments.error_rates local-disturbance --test-name="hu-lei__gt" --relative-norm=1 --weight=0.03 --tolerance=0.01 --seed=$i
python -m src.experiments.error_rates local-disturbance --test-name="hu-lei__gt" --relative-norm=1 --weight=0.04 --tolerance=0.01 --seed=$i
python -m src.experiments.error_rates local-disturbance --test-name="bootstrap" --relative-norm=1 --weight=0.005 --tolerance=0.01 --seed=$i
python -m src.experiments.error_rates local-disturbance --test-name="bootstrap" --relative-norm=1 --weight=0.01 --tolerance=0.01 --seed=$i
python -m src.experiments.error_rates local-disturbance --test-name="bootstrap" --relative-norm=1 --weight=0.02 --tolerance=0.01 --seed=$i
python -m src.experiments.error_rates local-disturbance --test-name="bootstrap" --relative-norm=1 --weight=0.03 --tolerance=0.01 --seed=$i
python -m src.experiments.error_rates local-disturbance --test-name="bootstrap" --relative-norm=1 --weight=0.04 --tolerance=0.01 --seed=$i

# for Figure 3
python -m src.experiments.error_rates mixture-noise --kernel-type="gaussian" --noise-mean=0.05 --seed=$i
python -m src.experiments.error_rates mixture-noise --kernel-type="gaussian" --noise-mean=0.075 --seed=$i
python -m src.experiments.error_rates mixture-noise --kernel-type="gaussian" --noise-mean=0.1 --seed=$i
python -m src.experiments.error_rates mixture-noise --kernel-type="linear" --noise-mean=0.05 --seed=$i
python -m src.experiments.error_rates mixture-noise --kernel-type="linear" --noise-mean=0.075 --seed=$i
python -m src.experiments.error_rates mixture-noise --kernel-type="linear" --noise-mean=0.1 --seed=$i

# for Figure 4
python -m src.experiments.error_rates output-kernel --kernel-type="gaussian" --kernel-parameter=0.05 --seed=$i
python -m src.experiments.error_rates output-kernel --kernel-type="gaussian" --kernel-parameter=0.1 --seed=$i
python -m src.experiments.error_rates output-kernel --kernel-type="gaussian" --kernel-parameter=0.15 --seed=$i
python -m src.experiments.error_rates output-kernel --kernel-type="polynomial" --kernel-parameter=1 --seed=$i
python -m src.experiments.error_rates output-kernel --kernel-type="polynomial" --kernel-parameter=2 --seed=$i
python -m src.experiments.error_rates output-kernel --kernel-type="polynomial" --kernel-parameter=3 --seed=$i

# for Figure 6
python -m src.experiments.error_rates dataset-size --test-name="bootstrap" --size=20 --seed=$i
python -m src.experiments.error_rates dataset-size --test-name="bootstrap" --size=50 --seed=$i
python -m src.experiments.error_rates dataset-size --test-name="bootstrap" --size=100 --seed=$i
python -m src.experiments.error_rates dataset-size --test-name="bootstrap" --size=250 --seed=$i

done ```

You can create the corresponding figures (selectively) from the following commands.

bash python -m src.figures.error_rates --figure=2 python -m src.figures.error_rates --figure=3 python -m src.figures.error_rates --figure=4 python -m src.figures.error_rates --figure=6

System monitoring

We repeat our system monitoring tutorial experiment for 50 different random seeds. You can adjust this as needed.

```bash for i in $(seq 0 49); do python -m src.experiments.monitoring --dimension=2 --disturbance=0.1 --seed=$i python -m src.experiments.monitoring --dimension=4 --disturbance=0.1 --seed=$i python -m src.experiments.monitoring --dimension=8 --disturbance=0.1 --seed=$i python -m src.experiments.monitoring --dimension=16 --disturbance=0.1 --seed=$i python -m src.experiments.monitoring --dimension=16 --disturbance=0.5 --seed=$i python -m src.experiments.monitoring --dimension=16 --disturbance=0.75 --seed=$i done

python -m src.figures.monitoring ```

Owner

  • Name: Data Science in Mechanical Engineering (DSME)
  • Login: Data-Science-in-Mechanical-Engineering
  • Kind: organization
  • Location: Aachen, Germany

Public code repository of the Institute for Data Science in Mechanical Engineering at the RWTH Aachen University

GitHub Events

Total
  • Watch event: 2
  • Member event: 2
  • Push event: 5
  • Fork event: 2
  • Create event: 1
Last Year
  • Watch event: 2
  • Member event: 2
  • Push event: 5
  • Fork event: 2
  • Create event: 1

Dependencies

requirements.txt pypi
  • Pygments ==2.19.1
  • absl-py ==2.1.0
  • cloudpickle ==3.1.0
  • contourpy ==1.3.1
  • cycler ==0.12.1
  • decorator ==5.1.1
  • dm-tree ==0.1.8
  • docstring_parser ==0.16
  • fonttools ==4.55.3
  • gast ==0.6.0
  • jax ==0.4.38
  • jaxlib ==0.4.38
  • kiwisolver ==1.4.8
  • markdown-it-py ==3.0.0
  • matplotlib ==3.10.0
  • mdurl ==0.1.2
  • ml_dtypes ==0.5.0
  • numpy ==2.2.1
  • opt_einsum ==3.4.0
  • packaging ==24.2
  • pandas ==2.2.3
  • pillow ==11.1.0
  • pyparsing ==3.2.1
  • python-dateutil ==2.9.0.post0
  • pytz ==2024.2
  • rich ==13.9.4
  • scipy ==1.15.0
  • seaborn ==0.13.2
  • shtab ==1.7.1
  • six ==1.17.0
  • tensorflow-probability ==0.25.0
  • tqdm ==4.67.1
  • typeguard ==4.4.1
  • typing_extensions ==4.12.2
  • tyro ==0.9.13
  • tzdata ==2024.2