Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/gperdrizet/ensembleset

Ensemble dataset generator for tabular data prediction and modeling projects.

https://github.com/gperdrizet/ensembleset

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.6%) to scientific vocabulary

Keywords

classification ensemble feature-engineering machine-learning regression scikit-learn
Last synced: 6 months ago · JSON representation

Repository

Ensemble dataset generator for tabular data prediction and modeling projects.

Basic Info
Statistics
  • Stars: 1
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 20
Topics
classification ensemble feature-engineering machine-learning regression scikit-learn
Created 10 months ago · Last pushed 9 months ago
Metadata Files
Readme License

README.md

EnsembleSet

PyPI release Python CIDevcontainer

EnsembleSet generates dataset ensembles by applying a randomized sequence of feature engineering methods to a randomized subset of input features.

1. Installation

Install the pre-release alpha from PyPI with:

bash pip install ensembleset

2. Usage

See the example usage notebook.

Initialize an EnsembleSet class instance, passing in the label name and training DataFrame. Optionally, include a test DataFrame and/or list of any string features and the path where you want EnsembleSet to put data. Then call the make_datasets() to generate an EnsembleSet, specifying:

  1. The number of individual datasets to generate.
  2. The fraction of features to randomly select for each feature engineering step.
  3. The number of feature engineering steps to run.

```python import ensembleset.dataset as ds

dataensemble=ds.DataSet( label='labelcolumnname', # Required traindata=traindf, # Required testdata=testdf, # Optional, defaults to None stringfeatures=['stringfeaturecolumnnames'], # Optional, defaults to None datadirectory='path/to/ensembleset/data' # Optional, defaults to ./data )

dataensemble.makedatasets( ndatasets=10, # Required fractionfeatures=0.1, # Required n_steps=5 # Required ) ```

The above call to make_datasets() will generate 10 different datasets using a random sequence of 5 feature engineering techniques applied to a randomly selected 10% of features. The feature selection is re-calculated after each feature engineering step. Each feature engineering step is applied to the test set if one is provided with a minimum of data leakage (e.g. gaussian KDE is calculated from training data only and then applied to training and testing data).

By default, generated datasets will be saved to HDF5 in data/dataset.h5 using the following structure:

text dataset.h5 ├──train │ ├── labels | ├── 1 | ├── . | ├── . | ├── . | └── n │ └──test ├── labels ├── 1 ├── . ├── . ├── . └── n

3. Feature engineering

The currently implemented pool of feature engineering methods are:

  1. One-hot encoding for string features
  2. Ordinal encoding for string features
  3. Log features with bases 2, e or 10
  4. Ratio features
  5. Exponential features with base 2 or e
  6. Sum features with 2, 3, or 4
  7. Difference features with 2, 3 or 4 subtrahends
  8. Polynomial features with degree 2 or 3
  9. Spline features with degree 2, 3 or 4
  10. Quantized features with using randomly selected k-bins
  11. Smoothed features with gaussian kernel density estimation

Major feature engineering parameters are also randomly selected for each step.

Owner

  • Name: George Perdrizet
  • Login: gperdrizet
  • Kind: user

Chemistry Professor and Springboard student. Learning how to engineer and model data at scale.

GitHub Events

Total
  • Release event: 21
  • Watch event: 1
  • Delete event: 3
  • Push event: 84
  • Pull request event: 17
  • Create event: 21
Last Year
  • Release event: 21
  • Watch event: 1
  • Delete event: 3
  • Push event: 84
  • Pull request event: 17
  • Create event: 21

Committers

Last synced: 8 months ago

All Time
  • Total Commits: 139
  • Total Committers: 1
  • Avg Commits per committer: 139.0
  • Development Distribution Score (DDS): 0.0
Past Year
  • Commits: 139
  • Committers: 1
  • Avg Commits per committer: 139.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
George Perdrizet g****e@p****g 139
Committer Domains (Top 20 + Academic)
perdrizet.org: 1

Issues and Pull Requests

Last synced: 8 months ago

All Time
  • Total issues: 0
  • Total pull requests: 11
  • Average time to close issues: N/A
  • Average time to close pull requests: 13 minutes
  • Total issue authors: 0
  • Total pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 11
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 11
  • Average time to close issues: N/A
  • Average time to close pull requests: 13 minutes
  • Issue authors: 0
  • Pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 11
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
  • gperdrizet (21)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 154 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 19
  • Total maintainers: 1
pypi.org: ensembleset

Ensemble dataset generator for tabular data prediction and modeling projects.

  • Versions: 19
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 154 Last month
Rankings
Dependent packages count: 9.2%
Average: 30.5%
Dependent repos count: 51.8%
Maintainers (1)
gperdrizet
Last synced: 6 months ago

Dependencies

pyproject.toml pypi
  • h5py *
  • pandas *
requirements.txt pypi
  • build *
  • h5py ==3.13.0
  • hatchling *
  • pandas *
  • twine *
.github/workflows/publish_pypi.yml actions
  • actions/checkout v4 composite
  • actions/download-artifact v4 composite
  • actions/setup-python v5 composite
  • actions/upload-artifact v4 composite
  • pypa/gh-action-pypi-publish release/v1 composite
.github/workflows/python_ci.yml actions
  • actions/checkout v4 composite
  • actions/setup-python v3 composite
build_requirements.txt pypi
  • build *
  • coverage *
  • flake8 *
  • h5py *
  • hatchling *
  • pandas *
  • scikit-learn *
  • twine *