https://github.com/abhayspawar/featexp
Feature exploration for supervised learning
Science Score: 33.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
✓DOI references
Found 1 DOI reference(s) in README -
✓Academic publication links
Links to: zenodo.org -
✓Committers with academic emails
1 of 5 committers (20.0%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (9.0%) to scientific vocabulary
Keywords
Repository
Feature exploration for supervised learning
Basic Info
- Host: GitHub
- Owner: abhayspawar
- License: mit
- Language: Jupyter Notebook
- Default Branch: master
- Size: 4.95 MB
Statistics
- Stars: 764
- Watchers: 21
- Forks: 163
- Open Issues: 3
- Releases: 0
Topics
Metadata Files
README.md
featexp
Feature exploration for supervised learning. Helps with feature understanding, identifying noisy features, feature debugging, leakage detection and model monitoring.
Installation
pip install featexp
Using featexp
Detailed Medium post on using featexp. Translations from web: Chinese, Russian
featexp draws plots similar to partial dependence plots, but directly from data instead of using a trained model like current implementations of pdp do. Since it draws plots from data directly, it helps with understanding the features well and building better ML models.
```bash from featexp import getunivariateplots getunivariateplots(data=datatrain, targetcol='target', datatest=datatest, featureslist=['DAYSEMPLOYED'])
datatest and featureslist are optional.
Draws plots for all columns if features_list not passed
Draws only train data plots if no test_data passed
```
featexp bins a feature into equal population bins and shows the mean value of the dependent variable (target) in each bin. Here's how to read these plots:
- The trend plot on the left helps you understand the relationship between target and feature.
- Population distribution helps you make sure the feature is correct.
- Also, shows the number of trend direction changes and the correlation between train and test trend which can be used to identify noisy features. A high number of trend changes or low trend correlation implies high noise.
Example of a noisy feature: Has low trend correlation
Getting binned feature stats
Returns mean target and population in each bin of a feature
```bash from featexp import univariateplotter binneddatatrain, binneddatatest = univariateplotter(data=datatrain, targetcol='target', feature='DAYSEMPLOYED', datatest=data_test)
For only train data
binneddatatrain = univariateplotter(data=datatrain, targetcol='target', feature='DAYSEMPLOYED') ```
Getting stats for all features
Returns trend changes and trend correlation for all features in a dataframe
```bash from featexp import gettrendstatsfeature stats = gettrendstats(data=datatrain, targetcol='target', datatest=data_test)
data_test is optional. If not passed, trend correlations aren't calculated.
```
Returns a dataframe with trend changes and trend correlation which can be used for dropping the noisy features, etc.
Leakage detection
It helps with identifying why a feature is leaky which helps with debugging.
Nulls have 0% mean target and 100% mean target in other bins. Implies this feature is populated only for target = 1.
Citing featexp
GitHub Events
Total
- Watch event: 3
- Fork event: 1
Last Year
- Watch event: 3
- Fork event: 1
Committers
Last synced: almost 3 years ago
All Time
- Total Commits: 90
- Total Committers: 5
- Avg Commits per committer: 18.0
- Development Distribution Score (DDS): 0.122
Top Committers
| Name | Commits | |
|---|---|---|
| Abhay Pawar | a****r@g****m | 79 |
| Abhay Pawar | a****r@s****m | 5 |
| Pau Roger Puig-Sureda | p****u@f****m | 3 |
| Pau Roger Puig-Sureda | p****r@a****u | 2 |
| Miguel Mendez | m****z@g****m | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 23
- Total pull requests: 5
- Average time to close issues: about 2 months
- Average time to close pull requests: 26 days
- Total issue authors: 21
- Total pull request authors: 4
- Average comments per issue: 1.96
- Average comments per pull request: 1.2
- Merged pull requests: 3
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- Emily-Zh (2)
- e184633 (2)
- AWalkInClouds (1)
- mrxuehb (1)
- wubinbai (1)
- pauroger (1)
- anzhizh (1)
- michaelabehsera (1)
- chnzhangrui (1)
- StarJu (1)
- XiaolinZHONG (1)
- YasinZhao (1)
- snehamaheshwari07 (1)
- joanap (1)
- gig67 (1)
Pull Request Authors
- pauroger (2)
- longmfe (1)
- mikenyaga (1)
- mmeendez8 (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 2
-
Total downloads:
- pypi 1,408 last-month
- Total docker downloads: 58
-
Total dependent packages: 0
(may contain duplicates) -
Total dependent repositories: 5
(may contain duplicates) - Total versions: 8
- Total maintainers: 1
proxy.golang.org: github.com/abhayspawar/featexp
- Documentation: https://pkg.go.dev/github.com/abhayspawar/featexp#section-documentation
- License: mit
-
Latest release: v0.0.7
published over 5 years ago
Rankings
pypi.org: featexp
Feature exploration for supervised learning
- Homepage: https://github.com/abhayspawar/featexp
- Documentation: https://featexp.readthedocs.io/
- License: MIT License
-
Latest release: 0.0.7
published over 5 years ago
Rankings
Maintainers (1)
Dependencies
- ipykernel >=5.1.3 development
- matplotlib * development
- numpy * development
- pandas * development
- scikit-learn >=0.22.1 development
- virtualenvwrapper >=4.8.4 development
- xgboost >=1.0.0 development
- pandas *