Data

Tools

Indexes

Applications

Experiments

Open Source Science

tree-influence

Influence Estimation for Gradient-Boosted Decision Trees

https://github.com/jjbrophy47/tree_influence

Science Score: 31.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.8%) to scientific vocabulary

Keywords

boostin explainability gradient-boosted-trees influence-estimation influence-functions influential-examples instance-attribution instance-based interpretability tracin
Last synced: 6 months ago · JSON representation ·

Repository

Influence Estimation for Gradient-Boosted Decision Trees

Basic Info
  • Host: GitHub
  • Owner: jjbrophy47
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 5.38 MB
Statistics
  • Stars: 23
  • Watchers: 4
  • Forks: 9
  • Open Issues: 1
  • Releases: 0
Topics
boostin explainability gradient-boosted-trees influence-estimation influence-functions influential-examples instance-attribution instance-based interpretability tracin
Created almost 5 years ago · Last pushed over 1 year ago
Metadata Files
Readme License Citation

README.md

TreeInfluence: Influence Estimation for Gradient-Boosted Decision Trees

PyPi version Python version Github License Build

tree-influence is a python library that implements influence estimation for gradient-boosted decision trees (GBDTs), adapting popular techniques such as TracIn and Influence Functions to GBDTs. This library is compatible with all major GBDT frameworks including LightGBM, XGBoost, CatBoost, and SKLearn.

illustration

Installation

shell pip install tree-influence

Usage

Simple example using BoostIn to identify the most influential training instances to a given test instance:

```python import numpy as np from sklearn.datasets import loadiris from sklearn.modelselection import traintestsplit from lightgbm import LGBMClassifier from tree_influence.explainers import BoostIn

load iris data

data = load_iris() X, y = data['data'], data['target']

use two classes, then split into train and test

idxs = np.where(y != 2)[0] X, y = X[idxs], y[idxs] Xtrain, Xtest, ytrain, ytest = traintestsplit(X, y, testsize=0.1, randomstate=1)

train GBDT model

model = LGBMClassifier().fit(Xtrain, ytrain)

fit influence estimator

explainer = BoostIn().fit(model, Xtrain, ytrain)

estimate training influences on each test instance

influence = explainer.getlocalinfluence(Xtest, ytest) # shape=(no. train, no. test)

extract influence values for the first test instance

values = influence[:, 0] # shape=(no. train,)

sort training examples from:

- most positively influential (decreases loss of the test instance the most), to

- most negatively influential (increases loss of the test instance the most)

training_idxs = np.argsort(values)[::-1] ```

Supported Estimators

tree-influence supports the following influence-estimation techniques in GBDTs:

| Method | Description | | -------| ----------- | | BoostIn | Traces the influence of a training instance throughout the training process (adaptation of TracIn). | | TREX | Trains a surrogate kernel model that approximates the original model and decomposes any prediction into a weighted sum of the training examples (adaptation of representer-point methods). | | LeafInfluence | Estimates the impact of a training example on the final GBDT model (adaptation of influence functions). | | TreeSim | Computes influence via similarity in tree-kernel space. | | LOO | Leave-one-out retraining, measures the influence of a training instance by removing and retraining without that instance.

License

Apache License 2.0.

Reference

Brophy, Hammoudeh, and Lowd. Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees. Journal of Machine Learning Research (JMLR), 2023.

@article{brophy2023treeinfluence, author = {Jonathan Brophy and Zayd Hammoudeh and Daniel Lowd}, title = {Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees}, journal = {Journal of Machine Learning Research}, year = {2023}, volume = {24}, number = {154}, pages = {1--48}, url = {http://jmlr.org/papers/v24/22-0449.html}, }

Owner

  • Name: Jonathan Brophy
  • Login: jjbrophy47
  • Kind: user
  • Location: Portland, OR
  • Company: University of Oregon

PhD from UO.

Citation (CITATION) 

@article{brophy2023treeinfluence,
  author  = {Jonathan Brophy and Zayd Hammoudeh and Daniel Lowd},
  title   = {Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees},
  journal = {Journal of Machine Learning Research},
  year    = {2023},
  volume  = {24},
  number  = {154},
  pages   = {1--48},
  url     = {http://jmlr.org/papers/v24/22-0449.html},
}

GitHub Events

Total
  • Watch event: 4
  • Issue comment event: 1
  • Fork event: 2
Last Year
  • Watch event: 4
  • Issue comment event: 1
  • Fork event: 2

Committers

Last synced: almost 3 years ago

All Time
  • Total Commits: 319
  • Total Committers: 1
  • Avg Commits per committer: 319.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
jbrophy j****7@g****m 319

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 4
  • Total pull requests: 2
  • Average time to close issues: about 1 month
  • Average time to close pull requests: about 22 hours
  • Total issue authors: 4
  • Total pull request authors: 1
  • Average comments per issue: 2.25
  • Average comments per pull request: 0.5
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 1
  • Pull requests: 0
  • Average time to close issues: 2 months
  • Average time to close pull requests: N/A
  • Issue authors: 1
  • Pull request authors: 0
  • Average comments per issue: 3.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • S2378kmeth (1)
  • aclarkse (1)
  • Yarden234 (1)
  • LornaAS93 (1)
Pull Request Authors
  • ZaydH (2)
Top Labels
Issue Labels
bug (1)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 135 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 1
  • Total versions: 6
  • Total maintainers: 1
pypi.org: tree-influence

Influence Estimation for Gradient-Boosted Decision Trees

  • Versions: 6
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 135 Last month
Rankings
Dependent packages count: 10.1%
Forks count: 13.3%
Average: 15.8%
Stargazers count: 16.1%
Downloads: 18.1%
Dependent repos count: 21.6%
Maintainers (1)
jbrophy
Last synced: 6 months ago

Dependencies

setup.py pypi
  • numpy >=1.22
  • scikit-learn >=0.24.2
  • torch >=1.9.0
requirements.txt pypi
  • catboost ==0.26
  • configargparse ==1.5.2
  • cython ==0.29.23
  • lightgbm ==3.2.1
  • mat73 ==0.50
  • matplotlib ==3.4.2
  • numpy >=1.22.0
  • openpyxl ==3.0.7
  • pandas ==1.3.1
  • scikit-learn >=0.24.2
  • seaborn ==0.11.1
  • torch >=1.9.0
  • tqdm ==4.61.2
  • twine ==3.8.0
  • xgboost ==1.6.1
  • xlrd ==2.0.1
.github/workflows/wheels.yml actions
  • actions/checkout v2 composite
  • actions/download-artifact v2 composite
  • actions/upload-artifact v2 composite
  • pypa/cibuildwheel v2.3.1 composite
  • pypa/gh-action-pypi-publish v1.4.2 composite
pyproject.toml pypi
tree_influence/explainers/parsers/setup.py pypi