Empulse

Empulse is a package aimed to enable value-driven and cost-sensitive analysis in Python. The package implements popular value-driven and cost-sensitive metrics and algorithms in accordance to sci-kit learn conventions. This allows the measures to seamlessly integrate into existing ML workflows.

Installation

Empulse requires python 3.10 or higher.

Install empulse via pip with

bash pip install empulse

Documentation

You can find the documentation here.

Features

• Ready to use out of the box with scikit-learn
• Use case specific profit and cost metrics
• Flexible profit-driven and cost-sensitive models
• Easy passing of instance-dependent costs
• Cost-aware resampling and relabeling
• Find the optimal decision threshold
• Easy access to real-world datasets for benchmarking

Take the tour

Ready to use out of the box with scikit-learn

All components of the package are designed to work seamlessly with scikit-learn.

Models are implemented as scikit-learn estimators and can be used anywhere a scikit-learn estimator can be used.

Pipelines

```python from empulse.models import CSLogitClassifier from sklearn.datasets import make_classification from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler

X, y = makeclassification() pipeline = Pipeline([ ("scaler", StandardScaler()), ("model", CSLogitClassifier()) ]) pipeline.fit(X, y, modelfpcost=2, model_fncost=1) ```

Cross-validation

```python from sklearn.modelselection import crossval_score

crossvalscore( pipeline, X, y, scoring="rocauc", params={"modelfpcost": 2, "model_fncost": 1} ) ```

Grid search

```python from sklearn.model_selection import GridSearchCV

paramgrid = {"modelC": [0.1, 1, 10]} gridsearch = GridSearchCV(pipeline, paramgrid, scoring="rocauc") gridsearch.fit(X, y, modelfpcost=2, model_fncost=1) ```

All metrics can easily be converted as scikit-learn scorers and can be used in the same way as any other scikit-learn scorer.

```python from empulse.metrics import expectedcostloss from sklearn.metrics import make_scorer

scorer = makescorer( expectedcostloss, responsemethod="predictproba", greaterisbetter=False, fpcost=2, fncost=1 ) crossval_score(pipeline, X, y, scoring=scorer) ```

Use case specific profit and cost metrics

Empulse offers a wide range of profit and cost metrics that are tailored to specific use cases such as: - customer churn, - customer acquisition, - credit scoring, - and fraud detection (coming soon).

For other use cases, the package provides a generic implementations for: - the Metric class to define your own custom metrics, - the cost loss, - the expected cost loss, - the expected log cost loss, - the savings score, - the expected savings score, - and the maximum profit score.

Flexible profit-driven and cost-sensitive models

Empulse provides a range of profit-driven and cost-sensitive models such as: - CSLogitClassifier, - CSBoostClassifier, - B2BoostClassifier, - RobustCSClassifier, - ProfLogitClassifier, - BiasRelabelingClassifier, - BiasResamplingClassifier, - and BiasReweighingClassifier.

Each classifier tries to balance ease of use through good defaults and flexibility through a wide range of parameters.

For instance, the CSLogitClassifier allows you to change the loss function and the optimization method:

```python import numpy as np from empulse.models import CSLogitClassifier from empulse.metrics import expectedsavingsscore from scipy.optimize import minimize, OptimizeResult

def optimize(objective, X, *kwargs) -> OptimizeResult: initialguess = np.zeros(X.shape[1]) result = minimize( lambda x: -objective(x), # inverse objective function to maximize initialguess, method='BFGS', *kwargs ) return result model = CSLogitClassifier(loss=expectedsavingsscore, optimize_fn=optimize) ```

Easy passing of instance-dependent costs

Instance-dependent costs can easily be passed to the models through metadata routing.

For instance, the instance-dependent costs are passed dynamically to each fold of the cross-validation through requesting the costs in the set_fit_request method of the model and the set_score_request method of the scorer.

```python import numpy as np from empulse.models import CSLogitClassifier from empulse.metrics import expectedcostloss from sklearn import setconfig from sklearn.datasets import makeclassification from sklearn.modelselection import crossvalscore from sklearn.metrics import makescorer from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler

setconfig(enablemetadata_routing=True)

X, y = makeclassification() fpcost = np.random.rand(y.size) fn_cost = np.random.rand(y.size)

pipeline = Pipeline([ ("scale", StandardScaler()), ("model", CSLogitClassifier().setfitrequest(fpcost=True, fncost=True)) ])

scorer = makescorer( expectedcostloss, responsemethod="predictproba", greaterisbetter=False, ).setscorerequest(fpcost=True, fn_cost=True)

crossvalscore(pipeline, X, y, scoring=scorer, params={"fpcost": fpcost, "fncost": fncost}) ```

Cost-aware resampling and relabeling

Empulse uses the imbalanced-learn package to provide cost-aware resampling and relabeling techniques: - CostSensitiveSampler - BiasResampler - BiasRelabler

```python from empulse.samplers import CostSensitiveSampler from sklearn.datasets import make_classification

X, y = makeclassification() sampler = CostSensitiveSampler() Xresampled, yresampled = sampler.fitresample(X, y, fpcost=2, fncost=1) ```

They can be used in an imbalanced-learn pipeline:

```python import numpy as np from empulse.samplers import CostSensitiveSampler from imblearn.pipeline import Pipeline from sklearn import setconfig from sklearn.datasets import makeclassification from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression

setconfig(enablemetadata_routing=True)

X, y = makeclassification() fpcost = np.random.rand(y.size) fncost = np.random.rand(y.size) pipeline = Pipeline([ ("scaler", StandardScaler()), ("sampler", CostSensitiveSampler().setfitresamplerequest(fpcost=True, fncost=True)), ("model", LogisticRegression()) ])

pipeline.fit(X, y, fpcost=fpcost, fncost=fncost) ```

Find the optimal decision threshold

Empulse provides the CSThresholdClassifier which allows you to find the optimal decision threshold for a given cost matrix to minimize the expected cost loss.

The meta-estimator changes the predict method of the base estimator to predict the class with the lowest expected cost.

```python from empulse.models import CSThresholdClassifier from sklearn.datasets import makeclassification from sklearn.linearmodel import LogisticRegression

X, y = makeclassification() model = CSThresholdClassifier(estimator=LogisticRegression()) model.fit(X, y) model.predict(X, fpcost=2, fn_cost=1) ```

Metrics like the maximum profit score conveniently return the optimal target threshold. For example the Expected Maximum Profit measure for customer churn (EMPC) tells you what fraction of the customer base should be targeted to maximize profit.

```python from empulse.metrics import empc from sklearn.datasets import makeclassification from sklearn.linearmodel import LogisticRegression

X, y = makeclassification() model = LogisticRegression() predictions = model.fit(X, y).predictproba(X)[:, 1]

score, threshold = empc(y, predictions, clv=50) ```

This score can then be converted to a decision threshold by using the classification_threshold function.

```python from empulse.metrics import classification_threshold

decisionthreshold = classificationthreshold(y, predictions, customer_threshold=threshold) ```

This can then be combined with sci-kit learn's FixedThresholdClassifier to create a model that predicts the class with the highest expected profit.

```python from sklearn.model_selection import FixedThresholdClassifier

model = FixedThresholdClassifier(estimator=model, threshold=decision_threshold) model.predict(X) ```

Easy access to real-world datasets for benchmarking

Empulse provides easy access to real-world datasets for benchmarking cost-sensitive models.

Each dataset returns the features, the target, and the instance-dependent costs, ready to use in a cost-sensitive model.

```python from empulse.datasets import loadgivemesomecredit from empulse.models import CSLogitClassifier from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler

X, y, tpcost, fpcost, tncost, fncost = loadgivemesomecredit(returnXy_costs=True)

pipeline = Pipeline([ ('scaler', StandardScaler()), ('model', CSLogitClassifier()) ]) pipeline.fit( X, y, modeltpcost=tpcost, modelfpcost=fpcost, modeltncost=tncost, modelfncost=fncost ) ```