tsfeast

A collection of Scikit-Learn compatible time series transformers and tools.

Installation

Create a virtual environment and install:

From PyPi

bash pip install tsfeast

From this repo

bash pip install git+https://github.com/chris-santiago/tsfeast.git

Use

Preliminaries

This example shows both the use of individual transformers and the TimeSeriesFeatures convenience class that wraps multiple transformers. Both methods are compatible with Scikit-Learn Pipeline objects.

```python import warnings warnings.filterwarnings("ignore") # ignore pandas concat warnings from statsmodels

import pandas as pd import numpy as np import matplotlib.pyplot as plt

from sklearn.linearmodel import LinearRegression, Lasso, PoissonRegressor from sklearn.pipeline import Pipeline from sklearn.featureselection import SelectKBest from sklearn.metrics import meansquarederror, meanabsolutepercentageerror, meanabsoluteerror from sklearn.preprocessing import MinMaxScaler, StandardScaler from statsmodels.tsa.arimaprocess import armageneratesample from steps.forward import ForwardSelector

from tsfeast.transformers import DateTimeFeatures, InteractionFeatures, LagFeatures from tsfeast.tsfeatures import TimeSeriesFeatures from tsfeast.funcs import getdatetimefeatures from tsfeast.utils import plot_diag from tsfeast.models import ARMARegressor ```

```python def makedummydata(n=200): nlags = 2 coefs = {'ar': [1, -0.85], 'ma': [1, 0], 'trend': 3.2, 'bdaysinmonth': 231, 'marketing': 0.0026} rng = np.random.defaultrng(seed=42)

sales = pd.DataFrame({
    'date': pd.date_range(end='2020-08-31', periods=n, freq='M'),
    'sales_base': rng.poisson(200, n),
    'sales_ar': arma_generate_sample(ar=coefs['ar'], ma=coefs['ma'], nsample=n, scale=100),
    'sales_trend': [x * coefs['trend'] + rng.poisson(300) for x in range(1, n+1)],
})

sales = sales.join(get_datetime_features(sales['date'])[['bdays_in_month', 'quarter']])
sales['sales_per_day'] = sales['bdays_in_month'] * coefs['bdays_in_month'] + rng.poisson(100, n)

sales['mkt_base'] = rng.normal(1e6, 1e4, n)
sales['mkt_trend'] = np.array([x * 5e3 for x in range(1, n+1)]) + rng.poisson(100)
sales['mkt_season'] = np.where(sales['quarter'] == 3, sales['mkt_base'] * .35, 0)
sales['mkt_total'] = sales.loc[:, 'mkt_base': 'mkt_season'].sum(1) + rng.poisson(100, n)
sales['sales_mkting'] = sales['mkt_total'].shift(n_lags) * coefs['marketing']

final = pd.DataFrame({
    'y': sales[['sales_base', 'sales_ar', 'sales_trend', 'sales_per_day', 'sales_mkting']].sum(1).astype(int),
    'date': sales['date'],
    'marketing': sales['mkt_total'],
    'x2': rng.random(n),
    'x3': rng.normal(loc=320, scale=4, size=n)
})
return sales.iloc[2:, :], final.iloc[2:, :]

```

python def get_results(estimator, x_train, x_test, y_train, y_test): return pd.DataFrame( { 'training': [ mean_absolute_error(y_train, estimator.predict(x_train)), mean_absolute_percentage_error(y_train, estimator.predict(x_train)) ], 'testing': [ mean_absolute_error(y_test, estimator.predict(x_test)), mean_absolute_percentage_error(y_test, estimator.predict(x_test)) ], }, index = ['MAE', 'MAPE'] )

Example Data

The dummy dataset in this example includes trend, seasonal, autoregressive and other factor components. Below, we visualize the individual components (comps) and features of the dummy dataset data.

python comps, data = make_dummy_data()

Sales Components

python comps.head()

python for col in comps.columns: print(f'Column: {col}') plt.figure(figsize=(10, 5)) plt.plot(comps[col]) plt.show()

Column: date

Column: sales_base

Column: sales_ar

Column: sales_trend

Column: bdays_in_month

Column: quarter

Column: sales_per_day

Column: mkt_base

Column: mkt_trend

Column: mkt_season

Column: mkt_total

Column: sales_mkting

Dummy Dataset

python data.head()

python for col in data.columns: print(f'Column: {col}') plt.figure(figsize=(10, 5)) plt.plot(data[col]) plt.show()

Column: y

Column: date

Column: marketing

Column: x2

Column: x3

python X = data.iloc[:, 1:] y = data.iloc[:, 0] x_train, x_test = X.iloc[:-40, :], X.iloc[-40:, :] y_train, y_test = y.iloc[:-40], y.iloc[-40:]

Individual Transformers

tsfeast provides individual time series transformers that can be used by themselves or within Scikit-Learn Pipeline objects:

|Transformer|Parameters|Description| |-----------|----------|-----------| |OriginalFeatures|None|Passes original features through pipeline.| |Scaler|None|Wraps Scikit-Learn StandardScaler to maintain DataFrame columns.| |DateTimeFeatures|datecol: str, dtformat: str, freq: str|Generates datetime features from a given date column.| |LaggedFeatures|nlags: int, fillna: bool|Generate lag features.| |RollingFeatures|windowlengths: List[int], fillna: bool|Generate rolling features (mean, std, min, max) for each specified window length.| |EwmaFeatures|windowlengths: List[int], fillna: bool|Generate exponentially-weighted moving average for each specified window length.| |ChangeFeatures|periodlengths: List[int], fillna: bool|Generate percent change for all features for each specified period length.| |DifferenceFeatures|n_diffs: int, fillna: bool|Generate n differences for all features.| |PolyFeatures|degree: int|Generate polynomial features.| |InteractionFeatures|None|Wraps Scikit-Learn PolynomialFeatures to generate interaction features and maintain DataFrame columns.|

Notes on Pipeline Use

Behavior of Scikit-Learn Pipeline objects is appropriate and intended for independent data observations, but not necessarily appropriate for the temporal dependencies inherent in time series.

Scikit-Learn pipelines only call the .transform() method during the .predict() method, which is appropriate to prevent data leakage in predictions. However, most of the transformers in this package take a set of features and generate new features; there's no inherent method to transform some time series features given a fitted estimator.

For time series lags, changes, etc., we have access to past data for feature generation without risk of data leakage; certain features (e.g. lags) require this to avoid NaNs or zeros. This behavior is appropriate for time series transformations, only.

Generate DateTime Features

python dt = DateTimeFeatures(date_col='date') dt.fit_transform(X, y)

Generate Interaction Features

python feat = LagFeatures(n_lags=4) feat.fit_transform(X.iloc[:, 1:], y) # skipping date column

TimeSeriesFeatures Class

tsfeast also includes a TimeSeriesFeatures class that generates multiple time series features in one transformer. The only required parameter is the column of datetimes; the optional parameters control what additional transformers are included.

|Parameter|Type|Description| |---------|----|-----------| |datetime|str|Column that holds datetime information| |trend|str|Trend to include, options are {'n': no trend, 'c': constant only, 't': linear trend, 'ct': constant and linear trend, 'ctt': constant, linear and quadratric trend}; defaults to no trend| |lags|int|Number of lags to include (optional).| |rolling|List[int]|Number of rolling windows to include (optional).| |ewma|List[int]|Number of ewma windows to include (optional).| |pct_chg|List[int]|Periods to use for percent change features (optional).| |diffs|int|Number of differences to include (optional).| |polynomial|int|Polynomial(s) to include (optional).| |interactions|bool|Whether to include interactions of original featutes; deault True.| |fillna|bool|Whether to fill NaN values with zero; default True.|

python feat = TimeSeriesFeatures( datetime='date', trend='t', lags=4, interactions=False, polynomial=3 ) features = feat.fit_transform(X, y)

python features.head()

python [x for x in features.columns]

['trend',
 'original__marketing',
 'original__x2',
 'original__x3',
 'datetime__year',
 'datetime__quarter',
 'datetime__month',
 'datetime__days_in_month',
 'datetime__bdays_in_month',
 'datetime__leap_year',
 'features__lags__marketing_lag_1',
 'features__lags__x2_lag_1',
 'features__lags__x3_lag_1',
 'features__lags__marketing_lag_2',
 'features__lags__x2_lag_2',
 'features__lags__x3_lag_2',
 'features__lags__marketing_lag_3',
 'features__lags__x2_lag_3',
 'features__lags__x3_lag_3',
 'features__lags__marketing_lag_4',
 'features__lags__x2_lag_4',
 'features__lags__x3_lag_4',
 'features__polynomial__marketing^2',
 'features__polynomial__x2^2',
 'features__polynomial__x3^2',
 'features__polynomial__marketing^3',
 'features__polynomial__x2^3',
 'features__polynomial__x3^3']

Pipeline Example

The TimeSeriesFeatures class can be used as a feature generation step within a Scikit-Learn Pipeline. Given the temporal nature of the data and models, this may not be appropriate for all use cases-- though the class remains fully compatible with Pipeline objects.

We'll instantiate a TimeSeriesFeatures object with a linear trend, four lags and no interactions. Our pipeline will include feature generation, feature scaling and feature selection steps, before modeling with ordinary least squares.

Note: the ForwardSelector class is available in the step-select package (https://pypi.org/project/step-select/).

The pipeline creates a total of 22 features, before selecting only four to use in the final model. Note that 3 of the 4 final features corresponed with features from our "true model" that created the dummy dataset ('trend', 'datetime_bdaysinmonth' and 'marketinglag_2').

Regression diagnostic plots show evidence of slightly non-normal residuals and (1) autoregressive term (again, as specified in the "true model"). We'll address the autoregressive term in the next example.

```python feat = TimeSeriesFeatures( datetime='date', trend='t', lags=4, interactions=False )

pl = Pipeline([ ('featureextraction', feat), ('scaler', StandardScaler()), ('featureselection', ForwardSelector(metric='bic')), ('regression', LinearRegression()) ])

pl.fit(xtrain, ytrain) ```

Pipeline(steps=[('feature_extraction',
                 TimeSeriesFeatures(datetime='date', interactions=False, lags=4,
                                    trend='t')),
                ('scaler', StandardScaler()),
                ('feature_selection', ForwardSelector(metric='bic')),
                ('regression', LinearRegression())])

python pl.named_steps.feature_extraction.output_features_

python new_features = pl.named_steps.feature_extraction.feature_names_ mask = pl.named_steps.feature_selection.get_support() new_features[mask]

Index(['trend', 'datetime__bdays_in_month', 'features__lags__marketing_lag_2',
       'features__lags__x3_lag_2'],
      dtype='object')

python get_results(pl, x_train, x_test, y_train, y_test)

python resid = (y_train - pl.predict(x_train)) plot_diag(resid.iloc[2:]) # throw out first two residuals b/c of lags

ARMA Regressor

tsfeast includes a models module that provides an ARMARegressor class for extending Scikit-Learn regressors by adding support for AR/MA or ARIMA residuals. It accepts an arbitrary Scikit-Learn regressor and a tuple indicating the (p,d,q) order for the residuals model.

|Attribute|Description| |---------|-----------| |estimator|The Scikit-Learn regressor.| |order|The (p,d,q,) order of the ARMA model.| |intercept_|The fitted estimator's intercept.| |coef_|The fitted estimator's coefficients.| |arma_|The fitted ARMA model.| |fitted_values_|The combined estimator and ARMA fitted values.| |resid_|The combined estimator and ARMA residual values.|

Note The predict method should not be used to get fitted values from the training set; rather, users should access this same data using the fitted_values_ attribute. The predict method calls the ARMA regresor's forecast method, which generates predictions from the last time step in the training data, thus would not match, temporally, in a predict call with training data.

The pipeline follows the same steps as the previous example, with the only change beging the regression model-- in this case, the ARMARegressor. Metrics on test set slightly improve and we no longer see evidence of autoregressive term in the residuals.

```python feat = TimeSeriesFeatures( datetime='date', trend='t', lags=4, interactions=False )

mod = ARMARegressor( estimator=PoissonRegressor(), order=(1,0,0) )

pl = Pipeline([ ('featureextraction', feat), ('scaler', StandardScaler()), ('featureselection', ForwardSelector(metric='bic')), ('regression', mod) ])

pl.fit(xtrain, ytrain) ```

Pipeline(steps=[('feature_extraction',
                 TimeSeriesFeatures(datetime='date', interactions=False, lags=4,
                                    trend='t')),
                ('scaler', StandardScaler()),
                ('feature_selection', ForwardSelector(metric='bic')),
                ('regression', ARMARegressor(estimator=PoissonRegressor()))])

python new_features = pl.named_steps.feature_extraction.feature_names_ mask = pl.named_steps.feature_selection.get_support() new_features[mask]

Index(['trend', 'datetime__bdays_in_month', 'features__lags__marketing_lag_2',
       'features__lags__x3_lag_2'],
      dtype='object')

python get_results(pl, x_train, x_test, y_train, y_test)

python plot_diag(pl.named_steps.regression.resid_)