cvxcovariance

Quick Start in Online IDEs

GitHub Codespaces

Start coding instantly in a pre-configured cloud development environment.

The cvxcovariance package provides simple tools for creating an estimate $\hat\Sigmat$ of the covariance $\Sigmat$ of the $n$-dimensional return vectors $rt$, $t=1,2,\ldots$, based on the observed returns $r1, \ldots, r{t-1}$. (Here $rt$ is the return from $t-1$ to $t$.) The covariance predictor $\hat\Sigmat$ is generated by blending $K$ different "expert" predictors $\hat\Sigmat^{(1)},\ldots,\hat\Sigma_t^{(K)}$, by solving a convex optimization problem at each time step.

For a detailed description of the methodology, see our manuscript A Simple Method for Predicting Covariance Matrices of Financial Returns (in particular Section 3).

In the simplest case the user provides a $T\times n$ pandas DataFrame of returns $r1,\ldots,rT$ and $K$ half-life pairs, and gets back covariance predictors for each time step. (The $K$ experts are computed as iterated exponentially weighted moving average (IEWMA) predictors as described in Section 2.5 of the paper.) In the more general case, the user provides the $K$ expert predictors $\hat\Sigmat^{(1)},\ldots,\hat\Sigmat^{(K)}$, $t=1,\ldots,T$, and these are blended together by solving the convex optimization problems. In either case the result is returned as an iterator object over namedtuples: Result = namedtuple("Result", ["time", "mean", "covariance", "weights"]).

Note: at time $t$ the user is provided with $\hat\Sigma_{t+1}$, $\textit{i.e.}$, the covariance matrix for the next time step. So Result.covariance returns the covariance prediction for time+1.

Installation

To install the package, run the following command in the terminal:

bash pip install cvxcovariance

Usage

There are two alternative ways to use the package. The first is to use the from_ewmas function to create a combined multiple IEWMA (CM-IEWMA) predictor. The second is to provide your own covariance experts, via dictionaries, and pass them to the from_sigmas function. Both functions return an object of the _CovarianceCombination class, which can be used to solve the covariance combination problem.

CM-IEWMA

The from_ewmas function takes as input a pandas DataFrame of returns and the IEWMA half-life pairs (each pair consists of one half-life for volatility estimation and one for correlation estimation), and returns an iterator object that iterates over the CM-IEWMA covariance predictors defined via namedtuples. Through the namedtuple you can access the time, mean, covariance, and weights attributes. time is the timestamp. mean is the estimated mean of the return at the $\textit{next}$ timestamp, $\textit{i.e.}$ time+1, if the user wants to estimate the mean; per default the mean is set to zero, which is a reasonable assumption for many financial returns. covariance is the estimated covariance matrix for the $\textit{next}$ timestamp, $\textit{i.e.}$ time+1. weights are the $K$ weights attributed to the experts. Here is an example:

```python import pandas as pd from cvx.covariance.combination import from_ewmas

Load return data

returns = pd.readcsv("data/ff5norf.csv", indexcol=0, header=0, parse_dates=True).iloc[:1000] n = returns.shape[1]

Define half-life pairs for K=3 experts, (halflifevola, halflifecov)

halflife_pairs = [(10, 21), (21, 63), (63, 125)]

Define the covariance combinator

combinator = fromewmas(returns, halflifepairs, minperiodsvola=n, # min periods for volatility estimation minperiodscov=3 * n) # min periods for correlation estimation

Solve combination problem and loop through combination results to get predictors

covariancepredictors = {} for predictor in combinator.solve(window=10): # lookback window for optimization # From predictor we can access predictor.time, predictor.mean (=0 here), # predictor.covariance, and predictor.weights covariancepredictors[predictor.time] = predictor.covariance ```

Here covariance_predictors[t] is the covariance prediction for time $t+1$, $\textit{i.e.}$, it is uses knowledge of $r1,\ldots,rt$.

General covariance combination

The from_sigmas function takes as input a pandas DataFrame of returns and a dictionary of covariance predictors {key: {time: sigma}, where key is the key of an expert predictor and {time: sigma} is the expert predictions. For example, here we combine two EWMA covariance predictors from pandas:

```python import pandas as pd from cvx.covariance.combination import from_sigmas

Load return data

returns = pd.readcsv("data/ff5norf.csv", indexcol=0, header=0, parse_dates=True).iloc[:1000] n = returns.shape[1]

Define 21 and 63 day EWMAs as dictionaries (K=2 experts)

ewma21 = returns.ewm(halflife=21, minperiods=5 * n).cov().dropna() expert1 = {time: ewma21.loc[time] for time in ewma21.index.getlevelvalues(0).unique()} ewma63 = returns.ewm(halflife=63, minperiods=5 * n).cov().dropna() expert2 = {time: ewma63.loc[time] for time in ewma63.index.getlevelvalues(0).unique()}

Create expert dictionary

experts = {1: expert1, 2: expert2}

Define the covariance combinator

combinator = from_sigmas(sigmas=experts, returns=returns)

Solve combination problem and loop through combination results to get predictors

covariancepredictors = {} for predictor in combinator.solve(window=10): # From predictor we can access predictor.time, predictor.mean (=0 here), # predictor.covariance, and predictor.weights covariancepredictors[predictor.time] = predictor.covariance ```

Here covariance_predictors[t] is the covariance prediction for time $t+1$, $\textit{i.e.}$, it is uses knowledge of $r1,\ldots,rt$.

Poetry

We assume you share already the love for Poetry. Once you have installed poetry you can perform

bash make install

to replicate the virtual environment we have defined in pyproject.toml and locked in poetry.lock.

Jupyter

We install JupyterLab on fly within the aforementioned virtual environment. Executing

bash make jupyter

will install and start the jupyter lab.

Citing

If you want to reference our paper in your research, please consider citing us by using the following BibTeX:

BibTeX @article{johansson2023covariance, url = {http://dx.doi.org/10.1561/0800000047}, year = {2023}, volume = {12}, journal = {Foundations and Trends in Econometrics}, title = {A Simple Method for Predicting Covariance Matrices of Financial Returns}, doi = {10.1561/0800000047}, issn = {1551-3076}, number = {4}, pages = {324-407}, author = {Kasper Johansson and Mehmet G. Ogut and Markus Pelger and Thomas Schmelzer and Stephen Boyd} }