https://github.com/beniaminogreen/zoomergp

https://github.com/beniaminogreen/zoomergp

Repository

README.Rmd

---
output: github_document
always_allow_html: true
---

```{r, include=F, echo=F}
devtools::load_all()
library(tidyverse)
```

# ZoomerGP: Gaussian Process Regression in R

`zoomerGP` provides a fast and composable language to write, fit, and explore Gaussian Process models in R. The package allows users to compose a Gaussian process kernel using a consise formula syntax, and tune hyperparameters through type-2 ML or Variational Inference (SVGD). Both Gaussian and non-Gaussian Likelihoods are supported. 

## Kernel formulas:

`ZoomerGP`'s flagship feature is a tidy formula interface that allows users to cleanly specify complex Gaussian-process models.  As an example, you can specify and fit a Gaussian Process with an RBF covariance function to a dataset with the following formula:

```{r, eval=F}
gaussian_process(y ~ rbf(x), data = data)
```

This syntax also allows you to arbitrarily combine different kernels buy adding and multiplying them. As an example, one can try to fit a periodic trend that changes over time and a long-term trend to the data by using the following code: 


```{r, eval=F}
gaussian_process(y ~ periodic(x) * rbf(x) + rbf(x), data = data)
```

Each kernel can also act on multiple variables, as shown below: 

```{r, eval =F}
gaussian_process(a ~ periodic(b,c) * rbf(d,e,f,g) + rbf(h), data = data)
```

## Avaliable Kernels: 

At present the following kernels are implemented: 


  | Kernel                     | Function Name            | Notes | 
| :--------------------------|:-------------------:|:-----:|
| Squared Exponential        | rbf()               | Results in a smooth, infinitely-differentiable posterior. Too smooth for most applications |
| Spectral Mixture           | spectral[0-5]()     | Spectal mixture kernel with 1,2,3,4, or 5 components. Can recover any stationary covariance function but is very expensive to fit. |
| Linear                     | linear()            | Recovers a linear trend |
| Indicator                  | indicator()         | Equal to $sigma^2$ if $x=x'$, zero otherwise. Useful when accounting for clustering.  |
| If / mask                         | mask()                | Equal of one if both $x$ and $x'$ are equal to one, zero otherwise. Most useful when 'masking' out another kernel so it only acts on certain pairs of inputs  |
| Periodic                        | periodic()                | Periodic similarity function. Can accept as a keyword argument a vector of periods associated with each dimension.  |


## Fitting to Large Datasets with Sparse Approximations.

Gaussian Process Regression are computationally intensive, and generally take
$O(n^3)$ computations to fit. To allow the method to scale to large datasets,
this package implements the projected process approximation as described in
Rassmusen and Williams's (2005) Gaussian Processes for Machine Learning. To
turn this approximation on, use the `sparse=T` argument when fitting the
gaussian processes please be aware that you may also need to change
the default value of the number of inducing points (`n_points`) for numerical
stability.

## Optimization Methods

Currently, we the package supports 3 engines for hyperparameter optimization.
`bfgs` is selected by default, but these can be changed using the
training_method argument to `gaussian_process` function

* `bfgs` - as implimented in the optim package in R. Will sometimes fail to
  coverge, leading to an error.
* `rgenoud` - as implimented in the `rgenoud package.` This is likely the most
  robust optimizer as it avoids getting stuck in local minima, but also the
  most computationally intensive.
* `coin` - a learning-rate free algorithm that is implimented in Rust. Seems to
  work well across a variety of problems, but can get stuck in local minima.

## Fitting Non-Gaussian Likelihoods with Variational Inference

`zoomerGP` also implements a variant of the [Stein Variational Gaussian Processes](https://arxiv.org/abs/2009.12141) algorithm to perform variational inference over hyperparameters and latent function values when the likelihood is non-Gaussian. Specifically, we use the [Coin Sampling](https://arxiv.org/abs/2301.11294) algorithm proposed by the same authors to provide a robust Variational Inference engine that does not require tuning by the user. 

At the moment, inference for the following non-Gaussian likelihoods are implemented: 

- logistic regression (via `the logistic_gaussian_process` function).

# Installation Instructions

## Installing Rust

If your operating system or version of R is not installed, you must have the
[Rust compiler](https://www.rust-lang.org/tools/install) installed to compile
this package from sources. After the package is compiled, Rust is no longer
required, and can be safely uninstalled.

#### Installing Rust on Linux or Mac:

To install Rust on Linux or Mac, you can simply run the following snippet in
your terminal.

``` sh
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

#### Installing Rust on Windows:

To install Rust on windows, you can use the Rust installation wizard,
`rustup-init.exe`, found [at this
site](https://forge.rust-lang.org/infra/other-installation-methods.html).
Depending on your version of Windows, you may see an error that looks something like this:

```
error: toolchain 'stable-x86_64-pc-windows-gnu' is not installed
```

In this case, you should run `rustup install stable-x86_64-pc_windows-gnu` to
install the missing toolchain. If you're missing another toolchain, simply type
this in the place of `stable-x86_64-pc_windows-gnu` in the command above.

### Installing Package from Github:

Once you have rust installed Rust, you should be able to install the package
with either the install.packages function as above, or using the
`install_github` function from the `devtools` package or with the `pkg_install`
function from the `pak` package.

``` r
## Install with devtools
# install.packages("devtools")
devtools::install_github("beniaminogreen/GPR")

## Install with pak
# install.packages("pak")
pak::pkg_install("beniaminogreen/GPR")
```

### Loading The Package

Once the package is installed, you can load it into memory as usual by typing:

```{r, warning = FALSE, message = FALSE, eval = F}
library(zoomerGP)
```

Owner

GitHub Events

Committers

Last synced: 7 months ago

Issues and Pull Requests

Last synced: 7 months ago