https://github.com/cobrbra/rpensemble

Extension of the package RPEnsemble to incorporate extra functionality for supervised dimension reduction and sufficient subspace identification.

https://github.com/cobrbra/rpensemble

Repository

Extension of the package RPEnsemble to incorporate extra functionality for supervised dimension reduction and sufficient subspace identification.

README.Rmd

---
output: github_document
---



```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)
```

# RPEnsemble


[![R-CMD-check](https://github.com/cobrbra/RPEnsemble/workflows/R-CMD-check/badge.svg)](https://github.com/cobrbra/RPEnsemble/actions)


This is an R package extending RPEnsemble (created by my PhD supervisor Timothy Cannings and available on [CRAN](https://cran.r-project.org/web/packages/RPEnsemble/index.html)). RPEnsemble contains tools to approach classification tasks using the random projections method of [Cannings and Samworth, 2017](https://rss.onlinelibrary.wiley.com/doi/full/10.1111/rssb.12228). This version of the package incorporates extra functionality for supervised dimensionality reduction and sufficient subspace estimation.

## Installation

You can install this development version from [GitHub](https://github.com/) with:

``` r
# install.packages("devtools")
devtools::install_github("cobrbra/RPEnsemble")
```

## Supervised Dimension Reduction
Supervised dimensionality reduction concerns the following problem: given inputs X and outputs Y in a prediction problem (regression or classification, although here we focus on classification), can we form a low-dimensional representation of X that contains the maximal amount (ideally, all) information about Y contained in X.

The methodology here deals with finding maximally informative *linear* representations of X, i.e. representations that can be expressed AX, where A is some dimension-shrinking matrix. In the case where such a matrix can be found, we may also refer to the problem of identifying the minimal subspace spanned by such a matrix as *sufficient subspace estimation*.

## Example
First, lets generate some data. We'll use an inbuilt data generating mechanism that creates some 2D 'doughnuts' embedded in higher-dimensional (in this case, 4D) space.

```{r}
library(RPEnsemble)
set.seed(1234)

training <- RPModel(Model.No = 5, n = 500, p = 4)

pairs(training$data$x, col = ifelse(training$data$y == 1, "red", "blue"))

```

We may wish to extract a) estimates of the sufficient subspace (in this case, the subspace spanned by the first two canonical basis vectors), and b) low-dimensional representations of our data points. We may do the first with the following

```{r}
subspaces <- RPDecomposeA(XTrain = training$data$x, 
                          YTrain = training$data$y, 
                          B1 = 500,
                          B2 = 50, 
                          d = 2, 
                          base = "knn", 
                          estmethod = "randomsplit")
```

The object `subspaces` contains a SVD-style output. It contains an element `eigv`, a vector of length equal to the data's original dimension, showing the relative prioritisation of subsequently identified subspace dimensions. Looking at these weights plotted below, we can see that the decomposition has clearly identified two main directions.

```{r}
plot(1:4, subspaces$eigv, xlab = "Subspace dimension", ylab = "Eigenvalue", ylim = c(0,1))
```

We can investigate the subspace directions (columns of the element `v`) associated with these first two eigenvalues. We see that they have weight concentrated in the dimensions in which the original doughnuts originated.

```{r}
plot(1:4, subspaces$v[, 1], col = "red", ylim = c(-1,1), 
     xlab = "Index", ylab = "Weights");
points(1:4, subspaces$v[, 2], col = "blue")
lines(1:4, rep(0,4), lty = 2)
```

We can also look at the discovered 2D representation of the data.

```{r}
reduced_data <- RPReduce(XTrain = training$data$x, 
                         YTrain = training$data$y,
                         XTest = training$data$x,
                         reduced_dim = 2,
                         d = 2, 
                         decomposition = subspaces)

plot(reduced_data$Dim_1, reduced_data$Dim_2, col = ifelse(training$data$y == 1, "red", "blue"), xlab = "Reduced dimension 1", ylab = "Reduced dimension 2")
```


## Contact
Feel free to contact me via [email](cobrbradley@gmail.com). Alternatively, leave a pull request or issue.

Owner

GitHub Events