shadowvimp

https://github.com/oktawiastaburo/shadowvimp

Repository

README.Rmd

---
output: github_document
---



```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%",
  fig.width = 30,
  fig.height = 20,
  fig.align = "center"
)
```

# shadowVIMP


[![R-CMD-check](https://github.com/OktawiaStaburo/shadowVIMP/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/OktawiaStaburo/shadowVIMP/actions)




## Overview

`shadowVIMP` is an R package for variable selection that reduces the number of covariates in a statistically rigorous and informed way, identifying the most informative predictors. This package implements a method that performs statistical tests on the Variable Importance Measures (VIMP) obtained from the Random Forest (RF) algorithm to determine whether each covariate is statistically significant and truly informative. In contrast to widely used methods, such as selecting the top *n* covariates with the highest VIMP or choosing covariates with a VIMP above a certain threshold, the method implemented in `shadowVIMP` allows for a statistical justification of whether a given VIMP is sufficiently large to be unlikely due to chance. The main function of the package, `shadow_vimp()`, outputs a table indicating whether each covariate is informative, along with its associated (adjusted) p-values. In addition, the `plot_vimps()` function provides a convenient way to visualise the VIMPs obtained from the algorithm, including unadjusted, FDR- and FWER-adjusted p-values. Details on the method, a realistic example of its usage, and guidance on interpreting the results can be found in the vignette: `vignette("shadowVIMP-vignette")`.


## Installation

The `shadowVIMP` package can be installed in the following way::

``` {r, eval = FALSE}
install.packages("shadowVIMP")
```


## Usage

Imagine you are working with a dataset with many variables and you want to select the subset of the most informative covariates. Instead of computing the variable importance, displaying it on variable importance plots, and (somewhat arbitrarily) selecting the set of informative features, you can use the method implemented in this package and select the subset of features in a more robust way. The following example shows the basic use case.

```{r example, message = FALSE, warning = FALSE}
library(shadowVIMP)
library(magrittr)
library(ggplot2)

data(mtcars)

# For reproducibility
set.seed(789)
# Value of num.threads parameter
global_num_threads <- 1

# Standard usage
# WARNING 1: When working with real data, increase the value of the niters parameter or leave it at the default value.
# WARNING 2: To avoid potential issues with using multiple threads on CRAN, we set num.threads to 1, by default it is set to half of the available threads, which speeds up computation.
vimp_seq <- shadow_vimp(data = mtcars, outcome_var = "vs", niters = c(30, 100, 150), num.threads = global_num_threads)

# Summary of the results
vimp_seq

# Print informative covariates according to the pooled criterion (with and without p-value correction)
vimp_seq$final_dec_pooled

# The significance level used in each step of the procedure:
vimp_seq$alpha

# Were all covariates deemed insignificant at any step of the pre-selection? If so, which step?
# If `step_all_covariates_removed == 0`, then at least one covariate survived the pre-selection
vimp_seq$step_all_covariates_removed

# Check the time needed to execute each step of the algorithm and the entire procedure
vimp_seq$time_elapsed

# Check the call code that was used to create the inspected object
vimp_seq$call

# Check the VIMPs of the covariates and their shadows from the last step of the procedure
vimp_seq$vimp_history %>% head()

# Inspect in detail two steps of pre-selection
#  vimp_seq$pre_selection$step_1
#  vimp_seq$pre_selection$step_2
```

You can visualize your results in the following way:

```{r example_cont, cache=FALSE}
plot_vimps(
  shadow_vimp_out = vimp_seq,
  text_size = 10,
  legend.text = element_text(size = 40),
  axis.text = element_text(size = 30)
) +
  patchwork::plot_annotation(
    title = "shadowVIMP Results",
  ) &
  theme(plot.title = element_text(size = 45, face = "bold"))
```

For a more realistic and detailed example of how to use this package, and the theory behind the method, see `vignette("shadowVIMP-vignette")`.

## Getting help

If you encounter a clear bug, please file an issue with a minimal reproducible example.

Owner

GitHub Events

Issues and Pull Requests

Last synced: 10 months ago

Packages