mverse

R package for multiverse analysis. Extends R package multiverse with student and analyst friendly interfaces.

https://github.com/mverseanalysis/mverse

Repository

R package for multiverse analysis. Extends R package multiverse with student and analyst friendly interfaces.

README.Rmd

---
output: 
  bookdown::github_document2:
    number_sections: false
references:
  - id: hurricane
    title: 'Female hurricanes are deadlier than male hurricanes'
    type: article-journal
    issued:
      year: 2014
      month: 6
      day: 17
    doi: \doi{10.1073/pnas.1402786111}
    journal: Proceeedings fo the National Academiy of Sciences of the United States of America
    volume: 111
    issue: 24
    page: 8782-8787
    author:
    - given: Kiju
      family: Jung
    - given: Sharon
      family: Shavitt
    - given: Madhu
      family: Viswanathan
    - given: "Joseph M."
      family: Hilbe
  - id: multiverseR
    title: 'multiverse: Multiplexing Alternative Data Analyses in R Notebooks (Version 0.5.0)'
    type: article-journal
    URL: https://github.com/MUCollective/multiverse
    issued:
      year: 2021
    journal: OSF Preprints
    author:
    - given: Abhraneel
      family: Sarma
    - given: Alex
      family: Kale
    - given: Michael
      family: Moon
    - given: Nathan
      family: Taback
    - given: Fanny
      family: Chevalier
    - given: Jessica
      family: Hullman
    - given: Matthew
      family: Kay
  - id: multiverse
    title: 'Increasing transparency through a multiverse analysis'
    type: article-journal
    issued:
      year: 2016
      month: 9
      day: 29
    doi: \doi{10.1177/1745691616658637}
    journal: Perspectives on Psychological Science
    volume: 11
    issue: 5
    page: 702-712
    author:
    - given: Sara
      family: Steegen
    - given: Francis
      family: Tuerlinckx
    - given: Andrew
      family: Gelman
    - given: Wolf
      family: Vanpaemel
  - id: speccurve
    title: 'Specification curve analysis'
    type: article-journal
    issued:
      year: 2020
      month: 7
      day: 27
    doi: \doi{10.1038/s41562-020-0912-z}
    journal: Nature Human Behaviour
    volume: 4
    page: 1208-1214
    author:
      - given: Uri
        family: Simonsohn
      - given: "Joseph P."
        family: Simmons
      - given: "Leif D."
        family: Nelson
---



```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%",
  echo = FALSE
)
```

# mverse


[![CRAN status](https://www.r-pkg.org/badges/version/mverse)](https://CRAN.R-project.org/package=mverse)
[![R-CMD-check](https://github.com/mverseanalysis/mverse/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/mverseanalysis/mverse/actions/workflows/R-CMD-check.yaml)
[![codecov](https://codecov.io/gh/mverseanalysis/mverse/branch/master/graph/badge.svg?token=WP7ZLOlHzY)](https://app.codecov.io/gh/mverseanalysis/mverse)


_mverse_ is an extension to multiverse package [@multiverseR]
which allows users create explorable multiverse analysis [@multiverse]
in R. This extension provides user friendly abstraction
and a set of examples for researchers, educators,
and students in statistics.

## Installation

You can install the released version of mverse from [CRAN](https://CRAN.R-project.org) with:

``` r
install.packages("mverse")
```

You can install the development version from [GitHub](https://github.com/) with:

``` r
# install.packages("devtools")
devtools::install_github("mverseanalysis/mverse", build_vignettes = TRUE)
```

## Usage

The following demonstration performs a multiverse analysis using `hurricane` dataset [@hurricane] included in the library. We first create 6 universes as described in Figure 1. A filter _branch_ with 2 _options_ and a mutate _branch_ with 3 _options_ results in 6 _universes_ in total. We then fit a Poisson regression model across the multiverse and inspect a coefficient estimate. See `vignette("hurricane")` for a detailed analysis as well as the terminologies used.

```{r tree, message=FALSE, fig.asp=9/16, dpi=144}
library(ggplot2)
library(igraph)
library(ggraph)
library(mverse)
outliers <- filter_branch(
  ! Name %in% c("Katrina"),
  ! Name %in% c("Katrina", "Audrey")
)
strength <- mutate_branch(
  NDAM, HighestWindSpeed, Minpressure_Updated_2014, 
)
mv <- create_multiverse(hurricane) %>%
  add_filter_branch(outliers) %>%
  add_mutate_branch(strength) 
combs <- summary(mv) %>%
  dplyr::select(outliers_branch_code, strength_branch_code)
edges <- rbind(
  combs %>%
    dplyr::mutate(from = "Data",
           to = outliers_branch_code) %>%
    dplyr::distinct(from, to),
  combs %>%
    dplyr::mutate(from = outliers_branch_code,
           to = strength_branch_code) %>%
    dplyr::distinct(from, to)) %>%
  dplyr::mutate(
    branch =  factor(c(
      "Filter Branch: Outliers", "Filter Branch: Outliers",
      "Mutate Branch: Strength", "Mutate Branch: Strength", 
      "Mutate Branch: Strength", "Mutate Branch: Strength", 
      "Mutate Branch: Strength", "Mutate Branch: Strength"
    )),
    options = factor(c(1, 2, rep(1:3, 2)))
  )
froms <- unique(as.character(edges$from))
tos <- unique(as.character(edges$to))
nodes <- data.frame(
  name = unique(c(froms, tos)),
  branch = c("Filter Branch: Outliers", "Mutate Branch: Strength", "Mutate Branch: Strength", rep(NA, 3))
)
g <- graph_from_data_frame(edges, vertices = nodes)
ggraph(g, layout = 'dendrogram', circular = FALSE) +
  geom_edge_bend(aes(alpha = edge_attr(g, "options"), color = edge_attr(g, "branch")), width = 1, show.legend = FALSE) +
  geom_node_point(aes(color = branch), size = 4) +
  scale_color_discrete(name = NULL, labels = c(
    "Filter Branch: Outlier", "Mutate Branch: Strength", "Universes"
  )) +
  geom_node_text(aes(label = name),
                 vjust = -1.2, hjust = 1, nudge_y = -.05) +
  theme_void() +
  coord_flip() +
  scale_y_reverse(expand = c(0, .6)) +
  scale_x_continuous(expand = c(0, .5)) +
  theme(legend.position = "left")
```

Figure 1. Having one branch with 2 options and another with 3 results in 2 x 3 = 6 universes in total.

### Initiate

First, we start by loading the library and defining a `mverse` object with the dataset of interest. 

```{r echo=TRUE, message=FALSE}
library(mverse)
mv <- mverse(hurricane)
```

### Define Branches

We use the `*_branch()` methods to define branches. `filter_branch()` defines filtering operations using `dplyr::filter()` with different options for the filter. 

```{r echo=TRUE}
outliers <- filter_branch(
  ! Name %in% c("Katrina"),
  ! Name %in% c("Katrina", "Audrey")
)
```

`mutate_branch()` multiplexes `dplyr::mutate()` to add a new column in the dataset. 

```{r echo=TRUE}
strength <- mutate_branch(
  NDAM, HighestWindSpeed, Minpressure_Updated_2014)
```

In order to fit a Poisson regression, we need to specify the model using R's formula syntax and the underlying distribution using `family`. In `mverse`, we provide the specifications using `formula_branch()` and `family_branch()`. In this demonstration, we only define a single option for both formula and family but it is possible to provide multiple options for them as well. 

```{r echo=TRUE}
model <- formula_branch(alldeaths ~ strength * MasFem)
distribution <- family_branch(poisson)
```

### Add Branches

After defining the branches, we can add the branch objects to the `mverse` object using `add_*_branch()` methods.

```{r echo=TRUE}
mv <- mv %>%
  add_filter_branch(outliers) %>%
  add_mutate_branch(strength) %>%
  add_formula_branch(model) %>%
  add_family_branch(distribution)
```

### Fit Model

`glm_mverse()` multiplexes `stats::glm()` function and fits a GLM in each universe according to the specifications provided by `add_fomula_branch()` and `add_family_branch()`. 

```{r echo=TRUE}
mv <- mv %>% glm_mverse()
```


### Extract Results

After completing the analysis, we can extract the results using `summary()`. The method returns a table with branching options, estimates, 95\% confidence intervals for all regression terms across the multiverse. 

```{r echo=TRUE}
res <- summary(mv)
res
```

The resulting data is a `tibble` object and we can use regular `tidyverse` grammar to manipulate the data. In the code below, we specifically focus on the estimated coefficient for `MasFem` and its confidence intervals.

```{r echo=TRUE, message=FALSE}
library(dplyr)
res %>%
  filter(term == "MasFem") %>%
  select(outliers_branch, strength_branch, term, estimate, conf.low, conf.high)
```

### Plot a Specification Curve

We can also inspect the result graphically using `spec_curve()`. The method builds a specification curve     [@speccurve] for a term in the regression model specified by `var`. The method also allows multiple ways of sorting the estimates. See `?spec_curve` for details.

```{r echo=TRUE, dpi=144, fig.asp=1.5}
spec_summary(mv, var = "MasFem") %>% 
  spec_curve(spec_matrix_spacing = 4) +
  labs(colour = "Significant at 0.05")
```

## References

GitHub Events

Committers

Last synced: about 3 years ago

Issues and Pull Requests

Last synced: 7 months ago

Packages