datawizard

datawizard: An R Package for Easy Data Preparation and Statistical Transformations - Published in JOSS (2022)

https://github.com/easystats/datawizard

Keywords

Keywords from Contributors

Scientific Fields

Repository

Magic potions to clean and transform your data 🧙

README.Rmd

---
output: github_document
---

# `datawizard`: Easy Data Wrangling and Statistical Transformations 

```{r, echo=FALSE, warning=FALSE, message=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  dpi = 300,
  out.width = "100%",
  fig.path = "man/figures/",
  comment = "#>"
)

set.seed(333)
library(datawizard)
```

[![DOI](https://joss.theoj.org/papers/10.21105/joss.04684/status.svg)](https://doi.org/10.21105/joss.04684)
[![downloads](https://cranlogs.r-pkg.org/badges/datawizard)](https://cran.r-project.org/package=datawizard)
[![total](https://cranlogs.r-pkg.org/badges/grand-total/datawizard)](https://cranlogs.r-pkg.org/)









`{datawizard}` is a lightweight package to easily manipulate, clean, transform, and prepare your data for analysis. It is part of the [easystats ecosystem](https://easystats.github.io/easystats/), a suite of R packages to deal with your entire statistical analysis, from cleaning the data to reporting the results.

It covers two aspects of data preparation:

- **Data manipulation**: `{datawizard}` offers a very similar set of functions to that of the *tidyverse* packages, such as a `{dplyr}` and `{tidyr}`, to select, filter and reshape data, with a few key differences. 1) All data manipulation functions start with the prefix `data_*` (which makes them easy to identify). 2) Although most functions can be used exactly as their *tidyverse* equivalents, they are also string-friendly (which makes them easy to program with and use inside functions). Finally, `{datawizard}` is super lightweight (no dependencies, similar to [poorman](https://github.com/nathaneastwood/poorman)), which makes it awesome for developers to use in their packages.

- **Statistical transformations**: `{datawizard}` also has powerful functions to easily apply common data [transformations](https://easystats.github.io/datawizard/reference/index.html#statistical-transformations), including standardization, normalization, rescaling, rank-transformation, scale reversing, recoding, binning, etc.











# Installation

[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/datawizard)](https://cran.r-project.org/package=datawizard) [![datawizard status badge](https://easystats.r-universe.dev/badges/datawizard)](https://easystats.r-universe.dev) [![codecov](https://codecov.io/gh/easystats/datawizard/branch/main/graph/badge.svg)](https://app.codecov.io/gh/easystats/datawizard) [![R-CMD-check](https://github.com/easystats/datawizard/workflows/R-CMD-check/badge.svg?branch=main)](https://github.com/easystats/datawizard/actions)

Type | Source | Command
---|---|---
Release | CRAN | `install.packages("datawizard")`
Development | r-universe | `install.packages("datawizard", repos = "https://easystats.r-universe.dev")`
Development | GitHub | `remotes::install_github("easystats/datawizard")`

> **Tip**
>
> **Instead of `library(datawizard)`, use `library(easystats)`.**
> **This will make all features of the  easystats-ecosystem available.**
>
> **To stay updated, use `easystats::install_latest()`.**

# Citation

To cite the package, run the following command:

```{r, comment=""}
citation("datawizard")
```

# Features

[![Documentation](https://img.shields.io/badge/documentation-datawizard-orange.svg?colorB=E91E63)](https://easystats.github.io/datawizard/)
[![Blog](https://img.shields.io/badge/blog-easystats-orange.svg?colorB=FF9800)](https://easystats.github.io/blog/posts/)
[![Features](https://img.shields.io/badge/features-datawizard-orange.svg?colorB=2196F3)](https://easystats.github.io/datawizard/reference/index.html)

Most courses and tutorials about statistical modeling assume that you are working with a clean and tidy dataset. In practice, however, a major part of doing statistical modeling is preparing your data--cleaning up values, creating new columns, reshaping the dataset, or transforming some variables. `{datawizard}` provides easy to use tools to perform these common, critical, and sometimes tedious data preparation tasks.

## Data wrangling

### Select, filter and remove variables

The package provides helpers to filter rows meeting certain conditions...

```{r}
data_match(mtcars, data.frame(vs = 0, am = 1))
```

... or logical expressions:

```{r}
data_filter(mtcars, vs == 0 & am == 1)
```

Finding columns in a data frame, or retrieving the data of selected columns, can be  achieved using `extract_column_names()` or `data_select()`:

```{r}
# find column names matching a pattern
extract_column_names(iris, starts_with("Sepal"))

# return data columns matching a pattern
data_select(iris, starts_with("Sepal")) |> head()
```

It is also possible to extract one or more variables:

```{r}
# single variable
data_extract(mtcars, "gear")

# more variables
head(data_extract(iris, ends_with("Width")))
```

Due to the consistent API, removing variables is just as simple:

```{r}
head(data_remove(iris, starts_with("Sepal")))
```

### Reorder or rename

```{r}
head(data_relocate(iris, select = "Species", before = "Sepal.Length"))
```

```{r}
head(data_rename(iris, c("Sepal.Length", "Sepal.Width"), c("length", "width")))
```

### Merge

```{r}
x <- data.frame(a = 1:3, b = c("a", "b", "c"), c = 5:7, id = 1:3)
y <- data.frame(c = 6:8, d = c("f", "g", "h"), e = 100:102, id = 2:4)

x
y

data_merge(x, y, join = "full")

data_merge(x, y, join = "left")

data_merge(x, y, join = "right")

data_merge(x, y, join = "semi", by = "c")

data_merge(x, y, join = "anti", by = "c")

data_merge(x, y, join = "inner")

data_merge(x, y, join = "bind")
```

### Reshape

A common data wrangling task is to reshape data.

Either to go from wide/Cartesian to long/tidy format

```{r}
wide_data <- data.frame(replicate(5, rnorm(10)))

head(data_to_long(wide_data))
```

or the other way

```{r}
long_data <- data_to_long(wide_data, rows_to = "Row_ID") # Save row number

data_to_wide(long_data,
  names_from = "name",
  values_from = "value",
  id_cols = "Row_ID"
)
```

### Empty rows and columns

```{r}
tmp <- data.frame(
  a = c(1, 2, 3, NA, 5),
  b = c(1, NA, 3, NA, 5),
  c = c(NA, NA, NA, NA, NA),
  d = c(1, NA, 3, NA, 5)
)

tmp

# indices of empty columns or rows
empty_columns(tmp)
empty_rows(tmp)

# remove empty columns or rows
remove_empty_columns(tmp)
remove_empty_rows(tmp)

# remove empty columns and rows
remove_empty(tmp)
```

### Recode or cut dataframe

```{r}
set.seed(123)
x <- sample(1:10, size = 50, replace = TRUE)

table(x)

# cut into 3 groups, based on distribution (quantiles)
table(categorize(x, split = "quantile", n_groups = 3))
```

## Data Transformations

The packages also contains multiple functions to help transform data.

### Standardize

For example, to standardize (*z*-score) data:

```{r}
# before
summary(swiss)

# after
summary(standardize(swiss))
```

### Winsorize

To winsorize data:

```{r}
# before
anscombe

# after
winsorize(anscombe)
```

### Center

To grand-mean center data

```{r}
center(anscombe)
```

### Ranktransform

To rank-transform data:

```{r}
# before
head(trees)

# after
head(ranktransform(trees))
```

### Rescale

To rescale a numeric variable to a new range:

```{r}
change_scale(c(0, 1, 5, -5, -2))
```

### Rotate or transpose

```{r}
x <- mtcars[1:3, 1:4]

x

data_rotate(x)
```


## Data properties

`datawizard` provides a way to provide comprehensive descriptive summary for all variables in a dataframe:

```{r}
data(iris)
describe_distribution(iris)
```

Or even just a variable

```{r}
describe_distribution(mtcars$wt)
```

There are also some additional data properties that can be computed using this package.

```{r}
x <- (-10:10)^3 + rnorm(21, 0, 100)
smoothness(x, method = "diff")
```

## Function design and pipe-workflow

The design of the `{datawizard}` functions follows a design principle that makes it easy for user to understand and remember how functions work:

1. the first argument is the data
2. for methods that work on data frames, two arguments are following to `select` and `exclude` variables
3. the following arguments are arguments related to the specific tasks of the functions

Most important, functions that accept data frames usually have this as their first argument, and also return a (modified) data frame again. Thus, `{datawizard}` integrates smoothly into a "pipe-workflow".

```{r}
iris |>
  # all rows where Species is "versicolor" or "virginica"
  data_filter(Species %in% c("versicolor", "virginica")) |>
  # select only columns with "." in names (i.e. drop Species)
  data_select(contains("\\.")) |>
  # move columns that ends with "Length" to start of data frame
  data_relocate(ends_with("Length")) |>
  # remove fourth column
  data_remove(4) |>
  head()
```

# Contributing and Support

In case you want to file an issue or contribute in another way to the package, please follow [this guide](https://easystats.github.io/datawizard/CONTRIBUTING.html). For questions about the functionality, you may either contact us via email or also file an issue.

# Code of Conduct

Please note that this project is released with a
[Contributor Code of Conduct](https://easystats.github.io/datawizard/CODE_OF_CONDUCT.html). By participating in this project you agree to abide by its terms.

Owner

JOSS Publication

datawizard: An R Package for Easy Data Preparation and Statistical Transformations

Published

October 09, 2022

DOI

10.21105/joss.04684

Volume 7, Issue 78, Page 4684

Authors

Indrajeet Patil
cynkra Analytics GmbH, Germany

Dominique Makowski
Nanyang Technological University, Singapore

Mattan S. Ben-Shachar
Ben-Gurion University of the Negev, Israel

Brenton M. Wiernik
Independent Researcher

Etienne Bacher
Luxembourg Institute of Socio-Economic Research (LISER), Luxembourg

Daniel Lüdecke
University Medical Center Hamburg-Eppendorf, Germany

Editor

Øystein Sørensen

Tags

easystats

View PDF Review Thread Software Archive

GitHub Events

Committers

Last synced: 5 months ago

Issues and Pull Requests

Last synced: 4 months ago

Packages