Data

Tools

Indexes

Applications

Experiments

Open Source Science

widyr

Widen, process, and re-tidy a dataset

https://github.com/juliasilge/widyr

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 12 committers (8.3%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (17.7%) to scientific vocabulary

Keywords from Contributors

tidyverse tidy-data ggplot-extension transition data-manipulation grammar
Last synced: 11 months ago · JSON representation

Repository

Widen, process, and re-tidy a dataset

Basic Info
Statistics
  • Stars: 329
  • Watchers: 14
  • Forks: 29
  • Open Issues: 18
  • Releases: 0
Created about 10 years ago · Last pushed over 3 years ago
Metadata Files
Readme License Code of conduct

README.Rmd 

---
output: github_document
---




```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%",
  message = FALSE
)
suppressPackageStartupMessages(library(dplyr))
```

# widyr: Widen, process, and re-tidy a dataset

**Authors:** [Julia Silge](https://juliasilge.com/), [David Robinson](http://varianceexplained.org/)

**License:** [MIT](https://opensource.org/licenses/MIT)


[![R-CMD-check](https://github.com/juliasilge/widyr/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/juliasilge/widyr/actions/workflows/R-CMD-check.yaml)
[![CRAN_Status_Badge](http://www.r-pkg.org/badges/version/widyr)](https://cran.r-project.org/package=widyr)
[![Codecov test coverage](https://codecov.io/gh/juliasilge/widyr/branch/main/graph/badge.svg)](https://app.codecov.io/gh/juliasilge/widyr?branch=main)


This package wraps the pattern of un-tidying data into a wide matrix, performing some processing, then turning it back into a tidy form. This is useful for several mathematical operations such as co-occurrence counts, correlations, or clustering that are best done on a wide matrix.

## Installation

You can install the released version of widyr from [CRAN](https://CRAN.R-project.org) with:

``` r
install.packages("widyr")
```

And the development version from [GitHub](https://github.com/) with:

``` r
# install.packages("devtools")
devtools::install_github("juliasilge/widyr")
```

## Towards a precise definition of "wide" data

The term "wide data" has gone out of fashion as being "imprecise" [(Wickham 2014)](http://vita.had.co.nz/papers/tidy-data.pdf), but I think with a proper definition the term could be entirely meaningful and useful.

A **wide** dataset is one or more matrices where:

* Each row is one **item**
* Each column is one **feature**
* Each value is one **observation**
* Each matrix is one **variable**

When would you want data to be wide rather than tidy? Notable examples include classification, clustering, correlation, factorization, or other operations that can take advantage of a matrix structure. In general, when you want to **compare between pairs of items** rather than compare between variables or between groups of observations, this is a useful structure.

The widyr package is based on the observation that during a tidy data analysis, you often want data to be wide only *temporarily*, before returning to a tidy structure for visualization and further analysis. widyr makes this easy through a set of `pairwise_` functions.

## Example: gapminder

Consider the gapminder dataset in the [gapminder package](https://CRAN.R-project.org/package=gapminder).

```{r}
library(dplyr)
library(gapminder)

gapminder
```

This tidy format (one-row-per-country-per-year) is very useful for grouping, summarizing, and filtering operations. But if we want to *compare* countries (for example, to find countries that are similar to each other), we would have to reshape this dataset. Note that here, each country is an **item**, while each year is the **feature**.

#### Pairwise operations

The widyr package offers `pairwise_` functions that operate on pairs of items within data. An example is `pairwise_dist`:

```{r}
library(widyr)

gapminder %>%
  pairwise_dist(country, year, lifeExp)
```

This finds the Euclidean distance between the `lifeExp` value in each pair of countries. It knows which values to compare between countries with `year`, which is the feature column.

We could find the closest pairs of countries overall with `arrange()`:

```{r}
gapminder %>%
  pairwise_dist(country, year, lifeExp) %>%
  arrange(distance)
```

Notice that this includes duplicates (Germany/Belgium and Belgium/Germany). To avoid those (the upper triangle of the distance matrix), use `upper = FALSE`:

```{r}
gapminder %>%
  pairwise_dist(country, year, lifeExp, upper = FALSE) %>%
  arrange(distance)
```

In some analyses, we may be interested in correlation rather than distance of pairs. For this we would use `pairwise_cor`:

```{r}
gapminder %>%
  pairwise_cor(country, year, lifeExp, upper = FALSE)
```

### Code of Conduct

This project is released with a [Contributor Code of Conduct](https://www.contributor-covenant.org/version/2/1/CODE_OF_CONDUCT.html). By contributing to this project, you agree to abide by its terms.

Owner

  • Name: Julia Silge
  • Login: juliasilge
  • Kind: user
  • Location: Salt Lake City, UT
  • Company: @posit-pbc

Data science and MLOps with #rstats, text mining, 💖

GitHub Events

Total
  • Watch event: 4
Last Year
  • Watch event: 4

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 89
  • Total Committers: 12
  • Avg Commits per committer: 7.417
  • Development Distribution Score (DDS): 0.629
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
David Robinson a****d@g****m 33
Julia Silge j****e@g****m 20
dgrtwo d****o@p****u 18
Dave Robinson d****n@s****m 9
Kanishka Misra m****e@g****m 2
Simon P. Couch s****h@g****m 1
Matt Dupree k****3@g****m 1
Lukas Jung j****s@g****t 1
Katie Daisey k****y@y****m 1
Ben Marwick b****k@h****m 1
mbeigel m****r@g****m 1
David Robinson d****n@D****e 1
Committer Domains (Top 20 + Academic)
davids-mbp.fios-router.home: 1 gmx.net: 1 stackoverflow.com: 1 princeton.edu: 1

Issues and Pull Requests

Last synced: 12 months ago

All Time
  • Total issues: 32
  • Total pull requests: 14
  • Average time to close issues: about 1 year
  • Average time to close pull requests: 10 months
  • Total issue authors: 31
  • Total pull request authors: 10
  • Average comments per issue: 1.63
  • Average comments per pull request: 2.29
  • Merged pull requests: 8
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • hope-data-science (2)
  • lhdjung (1)
  • ghost (1)
  • DiabbZegpi (1)
  • michaelgaunt404 (1)
  • dhalpern (1)
  • sebastian-fox (1)
  • sjspielman (1)
  • dsobolew (1)
  • dgrtwo (1)
  • jeffreyxparker (1)
  • aalsharef (1)
  • wvictor14 (1)
  • kanishkamisra (1)
  • double-y (1)
Pull Request Authors
  • kanishkamisra (3)
  • juliasilge (3)
  • Katiedaisey (1)
  • twolodzko (1)
  • benmarwick (1)
  • kmdupr33 (1)
  • lhdjung (1)
  • simonpcouch (1)
  • strboul (1)
  • brooklynbagel (1)
Top Labels
Issue Labels
enhancement (1)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • cran 1,836 last-month
  • Total docker downloads: 1,441
  • Total dependent packages: 3
  • Total dependent repositories: 21
  • Total versions: 6
  • Total maintainers: 1
cran.r-project.org: widyr

Widen, Process, then Re-Tidy Data

  • Versions: 6
  • Dependent Packages: 3
  • Dependent Repositories: 21
  • Downloads: 1,836 Last month
  • Docker Downloads: 1,441
Rankings
Stargazers count: 1.2%
Forks count: 2.6%
Dependent repos count: 6.1%
Downloads: 8.6%
Average: 9.0%
Dependent packages count: 10.9%
Docker downloads count: 24.9%
Maintainers (1)
julia.silge@gmail.com
Last synced: 11 months ago

Dependencies

DESCRIPTION cran
  • Matrix * imports
  • broom * imports
  • dplyr * imports
  • purrr * imports
  • reshape2 * imports
  • rlang * imports
  • tibble * imports
  • tidyr * imports
  • tidytext * imports
  • countrycode * suggests
  • covr * suggests
  • fuzzyjoin * suggests
  • gapminder * suggests
  • ggplot2 * suggests
  • ggraph * suggests
  • igraph * suggests
  • irlba * suggests
  • janeaustenr * suggests
  • knitr * suggests
  • maps * suggests
  • rmarkdown * suggests
  • testthat * suggests
  • unvotes >= 0.3.0 suggests