Data

Tools

Indexes

Applications

Experiments

Open Source Science

bestNormalize

Estimate a suite of normalizing transformations, including a new adaptation of a technique based on ranks which can guarantee normally distributed transformed data if there are no ties: ordered quantile normalization (ORQ).

https://github.com/petersonr/bestnormalize

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    3 of 6 committers (50.0%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (16.2%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Estimate a suite of normalizing transformations, including a new adaptation of a technique based on ranks which can guarantee normally distributed transformed data if there are no ties: ordered quantile normalization (ORQ).

Basic Info
  • Host: GitHub
  • Owner: petersonR
  • Language: R
  • Default Branch: master
  • Homepage:
  • Size: 6.23 MB
Statistics
  • Stars: 41
  • Watchers: 5
  • Forks: 7
  • Open Issues: 1
  • Releases: 16
Created over 8 years ago · Last pushed over 2 years ago
Metadata Files
Readme

README.Rmd 

---
output: github_document
---



```{r, echo = FALSE, include=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "README-"
)
library(bestNormalize)
```


# bestNormalize: Flexibly calculate the best normalizing transformation for a vector

[![CRAN version](https://www.r-pkg.org/badges/version/bestNormalize)](https://cran.r-project.org/package=bestNormalize)
[![R-CMD-check](https://github.com/petersonR/bestNormalize/workflows/R-CMD-check/badge.svg)](https://github.com/petersonR/bestNormalize/actions)
[![downloads](https://cranlogs.r-pkg.org/badges/bestNormalize)](https://cran.r-project.org/package=bestNormalize)
[![Codecov test coverage](https://codecov.io/gh/petersonR/bestNormalize/branch/master/graph/badge.svg)](https://app.codecov.io/gh/petersonR/bestNormalize?branch=master)


The `bestNormalize` R package was designed to help find a normalizing transformation for a vector. There are many techniques that have been developed in this aim, however each has been subject to their own strengths/weaknesses, and it is unclear on how to decide which will work best until the data is observed. This package will look at a range of possible transformations and return the best one, i.e. the one that makes it look the *most* normal. 

Note that some authors use the term "normalize" differently than in this package. We define "normalize": to transform a vector of data in such a way that the transformed values follow a Gaussian distribution (or equivalently, a bell curve). This is in contrast to other such techniques designed to transform values to the 0-1 range, or to the -1 to 1 range. 

This package also introduces a new adaptation of a normalization technique, which we call Ordered Quantile normalization (`orderNorm()`, or ORQ). ORQ transforms the data based off of a rank mapping to the normal distribution.  This allows us to *guarantee* normally distributed transformed data (if ties are not present). The adaptation uses a shifted logit approximation on the ranks transformation to perform the transformation on newly observed data outside of the original domain. On new data within the original domain, the transformation uses linear interpolation of the fitted transformation.

To evaluate the efficacy of the normalization technique, the `bestNormalize()` function implements repeated cross-validation to estimate the Pearson's P statistic divided by its degrees of freedom. This is called the "Normality statistic", and if it is close to 1 (or less), then the transformation can be thought of as working well. The function is designed to select the transformation that produces the lowest P / df value, when estimated on out-of-sample data (estimating this on in-sample data will always choose the orderNorm technique, and is generally not the main goal of these procedures). 

## Installation

You can install the most recent (devel) version of bestNormalize from GitHub with:

```{r gh-installation, eval = FALSE}
# install.packages("devtools")
devtools::install_github("petersonR/bestNormalize")
```

Or, you can download it from CRAN with:

```{r cran-installation, eval = FALSE}
install.packages("bestNormalize")
```

## Example

In this example, we generate 1000 draws from a gamma distribution, and normalize them:

```{r, eval = FALSE}
library(bestNormalize)
```

```{r example}
set.seed(100)
x <- rgamma(1000, 1, 1)

# Estimate best transformation with repeated cross-validation
BN_obj <- bestNormalize(x, allow_lambert_s = TRUE)
BN_obj

# Perform transformation
gx <- predict(BN_obj)

# Perform reverse transformation
x2 <- predict(BN_obj, newdata = gx, inverse = TRUE)

# Prove the transformation is 1:1
all.equal(x2, x)

```

As of version 1.3, the package supports leave-one-out cross-validation as well. ORQ normalization works very well when the size of the test dataset is low relative to the training data set, so it will often be selected via leave-one-out cross-validation (which is why we set `allow_orderNorm = FALSE` here).

```{r}
(BN_loo <- bestNormalize(x, allow_orderNorm = FALSE, allow_lambert_s = TRUE, loo = TRUE))
```

It is also possible to visualize these transformations: 

```{r, eval = FALSE}
plot(BN_obj, leg_loc = "bottomright")
```


For a more in depth tutorial, please consult [the package vignette](https://CRAN.R-project.org/package=bestNormalize/vignettes/bestNormalize.html), or the [package website](https://petersonr.github.io/bestNormalize/).

Owner

  • Name: Ryan Peterson
  • Login: petersonR
  • Kind: user
  • Location: Denver, CO

Assistant Professor of Biostatistics at the Colorado School of Public Health at the University of Colorado.

GitHub Events

Total
  • Issues event: 1
  • Watch event: 1
  • Issue comment event: 1
Last Year
  • Issues event: 1
  • Watch event: 1
  • Issue comment event: 1

Committers

Last synced: 10 months ago

All Time
  • Total Commits: 181
  • Total Committers: 6
  • Avg Commits per committer: 30.167
  • Development Distribution Score (DDS): 0.525
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Ryan Peterson r****n@u****u 86
Ryan Peterson r****n@c****u 74
Ryan Peterson r****2@g****m 15
rempsyc r****t@m****a 3
Max Kuhn m****n@g****m 2
Emil Hvitfeldt e****t@g****m 1
Committer Domains (Top 20 + Academic)
mail.mcgill.ca: 1 cuanschutz.edu: 1 uiowa.edu: 1

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 20
  • Total pull requests: 5
  • Average time to close issues: about 2 months
  • Average time to close pull requests: 17 days
  • Total issue authors: 16
  • Total pull request authors: 4
  • Average comments per issue: 2.4
  • Average comments per pull request: 2.4
  • Merged pull requests: 5
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 1
  • Pull requests: 0
  • Average time to close issues: 2 minutes
  • Average time to close pull requests: N/A
  • Issue authors: 1
  • Pull request authors: 0
  • Average comments per issue: 1.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • topepo (3)
  • rempsyc (2)
  • gmgeorg (2)
  • Shmuppel (1)
  • guidohooiveld (1)
  • shabbybanks (1)
  • santarabantoosoo (1)
  • VladPerervenko (1)
  • navankurverma (1)
  • suntzuisafterU (1)
  • cardiomoon (1)
  • kadyb (1)
  • lizlaw (1)
  • EmilHvitfeldt (1)
  • MelissaRS (1)
Pull Request Authors
  • topepo (2)
  • EmilHvitfeldt (1)
  • rempsyc (1)
  • petersonR (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 2
  • Total downloads:
    • cran 4,410 last-month
  • Total docker downloads: 74
  • Total dependent packages: 5
    (may contain duplicates)
  • Total dependent repositories: 17
    (may contain duplicates)
  • Total versions: 25
  • Total maintainers: 1
cran.r-project.org: bestNormalize

Normalizing Transformation Functions

  • Versions: 19
  • Dependent Packages: 5
  • Dependent Repositories: 17
  • Downloads: 4,410 Last month
  • Docker Downloads: 74
Rankings
Dependent repos count: 6.9%
Downloads: 8.0%
Stargazers count: 8.5%
Average: 8.6%
Forks count: 8.8%
Dependent packages count: 10.7%
Maintainers (1)
ryan.a.peterson@cuanschutz.edu
Last synced: 10 months ago
conda-forge.org: r-bestnormalize
  • Versions: 6
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 34.0%
Stargazers count: 41.4%
Average: 43.6%
Forks count: 47.7%
Dependent packages count: 51.2%
Last synced: 10 months ago

Dependencies

DESCRIPTION cran
  • R >= 3.1.0 depends
  • LambertW >= 0.6.5 imports
  • butcher * imports
  • doParallel * imports
  • doRNG * imports
  • dplyr * imports
  • foreach * imports
  • methods * imports
  • nortest * imports
  • purrr * imports
  • recipes * imports
  • tibble * imports
  • MASS * suggests
  • covr * suggests
  • ggplot2 * suggests
  • knitr * suggests
  • mgcv * suggests
  • parallel * suggests
  • rlang * suggests
  • rmarkdown * suggests
  • scales * suggests
  • testthat * suggests
.github/workflows/R-CMD-check.yaml actions
  • actions/cache v2 composite
  • actions/checkout v2 composite
  • actions/upload-artifact main composite
  • r-lib/actions/setup-pandoc v1 composite
  • r-lib/actions/setup-r v1 composite
.github/workflows/pkgdown.yaml actions
  • actions/cache v2 composite
  • actions/checkout v2 composite
  • r-lib/actions/setup-pandoc v1 composite
  • r-lib/actions/setup-r v1 composite
.github/workflows/pr-commands.yaml actions
  • actions/checkout v2 composite
  • r-lib/actions/pr-fetch v1 composite
  • r-lib/actions/pr-push v1 composite
  • r-lib/actions/setup-r v1 composite
.github/workflows/test-coverage.yaml actions
  • actions/cache v2 composite
  • actions/checkout v2 composite
  • r-lib/actions/setup-r v1 composite