jlmerclusterperm
Fast cluster-based permutation test for densely-sampled, multi-level time series data (ex: eyetracking, EEG)
Science Score: 49.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 9 DOI reference(s) in README -
✓Academic publication links
Links to: zenodo.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (18.3%) to scientific vocabulary
Keywords
cluster-based-permutation-test
eeg
eyetracking
mixed-effects-models
timeseries
Last synced: 6 months ago
·
JSON representation
Repository
Fast cluster-based permutation test for densely-sampled, multi-level time series data (ex: eyetracking, EEG)
Basic Info
- Host: GitHub
- Owner: yjunechoe
- License: other
- Language: R
- Default Branch: main
- Homepage: https://yjunechoe.github.io/jlmerclusterperm/
- Size: 19.2 MB
Statistics
- Stars: 13
- Watchers: 1
- Forks: 1
- Open Issues: 5
- Releases: 14
Topics
cluster-based-permutation-test
eeg
eyetracking
mixed-effects-models
timeseries
Created almost 3 years ago
· Last pushed 12 months ago
Metadata Files
Readme
Changelog
Contributing
License
Code of conduct
Codemeta
README.Rmd
---
output: github_document
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.align = "center",
fig.path = "man/figures/README-",
asciicast_theme = if (Sys.getenv("IN_PKGDOWN") == "true") "pkgdown" else "readme"
)
asciicast::init_knitr_engine(
echo = TRUE,
echo_input = FALSE,
timeout = 60,
record_env = c("asciicast_cols" = 80),
same_process = TRUE
)
```
# jlmerclusterperm
[](https://CRAN.R-project.org/package=jlmerclusterperm)
[](https://yjunechoe.r-universe.dev/jlmerclusterperm)
[](https://github.com/yjunechoe/jlmerclusterperm/actions/workflows/R-CMD-check.yaml)
[](https://github.com/yjunechoe/jlmerclusterperm/actions?query=workflow%3Apkgcheck)
[](https://app.codecov.io/gh/yjunechoe/jlmerclusterperm?branch=main)
[](https://cranlogs.r-pkg.org/badges/grand-total/jlmerclusterperm)
Julia [GLM.jl](https://github.com/JuliaStats/GLM.jl) and [MixedModels.jl](https://github.com/JuliaStats/MixedModels.jl) based implementation of the cluster-based permutation test for time series data, powered by [JuliaConnectoR](https://github.com/stefan-m-lenz/JuliaConnectoR).
```{r, echo = FALSE}
knitr::include_graphics("man/figures/clusterpermute_animation.gif", error = FALSE)
```
## Installation and usage
### Zero-setup test drive
As of March 2025, **Google Colab** supports Julia. This means `{jlmerclusterperm}` *just works* out of the box. Try it out in a [demo notebook](https://colab.research.google.com/drive/1pTXGbuoQKka5Tm8qnyaHrMHs0Z-ALD7k?usp=sharing) that runs some of the code from the [Ito et al. 2018 case study vignette](https://yjunechoe.github.io/jlmerclusterperm/articles/Ito-et-al-2018.html).
### Local setup
Install the released version of jlmerclusterperm from CRAN:
```{r, eval = FALSE}
install.packages("jlmerclusterperm")
```
Or install the development version from [GitHub](https://github.com/yjunechoe/jlmerclusterperm) with:
```{r, eval = FALSE}
# install.packages("remotes")
remotes::install_github("yjunechoe/jlmerclusterperm")
```
Using `jlmerclusterperm` requires a prior installation of the Julia programming language, which can be downloaded from either the [official website](https://julialang.org/) or using the command line utility [juliaup](https://github.com/JuliaLang/juliaup). Julia version >=1.8 is required and [1.9](https://julialang.org/blog/2023/04/julia-1.9-highlights/#caching_of_native_code) or higher is preferred for the substantial speed improvements.
Before using functions from `jlmerclusterperm`, an initial setup is required via calling `jlmerclusterperm_setup()`. The very first call on a system will install necessary dependencies (this only happens once and takes around 10-15 minutes).
Subsequent calls to `jlmerclusterperm_setup()` incur a small overhead of around 30 seconds, plus slight delays for first-time function calls. You pay up front for start-up and warm-up costs and get blazingly-fast functions from the package.
```{r setup, include = FALSE}
library(jlmerclusterperm)
jlmerclusterperm_setup(cache_dir = tempdir())
```
```{asciicast setup-io}
# Both lines must be run at the start of each new session
library(jlmerclusterperm)
jlmerclusterperm_setup()
```
See the [Get Started](https://yjunechoe.github.io/jlmerclusterperm/articles/jlmerclusterperm.html) page on the [package website](https://yjunechoe.github.io/jlmerclusterperm/) for background and tutorials.
## Quick tour of package functionalities
### Wholesale CPA with `clusterpermute()`
A time series data:
```{r chickweight, out.width = "75%"}
chickweights <- ChickWeight
chickweights$Time <- as.integer(factor(chickweights$Time))
matplot(
tapply(chickweights$weight, chickweights[c("Time", "Diet")], mean),
type = "b", lwd = 3, ylab = "Weight", xlab = "Time"
)
```
```{asciicast chickweight-io, include = FALSE}
chickweights <- ChickWeight
chickweights$Time <- as.integer(factor(chickweights$Time))
```
Preparing a specification object with `make_jlmer_spec()`:
```{r spec, include = FALSE}
chickweights_spec <- make_jlmer_spec(
formula = weight ~ 1 + Diet,
data = chickweights,
subject = "Chick", time = "Time"
)
```
```{asciicast spec-io}
chickweights_spec <- make_jlmer_spec(
formula = weight ~ 1 + Diet,
data = chickweights,
subject = "Chick", time = "Time"
)
chickweights_spec
```
Cluster-based permutation test with `clusterpermute()`:
```{asciicast JIT, include = FALSE}
clusterpermute(chickweights_spec, threshold = 2.5, nsim = 2)
```
```{asciicast CPA-io}
set_rng_state(123L)
clusterpermute(
chickweights_spec,
threshold = 2.5,
nsim = 100
)
```
Including random effects:
```{asciicast reCPA-io}
chickweights_re_spec <- make_jlmer_spec(
formula = weight ~ 1 + Diet + (1 | Chick),
data = chickweights,
subject = "Chick", time = "Time"
)
set_rng_state(123L)
clusterpermute(
chickweights_re_spec,
threshold = 2.5,
nsim = 100
)$empirical_clusters
```
### Piecemeal approach to CPA
Computing time-wise statistics of the observed data:
```{r empirical_statistics, out.width = "75%"}
empirical_statistics <- compute_timewise_statistics(chickweights_spec)
matplot(t(empirical_statistics), type = "b", pch = 1, lwd = 3, ylab = "t-statistic")
abline(h = 2.5, lty = 3)
```
```{asciicast empirical_statistics-io, include = FALSE}
empirical_statistics <- compute_timewise_statistics(chickweights_spec)
```
Identifying empirical clusters:
```{asciicast empirical_clusters}
empirical_clusters <- extract_empirical_clusters(empirical_statistics, threshold = 2.5)
empirical_clusters
```
Simulating the null distribution:
```{asciicast null_statistics}
set_rng_state(123L)
null_statistics <- permute_timewise_statistics(chickweights_spec, nsim = 100)
null_cluster_dists <- extract_null_cluster_dists(null_statistics, threshold = 2.5)
null_cluster_dists
```
Significance testing the cluster-mass statistic:
```{asciicast calculate_clusters_pvalues}
calculate_clusters_pvalues(empirical_clusters, null_cluster_dists, add1 = TRUE)
```
Iterating over a range of threshold values:
```{asciicast walk_threshold_steps, message = FALSE}
walk_threshold_steps(empirical_statistics, null_statistics, steps = c(2, 2.5, 3))
```
## Acknowledgments
- The paper [Maris & Oostenveld (2007)](https://doi.org/10.1016/j.jneumeth.2007.03.024) which originally proposed the cluster-based permutation analysis.
- The [JuliaConnectoR](https://github.com/stefan-m-lenz/JuliaConnectoR) package for powering the R interface to Julia.
- The Julia packages [GLM.jl](https://github.com/JuliaStats/GLM.jl) and [MixedModels.jl](https://github.com/JuliaStats/MixedModels.jl) for fast implementations of (mixed effects) regression models.
- Existing implementations of CPA in R ([permuco](https://jaromilfrossard.github.io/permuco/), [permutes](https://cran.r-project.org/package=permutes), etc.) whose designs inspired the CPA interface in jlmerclusterperm.
## Citations
If you use jlmerclusterperm for cluster-based permutation test with mixed-effects models in your research, please cite one (or more) of the following as you see fit.
To cite jlmerclusterperm:
- Choe, J. (`r format(Sys.Date(), "%Y")`). jlmerclusterperm: Cluster-Based Permutation Analysis for Densely Sampled Time Data. R package version `r as.character(utils::packageVersion('jlmerclusterperm'))`. [10.32614/CRAN.package.jlmerclusterperm](https://doi.org/10.32614/CRAN.package.jlmerclusterperm).
To cite the cluster-based permutation test:
- Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG- and MEG-data. _Journal of Neuroscience Methods, 164_, 177–190. doi: 10.1016/j.jneumeth.2007.03.024.
To cite the Julia programming language:
- Bezanson, J., Edelman, A., Karpinski, S., & Shah, V. B. (2017). Julia: A Fresh Approach to Numerical Computing. _SIAM Review, 59_(1), 65–98. doi: 10.1137/141000671.
To cite the GLM.jl and MixedModels.jl Julia libraries, consult their Zenodo pages:
- GLM: https://doi.org/10.5281/zenodo.3376013
- MixedModels: https://zenodo.org/badge/latestdoi/9106942
```{asciicast close-io, include = FALSE}
JuliaConnectoR::stopJulia()
```
```{r close, include = FALSE}
JuliaConnectoR::stopJulia()
```
```{r srr, include = FALSE}
#' @srrstats {G1.0} References Maris & Oostenveld (2007) which originally proposed the cluster-based permutation analysis.
#' @srrstats {G1.1} Package is an improvement over existing implementations in R (mainly in speed and interpretability).
#' This is explained in the readme and the case study vignettes.
#' @srrstats {G1.2} Lifecycle is active and stable.
#' @srrstats {G1.3} The many moving parts are explained across the readme, the function documentation, and the topics/case study vignettes.
#' Users are assumed to already have familiarity with (genearlized, mixed-effects) regression to use this package.
#' @srrstats {G1.4} `roxygen2` is used throughout the package.
```
Owner
- Name: June Choe
- Login: yjunechoe
- Kind: user
- Website: yjunechoe.github.io
- Twitter: yjunechoe
- Repositories: 13
- Profile: https://github.com/yjunechoe
PhD student in Linguistics at the University of Pennsylvania
CodeMeta (codemeta.json)
{
"@context": "https://doi.org/10.5063/schema/codemeta-2.0",
"@type": "SoftwareSourceCode",
"identifier": "jlmerclusterperm",
"description": "An implementation of fast cluster-based permutation analysis (CPA) for densely-sampled time data developed in Maris & Oostenveld, 2007 <doi:10.1016/j.jneumeth.2007.03.024>. Supports (generalized, mixed-effects) regression models for the calculation of timewise statistics. Provides both a wholesale and a piecemeal interface to the CPA procedure with an emphasis on interpretability and diagnostics. Integrates 'Julia' libraries 'MixedModels.JL' and 'GLM.JL' for performance improvements, with additional functionalities for interfacing with 'Julia' from 'R' powered by the 'JuliaConnectoR' package.",
"name": "jlmerclusterperm: Cluster-Based Permutation Analysis for Densely Sampled Time Data",
"relatedLink": "https://yjunechoe.github.io/jlmerclusterperm/",
"codeRepository": "https://github.com/yjunechoe/jlmerclusterperm",
"issueTracker": "https://github.com/yjunechoe/jlmerclusterperm/issues",
"license": "https://spdx.org/licenses/MIT",
"version": "1.0.0",
"programmingLanguage": {
"@type": "ComputerLanguage",
"name": "R",
"url": "https://r-project.org"
},
"runtimePlatform": "R version 4.3.0 (2023-04-21 ucrt)",
"author": [
{
"@type": "Person",
"givenName": "June",
"familyName": "Choe",
"email": "jchoe001@gmail.com",
"@id": "https://orcid.org/0000-0002-0701-921X"
}
],
"copyrightHolder": [
{
"@type": "Person",
"givenName": "June",
"familyName": "Choe",
"email": "jchoe001@gmail.com",
"@id": "https://orcid.org/0000-0002-0701-921X"
}
],
"maintainer": [
{
"@type": "Person",
"givenName": "June",
"familyName": "Choe",
"email": "jchoe001@gmail.com",
"@id": "https://orcid.org/0000-0002-0701-921X"
}
],
"softwareSuggestions": [
{
"@type": "SoftwareApplication",
"identifier": "broom",
"name": "broom",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=broom"
},
{
"@type": "SoftwareApplication",
"identifier": "broom.mixed",
"name": "broom.mixed",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=broom.mixed"
},
{
"@type": "SoftwareApplication",
"identifier": "covr",
"name": "covr",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=covr"
},
{
"@type": "SoftwareApplication",
"identifier": "dplyr",
"name": "dplyr",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=dplyr"
},
{
"@type": "SoftwareApplication",
"identifier": "eyetrackingR",
"name": "eyetrackingR",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=eyetrackingR"
},
{
"@type": "SoftwareApplication",
"identifier": "forcats",
"name": "forcats",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=forcats"
},
{
"@type": "SoftwareApplication",
"identifier": "future",
"name": "future",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=future"
},
{
"@type": "SoftwareApplication",
"identifier": "ggplot2",
"name": "ggplot2",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=ggplot2"
},
{
"@type": "SoftwareApplication",
"identifier": "knitr",
"name": "knitr",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=knitr"
},
{
"@type": "SoftwareApplication",
"identifier": "MASS",
"name": "MASS",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=MASS"
},
{
"@type": "SoftwareApplication",
"identifier": "patchwork",
"name": "patchwork",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=patchwork"
},
{
"@type": "SoftwareApplication",
"identifier": "peekbankr",
"name": "peekbankr",
"sameAs": "https://github.com/langcog/peekbankr"
},
{
"@type": "SoftwareApplication",
"identifier": "readr",
"name": "readr",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=readr"
},
{
"@type": "SoftwareApplication",
"identifier": "rmarkdown",
"name": "rmarkdown",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=rmarkdown"
},
{
"@type": "SoftwareApplication",
"identifier": "scales",
"name": "scales",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=scales"
},
{
"@type": "SoftwareApplication",
"identifier": "testthat",
"name": "testthat",
"version": ">= 3.0.0",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=testthat"
},
{
"@type": "SoftwareApplication",
"identifier": "tibble",
"name": "tibble",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=tibble"
}
],
"softwareRequirements": {
"1": {
"@type": "SoftwareApplication",
"identifier": "R",
"name": "R",
"version": ">= 3.5"
},
"2": {
"@type": "SoftwareApplication",
"identifier": "cli",
"name": "cli",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=cli"
},
"3": {
"@type": "SoftwareApplication",
"identifier": "generics",
"name": "generics",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=generics"
},
"4": {
"@type": "SoftwareApplication",
"identifier": "JuliaConnectoR",
"name": "JuliaConnectoR",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=JuliaConnectoR"
},
"5": {
"@type": "SoftwareApplication",
"identifier": "lme4",
"name": "lme4",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=lme4"
},
"6": {
"@type": "SoftwareApplication",
"identifier": "parallel",
"name": "parallel"
},
"7": {
"@type": "SoftwareApplication",
"identifier": "stats",
"name": "stats"
},
"8": {
"@type": "SoftwareApplication",
"identifier": "utils",
"name": "utils"
},
"SystemRequirements": null
},
"fileSize": "1860.225KB",
"releaseNotes": "https://github.com/yjunechoe/jlmerclusterperm/blob/master/NEWS.md",
"readme": "https://github.com/yjunechoe/jlmerclusterperm/blob/main/README.md",
"contIntegration": [
"https://github.com/yjunechoe/jlmerclusterperm/actions/workflows/R-CMD-check.yaml",
"https://github.com/yjunechoe/jlmerclusterperm/actions?query=workflow%3Apkgcheck",
"https://app.codecov.io/gh/yjunechoe/jlmerclusterperm?branch=main"
],
"developmentStatus": "https://www.repostatus.org/#active"
}
GitHub Events
Total
- Issues event: 1
- Watch event: 3
- Push event: 5
- Pull request event: 1
- Create event: 1
Last Year
- Issues event: 1
- Watch event: 3
- Push event: 5
- Pull request event: 1
- Create event: 1
Issues and Pull Requests
Last synced: over 1 year ago
All Time
- Total issues: 20
- Total pull requests: 14
- Average time to close issues: 20 days
- Average time to close pull requests: 30 days
- Total issue authors: 4
- Total pull request authors: 1
- Average comments per issue: 5.25
- Average comments per pull request: 0.14
- Merged pull requests: 13
- Bot issues: 2
- Bot pull requests: 0
Past Year
- Issues: 5
- Pull requests: 4
- Average time to close issues: 22 days
- Average time to close pull requests: 2 days
- Issue authors: 1
- Pull request authors: 1
- Average comments per issue: 0.8
- Average comments per pull request: 0.0
- Merged pull requests: 4
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- yjunechoe (13)
- XiaoyuZeng (2)
- github-actions[bot] (2)
- achetverikov (1)
Pull Request Authors
- yjunechoe (20)
Top Labels
Issue Labels
documentation (2)
Pull Request Labels
Packages
- Total packages: 1
-
Total downloads:
- cran 675 last-month
- Total dependent packages: 0
- Total dependent repositories: 0
- Total versions: 12
- Total maintainers: 1
cran.r-project.org: jlmerclusterperm
Cluster-Based Permutation Analysis for Densely Sampled Time Data
- Homepage: https://github.com/yjunechoe/jlmerclusterperm
- Documentation: http://cran.r-project.org/web/packages/jlmerclusterperm/jlmerclusterperm.pdf
- License: MIT + file LICENSE
-
Latest release: 1.1.4
published over 1 year ago
Rankings
Stargazers count: 26.1%
Forks count: 28.7%
Dependent packages count: 29.1%
Dependent repos count: 34.8%
Average: 41.7%
Downloads: 89.6%
Maintainers (1)
Last synced:
6 months ago
Dependencies
DESCRIPTION
cran
- JuliaConnectoR * imports
- cli * imports
- generics * imports
- lme4 * imports
- parallel * imports
- stats * imports
- utils * imports
.github/workflows/R-CMD-check.yaml
actions
- actions/checkout v3 composite
- r-lib/actions/check-r-package v2 composite
- r-lib/actions/setup-r v2 composite
- r-lib/actions/setup-r-dependencies v2 composite
.github/workflows/julia-style.yaml
actions
- actions/checkout v1 composite
- julia-actions/setup-julia latest composite
- reviewdog/action-suggester v1 composite
.github/workflows/pkgcheck.yaml
actions
- ropensci-review-tools/pkgcheck-action main composite
.github/workflows/pr-commands.yaml
actions
- actions/checkout v3 composite
- r-lib/actions/pr-fetch v2 composite
- r-lib/actions/pr-push v2 composite
- r-lib/actions/setup-r v2 composite
- r-lib/actions/setup-r-dependencies v2 composite
.github/workflows/test-coverage.yaml
actions
- actions/checkout v3 composite
- actions/upload-artifact v3 composite
- r-lib/actions/setup-r v2 composite
- r-lib/actions/setup-r-dependencies v2 composite