EmpiricalCalibration
An R package for performing empirical calibration of observational study estimates
Science Score: 39.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 5 DOI reference(s) in README -
○Academic publication links
-
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (12.9%) to scientific vocabulary
Keywords
Keywords from Contributors
Repository
An R package for performing empirical calibration of observational study estimates
Basic Info
- Host: GitHub
- Owner: OHDSI
- Language: R
- Default Branch: main
- Homepage: http://ohdsi.github.io/EmpiricalCalibration/
- Size: 23.7 MB
Statistics
- Stars: 10
- Watchers: 18
- Forks: 9
- Open Issues: 1
- Releases: 7
Topics
Metadata Files
README.md
EmpiricalCalibration
EmpiricalCalibration is part of HADES.
Introduction
This R package contains routines for performing empirical calibration of observational study estimates. By using a set of negative control hypotheses we can estimate the empirical null distribution of a particular observational study setup. This empirical null distribution can be used to compute a calibrated p-value, which reflects the probability of observing an estimated effect size when the null hypothesis is true taking both random and systematic error into account, as described in the paper Interpreting observational studies: why empirical calibration is needed to correct p-values.
Also supported is empirical calibration of confidence intervals, based on the results for a set of negative and positive controls, as described in the paper Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data.
Features
- Estimate the empirical null distribution given the effect estimates of a set of negative controls.
- Estimate the calibrated p-value of a given hypothesis given the estimated empirical null distribution.
- Estimate a systematic error distribution given the effect estimates for a set of negative and positive controls.
- Estimate the calibrated confidence interval for a given estimate given the systematic error distribution.
- Estimate a calibrated log likelihood ratio, for use in maximum sequential probability ratio testing (MaxSPRT).
- Produce various plots for evaluating the empirical calibration.
- Contains the data sets from the papers for illustration.
Screenshots and examples
```r data(sccs) #Load one of the included data sets negatives <- sccs[sccs$groundTruth == 0,] #Select the negative controls null <- fitNull(logRr = negatives$logRr, seLogRr = negatives$seLogRr) #Fit the null distribution positive <- sccs[sccs$groundTruth == 1,] #Select the positive control
Create the plot above:
plotCalibrationEffect(logRrNegatives = negatives$logRr, seLogRrNegatives = negatives$seLogRr, logRrPositives = positive$logRr, seLogRrPositives = positive$seLogRr, null = null)
Compute the calibrated p-value:
calibrateP(null = null, logRr = positive$logRr, seLogRr = positive$seLogRr) #Compute calibrated p-value [1] 0.8390598 ```
Technology
This is a pure R package.
System requirements
Requires R (version 3.1.0 or newer).
Installation
In R, use the following commands to install the latest stable version from CRAN:
r
install.packages("EmpiricalCalibration")
To install the latest development version directly from GitHub, use:
r
install.packages("remotes")
library(remotes)
install_github("ohdsi/EmpiricalCalibration", ref = "develop")
User Documentation
Documentation can be found on the package website.
PDF versions of the documentation is also available:
- Vignette: Empirical calibration of p-values
- Vignette: Empirical calibration of confidence intervals
- Vignette: Empirical calibration and MaxSPRT
- Package manual: EmpiricalCalibration.pdf
Support
- Developer questions/comments/feedback: OHDSI Forum
- We use the GitHub issue tracker for all bugs/issues/enhancements
Contributing
Read here how you can contribute to this package.
License
EmpiricalCalibration is licensed under Apache License 2.0
Development
This package has been developed in RStudio.
Development status
This package is ready for use.
Owner
- Name: Observational Health Data Sciences and Informatics
- Login: OHDSI
- Kind: organization
- Website: http://ohdsi.org
- Repositories: 285
- Profile: https://github.com/OHDSI
GitHub Events
Total
- Create event: 1
- Release event: 1
- Issues event: 1
- Push event: 7
Last Year
- Create event: 1
- Release event: 1
- Issues event: 1
- Push event: 7
Committers
Last synced: 11 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| schuemie | s****e@o****g | 144 |
| Schuemie | M****I@i****m | 35 |
| Marc Suchard | m****d@g****m | 27 |
| Admin_mschuemi | A****i@i****m | 12 |
| arekkas | a****s@e****l | 7 |
| HAK16 | h****z@g****m | 5 |
| keesvanbochove | k****s@t****l | 4 |
| jgilber2 | J****2@i****m | 4 |
| cyang-emc | c****g@e****l | 4 |
| egarcialara | e****a@h****m | 1 |
| dependabot[bot] | 4****] | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 10 months ago
All Time
- Total issues: 4
- Total pull requests: 5
- Average time to close issues: 3 months
- Average time to close pull requests: about 20 hours
- Total issue authors: 4
- Total pull request authors: 5
- Average comments per issue: 0.75
- Average comments per pull request: 1.2
- Merged pull requests: 4
- Bot issues: 0
- Bot pull requests: 1
Past Year
- Issues: 0
- Pull requests: 1
- Average time to close issues: N/A
- Average time to close pull requests: about 17 hours
- Issue authors: 0
- Pull request authors: 1
- Average comments per issue: 0
- Average comments per pull request: 0.0
- Merged pull requests: 1
- Bot issues: 0
- Bot pull requests: 1
Top Authors
Issue Authors
- ericaVoss (1)
- fabkury (1)
- mvankessel-EMC (1)
- schuemie (1)
- azimov (1)
Pull Request Authors
- dependabot[bot] (2)
- HAK16 (1)
- rekkasa (1)
- keesvanbochove (1)
- azimov (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 1
-
Total downloads:
- cran 906 last-month
- Total docker downloads: 42,216
- Total dependent packages: 1
- Total dependent repositories: 29
- Total versions: 15
- Total maintainers: 1
cran.r-project.org: EmpiricalCalibration
Routines for Performing Empirical Calibration of Observational Study Estimates
- Homepage: https://ohdsi.github.io/EmpiricalCalibration/
- Documentation: http://cran.r-project.org/web/packages/EmpiricalCalibration/EmpiricalCalibration.pdf
- License: Apache License 2.0
-
Latest release: 3.1.4
published over 1 year ago
Rankings
Maintainers (1)
Dependencies
- R >= 3.5.0 depends
- Rcpp * imports
- ggplot2 >= 2.0.0 imports
- gridExtra * imports
- methods * imports
- rlang * imports
- Cyclops * suggests
- Sequential * suggests
- knitr * suggests
- markdown * suggests
- rmarkdown * suggests
- survival * suggests
- testthat * suggests
- actions/checkout v3 composite
- actions/create-release v1 composite
- actions/download-artifact v2 composite
- actions/upload-artifact v2 composite
- r-lib/actions/check-r-package v2 composite
- r-lib/actions/setup-pandoc v2 composite
- r-lib/actions/setup-r v2 composite
- r-lib/actions/setup-r-dependencies v2 composite
- r-lib/actions/setup-tinytex v2 composite
- actions/checkout v3 composite
- r-lib/actions/check-r-package v2 composite
- r-lib/actions/setup-pandoc v2 composite
- r-lib/actions/setup-r v2 composite
- r-lib/actions/setup-r-dependencies v2 composite
- r-lib/actions/setup-tinytex v2 composite