rmweather

Tools to Conduct Meteorological Normalisation on Air Quality Data

https://github.com/skgrange/rmweather

Science Score: 49.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
    Found 4 DOI reference(s) in README
  • Academic publication links
    Links to: sciencedirect.com, science.org, frontiersin.org
  • Committers with academic emails
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (12.4%) to scientific vocabulary
Last synced: 7 months ago · JSON representation

Repository

Tools to Conduct Meteorological Normalisation on Air Quality Data

Basic Info
  • Host: GitHub
  • Owner: skgrange
  • License: gpl-3.0
  • Language: R
  • Default Branch: master
  • Homepage:
  • Size: 1.78 MB
Statistics
  • Stars: 53
  • Watchers: 4
  • Forks: 8
  • Open Issues: 1
  • Releases: 0
Created about 8 years ago · Last pushed 8 months ago
Metadata Files
Readme Changelog License

README.md

rmweather

Lifecycle Status CRAN status CRAN log

Introduction

rmweather is an R package to conduct meteorological/weather normalisation on air quality so trends and interventions can be investigated in a robust way. For those who are aware of my previous research, rmweather is the "Mk.II" package of normalweatherr. rmweather does less than normalweatherr, but it is faster and easier to use.

Installation

rmweather is available from CRAN and can be installed in the normal way:

```

Install rmweather from CRAN

install.packages("rmweather") ```

Development version

To install the development version of rmweather, the remotes package will need to be installed first. Then:

```

Load helper package

library(remotes)

Install rmweather

install_github("skgrange/rmweather") ```

Example usage

rmweather contains example data from London which can be used to show the meteorological normalisation procedure. The example data are daily means of NO2 and NOx observations at London Marylebone Road. The accompanying surface meteorological data are from London Heathrow, a major airport located 23 km west of Central London.

Most of rmweather's functions begin with rmw_ so are easy to track and find help for. In this example, we have used dplyr and the pipe (%>% and pronounced as "then") for clarity. The example takes about 30 seconds on my (laptop) system and the model has an R2 value of 76 %.

```

Load packages

library(dplyr) library(rmweather) library(ranger)

Have a look at rmweather's example data, from london

head(data_london)

Prepare data for modelling

Only use data with valid wind speeds, no2 will become the dependent variable

datalondonprepared <- datalondon %>% filter(variable == "no2", !is.na(ws)) %>% rmwprepare_data(na.rm = TRUE)

Grow/train a random forest model and then create a meteorological normalised trend

listnormalised <- rmwdoall( datalondonprepared, variables = c( "dateunix", "dayjulian", "weekday", "airtemp", "rh", "wd", "ws", "atmosphericpressure" ), ntrees = 300, n_samples = 300, verbose = TRUE )

What units are in the list?

names(list_normalised)

Check model object's performance

rmwmodelstatistics(list_normalised$model)

Plot variable importances

listnormalised$model %>% rmwmodelimportance() %>% rmwplot_importance()

Check if model has suffered from overfitting

rmwpredictthetestset( model = listnormalised$model, df = listnormalised$observations ) %>% rmwplottest_prediction()

How long did the process take?

listnormalised$elapsedtimes

Plot normalised trend

rmwplotnormalised(list_normalised$normalised)

Investigate partial dependencies, if variable is NA, predict all

datapd <- rmwpartialdependencies( model = listnormalised$model, df = list_normalised$observations, variable = NA )

Plot partial dependencies

datapd %>% filter(variable != "dateunix") %>% rmwplotpartial_dependencies() ```

The meteorologically normalised trend produced is below.

Examples and citations

For usage examples see:

Grange, S. K., Carslaw, D. C., Lewis, A. C., Boleti, E., and Hueglin, C. (2018). Random forest meteorological normalisation models for Swiss PM10 trend analysis. Atmospheric Chemistry and Physics 18.9, pp. 6223--6239.

Grange, S. K. and Carslaw, D. C. (2019). Using meteorological normalisation to detect interventions in air quality time series. Science of The Total Environment 653, pp. 578--588.

The use of rmweather for prediction or counterfactual/business as usual scenarios

A second usage of rmweather became established in 2020 to help researchers quantify the effects of the COVID-19 related restrictions on air quality. Briefly, the approach involves the training of random forest models to explain pollutant concentrations based on meteorological and time variables for a training period, say, between 2018 and 2019. After the training period, the model is used in predictive-mode using the experienced meteorological conditions. The predicted time series can be thought of as a counterfactual or business-as-usual (BAU) scenario which the observed time series can be compared with. Critically, an approach like this accounts for the meteorological conditions observed in 2020, which in many locations was unusual and complicates simple analyses. The meteorological sampling and normalisation step is not required for this analysis, but this has been confused in the literature.

Examples of counterfactural modelling

Grange, S. K., Lee, J. D., Drysdale, W. S., Lewis, A. C., Hueglin, C., Emmenegger, L., and Carslaw, D. C. (2021). COVID-19 lockdowns highlight a risk of increasing ozone pollution in European urban areas. Atmospheric Chemistry and Physics 21.5, pp. 4169--4185.

Wang, Y., Wen, Y., Wang, Y., Zhang, S., Zhang, K. M., Zheng, H., Xing, J., Wu, Y., and Hao, J. (2020). Four-Month Changes in Air Quality during and after the COVID-19 Lockdown in Six Megacities in China. Environmental Science and Technology Letters 7.11, pp. 802--808.

Fenech, S., Aquilina, N. J., Ryan, V. (2021) COVID-19-Related Changes in NO2 and O3 Concentrations and Associated Health Effects in Malta. Frontiers in Sustainable Cities 3.631280, pp. 1--12.

Shi, Z., Song, C., Liu, B., Lu, G., Xu, J., Van Vu, T., Elliott, R. J. R., Li, W., Bloss, W. J., and Harrison, R. M. (2021). Abrupt but smaller than expected changes in surface air quality attributable to COVID-19 lockdowns. Science Advances 7.3, eabd6696.

See also

Owner

  • Name: Stuart Grange
  • Login: skgrange
  • Kind: user
  • Location: Dübendorf, Switzerland

An environmental and data scientist with an air quality focus. Programmatic skills and atmospheric knowledge are used to answer tricky questions.

GitHub Events

Total
  • Watch event: 6
  • Issue comment event: 1
  • Push event: 4
Last Year
  • Watch event: 6
  • Issue comment event: 1
  • Push event: 4

Committers

Last synced: 8 months ago

All Time
  • Total Commits: 148
  • Total Committers: 2
  • Avg Commits per committer: 74.0
  • Development Distribution Score (DDS): 0.405
Past Year
  • Commits: 4
  • Committers: 1
  • Avg Commits per committer: 4.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
skgrange s****e@g****m 88
skgrange s****e@g****m 60

Issues and Pull Requests

Last synced: 7 months ago

All Time
  • Total issues: 6
  • Total pull requests: 0
  • Average time to close issues: 14 days
  • Average time to close pull requests: N/A
  • Total issue authors: 6
  • Total pull request authors: 0
  • Average comments per issue: 0.83
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 1
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 1
  • Pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • skgrange (1)
  • mmmilkbottle (1)
  • teunbrand (1)
  • ghost (1)
  • Medgar21 (1)
  • maxbre (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • cran 326 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 11
  • Total maintainers: 1
cran.r-project.org: rmweather

Tools to Conduct Meteorological Normalisation and Counterfactual Modelling for Air Quality Data

  • Versions: 11
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 326 Last month
Rankings
Stargazers count: 8.1%
Forks count: 9.1%
Average: 24.0%
Dependent packages count: 29.8%
Dependent repos count: 35.5%
Downloads: 37.4%
Maintainers (1)
Last synced: 7 months ago

Dependencies

DESCRIPTION cran
  • R >= 3.2.0 depends
  • dplyr >= 1.0.1 imports
  • ggplot2 * imports
  • lubridate * imports
  • magrittr * imports
  • pdp * imports
  • purrr * imports
  • ranger * imports
  • stringr * imports
  • strucchange * imports
  • tibble * imports
  • tidyr * imports
  • viridis * imports
  • openair * suggests
  • testthat * suggests
.github/workflows/rhub.yaml actions
  • r-hub/actions/checkout v1 composite
  • r-hub/actions/platform-info v1 composite
  • r-hub/actions/run-check v1 composite
  • r-hub/actions/setup v1 composite
  • r-hub/actions/setup-deps v1 composite
  • r-hub/actions/setup-r v1 composite