agb-products

Light pre-processing of AGB data products to get them into a common format

https://github.com/ecosystemecologylab/agb-products

Science Score: 52.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
  • Academic publication links
  • Academic email domains
  • Institutional organization owner
    Organization ecosystemecologylab has institutional domain (nature.arizona.edu)
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (15.5%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

Light pre-processing of AGB data products to get them into a common format

Basic Info
  • Host: GitHub
  • Owner: EcosystemEcologyLab
  • License: mit
  • Language: R
  • Default Branch: main
  • Size: 413 KB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 1
  • Open Issues: 1
  • Releases: 0
Created over 1 year ago · Last pushed about 1 year ago
Metadata Files
Readme Contributing License Citation

README.md

Above-ground Biomass Raster Pre-Processing

Project Status: Active – The project has reached a stable, usable
state and is being actively
developed.

The goal of this repository is to do some light pre-processing to above-ground biomass data products and save them to the “snow” server in a common format.

Pre-processing steps currently taken:

  • Name all layers appropriately
    • If they are years, use the year (e.g. "2005")
    • If they represent a time range, use the time range (e.g. "2005-2010 mean" or "2009, 2012-2014 mean")
  • Convert to units of Mg/ha
  • Rasterize vector products
  • Save as cloud optimized geotiff(s)

Other possible steps that could be added in the future:

  • Re-project to a common CRS
  • Include layers besides AGB. For example, some products have layers with standard deviation or quality control flags that are currently removed during this pre-processing.

Example Function

In R/ there is a collection of clean_*() functions defined. One of the simpler ones is for the Xu et al. dataset:

``` r

Function has arguments for input and output path. Input might be a directory in the case of multiple tiles

cleanxu <- function( input = "/Volumes/moore/Xu/test10acdabpredcorr20002019v2.tif", output = "/Volumes/moore/AGBcleaned/xu/xu.tif" ) { # Read in file(s) xuagbraw <- terra::rast(input) # Do any necessary conversions or subsetting xuagb <- xuagbraw * 2.2 # Add metadata including layer names units(xuagb) <- "Mg/ha" varnames(xuagb) <- "AGB" names(xu_agb) <- 2000:2019

# Write to COG terra::writeRaster(xu_agb, output, filetype = "COG", overwrite = TRUE) # Return output path output } ```

If you have snow mounted to /Volumes/moore/ then all you need to do is run clean_xu() to read in the downloaded raw data, do the processing, and save it out as a COG to AGB_cleaned/xu.tif.

[!NOTE]

This will actually write two files, xu.tiff and a “sidecar” file xu.tiff.aux.json containing some metadata (just the units in this example). terra::rast("path/to/xu.tiff") will automatically also read in the sidecar file if it is in the same directory.

Using targets

These functions are set up to all be run as part of a targets pipeline using just the command tar_make(). The pipeline (defined in _targets.R) tracks both input and output files as well as the code in the clean_*() functions so if there are any changes (e.g. adding a new tile for the ESA CCI dataset), running tar_make() should automatically update the output data in the AGB_cleaned/ folder on snow.

Setup with renv

The R package dependencies for this repository are tracked with renv. When you first open this project, you’ll be prompted to install the dependencies with renv::restore(). This will attempt to install the exact versions of all packages used. Sometimes, especially when a package needs to be built from source, this will fail. You have the option of turning off renv with renv::deactivate(clean = TRUE), installing the most recent versions of required R packages (the ones in _targets_packages.R, and then re-initializing renv with renv::init(). However, note that changes in R packages could require you to update code in the cleaning functions or in _targets.R.


Developed in collaboration with the University of Arizona CCT Data Science team

Owner

  • Name: The Global Ecosystems Lab
  • Login: EcosystemEcologyLab
  • Kind: organization
  • Location: United States of America

Moore lab at the University of Arizona. We study ecosystems and how they respond to, and influence global environmental change.

Citation (CITATION.cff)

# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: Above-ground Biomass Raster Pre-Processing
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Eric R.
    family-names: Scott
    email: ericrscott@arizona.edu
    affiliation: >-
      University of Arizona, Communications & Cyber
      Technologies
    orcid: 'https://orcid.org/0000-0002-7430-7879'
  - given-names: Charles
    family-names: Devine
    affiliation: >-
      University of Arizona, School of Natural Resources &
      The Environment
    orcid: 'https://orcid.org/0000-0003-1651-7613'
  - given-names: Yang
    family-names: Li
    affiliation: >-
      University of Arizona, School of Natural Resources &
      The Environment
    orcid: 'https://orcid.org/0000-0002-6463-6441'
  - given-names: David J.P.
    family-names: Moore
    email: davidjpmoore@arizona.edu
    affiliation: >-
      University of Arizona, School of Natural Resources &
      The Environment
    orcid: 'https://orcid.org/0000-0002-6462-3288'
repository-code: 'https://github.com/EcosystemEcologyLab/AGB-products'
license: MIT
version: v0.1.0

GitHub Events

Total
  • Delete event: 1
  • Push event: 2
  • Pull request event: 2
  • Create event: 1
Last Year
  • Delete event: 1
  • Push event: 2
  • Pull request event: 2
  • Create event: 1

Dependencies

requirements.txt pypi
  • GDAL ==3.9.1
  • affine ==2.4.0
  • attrs ==23.2.0
  • basemap ==1.4.1
  • basemap-data ==1.3.2
  • certifi ==2024.7.4
  • cftime ==1.6.4
  • charset-normalizer ==3.3.2
  • click ==8.1.7
  • click-plugins ==1.1.1
  • cligj ==0.7.2
  • contourpy ==1.2.1
  • cycler ==0.12.1
  • fiona ==1.9.6
  • fonttools ==4.53.1
  • idna ==3.7
  • joblib ==1.4.2
  • kiwisolver ==1.4.5
  • littleutils ==0.2.4
  • lytools ==0.0.107
  • matplotlib ==3.8.4
  • netCDF4 ==1.7.1.post1
  • numpy ==1.26.4
  • outdated ==0.2.2
  • packaging ==23.2
  • pandas ==2.2.2
  • pillow ==10.4.0
  • psutil ==6.0.0
  • pyparsing ==3.1.2
  • pyproj ==3.6.1
  • pyshp ==2.3.1
  • python-dateutil ==2.9.0.post0
  • pytz ==2024.1
  • rasterio ==1.3.10
  • requests ==2.32.3
  • rioxarray ==0.17.0
  • scikit-learn ==1.5.1
  • scipy ==1.14.0
  • seaborn ==0.13.2
  • setuptools ==72.1.0
  • six ==1.16.0
  • snuggs ==1.4.7
  • threadpoolctl ==3.5.0
  • tqdm ==4.66.4
  • tzdata ==2024.1
  • urllib3 ==2.2.2
  • wheel ==0.43.0
  • xarray ==2024.7.0