TopoPyScale

TopoPyScale: A Python Package for Hillslope Climate Downscaling - Published in JOSS (2023)

https://github.com/arcticsnow/topopyscale

Science Score: 95.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
    Found 10 DOI reference(s) in README and JOSS metadata
  • Academic publication links
    Links to: joss.theoj.org, zenodo.org
  • Committers with academic emails
    1 of 10 committers (10.0%) from academic institutions
  • Institutional organization owner
  • JOSS paper metadata
    Published in Journal of Open Source Software

Keywords

climate climate-science clustering dem downscaling era5 geoscience joss science timeseries xarray

Scientific Fields

Mathematics Computer Science - 37% confidence
Last synced: 4 months ago · JSON representation

Repository

TopoPyScale: a Python library to perform simplistic climate downscaling at the hillslope scale

Basic Info
Statistics
  • Stars: 50
  • Watchers: 4
  • Forks: 16
  • Open Issues: 24
  • Releases: 18
Topics
climate climate-science clustering dem downscaling era5 geoscience joss science timeseries xarray
Created about 4 years ago · Last pushed 4 months ago
Metadata Files
Readme License

README.md

DOI badge

DOI GitHub license GitHub release (latest by date) PyPI Downloads Test

Binder Notebooks Examples: Binder

TopoPyScale

Python version of Toposcale packaged as a Pypi library. Toposcale is an original idea of Joel Fiddes to perform topography-based downscaling of climate data to the hillslope scale.

Documentation avalaible: https://topopyscale.readthedocs.io

References:

  • Filhol et al., (2023). TopoPyScale: A Python Package for Hillslope Climate Downscaling. Journal of Open Source Software, 8(86), 5059, https://doi.org/10.21105/joss.05059

And the original method it relies on: - Fiddes, J. and Gruber, S.: TopoSCALE v.1.0: downscaling gridded climate data in complex terrain, Geosci. Model Dev., 7, 387–405, https://doi.org/10.5194/gmd-7-387-2014, 2014. - Fiddes, J. and Gruber, S.: TopoSUB: a tool for efficient large area numerical modelling in complex topography at sub-grid scales, Geosci. Model Dev., 5, 1245–1257, https://doi.org/10.5194/gmd-5-1245-2012, 2012.

Kristoffer Aalstad has a Matlab implementation: https://github.com/krisaalstad/TopoLAB

Contribution Workflow

  1. All contribution welcome!
  2. Found a bug -> Check the issue page. If you have a solution let us know.
  3. No idea on moving furhter -> then create a new issue
  4. Wanna develop a new feature/idea? -> create a new branch. Go wild. Merge with main branch when accomplished.
  5. Create release version when significant improvements and bug fixes have been done. Coordinate with others on Discussions

Create a new release: Follow procedure and conventions described in: https://www.youtube.com/watch?v=Ob9llA_QhQY

Our forum is now on Github Discussions. Come visit!

Design

  1. Inputs
    • Climate data from reanalysis (ERA5, etc)
    • Climate data from future projections (CORDEX) (TBD)
    • DEM from local source, or fetch from public repository: SRTM, ArcticDEM, ASTER
  2. Run TopoScale
    • compute derived values (from DEM)
    • toposcale (k-mean clustering)
    • interpolation (bilinear, inverse square dist.)
  3. Output
    • Cryogrid format
    • FSM format
    • CROCUS format
    • Snowmodel format
    • basic netcfd
    • For each method, have the choice to output either the abstract cluster points, or the gridded product after interpolation
  4. Validation toolset
    • validation to local observation timeseries
    • plotting
  5. Gap filling algorithm
    • random forest temporal gap filling (TBD)

Validation (4) and Gap filling (4) are future implementation.

Installation

We have now added an environments.yml file to handle versions of depencencies that are tested with the current codebase, to use this run:

conda env create -f environment.yml

Alternatively you can follow this method for dependencies (to be deprecated):

```bash conda create -n downscaling python ipython conda activate downscaling

Recomended way to install dependencies:

conda install -c conda-forge xarray matplotlib scikit-learn pandas numpy netcdf4 h5netcdf rasterio pyproj dask rioxarray

install forked version of Topocalc compatible with Python >3.9 (tested with 3.13)

pip install pip@git+https://github.com/ArcticSnow/topocalc ```

Then install the code:

```

OPTION 1 (Pypi release):

pip install TopoPyScale

OPTION 2 (development):

cd github # navigate to where you want to clone TopoPyScale git clone git@github.com:ArcticSnow/TopoPyScale.git pip install -e TopoPyScale #install a development version

----------------------------------------------------------

OPTIONAL: if using jupyter lab

add this new Python kernel to your jupyter lab PATH

python -m ipykernel install --user --name downscaling

Tool for generating documentation from code docstring

pip install lazydocs ```

Then you need to setup your cdsapi with the Copernicus API key system. Follow this tutorial after creating an account with Copernicus. On Linux, create a file nano ~/.cdsapirc with inside:

url: https://cds.climate.copernicus.eu/api/v2 key: {uid}:{api-key}

Basic usage

  1. Setup your Python environment
  2. Create your project directory
  3. Configure the file config.ini to fit your problem (see config.yml for an example)
  4. Run TopoPyScale

```python import pandas as pd from TopoPyScale import topoclass as tc from matplotlib import pyplot as plt

========= STEP 1 ==========

Load Configuration

configfile = './config.yml' mp = tc.Topoclass(configfile)

Compute parameters of the DEM (slope, aspect, sky view factor)

mp.getera5() mp.computedem_param()

========== STEP 2 ===========

Extract DEM parameters for points of interest (centroids or physical points)

mp.extracttopoparam()

----- Option 1:

Compute clustering of the input DEM and extract cluster centroids

mp.extractdemcluster_param()

plot clusters

mp.toposub.plotclustersmap()

plot sky view factor

mp.toposub.plotclustersmap(var='svf', cmap=plt.cm.viridis)

------ Option 2:

inidicate in the config file the .csv file containing a list of point coordinates (!!! must same coordinate system as DEM !!!)

mp.extractptsparam(method='linear',index_col=0)

========= STEP 3 ==========

compute solar geometry and horizon angles

mp.computesolargeometry() mp.compute_horizon()

========= STEP 4 ==========

Perform the downscaling

mp.downscale_climate()

========= STEP 5 ==========

explore the downscaled dataset. For instance the temperature difference between each point and the first one

(mp.downscaledpts.t-mp.downscaledpts.t.isel(point_id=0)).plot() plt.show()

========= STEP 6 ==========

Export output to desired format

mp.to_netcdf() ```

TopoClass will create a file structure in the project folder (see below). TopoPyScale assumes you have a DEM in GeoTiFF, and a set of climate data in netcdf (following ERA5 variable conventions). TopoPyScale can easier segment the DEM using clustering (e.g. K-mean), or a list of predefined point coordinates in pts_list.csv can be provided. Make sure all parameters in config.ini are correct. my_project/ ├── inputs/ ├── dem/ ├── my_dem.tif └── pts_list.csv (optional) └── climate/ ├── PLEV*.nc └── SURF*.nc ├── outputs/ └── config.ini

Owner

  • Name: Simon Filhol
  • Login: ArcticSnow
  • Kind: user
  • Location: Norway
  • Company: University of Oslo

JOSS Publication

TopoPyScale: A Python Package for Hillslope Climate Downscaling
Published
June 23, 2023
Volume 8, Issue 86, Page 5059
Authors
Simon Filhol ORCID
Department of Geosciences, University of Oslo, Norway
Joel Fiddes ORCID
WSL Institute for Snow and Avalanche Research SLF, Switzerland
Kristoffer Aalstad ORCID
Department of Geosciences, University of Oslo, Norway
Editor
Hugo Ledoux ORCID
Tags
climate science downscaling xarray dem segmentation snow permafrost

GitHub Events

Total
  • Create event: 2
  • Commit comment event: 4
  • Release event: 3
  • Issues event: 17
  • Watch event: 6
  • Delete event: 3
  • Issue comment event: 39
  • Member event: 1
  • Push event: 62
  • Pull request event: 4
  • Fork event: 6
Last Year
  • Create event: 2
  • Commit comment event: 4
  • Release event: 3
  • Issues event: 17
  • Watch event: 6
  • Delete event: 3
  • Issue comment event: 39
  • Member event: 1
  • Push event: 62
  • Pull request event: 4
  • Fork event: 6

Committers

Last synced: 5 months ago

All Time
  • Total Commits: 608
  • Total Committers: 10
  • Avg Commits per committer: 60.8
  • Development Distribution Score (DDS): 0.275
Past Year
  • Commits: 76
  • Committers: 5
  • Avg Commits per committer: 15.2
  • Development Distribution Score (DDS): 0.382
Top Committers
Name Email Commits
arcticsnow s****l@p****m 441
Joel Fiddes j****s@g****m 127
paswyss p****s@p****m 26
FILHOL Simon s****l@m****r 6
Nilick 6****K 3
Kristoffer Aalstad k****d@g****m 1
Hugo Ledoux h****x@t****l 1
Esteban Alonso e****l@g****m 1
Marco Mazzolini m****z@n****3 1
Joel Fiddes j****l@j****e 1
Committer Domains (Top 20 + Academic)

Issues and Pull Requests

Last synced: 4 months ago

All Time
  • Total issues: 87
  • Total pull requests: 23
  • Average time to close issues: 7 months
  • Average time to close pull requests: about 7 hours
  • Total issue authors: 10
  • Total pull request authors: 6
  • Average comments per issue: 2.44
  • Average comments per pull request: 0.61
  • Merged pull requests: 22
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 9
  • Pull requests: 6
  • Average time to close issues: about 2 months
  • Average time to close pull requests: about 21 hours
  • Issue authors: 5
  • Pull request authors: 3
  • Average comments per issue: 3.0
  • Average comments per pull request: 1.17
  • Merged pull requests: 5
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • ArcticSnow (53)
  • joelfiddes (20)
  • paswyss (4)
  • NILICK (3)
  • MatteaE (2)
  • Giulia-cima (1)
  • srinivas2036 (1)
  • dvalters (1)
  • mvantiggelen (1)
  • bare92 (1)
Pull Request Authors
  • ArcticSnow (12)
  • joelfiddes (4)
  • NILICK (3)
  • mmazzolini (2)
  • ealonsogzl (1)
  • paswyss (1)
Top Labels
Issue Labels
enhancement (38) topo_scale (16) bug (13) topoClass (7) debug (6) topo_sub (4) dem (4) fetch_data (4) good first issue (3) parallelize (3) new feature (2) documentation (2) meteo_util (2) abandoned (1)
Pull Request Labels

Packages

  • Total packages: 3
  • Total downloads:
    • pypi 266 last-month
  • Total dependent packages: 0
    (may contain duplicates)
  • Total dependent repositories: 0
    (may contain duplicates)
  • Total versions: 51
  • Total maintainers: 1
proxy.golang.org: github.com/ArcticSnow/TopoPyScale
  • Versions: 18
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.4%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 4 months ago
proxy.golang.org: github.com/arcticsnow/topopyscale
  • Versions: 18
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.4%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 4 months ago
pypi.org: topopyscale

TopoPyScale, a Python package to perform simplistic climate downscaling at the hillslope scale.

  • Versions: 15
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 266 Last month
  • Docker Downloads: 0
Rankings
Dependent packages count: 6.6%
Downloads: 13.7%
Stargazers count: 14.9%
Average: 17.1%
Forks count: 19.6%
Dependent repos count: 30.6%
Maintainers (1)
Last synced: 4 months ago

Dependencies

setup.py pypi
  • cdsapi *
  • configobj *
  • dask *
  • matplotlib *
  • munch *
  • numpy *
  • pandas *
  • pvlib *
  • pyproj *
  • rasterio *
  • scikit-learn *
  • topocalc *
  • xarray *
.github/workflows/draft-pdf.yml actions
  • actions/checkout v2 composite
  • actions/upload-artifact v1 composite
  • openjournals/openjournals-draft-action master composite
.github/workflows/python-publish.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
  • pypa/gh-action-pypi-publish 27b31702a0e7fc50959f5ad993c78deac1bdfc29 composite
.github/workflows/test_topopyscale.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
environment.yml pypi
  • cdsapi ==0.6.1
  • gitpython ==3.1.31
  • munch ==2.5.0
  • numba ==0.57.0
  • numpy ==1.24.2
  • pvlib ==0.9.5
  • pyModis ==2.3.0
  • pydap ==3.4.0
  • requests ==2.30.0
  • scipy ==1.10.1
  • seaborn ==0.12.2
  • topocalc ==0.5.0