asli-pipeline

This repository contains a pipeline for operational execution of the Amundsen Sea Ice Low calculations, provided in the asli package. The functions in the asli package are described in detail in the package repository amundsen-sea-low-index.

https://github.com/antarctica/asli-pipeline

Science Score: 75.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
    Found 6 DOI reference(s) in README
  • Academic publication links
    Links to: zenodo.org
  • Academic email domains
  • Institutional organization owner
    Organization antarctica has institutional domain (www.bas.ac.uk)
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (16.0%) to scientific vocabulary

Keywords

arco era5 jasmin pipeline s3
Last synced: 6 months ago · JSON representation ·

Repository

This repository contains a pipeline for operational execution of the Amundsen Sea Ice Low calculations, provided in the asli package. The functions in the asli package are described in detail in the package repository amundsen-sea-low-index.

Basic Info
Statistics
  • Stars: 0
  • Watchers: 4
  • Forks: 0
  • Open Issues: 1
  • Releases: 2
Topics
arco era5 jasmin pipeline s3
Created almost 2 years ago · Last pushed 12 months ago
Metadata Files
Readme License Citation

README.md

asli-pipeline

DOI Dev Status Static Badge Documentation <!-- badges: end -->

This repository contains a pipeline for operational execution of the Amundsen Sea Ice Low calculations, provided in the asli package. The functions in the asli package are described in detail in the package repository amundsen-sea-low-index (Hosking & Wilby 2024), and in Hosking et al. (2016).

This pipeline was built using the icenet-pipeline as a template (Byrne et al. 2024).

Get the repository

Clone this repository into a directory on your computer or HPC. git clone git@github.com:antarctica/asli-pipeline.git asli-pipeline

Creating an environment

```bash

If you are working on JASMIN you will need to load in jaspy and jasr

module load jaspy module load jasr

Or, on the BAS HPC:

module load mamba/r-* # any version above 4.* module load python/3.12.3/gcc-11.4.1-n3s7

python -m venv asli_env

source asli_env/bin/activate ```

Installing dependencies

To install all dependencies, inlcuding the asli package, run: ```bash pip install -r requirements.txt

For R, we are using {renv} to manage dependencies

install.R uses renv::restore, in combination with

automatic distro detection to install R & system

dependencies

R -f install.R ```

Setting up Climate Data Store API

The asli package will not be able to download ERA5 data without access to the Copernicus Climate Data Store.

Follow these instructions to set up CDS API access: How to Use The CDS API.

```bash nano $HOME/.cdsapirc

Paste in your {uid} and {api-key}

```

Configuration

This pipeline revolves around the ENVS file to provide the necessary configuration items. This can easily be derived from the ENVS.example file to a new file, then symbolically linked. Comments are available in ENVS.example to assist you with the editing process. ```bash cp ENVS.example ENVS.myconfig ln -sf ENVS.myconfig ENVS

Edit ENVS.myconfig to customise parameters for the pipeline

```

Documentation

Please refer to the documentation for a full breakdown of functionality.

Citation

If you use this pipeline in your work, please cite this repository by using the 'Cite this repostory' button on the top right of this repository.

Acknowledgements

This work used JASMIN, the UK’s collaborative data analysis environment (https://www.jasmin.ac.uk).

The asli package uses data from Hersbach, H. et al., (2018) downloaded from the Copernicus Climate Change Service (2023). This software is used to download the data. Therefore these sources are cited below without a specific access date.

References

Brown, M. J., & Chevuturi, A. objectstoretutorial [Computer software]. https://github.com/NERC-CEH/objectstoretutorial

Byrne, J., Ubald, B. N., & Chan, R. icenet-pipeline (Version v0.2.9) [Computer software]. https://github.com/icenet-ai/icenet-pipeline

Copernicus Climate Change Service (2023): ERA5 hourly data on single levels from 1940 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS), DOI: 10.24381/cds.adbb2d47.

Hersbach, H., Bell, B., Berrisford, P., Biavati, G., Horányi, A., Muñoz Sabater, J., Nicolas, J., Peubey, C., Radu, R., Rozum, I., Schepers, D., Simmons, A., Soci, C., Dee, D., Thépaut, J-N. (2018): ERA5 hourly data on single levels from 1940 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS), DOI: 10.24381/cds.adbb2d47.

Hosking, J. S., A. Orr, T. J. Bracegirdle, and J. Turner (2016), Future circulation changes off West Antarctica: Sensitivity of the Amundsen Sea Low to projected anthropogenic forcing, Geophys. Res. Lett., 43, 367–376, doi:10.1002/2015GL067143.

Hosking, J. S., & Wilby, D. asli [Computer software]. https://github.com/scotthosking/amundsen-sea-low-index

Lawrence, B. N. , Bennett, V. L., Churchill, J., Juckes, M., Kershaw, P., Pascoe, S., Pepler, S., Pritchard, M. and Stephens, A. (2013) Storing and manipulating environmental big data with JASMIN. In: IEEE Big Data, October 6-9, 2013, San Francisco.

Owner

  • Name: British Antarctic Survey
  • Login: antarctica
  • Kind: organization
  • Email: info@bas.ac.uk
  • Location: Cambridge, UK

British Antarctic Survey (BAS) delivers world-leading interdisciplinary research in the Polar Regions

Citation (CITATION.cff)

cff-version: 1.2.0
title: asli-pipeline
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Thomas
    family-names: Zwagerman
    email: thozwa@bas.ac.uk
    affiliation: British Antarctic Survey
    orcid: 'https://orcid.org/0009-0003-3742-3234'
  - given-names: David
    family-names: Wilby
    email: dalby@bas.ac.uk
    affiliation: British Antarctic Survey
    orcid: 'https://orcid.org/0000-0002-6553-8739'
repository-code: 'https://github.com/antarctica/boost-eds-pipeline'
abstract: >-
  This repository contains a pipeline for operational execution of the Amundsen Sea Ice Low calculations, provided in the asli package.
  The functions in the asli package are described in detail in the package repository amundsen-sea-low-index.
keywords:
  - object storage
  - JASMIN
  - cloud compute
  - cloud
  - pipeline
  - Datalabs
license: MIT

GitHub Events

Total
  • Create event: 10
  • Release event: 2
  • Issues event: 28
  • Delete event: 8
  • Issue comment event: 17
  • Push event: 58
  • Pull request review event: 9
  • Pull request review comment event: 11
  • Pull request event: 10
Last Year
  • Create event: 10
  • Release event: 2
  • Issues event: 28
  • Delete event: 8
  • Issue comment event: 17
  • Push event: 58
  • Pull request review event: 9
  • Pull request review comment event: 11
  • Pull request event: 10

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 6
  • Total pull requests: 3
  • Average time to close issues: about 1 month
  • Average time to close pull requests: 1 minute
  • Total issue authors: 1
  • Total pull request authors: 1
  • Average comments per issue: 0.33
  • Average comments per pull request: 0.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 6
  • Pull requests: 3
  • Average time to close issues: about 1 month
  • Average time to close pull requests: 1 minute
  • Issue authors: 1
  • Pull request authors: 1
  • Average comments per issue: 0.33
  • Average comments per pull request: 0.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • thomaszwagerman (19)
Pull Request Authors
  • thomaszwagerman (8)
  • dependabot[bot] (1)
Top Labels
Issue Labels
Pull Request Labels
dependencies (1)

Dependencies

requirements.txt pypi
  • Cartopy ==0.23.0
  • Jinja2 ==3.1.4
  • MarkupSafe ==2.1.5
  • PyYAML ==6.0.1
  • Pygments ==2.18.0
  • affine ==2.4.0
  • aiobotocore ==2.13.1
  • aiohttp ==3.10.2
  • aioitertools ==0.11.0
  • aiosignal ==1.3.1
  • asciitree ==0.3.3
  • asttokens ==2.4.1
  • attrs ==23.2.0
  • bas-geoplot ==0.1.0
  • botocore ==1.34.131
  • branca ==0.7.2
  • cads-api-client ==1.1.0
  • cdsapi ==0.7.0
  • certifi ==2024.7.4
  • cftime ==1.6.4
  • charset-normalizer ==3.3.2
  • click ==8.1.7
  • click-plugins ==1.1.1
  • cligj ==0.7.2
  • cloudpickle ==3.0.0
  • comm ==0.2.2
  • contextily ==1.6.0
  • contourpy ==1.2.1
  • cycler ==0.12.1
  • dask ==2024.7.0
  • debugpy ==1.8.2
  • decorator ==5.1.1
  • executing ==2.0.1
  • fasteners ==0.19
  • folium ==0.17.0
  • fonttools ==4.53.1
  • frozenlist ==1.4.1
  • fsspec ==2024.6.1
  • geographiclib ==2.0
  • geopandas ==1.0.1
  • geoplot ==0.5.1
  • geopy ==2.4.1
  • idna ==3.7
  • imageio ==2.34.2
  • iniconfig ==2.0.0
  • ipykernel ==6.29.5
  • ipython ==8.26.0
  • jedi ==0.19.1
  • jmespath ==1.0.1
  • joblib ==1.4.2
  • jsonpickle ==3.2.2
  • jsonschema ==4.23.0
  • jsonschema-specifications ==2023.12.1
  • jupyter_client ==8.6.2
  • jupyter_core ==5.7.2
  • kiwisolver ==1.4.5
  • lazy_loader ==0.4
  • locket ==1.0.0
  • mapclassify ==2.7.0
  • matplotlib ==3.9.1
  • matplotlib-inline ==0.1.7
  • mercantile ==1.2.1
  • meshiphi ==2.1.11
  • multidict ==6.0.5
  • multiurl ==0.3.1
  • nest-asyncio ==1.6.0
  • netCDF4 ==1.7.1.post1
  • networkx ==3.3
  • numcodecs ==0.13.0
  • numpy ==2.0.0
  • packaging ==24.1
  • pandas ==2.2.2
  • parso ==0.8.4
  • partd ==1.4.2
  • pexpect ==4.9.0
  • pillow ==10.4.0
  • platformdirs ==4.2.2
  • pluggy ==1.5.0
  • prompt_toolkit ==3.0.47
  • psutil ==6.0.0
  • ptyprocess ==0.7.0
  • pure-eval ==0.2.2
  • pyogrio ==0.9.0
  • pyparsing ==3.1.2
  • pyproj ==3.6.1
  • pyshp ==2.3.1
  • pytest ==8.3.2
  • python-dateutil ==2.9.0.post0
  • pytz ==2024.1
  • pyzmq ==26.0.3
  • rasterio ==1.3.10
  • referencing ==0.35.1
  • requests ==2.32.3
  • rioxarray ==0.17.0
  • rpds-py ==0.19.1
  • s3fs ==2024.6.1
  • scikit-image ==0.24.0
  • scikit-learn ==1.5.1
  • scipy ==1.14.0
  • seaborn ==0.13.2
  • setuptools ==70.3.0
  • shapely ==2.0.5
  • six ==1.16.0
  • snuggs ==1.4.7
  • stack-data ==0.6.3
  • threadpoolctl ==3.5.0
  • tifffile ==2024.7.2
  • toolz ==0.12.1
  • tornado ==6.4.1
  • tqdm ==4.66.4
  • traitlets ==5.14.3
  • typing_extensions ==4.12.2
  • tzdata ==2024.1
  • urllib3 ==1.26.19
  • wcwidth ==0.2.13
  • wrapt ==1.16.0
  • xarray ==2024.6.0
  • xyzservices ==2024.6.0
  • yarl ==1.9.4
  • zarr ==2.18.2