Data

Tools

Indexes

Applications

Experiments

Open Source Science

gt4py

Python library for generating high-performance implementations of stencil kernels for weather and climate modeling from a domain-specific language (DSL).

https://github.com/GridTools/gt4py

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    3 of 34 committers (8.8%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.5%) to scientific vocabulary

Keywords from Contributors

fpga high-level-synthesis programming-language vivado-hls radiuss build-tools hpsf spack
Last synced: 6 months ago · JSON representation

Repository

Python library for generating high-performance implementations of stencil kernels for weather and climate modeling from a domain-specific language (DSL).

Basic Info
Statistics
  • Stars: 128
  • Watchers: 11
  • Forks: 52
  • Open Issues: 174
  • Releases: 10
Created over 6 years ago · Last pushed 6 months ago
Metadata Files
Readme Changelog Contributing License Authors

README.md

logo

license slack

Daily CI test-components qa

uv Nox

GT4Py: GridTools for Python

GT4Py is a Python library for generating high performance implementations of stencil kernels from a high-level definition using regular Python functions. GT4Py is part of the GridTools framework, a set of libraries and utilities to develop performance portable applications in the area of weather and climate modeling.

NOTE: The gt4py.next subpackage contains a new version of GT4Py which is not compatible with the current stable version defined in gt4py.cartesian. The new version is still experimental.

📃 Description

GT4Py is a Python library for expressing computational motifs as found in weather and climate applications. These computations are expressed in a domain specific language (GTScript) which is translated to high-performance implementations for CPUs and GPUs.

The DSL expresses computations on a 3-dimensional Cartesian grid. The horizontal axes (I, J) are always computed in parallel, while the vertical (K) can be iterated in sequential, forward or backward, order. Cartesian offsets are expressed relative to a center index.

In addition, GT4Py provides functions to allocate arrays with memory layout suited for a particular backend.

The following backends are supported:

  • numpy: Pure-Python backend
  • gt:cpu_ifirst: GridTools C++ CPU backend using I-first data ordering
  • gt:cpu_kfirst: GridTools C++ CPU backend using K-first data ordering
  • gt:gpu: GridTools backend for CUDA
  • cuda: CUDA backend minimally using utilities from GridTools
  • dace:cpu: Dace code-generated CPU backend
  • dace:gpu: Dace code-generated GPU backend

🚜 Installation

GT4Py can be installed as a regular Python package using uv, pip or any other PEP-517 compatible frontend. We strongly recommended to useuv to create and manage virtual environments for your own projects.

⚙ Configuration

Other useful available environment variables are:

  • CUDA_ARCH: Set the compute capability of the NVIDIA GPU if it is not detected automatically by cupy.
  • CXX: Set the C++ compiler.
  • GT_CACHE_DIR_NAME: Name of the compiler's cache directory (defaults to .gt_cache)
  • GT_CACHE_ROOT: Path to the compiler cache (defaults to ./)

More options and details are available in config.py.

🛠 Development Instructions

Follow the installation instructions below to initialize a development virtual environment containing an editable installation of the GT4Py package. Make sure you read the CONTRIBUTING.md and CODING_GUIDELINES.md documents before you start working on the project.

Development Environment Installation using uv

GT4Py uses the uv project manager for the development workflow. uv is a versatile tool that consolidates functionality usually distributed across different applications into subcommands.

  • The uv pip subcommand provides a fast Python package manager, emulating pip.
  • The uv export | lock | sync subcommands manage dependency versions in a manner similar to the pip-tools command suite.
  • The uv init | add | remove | build | publish | ... subcommands facilitate project development workflows, akin to hatch.
  • The uv tool subcommand serves as a runner for Python applications in isolation, similar to pipx.
  • The uv python subcommands manage different Python installations and versions, much like pyenv.

We require a reasonably recent version of uv, which can be installed in various ways (see its installation instructions), with the recommended method being the standalone installer:

bash $ curl -LsSf https://astral.sh/uv/install.sh | sh

Once uv is installed in your system, it is enough to clone this repository and let uv handling the installation of the development environment.

```bash

Clone the repository

git clone https://github.com/gridtools/gt4py.git cd gt4py

Let uv create the development environment at .venv.

uv sync --extra cartesian # for gt4py.cartesian. Include DaCe with: --group dace-cartesian uv sync --extra next # for gt4py.next. Include DaCe with: --group dace-next

The --extra cartesian (or --extra next) option tells uv to install

the optional run-time dependencies of gt4py.cartesian (or gt4py.next),

and thus it is not strictly necessary.

The 'dev' group with the core development dependencies is installed by default.

DaCe-related dependencies are different for gt4py.cartesian and gt4py.next,

so they won't be installed unless the specific dace group option is added.

Finally, activate the virtual environment and start writing code!

source .venv/bin/activate ```

The newly created venv is a standard Python virtual environment preconfigured with all necessary runtime and development dependencies. Additionally, the gt4py package is installed in editable mode, allowing for seamless development and testing. To install new packages in this environment, use the uv pip subcommand which emulates the pip interface and is generally much faster than the original pip tool (which is also available within the venv although its use is discouraged).

The pyproject.toml file contains both the definition of the gt4py Python distribution package and the settings of the development tools used in this project, most notably uv, ruff, and mypy. It also contains dependency groups (see PEP 735 for further reference) with the development requirements listed in different groups (build, docs, lint, test, typing, ...) and collected together in the general dev group, which gets installed by default by uv as mentioned above.

Development Tasks (/scripts)

Recurrent development tasks like bumping versions of used development tools or required third party dependencies have been collected as separate modules in the /scripts folder, and collected as subcommands of the scripts-cli.py tool. Read the tool help for a brief description of every task and always use this tool to update the versions and sync the version configuration across different files (e.g. pyproject.toml and .pre-commit-config.yaml).

📖 Documentation

GT4Py uses the Sphinx tool for the documentation. To build browseable HTML documentation, install the required tools provided in the docs dependency group:

bash uv sync --group docs --extra standard # or --group dev

(Note that most likely these tools are already installed in your development environment, since the docs group is included in the dev group, which installed by default by uv sync if no dependency groups are specified.)

Once the requirements are already installed, then build the docs using:

bash cd gt4py/docs/user/cartesian make html # run 'make help' for a list of targets

⚖️ License

GT4Py is licensed under the terms of the BSD-3-Clause.

Owner

  • Name: GridTools
  • Login: GridTools
  • Kind: organization

GitHub Events

Total
  • Create event: 66
  • Commit comment event: 4
  • Release event: 2
  • Issues event: 35
  • Watch event: 15
  • Delete event: 32
  • Issue comment event: 377
  • Push event: 761
  • Pull request review comment event: 1,947
  • Pull request review event: 1,795
  • Pull request event: 784
  • Fork event: 3
Last Year
  • Create event: 66
  • Commit comment event: 4
  • Release event: 2
  • Issues event: 35
  • Watch event: 15
  • Delete event: 32
  • Issue comment event: 377
  • Push event: 761
  • Pull request review comment event: 1,947
  • Pull request review event: 1,795
  • Pull request event: 784
  • Fork event: 3

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 1,564
  • Total Committers: 34
  • Avg Commits per committer: 46.0
  • Development Distribution Score (DDS): 0.848
Past Year
  • Commits: 477
  • Committers: 15
  • Avg Commits per committer: 31.8
  • Development Distribution Score (DDS): 0.742
Top Committers
Name Email Commits
Hannes Vogt h****s@h****e 237
Till Ehrengruber t****r@c****h 200
edopao e****e@c****h 191
Johann Dahm j****m@g****m 158
Enrique González Paredes e****g@c****h 143
gronerl l****r@c****h 105
Philip Müller 1****r 92
Felix Thaler t****r@c****h 70
Rico Haeuselmann r****n@g****h 67
Roman Cattaneo r****c 62
Nicoletta Farabullini 4****i 54
Rico Häuselmann r****h@c****h 41
eddie-c-davis 2****s 27
SF-N s****i@e****t 23
Péter Kardos k****4@h****m 19
Florian Deconinck d****n@g****m 16
Stefano Ubbiali s****i@p****h 11
Tobias Wicky-Pfund w****b@g****m 10
Eddie Davis e****d@v****m 7
Samuel k****l@g****m 6
mroethlin 5****n 4
Ioannis Magkanaris i****s@c****h 3
ninaburg 8****g 3
BenWeber42 d****r@g****m 2
Christos Kotsalos c****s@c****h 2
Magdalena l****m@e****h 2
abishekg7 5****7 2
Kalman Szenes 5****s 1
Oliver Elbert o****6@g****m 1
Oliver Fuhrer o****r 1
and 4 more...
Committer Domains (Top 20 + Academic)
cscs.ch: 8 noaa.gov: 1 ethz.ch: 1 vulcan.com: 1 phys.ethz.ch: 1 ecmwf.int: 1 gmx.ch: 1 havogt.de: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 127
  • Total pull requests: 1,554
  • Average time to close issues: 12 months
  • Average time to close pull requests: 21 days
  • Total issue authors: 25
  • Total pull request authors: 25
  • Average comments per issue: 0.9
  • Average comments per pull request: 0.77
  • Merged pull requests: 1,120
  • Bot issues: 0
  • Bot pull requests: 6
Past Year
  • Issues: 46
  • Pull requests: 932
  • Average time to close issues: 18 days
  • Average time to close pull requests: 7 days
  • Issue authors: 12
  • Pull request authors: 16
  • Average comments per issue: 0.43
  • Average comments per pull request: 0.65
  • Merged pull requests: 664
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • havogt (30)
  • tehrengruber (20)
  • romanc (12)
  • jdahm (12)
  • egparedes (8)
  • FlorianDeconinck (6)
  • gronerl (6)
  • philip-paul-mueller (4)
  • edopao (4)
  • petiaccja (3)
  • xyuan (3)
  • muellch (2)
  • samkellerhals (2)
  • nfarabullini (2)
  • mbianco (2)
Pull Request Authors
  • edopao (374)
  • havogt (296)
  • tehrengruber (243)
  • philip-paul-mueller (162)
  • romanc (127)
  • egparedes (98)
  • SF-N (48)
  • nfarabullini (43)
  • DropD (39)
  • FlorianDeconinck (30)
  • samkellerhals (15)
  • twicki (14)
  • petiaccja (14)
  • xyuan (8)
  • kotsaloscv (8)
Top Labels
Issue Labels
gt4py.cartesian (36) gt4py.next (32) triage: bug (22) triage: enhancement (21) module: backend (18) declarative (14) module: frontend (10) module: dace (9) priority: low (7) good first issue (5) priority: intermediate (4) module: tests (4) priority: high (3) module: analysis (3) triage: invalid (2) module: storage (2) triage: exploration (2) cartesian (2) triage: performance (1) infrastructure (1) triage: question (1) module: eve (1) module: documentation (1) feedback wanted (1)
Pull Request Labels
move to main (7) dependencies (6) gt4py.cartesian (6)

Packages

  • Total packages: 3
  • Total downloads:
    • pypi 30,087 last-month
  • Total dependent packages: 0
    (may contain duplicates)
  • Total dependent repositories: 2
    (may contain duplicates)
  • Total versions: 29
  • Total maintainers: 1
proxy.golang.org: github.com/GridTools/gt4py
  • Versions: 10
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.4%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 6 months ago
proxy.golang.org: github.com/gridtools/gt4py
  • Versions: 10
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.4%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 6 months ago
pypi.org: gt4py

Python library for generating high-performance implementations of stencil kernels for weather and climate modeling from a domain-specific language (DSL)

  • Versions: 9
  • Dependent Packages: 0
  • Dependent Repositories: 2
  • Downloads: 30,087 Last month
Rankings
Forks count: 6.3%
Dependent packages count: 7.3%
Stargazers count: 7.5%
Dependent repos count: 11.8%
Average: 14.2%
Downloads: 38.3%
Maintainers (1)
gridtools
Last synced: 6 months ago

Dependencies

requirements-dev.txt pypi
  • bump2version >=1.0.1 development
  • check-manifest >=0.40 development
  • darglint >=1.6 development
  • factory-boy >=3.1 development
  • flake8 >=5.0.4 development
  • flake8-bugbear >=20.11.1 development
  • flake8-builtins >=1.5.3 development
  • flake8-debugger >=4.0.0 development
  • flake8-docstrings >=1.5.0 development
  • flake8-eradicate >=1.0.0 development
  • flake8-mutable >=1.2.0 development
  • flake8-rst-docstrings >=0.0.14 development
  • hypothesis >=4.14 development
  • isort * development
  • mypy >=0.980 development
  • pre-commit * development
  • psutil >=5.0 development
  • pygments >=2.7 development
  • pytest * development
  • pytest-cache >=1.0 development
  • pytest-factoryboy >=2.0 development
  • pytest-xdist >=2.4 development
  • seed-isort-config * development
  • setuptools >=40.8.0 development
  • sphinx * development
  • sphinx_rtd_theme * development
  • tox >=3.14 development
.github/workflows/_disabled/documentation.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/_disabled/eve_coverage.yml actions
.github/workflows/_disabled/gt4py-sphinx.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v1 composite
.github/workflows/_disabled/gt4py_coverage.yml actions
.github/workflows/_disabled/template_coverage.yml actions
  • actions/checkout v3 composite
  • actions/github-script v6 composite
  • actions/setup-python v4 composite
  • codecov/codecov-action v3 composite
.github/workflows/code-quality.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • pre-commit/action v3.0.0 composite
.github/workflows/daily-ci.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • slackapi/slack-github-action v1.23.0 composite
.github/workflows/deploy-release.yml actions
  • actions/checkout master composite
  • actions/setup-python v3 composite
  • pypa/gh-action-pypi-publish release/v1 composite
.github/workflows/test-cartesian-fallback.yml actions
.github/workflows/test-cartesian.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v4 composite
.github/workflows/test-eve-fallback.yml actions
.github/workflows/test-eve.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/test-next-fallback.yml actions
.github/workflows/test-next.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/test-storage-fallback.yml actions
.github/workflows/test-storage.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
min-extra-requirements-test.txt pypi
  • astunparse ==1.6.3 test
  • attrs ==21.3 test
  • black ==22.3 test
  • boltons ==20.1 test
  • cached-property ==1.5.1 test
  • clang-format ==9.0 test
  • click ==8.0.0 test
  • cmake ==3.22 test
  • cogapp ==3.3 test
  • coverage ==5.0 test
  • cytoolz ==0.12.0 test
  • dace ==0.14.2 test
  • darglint ==1.6 test
  • deepdiff ==5.6.0 test
  • devtools ==0.6 test
  • factory-boy ==3.1 test
  • flake8 ==5.0.4 test
  • flake8-bugbear ==20.11.1 test
  • flake8-builtins ==1.5.3 test
  • flake8-debugger ==4.0.0 test
  • flake8-docstrings ==1.5.0 test
  • flake8-eradicate ==1.3.0 test
  • flake8-mutable ==1.2.0 test
  • flake8-pyproject ==1.2.2 test
  • flake8-rst-docstrings ==0.0.14 test
  • frozendict ==2.3 test
  • gridtools-cpp ==2.3.1 test
  • hypothesis ==6.0.0 test
  • importlib-resources ==5.0 test
  • isort ==5.10 test
  • jax ==0.4.13 test
  • jinja2 ==3.0.0 test
  • jupytext ==1.14 test
  • lark ==1.1.2 test
  • mako ==1.1 test
  • mypy ==1.0 test
  • nanobind ==1.4.0 test
  • ninja ==1.10 test
  • numpy ==1.21.2 test
  • packaging ==20.0 test
  • pip-tools ==6.10 test
  • pipdeptree ==2.3 test
  • pre-commit ==2.17 test
  • psutil ==5.0 test
  • pybind11 ==2.5 test
  • pygments ==2.7 test
  • pytest ==7.0 test
  • pytest-cache ==1.0 test
  • pytest-cov ==2.8 test
  • pytest-factoryboy ==2.0.3 test
  • pytest-xdist ==2.4 test
  • ruff ==0.0.265 test
  • scipy ==1.7.2 test
  • setuptools ==65.5.0 test
  • sphinx ==4.4 test
  • sphinx_rtd_theme ==1.0 test
  • sympy ==1.7 test
  • tabulate ==0.8.10 test
  • tomli ==2.0.1 test
  • tox ==3.2.0 test
  • types-all ==1.0.0 test
  • typing-extensions ==4.2 test
  • xxhash ==1.4.4 test
min-requirements-test.txt pypi
  • astunparse ==1.6.3 test
  • attrs ==21.3 test
  • black ==22.3 test
  • boltons ==20.1 test
  • cached-property ==1.5.1 test
  • clang-format ==9.0 test
  • click ==8.0.0 test
  • cmake ==3.22 test
  • cogapp ==3.3 test
  • coverage ==5.0 test
  • cytoolz ==0.12.0 test
  • darglint ==1.6 test
  • deepdiff ==5.6.0 test
  • devtools ==0.6 test
  • factory-boy ==3.1 test
  • flake8 ==5.0.4 test
  • flake8-bugbear ==20.11.1 test
  • flake8-builtins ==1.5.3 test
  • flake8-debugger ==4.0.0 test
  • flake8-docstrings ==1.5.0 test
  • flake8-eradicate ==1.3.0 test
  • flake8-mutable ==1.2.0 test
  • flake8-pyproject ==1.2.2 test
  • flake8-rst-docstrings ==0.0.14 test
  • frozendict ==2.3 test
  • gridtools-cpp ==2.3.1 test
  • hypothesis ==6.0.0 test
  • importlib-resources ==5.0 test
  • isort ==5.10 test
  • jinja2 ==3.0.0 test
  • jupytext ==1.14 test
  • lark ==1.1.2 test
  • mako ==1.1 test
  • mypy ==1.0 test
  • nanobind ==1.4.0 test
  • ninja ==1.10 test
  • numpy ==1.21.2 test
  • packaging ==20.0 test
  • pip-tools ==6.10 test
  • pipdeptree ==2.3 test
  • pre-commit ==2.17 test
  • psutil ==5.0 test
  • pybind11 ==2.5 test
  • pygments ==2.7 test
  • pytest ==7.0 test
  • pytest-cache ==1.0 test
  • pytest-cov ==2.8 test
  • pytest-factoryboy ==2.0.3 test
  • pytest-xdist ==2.4 test
  • ruff ==0.0.265 test
  • setuptools ==65.5.0 test
  • sphinx ==4.4 test
  • sphinx_rtd_theme ==1.0 test
  • tabulate ==0.8.10 test
  • tomli ==2.0.1 test
  • tox ==3.2.0 test
  • types-all ==1.0.0 test
  • typing-extensions ==4.2 test
  • xxhash ==1.4.4 test
pyproject.toml pypi
  • astunparse >=1.6.3;python_version<'3.9'
  • attrs >=21.3
  • black >=22.3
  • boltons >=20.1
  • cached-property >=1.5.1
  • click >=8.0.0
  • cmake >=3.22
  • cytoolz >=0.12.0
  • deepdiff >=5.6.0
  • devtools >=0.6
  • frozendict >=2.3
  • gridtools-cpp >=2.3.1,==2.*
  • importlib-resources >=5.0;python_version<'3.9'
  • jinja2 >=3.0.0
  • lark >=1.1.2
  • mako >=1.1
  • nanobind >=1.4.0
  • ninja >=1.10
  • numpy >=1.21.2
  • packaging >=20.0
  • pybind11 >=2.5
  • setuptools >=65.5.0
  • tabulate >=0.8.10
  • typing-extensions >=4.2,<4.6.0
  • xxhash >=1.4.4,<3.1.0
requirements-dev.in pypi
  • clang-format >=9.0 development
  • cogapp >=3.3 development
  • coverage >=5.0 development
  • darglint >=1.6 development
  • factory-boy >=3.1 development
  • flake8 >=5.0.4 development
  • flake8-bugbear >=20.11.1 development
  • flake8-builtins >=1.5.3 development
  • flake8-debugger >=4.0.0 development
  • flake8-docstrings >=1.5.0 development
  • flake8-eradicate >=1.3.0 development
  • flake8-mutable >=1.2.0 development
  • flake8-pyproject >=1.2.2 development
  • flake8-rst-docstrings >=0.0.14 development
  • hypothesis >=6.0.0 development
  • isort >=5.10 development
  • jupytext >=1.14 development
  • mypy >=1.0 development
  • pip-tools >=6.10 development
  • pipdeptree >=2.3 development
  • pre-commit >=2.17 development
  • psutil >=5.0 development
  • pygments >=2.7 development
  • pytest >=7.0 development
  • pytest-cache >=1.0 development
  • pytest-cov >=2.8 development
  • pytest-factoryboy >=2.0.3 development
  • pytest-xdist >=2.4 development
  • ruff >=0.0.265 development
  • sphinx >=4.4 development
  • sphinx_rtd_theme >=1.0 development
  • tomli >=2.0.1 development
  • tox >=3.2.0 development
  • types-all >=1.0.0 development