Data

Tools

Indexes

Applications

Experiments

Open Source Science

verif

Graphical tool for creating verification plots of weather forecasts

https://github.com/wfrt/verif

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ✓
    DOI references
    Found 4 DOI reference(s) in README
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    2 of 4 committers (50.0%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (17.4%) to scientific vocabulary

Keywords

evaluation forecasting prediction statistics verification weather
Last synced: 6 months ago · JSON representation

Repository

Graphical tool for creating verification plots of weather forecasts

Basic Info
  • Host: GitHub
  • Owner: WFRT
  • License: other
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 4.41 MB
Statistics
  • Stars: 96
  • Watchers: 14
  • Forks: 40
  • Open Issues: 3
  • Releases: 13
Topics
evaluation forecasting prediction statistics verification weather
Created over 10 years ago · Last pushed 11 months ago
Metadata Files
Readme License

README.rst 

Forecast verification software
==============================
.. image:: https://img.shields.io/github/v/release/WFRT/verif.svg
  :target: https://github.com/WFRT/verif/releases
  :alt: Release badge
.. image:: https://github.com/WFRT/verif/actions/workflows/python-package.yml/badge.svg
  :target: https://github.com/WFRT/verif/actions/workflows/python-package.yml
  :alt: CI badge
.. image:: https://img.shields.io/badge/DOI-10.1175%2FBAMS--D--22--0253.1-blue
  :target: https://doi.org/10.1175/BAMS-D-22-0253.1
  :alt: Bams Article

Verif is a command-line tool that lets you verify the quality of weather forecasts for point
locations. It can also compare forecasts from different forecasting systems (that have different
models, post-processing methods, etc).

The program reads files with observations and forecasts in a specific format (see "Input files"
below). The input files contain information about dates, forecast lead times, and locations such
that statistics can be aggregated across different dimensions. To ensure a fair comparison among
files, Verif will discard data points where one or more forecast systems have missing forecasts.
Since Verif is a command-line tool, it can be used in scripts to automatically create
verification figures.

Verif version 1.3 has been released (see "Installation Instruction" below). We welcome suggestions
for improvements. Verif is developed by Thomas Nipen (thomasn@met.no), with contributions from many.

Features
--------

* Deterministic metrics such as MAE, bias, correlation, RMSE (e.g. ``-m mae``)
* Threshold-based metrics such as the false alarm rate, ETS, EDI, Yule's Q (e.g. ``-m ets``)
* Probabilistic metrics such as brier score, PIT-histogram, reliability diagrams (e.g. ``-m bs``)
* Special plots like Taylor diagrams (``-m taylor``), quantile-quantile plots (``-m qq``).
* Plot scores as a function of date, lead time, station altitude/lat/longitude (e.g. ``-x date``)
* Show scores on maps (``-type map``)
* Subset the data by specifying a date range and lat/lon range (``-latrange 58,60``)
* Export to text (``-type text``)
* Options to adjust font sizes, label positions, tick marks, legends, etc (``-labfs 14``)
* Anomaly statistics relative to a baseline like climatology (``-c climfile.txt``)
* Output to png, jpeg, eps, etc and specify dimensions and resolution (``-f image.png -dpi 300``)

Resources
---------

* Check out the **wiki** at https://github.com/WFRT/verif/wiki.
* Found a bug? Please report it in the **issue tracker** at https://github.com/WFRT/verif/issues.
* Reach out to the Verif community in the **discussions** at https://github.com/WFRT/verif/discussions.
* Check out our `article `_ published in BAMS.

.. image:: other/image.jpg
    :alt: Example plots
    :width: 400
    :align: center

Getting started
---------------

The easiest way to get started is to install verif with pip:

.. code-block:: bash

    pip3 install verif

The `verif` command-line program should then be available.

Note that the python package "cartopy" is optional and not installed by default when verif is
installed, Verif uses "cartopy" for creating a background map when verification scores are plotted
on a map. To install Cartopy run

.. code-block:: bash

    pip3 install cartopy

To upgrade to a newer version of Verif, run the following:

.. code-block:: bash

   pip3 install verif --upgrade

Examples
--------
To test Verif, you can download example datasets from the github
`discussion page `_. For example, download the following two files from the wind speed dataset: `MEPS.nc `_ (2.5 km regional model; 20MB file size) and `ECMWF.nc `_ (0.2° global model; 24MB file size). Then run the following
commands to test out the software:

.. code-block:: bash

   # Shows mean absolute error as a function of lead-time
   verif MEPS.nc ECMWF.nc -m mae
   # Shows average observed and forecasted values as a function on time
   verif MEPS.nc ECMWF.nc -m obsfcst -x time
   # Shows equitable threat score as a function of threshold
   verif MEPS.nc ECMWF.nc -m ets
   # Shows a reliability diagram for a threshold of 13.9 m/s (gale force winds)
   verif MEPS.nc ECMWF.nc -m reliability -r 13.9
   # Shows Brier skill score as a function of threshold
   verif MEPS.nc ECMWF.nc -m bss -x threshold

How to cite
-----------
Nipen, T. N., R. B. Stull, C. Lussana, and I. A. Seierstad, 2023: `Verif: A Weather-Prediction Verification Tool for Effective Product Development `_. Bull. Amer. Meteor.  Soc., 104, E1610–E1618, https://doi.org/10.1175/BAMS-D-22-0253.1.

Copyright and license
---------------------

Copyright © 2013-2025 UBC Weather Forecast Research Team. Verif is licensed under the 3-clause
BSD license. See LICENSE file.

Owner

  • Name: Weather Forecast Research Team
  • Login: WFRT
  • Kind: organization

GitHub Events

Total
  • Issues event: 3
  • Watch event: 9
  • Delete event: 1
  • Issue comment event: 2
  • Push event: 10
  • Gollum event: 80
  • Pull request event: 4
  • Fork event: 1
  • Create event: 2
Last Year
  • Issues event: 3
  • Watch event: 9
  • Delete event: 1
  • Issue comment event: 2
  • Push event: 10
  • Gollum event: 80
  • Pull request event: 4
  • Fork event: 1
  • Create event: 2

Committers

Last synced: over 1 year ago

All Time
  • Total Commits: 811
  • Total Committers: 4
  • Avg Commits per committer: 202.75
  • Development Distribution Score (DDS): 0.009
Past Year
  • Commits: 20
  • Committers: 1
  • Avg Commits per committer: 20.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Thomas Nipen t****n@g****m 804
Tomas Barton t****n@u****a 4
David Siuta d****a@e****a 2
tc637 t****4@h****m 1
Committer Domains (Top 20 + Academic)
eos.ubc.ca: 1 ualberta.ca: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 98
  • Total pull requests: 7
  • Average time to close issues: over 1 year
  • Average time to close pull requests: 4 months
  • Total issue authors: 12
  • Total pull request authors: 5
  • Average comments per issue: 0.45
  • Average comments per pull request: 1.29
  • Merged pull requests: 4
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 2
  • Pull requests: 3
  • Average time to close issues: about 7 hours
  • Average time to close pull requests: 2 days
  • Issue authors: 1
  • Pull request authors: 3
  • Average comments per issue: 0.5
  • Average comments per pull request: 0.33
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • tnipen (84)
  • martadd (2)
  • evenmn (2)
  • dsiuta (2)
  • DikraK (1)
  • Marcoii1801 (1)
  • agiord (1)
  • MigueLesPaul (1)
  • cjmelick (1)
  • brey (1)
  • mokrzesa (1)
  • dhanraj125 (1)
Pull Request Authors
  • tommz9 (3)
  • tnipen (2)
  • evenmn (1)
  • yma042 (1)
  • mokrzesa (1)
Top Labels
Issue Labels
wontfix (5) enhancement (4) bug (1) duplicate (1)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 305 last-month
  • Total dependent packages: 1
  • Total dependent repositories: 5
  • Total versions: 17
  • Total maintainers: 1
pypi.org: verif

A verification program for meteorological forecasts and observations

  • Versions: 17
  • Dependent Packages: 1
  • Dependent Repositories: 5
  • Downloads: 305 Last month
Rankings
Dependent packages count: 4.8%
Forks count: 6.5%
Dependent repos count: 6.6%
Stargazers count: 7.8%
Average: 8.7%
Downloads: 17.7%
Maintainers (1)
tnipen
Last synced: 6 months ago

Dependencies

requirements.txt pypi
  • coveralls *
  • future *
  • matplotlib *
  • netCDF4 *
  • numpy >=1.7
  • pep8 *
  • scipy *
setup.py pypi
  • future *
  • matplotlib *
  • netCDF4 *
  • numpy >=1.7
  • scipy *
  • six *
.github/workflows/codeql-analysis.yml actions
  • actions/checkout v3 composite
  • github/codeql-action/analyze v2 composite
  • github/codeql-action/autobuild v2 composite
  • github/codeql-action/init v2 composite
.github/workflows/python-package.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite