NEMSEER

NEMSEER: A Python package for downloading and handling historical National Electricity Market forecast data produced by the Australian Energy Market Operator - Published in JOSS (2023)

https://github.com/unsw-ceem/nemseer

Science Score: 100.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
    Found 8 DOI reference(s) in README and JOSS metadata
  • Academic publication links
    Links to: researchgate.net, joss.theoj.org
  • Committers with academic emails
  • Institutional organization owner
    Organization unsw-ceem has institutional domain (ceem.unsw.edu.au)
  • JOSS paper metadata
    Published in Journal of Open Source Software

Keywords

aemo australia energy forecast national-electricity-market nem nemweb pasa pre-dispatch python

Scientific Fields

Earth and Environmental Sciences Physical Sciences - 76% confidence
Medicine Life Sciences - 40% confidence
Artificial Intelligence and Machine Learning Computer Science - 40% confidence
Last synced: 4 months ago · JSON representation ·

Repository

A package for downloading and handling forecasts for the National Electricity Market (NEM) from the Australian Energy Market Operator (AEMO).

Basic Info
Statistics
  • Stars: 31
  • Watchers: 3
  • Forks: 6
  • Open Issues: 7
  • Releases: 19
Topics
aemo australia energy forecast national-electricity-market nem nemweb pasa pre-dispatch python
Created over 3 years ago · Last pushed 5 months ago
Metadata Files
Readme Changelog Contributing License Citation

README.md

nemseer

PyPI version Continuous Integration and Deployment Documentation Status codecov pre-commit.ci status Code style: black DOI

A package for downloading and handling historical National Electricity Market (NEM) forecast data produced by the Australian Energy Market Operator (AEMO).

Installation

bash pip install nemseer

Many nemseer use-cases require NEMOSIS, which can also be installed using pip:

bash pip install nemosis

Overview

nemseer allows you to access historical AEMO pre-dispatch and Projected Assessment of System Adequacy (PASA) forecast[^1] data available through the MMSDM Historical Data SQLLoader. nemseer can then compile this data into pandas DataFrames or xarray Datasets.

An overview of nemseer functionality and potential use-cases are provided in the JOSS paper for this package.

forecast_overview

Source: Reserve services in the National Electricity Market, AEMC, 2021

Whereas PASA processes are primarily used to assess resource adequacy based on technical inputs and assumptions for resources in the market (i.e. used to answer questions such as "can operational demand be met in the forecast horizon with a sufficient safety (reserve) margin?"), pre-dispatch processes incorporate the latest set of market participant offers and thus produce regional prices forecasts for energy and frequency control ancillary services (FCAS). Overviews of the various pre-dispatch and PASA processes can be found in the glossary.

[^1]: We use the term "forecast" loosely, especially given that these "forecasts" change once participants update offer information (e.g. through rebidding) or submit revised resource availabilities and energy constraints. Both of these are intended outcomes of these "ahead processes", which are run to provide system and market information to participants to inform their decision-making. However, to avoid confusion and to ensure consistency with the language used by AEMO, we use the terms "forecast" (or outputs) and "forecast types" (or ahead processes) in nemseer.

nemseer enables you to download and work with data for the following forecast types. Where available, AEMO process and table descriptions are linked:

  1. 5-minute pre-dispatch (P5MIN: Table descriptions)
  2. Pre-dispatch (PREDISPATCH: Table descriptions)
  3. Pre-dispatch Projected Assessment of System Adequacy (PDPASA: Tables and Descriptions)
  4. Short Term Projected Assessment of System Adequacy (STPASA: Table descriptions)
  5. Medium Term Projected Assessment of System Adequacy (MTPASA: Table descriptions)

Another helpful reference for PASA information is AEMO's Reliability Standard Implementation Guidelines.

ST PASA Replacement Project

Note that the methodologies for PD PASA and ST PASA are being reviewed by AEMO. In particular, the ST PASA Replacement project will combine PD PASA and ST PASA into ST PASA. For more detail, refer to the final determination of the rule change and the AEMO ST PASA Replacement Project home page.

Usage

Glossary

The glossary contains overviews of the PASA and pre-dispatch processes, and descriptions of terminology used in nemseer.

Quick start

Check out the Quick start for guide on to use nemseer.

Examples

Some use case examples have been included in the Examples section of the documentation.

Support

If you are having an issue with this software that has not already been raised in the issues register, please raise a new issue.

Contributing

Interested in contributing? Check out the contributing guidelines, which also includes steps to install nemseer for development.

Please note that this project is released with a Code of Conduct. By contributing to this project, you agree to abide by its terms.

Citation

If you use nemseer, please cite the JOSS paper for this package

If you use code or analysis from any of the demand error and/or price convergence examples in the documentation, please also cite NEMOSIS via this conference paper

Licenses

nemseer was created by Abhijith Prakash. It is licensed under the terms of GNU GPL-3.0-or-later licences.

The content within the documentation for this project is licensed under a Creative Commons Attribution 4.0 International License.

Credits

nemseer was created with cookiecutter and the py-pkgs-cookiecutter template.

Development of nemseer was funded by the UNSW Digital Grid Futures Institute.

Contributor Acknowledgements

Thanks to:

  • Nicholas Gorman for reviewing nemseer code
  • Krisztina Katona for reviewing and improving the glossary
  • Dylan McConnell for assistance in interpreting PASA run types
  • Declan Heim for suggesting improvements to nemseer examples

Owner

  • Name: Collaboration on Energy and Environmental Markets (CEEM)
  • Login: UNSW-CEEM
  • Kind: organization
  • Location: Sydney Australia

JOSS Publication

NEMSEER: A Python package for downloading and handling historical National Electricity Market forecast data produced by the Australian Energy Market Operator
Published
December 09, 2023
Volume 8, Issue 92, Page 5883
Authors
Abhijith Prakash ORCID
School of Electrical Engineering and Telecommunications, University of New South Wales, Australia, Collaboration on Energy and Environmental Markets (CEEM), University of New South Wales, Australia
Anna Bruce ORCID
School of Photovoltaics and Renewable Energy Engineering, University of New South Wales, Australia, Collaboration on Energy and Environmental Markets (CEEM), University of New South Wales, Australia
Iain MacGill ORCID
School of Electrical Engineering and Telecommunications, University of New South Wales, Australia, Collaboration on Energy and Environmental Markets (CEEM), University of New South Wales, Australia
Editor
Arfon Smith ORCID
Tags
NEM National Electricity Market Forecast PASA Pre dispatch

Citation (CITATION.cff)

cff-version: "1.2.0"
authors:
- family-names: Prakash
  given-names: Abhijith
  orcid: "https://orcid.org/0000-0002-2945-4757"
- family-names: Bruce
  given-names: Anna
  orcid: "https://orcid.org/0000-0003-1820-4039"
- family-names: MacGill
  given-names: Iain
  orcid: "https://orcid.org/0000-0002-9587-6835"
contact:
- family-names: Prakash
  given-names: Abhijith
  orcid: "https://orcid.org/0000-0002-2945-4757"
doi: v1.0.7
message: If you use this software, please cite our article in the
  Journal of Open Source Software.
preferred-citation:
  authors:
  - family-names: Prakash
    given-names: Abhijith
    orcid: "https://orcid.org/0000-0002-2945-4757"
  - family-names: Bruce
    given-names: Anna
    orcid: "https://orcid.org/0000-0003-1820-4039"
  - family-names: MacGill
    given-names: Iain
    orcid: "https://orcid.org/0000-0002-9587-6835"
  date-published: 2023-12-09
  doi: 10.21105/joss.05883
  issn: 2475-9066
  issue: 92
  journal: Journal of Open Source Software
  publisher:
    name: Open Journals
  start: 5883
  title: "NEMSEER: A Python package for downloading and handling
    historical National Electricity Market forecast data produced by the
    Australian Energy Market Operator"
  type: article
  url: "https://joss.theoj.org/papers/10.21105/joss.05883"
  volume: 8
title: "NEMSEER: A Python package for downloading and handling
  historical National Electricity Market forecast data produced by the
  Australian Energy Market Operator"

GitHub Events

Total
  • Issues event: 4
  • Watch event: 10
  • Delete event: 1
  • Issue comment event: 3
  • Push event: 11
  • Pull request event: 1
  • Fork event: 2
  • Create event: 2
Last Year
  • Issues event: 4
  • Watch event: 10
  • Delete event: 1
  • Issue comment event: 3
  • Push event: 11
  • Pull request event: 1
  • Fork event: 2
  • Create event: 2

Committers

Last synced: 5 months ago

All Time
  • Total Commits: 555
  • Total Committers: 3
  • Avg Commits per committer: 185.0
  • Development Distribution Score (DDS): 0.034
Past Year
  • Commits: 1
  • Committers: 1
  • Avg Commits per committer: 1.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
prakaa a****7@g****m 536
pre-commit-ci[bot] 6****] 18
Krasztana2 1****2 1

Issues and Pull Requests

Last synced: 4 months ago

All Time
  • Total issues: 33
  • Total pull requests: 36
  • Average time to close issues: 11 days
  • Average time to close pull requests: 8 days
  • Total issue authors: 6
  • Total pull request authors: 3
  • Average comments per issue: 1.0
  • Average comments per pull request: 1.03
  • Merged pull requests: 33
  • Bot issues: 0
  • Bot pull requests: 18
Past Year
  • Issues: 4
  • Pull requests: 1
  • Average time to close issues: 3 months
  • Average time to close pull requests: N/A
  • Issue authors: 3
  • Pull request authors: 1
  • Average comments per issue: 0.0
  • Average comments per pull request: 1.0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 1
Top Authors
Issue Authors
  • prakaa (28)
  • atticmuses (2)
  • nick-gorman (1)
  • notuntoward (1)
  • JTOrange (1)
  • nickmasonsmith (1)
Pull Request Authors
  • pre-commit-ci[bot] (25)
  • prakaa (17)
  • Krasztana2 (1)
Top Labels
Issue Labels
enhancement (9) bug (7) documentation (5) good first issue (2) test (2) dependency (1)
Pull Request Labels
documentation (5) enhancement (4) bug (1)

Dependencies

poetry.lock pypi
  • alabaster 0.7.12 develop
  • appnope 0.1.3 develop
  • asttokens 2.0.5 develop
  • atomicwrites 1.4.0 develop
  • attrs 21.4.0 develop
  • babel 2.10.3 develop
  • backcall 0.2.0 develop
  • beautifulsoup4 4.11.1 develop
  • certifi 2022.6.15 develop
  • charset-normalizer 2.1.0 develop
  • colorama 0.4.5 develop
  • coverage 6.4.1 develop
  • decorator 5.1.1 develop
  • docutils 0.18.1 develop
  • executing 0.8.3 develop
  • feather-format 0.4.1 develop
  • flake8 4.0.1 develop
  • furo 2022.6.21 develop
  • idna 3.3 develop
  • imagesize 1.4.1 develop
  • importlib-metadata 4.12.0 develop
  • iniconfig 1.1.1 develop
  • ipdb 0.13.9 develop
  • ipython 8.4.0 develop
  • jedi 0.18.1 develop
  • jinja2 3.1.2 develop
  • joblib 1.1.0 develop
  • markupsafe 2.1.1 develop
  • matplotlib-inline 0.1.3 develop
  • mccabe 0.6.1 develop
  • mypy 0.961 develop
  • mypy-extensions 0.4.3 develop
  • nemosis 3.1.0 develop
  • numpy 1.23.0 develop
  • packaging 21.3 develop
  • pandas 1.4.3 develop
  • parso 0.8.3 develop
  • pexpect 4.8.0 develop
  • pickleshare 0.7.5 develop
  • pluggy 1.0.0 develop
  • pockets 0.9.1 develop
  • prompt-toolkit 3.0.30 develop
  • ptyprocess 0.7.0 develop
  • pure-eval 0.2.2 develop
  • py 1.11.0 develop
  • pyarrow 8.0.0 develop
  • pycodestyle 2.8.0 develop
  • pyflakes 2.4.0 develop
  • pygments 2.12.0 develop
  • pyparsing 3.0.9 develop
  • pytest 7.1.2 develop
  • pytest-cov 3.0.0 develop
  • python-dateutil 2.8.2 develop
  • pytz 2022.1 develop
  • requests 2.28.1 develop
  • setuptools 63.1.0 develop
  • six 1.16.0 develop
  • snowballstemmer 2.2.0 develop
  • soupsieve 2.3.2.post1 develop
  • sphinx 5.0.2 develop
  • sphinx-basic-ng 0.0.1a12 develop
  • sphinx-copybutton 0.5.0 develop
  • sphinxcontrib-applehelp 1.0.2 develop
  • sphinxcontrib-devhelp 1.0.2 develop
  • sphinxcontrib-htmlhelp 2.0.0 develop
  • sphinxcontrib-jsmath 1.0.1 develop
  • sphinxcontrib-napoleon 0.7 develop
  • sphinxcontrib-qthelp 1.0.3 develop
  • sphinxcontrib-serializinghtml 1.1.5 develop
  • stack-data 0.3.0 develop
  • toml 0.10.2 develop
  • tomli 2.0.1 develop
  • traitlets 5.3.0 develop
  • typing-extensions 4.3.0 develop
  • urllib3 1.26.9 develop
  • wcwidth 0.2.5 develop
  • xarray 2022.3.0 develop
  • xlrd 2.0.1 develop
  • zipp 3.8.0 develop
  • greenlet 1.1.2
  • sqlalchemy 1.4.39
pyproject.toml pypi
  • SQLAlchemy ^1.4.39
  • attrs ^21
  • beautifulsoup4 ^4
  • nemosis ^3
  • pandas ^1
  • python ^3.8
  • requests ^2
  • xarray ^2022
.github/workflows/cicd.yml actions
  • actions/cache v2 composite
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • codecov/codecov-action v3 composite
  • snok/install-poetry v1 composite