A Framework to Quality Control Oceanographic Data

A Framework to Quality Control Oceanographic Data - Published in JOSS (2020)

https://github.com/castelao/cotede

Science Score: 98.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 16 DOI reference(s) in README and JOSS metadata
✓
Academic publication links
Links to: arxiv.org, joss.theoj.org, zenodo.org
○
Committers with academic emails
○
Institutional organization owner
✓
JOSS paper metadata
Published in Journal of Open Source Software

Scientific Fields

Earth and Environmental Sciences Physical Sciences - 87% confidence

Mathematics Computer Science - 84% confidence

Last synced: 4 months ago · JSON representation ·

Repository

Quality Control of Oceanographic Data

Basic Info

Host: GitHub
Owner: castelao
License: bsd-3-clause
Language: Python
Default Branch: master
Homepage: https://cotede.readthedocs.io
Size: 1.24 MB

Statistics

Stars: 56
Watchers: 13
Forks: 20
Open Issues: 10
Releases: 5

Created over 12 years ago · Last pushed over 2 years ago

Metadata Files

Readme Changelog Contributing License Citation Authors Zenodo

README.rst

======
CoTeDe
======

.. image:: https://joss.theoj.org/papers/10.21105/joss.02063/status.svg
   :target: https://doi.org/10.21105/joss.02063

.. image:: https://zenodo.org/badge/10284681.svg
   :target: https://zenodo.org/badge/latestdoi/10284681

.. image:: https://readthedocs.org/projects/cotede/badge/?version=latest
   :target: https://cotede.readthedocs.io/en/latest/?badge=latest
   :alt: Documentation Status

.. image:: https://github.com/castelao/CoTeDe/actions/workflows/ci.yml/badge.svg
   :target: https://github.com/castelao/CoTeDe/actions/workflows/ci.yml)

.. image:: https://codecov.io/gh/castelao/CoTeDe/branch/master/graph/badge.svg
   :target: https://codecov.io/gh/castelao/CoTeDe

.. image:: https://img.shields.io/pypi/v/cotede.svg
   :target: https://pypi.python.org/pypi/cotede

.. image:: https://mybinder.org/badge_logo.svg
   :target: https://mybinder.org/v2/gh/castelao/CoTeDe/master?filepath=docs%2Fnotebooks


`CoTeDe `_ is an Open Source Python package to quality control (QC) oceanographic data such as temperature and salinity.
It was designed to attend individual scientists as well as real-time operations on large data centers.
To achieve that, CoTeDe is highly customizable, giving the user full control to compose the desired set of tests including the specific parameters of each test, or choose from a list of preset QC procedures.

I believe that we can do better than we have been doing with more flexible classification techniques, which includes machine learning. My goal is to minimize the burden on manual expert QC improving the consistency, performance, and reliability of the QC procedure for oceanographic data, especially for real-time operations.

CoTeDe is the result from several generations of quality control systems that started in 2006 with real-time QC of TSGs and were later expanded for other platforms including CTDs, XBTs, gliders, and others.


----------
Why CoTeDe
----------

CoTeDe contains several QC procedures that can be easily combined in different ways:

- Pre-set standard tests according to the recommendations by GTSPP, EGOOS, XBT, Argo or QARTOD;
- Custom set of tests, including user defined thresholds;
- Two different fuzzy logic approaches: as proposed by Timms et. al 2011 & Morello et. al. 2014, and using usual defuzification by the bisector;
- A novel approach based on Anomaly Detection, described by `Castelao 2021 `_ (available since 2014 ``_).

Each measuring platform is a different realm with its own procedures, metadata, and meaningful visualization. 
So CoTeDe focuses on providing a robust framework with the procedures and lets each application, and the user, to decide how to drive the QC.
For instance, the `pySeabird package `_ is another package that understands CTD and uses CoTeDe as a plugin to QC.

-------------
Documentation
-------------

A detailed documentation is available at http://cotede.readthedocs.org, while a collection of notebooks with examples is available at
http://nbviewer.ipython.org/github/castelao/CoTeDe/tree/master/docs/notebooks/

--------
Citation
--------

If you use CoTeDe, or replicate part of it, in your work/package, please consider including the reference:

Castelão, G. P., (2020). A Framework to Quality Control Oceanographic Data. Journal of Open Source Software, 5(48), 2063, https://doi.org/10.21105/joss.02063

::

  @article{Castelao2020,
    doi = {10.21105/joss.02063},
    url = {https://doi.org/10.21105/joss.02063},
    year = {2020},
    publisher = {The Open Journal},
    volume = {5},
    number = {48},
    pages = {2063},
    author = {Guilherme P. Castelao},
    title = {A Framework to Quality Control Oceanographic Data},
    journal = {Journal of Open Source Software}
  }

For the Anomaly Detection techinique specifically, which was implemented in CoTeDe, please include the reference:

Castelão, G. P. (2021). A Machine Learning Approach to Quality Control Oceanographic Data. Computers & Geosciences, https://doi.org/10.1016/j.cageo.2021.104803

::

  @article{Castelao2021,
    doi = {10.1016/j.cageo.2021.104803},
    url = {https://doi.org/10.1016/j.cageo.2021.104803},
    year = {2021},
    publisher = {Elsevier},
    author = {Guilherme P. Castelao},
    title = {A Machine Learning Approach to Quality Control Oceanographic Data},
    journal = {Computers and Geosciences}
  }

If you are concerned about reproducibility, please include the DOI provided by Zenodo on the top of this page, which is associated with a specific release (version).

Owner

Name: Guilherme Castelão
Login: castelao
Kind: user
Location: CO
Company: @NREL

Website: www.castelao.net
Repositories: 80
Profile: https://github.com/castelao

multi-class: PhD in Physical Oceanography, offshore solo sailor, Rustacean and Pythonista.

JOSS Publication

A Framework to Quality Control Oceanographic Data

Published

April 07, 2020

DOI

10.21105/joss.02063

Volume 5, Issue 48, Page 2063

Authors

Guilherme P. Castelao

Scripps Institution of Oceanography

Editor

Kristen Thyng

Citation (CITATION.cff)

cff-version: 1.1.0
message: If you use this software, please cite it using these metadata.
title: 'A Framework to Quality Control Oceanographic Data'
authors:
- given-names: Guilherme
  family-names: Castelao
  affiliation: Scripps Institution of Oceanography - UC San Diego
  orcid: https://orcid.org/0000-0002-6765-0708
version: 0.23.8
doi: 10.21105/joss.02063
date-released: 2021-01-26
repository-code: https://github.com/castelao/CoTeDe
license: BSD-3 Clause

GitHub Events

Total

Issues event: 1
Watch event: 6
Fork event: 3

Last Year

Issues event: 1
Watch event: 6
Fork event: 3

Committers

Last synced: 5 months ago

All Time

Total Commits: 1,026
Total Committers: 4
Avg Commits per committer: 256.5
Development Distribution Score (DDS): 0.004

Past Year

Commits: 0
Committers: 0
Avg Commits per committer: 0.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
Gui	g**e@c**t	1,022
Bill Mills	m**j@g**m	2
Simon Good	s****d	1
Kristen Thyng	k**g@g**m	1

Committer Domains (Top 20 + Academic)

castelao.net: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time

Total issues: 42
Total pull requests: 32
Average time to close issues: 7 months
Average time to close pull requests: 2 months
Total issue authors: 9
Total pull request authors: 6
Average comments per issue: 0.45
Average comments per pull request: 0.97
Merged pull requests: 24
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 1
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 1
Pull request authors: 0
Average comments per issue: 0.0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

castelao (29)
s-good (4)
jessicaaustin (3)
evanleeturner (1)
shaunwbell (1)
guillotp (1)
alkis05 (1)
BillMills (1)
lidong74 (1)

Pull Request Authors

castelao (22)
BillMills (3)
jessicaaustin (2)
s-good (2)
electricsam (2)
kthyng (1)

Top Labels

Issue Labels

bug (17) enhancement (6)

Pull Request Labels

bug (1)

Packages

Total packages: 2
Total downloads:
- pypi 546 last-month

Total dependent packages: 1
(may contain duplicates)
Total dependent repositories: 5
(may contain duplicates)
Total versions: 31
Total maintainers: 1

pypi.org: cotede

Quality Control of Oceanographic Data

Documentation: https://cotede.readthedocs.io/
License: Copyright (c) 2011-2023, Guilherme Pimenta Castelão All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the CoTeDe Team nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Latest release: 0.23.9
published over 2 years ago

Versions: 28
Dependent Packages: 1
Dependent Repositories: 5
Downloads: 546 Last month
Docker Downloads: 0

Rankings

Docker downloads count: 4.3%

Dependent packages count: 4.8%

Dependent repos count: 6.6%

Average: 10.4%

Downloads: 25.9%

Maintainers (1)

castelao

Last synced: 4 months ago

conda-forge.org: cotede

Homepage: http://cotede.castelao.net
License: BSD-3-Clause
Latest release: 0.23.7
published over 4 years ago

Versions: 3
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent repos count: 34.0%

Forks count: 39.0%

Stargazers count: 40.1%

Average: 41.1%

Dependent packages count: 51.2%

Last synced: 4 months ago

Dependencies

.github/workflows/ci.yml actions

actions/cache v2 composite
actions/checkout v3 composite
actions/setup-python v2 composite

.github/workflows/publish-to-test-pypi.yml actions

actions/checkout v3 composite
actions/setup-python v3 composite

docs/environment.yml conda

matplotlib
netcdf4
numpy
pip
python
scipy

environment.yml conda

bokeh 2.2.3.*
matplotlib 3.3.2.*
netcdf4 1.5.3.*
numpy 1.19.2.*
oceansdb 0.8.13.*
pip
python 3.8.5.*
scipy 1.5.2.*
xarray 0.16.2.*

pyproject.toml pypi

Click >=6.6
numpy >=1.20
oceansdb >= 0.8.13
scipy >= 1.0.0

setup.py pypi

A Framework to Quality Control Oceanographic Data

Science Score: 98.0%

Scientific Fields

Repository

Basic Info

Statistics

Metadata Files

README.rst

Owner

JOSS Publication

A Framework to Quality Control Oceanographic Data

Authors

Editor

Tags

Citation (CITATION.cff)

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: cotede

Rankings

Maintainers (1)

conda-forge.org: cotede

Rankings

Dependencies