oceandatatools: a MATLAB toolbox for interacting with bulk freely-available oceanographic data

Copyright (c) 2020 lnferris

oceandatatools simplifies the process of extracting, formatting, and visualizing freely-available oceanographic data. While a wealth of oceanographic data is accessible online, some end-users may be dissuaded from utilizing this data due to the overhead associated with obtaining and formatting it into usable data structures. oceandatatools solves this problem by allowing the user to transform common oceanographic data sources into uniform structs, call generalized functions on these structs, easily perform custom calculations, and make graphics.

Find a bug, have a question, or want to chat about contributing? Open an issue or email lnferris@alum.mit.edu.

Getting Started

Dependencies

Accessing Help

How to Contribute

Contents

Finding Data

Citing ODT

Getting Started

1. Download bathymetry.
2. Download nctoolbox. You will need to run the command setup_nctoolbox at the beginning of each MATLAB session.
3. Add oceandatatools and nctoolbox to the path. Specifically, the following folders must be added to the path:

• oceandatatools/oceandatatools
• oceandatatools/oceandatatools/utilities
• nctoolbox/

1. Run each demonstration in demos/demos.m, which contains example usages for all functions. All required test data is included in data/.

Functions are named using a two-part system. The prefix (argo_, bathymetry_, general_, etc.) indicates the appropriate data source, while the suffix (\_build, \_profiles, \_section, etc.) indicates the action performed. Functions with the \_build suffix load raw data into uniform structs (e.g. argo, cruise, hycom, mercator, woa, wod). Uniform structs created by \_build functions are compatable with any general_ function.

Data sources currently supported: | Data Source | DOI, Product Code, or Link | |:-- |:--| | Argo floats | doi:10.17882/42182 | | Smith & Sandwell bathymetry | doi:10.1126/science.277.5334.1956 | | IOOS Glider DAC | https://gliders.ioos.us/ | | MOCHA Climatology | doi:10.7282/T3XW4N4M | | HYbrid Coordinate Ocean Model | https://hycom.org | | CMEMS Global Ocean 1/12° Physics Analysis and Forecast | GLOBALANALYSISFORECAST_ PHY001024 | | CMEMS Global Ocean Waves Multi Year | GLOBALREANALYSISWAV001032 | | GO-SHIP hydrographic cruises | https://www.go-ship.org/ | | World Ocean Atlas 2018 | https://www.ncei.noaa.gov/products/world-ocean-atlas | | World Ocean Database | https://www.ncei.noaa.gov/products/world-ocean-database |

Main functions are located in oceandatatools/. Demonstrations are located in demos/. Test datas are located in data/. Shell scripts for batch downloading data are located in shell_scripts/. While shell scripts can be run directly in a macOS Terminal, running them in Windows requires Cygwin (and perhaps slight modification of commands). Python syntax examples are located in python/, which may be grow to become a module in the future.

Dependencies

The only true dependency is nctoolbox.

It is recommended to also download Gibbs-SeaWater (GSW) Oceanographic Toolbox. A benefit of oceandatatools is that neatly packs data into uniform structs; at which point a user can easily apply custom calculations or functions from other toolboxes such as GSW. See an example.

Accessing Help

To access help, run the command doc ocean_data_tools.

How to Contribute

• Want to make changes or add a new function? (1) Fork the repository (make your own separate copy), (2) make changes, and (3) open a 'pull request'. Once approved, it can be merged into the master branch. If you wish to chat beforehand about your contribution, open an issue or email lnferris@alum.mit.edu.
• Don't use git often and don't want to remember all the terminal commands? Download GitHub Desktop.
• Find a bug in the code? Open an 'issue' to notify contributors and create an official record.

Before contributing, please see Contents and consider how your function fits into oceandatatools and its ethos of structure arrays. At a minimum, functions must be well-documented and address a specific freely-available oceanographic data source which can be accessed by anyone online.

Adding a new function isn't the only way to contribute. Python, Julia, etc. translations of existing Matlab functions are also welcomed!

If you are interested in becoming a formal collaborator (e.g. have direct access and co-manage this repository), please reach out.

Contents

Building uniform structs from data sources

General functions for subsetting and plotting uniform structs

Plotting gridded data without building structs

Adding bathymetry to existing plots

Additional functions for inspecting Argo data

Miscellaneous utilities

Building uniform structs from data sources

argo_build searches the locally-stored Argo profiles matching the specified region & time period and builds a uniform struct

glider_build loads an archived glider survey (downloaded from gliders.ioos.us/erddap) and builds a uniform struct

mochabuildprofiles builds a uniform struct of profiles from the MOCHA Mid-Atlantic Bight climatology

modelbuildprofiles builds a uniform struct of profiles from HYCOM or Operational Mercator CMEMS GLOBALANALYSISFORECASTPHY001_024

waves_build builds a uniform struct of timeseries from CMEMS Global Ocean Waves Multi Year product GLOBALREANALYSISWAV001032

whpcruisebuild builds a uniform struct of profiles from GO-SHIP cruise data in WHP-Exchange Format

woabuildprofiles builds a uniform struct of profiles from World Ocean Atlas 2018 Statistical Mean for All Decades, Objectively Analyzed Mean Fields

wod_build builds a uniform struct of profiles from World Ocean Database data

Don't see a function yet for your preferred data source? Email lnferris@alum.mit.edu to request or contribute.

General functions for subsetting and plotting uniform structs

generaldepthsubset subsets a uniform struct by depth

general_map plots coordinate locations in a uniform struct, with optional bathymetry contours

general_profiles plots vertical profiles in a uniform struct

generalregionsubset subsets a uniform struct by polygon region

generalremoveduplicates removes spatially (or spatially and temporally) non-unique profiles from a uniform struct

general_section plots a data section from a uniform struct

Plotting gridded data without building structs

mochadomainplot plots a 3-D domain from the MOCHA Mid-Atlantic Bight climatology

mochasimpleplot plots a 2-D layer from the MOCHA Mid-Atlantic Bight climatology

modeldomainplot plots a 3-D domain from HYCOM or Operational Mercator CMEMS GLOBALANALYSISFORECASTPHY001_024

modelsimpleplot plots a 2-D layer from HYCOM or Operational Mercator CMEMS GLOBALANALYSISFORECASTPHY001_024

woadomainplot plots a 3-D domain from World Ocean Atlas 2018 Statistical Mean for All Decades, Objectively Analyzed Mean Fields

woasimpleplot plots a 2-D layer from World Ocean Atlas 2018 Statistical Mean for All Decades, Objectively Analyzed Mean Fields

Adding bathymetry to existing plots

bathymetry_extract extracts a region of Smith & Sandwell Global Topography and outputs as arrays

bathymetry_plot adds bathymetry to 2-D (latitude vs. longitude) or 3-D (latitude vs. longitude vs. depth) data plots

bounding_region finds the rectangular region around a uniform struct and/or list of coordinates to pass as an argument for other bathymetry functions

bathymetry_section adds Smith & Sandwell Global Topography to a section from plot using bathymetry data nearest to specified coordinates

Additional functions for inspecting Argo data

argoplatformmap plots locations of Argo profiles in a uniform struct, coloring markers by platform (individual Argo float)

argoplatformsubset subsets a uniform struct of Argo data to one platform (individual Argo float)

argoprofilesmap plots coordinate locations of Argo profiles in uniform struct argo, using colors corresponding to argo_profiles called on the same struct

argo_profiles plots vertical Argo profiles in uniform struct argo, using colors corresponding to argoprofilesmap called on the same struct

Miscellaneous utilities

region_select creates coordinate list (which represents vertices of a polygon region) by clicking stations on a plot

transect_select creates a coordinate list (which represents a virtual transect) by clicking stations on a plot

Finding Data

There two types of datasets: those that need to be downloaded manually¹ and those that can be accessed remotely² through OpenDAP (e.g. the data can be accessed directly on the the internet using a url).

argo¹

Download Argo data directly from GDAC FTP servers using either the Coriolis selection tool, or the US GDAC. See the Argo User's Manual for more information.

Alternatively run shellscripts/downloadargo to download data via File Transfer Protocol.

bathymetry¹

To get bathymetry data (for bathymetry_dir), download Smith & Sandwell under Global Topography V19.1 in netcdf form (topo_20.1.nc).

glider¹

Vist gliders.ioos.us/erddap. Click "View a List of All 779 Datasets" or use the "Advanced Search". After choosing a dataset, navigate to the Data Access Form. To get started, select these variables:

Scroll to "File type:". In the drop-down menu, select ".nc". Click "Submit".

mocha²

The url for MOCHA Mid-Atlantic Bight climatology is embedded. See Rutgers Marine catalog.

model^1,2

HYCOM data may be accessed remotely using OpenDAP. Get the data url by visiting the HYCOM website. For example, click Access Data Here -> GLBv0.08/expt57.7 (Jun-01-2017 to Sep-30-2017)/ -> Hindcast Data: Jun-01-2017 to Sep-30-2017. Click on the OpenDAP link. Copy the url as and use this as the source in ``modelbuild_profiles``.

Alteratively, download subsetted HYCOM data using NCSS. Get the data url by visiting the HYCOM website. For example, click Access Data Here -> GLBv0.08/expt57.7 (Jun-01-2017 to Sep-30-2017)/ -> Hindcast Data: Jun-01-2017 to Sep-30-2017. Click on the NetcdfSubset link. Set constraints and copy the NCSS Request URL at the bottom of the page. Run **shellscripts/downloadhycomlite. To download multiple months or years, run **shellscripts/downloadhycombulkdaily (partition files by day) or shellscripts/downloadhycombulkmonthly (partition files by month). Please use responsibly.

For Mercator, download Copernicus Marine data directly from FTP servers. First make a Copernicus account. Use the selection tool to download GLOBALANALYSISFORECASTPHY001024. Alternatively run **shellscripts/download_mercator**. Before running the script, follow the instructions for modifying your ~/.netrc file in the comments of the script.

waves¹

First make a Copernicus account. Use the selection tool to download CMEMS Global Ocean Waves Multi Year product GLOBALREANALYSISWAV001032.

whp_cruise¹

For GO-SHIP data, get CTD data (for ctdo_dir) by choosing a GO-SHIP cruise and downloading the CTD data in whpnetcdf format. More information about whpnetcdf parameters is available here. Get LADCP data (for uv_dir, wke_dir) here. There is information about LACDP processing here.

woa²

Functions build the World Ocean Atlas url at maximum resolution based on arguments, but coarser resolutions and seasonal climatologies are available at the NODC website. Note NCEI is scheduled to update data urls in the near future. Functions will be updated as such.

wod¹

Search the World Ocean Database and select products.

Citing ODT

Ferris, L., (2020). oceandatatools: A MATLAB toolbox for interacting with bulk freely-available oceanographic data. Journal of Open Source Software, 5(54), 2497. https://doi.org/10.21105/joss.02497