ghcndata.jl
Helps access the Global Historical Climatological Network's daily data
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
✓Committers with academic emails
2 of 5 committers (40.0%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.6%) to scientific vocabulary
Keywords from Contributors
Repository
Helps access the Global Historical Climatological Network's daily data
Basic Info
Statistics
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 0
- Releases: 6
Metadata Files
README.md
GHCNData
Utility functionality to help getting hold of daily data from the Global Historical Climatology Network archive.
If you use this data, you should acknowledge it appropriately. Instruction for doing so can be found at the top of NOAA's readme.
Why Bother?
While the GHCN data is fairly straightforward, it's not as simple as just downloading a single file and opening it as a DataFrame.
There are a few different kinds of files that you need to be aware of, each of which has a well-documented but non-standard format.
As such, it makes sense to implement the functionality to load the files in a format more amenable to standard workflows.
Usage
Data Loading
This package provides helper functions to download and load the data offered by NOAA. There are four core functions that you should be aware of
julia
load_station_metadata
load_inventories
load_data_file
load_countries_metadata
Each of these functions download the corresponding data using DataDeps.jl if it's not already available, and parses it into a DataFrame.
NOAA's documentation is the best place to look to understand these files, but the docstrings in this package provide a brief overview.
Typical Workflows
Commonly, you'll want to load all of the data associated with a particular collection of stations in a particular region of the world. There are basically two steps to do this:
- Use
load_inventories()to find out which stations exist at which latitudes / longitude, and their corresponding ID. - Use
load_data_file(station_id)to load each station that you've found in your region of interest.
For an example of this kind of thing, see the code for select_data in dataset_loading.jl.
You might also be interested in, for example, the properties of the station in question (e.g. its elevation). For that data, use load_station_metadata().
Helper Functions
This package presently provides two bits of functionality to process the data a bit once it's been loaded.
select_data pretty much implements the workflow discussed above.
convert_to_time_series "stacks" the output of load_data_file, converting from 1 row == 1 month (different day's data live in different columns in the raw data), to a format in which 1 row == 1 day.
Both functions are quite opinionated, so while they're hopefully helpful examples of things that you might want to do with the GHCN data, you'll probably need to tweak them a bit for your use-case.
Missing Functionality and Contributing
If you build on this functionality, please consider contributing back so that we can make all of our lives easier! Similarly, please open an issue (or, even better, a PR) if you feel that something that would be useful is missing.
Development has been driven on an as-needed basis, so while this is package will grab most (all?) of the daily data for you, it is a little sparse on utility functionality.
In particular, please note that convert_to_time_series and select_data may not make assumptions about the data that are appropriate for your use case. If in doubt, I would recommend using the functionality in dataset_loading.jl, as it just provides helpful functionality to extract the data.
Moreover, it doesn't currently implement anything to grab or process the monthly data, but it should be a straightforward extension of the existing functionality to do so.
Bug Reporting
If you either find a bug, or think something looks suspicious, please open an issue / PR. When considering whether or not to open an issue / PR, note that it's generally better to open an issue erroneously (no harm is done if it turns out there wasn't a problem after all) than it is for a problem to slip by (data-related bugs cause papers to be retracted and generally hold back progress). If in doubt, open an issue.
Why are there so few tests?
Three of the four core functions listed above are lightly tested -- load_data_file has yet to be tested because, as presently implemented, the CI runner would need to download the entire collection of daily data for each run, which seems impractical. If you have any suggestions for how to alleviate this, please open an issue / PR!
Related Work
Scott Hosking provides similar functionality in a Python package.
Owner
- Name: Will Tebbutt
- Login: willtebbutt
- Kind: user
- Repositories: 8
- Profile: https://github.com/willtebbutt
GitHub Events
Total
Last Year
Committers
Last synced: 10 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| wt | w****t@i****k | 15 |
| WT | w****1@m****k | 4 |
| Lyndon White | l****e@i****k | 3 |
| Lyndon White | o****x@u****u | 1 |
| willtebbutt | w****3@c****k | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 10 months ago
All Time
- Total issues: 2
- Total pull requests: 7
- Average time to close issues: 31 minutes
- Average time to close pull requests: 4 months
- Total issue authors: 2
- Total pull request authors: 3
- Average comments per issue: 4.5
- Average comments per pull request: 1.0
- Merged pull requests: 6
- Bot issues: 0
- Bot pull requests: 1
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- willtebbutt (1)
- JuliaTagBot (1)
Pull Request Authors
- willtebbutt (5)
- github-actions[bot] (1)
- oxinabox (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 3
- Total downloads: unknown
-
Total dependent packages: 0
(may contain duplicates) -
Total dependent repositories: 0
(may contain duplicates) - Total versions: 18
proxy.golang.org: github.com/willtebbutt/GHCNData.jl
- Documentation: https://pkg.go.dev/github.com/willtebbutt/GHCNData.jl#section-documentation
- License: mit
-
Latest release: v0.1.5
published almost 3 years ago
Rankings
proxy.golang.org: github.com/willtebbutt/ghcndata.jl
- Documentation: https://pkg.go.dev/github.com/willtebbutt/ghcndata.jl#section-documentation
- License: mit
-
Latest release: v0.1.5
published almost 3 years ago
Rankings
juliahub.com: GHCNData
Helps access the Global Historical Climatological Network's daily data
- Documentation: https://docs.juliahub.com/General/GHCNData/stable/
- License: MIT
-
Latest release: 0.1.5
published almost 3 years ago
Rankings
Dependencies
- JuliaRegistries/TagBot v1 composite
- actions/checkout v1.0.0 composite
- actions/github-script 0.3.0 composite
- julia-actions/setup-julia latest composite
- actions/cache v1 composite
- actions/checkout v2 composite
- codecov/codecov-action v1 composite
- julia-actions/julia-buildpkg v1 composite
- julia-actions/julia-processcoverage v1 composite
- julia-actions/julia-runtest v1 composite
- julia-actions/setup-julia v1 composite