Data

Tools

Indexes

Applications

Experiments

Open Source Science

psi-collect

psi-collect: A Python module for post-storm image collection and cataloging - Published in JOSS (2020)

https://github.com/uncg-daisy/psi-collect

Science Score: 95.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 3 DOI reference(s) in README and JOSS metadata
  • ✓
    Academic publication links
    Links to: joss.theoj.org, zenodo.org
  • ✓
    Committers with academic emails
    2 of 10 committers (20.0%) from academic institutions
  • ○
    Institutional organization owner
  • ✓
    JOSS paper metadata
    Published in Journal of Open Source Software

Keywords

coastal hurricane imagery noaa storms
Last synced: 6 months ago · JSON representation

Repository

Collect Post-Storm Imagery from NOAA

Basic Info
  • Host: GitHub
  • Owner: UNCG-DAISY
  • License: mit
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 287 MB
Statistics
  • Stars: 5
  • Watchers: 5
  • Forks: 3
  • Open Issues: 27
  • Releases: 6
Topics
coastal hurricane imagery noaa storms
Created about 6 years ago · Last pushed about 3 years ago
Metadata Files
Readme Changelog Contributing License Code of conduct

README.md

Post-Storm Imagery Collection

Codecov PyPI Travis-CI

Maintenance Last Commit

JOSS DOI

ReadTheDocs Documentation

This package downloads, catalogs, and stores NOAA emergency response imagery

Quick Start

There are two ways to get started with psi-collect — using the version on PyPI or installing directly from this GitHub repository. The version currently on PyPI does not work with images from the two most recent Hurricanes (Hx Delta and Hx Zeta). The version here on GitHub is modified and works with these storms (but we still need to make a few more modifications before this new version is packaged for PyPI).

from PyPI:

  1. Install Python 3.6, 3.7, or 3.8 (Download Here)
  2. Run pip3 install psi-collect in your favorite terminal
  3. Run pstorm collect -h for help on collecting images or pstorm catalog -h for help on cataloging local archives

from GitHub Repository

  1. Install Python 3.6, 3.7, or 3.8 (Download Here)
  2. Make sure you have dependecies installed (check them out in requirements.txt)
  3. Run pip install git+https://github.com/UNCG-DAISY/psi-collect in your favorite terminal
  4. Run pstorm collect -h for help on collecting images or pstorm catalog -h for help on cataloging local archives

Usage Guide

Check out the documentation for Collecting and Cataloging images to see usage.

An Example Use of psi-collect: Labeling Imagery

We labeled storm impacts for ~300 images from Hurricane Florence using the Coastal Image Labeler. The labels are available on figshare.

The specific NOAA imagery to link to these labels can be retrieved with psi-collect via:

pstorm collect -s Florence -a 20180917a_jpgs -d

Contributing / Developing

Code of Conduct

We hope to foster an inclusive and respectful environment surrounding the contribution and discussion of our project. Make sure you understand our Code of Conduct.

Code Conventions

Before committing to the repository please read the project Code Conventions.

Project Pre-Requisites

  1. Python 3.6, 3.7, or 3.8 (Download Here)
  2. Pipenv (Run pip install pipenv)
  3. You will need to add the module path <parent dir>/psi-collect/psicollect/ to your PYTHONPATH (See Configuring Module)

If you get an error message that looks something like Module not found: "psicollect", then the PYTHONPATH is not configured correctly!

You will need this in order to run the project.

Installing Dependencies for Development

  1. Change current directory (cd) to psi-collect/ (the project root)
  2. Run pipenv install --dev to install dependencies
  3. Run pre-commit install to install style checking when committing

Testing the Collect Script

  1. Change current directory to the collector module (cd collector/)
  2. Either use pipenv run collect.py <args> or pipenv shell then collect.py <args>
  3. In addition tests can be run using pytest from the project root directory

The arguments for collect.py are listed here

NOAA Data 💾

  • NOAA landing page for the post-storm Emergency Response Imagery, here

Team Members and contributors

Owner

  • Name: UNCG DAISY
  • Login: UNCG-DAISY
  • Kind: organization
  • Location: The University of North Carolina at Greensboro, USA

the UNCG Environmental Data Science Collaboratory

JOSS Publication

psi-collect: A Python module for post-storm image collection and cataloging
Published
March 01, 2020
DOI
Volume 5, Issue 47, Page 2075
Authors
Matthew C. Moretz
Department of Computer Science, The University of North Carolina at Greensboro
Daniel Foster
Department of Computer Science, The University of North Carolina at Greensboro
John Weber
Department of Computer Science, The University of North Carolina at Greensboro
Rinty Chowdhury
Department of Computer Science, The University of North Carolina at Greensboro
Shah Nafis Rafique
Department of Computer Science, The University of North Carolina at Greensboro
Evan B. Goldstein ORCID
Department of Geography, Environment, and Sustainability, The University of North Carolina at Greensboro
Somya D. Mohanty ORCID
Department of Computer Science, The University of North Carolina at Greensboro
Editor
Katy Barnhart ORCID
Tags
coastal storm morphology hurricane severe weather NOAA
View PDF Review Thread Software Archive

GitHub Events

Total
Last Year

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 178
  • Total Committers: 10
  • Avg Commits per committer: 17.8
  • Development Distribution Score (DDS): 0.629
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
dependabot-preview[bot] 2****] 66
Evan B. Goldstein e****n 58
Matmorcat m****z@u****u 44
Chris Leaman c****n@g****m 4
Somya Mohanty m****a@g****m 1
Shah Nafis Rafique s****u@u****u 1
Matthew Conlin 3****t 1
Katherine Ratliff r****e@e****v 1
Jamison Valentine j****e@g****m 1
JWeb56 J****6 1
Committer Domains (Top 20 + Academic)
uncg.edu: 2 epa.gov: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 5
  • Total pull requests: 95
  • Average time to close issues: 2 months
  • Average time to close pull requests: 26 days
  • Total issue authors: 1
  • Total pull request authors: 4
  • Average comments per issue: 0.8
  • Average comments per pull request: 0.93
  • Merged pull requests: 10
  • Bot issues: 0
  • Bot pull requests: 92
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • ebgoldstein (5)
Pull Request Authors
  • dependabot-preview[bot] (81)
  • dependabot[bot] (11)
  • ebgoldstein (2)
  • jamisonvalentine (1)
Top Labels
Issue Labels
bug (2) documentation (2) enhancement (2)
Pull Request Labels
dependencies (92) security (4) bug (1)

Packages

  • Total packages: 3
  • Total downloads:
    • pypi 57 last-month
  • Total dependent packages: 0
    (may contain duplicates)
  • Total dependent repositories: 1
    (may contain duplicates)
  • Total versions: 39
  • Total maintainers: 2
proxy.golang.org: github.com/UNCG-DAISY/psi-collect
  • Versions: 10
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.4%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 6 months ago
proxy.golang.org: github.com/uncg-daisy/psi-collect
  • Versions: 10
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.4%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 6 months ago
pypi.org: psi-collect

Collection, aggregation, and cataloging of NOAA post-storm emergency response imagery.

  • Versions: 19
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 57 Last month
Rankings
Dependent packages count: 7.3%
Forks count: 16.9%
Dependent repos count: 22.1%
Average: 22.8%
Stargazers count: 23.2%
Downloads: 44.4%
Maintainers (2)
ebgoldstein Matmorcat
Last synced: 6 months ago

Dependencies

Pipfile pypi
  • bump2version * develop
  • codecov * develop
  • pre-commit * develop
  • pytest * develop
  • pytest-cov * develop
  • pytest-runner * develop
  • imageio *
  • pandas *
  • pillow *
  • requests *
  • requests-html *
  • tqdm *
Pipfile.lock pypi
  • appdirs ==1.4.4 develop
  • attrs ==20.3.0 develop
  • bump2version ==1.0.1 develop
  • certifi ==2020.12.5 develop
  • cfgv ==3.2.0 develop
  • chardet ==4.0.0 develop
  • codecov ==2.1.11 develop
  • coverage ==5.3.1 develop
  • distlib ==0.3.1 develop
  • filelock ==3.0.12 develop
  • identify ==1.5.11 develop
  • idna ==2.10 develop
  • iniconfig ==1.1.1 develop
  • nodeenv ==1.5.0 develop
  • packaging ==20.8 develop
  • pluggy ==0.13.1 develop
  • pre-commit ==2.9.3 develop
  • py ==1.10.0 develop
  • pyparsing ==2.4.7 develop
  • pytest ==6.2.1 develop
  • pytest-cov ==2.10.1 develop
  • pytest-runner ==5.2 develop
  • pyyaml ==5.3.1 develop
  • requests ==2.25.1 develop
  • six ==1.15.0 develop
  • toml ==0.10.2 develop
  • urllib3 ==1.26.2 develop
  • virtualenv ==20.2.2 develop
  • appdirs ==1.4.4
  • beautifulsoup4 ==4.9.3
  • bs4 ==0.0.1
  • certifi ==2020.12.5
  • chardet ==4.0.0
  • cssselect ==1.1.0
  • fake-useragent ==0.1.11
  • idna ==2.10
  • imageio ==2.9.0
  • lxml ==4.6.2
  • numpy ==1.19.4
  • pandas ==1.2.0
  • parse ==1.18.0
  • pillow ==8.1.0
  • pyee ==7.0.4
  • pyppeteer ==0.2.3
  • pyquery ==1.4.3
  • python-dateutil ==2.8.1
  • pytz ==2020.5
  • requests ==2.25.1
  • requests-html ==0.10.0
  • six ==1.15.0
  • soupsieve ==2.1
  • tqdm ==4.55.1
  • urllib3 ==1.26.2
  • w3lib ==1.22.0
  • websockets ==8.1
docs/requirements-docs.txt pypi
  • markdown ==3.3.3
  • mkdocs ==1.1.2
  • mkdocs-git-revision-date-plugin ==0.3.1
  • mkdocs-macros-plugin ==0.5.0
  • mkdocs-material ==6.2.3
  • mkdocs-minify-plugin ==0.3.0
  • pymdown-extensions ==8.1
requirements-dev.txt pypi
  • bump2version * development
  • codecov * development
  • pre-commit * development
  • pytest * development
  • pytest-cov * development
  • pytest-runner * development
requirements.txt pypi
  • imageio *
  • pandas *
  • pillow *
  • requests *
  • requests-html *
  • tqdm *
  • urllib3 <1.25
setup.py pypi
  • Pillow *
  • imageio *
  • pandas *
  • requests *
  • tqdm *