Data

Tools

Indexes

Applications

Experiments

Open Source Science

Blimpy

Blimpy: Breakthrough Listen I/O Methods for Python - Published in JOSS (2019)

https://github.com/ucberkeleyseti/blimpy

Science Score: 98.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 1 DOI reference(s) in JOSS metadata
  • ✓
    Academic publication links
    Links to: joss.theoj.org
  • ✓
    Committers with academic emails
    4 of 25 committers (16.0%) from academic institutions
  • ✓
    Institutional organization owner
    Organization ucberkeleyseti has institutional domain (seti.berkeley.edu)
  • ✓
    JOSS paper metadata
    Published in Journal of Open Source Software

Scientific Fields

Mathematics Computer Science - 34% confidence
Last synced: 6 months ago · JSON representation

Repository

Breakthrough Listen I/O Methods for Python

Basic Info
  • Host: GitHub
  • Owner: UCBerkeleySETI
  • License: bsd-3-clause
  • Language: Python
  • Default Branch: master
  • Homepage: https://blimpy.readthedocs.io
  • Size: 6.13 MB
Statistics
  • Stars: 50
  • Watchers: 14
  • Forks: 43
  • Open Issues: 12
  • Releases: 35
Created over 9 years ago · Last pushed over 3 years ago
Metadata Files
Readme Contributing License Code of conduct

README.md

Build Status Documentation Status codecov JOSS status

Breakthrough Listen I/O Methods for Python.

Filterbank + Raw file readers

This repository contains Python 2/3 readers for interacting with Sigproc filterbank (.fil), HDF5 (.h5) and guppi raw (.raw) files, as used in the Breakthrough Listen search for intelligent life.

Installation

System Dependencies

Sometimes the pip installation can fail if a system dependency is not installed. To fix this, make sure you have curl and install the required system dependencies with the command bellow:

Debian/Ubuntu

curl https://raw.githubusercontent.com/UCBerkeleySETI/blimpy/master/dependencies.txt | xargs -n 1 sudo apt install --no-install-recommends -y

Manual Installation

The latest release can be installed via pip directly from this repository:

python3 -m pip install -U git+https://github.com/UCBerkeleySETI/blimpy

Or, the latest version of the development code can be installed from the github repo and then run python setup.py install or pip install . (with sudo if required), or by using the following terminal command:

python3 -m pip install -U https://github.com/UCBerkeleySETI/blimpy/tarball/master

To install everything required to run the unit tests, run:

python3 -m pip install -e .[full]

You will need numpy, h5py, astropy, scipy, and matplotlib as dependencies. A pip install should pull in numpy, h5py, and astropy, but you may still need to install scipy and matplotlib separately. To interact with compressed files, you'll need the hdf5plugin package too.

Note that h5py generally needs to be installed in this way:

$ python3 -m pip install --no-binary=h5py h5py

Command line utilities

After installation, the following command will display the metadata (header) values and some information about the data matrix:

watutil -i

Other command line utilities available post-installation: * bl_scrunch, Scrunch an HDF5 file (.h5) or a Sigproc Filterbank file (.fil) to an output HDF5 file. This has the effect of down-sampling adjacent fine frequency cells. * bldice, Extract a smaller frequency region from a Filterbank file (.fil or .h5 file). * calcload, Calculate the Waterfall maxload value needed to load the entire data array for a given Filterbank file. * dsamp, Down-sample (time-dimension only) from one Filterbank file to another. * fil2h5, Convert a .fil file into .h5 format. * h52fil, Convert an .h5 file into .fil format. * peek, Display a selected portion of values from the data matrix of a Filterbank file. * rawhdr, Display the header fields of a raw guppi file. * rawutil, Plot data in a guppi raw file. * srcname, Patch the header sourcename field in a .h5 file. * stax, For a collection of .h5 or .fil files sharing the same frequency range, create a vertical stack of waterfall plots as a single PNG file. * stix, For a single very large Filterbank file, create a horizontal or vertical stack of waterfall plots as a single PNG file. * watutil, Information/read/write/plot utility for Filterbank files.

Use the -h flag to any of the above command line utilities to display their available arguments.

Reading blimpy filterbank files in .fil or .h5 format

The blimpy.Waterfall provides a Python API for interacting with filterbank data. It supports all BL filterbank data products; see this example Jupyter notebook for an overview.

From the python, ipython or jupiter notebook environments.

```python from blimpy import Waterfall fb = Waterfall('/path/to/filterbank.fil')

fb = Waterfall('/path/to/filterbank.h5') #works the same way

fb.info() data = fb.data ```

Reading guppi raw files

The Guppi Raw format can be read using the GuppiRaw class from guppi.py:

```python from blimpy import GuppiRaw gr = GuppiRaw('/path/to/guppirawfile.raw')

header, data = gr.readnextdata_block() ```

or

```python from blimpy import GuppiRaw gr = GuppiRaw('/path/to/guppirawfile.raw')

for header, datax, datay in gr.get_data(): # process data ```

Note: most users should start analysis with filterbank files, which are smaller in size and have been generated from the guppi raw files.

Using blimpy inside Docker

The blimpy images are pushed to a public repository after each successful build on Travis. If you have Docker installed, you can run the following commands to pull our images, which have the environment and dependencies set up for you.

docker pull fx196/blimpy:py3_kern_stable

Here is a more complete guide on using blimpy in Docker.

Further reading

A detailed overview of the data formats used in Breakthrough Listen can be found in our data format paper. An archive of data files from the Breakthrough Listen program is provided at seti.berkeley.edu/opendata.

If you have any requests or questions, please lets us know!

Owner

  • Name: UC Berkeley SETI Program
  • Login: UCBerkeleySETI
  • Kind: organization
  • Location: University of California, Berkeley - Berkeley, California, USA

JOSS Publication

Blimpy: Breakthrough Listen I/O Methods for Python
Published
October 21, 2019
DOI
Volume 4, Issue 42, Page 1554
Authors
Danny C. Price ORCID
Equal contribution from both authors, Department of Astronomy, University of California Berkeley, Berkeley CA 94720, United States of America, Centre for Astrophysics & Supercomputing, Swinburne University of Technology, Hawthorn, VIC 3122, Australia
J. Emilio Enriquez ORCID
Equal contribution from both authors, Department of Astronomy, University of California Berkeley, Berkeley CA 94720, United States of America, Department of Astrophysics/IMAPP, Radboud University, Nijmegen, Netherlands
Yuhong Chen
Department of Astronomy, University of California Berkeley, Berkeley CA 94720, United States of America
Mark Siebert
Department of Astronomy, University of California Berkeley, Berkeley CA 94720, United States of America
Editor
Juanjo Bazán ORCID
Tags
astronomy radio astronomy technosignatures SETI
View PDF Review Thread Software Archive

GitHub Events

Total
  • Watch event: 3
  • Fork event: 3
Last Year
  • Watch event: 3
  • Fork event: 3

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 1,451
  • Total Committers: 25
  • Avg Commits per committer: 58.04
  • Development Distribution Score (DDS): 0.716
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Danny Price d****n@t****m 412
Richard Elkins r****s@g****m 337
Emilio Enriquez j****z@g****m 278
yuhongc m****2@g****m 259
Mark Siebert m****5@c****u 53
Gijs Molenaar g****s@p****l 40
Luigi Cruz l****z@g****m 21
Bryan Brzycki b****i@b****u 10
Yunfan Zhang y****z@b****u 9
dan-rds d****r@u****u 6
Kevin Lacker l****r@g****m 5
greghell g****g@g****m 3
Danny Price d****r@b****t 2
Danny Price d****e@b****t 2
enriquez e****z@a****l 2
Matt Lebofsky - SETI m****y 2
Nick Richard n****d@r****m 2
Christian Gilbertson 3****l 1
Griffin Foster g****r@g****m 1
Jan Janssen j****n 1
Yuhong Chen y****n@b****t 1
Danny Price p****9@b****t 1
Lukas Finkbeiner 4****r 1
Steve Croft d****t@g****m 1
Wael Farah w****h@g****m 1
Committer Domains (Top 20 + Academic)
berkeley.edu: 2 blc00.bl.pvt: 1 blpc0.bl.pvt: 1 radwave.com: 1 astro.ru.nl: 1 bls0.bl.pvt: 1 blh0.bl.pvt: 1 ucsc.edu: 1 pythonic.nl: 1 cornell.edu: 1 thetelegraphic.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 53
  • Total pull requests: 47
  • Average time to close issues: 5 months
  • Average time to close pull requests: 6 days
  • Total issue authors: 14
  • Total pull request authors: 6
  • Average comments per issue: 3.28
  • Average comments per pull request: 1.28
  • Merged pull requests: 42
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • texadactyl (33)
  • telegraphic (8)
  • shamichatterjee (1)
  • Sofysicist (1)
  • thevorpalblade (1)
  • razman786 (1)
  • ZaynAmell (1)
  • daniestevez (1)
  • aleder (1)
  • siemion (1)
  • wfarah (1)
  • irfnt (1)
  • jeenriquez (1)
  • evanocathain (1)
Pull Request Authors
  • texadactyl (41)
  • lacker (2)
  • jan-janssen (1)
  • Akshay-E (1)
  • razman786 (1)
  • wfarah (1)
Top Labels
Issue Labels
enhancement (16) bug (15) testing (6) better diagnostics (4) documentation (3) help wanted (3) question (2) dependency surprise (1) Solved another way (1) Deferred to Hyperseti (1)
Pull Request Labels
duplicate (1) documentation (1)

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 817 last-month
  • Total dependent packages: 2
    (may contain duplicates)
  • Total dependent repositories: 8
    (may contain duplicates)
  • Total versions: 55
  • Total maintainers: 5
pypi.org: blimpy

Python utilities for Breakthrough Listen SETI observations

  • Versions: 47
  • Dependent Packages: 1
  • Dependent Repositories: 8
  • Downloads: 817 Last month
Rankings
Dependent packages count: 4.8%
Dependent repos count: 5.2%
Forks count: 6.4%
Average: 7.1%
Downloads: 9.3%
Stargazers count: 9.8%
Maintainers (5)
griffinfoster jeenriquez mattlebofsky telegraphic lacker
Last synced: 6 months ago
conda-forge.org: blimpy
  • Versions: 8
  • Dependent Packages: 1
  • Dependent Repositories: 0
Rankings
Forks count: 26.1%
Dependent packages count: 28.8%
Average: 32.0%
Dependent repos count: 34.0%
Stargazers count: 39.1%
Last synced: 6 months ago

Dependencies

requirements.txt pypi
  • Pillow *
  • astropy *
  • h5py *
  • hdf5plugin *
  • matplotlib *
  • numpy >=1.19
  • pandas *
  • psutil *
  • pyparsing ==2.4.7
  • scipy *
  • setuptools *
  • six *
requirements_test.txt pypi
  • codecov *
  • coverage *
  • pyslalib *
  • pytest *
  • pytest-cov *
  • pytest-order *
  • pytest-runner *
  • setigen *
.github/workflows/docker_build.yml actions
  • actions/checkout v2 composite
.github/workflows/push_docker.yml actions
  • actions/checkout v2 composite
  • docker/build-push-action v1 composite
.github/workflows/python_tests.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • codecov/codecov-action v1 composite
.github/workflows/pythonpublish.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v1 composite
Dockerfile docker
  • ubuntu 20.04 build