georinex - numerous bugfixes. Requires Python >= 3.8

FIxed numerous bugs, including xarray API update

Requires Python 3.8+ as Python 3.7 library support and wheel availability is dwindling.

- Python
Published by scivision over 2 years ago

georinex - add sp3a read, additional nav3 header

also fix sp3 bug in reading microseconds

- Python
Published by scivision over 4 years ago

georinex - use Python hatanaka package, install unlzw3

• implement hatanaka using python hatanaka package, which provides Python version agnostic wheels
• install unlzw3 by default

- Python
Published by scivision over 4 years ago

georinex - read robustness

multiple fixes to handle RINEX oddities

Requires Python >= 3.7 due to enhanced type annotation

- Python
Published by scivision almost 5 years ago

georinex - use src/ layout for robustness

also use entry_points instead of scripts

- Python
Published by scivision almost 6 years ago

georinex - add sp3-d format

Thanks to @fmeynadier for adding sp3-d format to Georinex.

- Python
Published by scivision over 6 years ago

georinex - increased robustness

• add ability to overwrite .nc files with "overwrite=True"
• make Rinex3 files handle INRSS
• add more header parameters
• make SP3 reading much more robust
• tolerate blank lines starting rinex file (default up to 10)
• use "unlzw3" for pure Python .Z files
• general test / CI / coverage improvements

- Python
Published by scivision over 6 years ago

georinex - NAV: allow same-time entries, fix pandas >= 0.25, OBS2 wrong obs num OK

• NAV3: allow multiple entries at same time, instead of discarding them as previous versions did
• fix allows Pandas >= 0.25 to be used, there was a bug where pandas 0.25 starting outputting dtype('O') that broke xarray.to_netcdf()
• OBS2: allow more flexibility where the RINEX file specifies the wrong number of observation types. This isn't foolproof, but at least tries to allow these RINEX files with incorrect headers.

- Python
Published by scivision over 6 years ago

georinex - OBS2: max per-system SV is now 36

thanks @nmayorov #47

- Python
Published by scivision over 6 years ago

georinex - add .sp3 SP3-C file reading, interval header metadata

• add .sp3 SP3-C file reading
• add 'interval' metadata to xarray.Dataset output, if available in source data file.

- Python
Published by scivision over 6 years ago

georinex - CRINEX Hatanaka auto-build

For better reliability and ease of use, auto-build Hatanaka CRINEX inline converter program by discovering C compiler and compiling automatically when needed.

Move CI to GitHub Actions

** Restricts Pandas < 0.25.0 for the moment as Pandas 0.25.0 disallows writing NetCDF .nc files due to a conflict with xarray and datetime64[ns] inline conversion. Would have been tricky to discover without periodic CI.

- Python
Published by scivision over 6 years ago

georinex - CI integration test, move complicated import functions to own files

created georinex/{geo,plots_geo}.py

- Python
Published by scivision almost 7 years ago

georinex - BUGFIX: NAV3 malformed files

BUGFIX: workaround bad lines in NAV3 files, filling those rows with NaN.

Ignore NAV3 fields "spare" and "FitInterval" which are missing and malformed in many NAV3 files.

Check number of times in file to see if it's as expected.

- Python
Published by scivision about 7 years ago

georinex - Overhaul for speed and robustness

Significant overhauling, deduplicating, robustness improvement for RINEX and CRINEX reading. Particular improvements for:

• CRINEX 1.x files
• uncompressed and compressed OBS 2.x files

In particular, the speed difference between "slow" and "fast" reading of OBS2 files is now less than 10% and memory usage is reduced proportional to tlim if specified.

Please let us know of any problem RINEX files.

- Python
Published by scivision about 7 years ago

georinex - Read by time interval for OBS, to_datetime() time conversion

For OBS2 and OBS3 files, added ability to read by time interval. That is, ready only every say 5 or 30 or whatever seconds. This can greatly speed up parsing of large files when the full time resolution is not needed. The option is interval= and any of float,int (seconds) or datetime.timedelta can be specified

A convenience function georinex.to_datetime() was added. This converts an xarray.DataArray of times to datetime.datetime ndarray--or scalar if it's only one time.

- Python
Published by scivision about 7 years ago

georinex - File extension agnostic

Continuing the work from v1.6.9, we make RINEX and Hatanaka CRINEX reading file-extension agnostic. Some organizations like to name their RINEX/CRINEX in non-standard ways.

This actually made the code simpler (!) and more functionalized.

However, compressed files of any type (.gz, .Z, .zip, etc.) are still decompressed by file extension. If this is an issue, we could consider looking at the file magic number, but I have only in the rarest situations seen where people are not using the standard general compressed file extensions.

- Python
Published by scivision over 7 years ago

georinex - BUGFIX: Galileo ionospheric correction, parametrize tests

Enhance test coverage ~ 94% by parametrizing unit tests

BUGFIX: match RINEX 3 spec for Galileo, only 3 ionospheric correction coefficients

- Python
Published by scivision over 7 years ago

georinex - enable StringIO input throughout program

Some users may wish to input io.StringIO instead of discrete files, which is now possible for all relevant functions.

- Python
Published by scivision over 7 years ago

georinex - better version detection, NAV3 spare detection

• detect RINEX type (obs, nav) based on header first line, not file extension. Despite the file naming convention being standard, it seems inevitable a mistake is made that is trivially detected from the first line of a RINEX file.
• add IRNSS NAV3
• better NAV3 spare field handling, not all files have all spare fields.

- Python
Published by scivision over 7 years ago

georinex - Better metadata, unit handling, rtklib format support

- Python
Published by scivision over 7 years ago

georinex - optional psutil RAM check, better metadata

- Python
Published by scivision over 7 years ago

georinex - output NAV2/3 iono parameters, use updated pytest 3.9+ fixtures

Improved output of NAV 2/3 parameters, especially ionospheric parameters.

Corrected NAV test file spacing issues

Use Pytest ≥ 3.9 tmp_path fixture.

- Python
Published by scivision over 7 years ago

georinex - BUGFIX mangled NAV3 data, plot galileo NAV

• BUGFIX https://github.com/scivision/georinex/issues/30 for irregular NAV3 data for Galileo
• functionalize irregular NAV3 data handling/checking
• plot galileo nav

- Python
Published by scivision over 7 years ago

georinex - Handle OBS2 mangled header/data times, dedupe

Fixes https://github.com/scivision/georinex/issues/29 mangled times in header and data. Dedupe time conversion for header

- Python
Published by scivision over 7 years ago

georinex - BUGFIX: NAV2 crash for irregular length (missing FitIntvl), improve GNSS location plots

• allow NAV2 files to intermittently be missing the final field (e.g. GPS FitIntvl)
• allow reading .nc files as well as RINEX for location plotting

- Python
Published by scivision over 7 years ago

georinex - Auto-allocate is smarter, reduced memory, auto-off

• Added pre-scan to autoallocation to reduce RAM usage by not always using absolute worst case.
• added auto-off (fast=False) if autoallocation senses file has an unusual/non-standard format
• dedupe code
• add obs.fast_processing=1 or 0 to indicate file was processed with fast=True or False respectively.

- Python
Published by scivision almost 8 years ago

georinex - Fast allocate OBS2

• fast=True default fast allocate based on file size and minimum SV=6 on average for OBS2
• quick check to see if RAM exceeded in OBS2 reads

- Python
Published by scivision almost 8 years ago

georinex - BUGFIX: missing header elements, minimal files, location plotter

• BUGFIX: handle (avoid crashing on) RINEX 2/3 OBS/NAV files with extremely minimal headers and missing data sanely
• optional location plotter, with optional output of geodetic GNSS receiver coordinates

- Python
Published by scivision almost 8 years ago

georinex - Batch conversion, rigourous exceptions, more test coverage

make rnx2hdf5 its own program to make use more obvious, with rigorous cross-platform test coverage

- Python
Published by scivision almost 8 years ago

georinex - Fix Hatanaka streaming for Windows, skip unlzw tests if not installed

Workaround for Python (including 3.7) subprocess bug on Windows that requires shell=True when using cwd=

allow missing unlzw by skipping those tests

- Python
Published by scivision almost 8 years ago

georinex - Doubled speed of OBS2 read, 30% speedup of NAV2

eliminating numpy.genfromtxt() in nav2.py and obs2.py brought further substantial speed increase. Opens door for future JIT, MyPy, etc. Default is now NOT verbose due to speed of reading. Use -v to show updates, plots.

- Python
Published by scivision almost 8 years ago

georinex - 10x overall speedup in RINEX 2 OBS reading

File that took one minute to read now takes only 5 seconds. This accomplished by using 3-D Numpy array preallocated by 10 ms fast prescan of OBS file.

- Python
Published by scivision almost 8 years ago

georinex - BUGFIX OBS2 with more than 10 measurements.

Note: this module would previously issue an error, not just silently give bad results. Now it works correctly.

Thanks @serioca fixes #25

- Python
Published by scivision almost 8 years ago

georinex - cleanup SV renaming

Some RINEX writers leave gaps in SV names. instead of G07 they write G 7, which making indexing messy. Now they are renamed in xarray.Dataset for easy, efficient indexing.

- Python
Published by scivision almost 8 years ago

georinex - Hatanaka read, API simplification, specify measurements to read

• Enable Hatanaka compressed RINEX .crx and .crx.gz reading using stream processing via Hatanaka's crx2rnx--no temporary files are used
• simplify API with georinex.load convenience method
• speed reading by optionally specifying meas=

- Python
Published by scivision almost 8 years ago

georinex - NAV file improvements

Add time reading from NAV files TimeRinex Add several tests for handling of large NAV files Add "use" option for RINEX 3 NAV files, to select specific satellite systems.

- Python
Published by scivision almost 8 years ago

georinex - More RINEX3 NAV formats

Improved multi-system RINEX 3 NAV files (Beidou, GNSS, GLONASS, Galileo, QZSS, SBAS)

Increased test coverage of NetCDF4 (HDF5) read/write

Failsoft more parts of code where user may not have every prereq

- Python
Published by scivision almost 8 years ago

georinex - Globbing, robust, .Z LZW read

• file globbing (input multiple files)
• read only data (nav, obs) specified by --tlim
• tolerate mangled parts of files (skips to next readable time step)
• increase speed by less eager reading of file
• using unlzw optionally, read .Z RINEX files, commonly used for RINEX 2.

- Python
Published by scivision almost 8 years ago

georinex - Times, functionalize, read .Z files, optional indicators

Break up functions into logically grouped files for better maintainabiility

Expose time selection API (read parts of a file)

TimeRinex.py print times in OBS file

Add ability to read .Z lzw files via unlzw

Make Indicators (SSI, LLI) optional -useindicators over 2x reading speedup, and nearly 50% space savings (disk, RAM)

- Python
Published by scivision almost 8 years ago

georinex - Rename, modernize, functionalize

Rename: PyRinex=>GeoRinex

Functionalize self-tests for easier diagnosis

Further functionalize/expose nice API, for example rinexheader to return dict() of header, with some mandatory values converted, and all others as str

pep8/type checking

- Python
Published by scivision almost 8 years ago

georinex - preserve first instance of duplicated time

From what I've seen, where an SV has a duplicated time entry, the data for that time is duplicated as well. So I discard all but the first of those duplicated times steps for that SV.

- Python
Published by scivision almost 8 years ago

georinex - Corrected NAV RINEX3 logic for irregular files

Made RINEX3 NAV reading for irregularly formatted files more robust.

Auto-read GZIP files as well as plaintext, transparently.

- Python
Published by scivision almost 8 years ago

georinex - Python >= 3.6, flake8 and mypy type

As part of a general move to increase geoscience/geospace coding quality, we have moved to flake8 + mypy type validation, hence Python >= 3.6 is required.

- Python
Published by scivision almost 8 years ago

georinex - Handle GPS RINEX 3 NAV edge case

...where FitInterval field is optional per RINEX 3.03 spec.

- Python
Published by scivision about 8 years ago

georinex - cleanup / bugfix

• fix bug reading RINEX 2 NAV
• cleanup / dedupe code
• add Galileo RINEX 3 NAV test

- Python
Published by scivision about 8 years ago

georinex - increase robustness of time stamp

Python < 3.7 handles microsecond precision, more than enough.

- Python
Published by scivision about 8 years ago

georinex - RINEX 3 non-integer second bugfix, increase robustness and speed

• added header mandatory fields sanity check
• simplified and sped up single-system RINEX 3 reading (pick single system from multi-system files).
• Multi-system RINEX 3 reading also available, but is significantly slower than single system reads due to non-uniform fields between satellite systems.
• BUGFIX: non-integer second timing crash

- Python
Published by scivision about 8 years ago

georinex - Add convenience readrinex() function, deprecate Py2.7

convenience function readrinex() handles OBS,NAV for RINEX 2 and 3.

added Python >= 3.5 type hinting

- Python
Published by scivision about 8 years ago

georinex - Rinex 2/3 OBS/NAV reading for one or many satellites

• greatly improve reading/conversion speed and memory usage by replacing Pandas NDPanel with xarray.Dataset
• much more extensive self-tests, tested against a wider variety of RINEX files.

- Python
Published by scivision about 8 years ago

georinex -

- Python
Published by scivision about 9 years ago

georinex - Initial release

- Python
Published by drhirsch over 9 years ago