Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of tum_esm_utils

tum_esm_utils - v2.8.0

🦚  New Features

  • Add function code.download_github_release_asset
  • Improve library import times (#28) when setting env var TUM_ESM_UTILS_EXPLICIT_IMPORTS=1 (see docs)
  • Add function datastructures.concat_lists
  • Add function system.get_physical_memory_usage
  • Add function timing.timed_section
  • Add function mathematics.divides_evenly (reason explained in the function docs)

🐜  Internal Changes

  • Debug shell.get_commit_sha if git is not installed on the system
  • Lower constraint for polars dependency to 1.16.0 because newer polars releases have memory leaks
  • Debug opus.OpusHTTPInterface.some_macro_is_running for OPUS 8.9 (some error codes changed for OPUS 8.9)

- Python
Published by dostuffthatmatters 12 months ago

tum_esm_utils - v2.7.0

🦚  New Features

  • column.astronomy module: Convenience function for astronomical calculations
  • column.ncep_profiles module: Loading GGG2020 atmospheric profiles into polars dataframes
  • column.averaging_kernel module: Functions to store, load and apply a column averaging kernel.

Use the new column module by installing tum_esm_utils[modeling].

- Python
Published by dostuffthatmatters about 1 year ago

tum_esm_utils - v2.6.0

🦚  New Features

  • opus.OpusHTTPInterface class Communicate with OPUS using its HTTP interface (see docs and API Reference)

🐜  Internal Changes

  • Fix bug when only reading the first channel from OPUS files
  • Refine pytests: add test groups, make tests run when .git folder is not present
  • Add repository to Zenodo

- Python
Published by dostuffthatmatters over 1 year ago

tum_esm_utils - v2.5.0

🦚  New Features

  • opus Module Read OPUS files. Credits to @FriedrichKlappenbach for decoding the OPUS file structure.

🐜  Internal Changes

  • Switch from YAPF to RUFF formatting (easier to configure and better maintained)
  • Rename polars dependency group to em27. The old name will be kept as an alias until the next breaking release (3.0.0)

- Python
Published by dostuffthatmatters over 1 year ago

tum_esm_utils - v2.4.0

🦚  New Features

  • timing.ExponentialBackoff Add class to exponentially increase the waiting time after an error see in docs

  • StricterBaseModel, StrictIPv4Adress More specialized Pydantic base classes see in docs

  • timing.date_span_intersection, timing.datetime_span_intersection Functions to compute the overlap of two date/datetime periods see in docs

🦀  Dependency Updates

  • Explicitly add Python 3.13 support (tests, PyPI package)

- Python
Published by dostuffthatmatters over 1 year ago

tum_esm_utils - v2.3.0

🦚  New Features

  • em27.PROFFAST_MULTIPLIERS, em27. PROFFAST_UNITS, ... Add EM27/SUN constants for instrument colors, serial numbers, data units, etc. see in docs

  • plotting.add_colorpatch_legend A function to add a "Colorpatch Legend" to a plot see in docs

  • RandomLabelGenerator Generate random names - like Docker container names curious-einstein, wild-picard and so on see in docs

  • tum_esm_utils.timing.parse_iso_8601_datetime Added because datetime.datetime.fromisoformat does not support all ISO 8601 timezone variants see in docs

  • tum_esm_utils.files.list_directory Adds more options the os.listdir method - only include directories/files, filter by regex, ignore by Unix shell patterns see in docs

  • tum_esm_utils.files.render_directory_tree Render a directory tree as a string see in docs

  • tum_esm_utils.text.simplify_string_characters Simplify a string by replacing special characters with their ASCII counterparts and removing unwanted characters with. For example, simplifystringcharacters("Héllo, wörld!") will return "hello-woerld". see in docs

  • tum_esm_utils.text.replace_consecutive_characters Remove consecutive whitespaces or other characters with tum_esm_utils.text.replace_consecutive_characters see in docs

🦀  Dependency Updates

- Python
Published by dostuffthatmatters almost 2 years ago

tum_esm_utils - v2.2.0

Upgrade Numpy (v1 to v2) and Polars (v0 to v1).

Add Version validator (#19).

- Python
Published by dostuffthatmatters almost 2 years ago

tum_esm_utils - v2.1.0

This release brings a significantly improved opus file validation based on Proffast 2.4. The old version could only handle a few reasons of file corruption. Furthermore the output did not list the files that the preprocessing could not even read.

Old validator:

python detection_results = tum_esm_utils.em27.detect_corrupt_ifgs( "/path/to/a/folder/with/interferograms" ) assert detection_results == { "md20220409s0e00a.0199": [ "charfilter 'GFW' is missing", "charfilter 'GBW' is missing", "charfilter 'HFL' is missing", "charfilter 'LWN' is missing", "charfilter 'TSC' is missing", ] }

New validator:

```python detectionresults = tumesmutils.em27.detectcorruptopusfiles( "/path/to/a/folder/with/interferograms" )

detectionresults == { 'md20220409s0e00a.0199': [ 'Charfilter "DUR" is missing', 'Charfilter "GBW" is missing', 'Charfilter "GFW" is missing', 'Charfilter "HFL" is missing', 'Charfilter "LWN" is missing', 'Charfilter "TSC" is missing', 'Differing sizes of dual channel IFGs!', 'IFG size too small!', 'Inconsistent dualifg!', 'Inconsistent parameter kind in OPUS file!' ], 'combinvparmsmaSN061210329-210329.csv': [ 'File not even readible by the parser' ], 'combinvparmsmcSN115_220602-220602.csv': [ 'File not even readible by the parser' ], 'md20220409s0e00a.0200': [ 'File not even readible by the parser' ] } ```

The function tum_esm_utils.em27.detect_corrupt_ifgs has been renamed to tum_esm_utils.em27.detect_corrupt_opus_files but is of course still available under the old name too (until the next breaking release).

- Python
Published by dostuffthatmatters about 2 years ago

tum_esm_utils - v2.0.0

This breaking release is necessary to make the polars library an optional dependency. Until now, every codebase that wanted to use the tum-esm-utils library would have to use the same major release of polars.

The library now has two optional modules: em27 and plotting, for which the dependencies (polars and matplotlib are not automatically installed). Install them using:

python pip install "tum-esm-utils[polars]" pip install "tum-esm-utils[plotting]" pip install "tum-esm-utils[polars,plotting]"

Due to this change, the base library is kept at ~12MB, including all (sub-)dependencies, and only if you want the plotting and EM27 extras, this grows to ~174MB.

🦞  Breaking changes

I included many breaking changes here, so there will not be a need for another breaking release soon.

  • Module interferograms has been renamed em27 and made optional
  • Removed the context and testing modules: functionality has been moved into timing and files
  • Renamed module github to code
  • Removed unused functionality and changed some return types (see #18 for complete list)
  • Dropped Python3.8 support because Matplotlib also dropped it
  • Changed from MIT to GPL license

🦚  New features

  • Added new optional plotting module
  • Refactored function to load Proffast2 output files
  • Added function to code module to load files from GitLab

- Python
Published by dostuffthatmatters about 2 years ago

tum_esm_utils - v1.9.0

New Features

  • Add a new method tum_esm_utils.files.read_last_n_lines (docs)
  • Add a new method tum_esm_utils.timing.parse_timezone_string (docs)

Dependencies

  • Update pytest@7.4.4 to 8.0.0

- Python
Published by dostuffthatmatters over 2 years ago

tum_esm_utils - v1.8.0

New Features

  • Add a new method tum_esm_utils.timing.date_range: https://tum-esm-utils.netlify.app/api-reference#date_range-1

Dependencies

  • Update polars@0.19 to 0.20

Internal Changes

  • Copy all the functions in the context module (all time related) to the new timing module
  • Add deprecation warnings with the link to the new timing module to all migrated functions in the context module
  • Use timing.date_range inside text.date_range (the latter is based on date strings like YYYYMMDD)

- Python
Published by dostuffthatmatters over 2 years ago

tum_esm_utils - 1.7.0

New Features

  • Added a new method to strictly validate paths using Pydantic root models: https://tum-esm-utils.netlify.app/example-usage#strict-path-validation-with-pydantic
  • Added a new very clean method to convert paths relative to the script into absolute files: https://tum-esm-utils.netlify.app/example-usage#get-absolute-paths
  • Added a function to set alarms in order to catch infinite loops in your code: https://tum-esm-utils.netlify.app/example-usage#set-alarms-to-catch-infinite-loops

All of the above is tested, statically typed and documented of course 🐝

- Python
Published by dostuffthatmatters over 2 years ago

tum_esm_utils - 1.6.0 - Significantly Improved Documentation

We moved from Docsify to Nextra to render the actual docs page. We moved from lazydocs to pydoc-markdown to automatically generate the markdown API reference from the codebase

Feel free to use this documentation setup in your own projects to improve the usability of the content you produce 🌈

API reference - Old | New - now all on one scrollable page

api-reference-old api-reference-new

Examples - Old | New - more and improved usage examples

examples-old examples-new

Search - Old | New - a search that is actually useful

search-old search-new

- Python
Published by dostuffthatmatters over 2 years ago