Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of https://github.com/google/etils

https://github.com/google/etils - v1.13.0

  • ecolab:
    • adhoc: Better error message for adhoc invalidate with epy.reraise
    • Fix Digraph patch_graphviz()
    • Fix pretty display with syntax-highlighting for ;l and ;h
    • adhoc.inspect: Cleaner display for protos
  • epath:
    • Fix error in path.is_file() when the file do not exists.
  • edc:
    • Add a edc.ContextStack() util for a thread-safe / contextvars-aware stack.
  • epy:
    • epy.pretty_repr supports collections.UserDict, pydantic, immutabledict objects.
    • epy.pretty_repr dict keys are pretty-printed.
    • Support __setattr__ and __delattr__ on LazyModules.
    • Fix epy.ContextVar descriptor that were sharing their state.
  • etree:
    • Add a etree.copy to copy a tree structure without modifying its leaves

- Python
Published by github-actions[bot] 8 months ago

https://github.com/google/etils - v1.12.2

  • enp:
    • Fix orbax ArraySpec issue.

- Python
Published by github-actions[bot] 12 months ago

https://github.com/google/etils - v1.12.1

  • enp:
    • Add enp.flatten / enp.unflatten to flatten batch shape using an einops-like syntax.

- Python
Published by github-actions[bot] 12 months ago

https://github.com/google/etils - v1.12.0

  • epy:
    • Add experimental util for ml_python + multi-process compatibility for adhoc-imports (when using PyGrain).
    • epy.lazy_api_imports: Make lazy importing thread safe.
    • epy.StrEnum: Fix != operator.
  • epath:
    • Supports hf:// urls.
  • ecolab:
    • Make mediapy lazy

- Python
Published by github-actions[bot] about 1 year ago

https://github.com/google/etils - v1.11.0

  • enp:
    • Make enp.testing.parametrize_xnp() import only requested xnp modules.
    • Fix orbax error when inspecting specs of an orbax checkpoint.
  • ecolab:
    • ecolab.inspect: Proto are better displayed (hide attributes DESCRIPTOR, Extensions in sub-section)
  • epy:
    • epy.lazy_api_imports: Fix infinite recursion when importing sub-module
  • exm:
    • Add dummy implementation of the API to simplify open-sourcing.

- Python
Published by github-actions[bot] over 1 year ago

https://github.com/google/etils - v1.10.0

  • epy:
    • Add frozen dataclass support for epy.ContextManager
    • Make epy.StrEnum truly case-insensitive
    • Support adhoc import of proto files with hyphen.
    • Add fiddle supports to epy.pretty_repr
  • enp: Add ArraySpec support for grain.python.SharedMemoryArrays.

- Python
Published by github-actions[bot] over 1 year ago

https://github.com/google/etils - v1.9.4

  • Return Python 3.10 support.

- Python
Published by github-actions[bot] over 1 year ago

https://github.com/google/etils - v1.9.3

  • eapp:
    • Allow extra kwargs in eapp.make_flags_parser()
  • epath:
    • Fix epath.Path pydantic deserialization for URI-style paths
  • epy:
    • Add epy.is_test to check whether we're running in a test environment.
    • Add epy.typing.Json.
    • Add epy.ExitStack which allows setting the contextmanagers during init.
    • Add proto support for epy.binary_adhoc
    • Hide reraise from the traceback for cleaner error messages
  • exm:
    • Add exm.url_to_python_only_logs() and exm.curr_job_name() to add artifact to Python only logs (without all the verbose C++ junk ).
    • Fix a bug which makes exm.current_experiment crash

- Python
Published by github-actions[bot] over 1 year ago

https://github.com/google/etils - v1.9.2

  • epath:
    • Support pydantic serialization of epath.Path

- Python
Published by github-actions[bot] over 1 year ago

https://github.com/google/etils - v1.9.1

  • epath:
    • Fix an infinite recursion on is_relative_to for Python>=3.12.

- Python
Published by github-actions[bot] over 1 year ago

https://github.com/google/etils - v1.9.0

  • epy:
    • Add epy.lazy_api_imports to lazy-import __init__.py symbols.
    • Removed: epy.cached_property
    • epy.lazy_imports: Error callback accept a str to auto-re-raise with additional info.
    • Fix mixing epy.lazy_imports() with epy.binary_adhoc().
  • ecolab:
    • Added reload_workspace=True to adhoc to auto-reload from workspace
    • Add ecolab.get_permalink()
    • Fix ecolab.inspect not finding static files when the kernel contain partial etils deps.
  • epath:
    • Fix error when importlib.resources.files return MultiplexedPath
    • Fix gs:// URI for 3.12
    • Fix .walk 3.12 error (topdown -> top_down rename)
  • Full compatibility with Python 3.12 (unit tests run on both 3.11 and 3.12).

- Python
Published by github-actions[bot] over 1 year ago

https://github.com/google/etils - v1.8.0

  • Drop Python 3.10 support.
  • epy:
    • epy.pretty_repr: Add support for namedtuple
  • ecolab:
    • Add ecolab.disp(obj)
    • Add ;h for syntax highlighting with auto-display
    • Fix proto error on import

- Python
Published by github-actions[bot] almost 2 years ago

https://github.com/google/etils - v1.7.0

  • epath:
    • Add mode to epath.Path.stat output. Does not work for Windows nor when tf.io.gfile is used.
    • Add .walk to epath.Path. Similar usage than pathlib.Path.walk
  • epy:
    • Added: epy.reverse_fstring: Reverse fstring parsing
    • Added: reload= for epy.binary_adhoc(), fixed behavior for consistency with ecolab.adhoc
    • Added: epy.pprint: Pretty print an object (including dataclass).
    • Added: epy.pretty_repr_top_level
    • Added: epy.wraps_cls equivalent of functools.wraps but for classes.
    • Breaking: epy.lazy_imports(error_callback=) has now signature (Exception) -> None (instead of (str) -> None)
    • Fixed: epy.pretty_repr missing trailing , for tuple with single element.
  • ecolab:
    • Changed: ecolab.auto_display: Better representation when line is displayed
    • Fix adhoc that delete sub-module when invalidate=False
    • adhoc with reload_mode=UPDATE_INPLACE now supports enums, so old versions compare equal to new versions. Enums compared as a is FOO might still fail.
    • adhoc with reload_mode=UPDATE_INPLACE is now much faster.
    • When using cell_autoreload=True the default reload_mode is now UPDATE_INPLACE.
    • Better error message for adhoc reload
  • exm:
    • Added: exm.set_citc_source() to specify which workspace to use when using XManager on Colab
  • etree:
    • Added is_leaf kwarg to .map and .parallel_map
  • enp:
    • Add ArraySpec support for flax.linen.summary.

- Python
Published by github-actions[bot] about 2 years ago

https://github.com/google/etils - v1.6.0

  • ecolab:
    • Added protobuf repeated fields support to ecolab.inspect
    • ecolab.auto_display:
      • Support specifiers to customize auto-display (;s, ;a, ;i, ;p,...)
      • Fixed auto-display when the line contain UTF-8 character
    • Fix a bug for ecolab.highlight_html to escape HTML string.
  • epy:
    • epy.lazy_imports() support adhoc imports (will re-create the original ecolab.adhoc context when resolved)
    • Added: epy.binary_adhoc() to add adhoc imports when using Python interpreter.
    • epy.lazy_imports() supports error and success callbacks (error_callback/success_callback).
    • Added: epy.pretty_repr support attr dataclass-like objects.
  • exm:
    • Adding artifacts can be used inside with xm.create_experiment()

- Python
Published by github-actions[bot] about 2 years ago

https://github.com/google/etils - v1.5.2

  • ecolab:
    • Fix import error in IPython when 7.0 <= version < 8
  • epath:
    • Fix resource_path when used on a adhoc-imported module.

- Python
Published by github-actions[bot] over 2 years ago

https://github.com/google/etils - v1.5.1

  • epath:
    • Fix glob issue when used with ffspec.

- Python
Published by github-actions[bot] over 2 years ago

https://github.com/google/etils - v1.5.0

  • ecolab:
    • Auto display statements ending with ; (assignments, return statements, expressions,...).
    • Adhoc proto now supports message extensions.
  • epath:
    • Add missing_ok=False kwargs to path.rmtree.
    • Add fsspec_backend relying on fsspec to handle GCS/S3 without needing TensorFlow. This means that gcsfs and s3fs become required dependencies to read respectively GCS and S3. TensorFlow is no more required. Note: If TensorFlow is installed, we still default to the tf_backend for backward compatibility.
    • Changed: path.glob raise an error if insufficient permission.
  • enp:
    • ArraySpec.from_array:
      • Fix when TF is in graph mode.
      • Add orbax metadata support
  • epy:
    • Add epy.pretty_repr for pretty print an object (including dataclass).
    • Add epy.diff_str for pretty diff print of 2 objects.
    • Add epy.is_namedtuple
  • etree:
    • Fix etree.map for namedtuple (Python backend)

- Python
Published by github-actions[bot] over 2 years ago

https://github.com/google/etils - v1.4.1

  • lazy_imports: More lazy imports
  • epath: Remove circular deps

- Python
Published by github-actions[bot] over 2 years ago

https://github.com/google/etils - v1.4.0

  • epy:
    • Add @epy.lazy_imports to lazyly import modules.
    • Fix @epy.frozen when class has custom __getattr__
  • ecolab:
    • ecolab.collapse()
      • Breaking: Remove widget=True argument to (widget always enabled).
        • Add expanded: bool kwargs to control whether the widget is expanded or collapsed by default.
    • Breaking: Remove keep_proto kwargs from clear_cached_modules and import_proto kwargs from adhoc (proto always supported)
    • Add expanded argument to ecolab.json() and change default behavior to False.
  • enp:
    • Make array spec (e.g. etree.spec_like()) hashable.
    • enp.lazy.is_tf returns True for tf.TensorSpec
    • Remove array_types.dtypes and array_types.typing (should use enp.dtypes and enp.typing instead).
  • epath:
    • Add owner and group to epath.Path.stat output. Does not work for Windows nor when tf.io.gfile is used.
  • etree:
    • Fix etree.map for collections.defaultdict
  • internal:
    • Add a unwrap_on_reload to save/restore original function after a module is reloaded (e.g. on colab)

- Python
Published by github-actions[bot] over 2 years ago

https://github.com/google/etils - v1.3.0

  • ecolab:
    • Add widget=True argument to ecolab.collapse() for better interactivity.
    • Add ecolab.highlight_html(code) to add syntax highlighting to cell outputs of specific objects.
  • edc:
    • Add contextvars option: Fields annotated as edc.ContextVars[T] are wrapped in contextvars.ContextVars.
    • Fix error when using _: dataclasses.KW_ONLY
    • __repr__ now also pretty-print nested dataclasses, list, dict,...
  • enp:
    • ArraySpec support grain.ArraySpec (allow support etree.spec_like(ds.element_spec) on grain datasets)
  • epy:
    • Better epy.Lines.make_block for custom pretty print classes, list,...

- Python
Published by github-actions[bot] almost 3 years ago

https://github.com/google/etils - v1.2.0

  • enp:
    • etree.spec_like support f32,... annotations (when fully defined).
  • etree:
    • Add optree backend.
  • ecolab:
    • Add ecolab.auto_inspect() to allow inspecting all colab outputs.
    • Fix various ecolab.inspect issues (e.g. when used on metaclasses,...).
    • Add ecolab.inspect support for Jupyter notebook (non-Colab).

- Python
Published by github-actions[bot] almost 3 years ago

https://github.com/google/etils - v1.1.1

  • enp:
    • Fix torch==2.0 compatibility.
  • Fix warning during pip install (missing epath-no-tf)

- Python
Published by github-actions[bot] almost 3 years ago

https://github.com/google/etils - v1.1.0

  • enp:
    • Add torch support!
    • Add enp.lazy.LazyArray to lazily check isinstance(array, enp.lazy.LazyArray) without triggering TF, jax,... imports
    • Add skip=['jnp', ...] kwarg to enp.testing.parametrize_xnp to exclude a specific xnp module from tests.
    • Add enp.compat function to fix compatibility issues between Jax, TF, torch, numpy (e.g. x.astype() missing from torch, but working in jax, tf, np).
    • Breaking: Move some functions from enp.linalg to enp.compat.
    • Add more enp.lazy. methods for conversions from/to dtype
  • ecolab:
    • Add ecolab.inspect for interactively inspect any Python objects.
    • Add ecolab.json for interactive expandable JSON display.
    • ecolab.auto_plot_array
      • Add torch
      • Small/large images (height outside 100-250px) are automatically scaled up/down. Can be disabled with ecolab.auto_plot_array(height=None)
      • Can overwrite mediapy default options with show_images_kwargs and show_videos_kwargs
    • Add ecolab.interruptible for graceful interruption of loops.
    • Fixed: Pytype/pylance support for lazy_imports: unlock auto-complete, docstring tooltip, do not trigger linter errors anymore ("xxx" is not definedPylancereportUndefinedVariable).
  • etree:
    • from etils import etree now expose the Python backend ( etree.map,...). Other backend are still available as previously (etree.jax.map,...)
  • epy:
    • Added: epy.splitby to split an iterator in 2, based on a predicate. (e.g. small, big = epy.splitby([20, 1, 30, 1, 2], lambda x: x > 10))

- Python
Published by github-actions[bot] almost 3 years ago

https://github.com/google/etils - v1.0.0

  • etree:
    • Added: etree.stack to stack/batch multiple trees of arrays together.
    • Added: etils.etree.py backend (pure Python implementation to avoid installing extra deps).
    • Added: etree.map (as convenience to access the corresponding map_structure).
  • enp:
    • Added: enp.batch_dot: Always dot product on the last axis with broadcasting support (while np.dot is inconsistent 1-D vs 2-D).
    • Added: enp.angle_between to compute angle between 2 n-dimensions vectors.
    • Changed: enp.project_onto_vector, enp.project_onto_plane supports broadcasting.
    • Fixed: TF accidentally imported when using enp.linalg functions.
  • ecolab:
    • Added: ecolab.patch_graphviz to fix Colab display with graphviz.
    • Added: ecolab.set_verbose to activate stderr logging on Colab.
    • Changed: ecolab.clear_cached_modules accept single str
    • Changed: ecolab.clear_cached_modules has a invalidate=False to not invalidate previous instances of the modules.
  • edc:
    • Added: Expose edc.repr, for functional use or directly assign class members (e.g. __repr__ = edc.repr)
  • eapp:
    • Fixed: eapp.better_logging() do not raise is_borg AttributeError anymore.
  • epath:
    • Fixed: local_path.copy('gs://') uses the correct backend.
  • All:
    • Changed: Better error message when missing import.

- Python
Published by github-actions[bot] about 3 years ago

https://github.com/google/etils - v0.9.0

  • eapp (Added):
    • Added: .make_flags_parser to define CLI flags through dataclasses.
    • Added: .better_logging to display logs by default, tqdm compatibility,...
  • epy:
    • Added: @epy.frozen class decorator to make class immutable
  • edc:
    • Added: Attributes of @edc.dataclass can be annotated with x: edc.AutoCast[T] (for auto-normalization).
  • ecolab:
    • Added: ecolab.clear_cached_modules to reload modules (useful for interactive development)
    • Added: etils.lazy_imports supports multi-import (e.g. using concurrent also trigger concurrent.futures import).

- Python
Published by github-actions[bot] over 3 years ago

https://github.com/google/etils - v0.8.0

  • epath:
    • Added: path.stat()
    • Fix performance issues for path.mkdir.
  • ecolab:
    • Added: from etils.lazy_imports import * alias of from etils.ecolab.lazy_imports import *.
    • Changed: Mutating a lazy_import module mutate the original one. This allow to mutate builtins module for example.
  • edc:
    • Changed: .replace only added once (not in subclasses).

- Python
Published by github-actions[bot] over 3 years ago

https://github.com/google/etils - v0.7.1

  • ecolab: Added dataclass_array to lazy_imports.

- Python
Published by github-actions[bot] over 3 years ago

https://github.com/google/etils - v0.7.0

  • array_types:
    • Added: More array types: complex64,...
    • Added: An experimental array_types.dtypes.DType to support more flexible dtype expression (AnyFloat, type union,...)
    • Changed: FloatArray, IntArray do supports any float, int without casting.
    • Changed (breaking): Array dtypes (e.g. (f32.dtype) are now array_types.dtypes.DType.
  • ecolab:
    • Added: lazy_imports.print_current_imports to display the active lazy imports (e.g. to add imports before publishing a colab).
  • epy:
    • Added: epy.ContextManager to create yield-based contextmanager class (see discussion)
    • Added: epy.issubclass (like issubclass but does not raises error for non-types)
    • Added: epy.groupby, like itertools.groupby but returns a dict.
    • Added: epy.Lines.make_block helper to create code blocks (function calls,...)
    • Fixed: epy.StrEnum raises better error message if invalid input.
  • epath
    • Added: epath.DEFINE_path for absl.flags support
    • Changed (breaking): Recursive glob (rglob, glob('**/')) now raise an error rather than being silently ignored.
    • Changed: path.as_uri() returns gs:// and s3:// (rather than file:///gs/)
    • Changed: Add __eq__ and __hash__ for resource path.
  • edc
    • Fixed: __repr__ is correctly added in Python 3.10 (#143)
    • Fixed: dc.frozen() compatibility with autograd.
    • Changed: dc.unfrozen()now supports jax.tree_map.
    • Changed: Better dc.unfrozen() repr which display overwritten fields.
  • enp:
    • Added: enp.check_and_normalize_arrays util to dynamically validate array dtype/shape from typing annotations.
    • Added: enp.linalg.normalize util.
    • Added: enp.project_onto_vector and enp.project_onto_plane geometry utils.
  • Other:

- Python
Published by github-actions[bot] over 3 years ago

https://github.com/google/etils - v0.6.0

  • epath:
    • Remove TensorFlow dependency from epath.Path by default. (For now accessing gs:// still require TF to be installed).
    • Add epath.testing.mock_epath to mock GCS calls.
  • epy.testing:
    • Add epy.testing.subtest for better subtests support.
    • Add epy.testing.non_hermetic to mark non-hermetic tests.
  • oss-kit:

- Python
Published by github-actions[bot] over 3 years ago