Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of https://github.com/google/trax

https://github.com/google/trax - v1.4.1

TODO(afrozm): Write this up.

- Python
Published by afrozenator over 4 years ago

https://github.com/google/trax - v1.4.0

TODO(afrozm): Write this up.

- Python
Published by afrozenator over 4 years ago

https://github.com/google/trax - v1.3.9

Some bug fixes.

- Python
Published by afrozenator almost 5 years ago

https://github.com/google/trax - v1.3.8

TODO(afrozm): Describe

- Python
Published by afrozenator almost 5 years ago

https://github.com/google/trax - v1.3.7

Lots of documentation, bugs squashed and misc changes.

Features - Load checkpoint for fine-tuning in https://github.com/google/trax/commit/0349c3f2c32ed83d0ba54a0f6652766ea5467cca by @henrykmichalewski ! - Added generic input pipeline for GLUE tasks in https://github.com/google/trax/commit/2f490e83a9be1b802dccf866c52c8908645e10d6 by @henrykmichalewski - thanks a lot! - Some nascent support for bfloat16s!

Models - Performer's Favor and CausalFavor - https://github.com/google/trax/commit/77db199392ff967e49156ccb24c916b270b47eca thanks @lukaszkaiser ! - Funnel-Transformer in https://github.com/google/trax/pull/1156 thanks a lot @mvxxx ! - BERT for Trax in https://github.com/google/trax/pull/1254 , https://github.com/google/trax/pull/1223, etc by @piotrekp1 - thanks a lot! - Residual Exchange Network by @kkanska in https://github.com/google/trax/commit/3a8f402b722a0d9ae934ed399e6532dba8401f9e ! Thanks!

PRs Merged - Fixing broken example links in https://github.com/google/trax/pull/1263 thanks @amtagrwl ! - Added WideResnet, Deconv etc Example Notebook in https://github.com/google/trax/pull/1259 , https://github.com/google/trax/pull/1202, https://github.com/google/trax/pull/1232 thanks a lot! @SauravMaheshkar ! - Remove implicit object from the base class in https://github.com/google/trax/pull/1228 thanks @HarshCasper ! - Fashion MNIST example in https://github.com/google/trax/pull/1199 thanks @Jimexist ! - Fix PretrainedBERT init in https://github.com/google/trax/pull/1135 by @hepaajan ! - Typo in the TransformerDecoder input parameters description in https://github.com/google/trax/pull/1100 by @kujaomega !

- Python
Published by afrozenator about 5 years ago

https://github.com/google/trax - v1.3.6

TODO(afrozm): Write something.

- Python
Published by afrozenator over 5 years ago

https://github.com/google/trax - v1.3.5

PRs: Thanks @NathanHowell for fixing a bunch of typos in #962 Thanks @DarrenZhang01 for contributing to the TF-Numpy extensions code in #954

ReversibleSerialTrainer for memory efficient, layer by layer training.

Miscellaneous other issues.

- Python
Published by afrozenator over 5 years ago

https://github.com/google/trax - v1.3.4

- Python
Published by afrozenator over 5 years ago

https://github.com/google/trax - v1.3.3

Minor Fixes: * Rename gumbel_sample to logsoftmax_sample in a2497cbd8477a11f2b96e87a4d53ce46b845ffa8 * A fix to storing checkpoints in Loop in 88b033c804a4da9021505f2ce8fbb5c7d6500574

- Python
Published by afrozenator over 5 years ago

https://github.com/google/trax - v1.3.2

Framework: * Data pipeline combinators in 17d44710bee74e8bb6e3b34baa4114b0c78160af * Multi-host training in the Loop api 524321e0f77afa58a9fba470e5bccb19ae6bdb92 * Auto-regressive sampling in fe0fa7886ee33bfac831a969a82b351ccb02941c * T2T Tokenizers 6306231ace01ab55c5df0c401b1f3cf4c7b6a32f * Early work on multi-task training in 423d664b470612ebbc0e5041e8288eb1a47c30ef thanks to @koz4k

Tasks / Gin files: * IMDB dataset in 08bdb50db41a089b6e12c61a8a29981225bf3566 * Machine Translation en-pl in 5fb8aa8c5cb86dabb2338938c745996d5d87d996

- Python
Published by afrozenator over 5 years ago

https://github.com/google/trax - v1.3.1

Miscellaneous fixes.

  • tl.Embedding now has the same signature as pytorch/tf
  • train.lr_schedule (function object) -> train.lr_schedule_fn (function)
  • Report loss back to training.Loop

- Python
Published by afrozenator over 5 years ago

https://github.com/google/trax - v1.3.0

Trax now has docs on - https://trax-ml.readthedocs.io/en/latest/trax.html thanks to @j2i2 !

Many usability changes, especially in trax.supervised.training.TrainTask/EvalTask/Loop, docs, comments etc.

  • flat saved model/checkpoint representation
  • lr schedule simplified, now they just take step number.
  • configs are now in supervised/configs and rl/configs.
  • RL obsolete code cleanup.

Also rapid development of the tf-numpy codebase !

- Python
Published by afrozenator over 5 years ago

https://github.com/google/trax - v1.2.4

Merged PRs:

  • #459 by @w4-sjcho - adding names to layers, aiding debuggability thanks a lot!
  • #256 and #300 by @stephenjfox and @satyarohith refining the README.md language, thanks a lot folks!
  • #313 #312 #436 #396 from @pkol with lots of bugfixes, thanks a lot @pkol !
  • #409 by @pkol -- a special shoutout to this PR, this fixes a long standing issue that prevented termination of the process by tracking the zombie threads -- thanks a lot for this @pkol specially !
  • #386 another shoutout to @pkol for an amazing speedup in the RL code -- thanks a lot again !
  • #344 a psum bugfix with tf backend from @fsx950223 - thanks a lot !
  • #335 a bugfix from @friesel - thanks a lot Phillip !
  • #315 better exception handling by @cool-RR - thanks a lot !

Reformer:

  • BERT initialization and finetuning by Nikita!
  • Many changes including ReformerLM on C4 dataset.

RL:

  • New 'light' RL code in the Trax supervised style, check it out!
  • AWR in the old code working with MuJoCo tasks.

And many more changes the Trax framework !

- Python
Published by afrozenator almost 6 years ago

https://github.com/google/trax - v1.2.3

Reformer

  • Reversible Transformer model for machine translation and other encoder-decoder tasks
  • Add code for beam search, sampling, and greedy decoding (see trax.models.beam_search.Search)
  • Memory-efficient attention classes have been re-written to use even less memory and to support faster decoding (see the new SelfAttention, LSHSelfAttention and EncDecAttention classes)

RL

  • Implemented the Advantage-Weighted Regression algorithm, a simple off-policy reinforcement learning algorithm.
  • Extracted out a PolicyBasedTrainer, so ppo_trainer.PPO and awr_trainer.AwrTrainer now both inherit from it.
  • Refactoring of the serialization code in the RL part, thanks to @koz4k !

Framework

  • A lot of code cleanup and refactoring of the core abstractions by Jonni, thanks Jonni!

TF Numpy

  • More ops added by @wangpengmit !

- Python
Published by afrozenator about 6 years ago

https://github.com/google/trax - v1.2.2

  • More bugfixes related to scan/remat.

- Python
Published by afrozenator about 6 years ago

https://github.com/google/trax - v1.2.1

  • Minor bugfix to GRU and scan.

- Python
Published by afrozenator about 6 years ago

https://github.com/google/trax - v1.2.0

New Models

  • Reformer Implementation - https://arxiv.org/abs/2001.04451 Thanks Nikita Kitaev, @lukaszkaiser and @levskaya !

Colabs

  • Reformer Colabs https://github.com/google/trax/commit/9dbf8636a34cffbd421bedb2a1e3d7fe006346c0
  • Transformer Colab https://github.com/google/trax/commit/d2c5b84b8db53888013c56980ccbafba077ee8f2

Framework Changes

  • Ongoing cleanups and API simplifications.
  • Optimization by @jekbradbury - thanks James!

PRs

  • Consistent logging absl.logging and setup.py fixes thanks to @lkhphuc in #198
  • Code cleanups by @cclauss in #196
  • Code cleanup by @pkol in #134 - thanks!
  • Bug fix by @pzielinski-nyc in #151 - thanks!

- Python
Published by afrozenator about 6 years ago

https://github.com/google/trax - v1.1.2

- Python
Published by afrozenator over 6 years ago