Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of tensorflow

tensorflow - TensorFlow 2.20.0

Release 2.20.0

TensorFlow

Breaking Changes

  • The tensorflow-io-gcs-filesystem package is now optional, due its uncertain, and limited support. To install it alongside tensorflow, run pip install "tensorflow[gcs-filesystem]".

Major Features and Improvements

  • tf.data
    • Adds autotune.min_parallelism to tf.data.Options to enable faster input pipeline warm up.
  • tf.lite
    • tf.lite will be deprecated, in favor of the new repo https://github.com/google-ai-edge/LiteRT.
    • The duplicated source will also be removed from the TF repo.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

1ndig0, 372046933, abhinav, afzpatel, Akhil Goel, Alain Carlucci, Aleksei, Alen Huang, Alex, Amrinfathima-Mcw, Aravindh Balaji, Armand Picard, Aseem Athale, Ashiq Imran, Assoap, Chao, Chase Riley Roberts, Chenhao Jiang, chunhsue, chuntl, Chunyu Jin, Corentin Kerisit, Crefeda Rodrigues, dependabot[bot], Dragan Mladjenovic, Elen Kalda, Felix Thomasmathibalan, gabeweisz, Gauri Deshpande, Georg Stefan Schmid, Guozhong Zhuang, Harsha H S, Harshith_N, Hugo Mano, Ian Tayler Lessa, Jack Wolfard, James Ward, Jane Liu, Jaroslav Sevcik, JD, Jerry-Ge, Jian Li, Jinzhe Zeng, jiunkaiy, Johannes Reifferscheid, johnnkp, junweifu, Kanvi Khanna, Kasper Nielsen, Linzb-Xyz, Luke Hutton, Mahmoud Abuzaina, Mathew Odden, Michael Platings, misterBart, Mitchell Ludwig, Mmakevic-Amd, mraunak, NamanAgarwal0905, Namrata-Ibm, Neuropilot-Captain, nhatle, Nicholas Wilson, Nikhil Shinde, Olli Lupton, Patrick J. Lopresti, Pavel Emeliyanenko, Pearu Peterson, pemeliya, Peng Sun, Philipp Hack, Pratham-Mcw, RahulSudarMCW, RakshithGB, Rakshithgb-Fujitsu, RuslanSemchenko, Ruturaj Vaidya, Sachin Muradi, sandeepgupta12, SaoirseARM, Sergey Kozub, Sevin Fide Varoglu, Shanbin Ke, Shaogang Wang, Shraiysh Vaishay, Siddhartha Menon, spiao, Swatheesh Muralidharan, Tai Ly, Terry Sun, Thibaut Goetghebuer-Planchon, Thomas Dickerson, Tilak, Tj Xu, Trevor Morris, tyb0807, vfdev, Wei Wang, wokron, wondertx, Xuefei Jiang, Yaowei Zhou, Zentrik, Ziyun Cheng, Zoranjovanovic-Ns

- C++
Published by tensorflow-jenkins 6 months ago

tensorflow - TensorFlow 2.19.1

Release 2.19.1

Bug Fixes and Other Changes

  • Fix save_model.save for Serving embedding and add SparseCore Reshard.

- C++
Published by tensorflow-jenkins 6 months ago

tensorflow - TensorFlow 2.20.0-rc0

Release 2.20.0

TensorFlow

Breaking Changes

  • The tensorflow-io-gcs-filesystem package is now optional, due its uncertain, and limited support. To install it alongside tensorflow, run pip install "tensorflow[gcs-filesystem]".

Major Features and Improvements

  • tf.data
    • Adds autotune.min_parallelism to tf.data.Options to enable faster input pipeline warm up.
  • tf.lite
    • tf.lite will be deprecated, in favor of the new repo https://github.com/google-ai-edge/LiteRT.
    • The duplicated source will also be removed from the TF repo.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

1ndig0, 372046933, abhinav, afzpatel, Akhil Goel, Alain Carlucci, Aleksei, Alen Huang, Alex, Amrinfathima-Mcw, Aravindh Balaji, Armand Picard, Aseem Athale, Ashiq Imran, Assoap, Chao, Chase Riley Roberts, Chenhao Jiang, chunhsue, chuntl, Chunyu Jin, Corentin Kerisit, Crefeda Rodrigues, dependabot[bot], Dragan Mladjenovic, Elen Kalda, Felix Thomasmathibalan, gabeweisz, Gauri Deshpande, Georg Stefan Schmid, Guozhong Zhuang, Harsha H S, Harshith_N, Hugo Mano, Ian Tayler Lessa, Jack Wolfard, James Ward, Jane Liu, Jaroslav Sevcik, JD, Jerry-Ge, Jian Li, Jinzhe Zeng, jiunkaiy, Johannes Reifferscheid, johnnkp, junweifu, Kanvi Khanna, Kasper Nielsen, Linzb-Xyz, Luke Hutton, Mahmoud Abuzaina, Mathew Odden, Michael Platings, misterBart, Mitchell Ludwig, Mmakevic-Amd, mraunak, NamanAgarwal0905, Namrata-Ibm, Neuropilot-Captain, nhatle, Nicholas Wilson, Nikhil Shinde, Olli Lupton, Patrick J. Lopresti, Pavel Emeliyanenko, Pearu Peterson, pemeliya, Peng Sun, Philipp Hack, Pratham-Mcw, RahulSudarMCW, RakshithGB, Rakshithgb-Fujitsu, RuslanSemchenko, Ruturaj Vaidya, Sachin Muradi, sandeepgupta12, SaoirseARM, Sergey Kozub, Sevin Fide Varoglu, Shanbin Ke, Shaogang Wang, Shraiysh Vaishay, Siddhartha Menon, spiao, Swatheesh Muralidharan, Tai Ly, Terry Sun, Thibaut Goetghebuer-Planchon, Thomas Dickerson, Tilak, Tj Xu, Trevor Morris, tyb0807, vfdev, Wei Wang, wokron, wondertx, Xuefei Jiang, Yaowei Zhou, Zentrik, Ziyun Cheng, Zoranjovanovic-Ns

- C++
Published by tensorflow-jenkins 7 months ago

tensorflow - TensorFlow 2.19.0

Release 2.19.0

TensorFlow

Breaking Changes

  • LiteRT, a.k.a. tf.lite:
    • C++ API:
      • The public constants tflite::Interpreter:kTensorsReservedCapacity and tflite::Interpreter:kTensorsCapacityHeadroom are now const references, rather than constexpr compile-time constants. (This is to enable better API compatibility for TFLite in Play services while preserving the implementation flexibility to change the values of these constants in the future.)
    • Python API:
      • tf.lite.Interpreter gives deprecation warning redirecting to its new location at ai_edge_litert.interpreter, as the API tf.lite.Interpreter will be deleted in TF 2.20. See the migration guide for details.

Known Caveats

Major Features and Improvements

  • tf.lite
    • tfl.Cast op is now supporting bfloat16 in runtime kernel.

Bug Fixes and Other Changes

  • We have stopped publishing libtensorflow packages but it can still be unpacked from the PyPI package.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Akhil Goel, akhilgoe, Alain Flaischer, Alex, Alexander Pivovarov, Alexander Shadchin, Alexis Praga, Amrinfathima-Mcw, Andrey Pikas, Andrey Portnoy, Ankur Singh, Ashiq Imran, Assoap, c8ef, charleshofer, Chase Riley Roberts, Chenhao Jiang, Chongyun Lee, Claudio Desouza, Corentin Godeau, Crefeda Rodrigues, Danny Burrow, dependabot[bot], Dimitris Vardoulakis, Dragan Mladjenovic, Elfie Guo, Emmanuel Ferdman, fiberflow, flyingcat, Gary Yi-Hung Chen, Georg Stefan Schmid, Gerwout Van Der Veen, Harsha H S, Harshit Monish, Hugo Mano, i.Pear, Ilia Sergachev, Jane Liu, Jaroslav Sevcik, Jc (Jonathan Chen), Jerry Ge, Jian Li, johndoknjas, Johnny, Jonathan Albrecht, Kaixi Hou, Kanvi Khanna, keerthanakadiri, Kevin Ji, Kiran Sai Ramineni, kwoncy2020, LakshmiKalaKadali, Lee, Jun Seok, Mahmoud Abuzaina, Matt Bahr, mayuyuace, Melissa Weber Mendonça, misterBart, Mkarpushin-Enhancelab, Mmakevic-Amd, mraunak, nallave, Nayana Thorat, Nayana-Ibm, nick.camarena, Nicolas Castet, Om Thakkar, oyzh, Parsa Homayouni, Patrick Toulme, Pavel Emeliyanenko, Pavithra Eswaramoorthy, Pearu Peterson, pemeliya, Philipp Hack, Ravi Kumar Soni, redwrasse, Ruturaj Vaidya, Sallenkey-Wei, Sandeep Gupta, Sergey Kozub, Sevin Fide Varoglu, Shanbin Ke, Shaogang Wang, Shixin Zhang, Shraiysh, Shu Wang, Silvio Traversaro, snadampal, Sunita Nadampalli, Tai Ly, Tatwai Chong, tchatow, tdanyluk, Terry Sun, Tilak, Tj Xu, Trevor Morris, Twice, vfdev, Vladimir Silyaev, Weisser, Pascal, wokron, Won Jeon, Xuefei Jiang, Zentrik, Zoranjovanovic-Ns

- C++
Published by tensorflow-jenkins 12 months ago

tensorflow - TensorFlow 2.18.1

Release 2.18.1

Security

Bug Fixes and Other Changes

  • Loosen ml_dtypes upperbound to < 1.0.0 to reduce conflicts when installed with other ML ecosystem components.

Breaking Changes

  • tf.lite
    • Interpreter:
      • tf.lite.Interpreter gives warning of future deletion and a redirection notice to its new location at ai_edge_litert.interpreter. See the migration guide for details.
  • Tensorflow-tpu for this patch is skipped due to some sparsecore related bugs. We suggest to upgrade to 2.19.0 instead.

- C++
Published by tensorflow-jenkins 12 months ago

tensorflow - TensorFlow 2.19.0-rc0

Release 2.19.0

TensorFlow

Breaking Changes

  • LiteRT, a.k.a. tf.lite:
    • C++ API:
    • The public constants tflite::Interpreter:kTensorsReservedCapacity and tflite::Interpreter:kTensorsCapacityHeadroom are now const references, rather than constexpr compile-time constants. (This is to enable better API compatibility for TFLite in Play services while preserving the implementation flexibility to change the values of these constants in the future.)
    • Interpreter:
      • tf.lite.Interpreter gives deprecation warning redirecting to its new location at ai_edge_litert.interpreter, as the API tf.lite.Interpreter will be deleted in TF 2.20. See the migration guide for details.

Known Caveats

Major Features and Improvements

  • tf.lite
    • tfl.Cast op is now supporting bfloat16 in runtime kernel.

Bug Fixes and Other Changes

  • We have stopped publishing libtensorflow packages but it can still be unpacked from the PyPI package.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Akhil Goel, akhilgoe, Alain Flaischer, Alex, Alexander Pivovarov, Alexander Shadchin, Alexis Praga, Amrinfathima-Mcw, Andrey Pikas, Andrey Portnoy, Ankur Singh, Ashiq Imran, Assoap, c8ef, charleshofer, Chase Riley Roberts, Chenhao Jiang, Chongyun Lee, Claudio Desouza, Corentin Godeau, Crefeda Rodrigues, Danny Burrow, dependabot[bot], Dimitris Vardoulakis, Dragan Mladjenovic, Elfie Guo, Emmanuel Ferdman, fiberflow, flyingcat, Gary Yi-Hung Chen, Georg Stefan Schmid, Gerwout Van Der Veen, Harsha H S, Harshit Monish, Hugo Mano, i.Pear, Ilia Sergachev, Jane Liu, Jaroslav Sevcik, Jc (Jonathan Chen), Jerry Ge, Jian Li, johndoknjas, Johnny, Jonathan Albrecht, Kaixi Hou, Kanvi Khanna, keerthanakadiri, Kevin Ji, Kiran Sai Ramineni, kwoncy2020, LakshmiKalaKadali, Lee, Jun Seok, Mahmoud Abuzaina, Matt Bahr, mayuyuace, Melissa Weber Mendonça, misterBart, Mkarpushin-Enhancelab, Mmakevic-Amd, mraunak, nallave, Nayana Thorat, Nayana-Ibm, nick.camarena, Nicolas Castet, Om Thakkar, oyzh, Parsa Homayouni, Patrick Toulme, Pavel Emeliyanenko, Pavithra Eswaramoorthy, Pearu Peterson, pemeliya, Philipp Hack, Ravi Kumar Soni, redwrasse, Ruturaj Vaidya, Sallenkey-Wei, Sandeep Gupta, Sergey Kozub, Sevin Fide Varoglu, Shanbin Ke, Shaogang Wang, Shixin Zhang, Shraiysh, Shu Wang, Silvio Traversaro, snadampal, Sunita Nadampalli, Tai Ly, Tatwai Chong, tchatow, tdanyluk, Terry Sun, Tilak, Tj Xu, Trevor Morris, Twice, vfdev, Vladimir Silyaev, Weisser, Pascal, wokron, Won Jeon, Xuefei Jiang, Zentrik, Zoranjovanovic-Ns

- C++
Published by tensorflow-jenkins 12 months ago

tensorflow - TensorFlow 2.18.0

Release 2.18.0

TensorFlow

Breaking Changes

  • tf.lite

    • C API:
      • An optional, fourth parameter was added TfLiteOperatorCreate as a step forward towards a cleaner API for TfLiteOperator. Function TfLiteOperatorCreate was added recently, in TensorFlow Lite version 2.17.0, released on 7/11/2024, and we do not expect there will be much code using this function yet. Any code breakages can be easily resolved by passing nullptr as the new, 4th parameter.

  • TensorRT support is disabled in CUDA builds for code health improvement.

  • Hermetic CUDA support is added.

Hermetic CUDA uses a specific downloadable version of CUDA instead of the user’s locally installed CUDA. Bazel will download CUDA, CUDNN and NCCL distributions, and then use CUDA libraries and tools as dependencies in various Bazel targets. This enables more reproducible builds for Google ML projects and supported CUDA versions.

Known Caveats

Major Features and Improvements

  • TensorFlow now supports and is compiled with NumPy 2.0 by default. Please see the NumPy 2 release notes and the NumPy 2 migration guide.
    • Note that NumPy's type promotion rules have been changed(See NEP 50for details). This may change the precision at which computations happen, leading either to type errors or to numerical changes to results.
    • Tensorflow will continue to support NumPy 1.26 until 2025, aligning with community standard deprecation timeline here.
  • tf.lite:
    • The LiteRT repo is live (see announcement), which means that in the coming months there will be changes to the development experience for TFLite. The TF Lite Runtime source will be moved later this year, and sometime after that we will start accepting contributions through that repo.
  • SignatureRunner is now supported for models with no signatures.

Bug Fixes and Other Changes

  • tf.data

    • Add optional synchronous argument to map, to specify that the map should run synchronously, as opposed to be parallelizable when options.experimental_optimization.map_parallelization=True. This saves memory compared to setting num_parallel_calls=1.
    • Add optional use_unbounded_threadpool argument to map, to specify that the map should use an unbounded threadpool instead of the default pool that is based on the number of cores on the machine. This can improve throughput for map functions which perform IO or otherwise release the CPU.
    • Add tf.data.experimental.get_model_proto to allow users to peek into the analytical model inside of a dataset iterator.

  • tf.lite

    • Dequantize op supports TensorType_INT4.
      • This change includes per-channel dequantization.
    • Add support for stablehlo.composite.
    • EmbeddingLookup op supports per-channel quantization and TensorType_INT4 values.
    • FullyConnected op supports TensorType_INT16 activation and TensorType_Int4 weight per-channel quantization.

  • tf.tensor_scatter_update, tf.tensor_scatter_add and of other reduce types.

    • Support bad_indices_policy.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Akhil Goel, akhilgoe, Alexander Pivovarov, Amir Samani, Andrew Goodbody, Andrey Portnoy, Anthony Platanios, bernardoArcari, Brett Taylor, buptzyb, Chao, Christian Clauss, Cocoa, Daniil Kutz, Darya Parygina, dependabot[bot], Dimitris Vardoulakis, Dragan Mladjenovic, Elfie Guo, eukub, Faijul Amin, flyingcat, Frédéric Bastien, ganyu.08, Georg Stefan Schmid, Grigory Reznikov, Harsha H S, Harshit Monish, Heiner, Ilia Sergachev, Jan, Jane Liu, Jaroslav Sevcik, Kaixi Hou, Kanvi Khanna, Kristof Maar, Kristóf Maár, LakshmiKalaKadali, Lbertho-Gpsw, lingzhi98, MarcoFalke, Masahiro Hiramori, Mmakevic-Amd, mraunak, Nobuo Tsukamoto, Notheisz57, Olli Lupton, Pearu Peterson, pemeliya, Peyara Nando, Philipp Hack, Phuong Nguyen, Pol Dellaiera, Rahul Batra, Ruturaj Vaidya, sachinmuradi, Sergey Kozub, Shanbin Ke, Sheng Yang, shengyu, Shraiysh, Shu Wang, Surya, sushreebarsa, Swatheesh-Mcw, syzygial, Tai Ly, terryysun, tilakrayal, Tj Xu, Trevor Morris, Tzung-Han Juang, wenchenvincent, wondertx, Xuefei Jiang, Ye Huang, Yimei Sun, Yunlong Liu, Zahid Iqbal, Zhan Lu, Zoranjovanovic-Ns, Zuri Obozuwa

- C++
Published by tensorflow-jenkins over 1 year ago

tensorflow - TensorFlow 2.17.1

Release 2.17.1

Bug Fixes and Other Changes

  • Add necessary header files in the aar library. These are needed if developers build apps with header files unpacked from tflite aar files from maven.
  • Implement Name() for GCSWritableFile to fix the profiler trace viewer cache file generation.
  • Fix cstring.h missing file issue with the Libtensorflow archive.

- C++
Published by tensorflow-jenkins over 1 year ago

tensorflow - TensorFlow 2.18.0-rc2

Release 2.18.0

TensorFlow

Breaking Changes

  • tf.lite

    • C API:
      • An optional, fourth parameter was added TfLiteOperatorCreate as a step forward towards a cleaner API for TfLiteOperator. Function TfLiteOperatorCreate was added recently, in TensorFlow Lite version 2.17.0, released on 7/11/2024, and we do not expect there will be much code using this function yet. Any code breakages can be easily resolved by passing nullptr as the new, 4th parameter.
    • SignatureRunner is now supported for models with no signatures.

  • TensorRT support is disabled in CUDA builds for code health improvement.

  • Hermetic CUDA support is added.

Hermetic CUDA uses a specific downloadable version of CUDA instead of the user’s locally installed CUDA. Bazel will download CUDA, CUDNN and NCCL distributions, and then use CUDA libraries and tools as dependencies in various Bazel targets. This enables more reproducible builds for Google ML projects and supported CUDA versions.

Known Caveats

Major Features and Improvements

  • TensorFlow now supports and is compiled with NumPy 2.0 by default. Please see the NumPy 2 release notes and the NumPy 2 migration guide.
    • Note that NumPy's type promotion rules have been changed(See NEP 50 for details). This may change the precision at which computations happen, leading either to type errors or to numerical changes to results.
    • Tensorflow will continue to support NumPy 1.26 until 2025, aligning with community standard deprecation timeline here.
  • tf.lite:
    • The LiteRT repo is live (see announcement), which means that in the coming months there will be changes to the development experience for TFLite. The TF Lite Runtime source will be moved later this year, and sometime after that we will start accepting contributions through that repo.

Bug Fixes and Other Changes

  • tf.data

    • Add optional synchronous argument to map, to specify that the map should run synchronously, as opposed to be parallelizable when options.experimental_optimization.map_parallelization=True. This saves memory compared to setting num_parallel_calls=1.
    • Add optional use_unbounded_threadpool argument to map, to specify that the map should use an unbounded threadpool instead of the default pool that is based on the number of cores on the machine. This can improve throughput for map functions which perform IO or otherwise release the CPU.
    • Add tf.data.experimental.get_model_proto to allow users to peek into the analytical model inside of a dataset iterator.

  • tf.lite

    • Dequantize op supports TensorType_INT4.
      • This change includes per-channel dequantization.
    • Add support for stablehlo.composite.
    • EmbeddingLookup op supports per-channel quantization and TensorType_INT4 values.
    • FullyConnected op supports TensorType_INT16 activation and TensorType_Int4 weight per-channel quantization.

  • tf.tensor_scatter_update, tf.tensor_scatter_add and of other reduce types.

    • Support bad_indices_policy.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Akhil Goel, akhilgoe, Alexander Pivovarov, Amir Samani, Andrew Goodbody, Andrey Portnoy, Anthony Platanios, bernardoArcari, Brett Taylor, buptzyb, Chao, Christian Clauss, Cocoa, Daniil Kutz, Darya Parygina, dependabot[bot], Dimitris Vardoulakis, Dragan Mladjenovic, Elfie Guo, eukub, Faijul Amin, flyingcat, Frédéric Bastien, ganyu.08, Georg Stefan Schmid, Grigory Reznikov, Harsha H S, Harshit Monish, Heiner, Ilia Sergachev, Jan, Jane Liu, Jaroslav Sevcik, Kaixi Hou, Kanvi Khanna, Kristof Maar, Kristóf Maár, LakshmiKalaKadali, Lbertho-Gpsw, lingzhi98, MarcoFalke, Masahiro Hiramori, Mmakevic-Amd, mraunak, Nobuo Tsukamoto, Notheisz57, Olli Lupton, Pearu Peterson, pemeliya, Peyara Nando, Philipp Hack, Phuong Nguyen, Pol Dellaiera, Rahul Batra, Ruturaj Vaidya, sachinmuradi, Sergey Kozub, Shanbin Ke, Sheng Yang, shengyu, Shraiysh, Shu Wang, Surya, sushreebarsa, Swatheesh-Mcw, syzygial, Tai Ly, terryysun, tilakrayal, Tj Xu, Trevor Morris, Tzung-Han Juang, wenchenvincent, wondertx, Xuefei Jiang, Ye Huang, Yimei Sun, Yunlong Liu, Zahid Iqbal, Zhan Lu, Zoranjovanovic-Ns, Zuri Obozuwa

- C++
Published by tensorflow-jenkins over 1 year ago

tensorflow - TensorFlow 2.18.0-rc1

Release 2.18.0

TensorFlow

Breaking Changes

  • tf.lite

    • C API:
      • An optional, fourth parameter was added TfLiteOperatorCreate as a step forward towards a cleaner API for TfLiteOperator. Function TfLiteOperatorCreate was added recently, in TensorFlow Lite version 2.17.0, released on 7/11/2024, and we do not expect there will be much code using this function yet. Any code breakages can be easily resolved by passing nullptr as the new, 4th parameter.
    • SignatureRunner is now supported for models with no signatures.

  • TensorRT support is disabled in CUDA builds for code health improvement.

  • Hermetic CUDA support is added.

Hermetic CUDA uses a specific downloadable version of CUDA instead of the user’s locally installed CUDA. Bazel will download CUDA, CUDNN and NCCL distributions, and then use CUDA libraries and tools as dependencies in various Bazel targets. This enables more reproducible builds for Google ML projects and supported CUDA versions.

Known Caveats

Major Features and Improvements

  • TensorFlow now supports and is compiled with NumPy 2.0 by default. Please see the NumPy 2 release notes and the NumPy 2 migration guide.
    • Note that NumPy's type promotion rules have been changed(See NEP 50for details). This may change the precision at which computations happen, leading either to type errors or to numerical changes to results.
    • Tensorflow will continue to support NumPy 1.26 until 2025, aligning with community standard deprecation timeline here.
  • tf.lite:
    • The LiteRT repo is live (see announcement), which means that in the coming months there will be changes to the development experience for TFLite. The TF Lite Runtime source will be moved later this year, and sometime after that we will start accepting contributions through that repo.

Bug Fixes and Other Changes

  • tf.data

    • Add optional synchronous argument to map, to specify that the map should run synchronously, as opposed to be parallelizable when options.experimental_optimization.map_parallelization=True. This saves memory compared to setting num_parallel_calls=1.
    • Add optional use_unbounded_threadpool argument to map, to specify that the map should use an unbounded threadpool instead of the default pool that is based on the number of cores on the machine. This can improve throughput for map functions which perform IO or otherwise release the CPU.
    • Add tf.data.experimental.get_model_proto to allow users to peek into the analytical model inside of a dataset iterator.

  • tf.lite

    • Dequantize op supports TensorType_INT4.
      • This change includes per-channel dequantization.
    • Add support for stablehlo.composite.
    • EmbeddingLookup op supports per-channel quantization and TensorType_INT4 values.
    • FullyConnected op supports TensorType_INT16 activation and TensorType_Int4 weight per-channel quantization.

  • tf.tensor_scatter_update, tf.tensor_scatter_add and of other reduce types.

    • Support bad_indices_policy.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Akhil Goel, akhilgoe, Alexander Pivovarov, Amir Samani, Andrew Goodbody, Andrey Portnoy, Anthony Platanios, bernardoArcari, Brett Taylor, buptzyb, Chao, Christian Clauss, Cocoa, Daniil Kutz, Darya Parygina, dependabot[bot], Dimitris Vardoulakis, Dragan Mladjenovic, Elfie Guo, eukub, Faijul Amin, flyingcat, Frédéric Bastien, ganyu.08, Georg Stefan Schmid, Grigory Reznikov, Harsha H S, Harshit Monish, Heiner, Ilia Sergachev, Jan, Jane Liu, Jaroslav Sevcik, Kaixi Hou, Kanvi Khanna, Kristof Maar, Kristóf Maár, LakshmiKalaKadali, Lbertho-Gpsw, lingzhi98, MarcoFalke, Masahiro Hiramori, Mmakevic-Amd, mraunak, Nobuo Tsukamoto, Notheisz57, Olli Lupton, Pearu Peterson, pemeliya, Peyara Nando, Philipp Hack, Phuong Nguyen, Pol Dellaiera, Rahul Batra, Ruturaj Vaidya, sachinmuradi, Sergey Kozub, Shanbin Ke, Sheng Yang, shengyu, Shraiysh, Shu Wang, Surya, sushreebarsa, Swatheesh-Mcw, syzygial, Tai Ly, terryysun, tilakrayal, Tj Xu, Trevor Morris, Tzung-Han Juang, wenchenvincent, wondertx, Xuefei Jiang, Ye Huang, Yimei Sun, Yunlong Liu, Zahid Iqbal, Zhan Lu, Zoranjovanovic-Ns, Zuri Obozuwa

- C++
Published by tensorflow-jenkins over 1 year ago

tensorflow - TensorFlow 2.18.0-rc0

Release 2.18.0

TensorFlow

Breaking Changes

  • tf.lite

    • C API:
      • An optional, fourth parameter was added TfLiteOperatorCreate as a step forward towards a cleaner API for TfLiteOperator. Function TfLiteOperatorCreate was added recently, in TensorFlow Lite version 2.17.0, released on 7/11/2024, and we do not expect there will be much code using this function yet. Any code breakages can be easily resolved by passing nullptr as the new, 4th parameter.
    • SignatureRunner is now supported for models with no signatures.

  • TensorRT support is disabled in CUDA builds for code health improvement.

  • Hermetic CUDA support is added.

Hermetic CUDA uses a specific downloadable version of CUDA instead of the user’s locally installed CUDA. Bazel will download CUDA, CUDNN and NCCL distributions, and then use CUDA libraries and tools as dependencies in various Bazel targets. This enables more reproducible builds for Google ML projects and supported CUDA versions.

Known Caveats

Major Features and Improvements

  • TensorFlow now supports and is compiled with NumPy 2.0 by default. Please see the NumPy 2 release notes and the NumPy 2 migration guide.
    • Note that NumPy's type promotion rules have been changed(See NEP 50for details). This may change the precision at which computations happen, leading either to type errors or to numerical changes to results.
    • Tensorflow will continue to support NumPy 1.26 until 2025, aligning with community standard deprecation timeline here.
  • tf.lite:
    • The LiteRT repo is live (see announcement), which means that in the coming months there will be changes to the development experience for TFLite. The TF Lite Runtime source will be moved later this year, and sometime after that we will start accepting contributions through that repo.

Bug Fixes and Other Changes

  • tf.data

    • Add optional synchronous argument to map, to specify that the map should run synchronously, as opposed to be parallelizable when options.experimental_optimization.map_parallelization=True. This saves memory compared to setting num_parallel_calls=1.
    • Add optional use_unbounded_threadpool argument to map, to specify that the map should use an unbounded threadpool instead of the default pool that is based on the number of cores on the machine. This can improve throughput for map functions which perform IO or otherwise release the CPU.
    • Add tf.data.experimental.get_model_proto to allow users to peek into the analytical model inside of a dataset iterator.

  • tf.lite

    • Dequantize op supports TensorType_INT4.
      • This change includes per-channel dequantization.
    • Add support for stablehlo.composite.
    • EmbeddingLookup op supports per-channel quantization and TensorType_INT4 values.
    • FullyConnected op supports TensorType_INT16 activation and TensorType_Int4 weight per-channel quantization.

  • tf.tensor_scatter_update, tf.tensor_scatter_add and of other reduce types.

    • Support bad_indices_policy.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Akhil Goel, akhilgoe, Alexander Pivovarov, Amir Samani, Andrew Goodbody, Andrey Portnoy, Anthony Platanios, bernardoArcari, Brett Taylor, buptzyb, Chao, Christian Clauss, Cocoa, Daniil Kutz, Darya Parygina, dependabot[bot], Dimitris Vardoulakis, Dragan Mladjenovic, Elfie Guo, eukub, Faijul Amin, flyingcat, Frédéric Bastien, ganyu.08, Georg Stefan Schmid, Grigory Reznikov, Harsha H S, Harshit Monish, Heiner, Ilia Sergachev, Jan, Jane Liu, Jaroslav Sevcik, Kaixi Hou, Kanvi Khanna, Kristof Maar, Kristóf Maár, LakshmiKalaKadali, Lbertho-Gpsw, lingzhi98, MarcoFalke, Masahiro Hiramori, Mmakevic-Amd, mraunak, Nobuo Tsukamoto, Notheisz57, Olli Lupton, Pearu Peterson, pemeliya, Peyara Nando, Philipp Hack, Phuong Nguyen, Pol Dellaiera, Rahul Batra, Ruturaj Vaidya, sachinmuradi, Sergey Kozub, Shanbin Ke, Sheng Yang, shengyu, Shraiysh, Shu Wang, Surya, sushreebarsa, Swatheesh-Mcw, syzygial, Tai Ly, terryysun, tilakrayal, Tj Xu, Trevor Morris, Tzung-Han Juang, wenchenvincent, wondertx, Xuefei Jiang, Ye Huang, Yimei Sun, Yunlong Liu, Zahid Iqbal, Zhan Lu, Zoranjovanovic-Ns, Zuri Obozuwa

- C++
Published by tensorflow-jenkins over 1 year ago

tensorflow - TensorFlow 2.17.0

Release 2.17.0

TensorFlow

Breaking Changes

  • GPU
    • Support for NVIDIA GPUs with compute capability 5.x (Maxwell generation) has been removed from TF binary distributions (Python wheels).

Major Features and Improvements

  • Add is_cpu_target_available, which indicates whether or not TensorFlow was built with support for a given CPU target. This can be useful for skipping target-specific tests if a target is not supported.

  • tf.data

    • Support data.experimental.distribued_save. distribued_save uses tf.data service (https://www.tensorflow.org/api_docs/python/tf/data/experimental/service) to write distributed dataset snapshots. The call is non-blocking and returns without waiting for the snapshot to finish. Setting wait=True to tf.data.Dataset.load allows the snapshots to be read while they are being written.

Bug Fixes and Other Changes

  • GPU
    • Support for NVIDIA GPUs with compute capability 8.9 (e.g. L4 & L40) has been added to TF binary distributions (Python wheels).
  • Replace DebuggerOptions of TensorFlow Quantizer, and migrate to DebuggerConfig of StableHLO Quantizer.
  • Add TensorFlow to StableHLO converter to TensorFlow pip package.
  • TensorRT support: this is the last release supporting TensorRT. It will be removed in the next release.

  • NumPy 2.0 support: TensorFlow is going to support NumPy 2.0 in the next release. It may break some edge cases of TensorFlow API usage.

  • tf.lite

    • Quantization for FullyConnected layer is switched from per-tensor to per-channel scales for dynamic range quantization use case (float32 inputs / outputs and int8 weights). The change enables new quantization schema globally in the converter and inference engine. The new behaviour can be disabled via experimental flag converter._experimental_disable_per_channel_quantization_for_dense_layers = True.
    • C API:
      • The experimental TfLiteRegistrationExternal type has been renamed as TfLiteOperator, and likewise for the corresponding API functions.
    • The Python TF Lite Interpreter bindings now have an option experimental_default_delegate_latest_features to enable all default delegate features.
    • Flatbuffer version update:
      • GetTemporaryPointer() bug fixed.

  • tf.data

    • Add wait to tf.data.Dataset.load. If True, for snapshots written with distributed_save, it reads the snapshot while it is being written. For snapshots written with regular save, it waits for the snapshot until it's finished. The default is False for backward compatibility. Users of distributed_save are recommended to set it to True.

  • tf.tpu.experimental.embedding.TPUEmbeddingV2

    • Add compute_sparse_core_stats for sparse core users to profile the data with this API to get the max_ids and max_unique_ids. These numbers will be needed to configure the sparse core embedding mid level api.
    • Remove the preprocess_features method since that's no longer needed.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Abdulaziz Aloqeely, Ahmad-M-Al-Khateeb, Akhil Goel, akhilgoe, Alexander Pivovarov, Amir Samani, Andrew Goodbody, Andrey Portnoy, Ashiq Imran, Ben Olson, Chao, Chase Riley Roberts, Clemens Giuliani, dependabot[bot], Dimitris Vardoulakis, Dragan Mladjenovic, ekuznetsov139, Elfie Guo, Faijul Amin, Gauri1 Deshpande, Georg Stefan Schmid, guozhong.zhuang, Hao Wu, Haoyu (Daniel), Harsha H S, Harsha Hs, Harshit Monish, Ilia Sergachev, Jane Liu, Jaroslav Sevcik, Jinzhe Zeng, Justin Dhillon, Kaixi Hou, Kanvi Khanna, LakshmiKalaKadali, Learning-To-Play, lingzhi98, Lu Teng, Matt Bahr, Max Ren, Meekail Zain, Mmakevic-Amd, mraunak, neverlva, nhatle, Nicola Ferralis, Olli Lupton, Om Thakkar, orangekame3, ourfor, pateldeev, Pearu Peterson, pemeliya, Peng Sun, Philipp Hack, Pratik Joshi, prrathi, rahulbatra85, Raunak, redwrasse, Robert Kalmar, Robin Zhang, RoboSchmied, Ruturaj Vaidya, sachinmuradi, Shawn Wang, Sheng Yang, Surya, Thibaut Goetghebuer-Planchon, Thomas Preud'Homme, tilakrayal, Tj Xu, Trevor Morris, wenchenvincent, Yimei Sun, zahiqbal, Zhu Jianjiang, Zoranjovanovic-Ns

- C++
Published by tensorflow-jenkins over 1 year ago

tensorflow - TensorFlow 2.17.0-rc1

Release 2.17.0

TensorFlow

Breaking Changes

  • GPU
    • Support for NVIDIA GPUs with compute capability 5.x (Maxwell generation) has been removed from TF binary distributions (Python wheels).

Major Features and Improvements

  • Add is_cpu_target_available, which indicates whether or not TensorFlow was built with support for a given CPU target. This can be useful for skipping target-specific tests if a target is not supported.

  • tf.data

    • Support data.experimental.distribued_save. distribued_save uses tf.data service (https://www.tensorflow.org/api_docs/python/tf/data/experimental/service) to write distributed dataset snapshots. The call is non-blocking and returns without waiting for the snapshot to finish. Setting wait=True to tf.data.Dataset.load allows the snapshots to be read while they are being written.

Bug Fixes and Other Changes

  • GPU
    • Support for NVIDIA GPUs with compute capability 8.9 (e.g. L4 & L40) has been added to TF binary distributions (Python wheels).
  • Replace DebuggerOptions of TensorFlow Quantizer, and migrate to DebuggerConfig of StableHLO Quantizer.
  • Add TensorFlow to StableHLO converter to TensorFlow pip package.
  • TensorRT support: this is the last release supporting TensorRT. It will be removed in the next release.

  • NumPy 2.0 support: TensorFlow is going to support NumPy 2.0 in the next release. It may break some edge cases of TensorFlow API usage.

  • tf.lite

    • Quantization for FullyConnected layer is switched from per-tensor to per-channel scales for dynamic range quantization use case (float32 inputs / outputs and int8 weights). The change enables new quantization schema globally in the converter and inference engine. The new behaviour can be disabled via experimental flag converter._experimental_disable_per_channel_quantization_for_dense_layers = True.
    • C API:
      • The experimental TfLiteRegistrationExternal type has been renamed as TfLiteOperator, and likewise for the corresponding API functions.
    • The Python TF Lite Interpreter bindings now have an option experimental_default_delegate_latest_features to enable all default delegate features.
    • Flatbuffer version update:
      • GetTemporaryPointer() bug fixed.

  • tf.data

    • Add wait to tf.data.Dataset.load. If True, for snapshots written with distributed_save, it reads the snapshot while it is being written. For snapshots written with regular save, it waits for the snapshot until it's finished. The default is False for backward compatibility. Users of distributed_save are recommended to set it to True.

  • tf.tpu.experimental.embedding.TPUEmbeddingV2

    • Add compute_sparse_core_stats for sparse core users to profile the data with this API to get the max_ids and max_unique_ids. These numbers will be needed to configure the sparse core embedding mid level api.
    • Remove the preprocess_features method since that's no longer needed.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Abdulaziz Aloqeely, Ahmad-M-Al-Khateeb, Akhil Goel, akhilgoe, Alexander Pivovarov, Amir Samani, Andrew Goodbody, Andrey Portnoy, Ashiq Imran, Ben Olson, Chao, Chase Riley Roberts, Clemens Giuliani, dependabot[bot], Dimitris Vardoulakis, Dragan Mladjenovic, ekuznetsov139, Elfie Guo, Faijul Amin, Gauri1 Deshpande, Georg Stefan Schmid, guozhong.zhuang, Hao Wu, Haoyu (Daniel), Harsha H S, Harsha Hs, Harshit Monish, Ilia Sergachev, Jane Liu, Jaroslav Sevcik, Jinzhe Zeng, Justin Dhillon, Kaixi Hou, Kanvi Khanna, LakshmiKalaKadali, Learning-To-Play, lingzhi98, Lu Teng, Matt Bahr, Max Ren, Meekail Zain, Mmakevic-Amd, mraunak, neverlva, nhatle, Nicola Ferralis, Olli Lupton, Om Thakkar, orangekame3, ourfor, pateldeev, Pearu Peterson, pemeliya, Peng Sun, Philipp Hack, Pratik Joshi, prrathi, rahulbatra85, Raunak, redwrasse, Robert Kalmar, Robin Zhang, RoboSchmied, Ruturaj Vaidya, sachinmuradi, Shawn Wang, Sheng Yang, Surya, Thibaut Goetghebuer-Planchon, Thomas Preud'Homme, tilakrayal, Tj Xu, Trevor Morris, wenchenvincent, Yimei Sun, zahiqbal, Zhu Jianjiang, Zoranjovanovic-Ns

- C++
Published by tensorflow-jenkins over 1 year ago

tensorflow - TensorFlow 2.16.2

Release 2.16.2

Bug Fixes and Other Changes

  • Fixed: Incorrect dependency metadata in TensorFlow Python packages causing installation failures with certain package managers such as Poetry.

- C++
Published by tensorflow-jenkins over 1 year ago

tensorflow - TensorFlow 2.17.0-rc0

Release 2.17.0

TensorFlow

Breaking Changes

  • GPU
    • Support for NVIDIA GPUs with compute capability 5.x (Maxwell generation) has been removed from TF binary distributions (Python wheels).

Major Features and Improvements

  • Add is_cpu_target_available, which indicates whether or not TensorFlow was built with support for a given CPU target. This can be useful for skipping target-specific tests if a target is not supported.

  • tf.data

    • Support data.experimental.distribued_save. distribued_save uses tf.data service (https://www.tensorflow.org/api_docs/python/tf/data/experimental/service) to write distributed dataset snapshots. The call is non-blocking and returns without waiting for the snapshot to finish. Setting wait=True to tf.data.Dataset.load allows the snapshots to be read while they are being written.

Bug Fixes and Other Changes

  • GPU
    • Support for NVIDIA GPUs with compute capability 8.9 (e.g. L4 & L40) has been added to TF binary distributions (Python wheels).
  • Replace DebuggerOptions of TensorFlow Quantizer, and migrate to DebuggerConfig of StableHLO Quantizer.
  • Add TensorFlow to StableHLO converter to TensorFlow pip package.
  • TensorRT support: this is the last release supporting TensorRT. It will be removed in the next release.

  • NumPy 2.0 support: TensorFlow is going to support NumPy 2.0 in the next release. It may break some edge cases of TensorFlow API usage.

  • tf.lite

    • Quantization for FullyConnected layer is switched from per-tensor to per-channel scales for dynamic range quantization use case (float32 inputs / outputs and int8 weights). The change enables new quantization schema globally in the converter and inference engine. The new behaviour can be disabled via experimental flag converter._experimental_disable_per_channel_quantization_for_dense_layers = True.
    • C API:
      • The experimental TfLiteRegistrationExternal type has been renamed as TfLiteOperator, and likewise for the corresponding API functions.
    • The Python TF Lite Interpreter bindings now have an option experimental_default_delegate_latest_features to enable all default delegate features.
    • Flatbuffer version update:
      • GetTemporaryPointer() bug fixed.

  • tf.data

    • Add wait to tf.data.Dataset.load. If True, for snapshots written with distributed_save, it reads the snapshot while it is being written. For snapshots written with regular save, it waits for the snapshot until it's finished. The default is False for backward compatibility. Users of distributed_save are recommended to set it to True.

  • tf.tpu.experimental.embedding.TPUEmbeddingV2

    • Add compute_sparse_core_stats for sparse core users to profile the data with this API to get the max_ids and max_unique_ids. These numbers will be needed to configure the sparse core embedding mid level api.
    • Remove the preprocess_features method since that's no longer needed.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Abdulaziz Aloqeely, Ahmad-M-Al-Khateeb, Akhil Goel, akhilgoe, Alexander Pivovarov, Amir Samani, Andrew Goodbody, Andrey Portnoy, Ashiq Imran, Ben Olson, Chao, Chase Riley Roberts, Clemens Giuliani, dependabot[bot], Dimitris Vardoulakis, Dragan Mladjenovic, ekuznetsov139, Elfie Guo, Faijul Amin, Gauri1 Deshpande, Georg Stefan Schmid, guozhong.zhuang, Hao Wu, Haoyu (Daniel), Harsha H S, Harsha Hs, Harshit Monish, Ilia Sergachev, Jane Liu, Jaroslav Sevcik, Jinzhe Zeng, Justin Dhillon, Kaixi Hou, Kanvi Khanna, LakshmiKalaKadali, Learning-To-Play, lingzhi98, Lu Teng, Matt Bahr, Max Ren, Meekail Zain, Mmakevic-Amd, mraunak, neverlva, nhatle, Nicola Ferralis, Olli Lupton, Om Thakkar, orangekame3, ourfor, pateldeev, Pearu Peterson, pemeliya, Peng Sun, Philipp Hack, Pratik Joshi, prrathi, rahulbatra85, Raunak, redwrasse, Robert Kalmar, Robin Zhang, RoboSchmied, Ruturaj Vaidya, sachinmuradi, Shawn Wang, Sheng Yang, Surya, Thibaut Goetghebuer-Planchon, Thomas Preud'Homme, tilakrayal, Tj Xu, Trevor Morris, wenchenvincent, Yimei Sun, zahiqbal, Zhu Jianjiang, Zoranjovanovic-Ns

- C++
Published by tensorflow-jenkins over 1 year ago

tensorflow - TensorFlow 2.15.1

Release 2.15.1

Bug Fixes and Other Changes

  • ml_dtypes runtime dependency is updated to 0.3.1 to fix package conflict issues

- C++
Published by tensorflow-jenkins almost 2 years ago

tensorflow - TensorFlow 2.16.1

Release 2.16.1

TensorFlow

  • TensorFlow Windows Build:
    • Clang is now the default compiler to build TensorFlow CPU wheels on the Windows Platform starting with this release. The currently supported version is LLVM/clang 17. The official Wheels-published on PyPI will be based on Clang; however, users retain the option to build wheels using the MSVC compiler following the steps mentioned in https://www.tensorflow.org/install/source_windows as has been the case before
    • TensorFlow 2.16 will be released as TF 2.16.1 (instead of 2.16.0). The patch release will be done as 2.16.2 during the next release cycle.

Breaking Changes

  • tf.summary.trace_on now takes a profiler_outdir argument. This must be set if profiler arg is set to True.

    • tf.summary.trace_export's profiler_outdir arg is now a no-op. Enabling the profiler now requires setting profiler_outdir in trace_on.

  • tf.estimator

    • The tf.estimator API is removed.
    • To continue using tf.estimator, you will need to use TF 2.15 or an earlier version.

  • Keras 3.0 will be the default Keras version. You may need to update your script to use Keras 3.0.

  • Please refer to the new Keras documentation for Keras 3.0 (https://keras.io/keras_3).

  • To continue using Keras 2.0, do the following.

    1. Install tf-keras via pip install tf-keras~=2.16
1.  To switch `tf.keras` to use Keras 2 (`tf-keras`), set the environment variable `TF_USE_LEGACY_KERAS=1` directly or in your python program with `import os;os.environ["TF_USE_LEGACY_KERAS"]="1"`. Please note that this will set it for all packages in your Python runtime program

1.  Change the keras import: replace `import tensorflow.keras as keras` or `import keras` with `import tf_keras as keras`. Update any `tf.keras` references to `keras`.
  • Apple Silicon users: If you previously installed TensorFlow using pip install tensorflow-macos, please update your installation method. Use pip install tensorflow from now on.
  • Mac x86 users: Mac x86 builds are being deprecated and will no longer be released as a Pip package from TF 2.17 onwards.

Known Caveats

  • Full aarch64 Linux and Arm64 macOS wheels are now published to the tensorflow pypi repository and no longer redirect to a separate package.

Major Features and Improvements

  • Support for Python 3.12 has been added.
  • tensorflow-tpu package is now available for easier TPU based installs.
  • TensorFlow pip packages are now built with CUDA 12.3 and cuDNN 8.9.7
  • Added experimental support for float16 auto-mixed precision using the new AMX-FP16 instruction set on X86 CPUs.

Bug Fixes and Other Changes

  • tf.lite

    • Added support for stablehlo.gather.
    • Added support for stablehlo.add.
    • Added support for stablehlo.multiply.
    • Added support for stablehlo.maximum.
    • Added support for stablehlo.minimum.
    • Added boolean parameter support for tfl.gather_nd.
    • C API:
      • New API functions:
        • tensorflow/lite/c/c_api_experimental.h:
          • TfLiteInterpreterGetVariableTensorCount
          • TfLiteInterpreterGetVariableTensor
          • TfLiteInterpreterGetBufferHandle
          • TfLiteInterpreterSetBufferHandle
        • tensorflow/lite/c/c_api_opaque.h:
          • TfLiteOpaqueTensorSetAllocationTypeToDynamic
      • API functions promoted from experimental to stable:
        • tensorflow/lite/c/c_api.h:
          • TfLiteInterpreterOptionsEnableCancellation
          • TfLiteInterpreterCancel
    • C++ API:
      • New virtual methods in the tflite::SimpleDelegateInterface class in tensorflow/lite/delegates/utils/simple_delegate.h, and likewise in the tflite::SimpleOpaqueDelegateInterface class in tensorflow/lite/delegates/utils/simple_opaque_delegate.h:
        • CopyFromBufferHandle
        • CopyToBufferHandle
        • FreeBufferHandle

  • tf.train.CheckpointOptions and tf.saved_model.SaveOptions

    • These now take in a new argument called experimental_sharding_callback. This is a callback function wrapper that will be executed to determine how tensors will be split into shards when the saver writes the checkpoint shards to disk. tf.train.experimental.ShardByTaskPolicy is the default sharding behavior, but tf.train.experimental.MaxShardSizePolicy can be used to shard the checkpoint with a maximum shard file size. Users with advanced use cases can also write their own custom tf.train.experimental.ShardingCallbacks.

  • tf.train.CheckpointOptions

    • Added experimental_skip_slot_variables (a boolean option) to skip restoring of optimizer slot variables in a checkpoint.

  • tf.saved_model.SaveOptions

    • SaveOptions now takes a new argument called experimental_debug_stripper. When enabled, this strips the debug nodes from both the node defs and the function defs of the graph. Note that this currently only strips the Assert nodes from the graph and converts them into NoOps instead.

Keras

  • keras.layers.experimental.DynamicEmbedding
    • Added DynamicEmbedding Keras layer
    • Added 'UpdateEmbeddingCallback`
    • DynamicEmbedding layer allows for the continuous updating of the vocabulary and embeddings during the training process. This layer maintains a hash table to track the most up-to-date vocabulary based on the inputs received by the layer and the eviction policy. When this layer is used with an UpdateEmbeddingCallback, which is a time-based callback, the vocabulary lookup tensor is updated at the time interval set in the UpdateEmbeddingCallback based on the most up-to-date vocabulary hash table maintained by the layer. If this layer is not used in conjunction with UpdateEmbeddingCallback the behavior of the layer would be same as keras.layers.Embedding.
  • keras.optimizers.Adam
    • Added the option to set adaptive epsilon to match implementations with Jax and PyTorch equivalents.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aakar Dwivedi, Akhil Goel, Alexander Grund, Alexander Pivovarov, Andrew Goodbody, Andrey Portnoy, Aneta Kaczyńska, AnetaKaczynska, ArkadebMisra, Ashiq Imran, Ayan Moitra, Ben Barsdell, Ben Creech, Benedikt Lorch, Bhavani Subramanian, Bianca Van Schaik, Chao, Chase Riley Roberts, Connor Flanagan, David Hall, David Svantesson, David Svantesson-Yeung, dependabot[bot], Dr. Christoph Mittendorf, Dragan Mladjenovic, ekuznetsov139, Eli Kobrin, Eugene Kuznetsov, Faijul Amin, Frédéric Bastien, fsx950223, gaoyiyeah, Gauri1 Deshpande, Gautam, Giulio C.N, guozhong.zhuang, Harshit Monish, James Hilliard, Jane Liu, Jaroslav Sevcik, jeffhataws, Jerome Massot, Jerry Ge, jglaser, jmaksymc, Kaixi Hou, kamaljeeti, Kamil Magierski, Koan-Sin Tan, lingzhi98, looi, Mahmoud Abuzaina, Malik Shahzad Muzaffar, Meekail Zain, mraunak, Neil Girdhar, Olli Lupton, Om Thakkar, Paul Strawder, Pavel Emeliyanenko, Pearu Peterson, pemeliya, Philipp Hack, Pierluigi Urru, Pratik Joshi, radekzc, Rafik Saliev, Ragu, Rahul Batra, rahulbatra85, Raunak, redwrasse, Rodrigo Gomes, ronaghy, Sachin Muradi, Shanbin Ke, shawnwang18, Sheng Yang, Shivam Mishra, Shu Wang, Strawder, Paul, Surya, sushreebarsa, Tai Ly, talyz, Thibaut Goetghebuer-Planchon, Tj Xu, Tom Allsop, Trevor Morris, Varghese, Jojimon, weihanmines, wenchenvincent, Wenjie Zheng, Who Who Who, Yasir Ashfaq, yasiribmcon, Yoshio Soma, Yuanqiang Liu, Yuriy Chernyshov

- C++
Published by tensorflow-jenkins almost 2 years ago

tensorflow - TensorFlow 2.16.0-rc0

Release 2.16.0

TensorFlow

  • TensorFlow Windows Build:
    • Clang is now the default compiler to build TensorFlow CPU wheels on the Windows Platform starting with this release. The currently supported version is LLVM/clang 17. The official Wheels-published on PyPI will be based on Clang; however, users retain the option to build wheels using the MSVC compiler following the steps mentioned in https://www.tensorflow.org/install/source_windows as has been the case before

Breaking Changes

  • tf.summary.trace_on now takes a profiler_outdir argument. This must be set if profiler arg is set to True.

    • tf.summary.trace_export's profiler_outdir arg is now a no-op. Enabling the profiler now requires setting profiler_outdir in trace_on.

  • tf.estimator

    • The tf.estimator API is removed.
    • To continue using tf.estimator, you will need to use TF 2.15 or an earlier version.

  • Keras 3.0 will be the default Keras version. You may need to update your script to use Keras 3.0.

  • Please refer to the new Keras documentation for Keras 3.0 (https://keras.io/keras_3).

  • To continue using Keras 2.0, do the following.

    1. Install tf-keras via pip install tf-keras~=2.16
1.  To switch tf.keras to use Keras 2 (tf-keras), set the environment variable TF_USE_LEGACY_KERAS=1 directly or in your python program by import os;os.environ["TF_USE_LEGACY_KERAS"]=1. Please note that this will set it for all packages in your Python runtime program
  • 1. Change import of keras from tensorflow as follows
  • import tensorflow.keras as keras and import keras to import tf_keras as keras
  • Apple Silicon users: If you previously installed TensorFlow using pip install tensorflow-macos, please update your installation method. Use pip install tensorflow from now on. Starting with TF 2.17, the tensorflow-macos package will no longer receive updates.

Known Caveats

  • Full aarch64 Linux and Arm64 macOS wheels are now published to the tensorflow pypi repository and no longer redirect to a separate package.

Major Features and Improvements

  • Support for Python 3.12 has been added.
  • tensorflow-tpu package is now available for easier TPU based installs.
  • TensorFlow pip packages are now built with CUDA 12.3 and cuDNN 8.9.7

Bug Fixes and Other Changes

  • tf.lite

    • Added support for stablehlo.gather.
    • Added support for stablehlo.add.
    • Added support for stablehlo.multiply.
    • Added support for stablehlo.maximum.
    • Added support for stablehlo.minimum.
    • Added boolean parameter support for tfl.gather_nd.

  • tf.train.CheckpointOptions and tf.saved_model.SaveOptions

    • These now take in a new argument called experimental_sharding_callback. This is a callback function wrapper that will be executed to determine how tensors will be split into shards when the saver writes the checkpoint shards to disk. tf.train.experimental.ShardByTaskPolicy is the default sharding behavior, but tf.train.experimental.MaxShardSizePolicy can be used to shard the checkpoint with a maximum shard file size. Users with advanced use cases can also write their own custom tf.train.experimental.ShardingCallbacks.

  • tf.train.CheckpointOptions

    • Added experimental_skip_slot_variables (a boolean option) to skip restoring of optimizer slot variables in a checkpoint.

  • tf.saved_model.SaveOptions

    • SaveOptions now takes a new argument called experimental_debug_stripper. When enabled, this strips the debug nodes from both the node defs and the function defs of the graph. Note that this currently only strips the Assert nodes from the graph and converts them into NoOps instead.

Keras

  • keras.layers.experimental.DynamicEmbedding
    • Added DynamicEmbedding Keras layer
    • Added 'UpdateEmbeddingCallback`
    • DynamicEmbedding layer allows for the continuous updating of the vocabulary and embeddings during the training process. This layer maintains a hash table to track the most up-to-date vocabulary based on the inputs received by the layer and the eviction policy. When this layer is used with an UpdateEmbeddingCallback, which is a time-based callback, the vocabulary lookup tensor is updated at the time interval set in the UpdateEmbeddingCallback based on the most up-to-date vocabulary hash table maintained by the layer. If this layer is not used in conjunction with UpdateEmbeddingCallback the behavior of the layer would be same as keras.layers.Embedding.
  • keras.optimizers.Adam
    • Added the option to set adaptive epsilon to match implementations with Jax and PyTorch equivalents.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aakar Dwivedi, Akhil Goel, Alexander Grund, Alexander Pivovarov, Andrew Goodbody, Andrey Portnoy, Aneta Kaczyńska, AnetaKaczynska, ArkadebMisra, Ashiq Imran, Ayan Moitra, Ben Barsdell, Ben Creech, Benedikt Lorch, Bhavani Subramanian, Bianca Van Schaik, Chao, Chase Riley Roberts, Connor Flanagan, David Hall, David Svantesson, David Svantesson-Yeung, dependabot[bot], Dr. Christoph Mittendorf, Dragan Mladjenovic, ekuznetsov139, Eli Kobrin, Eugene Kuznetsov, Faijul Amin, Frédéric Bastien, fsx950223, gaoyiyeah, Gauri1 Deshpande, Gautam, Giulio C.N, guozhong.zhuang, Harshit Monish, James Hilliard, Jane Liu, Jaroslav Sevcik, jeffhataws, Jerome Massot, Jerry Ge, jglaser, jmaksymc, Kaixi Hou, kamaljeeti, Kamil Magierski, Koan-Sin Tan, lingzhi98, looi, Mahmoud Abuzaina, Malik Shahzad Muzaffar, Meekail Zain, mraunak, Neil Girdhar, Olli Lupton, Om Thakkar, Paul Strawder, Pavel Emeliyanenko, Pearu Peterson, pemeliya, Philipp Hack, Pierluigi Urru, Pratik Joshi, radekzc, Rafik Saliev, Ragu, Rahul Batra, rahulbatra85, Raunak, redwrasse, Rodrigo Gomes, ronaghy, Sachin Muradi, Shanbin Ke, shawnwang18, Sheng Yang, Shivam Mishra, Shu Wang, Strawder, Paul, Surya, sushreebarsa, Tai Ly, talyz, Thibaut Goetghebuer-Planchon, Tj Xu, Tom Allsop, Trevor Morris, Varghese, Jojimon, weihanmines, wenchenvincent, Wenjie Zheng, Who Who Who, Yasir Ashfaq, yasiribmcon, Yoshio Soma, Yuanqiang Liu, Yuriy Chernyshov

- C++
Published by tensorflow-jenkins almost 2 years ago

tensorflow - TensorFlow 2.15.0

Release 2.15.0

TensorFlow

Breaking Changes

  • tf.types.experimental.GenericFunction has been renamed to tf.types.experimental.PolymorphicFunction.

Major Features and Improvements

  • oneDNN CPU performance optimizations Windows x64 & x86.

    • Windows x64 & x86 packages:
      • oneDNN optimizations are enabled by default on X86 CPUs
    • To explicitly enable or disable oneDNN optimizations, set the environment variable TF_ENABLE_ONEDNN_OPTS to 1 (enable) or 0 (disable) before running TensorFlow. To fall back to default settings, unset the environment variable.
    • oneDNN optimizations can yield slightly different numerical results compared to when oneDNN optimizations are disabled due to floating-point round-off errors from different computation approaches and orders.
    • To verify if oneDNN optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

  • Making the tf.function type system fully available:

    • tf.types.experimental.TraceType now allows custom tf.function inputs to declare Tensor decomposition and type casting support.
    • Introducing tf.types.experimental.FunctionType as the comprehensive representation of the signature of tf.function callables. It can be accessed through the function_type property of tf.functions and ConcreteFunctions. See the tf.types.experimental.FunctionType documentation for more details.

  • Introducing tf.types.experimental.AtomicFunction as the fastest way to perform TF computations in Python.

    • Can be accessed through inference_fn property of ConcreteFunctions
    • Does not support gradients.
    • See tf.types.experimental.AtomicFunction documentation for how to call and use it.

  • tf.data:

    • Moved option warm_start from tf.data.experimental.OptimizationOptions to tf.data.Options.

  • tf.lite:

    • sub_op and mul_op support broadcasting up to 6 dimensions.
    • The tflite::SignatureRunner class, which provides support for named parameters and for multiple named computations within a single TF Lite model, is no longer considered experimental. Likewise for the following signature-related methods of tflite::Interpreter:
      • tflite::Interpreter::GetSignatureRunner
      • tflite::Interpreter::signature_keys
      • tflite::Interpreter::signature_inputs
      • tflite::Interpreter::signature_outputs
      • tflite::Interpreter::input_tensor_by_signature
      • tflite::Interpreter::output_tensor_by_signature
    • Similarly, the following signature runner functions in the TF Lite C API are no longer considered experimental:
      • TfLiteInterpreterGetSignatureCount
      • TfLiteInterpreterGetSignatureKey
      • TfLiteInterpreterGetSignatureRunner
      • TfLiteSignatureRunnerAllocateTensors
      • TfLiteSignatureRunnerGetInputCount
      • TfLiteSignatureRunnerGetInputName
      • TfLiteSignatureRunnerGetInputTensor
      • TfLiteSignatureRunnerGetOutputCount
      • TfLiteSignatureRunnerGetOutputName
      • TfLiteSignatureRunnerGetOutputTensor
      • TfLiteSignatureRunnerInvoke
      • TfLiteSignatureRunnerResizeInputTensor
    • New C API function TfLiteExtensionApisVersion added to tensorflow/lite/c/c_api.h.
    • Add int8 and int16x8 support for RSQRT operator

  • Android NDK r25 is supported.

Bug Fixes and Other Changes

  • Add TensorFlow Quantizer to TensorFlow pip package.

  • tf.sparse.segment_sum tf.sparse.segment_mean tf.sparse.segment_sqrt_n SparseSegmentSum/Mean/SqrtN[WithNumSegments]

    • Added sparse_gradient option (default=false) that makes the gradient of these functions/ops sparse (IndexedSlices) instead of dense (Tensor), using new SparseSegmentSum/Mean/SqrtNGradV2 ops.

  • tf.nn.embedding_lookup_sparse

    • Optimized this function for some cases by fusing internal operations.

  • tf.saved_model.SaveOptions

    • Provided a new experimental_skip_saver argument which, if specified, will suppress the addition of SavedModel-native save and restore ops to the SavedModel, for cases where users already build custom save/restore ops and checkpoint formats for the model being saved, and the creation of the SavedModel-native save/restore ops simply cause longer model serialization times.

  • Add ops to tensorflow.raw_ops that were missing.

  • tf.CheckpointOptions

    • It now takes in a new argument called experimental_write_callbacks. These are callbacks that will be executed after a saving event finishes writing the checkpoint file.

  • Add an option disable_eager_executer_streaming_enqueue to tensorflow.ConfigProto.Experimental to control the eager runtime's behavior around parallel remote function invocations; when set to True, the eager runtime will be allowed to execute multiple function invocations in parallel.

  • tf.constant_initializer

    • It now takes a new argument called support_partition. If True, constant_initializers can create sharded variables. This is disabled by default, similar to existing behavior.

  • tf.lite

    • Added support for stablehlo.scatter.

  • tf.estimator

    • The tf.estimator API removal is in progress and will be targeted for the 2.16 release.

Keras

  • This will be the final release before the launch of Keras 3.0, when Keras will become multi-backend. For the compatibility page and other info, please see: https://github.com/keras-team/keras-core

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aiden Grossman, Akash Patel, Akhil Goel, Alexander Pivovarov, Andrew Goodbody, Ayan Moitra, Ben Barsdell, Ben Olson, Bhavani Subramanian, Boian Petkantchin, Bruce Lai, Chao Chen, Christian Steinmeyer, cjflan, David Korczynski, Donghak Park, Dragan Mladjenovic, Eli Kobrin, Fadi Arafeh, Feiyue Chen, Frédéric Bastien, guozhong.zhuang, halseycamilla, Harshavardhan Bellamkonda, James Ward, jameshollyer, Jane Liu, johnnkp, jswag180, justkw, Kanvi Khanna, Keith Smiley, Koan-Sin Tan, Kulin Seth, Kun-Lu, kushanam, Lu Teng, mdfaijul, Mehdi Drissi, mgokulkrish, mraunak, Mustafa Uzun, Namrata Bhave, Pavel Emeliyanenko, pemeliya, Peng Sun, Philipp Hack, Pratik Joshi, Rahul Batra, Raunak, redwrasse, Saoirse Stewart, SaoirseARM, seanshpark, Shanbin Ke, Spenser Bauman, Surya, sushreebarsa, Tai Ly, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, Tj Xu, Vladislav, weihanmines, Wen Chen, wenchenvincent, wenscarl, William Muir, Zhoulong, Jiang

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.14.1

Release 2.14.1

Security

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.15.0-rc1

Release 2.15.0

TensorFlow

Breaking Changes

  • tf.types.experimental.GenericFunction has been renamed to tf.types.experimental.PolymorphicFunction.

Known Caveats

Major Features and Improvements

  • oneDNN CPU performance optimizations Windows x64 & x86.

    • Windows x64 & x86 packages:
      • oneDNN optimizations are enabled by default on X86 CPUs
    • To explicitly enable or disable oneDNN optimizations, set the environment variable TF_ENABLE_ONEDNN_OPTS to 1 (enable) or 0 (disable) before running TensorFlow. To fall back to default settings, unset the environment variable.
    • oneDNN optimizations can yield slightly different numerical results compared to when oneDNN optimizations are disabled due to floating-point round-off errors from different computation approaches and orders.
    • To verify if oneDNN optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

  • Making the tf.function type system fully available:

    • tf.types.experimental.TraceType now allows custom tf.function inputs to declare Tensor decomposition and type casting support.
    • Introducing tf.types.experimental.FunctionType as the comprehensive representation of the signature of tf.function callables. It can be accessed through the function_type property of tf.functions and ConcreteFunctions. See the tf.types.experimental.FunctionType documentation for more details.

  • Introducing tf.types.experimental.AtomicFunction as the fastest way to perform TF computations in Python.

    • Can be accessed through inference_fn property of ConcreteFunctions
    • Does not support gradients.
    • See tf.types.experimental.AtomicFunction documentation for how to call and use it.

  • tf.data:

    • Moved option warm_start from tf.data.experimental.OptimizationOptions to tf.data.Options.

  • tf.lite:

    • sub_op and mul_op support broadcasting up to 6 dimensions.
    • The tflite::SignatureRunner class, which provides support for named parameters and for multiple named computations within a single TF Lite model, is no longer considered experimental. Likewise for the following signature-related methods of tflite::Interpreter:
      • tflite::Interpreter::GetSignatureRunner
      • tflite::Interpreter::signature_keys
      • tflite::Interpreter::signature_inputs
      • tflite::Interpreter::signature_outputs
      • tflite::Interpreter::input_tensor_by_signature
      • tflite::Interpreter::output_tensor_by_signature
    • Similarly, the following signature runner functions in the TF Lite C API are no longer considered experimental:
      • TfLiteInterpreterGetSignatureCount
      • TfLiteInterpreterGetSignatureKey
      • TfLiteInterpreterGetSignatureRunner
      • TfLiteSignatureRunnerAllocateTensors
      • TfLiteSignatureRunnerGetInputCount
      • TfLiteSignatureRunnerGetInputName
      • TfLiteSignatureRunnerGetInputTensor
      • TfLiteSignatureRunnerGetOutputCount
      • TfLiteSignatureRunnerGetOutputName
      • TfLiteSignatureRunnerGetOutputTensor
      • TfLiteSignatureRunnerInvoke
      • TfLiteSignatureRunnerResizeInputTensor
    • New C API function TfLiteExtensionApisVersion added to tensorflow/lite/c/c_api.h.
    • Add int8 and int16x8 support for RSQRT operator

  • Android NDK r25 is supported.

Bug Fixes and Other Changes

  • Add TensorFlow Quantizer to TensorFlow pip package.

  • tf.sparse.segment_sum tf.sparse.segment_mean tf.sparse.segment_sqrt_n SparseSegmentSum/Mean/SqrtN[WithNumSegments]

    • Added sparse_gradient option (default=false) that makes the gradient of these functions/ops sparse (IndexedSlices) instead of dense (Tensor), using new SparseSegmentSum/Mean/SqrtNGradV2 ops.

  • tf.nn.embedding_lookup_sparse

    • Optimized this function for some cases by fusing internal operations.

  • tf.saved_model.SaveOptions

    • Provided a new experimental_skip_saver argument which, if specified, will suppress the addition of SavedModel-native save and restore ops to the SavedModel, for cases where users already build custom save/restore ops and checkpoint formats for the model being saved, and the creation of the SavedModel-native save/restore ops simply cause longer model serialization times.

  • Add ops to tensorflow.raw_ops that were missing.

  • tf.CheckpointOptions

    • It now takes in a new argument called experimental_write_callbacks. These are callbacks that will be executed after a saving event finishes writing the checkpoint file.

  • Add an option disable_eager_executer_streaming_enqueue to tensorflow.ConfigProto.Experimental to control the eager runtime's behavior around parallel remote function invocations; when set to True, the eager runtime will be allowed to execute multiple function invocations in parallel.

  • tf.constant_initializer

    • It now takes a new argument called support_partition. If True, constant_initializers can create sharded variables. This is disabled by default, similar to existing behavior.

  • tf.lite

    • Added support for stablehlo.scatter.

  • tf.estimator

    • The tf.estimator API removal is in progress and will be targeted for the 2.16 release.

Keras

  • This will be the final release before the launch of Keras 3.0, when Keras will become multi-backend. For the compatibility page and other info, please see: https://github.com/keras-team/keras-core

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aiden Grossman, Akash Patel, Akhil Goel, Alexander Pivovarov, Andrew Goodbody, Ayan Moitra, Ben Barsdell, Ben Olson, Bhavani Subramanian, Boian Petkantchin, Bruce Lai, Chao Chen, Christian Steinmeyer, cjflan, David Korczynski, Donghak Park, Dragan Mladjenovic, Eli Kobrin, Fadi Arafeh, Feiyue Chen, Frédéric Bastien, guozhong.zhuang, halseycamilla, Harshavardhan Bellamkonda, James Ward, jameshollyer, Jane Liu, johnnkp, jswag180, justkw, Kanvi Khanna, Keith Smiley, Koan-Sin Tan, Kulin Seth, Kun-Lu, kushanam, Lu Teng, mdfaijul, Mehdi Drissi, mgokulkrish, mraunak, Mustafa Uzun, Namrata Bhave, Pavel Emeliyanenko, pemeliya, Peng Sun, Philipp Hack, Pratik Joshi, Rahul Batra, Raunak, redwrasse, Saoirse Stewart, SaoirseARM, seanshpark, Shanbin Ke, Spenser Bauman, Surya, sushreebarsa, Tai Ly, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, Tj Xu, Vladislav, weihanmines, Wen Chen, wenchenvincent, wenscarl, William Muir, Zhoulong, Jiang

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.15.0-rc0

Release 2.15.0

TensorFlow

Breaking Changes

  • tf.types.experimental.GenericFunction has been renamed to tf.types.experimental.PolymorphicFunction.

Major Features and Improvements

  • oneDNN CPU performance optimizations Windows x64 & x86.

    • Windows x64 & x86 packages:
      • oneDNN optimizations are enabled by default on X86 CPUs
    • To explicitly enable or disable oneDNN optimizations, set the environment variable TF_ENABLE_ONEDNN_OPTS to 1 (enable) or 0 (disable) before running TensorFlow. To fall back to default settings, unset the environment variable.
    • oneDNN optimizations can yield slightly different numerical results compared to when oneDNN optimizations are disabled due to floating-point round-off errors from different computation approaches and orders.
    • To verify if oneDNN optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

  • Making the tf.function type system fully available:

    • tf.types.experimental.TraceType now allows custom tf.function inputs to declare Tensor decomposition and type casting support.
    • Introducing tf.types.experimental.FunctionType as the comprehensive representation of the signature of tf.function callables. It can be accessed through the function_type property of tf.functions and ConcreteFunctions. See the tf.types.experimental.FunctionType documentation for more details.

  • Introducing tf.types.experimental.AtomicFunction as the fastest way to perform TF computations in Python.

    • Can be accessed through inference_fn property of ConcreteFunctions
    • Does not support gradients.
    • See tf.types.experimental.AtomicFunction documentation for how to call and use it.

  • tf.data:

    • Moved option warm_start from tf.data.experimental.OptimizationOptions to tf.data.Options.

  • tf.lite:

    • sub_op and mul_op support broadcasting up to 6 dimensions.
    • The tflite::SignatureRunner class, which provides support for named parameters and for multiple named computations within a single TF Lite model, is no longer considered experimental. Likewise for the following signature-related methods of tflite::Interpreter:
      • tflite::Interpreter::GetSignatureRunner
      • tflite::Interpreter::signature_keys
      • tflite::Interpreter::signature_inputs
      • tflite::Interpreter::signature_outputs
      • tflite::Interpreter::input_tensor_by_signature
      • tflite::Interpreter::output_tensor_by_signature
    • Similarly, the following signature runner functions in the TF Lite C API are no longer considered experimental:
      • TfLiteInterpreterGetSignatureCount
      • TfLiteInterpreterGetSignatureKey
      • TfLiteInterpreterGetSignatureRunner
      • TfLiteSignatureRunnerAllocateTensors
      • TfLiteSignatureRunnerGetInputCount
      • TfLiteSignatureRunnerGetInputName
      • TfLiteSignatureRunnerGetInputTensor
      • TfLiteSignatureRunnerGetOutputCount
      • TfLiteSignatureRunnerGetOutputName
      • TfLiteSignatureRunnerGetOutputTensor
      • TfLiteSignatureRunnerInvoke
      • TfLiteSignatureRunnerResizeInputTensor
    • New C API function TfLiteExtensionApisVersion added to tensorflow/lite/c/c_api.h.
    • Add int8 and int16x8 support for RSQRT operator

  • Android NDK r25 is supported.

Bug Fixes and Other Changes

  • Add TensorFlow Quantizer to TensorFlow pip package.

  • tf.sparse.segment_sum tf.sparse.segment_mean tf.sparse.segment_sqrt_n SparseSegmentSum/Mean/SqrtN[WithNumSegments]

    • Added sparse_gradient option (default=false) that makes the gradient of these functions/ops sparse (IndexedSlices) instead of dense (Tensor), using new SparseSegmentSum/Mean/SqrtNGradV2 ops.

  • tf.nn.embedding_lookup_sparse

    • Optimized this function for some cases by fusing internal operations.

  • tf.saved_model.SaveOptions

    • Provided a new experimental_skip_saver argument which, if specified, will suppress the addition of SavedModel-native save and restore ops to the SavedModel, for cases where users already build custom save/restore ops and checkpoint formats for the model being saved, and the creation of the SavedModel-native save/restore ops simply cause longer model serialization times.

Keras

Bug Fixes and Other Changes

  • Add ops to tensorflow.raw_ops that were missing.
  • tf.CheckpointOptions
    • It now takes in a new argument called experimental_write_callbacks. These are callbacks that will be executed after a saving event finishes writing the checkpoint file.
  • Add an option disable_eager_executer_streaming_enqueue to tensorflow.ConfigProto.Experimental to control the eager runtime's behavior around parallel remote function invocations; when set to True, the eager runtime will be allowed to execute multiple function invocations in parallel.

  • tf.constant_initializer

    • It now takes a new argument called support_partition. If True, constant_initializers can create sharded variables. This is disabled by default, similar to existing behavior.

  • tf.lite

    • Added support for stablehlo.scatter.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aiden Grossman, Akash Patel, Akhil Goel, Alexander Pivovarov, Andrew Goodbody, Ayan Moitra, Ben Barsdell, Ben Olson, Bhavani Subramanian, Boian Petkantchin, Bruce Lai, Chao Chen, Christian Steinmeyer, cjflan, David Korczynski, Donghak Park, Dragan Mladjenovic, Eli Kobrin, Fadi Arafeh, Feiyue Chen, Frédéric Bastien, guozhong.zhuang, halseycamilla, Harshavardhan Bellamkonda, James Ward, jameshollyer, Jane Liu, johnnkp, jswag180, justkw, Kanvi Khanna, Keith Smiley, Koan-Sin Tan, Kulin Seth, Kun-Lu, kushanam, Lu Teng, mdfaijul, Mehdi Drissi, mgokulkrish, mraunak, Mustafa Uzun, Namrata Bhave, Pavel Emeliyanenko, pemeliya, Peng Sun, Philipp Hack, Pratik Joshi, Rahul Batra, Raunak, redwrasse, Saoirse Stewart, SaoirseARM, seanshpark, Shanbin Ke, Spenser Bauman, Surya, sushreebarsa, Tai Ly, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, Tj Xu, Vladislav, weihanmines, Wen Chen, wenchenvincent, wenscarl, William Muir, Zhoulong, Jiang

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.14.0

Release 2.14.0

Tensorflow

Breaking Changes

  • Support for Python 3.8 has been removed starting with TF 2.14. The TensorFlow 2.13.1 patch release will still have Python 3.8 support.

  • tf.Tensor

    • The class hierarchy for tf.Tensor has changed, and there are now explicit EagerTensor and SymbolicTensor classes for eager and tf.function respectively. Users who relied on the exact type of Tensor (e.g. type(t) == tf.Tensor) will need to update their code to use isinstance(t, tf.Tensor). The tf.is_symbolic_tensor helper added in 2.13 may be used when it is necessary to determine if a value is specifically a symbolic tensor.

  • tf.compat.v1.Session

    • tf.compat.v1.Session.partial_run and tf.compat.v1.Session.partial_run_setup will be deprecated in the next release.

Known Caveats

  • tf.lite
    • when converter flag "experimenalusebufferoffset" is enabled, additional metadata is automatically excluded from the generated model. The behaviour is the same as "excludeconversionmetadata" is set
    • If the model is larger than 2GB, then we also require "excludeconversionmetadata" flag to be set

Major Features and Improvements

  • The tensorflow pip package has a new, optional installation method for Linux that installs necessary Nvidia CUDA libraries through pip. As long as the Nvidia driver is already installed on the system, you may now run pip install tensorflow[and-cuda] to install TensorFlow's Nvidia CUDA library dependencies in the Python environment. Aside from the Nvidia driver, no other pre-existing Nvidia CUDA packages are necessary.

  • Enable JIT-compiled i64-indexed kernels on GPU for large tensors with more than 2**32 elements.

    • Unary GPU kernels: Abs, Atanh, Acos, Acosh, Asin, Asinh, Atan, Cos, Cosh, Sin, Sinh, Tan, Tanh.
    • Binary GPU kernels: AddV2, Sub, Div, DivNoNan, Mul, MulNoNan, FloorDiv, Equal, NotEqual, Greater, GreaterEqual, LessEqual, Less.

  • tf.lite

    • Add experimental supports conversion of models that may be larger than 2GB before buffer deduplication

Bug Fixes and Other Changes

  • tf.py_function and tf.numpy_function can now be used as function decorators for clearer code: @tf.py_function(Tout=tf.float32) def my_fun(x): print("This always executes eagerly.") return x+1

  • tf.lite

    • Strided_Slice now supports UINT32.

  • tf.config.experimental.enable_tensor_float_32_execution

    • Disabling TensorFloat-32 execution now causes TPUs to use float32 precision for float32 matmuls and other ops. TPUs have always used bfloat16 precision for certain ops, like matmul, when such ops had float32 inputs. Now, disabling TensorFloat-32 by calling tf.config.experimental.enable_tensor_float_32_execution(False) will cause TPUs to use float32 precision for such ops instead of bfloat16.

  • tf.experimental.dtensor

    • API changes for Relayout. Added a new API, dtensor.relayout_like, for relayouting a tensor according to the layout of another tensor.
    • Added dtensor.get_default_mesh, for retrieving the current default mesh under the dtensor context.
    • *fft* ops now support dtensors with any layout. Fixed bug in 'fft2d/fft3d', 'ifft2d/ifft3d', 'rfft2d/rfft3d', and 'irfft2d/irfft3d' for sharde input. Refer to this blog post for details.

  • tf.experimental.strict_mode

    • Added a new API, strict_mode, which converts all deprecation warnings into runtime errors with instructions on switching to a recommended substitute.

  • TensorFlow Debugger (tfdbg) CLI: ncurses-based CLI for tfdbg v1 was removed.

  • TensorFlow now supports C++ RTTI on mobile and Android. To enable this feature, pass the flag --define=tf_force_rtti=true to Bazel when building TensorFlow. This may be needed when linking TensorFlow into RTTI-enabled programs since mixing RTTI and non-RTTI code can cause ABI issues.

  • tf.ones, tf.zeros, tf.fill, tf.ones_like, tf.zeros_like now take an additional Layout argument that controls the output layout of their results.

  • tf.nest and tf.data now support user defined classes implementing __tf_flatten__ and __tf_unflatten__ methods. See nest_util code examples for an example.

  • TensorFlow IO support is now available for Apple Silicon packages.

  • Refactor CpuExecutable to propagate LLVM errors.

Keras

Keras is a framework built on top of the TensorFlow. See more details on the Keras website.

Major Features and Improvements

  • tf.keras
    • Model.compile now support steps_per_execution='auto' as a parameter, allowing automatic tuning of steps per execution during Model.fit, Model.predict, and Model.evaluate for a significant performance boost.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aakar Dwivedi, Adrian Popescu, ag.ramesh, Akhil Goel, Albert Zeyer, Alex Rosen, Alexey Vishnyakov, Andrew Goodbody, angerson, Ashiq Imran, Ayan Moitra, Ben Barsdell, Bhavani Subramanian, Boian Petkantchin, BrianWieder, Chris Mc, cloudhan, Connor Flanagan, Daniel Lang, Daniel Yudelevich, Darya Parygina, David Korczynski, David Svantesson, dingyuqing05, Dragan Mladjenovic, dskkato, Eli Kobrin, Erick Ochoa, Erik Schultheis, Frédéric Bastien, gaikwadrahul8, Gauri1 Deshpande, guozhong.zhuang, H. Vetinari, Isaac Cilia Attard, Jake Hall, Jason Furmanek, Jerry Ge, Jinzhe Zeng, JJ, johnnkp, Jonathan Albrecht, jongkweh, justkw, Kanvi Khanna, kikoxia, Koan-Sin Tan, Kun-Lu, ltsai1, Lu Teng, luliyucoordinate, Mahmoud Abuzaina, mdfaijul, Milos Puzovic, Nathan Luehr, Om Thakkar, pateldeev, Peng Sun, Philipp Hack, pjpratik, Poliorcetics, rahulbatra85, rangjiaheng, Renato Arantes, Robert Kalmar, roho, Rylan Justice, Sachin Muradi, samypr100, Saoirse Stewart, Shanbin Ke, Shivam Mishra, shuw, Song Ziming, Stephan Hartmann, Sulav, sushreebarsa, T Coxon, Tai Ly, talyz, Thibaut Goetghebuer-Planchon, Thomas Preud'Homme, tilakrayal, Tirumalesh, Tj Xu, Tom Allsop, Trevor Morris, Varghese, Jojimon, Wen Chen, Yaohui Liu, Yimei Sun, Zhoulong Jiang, Zhoulong, Jiang

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.13.1

Release 2.13.1

Bug Fixes and Other Changes

  • Refactor CpuExecutable to propagate LLVM errors.

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.14.0-rc1

Release 2.14.0

Tensorflow

Breaking Changes

  • Support for Python 3.8 has been removed starting with TF 2.14. The TensorFlow 2.13.1 patch release will still have Python 3.8 support.

  • tf.Tensor

    • The class hierarchy for tf.Tensor has changed, and there are now explicit EagerTensor and SymbolicTensor classes for eager and tf.function respectively. Users who relied on the exact type of Tensor (e.g. type(t) == tf.Tensor) will need to update their code to use isinstance(t, tf.Tensor). The tf.is_symbolic_tensor helper added in 2.13 may be used when it is necessary to determine if a value is specifically a symbolic tensor.

  • tf.compat.v1.Session

    • tf.compat.v1.Session.partial_run and tf.compat.v1.Session.partial_run_setup will be deprecated in the next release.

  • tf.estimator

    • tf.estimator API will be removed in the next release. TF Estimator Python package will no longer be released.

Known Caveats

  • tf.lite
    • when converter flag "experimenalusebufferoffset" is enabled, additional metadata is automatically excluded from the generated model. The behaviour is the same as "excludeconversionmetadata" is set
    • If the model is larger than 2GB, then we also require "excludeconversionmetadata" flag to be set

Major Features and Improvements

  • The tensorflow pip package has a new, optional installation method for Linux that installs necessary Nvidia CUDA libraries through pip. As long as the Nvidia driver is already installed on the system, you may now run pip install tensorflow[and-cuda] to install TensorFlow's Nvidia CUDA library dependencies in the Python environment. Aside from the Nvidia driver, no other pre-existing Nvidia CUDA packages are necessary.

  • Enable JIT-compiled i64-indexed kernels on GPU for large tensors with more than 2**32 elements.

    • Unary GPU kernels: Abs, Atanh, Acos, Acosh, Asin, Asinh, Atan, Cos, Cosh, Sin, Sinh, Tan, Tanh.
    • Binary GPU kernels: AddV2, Sub, Div, DivNoNan, Mul, MulNoNan, FloorDiv, Equal, NotEqual, Greater, GreaterEqual, LessEqual, Less.

  • tf.lite

    • Add experimental supports conversion of models that may be larger than 2GB before buffer deduplication

Bug Fixes and Other Changes

  • tf.py_function and tf.numpy_function can now be used as function decorators for clearer code: @tf.py_function(Tout=tf.float32) def my_fun(x): print("This always executes eagerly.") return x+1

  • tf.lite

    • Strided_Slice now supports UINT32.

  • tf.config.experimental.enable_tensor_float_32_execution

    • Disabling TensorFloat-32 execution now causes TPUs to use float32 precision for float32 matmuls and other ops. TPUs have always used bfloat16 precision for certain ops, like matmul, when such ops had float32 inputs. Now, disabling TensorFloat-32 by calling tf.config.experimental.enable_tensor_float_32_execution(False) will cause TPUs to use float32 precision for such ops instead of bfloat16.

  • tf.experimental.dtensor

    • API changes for Relayout. Added a new API, dtensor.relayout_like, for relayouting a tensor according to the layout of another tensor.
    • Added dtensor.get_default_mesh, for retrieving the current default mesh under the dtensor context.
    • *fft* ops now support dtensors with any layout. Fixed bug in 'fft2d/ fft3d', 'ifft2d/ifft3d', 'rfft2d/rfft3d', and 'irfft2d/irfft3d' for sharded input.

  • tf.experimental.strict_mode

    • Added a new API, strict_mode, which converts all deprecation warnings into runtime errors with instructions on switching to recommended substitute.

  • TensorFlow Debugger (tfdbg) CLI: ncurses-based CLI for tfdbg v1 was removed.

  • TensorFlow now supports C++ RTTI on mobile and Android. To enable this feature, pass the flag --define=tf_force_rtti=true to Bazel when building TensorFlow. This may be needed when linking TensorFlow into RTTI-enabled programs since mixing RTTI and non-RTTI code can cause ABI issues.

  • tf.ones, tf.zeros, tf.fill, tf.ones_like, tf.zeros_like now take an additional Layout argument that controls the output layout of their results.

  • tf.nest and tf.data now support user defined classes implementing __tf_flatten__ and __tf_unflatten__ methods. See nest_util code examples for an example.

Keras

Keras is a framework built on top of the TensorFlow. See more details on the Keras website.

Major Features and Improvements

  • tf.keras
    • Model.compile now support steps_per_execution='auto' as a parameter, allowing automatic tuning of steps per execution during Model fit, Model.predict, and Model.evaluate for a significant performance boost.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aakar Dwivedi, Adrian Popescu, ag.ramesh, Akhil Goel, Albert Zeyer, Alex Rosen, Alexey Vishnyakov, Andrew Goodbody, angerson, Ashiq Imran, Ayan Moitra, Ben Barsdell, Bhavani Subramanian, Boian Petkantchin, BrianWieder, Chris Mc, cloudhan, Connor Flanagan, Daniel Lang, Daniel Yudelevich, Darya Parygina, David Korczynski, David Svantesson, dingyuqing05, Dragan Mladjenovic, dskkato, Eli Kobrin, Erick Ochoa, Erik Schultheis, Frédéric Bastien, gaikwadrahul8, Gauri1 Deshpande, georgiie, guozhong.zhuang, H. Vetinari, Isaac Cilia Attard, Jake Hall, Jason Furmanek, Jerry Ge, Jinzhe Zeng, JJ, johnnkp, Jonathan Albrecht, jongkweh, justkw, Kanvi Khanna, kikoxia, Koan-Sin Tan, Kun-Lu, Learning-To-Play, ltsai1, Lu Teng, luliyucoordinate, Mahmoud Abuzaina, mdfaijul, Milos Puzovic, Nathan Luehr, Om Thakkar, pateldeev, Peng Sun, Philipp Hack, pjpratik, Poliorcetics, rahulbatra85, rangjiaheng, Renato Arantes, Robert Kalmar, roho, Rylan Justice, Sachin Muradi, samypr100, Saoirse Stewart, Shanbin Ke, Shivam Mishra, shuw, Song Ziming, Stephan Hartmann, Sulav, sushreebarsa, T Coxon, Tai Ly, talyz, Tensorflow Jenkins, Thibaut Goetghebuer-Planchon, Thomas Preud'Homme, tilakrayal, Tirumalesh, Tj Xu, Tom Allsop, Trevor Morris, Varghese, Jojimon, Wen Chen, Yaohui Liu, Yimei Sun, Zhoulong Jiang, Zhoulong, Jiang

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.14.0-rc0

Release 2.14.0

Tensorflow

Breaking Changes

  • tf.Tensor

    • The class hierarchy for tf.Tensor has changed, and there are now explicit EagerTensor and SymbolicTensor classes for eager and tf.function respectively. Users who relied on the exact type of Tensor (e.g. type(t) == tf.Tensor) will need to update their code to use isinstance(t, tf.Tensor). The tf.is_symbolic_tensor helper added in 2.13 may be used when it is necessary to determine if a value is specifically a symbolic tensor.

  • tf.compat.v1.Session

    • tf.compat.v1.Session.partial_run and tf.compat.v1.Session.partial_run_setup will be deprecated in the next release.

Known Caveats

  • tf.lite
    • when converter flag "experimenalusebufferoffset" is enabled, additional metadata is automatically excluded from the generated model. The behaviour is the same as "excludeconversionmetadata" is set
    • If the model is larger than 2GB, then we also require "excludeconversionmetadata" flag to be set

Major Features and Improvements

  • Enable JIT-compiled i64-indexed kernels on GPU for large tensors with more than 2**32 elements.

    • Unary GPU kernels: Abs, Atanh, Acos, Acosh, Asin, Asinh, Atan, Cos, Cosh, Sin, Sinh, Tan, Tanh.
    • Binary GPU kernels: AddV2, Sub, Div, DivNoNan, Mul, MulNoNan, FloorDiv, Equal, NotEqual, Greater, GreaterEqual, LessEqual, Less.

  • tf.lite

    • Add experimental supports conversion of models that may be larger than 2GB before buffer deduplication

Bug Fixes and Other Changes

  • tf.py_function and tf.numpy_function can now be used as function decorators for clearer code: @tf.py_function(Tout=tf.float32) def my_fun(x): print("This always executes eagerly.") return x+1

  • tf.lite

    • Strided_Slice now supports UINT32.

  • tf.config.experimental.enable_tensor_float_32_execution

    • Disabling TensorFloat-32 execution now causes TPUs to use float32 precision for float32 matmuls and other ops. TPUs have always used bfloat16 precision for certain ops, like matmul, when such ops had float32 inputs. Now, disabling TensorFloat-32 by calling tf.config.experimental.enable_tensor_float_32_execution(False) will cause TPUs to use float32 precision for such ops instead of bfloat16.

  • tf.experimental.dtensor

    • API changes for Relayout. Added a new API, dtensor.relayout_like, for relayouting a tensor according to the layout of another tensor.
    • Added dtensor.get_default_mesh, for retrieving the current default mesh under the dtensor context.
    • *fft* ops now support dtensors with any layout. Fixed bug in 'fft2d/ fft3d', 'ifft2d/ifft3d', 'rfft2d/rfft3d', and 'irfft2d/irfft3d' for sharded input.

  • tf.experimental.strict_mode

    • Added a new API, strict_mode, which converts all deprecation warnings into runtime errors with instructions on switching to a recommended substitute.

  • TensorFlow Debugger (tfdbg) CLI: ncurses-based CLI for tfdbg v1 was removed.

  • TensorFlow now supports C++ RTTI on mobile and Android. To enable this feature, pass the flag --define=tf_force_rtti=true to Bazel when building TensorFlow. This may be needed when linking TensorFlow into RTTI-enabled programs since mixing RTTI and non-RTTI code can cause ABI issues.

  • tf.ones, tf.zeros, tf.fill, tf.ones_like, tf.zeros_like now take an additional Layout argument that controls the output layout of their results.

  • tf.nest and tf.data now support user defined classes implementing __tf_flatten__ and __tf_unflatten__ methods. See nest_util code examples for an example.

Keras

Keras is a framework built on top of the TensorFlow. See more details on the Keras website.

Major Features and Improvements

  • tf.keras
    • Model.compile now support steps_per_execution='auto' as a parameter, allowing automatic tuning of steps per execution during Model.fit, Model.predict, and Model.evaluate for a significant performance boost.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aakar Dwivedi, Adrian Popescu, ag.ramesh, Akhil Goel, Albert Zeyer, Alex Rosen, Alexey Vishnyakov, Andrew Goodbody, angerson, Ashiq Imran, Ayan Moitra, Ben Barsdell, Bhavani Subramanian, Boian Petkantchin, BrianWieder, Chris Mc, cloudhan, Connor Flanagan, Daniel Lang, Daniel Yudelevich, Darya Parygina, David Korczynski, David Svantesson, dingyuqing05, Dragan Mladjenovic, dskkato, Eli Kobrin, Erick Ochoa, Erik Schultheis, Frédéric Bastien, gaikwadrahul8, Gauri1 Deshpande, georgiie, guozhong.zhuang, H. Vetinari, Isaac Cilia Attard, Jake Hall, Jason Furmanek, Jerry Ge, Jinzhe Zeng, JJ, johnnkp, Jonathan Albrecht, jongkweh, justkw, Kanvi Khanna, kikoxia, Koan-Sin Tan, Kun-Lu, Learning-To-Play, ltsai1, Lu Teng, luliyucoordinate, Mahmoud Abuzaina, mdfaijul, Milos Puzovic, Nathan Luehr, Om Thakkar, pateldeev, Peng Sun, Philipp Hack, pjpratik, Poliorcetics, rahulbatra85, rangjiaheng, Renato Arantes, Robert Kalmar, roho, Rylan Justice, Sachin Muradi, samypr100, Saoirse Stewart, Shanbin Ke, Shivam Mishra, shuw, Song Ziming, Stephan Hartmann, Sulav, sushreebarsa, T Coxon, Tai Ly, talyz, Tensorflow Jenkins, Thibaut Goetghebuer-Planchon, Thomas Preud'Homme, tilakrayal, Tirumalesh, Tj Xu, Tom Allsop, Trevor Morris, Varghese, Jojimon, Wen Chen, Yaohui Liu, Yimei Sun, Zhoulong Jiang, Zhoulong, Jiang

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.12.1

Release 2.12.1

Bug Fixes and Other Changes

  • The use of the ambe config to build and test aarch64 is not needed. The ambe config will be removed in the future. Making cpuarm64pip.sh and cpuarm64nonpip.sh more similar for easier future maintenance.

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.13.0

Release 2.13.0

TensorFlow

Breaking Changes

  • The LMDB kernels have been changed to return an error. This is in preparation for completely removing them from TensorFlow. The LMDB dependency that these kernels are bringing to TensorFlow has been dropped, thus making the build slightly faster and more secure.

Major Features and Improvements

  • tf.lite

    • Added 16-bit and 64-bit float type support for built-in op cast.
    • The Python TF Lite Interpreter bindings now have an option experimental_disable_delegate_clustering to turn-off delegate clustering.
    • Added int16x8 support for the built-in op exp
    • Added int16x8 support for the built-in op mirror_pad
    • Added int16x8 support for the built-in ops space_to_batch_nd and batch_to_space_nd
    • Added 16-bit int type support for built-in op less, greater_than, equal
    • Added 8-bit and 16-bit support for floor_div and floor_mod.
    • Added 16-bit and 32-bit int support for the built-in op bitcast.
    • Added 8-bit/16-bit/32-bit int/uint support for the built-in op bitwise_xor
    • Added int16 indices support for built-in op gather and gather_nd.
    • Added 8-bit/16-bit/32-bit int/uint support for the built-in op right_shift
    • Added reference implementation for 16-bit int unquantized add.
    • Added reference implementation for 16-bit int and 32-bit unsigned int unquantized mul.
    • add_op supports broadcasting up to 6 dimensions.
    • Added 16-bit support for top_k.

  • tf.function

    • ConcreteFunction (tf.types.experimental.ConcreteFunction) as generated through get_concrete_function now performs holistic input validation similar to calling tf.function directly. This can cause breakages where existing calls pass Tensors with the wrong shape or omit certain non-Tensor arguments (including default values).

  • tf.nn

    • tf.nn.embedding_lookup_sparse and tf.nn.safe_embedding_lookup_sparse now support ids and weights described by tf.RaggedTensors.
    • Added a new boolean argument allow_fast_lookup to tf.nn.embedding_lookup_sparse and tf.nn.safe_embedding_lookup_sparse, which enables a simplified and typically faster lookup procedure.

  • tf.data

    • tf.data.Dataset.zip now supports Python-style zipping, i.e. Dataset.zip(a, b, c).
    • tf.data.Dataset.shuffle now supports tf.data.UNKNOWN_CARDINALITY When doing a "full shuffle" using dataset = dataset.shuffle(dataset.cardinality()). But remember, a "full shuffle" will load the full dataset into memory so that it can be shuffled, so make sure to only use this with small datasets or datasets of small objects (like filenames).

  • tf.math

    • tf.nn.top_k now supports specifying the output index type via parameter index_type. Supported types are tf.int16, tf.int32 (default), and tf.int64.

  • tf.SavedModel

    • Introduced class method tf.saved_model.experimental.Fingerprint.from_proto(proto), which can be used to construct a Fingerprint object directly from a protobuf.
    • Introduced member method tf.saved_model.experimental.Fingerprint.singleprint(), which provides a convenient way to uniquely identify a SavedModel.

Bug Fixes and Other Changes

  • tf.Variable

    • Changed resource variables to inherit from tf.compat.v2.Variable instead of tf.compat.v1.Variable. Some checks for isinstance(v, tf compat.v1.Variable) that previously returned True may now return False.

  • tf.distribute

    • Opened an experimental API, tf.distribute.experimental.coordinator.get_current_worker_index, for retrieving the worker index from within a worker, when using parameter server training with a custom training loop.

  • tf.experimental.dtensor

    • Deprecated dtensor.run_on in favor of dtensor.default_mesh to correctly indicate that the context does not override the mesh that the ops and functions will run on, it only sets a fallback default mesh.
    • List of members of dtensor.Layout and dtensor.Mesh have slightly changed as part of efforts to consolidate the C++ and Python source code with pybind11. Most notably, dtensor.Layout.serialized_string is removed.
    • Minor API changes to represent Single Device Layout for non-distributed Tensors inside DTensor functions. Runtime support will be added soon.

  • tf.experimental.ExtensionType

    • tf.experimental.ExtensionType now supports Python tuple as the type annotation of its fields.

  • tf.nest

    • Deprecated API tf.nest.is_sequence has now been deleted. Please use tf.nest.is_nested instead.

Keras

Keras is a framework built on top of the TensorFlow. See more details on the Keras website.

Breaking Changes

  • Removed the Keras scikit-learn API wrappers (KerasClassifier and KerasRegressor), which had been deprecated in August 2021. We recommend using SciKeras instead.
  • The default Keras model saving format is now the Keras v3 format: calling model.save("xyz.keras") will no longer create a H5 file, it will create a native Keras model file. This will only be breaking for you if you were manually inspecting or modifying H5 files saved by Keras under a .keras extension. If this breaks you, simply add save_format="h5" to your .save() call to revert back to the prior behavior.
  • Added keras.utils.TimedThread utility to run a timed thread every x seconds. It can be used to run a threaded function alongside model training or any other snippet of code.
  • In the keras PyPI package, accessible symbols are now restricted to symbols that are intended to be public. This may affect your code if you were using import keras and you used keras functions that were not public APIs, but were accessible in earlier versions with direct imports. In those cases, please use the following guideline: - The API may be available in the public Keras API under a different name, so make sure to look for it on keras.io or TensorFlow docs and switch to the public version. - It could also be a simple python or TF utility that you could easily copy over to your own codebase. In those case, just make it your own! - If you believe it should definitely be a public Keras API, please open a feature request in keras GitHub repo. - As a workaround, you could import the same private symbol keras keras.src, but keep in mind the src namespace is not stable and those APIs may change or be removed in the future.

Major Features and Improvements

  • Added F-Score metrics tf.keras.metrics.FBetaScore, tf.keras.metrics.F1Score, and tf.keras.metrics.R2Score.
  • Added activation function tf.keras.activations.mish.
  • Added experimental keras.metrics.experimental.PyMetric API for metrics that run Python code on the host CPU (compiled outside of the TensorFlow graph). This can be used for integrating metrics from external Python libraries (like sklearn or pycocotools) into Keras as first-class Keras metrics.
  • Added tf.keras.optimizers.Lion optimizer.
  • Added tf.keras.layers.SpectralNormalization layer wrapper to perform spectral normalization on the weights of a target layer.
  • The SidecarEvaluatorModelExport callback has been added to Keras as keras.callbacks.SidecarEvaluatorModelExport. This callback allows for exporting the model the best-scoring model as evaluated by a SidecarEvaluator evaluator. The evaluator regularly evaluates the model and exports it if the user-defined comparison function determines that it is an improvement.
  • Added warmup capabilities to tf.keras.optimizers.schedules.CosineDecay learning rate scheduler. You can now specify an initial and target learning rate, and our scheduler will perform a linear interpolation between the two after which it will begin a decay phase.
  • Added experimental support for an exactly-once visitation guarantee for evaluating Keras models trained with tf.distribute ParameterServerStrategy, via the exact_evaluation_shards argument in Model.fit and Model.evaluate.
  • Added tf.keras.__internal__.KerasTensor,tf.keras.__internal__.SparseKerasTensor, and tf.keras.__internal__.RaggedKerasTensor classes. You can use these classes to do instance type checking and type annotations for layer/model inputs and outputs.
  • All the tf.keras.dtensor.experimental.optimizers classes have been merged with tf.keras.optimizers. You can migrate your code to use tf.keras.optimizers directly. The API namespace for tf.keras.dtensor.experimental.optimizers will be removed in future releases.
  • Added support for class_weight for 3+ dimensional targets (e.g. image segmentation masks) in Model.fit.
  • Added a new loss, keras.losses.CategoricalFocalCrossentropy.
  • Remove the tf.keras.dtensor.experimental.layout_map_scope(). You can user the tf.keras.dtensor.experimental.LayoutMap.scope() instead.

Security

  • Fixes correct values rank in UpperBound and LowerBound CVE-2023-33976

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar, Aakar Dwivedi, Abinash Satapathy, Aditya Kane, ag.ramesh, Alexander Grund, Andrei Pikas, andreii, Andrew Goodbody, angerson, Anthony_256, Ashay Rane, Ashiq Imran, Awsaf, Balint Cristian, Banikumar Maiti (Intel Aipg), Ben Barsdell, bhack, cfRod, Chao Chen, chenchongsong, Chris Mc, Daniil Kutz, David Rubinstein, dianjiaogit, dixr, Dongfeng Yu, dongfengy, drah, Eric Kunze, Feiyue Chen, Frederic Bastien, Gauri1 Deshpande, guozhong.zhuang, hDn248, HYChou, ingkarat, James Hilliard, Jason Furmanek, Jaya, Jens Glaser, Jerry Ge, Jiao Dian'S Power Plant, Jie Fu, Jinzhe Zeng, Jukyy, Kaixi Hou, Kanvi Khanna, Karel Ha, karllessard, Koan-Sin Tan, Konstantin Beluchenko, Kulin Seth, Kun Lu, Kyle Gerard Felker, Leopold Cambier, Lianmin Zheng, linlifan, liuyuanqiang, Lukas Geiger, Luke Hutton, Mahmoud Abuzaina, Manas Mohanty, Mateo Fidabel, Maxiwell S. Garcia, Mayank Raunak, mdfaijul, meatybobby, Meenakshi Venkataraman, Michael Holman, Nathan John Sircombe, Nathan Luehr, nitins17, Om Thakkar, Patrice Vignola, Pavani Majety, per1234, Philipp Hack, pollfly, Prianka Liz Kariat, Rahul Batra, rahulbatra85, ratnam.parikh, Rickard Hallerbäck, Roger Iyengar, Rohit Santhanam, Roman Baranchuk, Sachin Muradi, sanadani, Saoirse Stewart, seanshpark, Shawn Wang, shuw, Srinivasan Narayanamoorthy, Stewart Miles, Sunita Nadampalli, SuryanarayanaY, Takahashi Shuuji, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, TJ, Tony Sung, Trevor Morris, unda, Vertexwahn, venkat2469, William Muir, Xavier Bonaventura, xiang.zhang, Xiao-Yong Jin, yleeeee, Yong Tang, Yuriy Chernyshov, Zhang, Xiangze, zhaozheng09

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.13.0-rc2

Release 2.13.0

TensorFlow

Breaking Changes

  • The LMDB kernels have been changed to return an error. This is in preparation for completely removing them from TensorFlow. The LMDB dependency that these kernels are bringing to TensorFlow has been dropped, thus making the build slightly faster and more secure.

Major Features and Improvements

  • tf.lite

    • Added 16-bit and 64-bit float type support for built-in op cast.
    • The Python TF Lite Interpreter bindings now have an option experimental_disable_delegate_clustering to turn-off delegate clustering.
    • Added int16x8 support for the built-in op exp
    • Added int16x8 support for the built-in op mirror_pad
    • Added int16x8 support for the built-in ops space_to_batch_nd and batch_to_space_nd
    • Added 16-bit int type support for built-in op less, greater_than, equal
    • Added 8-bit and 16-bit support for floor_div and floor_mod.
    • Added 16-bit and 32-bit int support for the built-in op bitcast.
    • Added 8-bit/16-bit/32-bit int/uint support for the built-in op bitwise_xor
    • Added int16 indices support for built-in op gather and gather_nd.
    • Added 8-bit/16-bit/32-bit int/uint support for the built-in op right_shift
    • Added reference implementation for 16-bit int unquantized add.
    • Added reference implementation for 16-bit int and 32-bit unsigned int unquantized mul.
    • add_op supports broadcasting up to 6 dimensions.
    • Added 16-bit support for top_k.

  • tf.function

    • ConcreteFunction (tf.types.experimental.ConcreteFunction) as generated through get_concrete_function now performs holistic input validation similar to calling tf.function directly. This can cause breakages where existing calls pass Tensors with the wrong shape or omit certain non-Tensor arguments (including default values).

  • tf.nn

    • tf.nn.embedding_lookup_sparse and tf.nn.safe_embedding_lookup_sparse now support ids and weights described by tf.RaggedTensors.
    • Added a new boolean argument allow_fast_lookup to tf.nn.embedding_lookup_sparse and tf.nn.safe_embedding_lookup_sparse, which enables a simplified and typically faster lookup procedure.

  • tf.data

    • tf.data.Dataset.zip now supports Python-style zipping, i.e. Dataset.zip(a, b, c).
    • tf.data.Dataset.shuffle now supports tf.data.UNKNOWN_CARDINALITY When doing a "full shuffle" using dataset = dataset.shuffle(dataset.cardinality()). But remember, a "full shuffle" will load the full dataset into memory so that it can be shuffled, so make sure to only use this with small datasets or datasets of small objects (like filenames).

  • tf.math

    • tf.nn.top_k now supports specifying the output index type via parameter index_type. Supported types are tf.int16, tf.int32 (default), and tf.int64.

  • tf.SavedModel

    • Introduced class method tf.saved_model.experimental.Fingerprint.from_proto(proto), which can be used to construct a Fingerprint object directly from a protobuf.
    • Introduced member method tf.saved_model.experimental.Fingerprint.singleprint(), which provides a convenient way to uniquely identify a SavedModel.

Bug Fixes and Other Changes

  • tf.Variable

    • Changed resource variables to inherit from tf.compat.v2.Variable instead of tf.compat.v1.Variable. Some checks for isinstance(v, tf compat.v1.Variable) that previously returned True may now return False.

  • tf.distribute

    • Opened an experimental API, tf.distribute.experimental.coordinator.get_current_worker_index, for retrieving the worker index from within a worker, when using parameter server training with a custom training loop.

  • tf.experimental.dtensor

    • Deprecated dtensor.run_on in favor of dtensor.default_mesh to correctly indicate that the context does not override the mesh that the ops and functions will run on, it only sets a fallback default mesh.
    • List of members of dtensor.Layout and dtensor.Mesh have slightly changed as part of efforts to consolidate the C++ and Python source code with pybind11. Most notably, dtensor.Layout.serialized_string is removed.
    • Minor API changes to represent Single Device Layout for non-distributed Tensors inside DTensor functions. Runtime support will be added soon.

  • tf.experimental.ExtensionType

    • tf.experimental.ExtensionType now supports Python tuple as the type annotation of its fields.

  • tf.nest

    • Deprecated API tf.nest.is_sequence has now been deleted. Please use tf.nest.is_nested instead.

Keras

Keras is a framework built on top of the TensorFlow. See more details on the Keras website.

Breaking Changes

  • Removed the Keras scikit-learn API wrappers (KerasClassifier and KerasRegressor), which had been deprecated in August 2021. We recommend using SciKeras instead.
  • The default Keras model saving format is now the Keras v3 format: calling model.save("xyz.keras") will no longer create a H5 file, it will create a native Keras model file. This will only be breaking for you if you were manually inspecting or modifying H5 files saved by Keras under a .keras extension. If this breaks you, simply add save_format="h5" to your .save() call to revert back to the prior behavior.
  • Added keras.utils.TimedThread utility to run a timed thread every x seconds. It can be used to run a threaded function alongside model training or any other snippet of code.
  • In the keras PyPI package, accessible symbols are now restricted to symbols that are intended to be public. This may affect your code if you were using import keras and you used keras functions that were not public APIs, but were accessible in earlier versions with direct imports. In those cases, please use the following guideline: - The API may be available in the public Keras API under a different name, so make sure to look for it on keras.io or TensorFlow docs and switch to the public version. - It could also be a simple python or TF utility that you could easily copy over to your own codebase. In those case, just make it your own! - If you believe it should definitely be a public Keras API, please open a feature request in keras GitHub repo. - As a workaround, you could import the same private symbol keras keras.src, but keep in mind the src namespace is not stable and those APIs may change or be removed in the future.

Major Features and Improvements

  • Added F-Score metrics tf.keras.metrics.FBetaScore, tf.keras.metrics.F1Score, and tf.keras.metrics.R2Score.
  • Added activation function tf.keras.activations.mish.
  • Added experimental keras.metrics.experimental.PyMetric API for metrics that run Python code on the host CPU (compiled outside of the TensorFlow graph). This can be used for integrating metrics from external Python libraries (like sklearn or pycocotools) into Keras as first-class Keras metrics.
  • Added tf.keras.optimizers.Lion optimizer.
  • Added tf.keras.layers.SpectralNormalization layer wrapper to perform spectral normalization on the weights of a target layer.
  • The SidecarEvaluatorModelExport callback has been added to Keras as keras.callbacks.SidecarEvaluatorModelExport. This callback allows for exporting the model the best-scoring model as evaluated by a SidecarEvaluator evaluator. The evaluator regularly evaluates the model and exports it if the user-defined comparison function determines that it is an improvement.
  • Added warmup capabilities to tf.keras.optimizers.schedules.CosineDecay learning rate scheduler. You can now specify an initial and target learning rate, and our scheduler will perform a linear interpolation between the two after which it will begin a decay phase.
  • Added experimental support for an exactly-once visitation guarantee for evaluating Keras models trained with tf.distribute ParameterServerStrategy, via the exact_evaluation_shards argument in Model.fit and Model.evaluate.
  • Added tf.keras.__internal__.KerasTensor,tf.keras.__internal__.SparseKerasTensor, and tf.keras.__internal__.RaggedKerasTensor classes. You can use these classes to do instance type checking and type annotations for layer/model inputs and outputs.
  • All the tf.keras.dtensor.experimental.optimizers classes have been merged with tf.keras.optimizers. You can migrate your code to use tf.keras.optimizers directly. The API namespace for tf.keras.dtensor.experimental.optimizers will be removed in future releases.
  • Added support for class_weight for 3+ dimensional targets (e.g. image segmentation masks) in Model.fit.
  • Added a new loss, keras.losses.CategoricalFocalCrossentropy.
  • Remove the tf.keras.dtensor.experimental.layout_map_scope(). You can user the tf.keras.dtensor.experimental.LayoutMap.scope() instead.

Security

  • N/A

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar, Aakar Dwivedi, Abinash Satapathy, Aditya Kane, ag.ramesh, Alexander Grund, Andrei Pikas, andreii, Andrew Goodbody, angerson, Anthony_256, Ashay Rane, Ashiq Imran, Awsaf, Balint Cristian, Banikumar Maiti (Intel Aipg), Ben Barsdell, bhack, cfRod, Chao Chen, chenchongsong, Chris Mc, Daniil Kutz, David Rubinstein, dianjiaogit, dixr, Dongfeng Yu, dongfengy, drah, Eric Kunze, Feiyue Chen, Frederic Bastien, Gauri1 Deshpande, guozhong.zhuang, hDn248, HYChou, ingkarat, James Hilliard, Jason Furmanek, Jaya, Jens Glaser, Jerry Ge, Jiao Dian'S Power Plant, Jie Fu, Jinzhe Zeng, Jukyy, Kaixi Hou, Kanvi Khanna, Karel Ha, karllessard, Koan-Sin Tan, Konstantin Beluchenko, Kulin Seth, Kun Lu, Kyle Gerard Felker, Leopold Cambier, Lianmin Zheng, linlifan, liuyuanqiang, Lukas Geiger, Luke Hutton, Mahmoud Abuzaina, Manas Mohanty, Mateo Fidabel, Maxiwell S. Garcia, Mayank Raunak, mdfaijul, meatybobby, Meenakshi Venkataraman, Michael Holman, Nathan John Sircombe, Nathan Luehr, nitins17, Om Thakkar, Patrice Vignola, Pavani Majety, per1234, Philipp Hack, pollfly, Prianka Liz Kariat, Rahul Batra, rahulbatra85, ratnam.parikh, Rickard Hallerbäck, Roger Iyengar, Rohit Santhanam, Roman Baranchuk, Sachin Muradi, sanadani, Saoirse Stewart, seanshpark, Shawn Wang, shuw, Srinivasan Narayanamoorthy, Stewart Miles, Sunita Nadampalli, SuryanarayanaY, Takahashi Shuuji, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, TJ, Tony Sung, Trevor Morris, unda, Vertexwahn, venkat2469, William Muir, Xavier Bonaventura, xiang.zhang, Xiao-Yong Jin, yleeeee, Yong Tang, Yuriy Chernyshov, Zhang, Xiangze, zhaozheng09

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.13.0-rc1

Release 2.13.0

TensorFlow

Breaking Changes

  • The LMDB kernels have been changed to return an error. This is in preparation for completely removing them from TensorFlow. The LMDB dependency that these kernels are bringing to TensorFlow has been dropped, thus making the build slightly faster and more secure.

Major Features and Improvements

  • tf.lite

    • Added 16-bit and 64-bit float type support for built-in op cast.
    • The Python TF Lite Interpreter bindings now have an option experimental_disable_delegate_clustering to turn-off delegate clustering.
    • Added int16x8 support for the built-in op exp
    • Added int16x8 support for the built-in op mirror_pad
    • Added int16x8 support for the built-in ops space_to_batch_nd and batch_to_space_nd
    • Added 16-bit int type support for built-in op less, greater_than, equal
    • Added 8-bit and 16-bit support for floor_div and floor_mod.
    • Added 16-bit and 32-bit int support for the built-in op bitcast.
    • Added 8-bit/16-bit/32-bit int/uint support for the built-in op bitwise_xor
    • Added int16 indices support for built-in op gather and gather_nd.
    • Added 8-bit/16-bit/32-bit int/uint support for the built-in op right_shift
    • Added reference implementation for 16-bit int unquantized add.
    • Added reference implementation for 16-bit int and 32-bit unsigned int unquantized mul.
    • add_op supports broadcasting up to 6 dimensions.
    • Added 16-bit support for top_k.

  • tf.function

    • ConcreteFunction (tf.types.experimental.ConcreteFunction) as generated through get_concrete_function now performs holistic input validation similar to calling tf.function directly. This can cause breakages where existing calls pass Tensors with the wrong shape or omit certain non-Tensor arguments (including default values).

  • tf.nn

    • tf.nn.embedding_lookup_sparse and tf.nn.safe_embedding_lookup_sparse now support ids and weights described by tf.RaggedTensors.
    • Added a new boolean argument allow_fast_lookup to tf.nn.embedding_lookup_sparse and tf.nn.safe_embedding_lookup_sparse, which enables a simplified and typically faster lookup procedure.

  • tf.data

    • tf.data.Dataset.zip now supports Python-style zipping, i.e. Dataset.zip(a, b, c).
    • tf.data.Dataset.shuffle now supports tf.data.UNKNOWN_CARDINALITY When doing a "full shuffle" using dataset = dataset.shuffle(dataset.cardinality()). But remember, a "full shuffle" will load the full dataset into memory so that it can be shuffled, so make sure to only use this with small datasets or datasets of small objects (like filenames).

  • tf.math

    • tf.nn.top_k now supports specifying the output index type via parameter index_type. Supported types are tf.int16, tf.int32 (default), and tf.int64.

  • tf.SavedModel

    • Introduced class method tf.saved_model.experimental.Fingerprint.from_proto(proto), which can be used to construct a Fingerprint object directly from a protobuf.
    • Introduced member method tf.saved_model.experimental.Fingerprint.singleprint(), which provides a convenient way to uniquely identify a SavedModel.

Bug Fixes and Other Changes

  • tf.Variable

    • Changed resource variables to inherit from tf.compat.v2.Variable instead of tf.compat.v1.Variable. Some checks for isinstance(v, tf compat.v1.Variable) that previously returned True may now return False.

  • tf.distribute

    • Opened an experimental API, tf.distribute.experimental.coordinator.get_current_worker_index, for retrieving the worker index from within a worker, when using parameter server training with a custom training loop.

  • tf.experimental.dtensor

    • Deprecated dtensor.run_on in favor of dtensor.default_mesh to correctly indicate that the context does not override the mesh that the ops and functions will run on, it only sets a fallback default mesh.
    • List of members of dtensor.Layout and dtensor.Mesh have slightly changed as part of efforts to consolidate the C++ and Python source code with pybind11. Most notably, dtensor.Layout.serialized_string is removed.
    • Minor API changes to represent Single Device Layout for non-distributed Tensors inside DTensor functions. Runtime support will be added soon.

  • tf.experimental.ExtensionType

    • tf.experimental.ExtensionType now supports Python tuple as the type annotation of its fields.

  • tf.nest

    • Deprecated API tf.nest.is_sequence has now been deleted. Please use tf.nest.is_nested instead.

Keras

Keras is a framework built on top of the TensorFlow. See more details on the Keras website.

Breaking Changes

  • Removed the Keras scikit-learn API wrappers (KerasClassifier and KerasRegressor), which had been deprecated in August 2021. We recommend using SciKeras instead.
  • The default Keras model saving format is now the Keras v3 format: calling model.save("xyz.keras") will no longer create a H5 file, it will create a native Keras model file. This will only be breaking for you if you were manually inspecting or modifying H5 files saved by Keras under a .keras extension. If this breaks you, simply add save_format="h5" to your .save() call to revert back to the prior behavior.
  • Added keras.utils.TimedThread utility to run a timed thread every x seconds. It can be used to run a threaded function alongside model training or any other snippet of code.
  • In the keras PyPI package, accessible symbols are now restricted to symbols that are intended to be public. This may affect your code if you were using import keras and you used keras functions that were not public APIs, but were accessible in earlier versions with direct imports. In those cases, please use the following guideline: - The API may be available in the public Keras API under a different name, so make sure to look for it on keras.io or TensorFlow docs and switch to the public version. - It could also be a simple python or TF utility that you could easily copy over to your own codebase. In those case, just make it your own! - If you believe it should definitely be a public Keras API, please open a feature request in keras GitHub repo. - As a workaround, you could import the same private symbol keras keras.src, but keep in mind the src namespace is not stable and those APIs may change or be removed in the future.

Major Features and Improvements

  • Added F-Score metrics tf.keras.metrics.FBetaScore, tf.keras.metrics.F1Score, and tf.keras.metrics.R2Score.
  • Added activation function tf.keras.activations.mish.
  • Added experimental keras.metrics.experimental.PyMetric API for metrics that run Python code on the host CPU (compiled outside of the TensorFlow graph). This can be used for integrating metrics from external Python libraries (like sklearn or pycocotools) into Keras as first-class Keras metrics.
  • Added tf.keras.optimizers.Lion optimizer.
  • Added tf.keras.layers.SpectralNormalization layer wrapper to perform spectral normalization on the weights of a target layer.
  • The SidecarEvaluatorModelExport callback has been added to Keras as keras.callbacks.SidecarEvaluatorModelExport. This callback allows for exporting the model the best-scoring model as evaluated by a SidecarEvaluator evaluator. The evaluator regularly evaluates the model and exports it if the user-defined comparison function determines that it is an improvement.
  • Added warmup capabilities to tf.keras.optimizers.schedules.CosineDecay learning rate scheduler. You can now specify an initial and target learning rate, and our scheduler will perform a linear interpolation between the two after which it will begin a decay phase.
  • Added experimental support for an exactly-once visitation guarantee for evaluating Keras models trained with tf.distribute ParameterServerStrategy, via the exact_evaluation_shards argument in Model.fit and Model.evaluate.
  • Added tf.keras.__internal__.KerasTensor,tf.keras.__internal__.SparseKerasTensor, and tf.keras.__internal__.RaggedKerasTensor classes. You can use these classes to do instance type checking and type annotations for layer/model inputs and outputs.
  • All the tf.keras.dtensor.experimental.optimizers classes have been merged with tf.keras.optimizers. You can migrate your code to use tf.keras.optimizers directly. The API namespace for tf.keras.dtensor.experimental.optimizers will be removed in future releases.
  • Added support for class_weight for 3+ dimensional targets (e.g. image segmentation masks) in Model.fit.
  • Added a new loss, keras.losses.CategoricalFocalCrossentropy.
  • Remove the tf.keras.dtensor.experimental.layout_map_scope(). You can user the tf.keras.dtensor.experimental.LayoutMap.scope() instead.

Security

  • N/A

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar, Aakar Dwivedi, Abinash Satapathy, Aditya Kane, ag.ramesh, Alexander Grund, Andrei Pikas, andreii, Andrew Goodbody, angerson, Anthony_256, Ashay Rane, Ashiq Imran, Awsaf, Balint Cristian, Banikumar Maiti (Intel Aipg), Ben Barsdell, bhack, cfRod, Chao Chen, chenchongsong, Chris Mc, Daniil Kutz, David Rubinstein, dianjiaogit, dixr, Dongfeng Yu, dongfengy, drah, Eric Kunze, Feiyue Chen, Frederic Bastien, Gauri1 Deshpande, guozhong.zhuang, hDn248, HYChou, ingkarat, James Hilliard, Jason Furmanek, Jaya, Jens Glaser, Jerry Ge, Jiao Dian'S Power Plant, Jie Fu, Jinzhe Zeng, Jukyy, Kaixi Hou, Kanvi Khanna, Karel Ha, karllessard, Koan-Sin Tan, Konstantin Beluchenko, Kulin Seth, Kun Lu, Kyle Gerard Felker, Leopold Cambier, Lianmin Zheng, linlifan, liuyuanqiang, Lukas Geiger, Luke Hutton, Mahmoud Abuzaina, Manas Mohanty, Mateo Fidabel, Maxiwell S. Garcia, Mayank Raunak, mdfaijul, meatybobby, Meenakshi Venkataraman, Michael Holman, Nathan John Sircombe, Nathan Luehr, nitins17, Om Thakkar, Patrice Vignola, Pavani Majety, per1234, Philipp Hack, pollfly, Prianka Liz Kariat, Rahul Batra, rahulbatra85, ratnam.parikh, Rickard Hallerbäck, Roger Iyengar, Rohit Santhanam, Roman Baranchuk, Sachin Muradi, sanadani, Saoirse Stewart, seanshpark, Shawn Wang, shuw, Srinivasan Narayanamoorthy, Stewart Miles, Sunita Nadampalli, SuryanarayanaY, Takahashi Shuuji, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, TJ, Tony Sung, Trevor Morris, unda, Vertexwahn, venkat2469, William Muir, Xavier Bonaventura, xiang.zhang, Xiao-Yong Jin, yleeeee, Yong Tang, Yuriy Chernyshov, Zhang, Xiangze, zhaozheng09

- C++
Published by tensorflow-jenkins over 2 years ago

tensorflow - TensorFlow 2.13.0-rc0

Release 2.13.0

TensorFlow

Breaking Changes

  • The LMDB kernels have been changed to return an error. This is in preparation for completely removing them from TensorFlow. The LMDB dependency that these kernels are bringing to TensorFlow has been dropped, thus making the build slightly faster and more secure.

Major Features and Improvements

  • tf.lite

    • Add 16-bit and 64-bit float type support for built-in op cast.
    • The Python TF Lite Interpreter bindings now have an option experimental_disable_delegate_clustering to turn-off delegate clustering.
    • Add int16x8 support for the built-in op exp
    • Add int16x8 support for the built-in op mirror_pad
    • Add int16x8 support for the built-in ops space_to_batch_nd and batch_to_space_nd
    • Add 16-bit int type support for built-in op less, greater_than, equal
    • Add 8-bit and 16-bit support for floor_div and floor_mod.
    • Add 16-bit and 32-bit int support for the built-in op bitcast.
    • Add 8-bit/16-bit/32-bit int/uint support for the built-in op bitwise_xor
    • Add int16 indices support for built-in op gather and gather_nd.
    • Add 8-bit/16-bit/32-bit int/uint support for the built-in op right_shift
    • Add reference implementation for 16-bit int unquantized add.
    • Add reference implementation for 16-bit int and 32-bit unsigned int unquantized mul.
    • add_op supports broadcasting up to 6 dimensions.
    • Add 16-bit support for top_k.

  • tf.function

    • ConcreteFunction (tf.types.experimental.ConcreteFunction) as generated through get_concrete_function now performs holistic input validation similar to calling tf.function directly. This can cause breakages where existing calls pass Tensors with the wrong shape or omit certain non-Tensor arguments (including default values).

  • tf.nn

    • tf.nn.embedding_lookup_sparse and tf.nn.safe_embedding_lookup_sparse now support ids and weights described by tf.RaggedTensors.
    • Added a new boolean argument allow_fast_lookup to tf.nn.embedding_lookup_sparse and tf.nn.safe_embedding_lookup_sparse, which enables a simplified and typically faster lookup procedure.

  • tf.data

    • tf.data.Dataset.zip now supports Python-style zipping, i.e. Dataset.zip(a, b, c).
    • tf.data.Dataset.shuffle now supports full shuffling. To specify that data should be fully shuffled, use dataset = dataset.shuffle(dataset.cardinality()). This will load the full dataset into memory so that it can be shuffled, so make sure to only use this with datasets of filenames or other small datasets.

  • tf.math

    • tf.nn.top_k now supports specifying the output index type via parameter index_type. Supported types are tf.int16, tf.int32 (default), and tf.int64.

  • tf.SavedModel

    • Introduce class method tf.saved_model.experimental.Fingerprint.from_proto(proto), which can be used to construct a Fingerprint object directly from a protobuf.
    • Introduce member method tf.saved_model.experimental.Fingerprint.singleprint(), which provides a convenient way to uniquely identify a SavedModel.

Bug Fixes and Other Changes

  • tf.Variable

    • Changed resource variables to inherit from tf.compat.v2.Variable instead of tf.compat.v1.Variable. Some checks for isinstance(v, tf compat.v1.Variable) that previously returned True may now return False.

  • tf.distribute

    • Opened an experimental API, tf.distribute.experimental.coordinator.get_current_worker_index, for retrieving the worker index from within a worker, when using parameter server training with a custom training loop.

  • tf.experimental.dtensor

    • Deprecated dtensor.run_on in favor of dtensor.default_mesh to correctly indicate that the context does not override the mesh that the ops and functions will run on, it only sets a fallback default mesh.
    • List of members of dtensor.Layout and dtensor.Mesh have slightly changed as part of efforts to consolidate the C++ and Python source code with pybind11. Most notably, Layout.serialized_string is removed.
    • Minor API changes to represent Single Device Layout for non-distributed Tensors inside DTensor functions. Runtime support will be added soon.

  • tf.experimental.ExtensionType

    • tf.experimental.ExtensionType now supports Python tuple as the type annotation of its fields.

  • tf.nest

    • Deprecated API tf.nest.is_sequence has now been deleted. Please use tf.nest.is_nested instead.

Keras

Keras is a framework built on top of the TensorFlow. See more details on the Keras website.

Breaking Changes

  • tf.keras

    • Removed the Keras scikit-learn API wrappers (KerasClassifier and KerasRegressor), which had been deprecated in August 2021. We recommend using SciKeras instead.
    • The default Keras model saving format is now the Keras v3 format: calling model.save("xyz.keras") will no longer create a H5 file, it will create a native Keras model file. This will only be breaking for you if you were manually inspecting or modifying H5 files saved by Keras under a .keras extension. If this breaks you, simply add save_format="h5" to your .save() call to revert back to the prior behavior.
    • Added keras.utils.TimedThread utility to run a timed thread every x seconds. It can be used to run a threaded function alongside model training or any other snippet of code.
    • In the keras PyPI package, accessible symbols are now restricted to symbols that are intended to be public. This may affect your code if you were using import keras and you used keras functions that were not public APIs, but were accessible in earlier versions with direct imports. In those cases, please use the following guideline:
      • The API may be available in the public Keras API under a different name, so make sure to look for it on keras.io or TensorFlow docs and switch to the public version.
      • It could also be a simple python or TF utility that you could easily copy over to your own codebase. In those case, just make it your own!
      • If you believe it should definitely be a public Keras API, please open a feature request in keras GitHub repo.
      • As a workaround, you could import the same private symbol keras keras.src, but keep in mind the src namespace is not stable and those APIs may change or be removed in the future.

Major Features and Improvements

  • tf.keras

    • Added F-Score metrics tf.keras.metrics.FBetaScore, tf.keras.metrics.F1Score, and tf.keras.metrics.R2Score.
    • Added activation function tf.keras.activations.mish.
    • Added experimental keras.metrics.experimental.PyMetric API for metrics that run Python code on the host CPU (compiled outside of the TensorFlow graph). This can be used for integrating metrics from external Python libraries (like sklearn or pycocotools) into Keras as first-class Keras metrics.
    • Added tf.keras.optimizers.Lion optimizer.
    • Added tf.keras.layers.SpectralNormalization layer wrapper to perform spectral normalization on the weights of a target layer.
    • The SidecarEvaluatorModelExport callback has been added to Keras as keras.callbacks.SidecarEvaluatorModelExport. This callback allows for exporting the model the best-scoring model as evaluated by a SidecarEvaluator evaluator. The evaluator regularly evaluates the model and exports it if the user-defined comparison function determines that it is an improvement.
    • Added warmup capabilities to tf.keras.optimizers.schedules.CosineDecay learning rate scheduler. You can now specify an initial and target learning rate, and our scheduler will perform a linear interpolation between the two after which it will begin a decay phase.
    • Added experimental support for an exactly-once visitation guarantee for evaluating Keras models trained with tf.distribute ParameterServerStrategy, via the exact_evaluation_shards argument in Model.fit and Model.evaluate.
    • Added tf.keras.__internal__.KerasTensor,tf.keras.__internal__.SparseKerasTensor, and tf.keras.__internal__.RaggedKerasTensor classes. You can use these classes to do instance type checking and type annotations for layer/model inputs and outputs.
    • All the tf.keras.dtensor.experimental.optimizers classes have been merged with tf.keras.optimizers. You can migrate your code to use tf.keras.optimizers directly. The API namespace for tf.keras.dtensor.experimental.optimizers will be removed in future releases.
    • Added support for class_weight for 3+ dimensional targets (e.g. image segmentation masks) in Model.fit.
    • Added a new loss, keras.losses.CategoricalFocalCrossentropy.
    • Remove the tf.keras.dtensor.experimental.layout_map_scope(). You can user the tf.keras.dtensor.experimental.LayoutMap.scope() instead.

Security

  • N/A

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar, Aakar Dwivedi, Abinash Satapathy, Aditya Kane, ag.ramesh, Alexander Grund, Andrei Pikas, andreii, Andrew Goodbody, angerson, Anthony_256, Ashay Rane, Ashiq Imran, Awsaf, Balint Cristian, Banikumar Maiti (Intel Aipg), Ben Barsdell, bhack, cfRod, Chao Chen, chenchongsong, Chris Mc, Daniil Kutz, David Rubinstein, dianjiaogit, dixr, Dongfeng Yu, dongfengy, drah, Eric Kunze, Feiyue Chen, Frederic Bastien, Gauri1 Deshpande, guozhong.zhuang, hDn248, HYChou, ingkarat, James Hilliard, Jason Furmanek, Jaya, Jens Glaser, Jerry Ge, Jiao Dian'S Power Plant, Jie Fu, Jinzhe Zeng, Jukyy, Kaixi Hou, Kanvi Khanna, Karel Ha, karllessard, Koan-Sin Tan, Konstantin Beluchenko, Kulin Seth, Kun Lu, Kyle Gerard Felker, Leopold Cambier, Lianmin Zheng, linlifan, liuyuanqiang, Lukas Geiger, Luke Hutton, Mahmoud Abuzaina, Manas Mohanty, Mateo Fidabel, Maxiwell S. Garcia, Mayank Raunak, mdfaijul, meatybobby, Meenakshi Venkataraman, Michael Holman, Nathan John Sircombe, Nathan Luehr, nitins17, Om Thakkar, Patrice Vignola, Pavani Majety, per1234, Philipp Hack, pollfly, Prianka Liz Kariat, Rahul Batra, rahulbatra85, ratnam.parikh, Rickard Hallerbäck, Roger Iyengar, Rohit Santhanam, Roman Baranchuk, Sachin Muradi, sanadani, Saoirse Stewart, seanshpark, Shawn Wang, shuw, Srinivasan Narayanamoorthy, Stewart Miles, Sunita Nadampalli, SuryanarayanaY, Takahashi Shuuji, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, TJ, Tony Sung, Trevor Morris, unda, Vertexwahn, venkat2469, William Muir, Xavier Bonaventura, xiang.zhang, Xiao-Yong Jin, yleeeee, Yong Tang, Yuriy Chernyshov, Zhang, Xiangze, zhaozheng09

- C++
Published by tensorflow-jenkins almost 3 years ago

tensorflow - TensorFlow 2.12.0

Release 2.12.0

TensorFlow

Breaking Changes

  • Build, Compilation and Packaging

    • Removed redundant packages tensorflow-gpu and tf-nightly-gpu. These packages were removed and replaced with packages that direct users to switch to tensorflow or tf-nightly respectively. Since TensorFlow 2.1, the only difference between these two sets of packages was their names, so there is no loss of functionality or GPU support. See https://pypi.org/project/tensorflow-gpu for more details.

  • tf.function:

    • tf.function now uses the Python inspect library directly for parsing the signature of the Python function it is decorated on. This change may break code where the function signature is malformed, but was ignored previously, such as:
      • Using functools.wraps on a function with different signature
      • Using functools.partial with an invalid tf.function input
    • tf.function now enforces input parameter names to be valid Python identifiers. Incompatible names are automatically sanitized similarly to existing SavedModel signature behavior.
    • Parameterless tf.functions are assumed to have an empty input_signature instead of an undefined one even if the input_signature is unspecified.
    • tf.types.experimental.TraceType now requires an additional placeholder_value method to be defined.
    • tf.function now traces with placeholder values generated by TraceType instead of the value itself.

  • Experimental APIs tf.config.experimental.enable_mlir_graph_optimization and tf.config.experimental.disable_mlir_graph_optimization were removed.

Major Features and Improvements

  • Support for Python 3.11 has been added.

  • Support for Python 3.7 has been removed. We are not releasing any more patches for Python 3.7.

  • tf.lite:

    • Add 16-bit float type support for built-in op fill.
    • Transpose now supports 6D tensors.
    • Float LSTM now supports diagonal recurrent tensors: https://arxiv.org/abs/1903.08023

  • tf.experimental.dtensor:

    • Coordination service now works with dtensor.initialize_accelerator_system, and enabled by default.
    • Add tf.experimental.dtensor.is_dtensor to check if a tensor is a DTensor instance.

  • tf.data:

    • Added support for alternative checkpointing protocol which makes it possible to checkpoint the state of the input pipeline without having to store the contents of internal buffers. The new functionality can be enabled through the experimental_symbolic_checkpoint option of tf.data.Options().
    • Added a new rerandomize_each_iteration argument for the tf.data.Dataset.random() operation, which controls whether the sequence of generated random numbers should be re-randomized every epoch or not (the default behavior). If seed is set and rerandomize_each_iteration=True, the random() operation will produce a different (deterministic) sequence of numbers every epoch.
    • Added a new rerandomize_each_iteration argument for the tf.data.Dataset.sample_from_datasets() operation, which controls whether the sequence of generated random numbers used for sampling should be re-randomized every epoch or not. If seed is set and rerandomize_each_iteration=True, the sample_from_datasets() operation will use a different (deterministic) sequence of numbers every epoch.

  • tf.test:

    • Added tf.test.experimental.sync_devices, which is useful for accurately measuring performance in benchmarks.

  • tf.experimental.dtensor:

    • Added experimental support to ReduceScatter fuse on GPU (NCCL).

Bug Fixes and Other Changes

  • tf.SavedModel:
    • Introduced new class tf.saved_model.experimental.Fingerprint that contains the fingerprint of the SavedModel. See the SavedModel Fingerprinting RFC for details.
    • Introduced API tf.saved_model.experimental.read_fingerprint(export_dir) for reading the fingerprint of a SavedModel.
  • tf.random
    • Added non-experimental aliases for tf.random.split and tf.random.fold_in, the experimental endpoints are still available so no code changes are necessary.
  • tf.experimental.ExtensionType
    • Added function experimental.extension_type.as_dict(), which converts an instance of tf.experimental.ExtensionType to a dict representation.
  • stream_executor
    • Top level stream_executor directory has been deleted, users should use equivalent headers and targets under compiler/xla/stream_executor.
  • tf.nn
    • Added tf.nn.experimental.general_dropout, which is similar to tf.random.experimental.stateless_dropout but accepts a custom sampler function.
  • tf.types.experimental.GenericFunction
    • The experimental_get_compiler_ir method supports tf.TensorSpec compilation arguments.
  • tf.config.experimental.mlir_bridge_rollout
    • Removed enums MLIR_BRIDGE_ROLLOUT_SAFE_MODE_ENABLED and MLIR_BRIDGE_ROLLOUT_SAFE_MODE_FALLBACK_ENABLED which are no longer used by the tf2xla bridge

Keras

Keras is a framework built on top of the TensorFlow. See more details on the Keras website.

Breaking Changes

tf.keras:

  • Moved all saving-related utilities to a new namespace, keras.saving, for example: keras.saving.load_model, keras.saving.save_model, keras.saving.custom_object_scope, keras.saving.get_custom_objects, keras.saving.register_keras_serializable,keras.saving.get_registered_name and keras.saving.get_registered_object. The previous API locations (in keras.utils and keras.models) will be available indefinitely, but we recommend you update your code to point to the new API locations.
    • Improvements and fixes in Keras loss masking:
    • Whether you represent a ragged tensor as a tf.RaggedTensor or using keras masking, the returned loss values should be the identical to each other. In previous versions Keras may have silently ignored the mask.
    • If you use masked losses with Keras the loss values may be different in TensorFlow 2.12 compared to previous versions.
    • In cases where the mask was previously ignored, you will now get an error if you pass a mask with an incompatible shape.

Major Features and Improvements

tf.keras:

  • The new Keras model saving format (.keras) is available. You can start using it via model.save(f"{fname}.keras", save_format="keras_v3"). In the future it will become the default for all files with the .keras extension. This file format targets the Python runtime only and makes it possible to reload Python objects identical to the saved originals. The format supports non-numerical state such as vocabulary files and lookup tables, and it is easy to customize in the case of custom layers with exotic elements of state (e.g. a FIFOQueue). The format does not rely on bytecode or pickling, and is safe by default. Note that as a result, Python lambdas are disallowed at loading time. If you want to use lambdas, you can pass safe_mode=False to the loading method (only do this if you trust the source of the model).
    • Added a model.export(filepath) API to create a lightweight SavedModel artifact that can be used for inference (e.g. with TF-Serving).
    • Added keras.export.ExportArchive class for low-level customization of the process of exporting SavedModel artifacts for inference. Both ways of exporting models are based on tf.function tracing and produce a TF program composed of TF ops. They are meant primarily for environments where the TF runtime is available, but not the Python interpreter, as is typical for production with TF Serving.
  • Added utility tf.keras.utils.FeatureSpace, a one-stop shop for structured data preprocessing and encoding.
  • Added tf.SparseTensor input support to tf.keras.layers.Embedding layer. The layer now accepts a new boolean argument sparse. If sparse is set to True, the layer returns a SparseTensor instead of a dense Tensor. Defaults to False.
  • Added jit_compile as a settable property to tf.keras.Model.
  • Added synchronized optional parameter to layers.BatchNormalization.
  • Added deprecation warning to layers.experimental.SyncBatchNormalization and suggested to use layers.BatchNormalization with synchronized=True instead.
  • Updated tf.keras.layers.BatchNormalization to support masking of the inputs (mask argument) when computing the mean and variance.
  • Add tf.keras.layers.Identity, a placeholder pass-through layer.
  • Add show_trainable option to tf.keras.utils.model_to_dot to display layer trainable status in model plots.
  • Add ability to save a tf.keras.utils.FeatureSpace object, via feature_space.save("myfeaturespace.keras"), and reload it via feature_space = tf.keras.models.load_model("myfeaturespace.keras").
    • Added utility tf.keras.utils.to_ordinal to convert class vector to ordinal regression / classification matrix.

Bug Fixes and Other Changes

  • N/A

Security

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar, Aakar Dwivedi, Abinash Satapathy, Aditya Kane, ag.ramesh, Alexander Grund, Andrei Pikas, andreii, Andrew Goodbody, angerson, Anthony_256, Ashay Rane, Ashiq Imran, Awsaf, Balint Cristian, Banikumar Maiti (Intel Aipg), Ben Barsdell, bhack, cfRod, Chao Chen, chenchongsong, Chris Mc, Daniil Kutz, David Rubinstein, dianjiaogit, dixr, Dongfeng Yu, dongfengy, drah, Eric Kunze, Feiyue Chen, Frederic Bastien, Gauri1 Deshpande, guozhong.zhuang, hDn248, HYChou, ingkarat, James Hilliard, Jason Furmanek, Jaya, Jens Glaser, Jerry Ge, Jiao Dian'S Power Plant, Jie Fu, Jinzhe Zeng, Jukyy, Kaixi Hou, Kanvi Khanna, Karel Ha, karllessard, Koan-Sin Tan, Konstantin Beluchenko, Kulin Seth, Kun Lu, Kyle Gerard Felker, Leopold Cambier, Lianmin Zheng, linlifan, liuyuanqiang, Lukas Geiger, Luke Hutton, Mahmoud Abuzaina, Manas Mohanty, Mateo Fidabel, Maxiwell S. Garcia, Mayank Raunak, mdfaijul, meatybobby, Meenakshi Venkataraman, Michael Holman, Nathan John Sircombe, Nathan Luehr, nitins17, Om Thakkar, Patrice Vignola, Pavani Majety, per1234, Philipp Hack, pollfly, Prianka Liz Kariat, Rahul Batra, rahulbatra85, ratnam.parikh, Rickard Hallerbäck, Roger Iyengar, Rohit Santhanam, Roman Baranchuk, Sachin Muradi, sanadani, Saoirse Stewart, seanshpark, Shawn Wang, shuw, Srinivasan Narayanamoorthy, Stewart Miles, Sunita Nadampalli, SuryanarayanaY, Takahashi Shuuji, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, TJ, Tony Sung, Trevor Morris, unda, Vertexwahn, Vinila S, William Muir, Xavier Bonaventura, xiang.zhang, Xiao-Yong Jin, yleeeee, Yong Tang, Yuriy Chernyshov, Zhang, Xiangze, zhaozheng09

- C++
Published by mihaimaruseac almost 3 years ago

tensorflow - TensorFlow 2.11.1

Release 2.11.1

Note: TensorFlow 2.10 was the last TensorFlow release that supported GPU on native-Windows. Starting with TensorFlow 2.11, you will need to install TensorFlow in WSL2, or install tensorflow-cpu and, optionally, try the TensorFlow-DirectML-Plugin. * Security vulnerability fixes will no longer be patched to this Tensorflow version. The latest Tensorflow version includes the security vulnerability fixes. You can update to the latest version (recommended) or patch security vulnerabilities yourself steps. You can refer to the release notes of the latest Tensorflow version for a list of newly fixed vulnerabilities. If you have any questions, please create a GitHub issue to let us know.

This release also introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins almost 3 years ago

tensorflow - TensorFlow 2.12.0-rc1

Release 2.12.0

Breaking Changes

  • Build, Compilation and Packaging

    • Removal of redundant packages: the tensorflow-gpu and tf-nightly-gpu packages have been effectively removed and replaced with packages that direct users to switch to tensorflow or tf-nightly respectively. The naming difference was the only difference between the two sets of packages ever since TensorFlow 2.1, so there is no loss of functionality or GPU support. See https://pypi.org/project/tensorflow-gpu for more details.

  • tf.function:

    • tf.function now uses the Python inspect library directly for parsing the signature of the Python function it is decorated on.
    • This can break certain cases that were previously ignored where the signature is malformed, such as:
      • Using functools.wraps on a function with different signature
      • Using functools.partial with an invalid tf.function input
    • tf.function now enforces input parameter names to be valid Python identifiers. Incompatible names are automatically sanitized similarly to existing SavedModel signature behavior.
    • Parameterless tf.functions are assumed to have an empty input_signature instead of an undefined one even if the input_signature is unspecified.
    • tf.types.experimental.TraceType now requires an additional placeholder_value method to be defined.
    • tf.function now traces with placeholder values generated by TraceType instead of the value itself.

  • Experimental APIs tf.config.experimental.enable_mlir_graph_optimization and tf.config.experimental.disable_mlir_graph_optimization were removed.

  • tf.keras:

    • Moved all saving-related utilities to a new namespace, keras.saving, i.e. keras.saving.load_model, keras.saving.save_model, keras.saving.custom_object_scope, keras.saving.get_custom_objects, keras.saving.register_keras_serializable,keras.saving.get_registered_name and keras.saving.get_registered_object. The previous API locations (in keras.utils and keras.models) will stay available indefinitely, but we recommend that you update your code to point to the new API locations.
    • Improvements and fixes in Keras loss masking:
      • Whether you represent a ragged tensor as a tf.RaggedTensor or using keras masking, the returned loss values should be the identical to each other. In previous versions Keras may have silently ignored the mask.
      • If you use masked losses with Keras the loss values may be different in TensorFlow 2.12 compared to previous versions.
      • In cases where the mask was previously ignored, you will now get an error if you pass a mask with an incompatible shape.

Major Features and Improvements

  • tf.lite:

    • Add 16-bit float type support for built-in op fill.
    • Transpose now supports 6D tensors.
    • Float LSTM now supports diagonal recurrent tensors: https://arxiv.org/abs/1903.08023

  • tf.keras:

    • The new Keras model saving format (.keras) is available. You can start using it via model.save(f"{fname}.keras", save_format="keras_v3"). In the future it will become the default for all files with the .keras extension. This file format targets the Python runtime only and makes it possible to reload Python objects identical to the saved originals. The format supports non-numerical state such as vocabulary files and lookup tables, and it is easy to customize in the case of custom layers with exotic elements of state (e.g. a FIFOQueue). The format does not rely on bytecode or pickling, and is safe by default. Note that as a result, Python lambdas are disallowed at loading time. If you want to use lambdas, you can pass safe_mode=False to the loading method (only do this if you trust the source of the model).
    • Added a model.export(filepath) API to create a lightweight SavedModel artifact that can be used for inference (e.g. with TF-Serving).
    • Added keras.export.ExportArchive class for low-level customization of the process of exporting SavedModel artifacts for inference. Both ways of exporting models are based on tf.function tracing and produce a TF program composed of TF ops. They are meant primarily for environments where the TF runtime is available, but not the Python interpreter, as is typical for production with TF Serving.
    • Added utility tf.keras.utils.FeatureSpace, a one-stop shop for structured data preprocessing and encoding.
    • Added tf.SparseTensor input support to tf.keras.layers.Embedding layer. The layer now accepts a new boolean argument sparse. If sparse is set to True, the layer returns a SparseTensor instead of a dense Tensor. Defaults to False.
    • Added jit_compile as a settable property to tf.keras.Model.
    • Added synchronized optional parameter to layers.BatchNormalization.
    • Added deprecation warning to layers.experimental.SyncBatchNormalization and suggested to use layers.BatchNormalization with synchronized=True instead.
    • Updated tf.keras.layers.BatchNormalization to support masking of the inputs (mask argument) when computing the mean and variance.
    • Add tf.keras.layers.Identity, a placeholder pass-through layer.
    • Add show_trainable option to tf.keras.utils.model_to_dot to display layer trainable status in model plots.
    • Add ability to save a tf.keras.utils.FeatureSpace object, via feature_space.save("myfeaturespace.keras"), and reload it via feature_space = tf.keras.models.load_model("myfeaturespace.keras").
    • Added utility tf.keras.utils.to_ordinal to convert class vector to ordinal regression / classification matrix.

  • tf.experimental.dtensor:

    • Coordination service now works with dtensor.initialize_accelerator_system, and enabled by default.
    • Add tf.experimental.dtensor.is_dtensor to check if a tensor is a DTensor instance.

  • tf.data:

    • Added support for alternative checkpointing protocol which makes it possible to checkpoint the state of the input pipeline without having to store the contents of internal buffers. The new functionality can be enabled through the experimental_symbolic_checkpoint option of tf.data.Options().
    • Added a new rerandomize_each_iteration argument for the tf.data.Dataset.random() operation, which controls whether the sequence of generated random numbers should be re-randomized every epoch or not (the default behavior). If seed is set and rerandomize_each_iteration=True, the random() operation will produce a different (deterministic) sequence of numbers every epoch.
    • Added a new rerandomize_each_iteration argument for the tf.data.Dataset.sample_from_datasets() operation, which controls whether the sequence of generated random numbers used for sampling should be re-randomized every epoch or not. If seed is set and rerandomize_each_iteration=True, the sample_from_datasets() operation will use a different (deterministic) sequence of numbers every epoch.

  • tf.test:

    • Added tf.test.experimental.sync_devices, which is useful for accurately measuring performance in benchmarks.

  • tf.experimental.dtensor:

    • Added experimental support to ReduceScatter fuse on GPU (NCCL).

Bug Fixes and Other Changes

  • tf.SavedModel:
    • Introduced new class tf.saved_model.experimental.Fingerprint that contains the fingerprint of the SavedModel. See the SavedModel Fingerprinting RFC for details.
    • Introduced API tf.saved_model.experimental.read_fingerprint(export_dir) for reading the fingerprint of a SavedModel.
  • tf.random
    • Added non-experimental aliases for tf.random.split and tf.random.fold_in, the experimental endpoints are still available so no code changes are necessary.
  • tf.experimental.ExtensionType
    • Added function experimental.extension_type.as_dict(), which converts an instance of tf.experimental.ExtensionType to a dict representation.
  • stream_executor
    • Top level stream_executor directory has been deleted, users should use equivalent headers and targets under compiler/xla/stream_executor.
  • tf.nn
    • Added tf.nn.experimental.general_dropout, which is similar to tf.random.experimental.stateless_dropout but accepts a custom sampler function.
  • tf.types.experimental.GenericFunction
    • The experimental_get_compiler_ir method supports tf.TensorSpec compilation arguments.
  • tf.config.experimental.mlir_bridge_rollout
    • Removed enums MLIR_BRIDGE_ROLLOUT_SAFE_MODE_ENABLED and MLIR_BRIDGE_ROLLOUT_SAFE_MODE_FALLBACK_ENABLED which are no longer used by the tf2xla bridge

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar, Aakar Dwivedi, Abinash Satapathy, Aditya Kane, ag.ramesh, Alexander Grund, Andrei Pikas, andreii, Andrew Goodbody, angerson, Anthony_256, Ashay Rane, Ashiq Imran, Awsaf, Balint Cristian, Banikumar Maiti (Intel Aipg), Ben Barsdell, bhack, cfRod, Chao Chen, chenchongsong, Chris Mc, Daniil Kutz, David Rubinstein, dianjiaogit, dixr, Dongfeng Yu, dongfengy, drah, Eric Kunze, Feiyue Chen, Frederic Bastien, Gauri1 Deshpande, guozhong.zhuang, hDn248, HYChou, ingkarat, James Hilliard, Jason Furmanek, Jaya, Jens Glaser, Jerry Ge, Jiao Dian'S Power Plant, Jie Fu, Jinzhe Zeng, Jukyy, Kaixi Hou, Kanvi Khanna, Karel Ha, karllessard, Koan-Sin Tan, Konstantin Beluchenko, Kulin Seth, Kun Lu, Kyle Gerard Felker, Leopold Cambier, Lianmin Zheng, linlifan, liuyuanqiang, Lukas Geiger, Luke Hutton, Mahmoud Abuzaina, Manas Mohanty, Mateo Fidabel, Maxiwell S. Garcia, Mayank Raunak, mdfaijul, meatybobby, Meenakshi Venkataraman, Michael Holman, Nathan John Sircombe, Nathan Luehr, nitins17, Om Thakkar, Patrice Vignola, Pavani Majety, per1234, Philipp Hack, pollfly, Prianka Liz Kariat, Rahul Batra, rahulbatra85, ratnam.parikh, Rickard Hallerbäck, Roger Iyengar, Rohit Santhanam, Roman Baranchuk, Sachin Muradi, sanadani, Saoirse Stewart, seanshpark, Shawn Wang, shuw, Srinivasan Narayanamoorthy, Stewart Miles, Sunita Nadampalli, SuryanarayanaY, Takahashi Shuuji, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, TJ, Tony Sung, Trevor Morris, unda, Vertexwahn, Vinila S, William Muir, Xavier Bonaventura, xiang.zhang, Xiao-Yong Jin, yleeeee, Yong Tang, Yuriy Chernyshov, Zhang, Xiangze, zhaozheng09

- C++
Published by tensorflow-jenkins almost 3 years ago

tensorflow - TensorFlow 2.12.0-rc0

Release 2.12.0

Breaking Changes

  • Build, Compilation and Packaging

    • Removal of redundant packages: the tensorflow-gpu and tf-nightly-gpu packages have been effectively removed and replaced with packages that direct users to switch to tensorflow or tf-nightly respectively. The naming difference was the only difference between the two sets of packages ever since TensorFlow 2.1, so there is no loss of functionality or GPU support. See https://pypi.org/project/tensorflow-gpu for more details.

  • tf.function:

    • tf.function now uses the Python inspect library directly for parsing the signature of the Python function it is decorated on.
    • This can break certain cases that were previously ignored where the signature is malformed, such as:
      • Using functools.wraps on a function with different signature
      • Using functools.partial with an invalid tf.function input
    • tf.function now enforces input parameter names to be valid Python identifiers. Incompatible names are automatically sanitized similarly to existing SavedModel signature behavior.
    • Parameterless tf.functions are assumed to have an empty input_signature instead of an undefined one even if the input_signature is unspecified.
    • tf.types.experimental.TraceType now requires an additional placeholder_value method to be defined.
    • tf.function now traces with placeholder values generated by TraceType instead of the value itself.

  • Experimental APIs tf.config.experimental.enable_mlir_graph_optimization and tf.config.experimental.disable_mlir_graph_optimization were removed.

  • tf.keras:

    • Moved all saving-related utilities to a new namespace, keras.saving, i.e. keras.saving.load_model, keras.saving.save_model, keras.saving.custom_object_scope, keras.saving.get_custom_objects, keras.saving.register_keras_serializable,keras.saving.get_registered_name and keras.saving.get_registered_object. The previous API locations (in keras.utils and keras.models) will stay available indefinitely, but we recommend that you update your code to point to the new API locations.
    • Improvements and fixes in Keras loss masking:
      • Whether you represent a ragged tensor as a tf.RaggedTensor or using keras masking, the returned loss values should be the identical to each other. In previous versions Keras may have silently ignored the mask.
      • If you use masked losses with Keras the loss values may be different in TensorFlow 2.12 compared to previous versions.
      • In cases where the mask was previously ignored, you will now get an error if you pass a mask with an incompatible shape.

  • tf.SavedModel:

    • Introduced new class tf.saved_model.experimental.Fingerprint that contains the fingerprint of the SavedModel. See the SavedModel Fingerprinting RFC for details.
    • Introduced API tf.saved_model.experimental.read_fingerprint(export_dir) for reading the fingerprint of a SavedModel.

Major Features and Improvements

  • tf.lite:

    • Add 16-bit float type support for built-in op fill.
    • Transpose now supports 6D tensors.
    • Float LSTM now supports diagonal recurrent tensors: https://arxiv.org/abs/1903.08023

  • tf.keras:

    • The new Keras model saving format (.keras) is available. You can start using it via model.save(f"{fname}.keras", save_format="keras_v3"). In the future it will become the default for all files with the .keras extension. This file format targets the Python runtime only and makes it possible to reload Python objects identical to the saved originals. The format supports non-numerical state such as vocabulary files and lookup tables, and it is easy to customize in the case of custom layers with exotic elements of state (e.g. a FIFOQueue). The format does not rely on bytecode or pickling, and is safe by default. Note that as a result, Python lambdas are disallowed at loading time. If you want to use lambdas, you can pass safe_mode=False to the loading method (only do this if you trust the source of the model).
    • Added a model.export(filepath) API to create a lightweight SavedModel artifact that can be used for inference (e.g. with TF-Serving).
    • Added keras.export.ExportArchive class for low-level customization of the process of exporting SavedModel artifacts for inference. Both ways of exporting models are based on tf.function tracing and produce a TF program composed of TF ops. They are meant primarily for environments where the TF runtime is available, but not the Python interpreter, as is typical for production with TF Serving.
    • Added utility tf.keras.utils.FeatureSpace, a one-stop shop for structured data preprocessing and encoding.
    • Added tf.SparseTensor input support to tf.keras.layers.Embedding layer. The layer now accepts a new boolean argument sparse. If sparse is set to True, the layer returns a SparseTensor instead of a dense Tensor. Defaults to False.
    • Added jit_compile as a settable property to tf.keras.Model.
    • Added synchronized optional parameter to layers.BatchNormalization.
    • Added deprecation warning to layers.experimental.SyncBatchNormalization and suggested to use layers.BatchNormalization with synchronized=True instead.
    • Updated tf.keras.layers.BatchNormalization to support masking of the inputs (mask argument) when computing the mean and variance.
    • Add tf.keras.layers.Identity, a placeholder pass-through layer.
    • Add show_trainable option to tf.keras.utils.model_to_dot to display layer trainable status in model plots.
    • Add ability to save a tf.keras.utils.FeatureSpace object, via feature_space.save("myfeaturespace.keras"), and reload it via feature_space = tf.keras.models.load_model("myfeaturespace.keras").
    • Added utility tf.keras.utils.to_ordinal to convert class vector to ordinal regression / classification matrix.

  • tf.experimental.dtensor:

    • Coordination service now works with dtensor.initialize_accelerator_system, and enabled by default.
    • Add tf.experimental.dtensor.is_dtensor to check if a tensor is a DTensor instance.

  • tf.data:

    • Added support for alternative checkpointing protocol which makes it possible to checkpoint the state of the input pipeline without having to store the contents of internal buffers. The new functionality can be enabled through the experimental_symbolic_checkpoint option of tf.data.Options().
    • Added a new rerandomize_each_iteration argument for the tf.data.Dataset.random() operation, which controls whether the sequence of generated random numbers should be re-randomized every epoch or not (the default behavior). If seed is set and rerandomize_each_iteration=True, the random() operation will produce a different (deterministic) sequence of numbers every epoch.
    • Added a new rerandomize_each_iteration argument for the tf.data.Dataset.sample_from_datasets() operation, which controls whether the sequence of generated random numbers used for sampling should be re-randomized every epoch or not. If seed is set and rerandomize_each_iteration=True, the sample_from_datasets() operation will use a different (deterministic) sequence of numbers every epoch.

  • tf.test:

    • Added tf.test.experimental.sync_devices, which is useful for accurately measuring performance in benchmarks.

  • tf.experimental.dtensor:

    • Added experimental support to ReduceScatter fuse on GPU (NCCL).

Bug Fixes and Other Changes

  • tf.random
    • Added non-experimental aliases for tf.random.split and tf.random.fold_in, the experimental endpoints are still available so no code changes are necessary.
  • tf.experimental.ExtensionType
    • Added function experimental.extension_type.as_dict(), which converts an instance of tf.experimental.ExtensionType to a dict representation.
  • stream_executor
    • Top level stream_executor directory has been deleted, users should use equivalent headers and targets under compiler/xla/stream_executor.
  • tf.nn
    • Added tf.nn.experimental.general_dropout, which is similar to tf.random.experimental.stateless_dropout but accepts a custom sampler function.
  • tf.types.experimental.GenericFunction
    • The experimental_get_compiler_ir method supports tf.TensorSpec compilation arguments.
  • tf.config.experimental.mlir_bridge_rollout
    • Removed enums MLIR_BRIDGE_ROLLOUT_SAFE_MODE_ENABLED and MLIR_BRIDGE_ROLLOUT_SAFE_MODE_FALLBACK_ENABLED which are no longer used by the tf2xla bridge

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar, Aakar Dwivedi, Abinash Satapathy, Aditya Kane, ag.ramesh, Alexander Grund, Andrei Pikas, andreii, Andrew Goodbody, angerson, Anthony_256, Ashay Rane, Ashiq Imran, Awsaf, Balint Cristian, Banikumar Maiti (Intel Aipg), Ben Barsdell, bhack, cfRod, Chao Chen, chenchongsong, Chris Mc, Daniil Kutz, David Rubinstein, dianjiaogit, dixr, Dongfeng Yu, dongfengy, drah, Eric Kunze, Feiyue Chen, Frederic Bastien, Gauri1 Deshpande, guozhong.zhuang, hDn248, HYChou, ingkarat, James Hilliard, Jason Furmanek, Jaya, Jens Glaser, Jerry Ge, Jiao Dian'S Power Plant, Jie Fu, Jinzhe Zeng, Jukyy, Kaixi Hou, Kanvi Khanna, Karel Ha, karllessard, Koan-Sin Tan, Konstantin Beluchenko, Kulin Seth, Kun Lu, Kyle Gerard Felker, Leopold Cambier, Lianmin Zheng, linlifan, liuyuanqiang, Lukas Geiger, Luke Hutton, Mahmoud Abuzaina, Manas Mohanty, Mateo Fidabel, Maxiwell S. Garcia, Mayank Raunak, mdfaijul, meatybobby, Meenakshi Venkataraman, Michael Holman, Nathan John Sircombe, Nathan Luehr, nitins17, Om Thakkar, Patrice Vignola, Pavani Majety, per1234, Philipp Hack, pollfly, Prianka Liz Kariat, Rahul Batra, rahulbatra85, ratnam.parikh, Rickard Hallerbäck, Roger Iyengar, Rohit Santhanam, Roman Baranchuk, Sachin Muradi, sanadani, Saoirse Stewart, seanshpark, Shawn Wang, shuw, Srinivasan Narayanamoorthy, Stewart Miles, Sunita Nadampalli, SuryanarayanaY, Takahashi Shuuji, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tirumalesh, TJ, Tony Sung, Trevor Morris, unda, Vertexwahn, Vinila S, William Muir, Xavier Bonaventura, xiang.zhang, Xiao-Yong Jin, yleeeee, Yong Tang, Yuriy Chernyshov, Zhang, Xiangze, zhaozheng09

- C++
Published by tensorflow-jenkins about 3 years ago

tensorflow - TensorFlow 2.11.0

Release 2.11.0

Breaking Changes

  • The tf.keras.optimizers.Optimizer base class now points to the new Keras optimizer, while the old optimizers have been moved to the tf.keras.optimizers.legacy namespace.

    If you find your workflow failing due to this change, you may be facing one of the following issues:

    • Checkpoint loading failure. The new optimizer handles optimizer state differently from the old optimizer, which simplifies the logic of checkpoint saving/loading, but at the cost of breaking checkpoint backward compatibility in some cases. If you want to keep using an old checkpoint, please change your optimizer to tf.keras.optimizer.legacy.XXX (e.g. tf.keras.optimizer.legacy.Adam).
    • TF1 compatibility. The new optimizer, tf.keras.optimizers.Optimizer, does not support TF1 any more, so please use the legacy optimizer tf.keras.optimizer.legacy.XXX. We highly recommend migrating your workflow to TF2 for stable support and new features.
    • Old optimizer API not found. The new optimizer, tf.keras.optimizers.Optimizer, has a different set of public APIs from the old optimizer. These API changes are mostly related to getting rid of slot variables and TF1 support. Please check the API documentation to find alternatives to the missing API. If you must call the deprecated API, please change your optimizer to the legacy optimizer.
    • Learning rate schedule access. When using a tf.keras.optimizers.schedules.LearningRateSchedule, the new optimizer's learning_rate property returns the current learning rate value instead of a LearningRateSchedule object as before. If you need to access the LearningRateSchedule object, please use optimizer._learning_rate.
    • If you implemented a custom optimizer based on the old optimizer. Please set your optimizer to subclass tf.keras.optimizer.legacy.XXX. If you want to migrate to the new optimizer and find it does not support your optimizer, please file an issue in the Keras GitHub repo.
    • Errors, such as Cannot recognize variable.... The new optimizer requires all optimizer variables to be created at the first apply_gradients() or minimize() call. If your workflow calls the optimizer to update different parts of the model in multiple stages, please call optimizer.build(model.trainable_variables) before the training loop.
    • Timeout or performance loss. We don't anticipate this to happen, but if you see such issues, please use the legacy optimizer, and file an issue in the Keras GitHub repo.

    The old Keras optimizer will never be deleted, but will not see any new feature additions. New optimizers (for example, tf.keras.optimizers.Adafactor) will only be implemented based on the new tf.keras.optimizers.Optimizer base class.

  • tensorflow/python/keras code is a legacy copy of Keras since the TensorFlow v2.7 release, and will be deleted in the v2.12 release. Please remove any import of tensorflow.python.keras and use the public API with from tensorflow import keras or import tensorflow as tf; tf.keras.

Major Features and Improvements

  • tf.lite:

    • New operations supported: tf.math.unsorted_segment_sum, tf.atan2 and tf.sign.
    • Updates to existing operations:
      • tfl.mul now supports complex32 inputs.

  • tf.experimental.StructuredTensor:

    • Introduced tf.experimental.StructuredTensor, which provides a flexible and TensorFlow-native way to encode structured data such as protocol buffers or pandas dataframes.

  • tf.keras:

    • Added a new get_metrics_result() method to tf.keras.models.Model.
      • Returns the current metrics values of the model as a dict.
    • Added a new group normalization layer - tf.keras.layers.GroupNormalization.
    • Added weight decay support for all Keras optimizers via the weight_decay argument.
    • Added the Adafactor optimizer - tf.keras.optimizers.Adafactor.
    • Added warmstart_embedding_matrix to tf.keras.utils.
      • This utility can be used to warmstart an embedding matrix, so you reuse previously-learned word embeddings when working with a new set of words which may include previously unseen words (the embedding vectors for unseen words will be randomly initialized).

  • tf.Variable:

    • Added CompositeTensor as a base class to ResourceVariable.
      • This allows tf.Variables to be nested in tf.experimental.ExtensionTypes.
    • Added a new constructor argument experimental_enable_variable_lifting to tf.Variable, defaulting to True.
      • When it's set to False, the variable won't be lifted out of tf.function; thus it can be used as a tf.function-local variable: during each execution of the tf.function, the variable will be created and then disposed, similar to a local (that is, stack-allocated) variable in C/C++. Currently, experimental_enable_variable_lifting=False only works on non-XLA devices (for example, under @tf.function(jit_compile=False)).

  • TF SavedModel:

    • Added fingerprint.pb to the SavedModel directory. The fingerprint.pb file is a protobuf containing the "fingerprint" of the SavedModel. See the RFC for more details regarding its design and properties.

  • TF pip:

    • Windows CPU-builds for x86/x64 processors are now built, maintained, tested and released by a third party: Intel. Installing the Windows-native pip packages for tensorflow or tensorflow-cpu would install Intel's tensorflow-intel package. These packages are provided on an as-is basis. TensorFlow will use reasonable efforts to maintain the availability and integrity of this pip package. There may be delays if the third party fails to release the pip package. For using TensorFlow GPU on Windows, you will need to install TensorFlow in WSL2.

Bug Fixes and Other Changes

  • tf.image:

    • Added an optional parameter return_index_map to tf.image.ssim, which causes the returned value to be the local SSIM map instead of the global mean.

  • TF Core:

    • tf.custom_gradient can now be applied to functions that accept "composite" tensors, such as tf.RaggedTensor, as inputs.
    • Fix device placement issues related to datasets with ragged tensors of strings (i.e. variant encoded data with types not supported on GPU).
    • experimental_follow_type_hints for tf.function has been deprecated. Please use input_signature or reduce_retracing to minimize retracing.

  • tf.SparseTensor:

    • Introduced set_shape, which sets the static dense shape of the sparse tensor and has the same semantics as tf.Tensor.set_shape.

Security

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar Dwivedi, Alexander Grund, alif_elham, Aman Agarwal, amoitra, Andrei Ivanov, andreii, Andrew Goodbody, angerson, Ashay Rane, Azeem Shaikh, Ben Barsdell, bhack, Bhavani Subramanian, Cedric Nugteren, Chandra Kumar Ramasamy, Christopher Bate, CohenAriel, Cotarou, cramasam, Enrico Minack, Francisco Unda, Frederic Bastien, gadagashwini, Gauri1 Deshpande, george, Jake, Jeff, Jerry Ge, Jingxuan He, Jojimon Varghese, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, kcoul, Keith Smiley, Kevin Hu, Kun Lu, kushanam, Lianmin Zheng, liuyuanqiang, Louis Sugy, Mahmoud Abuzaina, Marius Brehler, mdfaijul, Meenakshi Venkataraman, Milos Puzovic, mohantym, Namrata-Ibm, Nathan John Sircombe, Nathan Luehr, Olaf Lipinski, Om Thakkar, Osman F Bayram, Patrice Vignola, Pavani Majety, Philipp Hack, Prianka Liz Kariat, Rahul Batra, RajeshT, Renato Golin, riestere, Roger Iyengar, Rohit Santhanam, Rsanthanam-Amd, Sadeed Pv, Samuel Marks, Shimokawa, Naoaki, Siddhesh Kothadi, Simengliu-Nv, Sindre Seppola, snadampal, Srinivasan Narayanamoorthy, sushreebarsa, syedshahbaaz, Tamas Bela Feher, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tom Anderson, Tomohiro Endo, Trevor Morris, vibhutisawant, Victor Zhang, Vremold, Xavier Bonaventura, Yanming Wang, Yasir Modak, Yimei Sun, Yong Tang, Yulv-Git, zhuoran.liu, zotanika

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.10.1

Release 2.10.1

This release introduces several vulnerability fixes: * Fixes an OOB seg fault in DynamicStitch due to missing validation (CVE-2022-41883) * Fixes an overflow in tf.keras.losses.poisson (CVE-2022-41887) * Fixes a heap OOB failure in ThreadUnsafeUnigramCandidateSampler caused by missing validation (CVE-2022-41880) * Fixes a segfault in ndarray_tensor_bridge (CVE-2022-41884) * Fixes an overflow in FusedResizeAndPadConv2D (CVE-2022-41885) * Fixes a overflow in ImageProjectiveTransformV2 (CVE-2022-41886) * Fixes an FPE in tf.image.generate_bounding_box_proposals on GPU (CVE-2022-41888) * Fixes a segfault in pywrap_tfe_src caused by invalid attributes (CVE-2022-41889) * Fixes a CHECK fail in BCast (CVE-2022-41890) * Fixes a segfault in TensorListConcat (CVE-2022-41891) * Fixes a CHECK_EQ fail in TensorListResize (CVE-2022-41893) * Fixes an overflow in CONV_3D_TRANSPOSE on TFLite (CVE-2022-41894) * Fixes a heap OOB in MirrorPadGrad (CVE-2022-41895) * Fixes a crash in Mfcc (CVE-2022-41896) * Fixes a heap OOB in FractionalMaxPoolGrad (CVE-2022-41897) * Fixes a CHECK fail in SparseFillEmptyRowsGrad (CVE-2022-41898) * Fixes a CHECK fail in SdcaOptimizer (CVE-2022-41899) * Fixes a heap OOB in FractionalAvgPool and FractionalMaxPool(CVE-2022-41900) * Fixes a CHECK_EQ in SparseMatrixNNZ (CVE-2022-41901) * Fixes an OOB write in grappler (CVE-2022-41902) * Fixes a overflow in ResizeNearestNeighborGrad (CVE-2022-41907) * Fixes a CHECK fail in PyFunc (CVE-2022-41908) * Fixes a segfault in CompositeTensorVariantToComponents (CVE-2022-41909) * Fixes a invalid char to bool conversion in printing a tensor (CVE-2022-41911) * Fixes a heap overflow in QuantizeAndDequantizeV2 (CVE-2022-41910) * Fixes a CHECK failure in SobolSample via missing validation (CVE-2022-35935) * Fixes a CHECK fail in TensorListScatter and TensorListScatterV2 in eager mode (CVE-2022-35935)

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.9.3

Release 2.9.3

This release introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.8.4

Release 2.8.4

This release introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.11.0-rc2

Release 2.11.0

Breaking Changes

  • tf.keras.optimizers.Optimizer now points to the new Keras optimizer, and old optimizers have moved to the tf.keras.optimizers.legacy namespace.
    If you find your workflow failing due to this change, you may be facing one of the following issues:

    • Checkpoint loading failure. The new optimizer handles optimizer state differently from the old optimizer, which simplifies the logic of checkpoint saving/loading, but at the cost of breaking checkpoint backward compatibility in some cases. If you want to keep using an old checkpoint, please change your optimizer to tf.keras.optimizer.legacy.XXX (e.g. tf.keras.optimizer.legacy.Adam).
    • TF1 compatibility. The new optimizer, tf.keras.optimizers.Optimizer, does not support TF1 any more, so please use the legacy optimizer tf.keras.optimizer.legacy.XXX. We highly recommend to migrate your workflow to TF2 for stable support and new features.
    • Old optimizer API not found. The new optimizer, tf.keras.optimizers.Optimizer, has a different set of public APIs from the old optimizer. These API changes are mostly related to getting rid of slot variables and TF1 support. Please check the API documentation to find alternatives to the missing API. If you must call the deprecated API, please change your optimizer to the legacy optimizer.
    • Learning rate schedule access. When using a LearningRateSchedule, The new optimizer's learning_rate property returns the current learning rate value instead of a LearningRateSchedule object as before. If you need to access the LearningRateSchedule object, please use optimizer._learning_rate.
    • If you implemented a custom optimizer based on the old optimizer. Please set your optimizer to subclass tf.keras.optimizer.legacy.XXX. If you want to migrate to the new optimizer and find it does not support your optimizer, please file an issue in the Keras GitHub repo.
    • Errors, such as Cannot recognize variable.... The new optimizer requires all optimizer variables to be created at the first apply_gradients() or minimize() call. If your workflow calls the optimizer to update different parts of the model in multiple stages, please call optimizer.build(model.trainable_variables) before the training loop.
    • Timeout or performance loss. We don't anticipate this to happen, but if you see such issues, please use the legacy optimizer, and file an issue in the Keras GitHub repo.

    The old Keras optimizer will never be deleted, but will not see any new feature additions. New optimizers (for example, tf.keras.optimizers.Adafactor) will only be implemented based on tf.keras.optimizers.Optimizer, the new base class.

  • tensorflow/python/keras code is a legacy copy of Keras since 2.7 release, and will be deleted in 2.12 release. Please remove any import of tensorflow.python.keras and use public API with from tensorflow import keras or import tensorflow as tf; tf.keras.

Major Features and Improvements

  • tf.lite:

    • New operations supported: tf.unsortedsegmentmin, tf.atan2 and tf.sign.
    • Updates to existing operations:
      • tfl.mul now supports complex32 inputs.

  • tf.experimental.StructuredTensor

    • Introduced tf.experimental.StructuredTensor, which provides a flexible and TensorFlow-native way to encode structured data such as protocol buffers or pandas dataframes.

  • tf.keras:

    • Added a new get_metrics_result() method to tf.keras.models.Model.
      • Returns the current metrics values of the model as a dict.
    • Added a new group normalization layer - tf.keras.layers.GroupNormalization.
    • Added weight decay support for all Keras optimizers.
    • Added Adafactor optimizer tf.keras.optimizers.Adafactor.
    • Added warmstart_embedding_matrix to tf.keras.utils.
      • This utility can be used to warmstart an embeddings matrix, so you reuse previously-learned word embeddings when working with a new set of words which may include previously unseen words (the embedding vectors for unseen words will be randomly initialized).

  • tf.Variable:

    • Added CompositeTensor as a baseclass to ResourceVariable.
      • This allows tf.Variables to be nested in tf.experimental.ExtensionTypes.
    • Added a new constructor argument experimental_enable_variable_lifting to tf.Variable, defaulting to True.
      • When it's False, the variable won't be lifted out of tf.function, thus it can be used as a tf.function-local variable: during each execution of the tf.function, the variable will be created and then disposed, similar to a local (that is, stack-allocated) variable in C/C++. Currently, experimental_enable_variable_lifting=False only works on non-XLA devices (for example, under @tf.function(jit_compile=False)).

  • TF SavedModel:

    • Added fingerprint.pb to the SavedModel directory. The fingerprint.pb file is a protobuf containing the "fingerprint" of the SavedModel. See the RFC for more details regarding its design and properties.

  • TF pip:

    • Windows CPU-builds for x86/x64 processors are now built, maintained, tested and released by a third party: Intel. Installing the windows-native pip packages for tensorflow or tensorflow-cpu would install Intel's tensorflow-intel package. These packages are provided as-is. Tensorflow will use reasonable efforts to maintain the availability and integrity of this pip package. There may be delays if the third party fails to release the pip package. For using TensorFlow GPU on Windows, you will need to install TensorFlow in WSL2.

Bug Fixes and Other Changes

  • tf.image

    • Added an optional parameter return_index_map to tf.image.ssim which causes the returned value to be the local SSIM map instead of the global mean.

  • TF Core:

    • tf.custom_gradient can now be applied to functions that accept "composite" tensors, such as tf.RaggedTensor, as inputs.
    • Fix device placement issues related to datasets with ragged tensors of strings (i.e. variant encoded data with types not supported on GPU).
    • experimental_follow_type_hints for tf.function has been deprecated. Please use input_signature or reduce_retracing to minimize retracing.

  • tf.SparseTensor:

    • Introduced set_shape, which sets the static dense shape of the sparse tensor and has the same semantics as tf.Tensor.set_shape.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar Dwivedi, Alexander Grund, alif_elham, Aman Agarwal, amoitra, Andrei Ivanov, andreii, Andrew Goodbody, angerson, Ashay Rane, Azeem Shaikh, Ben Barsdell, bhack, Bhavani Subramanian, Cedric Nugteren, Chandra Kumar Ramasamy, Christopher Bate, CohenAriel, Cotarou, cramasam, Enrico Minack, Francisco Unda, Frederic Bastien, gadagashwini, Gauri1 Deshpande, george, Jake, Jeff, Jerry Ge, Jingxuan He, Jojimon Varghese, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, kcoul, Keith Smiley, Kevin Hu, Kun Lu, kushanam, Lianmin Zheng, liuyuanqiang, Louis Sugy, Mahmoud Abuzaina, Marius Brehler, mdfaijul, Meenakshi Venkataraman, Milos Puzovic, mohantym, Namrata-Ibm, Nathan John Sircombe, Nathan Luehr, Olaf Lipinski, Om Thakkar, Osman F Bayram, Patrice Vignola, Pavani Majety, Philipp Hack, Prianka Liz Kariat, Rahul Batra, RajeshT, Renato Golin, riestere, Roger Iyengar, Rohit Santhanam, Rsanthanam-Amd, Sadeed Pv, Samuel Marks, Shimokawa, Naoaki, Siddhesh Kothadi, Simengliu-Nv, Sindre Seppola, snadampal, Srinivasan Narayanamoorthy, sushreebarsa, syedshahbaaz, Tamas Bela Feher, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tom Anderson, Tomohiro Endo, Trevor Morris, vibhutisawant, Victor Zhang, Vremold, Xavier Bonaventura, Yanming Wang, Yasir Modak, Yimei Sun, Yong Tang, Yulv-Git, zhuoran.liu, zotanika

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.11.0-rc1

Release 2.11.0

Breaking Changes

  • tf.keras.optimizers.Optimizer now points to the new Keras optimizer, and old optimizers have moved to the tf.keras.optimizers.legacy namespace. If you find your workflow failing due to this change, you may be facing one of the following issues:

    • Checkpoint loading failure. The new optimizer handles optimizer state differently from the old optimizer, which simplies the logic of checkpoint saving/loading, but at the cost of breaking checkpoint backward compatibility in some cases. If you want to keep using an old checkpoint, please change your optimizer to tf.keras.optimizer.legacy.XXX (e.g. tf.keras.optimizer.legacy.Adam).
    • TF1 compatibility. The new optimizer, tf.keras.optimizers.Optimizer, does not support TF1 any more, so please use the legacy optimizer tf.keras.optimizer.legacy.XXX. We highly recommend to migrate your workflow to TF2 for stable support and new features.
    • Old optimizer API not found. The new optimizer, tf.keras.optimizers.Optimizer, has a different set of public APIs from the old optimizer. These API changes are mostly related to getting rid of slot variables and TF1 support. Please check the API documentation to find alternatives to the missing API. If you must call the deprecated API, please change your optimizer to the legacy optimizer.
    • Learning rate schedule access. When using a LearningRateSchedule, The new optimizer's learning_rate property returns the current learning rate value instead of a LearningRateSchedule object as before. If you need to access the LearningRateSchedule object, please use optimizer._learning_rate.
    • If you implemented a custom optimizer based on the old optimizer. Please set your optimizer to subclass tf.keras.optimizer.legacy.XXX. If you want to migrate to the new optimizer and find it does not support your optimizer, please file an issue in the Keras GitHub repo.
    • Errors, such as Cannot recognize variable.... The new optimizer requires all optimizer variables to be created at the first apply_gradients() or minimize() call. If your workflow calls optimizer to update different parts of model in multiple stages, please call optimizer.build(model.trainable_variables) before the training loop.
    • Timeout or performance loss. We don't anticipate this to happen, but if you see such issues, please use the legacy optimizer, and file an issue in the Keras GitHub repo.

    The old Keras optimizer will never be deleted, but will not see any new feature additions. New optimizers (for example, tf.keras.optimizers.Adafactor) will only be implemented based on tf.keras.optimizers.Optimizer, the new base class.

Major Features and Improvements

  • tf.lite:

    • New operations supported: tf.unsortedsegmentmin, tf.atan2 and tf.sign.
    • Updates to existing operations:
      • tfl.mul now supports complex32 inputs.

  • tf.experimental.StructuredTensor

    • Introduced tf.experimental.StructuredTensor, which provides a flexible and TensorFlow-native way to encode structured data such as protocol buffers or pandas dataframes.

  • tf.keras:

    • Added a new get_metrics_result() method to tf.keras.models.Model.
      • Returns the current metrics values of the model as a dict.
    • Added a new group normalization layer - tf.keras.layers.GroupNormalization.
    • Added weight decay support for all Keras optimizers.
    • Added Adafactor optimizer tf.keras.optimizers.Adafactor.
    • Added warmstart_embedding_matrix to tf.keras.utils.
      • This utility can be used to warmstart an embeddings matrix, so you reuse previously-learned word embeddings when working with a new set of words which may include previously unseen words (the embedding vectors for unseen words will be randomly initialized).

  • tf.Variable:

    • Added CompositeTensor as a baseclass to ResourceVariable.
      • This allows tf.Variables to be nested in tf.experimental.ExtensionTypes.
    • Added a new constructor argument experimental_enable_variable_lifting to tf.Variable, defaulting to True.
      • When it's False, the variable won't be lifted out of tf.function, thus it can be used as a tf.function-local variable: during each execution of the tf.function, the variable will be created and then disposed, similar to a local (that is, stack-allocated) variable in C/C++. Currently, experimental_enable_variable_lifting=False only works on non-XLA devices (for example, under @tf.function(jit_compile=False)).

  • TF SavedModel:

    • Added fingerprint.pb to the SavedModel directory. The fingerprint.pb file is a protobuf containing the "fingerprint" of the SavedModel. See the RFC for more details regarding its design and properties.

  • TF pip:

    • Windows CPU-builds for x86/x64 processors are now built, maintained, tested and released by a third party: Intel. Installing the windows-native pip packages for tensorflow or tensorflow-cpu would install Intel's tensorflow-intel package. These packages are provided as-is. Tensorflow will use reasonable efforts to maintain the availability and integrity of this pip package. There may be delays if the third party fails to release the pip package. For using TensorFlow GPU on Windows, you will need to install TensorFlow in WSL2.

Bug Fixes and Other Changes

  • tf.image

    • Added an optional parameter return_index_map to tf.image.ssim which causes the returned value to be the local SSIM map instead of the global mean.

  • TF Core:

    • tf.custom_gradient can now be applied to functions that accept "composite" tensors, such as tf.RaggedTensor, as inputs.
    • Fix device placement issues related to datasets with ragged tensors of strings (i.e. variant encoded data with types not supported on GPU).
    • experimental_follow_type_hints for tf.function has been deprecated. Please use input_signature or reduce_retracing to minimize retracing.

  • tf.SparseTensor:

    • Introduced set_shape, which sets the static dense shape of the sparse tensor and has the same semantics as tf.Tensor.set_shape.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar Dwivedi, Alexander Grund, alif_elham, Aman Agarwal, amoitra, Andrei Ivanov, andreii, Andrew Goodbody, angerson, Ashay Rane, Azeem Shaikh, Ben Barsdell, bhack, Bhavani Subramanian, Cedric Nugteren, Chandra Kumar Ramasamy, Christopher Bate, CohenAriel, Cotarou, cramasam, Enrico Minack, Francisco Unda, Frederic Bastien, gadagashwini, Gauri1 Deshpande, george, Jake, Jeff, Jerry Ge, Jingxuan He, Jojimon Varghese, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, kcoul, Keith Smiley, Kevin Hu, Kun Lu, kushanam, Lianmin Zheng, liuyuanqiang, Louis Sugy, Mahmoud Abuzaina, Marius Brehler, mdfaijul, Meenakshi Venkataraman, Milos Puzovic, mohantym, Namrata-Ibm, Nathan John Sircombe, Nathan Luehr, Olaf Lipinski, Om Thakkar, Osman F Bayram, Patrice Vignola, Pavani Majety, Philipp Hack, Prianka Liz Kariat, Rahul Batra, RajeshT, Renato Golin, riestere, Roger Iyengar, Rohit Santhanam, Rsanthanam-Amd, Sadeed Pv, Samuel Marks, Shimokawa, Naoaki, Siddhesh Kothadi, Simengliu-Nv, Sindre Seppola, snadampal, Srinivasan Narayanamoorthy, sushreebarsa, syedshahbaaz, Tamas Bela Feher, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tom Anderson, Tomohiro Endo, Trevor Morris, vibhutisawant, Victor Zhang, Vremold, Xavier Bonaventura, Yanming Wang, Yasir Modak, Yimei Sun, Yong Tang, Yulv-Git, zhuoran.liu, zotanika

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.11.0-rc0

Release 2.11.0

Breaking Changes

  • tf.keras.optimizers.Optimizer now points to the new Keras optimizer, and old optimizers have moved to the tf.keras.optimizers.legacy namespace. If you find your workflow failing due to this change, you may be facing one of the following issues:

    • Checkpoint loading failure. The new optimizer handles optimizer state differently from the old optimizer, which simplies the logic of checkpoint saving/loading, but at the cost of breaking checkpoint backward compatibility in some cases. If you want to keep using an old checkpoint, please change your optimizer to tf.keras.optimizer.legacy.XXX (e.g. tf.keras.optimizer.legacy.Adam).
    • TF1 compatibility. The new optimizer, tf.keras.optimizers.Optimizer, does not support TF1 any more, so please use the legacy optimizer tf.keras.optimizer.legacy.XXX. We highly recommend to migrate your workflow to TF2 for stable support and new features.
    • Old optimizer API not found. The new optimizer, tf.keras.optimizers.Optimizer, has a different set of public APIs from the old optimizer. These API changes are mostly related to getting rid of slot variables and TF1 support. Please check the API documentation to find alternatives to the missing API. If you must call the deprecated API, please change your optimizer to the legacy optimizer.
    • Learning rate schedule access. When using a LearningRateSchedule, The new optimizer's learning_rate property returns the current learning rate value instead of a LearningRateSchedule object as before. If you need to access the LearningRateSchedule object, please use optimizer._learning_rate.
    • If you implemented a custom optimizer based on the old optimizer. Please set your optimizer to subclass tf.keras.optimizer.legacy.XXX. If you want to migrate to the new optimizer and find it does not support your optimizer, please file an issue in the Keras GitHub repo.
    • Errors, such as Cannot recognize variable.... The new optimizer requires all optimizer variables to be created at the first apply_gradients() or minimize() call. If your workflow calls optimizer to update different parts of model in multiple stages, please call optimizer.build(model.trainable_variables) before the training loop.
    • Timeout or performance loss. We don't anticipate this to happen, but if you see such issues, please use the legacy optimizer, and file an issue in the Keras GitHub repo.

    The old Keras optimizer will never be deleted, but will not see any new feature additions. New optimizers (for example, tf.keras.optimizers.Adafactor) will only be implemented based on tf.keras.optimizers.Optimizer, the new base class.

Major Features and Improvements

  • tf.lite:

    • New operations supported: tf.unsortedsegmentmin, tf.atan2 and tf.sign.
    • Updates to existing operations:
      • tfl.mul now supports complex32 inputs.

  • tf.experimental.StructuredTensor

    • Introduced tf.experimental.StructuredTensor, which provides a flexible and TensorFlow-native way to encode structured data such as protocol buffers or pandas dataframes.

  • tf.keras:

    • Added a new get_metrics_result() method to tf.keras.models.Model.
      • Returns the current metrics values of the model as a dict.
    • Added a new group normalization layer - tf.keras.layers.GroupNormalization.
    • Added weight decay support for all Keras optimizers.
    • Added Adafactor optimizer tf.keras.optimizers.Adafactor.
    • Added warmstart_embedding_matrix to tf.keras.utils.
      • This utility can be used to warmstart an embeddings matrix, so you reuse previously-learned word embeddings when working with a new set of words which may include previously unseen words (the embedding vectors for unseen words will be randomly initialized).

  • tf.Variable:

    • Added CompositeTensor as a baseclass to ResourceVariable.
      • This allows tf.Variables to be nested in tf.experimental.ExtensionTypes.
    • Added a new constructor argument experimental_enable_variable_lifting to tf.Variable, defaulting to True.
      • When it's False, the variable won't be lifted out of tf.function, thus it can be used as a tf.function-local variable: during each execution of the tf.function, the variable will be created and then disposed, similar to a local (that is, stack-allocated) variable in C/C++. Currently, experimental_enable_variable_lifting=False only works on non-XLA devices (for example, under @tf.function(jit_compile=False)).

  • TF SavedModel:

    • Added fingerprint.pb to the SavedModel directory. The fingerprint.pb file is a protobuf containing the "fingerprint" of the SavedModel. See the RFC for more details regarding its design and properties.

  • TF pip:

    • Windows CPU-builds for x86/x64 processors are now built, maintained, tested and released by a third party: Intel. Installing the windows-native pip packages for tensorflow or tensorflow-cpu would install Intel's tensorflow-intel package. These packages are provided as-is. Tensorflow will use reasonable efforts to maintain the availability and integrity of this pip package. There may be delays if the third party fails to release the pip package. For using TensorFlow GPU on Windows, you will need to install TensorFlow in WSL2.

Bug Fixes and Other Changes

  • tf.image

    • Added an optional parameter return_index_map to tf.image.ssim which causes the returned value to be the local SSIM map instead of the global mean.

  • TF Core:

    • tf.custom_gradient can now be applied to functions that accept "composite" tensors, such as tf.RaggedTensor, as inputs.
    • Fix device placement issues related to datasets with ragged tensors of strings (i.e. variant encoded data with types not supported on GPU).
    • experimental_follow_type_hints for tf.function has been deprecated. Please use input_signature or reduce_retracing to minimize retracing.

  • tf.SparseTensor:

    • Introduced set_shape, which sets the static dense shape of the sparse tensor and has the same semantics as tf.Tensor.set_shape.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

103yiran, 8bitmp3, Aakar Dwivedi, Alexander Grund, alif_elham, Aman Agarwal, amoitra, Andrei Ivanov, andreii, Andrew Goodbody, angerson, Ashay Rane, Azeem Shaikh, Ben Barsdell, bhack, Bhavani Subramanian, Cedric Nugteren, Chandra Kumar Ramasamy, Christopher Bate, CohenAriel, Cotarou, cramasam, Enrico Minack, Francisco Unda, Frederic Bastien, gadagashwini, Gauri1 Deshpande, george, Jake, Jeff, Jerry Ge, Jingxuan He, Jojimon Varghese, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, kcoul, Keith Smiley, Kevin Hu, Kun Lu, kushanam, Lianmin Zheng, liuyuanqiang, Louis Sugy, Mahmoud Abuzaina, Marius Brehler, mdfaijul, Meenakshi Venkataraman, Milos Puzovic, mohantym, Namrata-Ibm, Nathan John Sircombe, Nathan Luehr, Olaf Lipinski, Om Thakkar, Osman F Bayram, Patrice Vignola, Pavani Majety, Philipp Hack, Prianka Liz Kariat, Rahul Batra, RajeshT, Renato Golin, riestere, Roger Iyengar, Rohit Santhanam, Rsanthanam-Amd, Sadeed Pv, Samuel Marks, Shimokawa, Naoaki, Siddhesh Kothadi, Simengliu-Nv, Sindre Seppola, snadampal, Srinivasan Narayanamoorthy, sushreebarsa, syedshahbaaz, Tamas Bela Feher, Tatwai Chong, Thibaut Goetghebuer-Planchon, tilakrayal, Tom Anderson, Tomohiro Endo, Trevor Morris, vibhutisawant, Victor Zhang, Vremold, Xavier Bonaventura, Yanming Wang, Yasir Modak, Yimei Sun, Yong Tang, Yulv-Git, zhuoran.liu, zotanika

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.10.0

Release 2.10.0

Breaking Changes

  • Causal attention in keras.layers.Attention and keras.layers.AdditiveAttention is now specified in the call() method via the use_causal_mask argument (rather than in the constructor), for consistency with other layers.
  • Some files in tensorflow/python/training have been moved to tensorflow/python/tracking and tensorflow/python/checkpoint. Please update your imports accordingly, the old files will be removed in Release 2.11.
  • tf.keras.optimizers.experimental.Optimizer will graduate in Release 2.11, which means tf.keras.optimizers.Optimizer will be an alias of tf.keras.optimizers.experimental.Optimizer. The current tf.keras.optimizers.Optimizer will continue to be supported as tf.keras.optimizers.legacy.Optimizer, e.g.,tf.keras.optimizers.legacy.Adam. Most users won't be affected by this change, but please check the API doc if any API used in your workflow is changed or deprecated, and make adaptions. If you decide to keep using the old optimizer, please explicitly change your optimizer to tf.keras.optimizers.legacy.Optimizer.
  • RNG behavior change for tf.keras.initializers. Keras initializers will now use stateless random ops to generate random numbers.
    • Both seeded and unseeded initializers will always generate the same values every time they are called (for a given variable shape). For unseeded initializers (seed=None), a random seed will be created and assigned at initializer creation (different initializer instances get different seeds).
    • An unseeded initializer will raise a warning if it is reused (called) multiple times. This is because it would produce the same values each time, which may not be intended.

Deprecations

  • The C++ tensorflow::Code and tensorflow::Status will become aliases of respectively absl::StatusCode and absl::Status in some future release.
    • Use tensorflow::OkStatus() instead of tensorflow::Status::OK().
    • Stop constructing Status objects from tensorflow::error::Code.
    • One MUST NOT access tensorflow::errors::Code fields. Accessing tensorflow::error::Code fields is fine.
      • Use the constructors such as tensorflow::errors:InvalidArgument to create status using an error code without accessing it.
      • Use the free functions such as tensorflow::errors::IsInvalidArgument if needed.
      • In the last resort, use e.g.static_cast<tensorflow::errors::Code>(error::Code::INVALID_ARGUMENT) or static_cast<int>(code) for comparisons.
  • tensorflow::StatusOr will also become in the future alias to absl::StatusOr, so use StatusOr::value instead of StatusOr::ConsumeValueOrDie.

Major Features and Improvements

  • tf.lite:

    • New operations supported:
      • tflite SelectV2 now supports 5D.
      • tf.einsum is supported with multiple unknown shapes.
      • tf.unsortedsegmentprod op is supported.
      • tf.unsortedsegmentmax op is supported.
      • tf.unsortedsegmentsum op is supported.
    • Updates to existing operations:
      • tfl.scatter_nd now supports I1 for update arg.
    • Upgrade Flatbuffers v2.0.5 from v1.12.0

  • tf.keras:

    • EinsumDense layer is moved from experimental to core. Its import path is moved from tf.keras.layers.experimental.EinsumDense to tf.keras.layers.EinsumDense.
    • Added tf.keras.utils.audio_dataset_from_directory utility to easily generate audio classification datasets from directories of .wav files.
    • Added subset="both" support in tf.keras.utils.image_dataset_from_directory,tf.keras.utils.text_dataset_from_directory, and audio_dataset_from_directory, to be used with the validation_split argument, for returning both dataset splits at once, as a tuple.
    • Added tf.keras.utils.split_dataset utility to split a Dataset object or a list/tuple of arrays into two Dataset objects (e.g. train/test).
    • Added step granularity to BackupAndRestore callback for handling distributed training failures & restarts. The training state can now be restored at the exact epoch and step at which it was previously saved before failing.
    • Added tf.keras.dtensor.experimental.optimizers.AdamW. This optimizer is similar as the existing keras.optimizers.experimental.AdamW, and works in the DTensor training use case.
    • Improved masking support for tf.keras.layers.MultiHeadAttention.
      • Implicit masks for query, key and value inputs will automatically be used to compute a correct attention mask for the layer. These padding masks will be combined with any attention_mask passed in directly when calling the layer. This can be used with tf.keras.layers.Embedding with mask_zero=True to automatically infer a correct padding mask.
      • Added a use_causal_mask call time arugment to the layer. Passing use_causal_mask=True will compute a causal attention mask, and optionally combine it with any attention_mask passed in directly when calling the layer.
    • Added ignore_class argument in the loss SparseCategoricalCrossentropy and metrics IoU and MeanIoU, to specify a class index to be ignored during loss/metric computation (e.g. a background/void class).
    • Added tf.keras.models.experimental.SharpnessAwareMinimization. This class implements the sharpness-aware minimization technique, which boosts model performance on various tasks, e.g., ResNet on image classification.

  • tf.data:

    • Added support for cross-trainer data caching in tf.data service. This saves computation resources when concurrent training jobs train from the same dataset. See (https://www.tensorflow.org/apidocs/python/tf/data/experimental/service#sharingtfdataservicewithconcurrenttrainers) for more details.
    • Added dataset_id to tf.data.experimental.service.register_dataset. If provided, tf.data service will use the provided ID for the dataset. If the dataset ID already exists, no new dataset will be registered. This is useful if multiple training jobs need to use the same dataset for training. In this case, users should call register_dataset with the same dataset_id.
    • Added a new field, inject_prefetch, to tf.data.experimental.OptimizationOptions. If it is set to True,tf.data will now automatically add a prefetch transformation to datasets that end in synchronous transformations. This enables data generation to be overlapped with data consumption. This may cause a small increase in memory usage due to buffering. To enable this behavior, set inject_prefetch=True in tf.data.experimental.OptimizationOptions.
    • Added a new value to tf.data.Options.autotune.autotune_algorithm: STAGEBASED. If the autotune algorithm is set to STAGEBASED, then it runs a new algorithm that can get the same performance with lower CPU/memory usage.
    • Added tf.data.experimental.from_list, a new API for creating Datasets from lists of elements.

  • tf.distribute:

    • Added tf.distribute.experimental.PreemptionCheckpointHandler to handle worker preemption/maintenance and cluster-wise consistent error reporting for tf.distribute.MultiWorkerMirroredStrategy. Specifically, for the type of interruption with advance notice, it automatically saves a checkpoint, exits the program without raising an unrecoverable error, and restores the progress when training restarts.

  • tf.math:

    • Added tf.math.approx_max_k and tf.math.approx_min_k which are the optimized alternatives to tf.math.top_k on TPU. The performance difference range from 8 to 100 times depending on the size of k. When running on CPU and GPU, a non-optimized XLA kernel is used.

  • tf.train:

    • Added tf.train.TrackableView which allows users to inspect the TensorFlow Trackable object (e.g. tf.Module, Keras Layers and models).

  • tf.vectorized_map:

    • Added an optional parameter: warn. This parameter controls whether or not warnings will be printed when operations in the provided fn fall back to a while loop.

  • XLA:

  • CPU performance optimizations:

    • x86 CPUs: oneDNN bfloat16 auto-mixed precision grappler graph optimization pass has been renamed from auto_mixed_precision_mkl to auto_mixed_precision_onednn_bfloat16. See example usage here.
    • aarch64 CPUs: Experimental performance optimizations from Compute Library for the Arm® Architecture (ACL) are available through oneDNN in the default Linux aarch64 package (pip install tensorflow).
      • The optimizations are disabled by default.
      • Set the environment variable TF_ENABLE_ONEDNN_OPTS=1 to enable the optimizations. Setting the variable to 0 or unsetting it will disable the optimizations.
      • These optimizations can yield slightly different numerical results from when they are off due to floating-point round-off errors from different computation approaches and orders.
      • To verify that the optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

Bug Fixes and Other Changes

  • New argument experimental_device_ordinal in LogicalDeviceConfiguration to control the order of logical devices. (GPU only)

  • tf.keras:

    • Changed the TensorBoard tag names produced by the tf.keras.callbacks.TensorBoard callback, so that summaries logged automatically for model weights now include either a /histogram or /image suffix in their tag names, in order to prevent tag name collisions across summary types.

  • When running on GPU (with cuDNN version 7.6.3 or later),tf.nn.depthwise_conv2d backprop to filter (and therefore also tf.keras.layers.DepthwiseConv2D) now operate deterministically (and tf.errors.UnimplementedError is no longer thrown) when op-determinism has been enabled via tf.config.experimental.enable_op_determinism. This closes issue 47174.

  • tf.random

    • Added tf.random.experimental.stateless_shuffle, a stateless version of tf.random.shuffle.

Security

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Abolfazl Shahbazi, Adam Lanicek, Amin Benarieb, andreii, Andrew Fitzgibbon, Andrew Goodbody, angerson, Ashiq Imran, Aurélien Geron, Banikumar Maiti (Intel Aipg), Ben Barsdell, Ben Mares, bhack, Bhavani Subramanian, Bill Schnurr, Byungsoo Oh, Chandra Sr Potula, Chengji Yao, Chris Carpita, Christopher Bate, chunduriv, Cliff Woolley, Cliffs Dover, Cloud Han, Code-Review-Doctor, DEKHTIARJonathan, Deven Desai, Djacon, Duncan Riach, fedotoff, fo40225, Frederic Bastien, gadagashwini, Gauri1 Deshpande, guozhong.zhuang, Hui Peng, James Gerity, Jason Furmanek, Jonathan Dekhtiar, Jueon Park, Kaixi Hou, Kanvi Khanna, Keith Smiley, Koan-Sin Tan, Kulin Seth, kushanam, Learning-To-Play, Li-Wen Chang, lipracer, liuyuanqiang, Louis Sugy, Lucas David, Lukas Geiger, Mahmoud Abuzaina, Marius Brehler, Maxiwell S. Garcia, mdfaijul, Meenakshi Venkataraman, Michal Szutenberg, Michele Di Giorgio, Mickaël Salamin, Nathan John Sircombe, Nathan Luehr, Neil Girdhar, Nils Reichardt, Nishidha Panpaliya, Nobuo Tsukamoto, Om Thakkar, Patrice Vignola, Philipp Hack, Pooya Jannaty, Prianka Liz Kariat, pshiko, Rajeshwar Reddy T, rdl4199, Rohit Santhanam, Rsanthanam-Amd, Sachin Muradi, Saoirse Stewart, Serge Panev, Shu Wang, Srinivasan Narayanamoorthy, Stella Stamenova, Stephan Hartmann, Sunita Nadampalli, synandi, Tamas Bela Feher, Tao Xu, Thibaut Goetghebuer-Planchon, Trevor Morris, Xiaoming (Jason) Cui, Yimei Sun, Yong Tang, Yuanqiang Liu, Yulv-Git, Zhoulong Jiang, ZihengJiang

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.9.2

Release 2.9.2

This releases introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.8.3

Release 2.8.3

This releases introduces several vulnerability fixes: * Fixes a CHECK failure in tf.reshape caused by overflows (CVE-2022-35934) * Fixes a CHECK failure in SobolSample caused by missing validation (CVE-2022-35935) * Fixes an OOB read in Gather_nd op in TF Lite (CVE-2022-35937) * Fixes a CHECK failure in TensorListReserve caused by missing validation (CVE-2022-35960) * Fixes an OOB write in Scatter_nd op in TF Lite (CVE-2022-35939) * Fixes an integer overflow in RaggedRangeOp (CVE-2022-35940) * Fixes a CHECK failure in AvgPoolOp (CVE-2022-35941) * Fixes a CHECK failures in UnbatchGradOp (CVE-2022-35952) * Fixes a segfault TFLite converter on per-channel quantized transposed convolutions (CVE-2022-36027) * Fixes a CHECK failures in AvgPool3DGrad (CVE-2022-35959) * Fixes a CHECK failures in FractionalAvgPoolGrad (CVE-2022-35963) * Fixes a segfault in BlockLSTMGradV2 (CVE-2022-35964) * Fixes a segfault in LowerBound and UpperBound (CVE-2022-35965) * Fixes a segfault in QuantizedAvgPool (CVE-2022-35966) * Fixes a segfault in QuantizedAdd (CVE-2022-35967) * Fixes a CHECK fail in AvgPoolGrad (CVE-2022-35968) * Fixes a CHECK fail in Conv2DBackpropInput (CVE-2022-35969) * Fixes a segfault in QuantizedInstanceNorm (CVE-2022-35970) * Fixes a CHECK fail in FakeQuantWithMinMaxVars (CVE-2022-35971) * Fixes a segfault in Requantize (CVE-2022-36017) * Fixes a segfault in QuantizedBiasAdd (CVE-2022-35972) * Fixes a CHECK fail in FakeQuantWithMinMaxVarsPerChannel (CVE-2022-36019) * Fixes a segfault in QuantizedMatMul (CVE-2022-35973) * Fixes a segfault in QuantizeDownAndShrinkRange (CVE-2022-35974) * Fixes segfaults in QuantizedRelu and QuantizedRelu6 (CVE-2022-35979) * Fixes a CHECK fail in FractionalMaxPoolGrad (CVE-2022-35981) * Fixes a CHECK fail in RaggedTensorToVariant (CVE-2022-36018) * Fixes a CHECK fail in QuantizeAndDequantizeV3 (CVE-2022-36026) * Fixes a segfault in SparseBincount (CVE-2022-35982) * Fixes a CHECK fail in Save and SaveSlices (CVE-2022-35983) * Fixes a CHECK fail in ParameterizedTruncatedNormal (CVE-2022-35984) * Fixes a CHECK fail in LRNGrad (CVE-2022-35985) * Fixes a segfault in RaggedBincount (CVE-2022-35986) * Fixes a CHECK fail in DenseBincount (CVE-2022-35987) * Fixes a CHECK fail in tf.linalg.matrix_rank (CVE-2022-35988) * Fixes a CHECK fail in MaxPool (CVE-2022-35989) * Fixes a CHECK fail in Conv2DBackpropInput (CVE-2022-35999) * Fixes a CHECK fail in EmptyTensorList (CVE-2022-35998) * Fixes a CHECK fail in tf.sparse.cross (CVE-2022-35997) * Fixes a floating point exception in Conv2D (CVE-2022-35996) * Fixes a CHECK fail in AudioSummaryV2 (CVE-2022-35995) * Fixes a CHECK fail in CollectiveGather (CVE-2022-35994) * Fixes a CHECK fail in SetSize (CVE-2022-35993) * Fixes a CHECK fail in TensorListFromTensor (CVE-2022-35992) * Fixes a CHECK fail in TensorListScatter and TensorListScatterV2 (CVE-2022-35991) * Fixes a CHECK fail in FakeQuantWithMinMaxVarsPerChannelGradient (CVE-2022-35990) * Fixes a CHECK fail in FakeQuantWithMinMaxVarsGradient (CVE-2022-36005) * Fixes a CHECK fail in tf.random.gamma (CVE-2022-36004) * Fixes a CHECK fail in RandomPoissonV2 (CVE-2022-36003) * Fixes a CHECK fail in Unbatch (CVE-2022-36002) * Fixes a CHECK fail in DrawBoundingBoxes (CVE-2022-36001) * Fixes a CHECK fail in Eig (CVE-2022-36000) * Fixes a null dereference on MLIR on empty function attributes (CVE-2022-36011) * Fixes an assertion failure on MLIR empty edge names (CVE-2022-36012) * Fixes a null-dereference in mlir::tfg::GraphDefImporter::ConvertNodeDef (CVE-2022-36013) * Fixes a null-dereference in mlir::tfg::TFOp::nameAttr (CVE-2022-36014) * Fixes an integer overflow in math ops (CVE-2022-36015) * Fixes a CHECK-fail in tensorflow::full_type::SubstituteFromAttrs (CVE-2022-36016) * Fixes an OOB read in Gather_nd op in TF Lite Micro (CVE-2022-35938)

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.7.4

Release 2.7.4

Note: This is the last release in the 2.7.x series

This releases introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.10.0-rc3

Release 2.10.0

Breaking Changes

  • Causal attention in keras.layers.Attention and keras.layers.AdditiveAttention is now specified in the call() method via the use_causal_mask argument (rather than in the constructor), for consistency with other layers.
  • Some files in tensorflow/python/training have been moved to tensorflow/python/tracking and tensorflow/python/checkpoint. Please update your imports accordingly, the old files will be removed in Release 2.11.
  • tf.keras.optimizers.experimental.Optimizer will graduate in Release 2.11, which means tf.keras.optimizers.Optimizer will be an alias of tf.keras.optimizers.experimental.Optimizer. The current tf.keras.optimizers.Optimizer will continue to be supported as tf.keras.optimizers.legacy.Optimizer, e.g.,tf.keras.optimizers.legacy.Adam. Most users won't be affected by this change, but please check the API doc if any API used in your workflow is changed or deprecated, and make adaptions. If you decide to keep using the old optimizer, please explicitly change your optimizer to tf.keras.optimizers.legacy.Optimizer.
  • RNG behavior change for tf.keras.initializers. Keras initializers will now use stateless random ops to generate random numbers.
    • Both seeded and unseeded initializers will always generate the same values every time they are called (for a given variable shape). For unseeded initializers (seed=None), a random seed will be created and assigned at initializer creation (different initializer instances get different seeds).
    • An unseeded initializer will raise a warning if it is reused (called) multiple times. This is because it would produce the same values each time, which may not be intended.

Deprecations

  • The C++ tensorflow::Code and tensorflow::Status will become aliases of respectively absl::StatusCode and absl::Status in some future release.
    • Use tensorflow::OkStatus() instead of tensorflow::Status::OK().
    • Stop constructing Status objects from tensorflow::error::Code.
    • One MUST NOT access tensorflow::errors::Code fields. Accessing tensorflow::error::Code fields is fine.
      • Use the constructors such as tensorflow::errors:InvalidArgument to create status using an error code without accessing it.
      • Use the free functions such as tensorflow::errors::IsInvalidArgument if needed.
      • In the last resort, use e.g.static_cast<tensorflow::errors::Code>(error::Code::INVALID_ARGUMENT) or static_cast<int>(code) for comparisons.
  • tensorflow::StatusOr will also become in the future alias to absl::StatusOr, so use StatusOr::value instead of StatusOr::ConsumeValueOrDie.

Major Features and Improvements

  • tf.lite:

    • New operations supported:
      • tflite SelectV2 now supports 5D.
      • tf.einsum is supported with multiple unknown shapes.
      • tf.unsortedsegmentprod op is supported.
      • tf.unsortedsegmentmax op is supported.
      • tf.unsortedsegmentsum op is supported.
    • Updates to existing operations:
      • tfl.scatter_nd now supports I1 for update arg.
    • Upgrade Flatbuffers v2.0.5 from v1.12.0

  • tf.keras:

    • EinsumDense layer is moved from experimental to core. Its import path is moved from tf.keras.layers.experimental.EinsumDense to tf.keras.layers.EinsumDense.
    • Added tf.keras.utils.audio_dataset_from_directory utility to easily generate audio classification datasets from directories of .wav files.
    • Added subset="both" support in tf.keras.utils.image_dataset_from_directory,tf.keras.utils.text_dataset_from_directory, and audio_dataset_from_directory, to be used with the validation_split argument, for returning both dataset splits at once, as a tuple.
    • Added tf.keras.utils.split_dataset utility to split a Dataset object or a list/tuple of arrays into two Dataset objects (e.g. train/test).
    • Added step granularity to BackupAndRestore callback for handling distributed training failures & restarts. The training state can now be restored at the exact epoch and step at which it was previously saved before failing.
    • Added tf.keras.dtensor.experimental.optimizers.AdamW. This optimizer is similar as the existing keras.optimizers.experimental.AdamW, and works in the DTensor training use case.
    • Improved masking support for tf.keras.layers.MultiHeadAttention.
      • Implicit masks for query, key and value inputs will automatically be used to compute a correct attention mask for the layer. These padding masks will be combined with any attention_mask passed in directly when calling the layer. This can be used with tf.keras.layers.Embedding with mask_zero=True to automatically infer a correct padding mask.
      • Added a use_causal_mask call time arugment to the layer. Passing use_causal_mask=True will compute a causal attention mask, and optionally combine it with any attention_mask passed in directly when calling the layer.
    • Added ignore_class argument in the loss SparseCategoricalCrossentropy and metrics IoU and MeanIoU, to specify a class index to be ignored during loss/metric computation (e.g. a background/void class).
    • Added tf.keras.models.experimental.SharpnessAwareMinimization. This class implements the sharpness-aware minimization technique, which boosts model performance on various tasks, e.g., ResNet on image classification.

  • tf.data:

    • Added support for cross-trainer data caching in tf.data service. This saves computation resources when concurrent training jobs train from the same dataset. See (https://www.tensorflow.org/apidocs/python/tf/data/experimental/service#sharingtfdataservicewithconcurrenttrainers) for more details.
    • Added dataset_id to tf.data.experimental.service.register_dataset. If provided, tf.data service will use the provided ID for the dataset. If the dataset ID already exists, no new dataset will be registered. This is useful if multiple training jobs need to use the same dataset for training. In this case, users should call register_dataset with the same dataset_id.
    • Added a new field, inject_prefetch, to tf.data.experimental.OptimizationOptions. If it is set to True,tf.data will now automatically add a prefetch transformation to datasets that end in synchronous transformations. This enables data generation to be overlapped with data consumption. This may cause a small increase in memory usage due to buffering. To enable this behavior, set inject_prefetch=True in tf.data.experimental.OptimizationOptions.
    • Added a new value to tf.data.Options.autotune.autotune_algorithm: STAGEBASED. If the autotune algorithm is set to STAGEBASED, then it runs a new algorithm that can get the same performance with lower CPU/memory usage.
    • Added tf.data.experimental.from_list, a new API for creating Datasets from lists of elements.

  • tf.distribute:

    • Added tf.distribute.experimental.PreemptionCheckpointHandler to handle worker preemption/maintenance and cluster-wise consistent error reporting for tf.distribute.MultiWorkerMirroredStrategy. Specifically, for the type of interruption with advance notice, it automatically saves a checkpoint, exits the program without raising an unrecoverable error, and restores the progress when training restarts.

  • tf.math:

    • Added tf.math.approx_max_k and tf.math.approx_min_k which are the optimized alternatives to tf.math.top_k on TPU. The performance difference range from 8 to 100 times depending on the size of k. When running on CPU and GPU, a non-optimized XLA kernel is used.

  • tf.train:

    • Added tf.train.TrackableView which allows users to inspect the TensorFlow Trackable object (e.g. tf.Module, Keras Layers and models).

  • tf.vectorized_map:

    • Added an optional parameter: warn. This parameter controls whether or not warnings will be printed when operations in the provided fn fall back to a while loop.

  • XLA:

  • CPU performance optimizations:

    • x86 CPUs: oneDNN bfloat16 auto-mixed precision grappler graph optimization pass has been renamed from auto_mixed_precision_mkl to auto_mixed_precision_onednn_bfloat16. See example usage here.
    • aarch64 CPUs: Experimental performance optimizations from Compute Library for the Arm® Architecture (ACL) are available through oneDNN in the default Linux aarch64 package (pip install tensorflow).
      • The optimizations are disabled by default.
      • Set the environment variable TF_ENABLE_ONEDNN_OPTS=1 to enable the optimizations. Setting the variable to 0 or unsetting it will disable the optimizations.
      • These optimizations can yield slightly different numerical results from when they are off due to floating-point round-off errors from different computation approaches and orders.
      • To verify that the optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

Bug Fixes and Other Changes

  • New argument experimental_device_ordinal in LogicalDeviceConfiguration to control the order of logical devices. (GPU only)

  • tf.keras:

    • Changed the TensorBoard tag names produced by the tf.keras.callbacks.TensorBoard callback, so that summaries logged automatically for model weights now include either a /histogram or /image suffix in their tag names, in order to prevent tag name collisions across summary types.

  • When running on GPU (with cuDNN version 7.6.3 or later),tf.nn.depthwise_conv2d backprop to filter (and therefore also tf.keras.layers.DepthwiseConv2D) now operate deterministically (and tf.errors.UnimplementedError is no longer thrown) when op-determinism has been enabled via tf.config.experimental.enable_op_determinism. This closes issue 47174.

  • tf.random

    • Added tf.random.experimental.stateless_shuffle, a stateless version of tf.random.shuffle.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Abolfazl Shahbazi, Adam Lanicek, Amin Benarieb, andreii, Andrew Fitzgibbon, Andrew Goodbody, angerson, Ashiq Imran, Aurélien Geron, Banikumar Maiti (Intel Aipg), Ben Barsdell, Ben Mares, bhack, Bhavani Subramanian, Bill Schnurr, Byungsoo Oh, Chandra Sr Potula, Chengji Yao, Chris Carpita, Christopher Bate, chunduriv, Cliff Woolley, Cliffs Dover, Cloud Han, Code-Review-Doctor, DEKHTIARJonathan, Deven Desai, Djacon, Duncan Riach, fedotoff, fo40225, Frederic Bastien, gadagashwini, Gauri1 Deshpande, guozhong.zhuang, Hui Peng, James Gerity, Jason Furmanek, Jonathan Dekhtiar, Jueon Park, Kaixi Hou, Kanvi Khanna, Keith Smiley, Koan-Sin Tan, Kulin Seth, kushanam, Learning-To-Play, Li-Wen Chang, lipracer, liuyuanqiang, Louis Sugy, Lucas David, Lukas Geiger, Mahmoud Abuzaina, Marius Brehler, Maxiwell S. Garcia, mdfaijul, Meenakshi Venkataraman, Michal Szutenberg, Michele Di Giorgio, Mickaël Salamin, Nathan John Sircombe, Nathan Luehr, Neil Girdhar, Nils Reichardt, Nishidha Panpaliya, Nobuo Tsukamoto, Om Thakkar, Patrice Vignola, Philipp Hack, Pooya Jannaty, Prianka Liz Kariat, pshiko, Rajeshwar Reddy T, rdl4199, Rohit Santhanam, Rsanthanam-Amd, Sachin Muradi, Saoirse Stewart, Serge Panev, Shu Wang, Srinivasan Narayanamoorthy, Stella Stamenova, Stephan Hartmann, Sunita Nadampalli, synandi, Tamas Bela Feher, Tao Xu, Thibaut Goetghebuer-Planchon, Trevor Morris, Xiaoming (Jason) Cui, Yimei Sun, Yong Tang, Yuanqiang Liu, Yulv-Git, Zhoulong Jiang, ZihengJiang

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.10.0-rc2

Release 2.10.0

Breaking Changes

  • Causal attention in keras.layers.Attention and keras.layers.AdditiveAttention is now specified in the call() method via the use_causal_mask argument (rather than in the constructor), for consistency with other layers.
  • Some files in tensorflow/python/training have been moved to tensorflow/python/tracking and tensorflow/python/checkpoint. Please update your imports accordingly, the old files will be removed in Release 2.11.
  • tf.keras.optimizers.experimental.Optimizer will graduate in Release 2.11, which means tf.keras.optimizers.Optimizer will be an alias of tf.keras.optimizers.experimental.Optimizer. The current tf.keras.optimizers.Optimizer will continue to be supported as tf.keras.optimizers.legacy.Optimizer, e.g.,tf.keras.optimizers.legacy.Adam. Most users won't be affected by this change, but please check the API doc if any API used in your workflow is changed or deprecated, and make adaptions. If you decide to keep using the old optimizer, please explicitly change your optimizer to tf.keras.optimizers.legacy.Optimizer.
  • RNG behavior change for tf.keras.initializers. Keras initializers will now use stateless random ops to generate random numbers.
    • Both seeded and unseeded initializers will always generate the same values every time they are called (for a given variable shape). For unseeded initializers (seed=None), a random seed will be created and assigned at initializer creation (different initializer instances get different seeds).
    • An unseeded initializer will raise a warning if it is reused (called) multiple times. This is because it would produce the same values each time, which may not be intended.

Major Features and Improvements

  • tf.lite:

    • New operations supported:
      • tflite SelectV2 now supports 5D.
      • tf.einsum is supported with multiple unknown shapes.
      • tf.unsortedsegmentprod op is supported.
      • tf.unsortedsegmentmax op is supported.
      • tf.unsortedsegmentsum op is supported.
    • Updates to existing operations:
      • tfl.scatter_nd now supports I1 for update arg.
    • Upgrade Flatbuffers v2.0.5 from v1.12.0

  • tf.keras:

    • EinsumDense layer is moved from experimental to core. Its import path is moved from tf.keras.layers.experimental.EinsumDense to tf.keras.layers.EinsumDense.
    • Added tf.keras.utils.audio_dataset_from_directory utility to easily generate audio classification datasets from directories of .wav files.
    • Added subset="both" support in tf.keras.utils.image_dataset_from_directory,tf.keras.utils.text_dataset_from_directory, and audio_dataset_from_directory, to be used with the validation_split argument, for returning both dataset splits at once, as a tuple.
    • Added tf.keras.utils.split_dataset utility to split a Dataset object or a list/tuple of arrays into two Dataset objects (e.g. train/test).
    • Added step granularity to BackupAndRestore callback for handling distributed training failures & restarts. The training state can now be restored at the exact epoch and step at which it was previously saved before failing.
    • Added tf.keras.dtensor.experimental.optimizers.AdamW. This optimizer is similar as the existing keras.optimizers.experimental.AdamW, and works in the DTensor training use case.
    • Improved masking support for tf.keras.layers.MultiHeadAttention.
      • Implicit masks for query, key and value inputs will automatically be used to compute a correct attention mask for the layer. These padding masks will be combined with any attention_mask passed in directly when calling the layer. This can be used with tf.keras.layers.Embedding with mask_zero=True to automatically infer a correct padding mask.
      • Added a use_causal_mask call time arugment to the layer. Passing use_causal_mask=True will compute a causal attention mask, and optionally combine it with any attention_mask passed in directly when calling the layer.
    • Added ignore_class argument in the loss SparseCategoricalCrossentropy and metrics IoU and MeanIoU, to specify a class index to be ignored during loss/metric computation (e.g. a background/void class).
    • Added tf.keras.models.experimental.SharpnessAwareMinimization. This class implements the sharpness-aware minimization technique, which boosts model performance on various tasks, e.g., ResNet on image classification.

  • tf.data:

    • Added support for cross-trainer data caching in tf.data service. This saves computation resources when concurrent training jobs train from the same dataset. See (https://www.tensorflow.org/apidocs/python/tf/data/experimental/service#sharingtfdataservicewithconcurrenttrainers) for more details.
    • Added dataset_id to tf.data.experimental.service.register_dataset. If provided, tf.data service will use the provided ID for the dataset. If the dataset ID already exists, no new dataset will be registered. This is useful if multiple training jobs need to use the same dataset for training. In this case, users should call register_dataset with the same dataset_id.
    • Added a new field, inject_prefetch, to tf.data.experimental.OptimizationOptions. If it is set to True,tf.data will now automatically add a prefetch transformation to datasets that end in synchronous transformations. This enables data generation to be overlapped with data consumption. This may cause a small increase in memory usage due to buffering. To enable this behavior, set inject_prefetch=True in tf.data.experimental.OptimizationOptions.
    • Added a new value to tf.data.Options.autotune.autotune_algorithm: STAGEBASED. If the autotune algorithm is set to STAGEBASED, then it runs a new algorithm that can get the same performance with lower CPU/memory usage.
    • Added tf.data.experimental.from_list, a new API for creating Datasets from lists of elements.

  • tf.distribute:

    • Added tf.distribute.experimental.PreemptionCheckpointHandler to handle worker preemption/maintenance and cluster-wise consistent error reporting for tf.distribute.MultiWorkerMirroredStrategy. Specifically, for the type of interruption with advance notice, it automatically saves a checkpoint, exits the program without raising an unrecoverable error, and restores the progress when training restarts.

  • tf.math:

    • Added tf.math.approx_max_k and tf.math.approx_min_k which are the optimized alternatives to tf.math.top_k on TPU. The performance difference range from 8 to 100 times depending on the size of k. When running on CPU and GPU, a non-optimized XLA kernel is used.

  • tf.train:

    • Added tf.train.TrackableView which allows users to inspect the TensorFlow Trackable object (e.g. tf.Module, Keras Layers and models).

  • tf.vectorized_map:

    • Added an optional parameter: warn. This parameter controls whether or not warnings will be printed when operations in the provided fn fall back to a while loop.

  • XLA:

    • MWMS is now compilable with XLA.

  • oneDNN CPU performance optimizations:

    • x86 CPUs: oneDNN bfloat16 auto-mixed precision grappler graph optimization pass has been renamed from auto_mixed_precision_mkl to auto_mixed_precision_onednn_bfloat16. See example usage here.
    • aarch64 CPUs: Experimental Arm Compute Library (ACL) CPU performance optimizations through oneDNN are available in the default Linux aarch64 package (pip install tensorflow).
      • The optimizations are disabled by default.
      • Set the environment variable TF_ENABLE_ONEDNN_OPTS=1 to enable the optimizations. Setting the variable to 0 or unsetting it will disable the optimizations.
      • These optimizations can yield slightly different numerical results from when they are off due to floating-point round-off errors from different computation approaches and orders.
      • To verify that the optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

Bug Fixes and Other Changes

  • New argument experimental_device_ordinal in LogicalDeviceConfiguration to control the order of logical devices. (GPU only)

  • tf.keras:

    • Changed the TensorBoard tag names produced by the tf.keras.callbacks.TensorBoard callback, so that summaries logged automatically for model weights now include either a /histogram or /image suffix in their tag names, in order to prevent tag name collisions across summary types.

  • When running on GPU (with cuDNN version 7.6.3 or later),tf.nn.depthwise_conv2d backprop to filter (and therefore also tf.keras.layers.DepthwiseConv2D) now operate deterministically (and tf.errors.UnimplementedError is no longer thrown) when op-determinism has been enabled via tf.config.experimental.enable_op_determinism. This closes issue 47174.

  • tf.random

    • Added tf.random.experimental.stateless_shuffle, a stateless version of tf.random.shuffle.

Deprecations

  • The C++ tensorflow::Code and tensorflow::Status will become aliases of respectively absl::StatusCode and absl::Status in some future release.
    • Use tensorflow::OkStatus() instead of tensorflow::Status::OK().
    • Stop constructing Status objects from tensorflow::error::Code.
    • One MUST NOT access tensorflow::errors::Code fields. Accessing tensorflow::error::Code fields is fine.
      • Use the constructors such as tensorflow::errors:InvalidArgument to create status using an error code without accessing it.
      • Use the free functions such as tensorflow::errors::IsInvalidArgument if needed.
      • In the last resort, use e.g.static_cast<tensorflow::errors::Code>(error::Code::INVALID_ARGUMENT) or static_cast<int>(code) for comparisons.
  • tensorflow::StatusOr will also become in the future alias to absl::StatusOr, so use StatusOr::value instead of StatusOr::ConsumeValueOrDie.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Abolfazl Shahbazi, Adam Lanicek, Amin Benarieb, andreii, Andrew Fitzgibbon, Andrew Goodbody, angerson, Ashiq Imran, Aurélien Geron, Banikumar Maiti (Intel Aipg), Ben Barsdell, Ben Mares, bhack, Bhavani Subramanian, Bill Schnurr, Byungsoo Oh, Chandra Sr Potula, Chengji Yao, Chris Carpita, Christopher Bate, chunduriv, Cliff Woolley, Cliffs Dover, Cloud Han, Code-Review-Doctor, DEKHTIARJonathan, Deven Desai, Djacon, Duncan Riach, fedotoff, fo40225, Frederic Bastien, gadagashwini, Gauri1 Deshpande, guozhong.zhuang, Hui Peng, James Gerity, Jason Furmanek, Jonathan Dekhtiar, Jueon Park, Kaixi Hou, Kanvi Khanna, Keith Smiley, Koan-Sin Tan, Kulin Seth, kushanam, Learning-To-Play, Li-Wen Chang, lipracer, liuyuanqiang, Louis Sugy, Lucas David, Lukas Geiger, Mahmoud Abuzaina, Marius Brehler, Maxiwell S. Garcia, mdfaijul, Meenakshi Venkataraman, Michal Szutenberg, Michele Di Giorgio, Mickaël Salamin, Nathan John Sircombe, Nathan Luehr, Neil Girdhar, Nils Reichardt, Nishidha Panpaliya, Nobuo Tsukamoto, Om Thakkar, Patrice Vignola, Philipp Hack, Pooya Jannaty, Prianka Liz Kariat, pshiko, Rajeshwar Reddy T, rdl4199, Rohit Santhanam, Rsanthanam-Amd, Sachin Muradi, Saoirse Stewart, Serge Panev, Shu Wang, Srinivasan Narayanamoorthy, Stella Stamenova, Stephan Hartmann, Sunita Nadampalli, synandi, Tamas Bela Feher, Tao Xu, Thibaut Goetghebuer-Planchon, Trevor Morris, Xiaoming (Jason) Cui, Yimei Sun, Yong Tang, Yuanqiang Liu, Yulv-Git, Zhoulong Jiang, ZihengJiang

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.10.0-rc1

Release 2.10.0

Breaking Changes

  • Causal attention in keras.layers.Attention and keras.layers.AdditiveAttention is now specified in the call() method via the use_causal_mask argument (rather than in the constructor), for consistency with other layers.
  • Some files in tensorflow/python/training have been moved to tensorflow/python/tracking and tensorflow/python/checkpoint. Please update your imports accordingly, the old files will be removed in Release 2.11.
  • tf.keras.optimizers.experimental.Optimizer will graduate in Release 2.11, which means tf.keras.optimizers.Optimizer will be an alias of tf.keras.optimizers.experimental.Optimizer. The current tf.keras.optimizers.Optimizer will continue to be supported as tf.keras.optimizers.legacy.Optimizer, e.g.,tf.keras.optimizers.legacy.Adam. Most users won't be affected by this change, but please check the API doc if any API used in your workflow is changed or deprecated, and make adaptions. If you decide to keep using the old optimizer, please explicitly change your optimizer to tf.keras.optimizers.legacy.Optimizer.
  • RNG behavior change for tf.keras.initializers. Keras initializers will now use stateless random ops to generate random numbers.
    • Both seeded and unseeded initializers will always generate the same values every time they are called (for a given variable shape). For unseeded initializers (seed=None), a random seed will be created and assigned at initializer creation (different initializer instances get different seeds).
    • An unseeded initializer will raise a warning if it is reused (called) multiple times. This is because it would produce the same values each time, which may not be intended.

Major Features and Improvements

  • tf.lite:

    • New operations supported:
      • tflite SelectV2 now supports 5D.
      • tf.einsum is supported with multiple unknown shapes.
      • tf.unsortedsegmentprod op is supported.
      • tf.unsortedsegmentmax op is supported.
      • tf.unsortedsegmentsum op is supported.
    • Updates to existing operations:
      • tfl.scatter_nd now supports I1 for update arg.
    • Upgrade Flatbuffers v2.0.5 from v1.12.0

  • tf.keras:

    • EinsumDense layer is moved from experimental to core. Its import path is moved from tf.keras.layers.experimental.EinsumDense to tf.keras.layers.EinsumDense.
    • Added tf.keras.utils.audio_dataset_from_directory utility to easily generate audio classification datasets from directories of .wav files.
    • Added subset="both" support in tf.keras.utils.image_dataset_from_directory,tf.keras.utils.text_dataset_from_directory, and audio_dataset_from_directory, to be used with the validation_split argument, for returning both dataset splits at once, as a tuple.
    • Added tf.keras.utils.split_dataset utility to split a Dataset object or a list/tuple of arrays into two Dataset objects (e.g. train/test).
    • Added step granularity to BackupAndRestore callback for handling distributed training failures & restarts. The training state can now be restored at the exact epoch and step at which it was previously saved before failing.
    • Added tf.keras.dtensor.experimental.optimizers.AdamW. This optimizer is similar as the existing keras.optimizers.experimental.AdamW, and works in the DTensor training use case.
    • Improved masking support for tf.keras.layers.MultiHeadAttention.
      • Implicit masks for query, key and value inputs will automatically be used to compute a correct attention mask for the layer. These padding masks will be combined with any attention_mask passed in directly when calling the layer. This can be used with tf.keras.layers.Embedding with mask_zero=True to automatically infer a correct padding mask.
      • Added a use_causal_mask call time arugment to the layer. Passing use_causal_mask=True will compute a causal attention mask, and optionally combine it with any attention_mask passed in directly when calling the layer.
    • Added ignore_class argument in the loss SparseCategoricalCrossentropy and metrics IoU and MeanIoU, to specify a class index to be ignored during loss/metric computation (e.g. a background/void class).
    • Added tf.keras.models.experimental.SharpnessAwareMinimization. This class implements the sharpness-aware minimization technique, which boosts model performance on various tasks, e.g., ResNet on image classification.

  • tf.data:

    • Added support for cross-trainer data caching in tf.data service. This saves computation resources when concurrent training jobs train from the same dataset. See (https://www.tensorflow.org/apidocs/python/tf/data/experimental/service#sharingtfdataservicewithconcurrenttrainers) for more details.
    • Added dataset_id to tf.data.experimental.service.register_dataset. If provided, tf.data service will use the provided ID for the dataset. If the dataset ID already exists, no new dataset will be registered. This is useful if multiple training jobs need to use the same dataset for training. In this case, users should call register_dataset with the same dataset_id.
    • Added a new field, inject_prefetch, to tf.data.experimental.OptimizationOptions. If it is set to True,tf.data will now automatically add a prefetch transformation to datasets that end in synchronous transformations. This enables data generation to be overlapped with data consumption. This may cause a small increase in memory usage due to buffering. To enable this behavior, set inject_prefetch=True in tf.data.experimental.OptimizationOptions.
    • Added a new value to tf.data.Options.autotune.autotune_algorithm: STAGEBASED. If the autotune algorithm is set to STAGEBASED, then it runs a new algorithm that can get the same performance with lower CPU/memory usage.
    • Added tf.data.experimental.from_list, a new API for creating Datasets from lists of elements.

  • tf.distribute:

    • Added tf.distribute.experimental.PreemptionCheckpointHandler to handle worker preemption/maintenance and cluster-wise consistent error reporting for tf.distribute.MultiWorkerMirroredStrategy. Specifically, for the type of interruption with advance notice, it automatically saves a checkpoint, exits the program without raising an unrecoverable error, and restores the progress when training restarts.

  • tf.math:

    • Added tf.math.approx_max_k and tf.math.approx_min_k which are the optimized alternatives to tf.math.top_k on TPU. The performance difference range from 8 to 100 times depending on the size of k. When running on CPU and GPU, a non-optimized XLA kernel is used.

  • tf.train:

    • Added tf.train.TrackableView which allows users to inspect the TensorFlow Trackable object (e.g. tf.Module, Keras Layers and models).

  • tf.vectorized_map:

    • Added an optional parameter: warn. This parameter controls whether or not warnings will be printed when operations in the provided fn fall back to a while loop.

  • XLA:

    • MWMS is now compilable with XLA.

  • oneDNN CPU performance optimizations:

    • x86 CPUs: oneDNN bfloat16 auto-mixed precision grappler graph optimization pass has been renamed from auto_mixed_precision_mkl to auto_mixed_precision_onednn_bfloat16. See example usage here.
    • aarch64 CPUs: Experimental oneDNN optimizations are available in the default Linux aarch64 package (pip install tensorflow).
      • The optimizations are disabled by default.
      • Set the environment variable TF_ENABLE_ONEDNN_OPTS=1 to enable the optimizations. Setting the variable to 0 or unsetting it will disable the optimizations.
      • These optimizations can yield slightly different numerical results from when they are off due to floating-point round-off errors from different computation approaches and orders.
      • To verify that the optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

Bug Fixes and Other Changes

  • New argument experimental_device_ordinal in LogicalDeviceConfiguration to control the order of logical devices. (GPU only)

  • tf.keras:

    • Changed the TensorBoard tag names produced by the tf.keras.callbacks.TensorBoard callback, so that summaries logged automatically for model weights now include either a /histogram or /image suffix in their tag names, in order to prevent tag name collisions across summary types.

  • When running on GPU (with cuDNN version 7.6.3 or later),tf.nn.depthwise_conv2d backprop to filter (and therefore also tf.keras.layers.DepthwiseConv2D) now operate deterministically (and tf.errors.UnimplementedError is no longer thrown) when op-determinism has been enabled via tf.config.experimental.enable_op_determinism. This closes issue 47174.

  • tf.random

    • Added tf.random.experimental.stateless_shuffle, a stateless version of tf.random.shuffle.

Deprecations

  • The C++ tensorflow::Code and tensorflow::Status will become aliases of respectively absl::StatusCode and absl::Status in some future release.
    • Use tensorflow::OkStatus() instead of tensorflow::Status::OK().
    • Stop constructing Status objects from tensorflow::error::Code.
    • One MUST NOT access tensorflow::errors::Code fields. Accessing tensorflow::error::Code fields is fine.
      • Use the constructors such as tensorflow::errors:InvalidArgument to create status using an error code without accessing it.
      • Use the free functions such as tensorflow::errors::IsInvalidArgument if needed.
      • In the last resort, use e.g.static_cast<tensorflow::errors::Code>(error::Code::INVALID_ARGUMENT) or static_cast<int>(code) for comparisons.
  • tensorflow::StatusOr will also become in the future alias to absl::StatusOr, so use StatusOr::value instead of StatusOr::ConsumeValueOrDie.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Abolfazl Shahbazi, Adam Lanicek, Amin Benarieb, andreii, Andrew Fitzgibbon, Andrew Goodbody, angerson, Ashiq Imran, Aurélien Geron, Banikumar Maiti (Intel Aipg), Ben Barsdell, Ben Mares, bhack, Bhavani Subramanian, Bill Schnurr, Byungsoo Oh, Chandra Sr Potula, Chengji Yao, Chris Carpita, Christopher Bate, chunduriv, Cliff Woolley, Cliffs Dover, Cloud Han, Code-Review-Doctor, DEKHTIARJonathan, Deven Desai, Djacon, Duncan Riach, fedotoff, fo40225, Frederic Bastien, gadagashwini, Gauri1 Deshpande, guozhong.zhuang, Hui Peng, James Gerity, Jason Furmanek, Jonathan Dekhtiar, Jueon Park, Kaixi Hou, Kanvi Khanna, Keith Smiley, Koan-Sin Tan, Kulin Seth, kushanam, Learning-To-Play, Li-Wen Chang, lipracer, liuyuanqiang, Louis Sugy, Lucas David, Lukas Geiger, Mahmoud Abuzaina, Marius Brehler, Maxiwell S. Garcia, mdfaijul, Meenakshi Venkataraman, Michal Szutenberg, Michele Di Giorgio, Mickaël Salamin, Nathan John Sircombe, Nathan Luehr, Neil Girdhar, Nils Reichardt, Nishidha Panpaliya, Nobuo Tsukamoto, Om Thakkar, Patrice Vignola, Philipp Hack, Pooya Jannaty, Prianka Liz Kariat, pshiko, Rajeshwar Reddy T, rdl4199, Rohit Santhanam, Rsanthanam-Amd, Sachin Muradi, Saoirse Stewart, Serge Panev, Shu Wang, Srinivasan Narayanamoorthy, Stella Stamenova, Stephan Hartmann, Sunita Nadampalli, synandi, Tamas Bela Feher, Tao Xu, Thibaut Goetghebuer-Planchon, Trevor Morris, Xiaoming (Jason) Cui, Yimei Sun, Yong Tang, Yuanqiang Liu, Yulv-Git, Zhoulong Jiang, ZihengJiang

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.10.0-rc0

Release 2.10.0

Breaking Changes

  • Causal attention in keras.layers.Attention and keras.layers.AdditiveAttention is now specified in the call() method via the use_causal_mask argument (rather than in the constructor), for consistency with other layers.
  • Some files in tensorflow/python/training have been moved to tensorflow/python/tracking and tensorflow/python/checkpoint. Please update your imports accordingly, the old files will be removed in Release 2.11.
  • tf.keras.optimizers.experimental.Optimizer will graduate in Release 2.11, which means tf.keras.optimizers.Optimizer will be an alias of tf.keras.optimizers.experimental.Optimizer. The current tf.keras.optimizers.Optimizer will continue to be supported as tf.keras.optimizers.legacy.Optimizer, e.g., tf.keras.optimizers.legacy.Adam. Most users won't be affected by this change, but please check the API doc if any API used in your workflow is changed or deprecated, and make adaptions. If you decide to keep using the old optimizer, please explicitly change your optimizer to tf.keras.optimizers.legacy.Optimizer.
  • RNG behavior change for tf.keras.initializers. Keras initializers will now use stateless random ops to generate random numbers.
    • Both seeded and unseeded initializers will always generate the same values every time they are called (for a given variable shape). For unseeded initializers (seed=None), a random seed will be created and assigned at initializer creation (different initializer instances get different seeds).
    • An unseeded initializer will raise a warning if it is reused (called) multiple times. This is because it would produce the same values each time, which may not be intended.

Major Features and Improvements

  • tf.lite:

    • New operations supported:
      • tflite SelectV2 now supports 5D.
      • tf.einsum is supported with multiple unknown shapes.
      • tf.unsortedsegmentprod op is supported.
      • tf.unsortedsegmentmax op is supported.
      • tf.unsortedsegmentsum op is supported.
    • Updates to existing operations:
      • tfl.scatter_nd now supports I1 for update arg.
    • Upgrade Flatbuffers v2.0.5 from v1.12.0

  • tf.keras:

    • EinsumDense layer moved from experimental to core. Its import path moved from tf.keras.layers.experimental.EinsumDense to tf.keras.layers.EinsumDense.
    • Added tf.keras.utils.audio_dataset_from_directory utility to easily generate audio classification datasets from directories of .wav files.
    • Added subset="both" support in tf.keras.utils.image_dataset_from_directory, tf.keras.utils.text_dataset_from_directory, and audio_dataset_from_directory, to be used with the validation_split argument, for returning both dataset splits at once, as a tuple.
    • Added tf.keras.utils.split_dataset utility to split a Dataset object or a list/tuple of arrays into two Dataset objects (e.g. train/test).
    • Added step granularity to BackupAndRestore callback for handling distributed training failures & restarts. The training state can now be restored at the exact epoch and step at which it was previously saved before failing.
    • Added tf.keras.dtensor.experimental.optimizers.AdamW. This optimizer is similar as the existing keras.optimizers.experimental.AdamW, and works in the DTensor training use case.
    • Improved masking support for tf.keras.layers.MultiHeadAttention.
      • Implicit masks for query, key and value inputs will automatically be used to compute a correct attention mask for the layer. These padding masks will be combined with any attention_mask passed in directly when calling the layer. This can be used with tf.keras.layers.Embedding with mask_zero=True to automatically infer a correct padding mask.
      • Added a use_causal_mask call time arugment to the layer. Passing use_causal_mask=True will compute a causal attention mask, and optionally combine it with any attention_mask passed in directly when calling the layer.
    • Added ignore_class argument in the loss SparseCategoricalCrossentropy and metrics IoU and MeanIoU, to specify a class index to be ignored during loss/metric computation (e.g. a background/void class).
    • Added tf.keras.models.experimental.SharpnessAwareMinimization. This class implements the sharpness-aware minimization technique, which boosts model performance on various tasks, e.g., ResNet on image classification.

  • tf.data:

    • Added support for cross-trainer data caching in tf.data service. This saves computation resources when concurrent training jobs train from the same dataset. See https://www.tensorflow.org/apidocs/python/tf/data/experimental/service#sharingtfdataservicewithconcurrenttrainers for more details.
    • Added dataset_id to tf.data.experimental.service.register_dataset. If provided, tf.data service will use the provided ID for the dataset. If the dataset ID already exists, no new dataset will be registered. This is useful if multiple training jobs need to use the same dataset for training. In this case, users should call register_dataset with the same dataset_id.
    • Added a new field, inject_prefetch, to tf.data.experimental.OptimizationOptions. If it is set to True, tf.data will now automatically add a prefetch transformation to datasets that end in synchronous transformations. This enables data generation to be overlapped with data consumption. This may cause a small increase in memory usage due to buffering. To enable this behavior, set inject_prefetch=True in tf.data.experimental.OptimizationOptions.
    • Added a new value to tf.data.Options.autotune.autotune_algorithm: STAGEBASED. If the autotune algorithm is set to STAGEBASED, then it runs a new algorithm that can get the same performance with lower CPU/memory usage.
    • Added tf.data.experimental.from_list, a new API for creating Datasets from lists of elements.

  • tf.distribute:

    • Added tf.distribute.experimental.PreemptionCheckpointHandler to handle worker preemption/maintenance and cluster-wise consistent error reporting for tf.distribute.MultiWorkerMirroredStrategy. Specifically, for the type of interruption with advance notice, it automatically saves a checkpoint, exits the program without raising an unrecoverable error, and restores the progress when training restarts.

  • tf.math:

    • Added tf.math.approx_max_k and tf.math.approx_min_k which are the optimized alternatives to tf.math.top_k on TPU. The performance difference range from 8 to 100 times depending on the size of k. When running on CPU and GPU, a non-optimized XLA kernel is used.

  • tf.train:

    • Added tf.train.TrackableView which allows users to inspect the TensorFlow Trackable object (e.g. tf.Module, Keras Layers and models).

  • tf.vectorized_map:

    • Added an optional parameter: warn. This parameter controls whether or not warnings will be printed when operations in the provided fn fall back to a while loop.

  • XLA:

    • MWMS is now compilable with XLA.

Bug Fixes and Other Changes

  • New argument experimental_device_ordinal in LogicalDeviceConfiguration to control the order of logical devices. (GPU only)

  • tf.keras:

    • Changed the TensorBoard tag names produced by the tf.keras.callbacks.TensorBoard callback, so that summaries logged automatically for model weights now include either a /histogram or /image suffix in their tag names, in order to prevent tag name collisions across summary types.

  • When running on GPU (with cuDNN version 7.6.3 or later), tf.nn.depthwise_conv2d backprop to filter (and therefore also tf.keras.layers.DepthwiseConv2D) now operate deterministically (and tf.errors.UnimplementedError is no longer thrown) when op-determinism has been enabled via tf.config.experimental.enable_op_determinism. This closes issue 47174.

  • tf.random

    • Added tf.random.experimental.stateless_shuffle, a stateless version of tf.random.shuffle.

Deprecations

  • The C++ tensorflow::Code and tensorflow::Status will become aliases of respectively absl::StatusCode and absl::Status in some future release.
    • Use tensorflow::OkStatus() instead of tensorflow::Status::OK().
    • Stop constructing Status objects from tensorflow::error::Code.
    • One MUST NOT access tensorflow::errors::Code fields. Accessing tensorflow::error::Code fields is fine.
      • Use the constructors such as tensorflow::errors:InvalidArgument to create status using an error code without accessing it.
      • Use the free functions such as tensorflow::errors::IsInvalidArgument if needed.
      • In the last resort, use e.g. static_cast<tensorflow::errors::Code>(error::Code::INVALID_ARGUMENT) or static_cast<int>(code) for comparisons.
  • tensorflow::StatusOr will also become in the future alias to absl::StatusOr, so use StatusOr::value instead of StatusOr::ConsumeValueOrDie.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Abolfazl Shahbazi, Adam Lanicek, Amin Benarieb, andreii, Andrew Fitzgibbon, Andrew Goodbody, angerson, Ashiq Imran, Aurélien Geron, Banikumar Maiti (Intel Aipg), Ben Barsdell, Ben Mares, bhack, Bhavani Subramanian, Bill Schnurr, Byungsoo Oh, Chandra Sr Potula, Chengji Yao, Chris Carpita, Christopher Bate, chunduriv, Cliff Woolley, Cliffs Dover, Cloud Han, Code-Review-Doctor, DEKHTIARJonathan, Deven Desai, Djacon, Duncan Riach, fedotoff, fo40225, Frederic Bastien, gadagashwini, Gauri1 Deshpande, guozhong.zhuang, Hui Peng, James Gerity, Jason Furmanek, Jonathan Dekhtiar, Jueon Park, Kaixi Hou, Kanvi Khanna, Keith Smiley, Koan-Sin Tan, Kulin Seth, kushanam, Learning-To-Play, Li-Wen Chang, lipracer, liuyuanqiang, Louis Sugy, Lucas David, Lukas Geiger, Mahmoud Abuzaina, Marius Brehler, Maxiwell S. Garcia, mdfaijul, Meenakshi Venkataraman, Michal Szutenberg, Michele Di Giorgio, Mickaël Salamin, Nathan John Sircombe, Nathan Luehr, Neil Girdhar, Nils Reichardt, Nishidha Panpaliya, Nobuo Tsukamoto, Om Thakkar, Patrice Vignola, Philipp Hack, Pooya Jannaty, Prianka Liz Kariat, pshiko, Rajeshwar Reddy T, rdl4199, Rohit Santhanam, Rsanthanam-Amd, Sachin Muradi, Saoirse Stewart, Serge Panev, Shu Wang, Srinivasan Narayanamoorthy, Stella Stamenova, Stephan Hartmann, Sunita Nadampalli, synandi, Tamas Bela Feher, Tao Xu, Thibaut Goetghebuer-Planchon, Trevor Morris, Xiaoming (Jason) Cui, Yimei Sun, Yong Tang, Yuanqiang Liu, Yulv-Git, Zhoulong Jiang, ZihengJiang

- C++
Published by tensorflow-jenkins over 3 years ago

tensorflow - TensorFlow 2.9.1

Release 2.9.1

Add an upper bound for protobuf in setup.py since protobuf after version 3.20 is currently incompatible with TensorFlow. See https://github.com/tensorflow/tensorflow/issues/53234, https://github.com/protocolbuffers/protobuf/issues/9954 and https://github.com/tensorflow/tensorflow/issues/56077.

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.8.2

Release 2.8.2

Add an upper bound for protobuf in setup.py since protobuf after version 3.20 is currently incompatible with TensorFlow. See https://github.com/tensorflow/tensorflow/issues/53234, https://github.com/protocolbuffers/protobuf/issues/9954 and https://github.com/tensorflow/tensorflow/issues/56077.

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.7.3

Release 2.7.3

Add an upper bound for protobuf in setup.py since protobuf after version 3.20 is currently incompatible with TensorFlow. See https://github.com/tensorflow/tensorflow/issues/53234, https://github.com/protocolbuffers/protobuf/issues/9954 and https://github.com/tensorflow/tensorflow/issues/56077.

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.6.5

Release 2.6.5

Add an upper bound for protobuf in setup.py since protobuf after version 3.20 is currently incompatible with TensorFlow. See https://github.com/tensorflow/tensorflow/issues/53234, https://github.com/protocolbuffers/protobuf/issues/9954 and https://github.com/tensorflow/tensorflow/issues/56077.

This is the final release in the 2.6.x series.

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.9.0

Release 2.9.0

Breaking Changes

  • Due to security issues in TF 2.8, all boosted trees code has now been removed (after being deprecated in TF 2.8). Users should switch to TensorFlow Decision Forests.
  • Build, Compilation and Packaging
    • TensorFlow is now compiled with _GLIBCXX_USE_CXX11_ABI=1. Downstream projects that encounter std::__cxx11 or [abi:cxx11] linker errors will need to adopt this compiler option. See the GNU C++ Library docs on Dual ABI.
    • TensorFlow Python wheels now specifically conform to manylinux2014, an upgrade from manylinux2010. The minimum Pip version supporting manylinux2014 is Pip 19.3 (see pypa/manylinux. This change may affect you if you have been using TensorFlow on a very old platform equivalent to CentOS 6, as manylinux2014 targets CentOS 7 as a compatibility base. Note that TensorFlow does not officially support either platform.
    • Discussion for these changes can be found on SIG Build's TensorFlow Community Forum thread
  • The tf.keras.mixed_precision.experimental API has been removed. The non-experimental symbols under tf.keras.mixed_precision have been available since TensorFlow 2.4 and should be used instead.
    • The non-experimental API has some minor differences from the experimental API. In most cases, you only need to make three minor changes:
      • Remove the word "experimental" from tf.keras.mixed_precision symbols. E.g., replace tf.keras.mixed_precision.experimental.global_policy with tf.keras.mixed_precision.global_policy.
      • Replace tf.keras.mixed_precision.experimental.set_policy with tf.keras.mixed_precision.set_global_policy. The experimental symbol set_policy was renamed to set_global_policy in the non-experimental API.
      • Replace LossScaleOptimizer(opt, "dynamic") with LossScaleOptimizer(opt). If you pass anything other than "dynamic" to the second argument, see (1) of the next section.
    • In the following rare cases, you need to make more changes when switching to the non-experimental API:
      • If you passed anything other than "dynamic" to the loss_scale argument (the second argument) of LossScaleOptimizer:
      • If you passed a value to the loss_scale argument (the second argument) of Policy:
        • The experimental version of Policy optionally took in a tf.compat.v1.mixed_precision.LossScale in the constructor, which defaulted to a dynamic loss scale for the "mixed_float16" policy and no loss scale for other policies. In Model.compile, if the model's policy had a loss scale, the optimizer would be wrapped with a LossScaleOptimizer. With the non-experimental Policy, there is no loss scale associated with the Policy, and Model.compile wraps the optimizer with a LossScaleOptimizer if and only if the policy is a "mixed_float16" policy. If you previously passed a LossScale to the experimental Policy, consider just removing it, as the default loss scaling behavior is usually what you want. If you really want to customize the loss scaling behavior, you can wrap your optimizer with a LossScaleOptimizer before passing it to Model.compile.
      • If you use the very rarely-used function tf.keras.mixed_precision.experimental.get_layer_policy:
        • Replace tf.keras.mixed_precision.experimental.get_layer_policy(layer) with layer.dtype_policy.
  • tf.mixed_precision.experimental.LossScale and its subclasses have been removed from the TF2 namespace. This symbols were very rarely used and were only useful in TF2 for use in the now-removed tf.keras.mixed_precision.experimental API. The symbols are still available under tf.compat.v1.mixed_precision.
  • The experimental_relax_shapes heuristic for tf.function has been deprecated and replaced with reduce_retracing which encompasses broader heuristics to reduce the number of retraces (see below)

Major Features and Improvements

  • tf.keras:

    • Added tf.keras.applications.resnet_rs models. This includes the ResNetRS50, ResNetRS101, ResNetRS152, ResNetRS200, ResNetRS270, ResNetRS350 and ResNetRS420 model architectures. The ResNetRS models are based on the architecture described in Revisiting ResNets: Improved Training and Scaling Strategies
    • Added tf.keras.optimizers.experimental.Optimizer. The reworked optimizer gives more control over different phases of optimizer calls, and is easier to customize. We provide Adam, SGD, Adadelta, AdaGrad and RMSprop optimizers based on tf.keras.optimizers.experimental.Optimizer. Generally the new optimizers work in the same way as the old ones, but support new constructor arguments. In the future, the symbols tf.keras.optimizers.Optimizer/Adam/etc will point to the new optimizers, and the previous generation of optimizers will be moved to tf.keras.optimizers.legacy.Optimizer/Adam/etc.
    • Added L2 unit normalization layer tf.keras.layers.UnitNormalization.
    • Added tf.keras.regularizers.OrthogonalRegularizer, a new regularizer that encourages orthogonality between the rows (or columns) or a weight matrix.
    • Added tf.keras.layers.RandomBrightness layer for image preprocessing.
    • Added APIs for switching between interactive logging and absl logging. By default, Keras always writes the logs to stdout. However, this is not optimal in a non-interactive environment, where you don't have access to stdout, but can only view the logs. You can use tf.keras.utils.disable_interactive_logging() to write the logs to ABSL logging. You can also use tf.keras.utils.enable_interactive_logging() to change it back to stdout, or tf.keras.utils.is_interactive_logging_enabled() to check if interactive logging is enabled.
    • Changed default value for the verbose argument of Model.evaluate() and Model.predict() to "auto", which defaults to verbose=1 for most cases and defaults to verbose=2 when used with ParameterServerStrategy or with interactive logging disabled.
    • Argument jit_compile in Model.compile() now applies to Model.evaluate() and Model.predict(). Setting jit_compile=True in compile() compiles the model's training, evaluation, and inference steps to XLA. Note that jit_compile=True may not necessarily work for all models.
    • Added DTensor-related Keras APIs under tf.keras.dtensor namespace. The APIs are still classified as experimental. You are welcome to try it out. Please check the tutoral and guide on https://www.tensorflow.org/ for more details about DTensor.

  • tf.lite:

    • Added TFLite builtin op support for the following TF ops:
      • tf.math.argmin/tf.math.argmax for input data type tf.bool on CPU.
      • tf.nn.gelu op for output data type tf.float32 and quantization on CPU.
    • Add nominal support for unsigned 16-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for unsigned 16-bit integer tensor types in cast op.
    • Experimental support for lowering list_ops.tensor_list_set_item with DynamicUpdateSlice.
    • Enabled a new MLIR-based dynamic range quantization backend by default
      • The new backend is used for post-training int8 dynamic range quantization and post-training float16 quantization.
      • Set experimental_new_dynamic_range_quantizer in tf.lite.TFLiteConverter to False to disable this change
    • Native TF Lite variables are now enabled during conversion by default on all v2 TfLiteConverter entry points. experimental_enable_resource_variables on tf.lite.TFLiteConverter is now True by default and will be removed in the future.

  • tf.function:

    • Custom classes used as arguments for tf.function can now specify rules regarding when retracing needs to occur by implementing the Tracing Protocol available through tf.types.experimental.SupportsTracingProtocol.
    • TypeSpec classes (as associated with ExtensionTypes) also implement the Tracing Protocol which can be overriden if necessary.
    • The newly introduced reduce_retracing option also uses the Tracing Protocol to proactively generate generalized traces similar to experimental_relax_shapes (which has now been deprecated).

  • Unified eager and tf.function execution:

    • Eager mode can now execute each op as a tf.function, allowing for more consistent feature support in future releases.
    • It is available for immediate use.
      • See the TF_RUN_EAGER_OP_AS_FUNCTION environment variable in eager context.
      • Eager performance should be similar with this feature enabled.
        • A roughly 5us per-op overhead may be observed when running many small functions.
        • Note a known issue with GPU performance.
      • The behavior of tf.function itself is unaffected.
    • Note: This feature will be enabled by default in an upcoming version of TensorFlow.

  • tf.experimental.dtensor: Added DTensor, an extension to TensorFlow for large-scale modeling with minimal changes to user code. You are welcome to try it out, though be aware that the DTensor API is experimental and up-to backward-incompatible changes. DTensor and Keras integration is published under tf.keras.dtensor in this release (refer to the tf.keras entry). The tutoral and guide for DTensor will be published on https://www.tensorflow.org/. Please stay tuned.

  • oneDNN CPU performance optimizations are available in Linux x86, Windows x86, and Linux aarch64 packages.

    • Linux x86 packages:
      • oneDNN optimizations are enabled by default on CPUs with neural-network-focused hardware features such as AVX512VNNI, AVX512BF16, AMX, etc. (Intel Cascade Lake and newer CPUs.)
      • For older CPUs, oneDNN optimizations are disabled by default.
    • Windows x86 package: oneDNN optimizations are disabled by default.
    • Linux aach64 (--config=mkl_aarch64) package:
      • Experimental oneDNN optimizations are disabled by default.
      • If you experience issues with oneDNN optimizations on, we recommend turning them off.
    • To explicitly enable or disable oneDNN optimizations, set the environment variable TF_ENABLE_ONEDNN_OPTS to 1 (enable) or 0 (disable) before running TensorFlow. (The variable is checked during import tensorflow.) To fall back to default settings, unset the environment variable.
    • These optimizations can yield slightly different numerical results from when they are off due to floating-point round-off errors from different computation approaches and orders.
    • To verify that the optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

Bug Fixes and Other Changes

  • tf.data:

    • Fixed bug in tf.data.experimental.parse_example_dataset when tf.io.RaggedFeatures would specify value_key but no partitions. Before the fix, setting value_key but no partitions would result in the feature key being replaced by the value key, e.g. {'value_key': <RaggedTensor>} instead of {'key': <RaggedTensor>}. Now the correct feature key will be used. This aligns the behavior of tf.data.experimental.parse_example_dataset to match the behavior of tf.io.parse_example.
    • Added a new field, filter_parallelization, to tf.data.experimental.OptimizationOptions. If it is set to True, tf.data will run Filter transformation with multiple threads. Its default value is False if not specified.

  • tf.keras:

    • Fixed bug in optimizers that prevented them from properly checkpointing slot variables when they are ShardedVariables (used for training with tf.distribute.experimental.ParameterServerStrategy).

  • tf.random:

    • Added tf.random.experimental.index_shuffle, for shuffling a sequence without materializing the sequence in memory.

  • tf.RaggedTensor:

    • Introduced tf.experimental.RowPartition, which encodes how one dimension in a RaggedTensor relates to another, into the public API.
    • Introduced tf.experimental.DynamicRaggedShape, which represents the shape of a RaggedTensor.

Security

  • Fixes a code injection in saved_model_cli (CVE-2022-29216)
  • Fixes a missing validation which causes TensorSummaryV2 to crash (CVE-2022-29193)
  • Fixes a missing validation which crashes QuantizeAndDequantizeV4Grad (CVE-2022-29192)
  • Fixes a missing validation which causes denial of service via DeleteSessionTensor (CVE-2022-29194)
  • Fixes a missing validation which causes denial of service via GetSessionTensor (CVE-2022-29191)
  • Fixes a missing validation which causes denial of service via StagePeek (CVE-2022-29195)
  • Fixes a missing validation which causes denial of service via UnsortedSegmentJoin (CVE-2022-29197)
  • Fixes a missing validation which causes denial of service via LoadAndRemapMatrix (CVE-2022-29199)
  • Fixes a missing validation which causes denial of service via SparseTensorToCSRSparseMatrix (CVE-2022-29198)
  • Fixes a missing validation which causes denial of service via LSTMBlockCell (CVE-2022-29200)
  • Fixes a missing validation which causes denial of service via Conv3DBackpropFilterV2 (CVE-2022-29196)
  • Fixes a CHECK failure in depthwise ops via overflows (CVE-2021-41197)
  • Fixes issues arising from undefined behavior stemming from users supplying invalid resource handles (CVE-2022-29207)
  • Fixes a segfault due to missing support for quantized types (CVE-2022-29205)
  • Fixes a missing validation which results in undefined behavior in SparseTensorDenseAdd (CVE-2022-29206)
  • Fixes a missing validation which results in undefined behavior in QuantizedConv2D (CVE-2022-29201)
  • Fixes an integer overflow in SpaceToBatchND (CVE-2022-29203)
  • Fixes a segfault and OOB write due to incomplete validation in EditDistance (CVE-2022-29208)
  • Fixes a missing validation which causes denial of service via Conv3DBackpropFilterV2 (CVE-2022-29204)
  • Fixes a denial of service in tf.ragged.constant due to lack of validation (CVE-2022-29202)
  • Fixes a segfault when tf.histogram_fixed_width is called with NaN values (CVE-2022-29211)
  • Fixes a core dump when loading TFLite models with quantization (CVE-2022-29212)
  • Fixes crashes stemming from incomplete validation in signal ops (CVE-2022-29213)
  • Fixes a type confusion leading to CHECK-failure based denial of service (CVE-2022-29209)
  • Fixes a heap buffer overflow due to incorrect hash function (CVE-2022-29210)
  • Updates curl to 7.83.1 to handle (CVE-2022-22576, (CVE-2022-27774, (CVE-2022-27775, (CVE-2022-27776, (CVE-2022-27778, (CVE-2022-27779, (CVE-2022-27780, (CVE-2022-27781, (CVE-2022-27782 and (CVE-2022-30115
  • Updates zlib to 1.2.12 after 1.2.11 was pulled due to security issue # Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aaron Debattista, Abel Soares Siqueira, Abhishek Varma, Andrei Ivanov, andreii, Andrew Goodbody, apeltop, Arnab Dutta, Ashiq Imran, Banikumar Maiti (Intel Aipg), Ben Greiner, Benjamin Peterson, bhack, Christopher Bate, chunduriv, Copybara-Service, DEKHTIARJonathan, Deven Desai, Duncan Riach, Eric Kunze, Everton Constantino, Faruk D, Fredrik Knutsson, gadagashwini, Gauri1 Deshpande, gtiHibGele, Guozhong Zhuang, Islem-Esi, Ivanov Viktor, Jason Furmanek, Jason Zaman, Jim, Jinzhe Zeng, John Laxson, Jonas Eschle, Jonas Eschle 'Mayou36, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, KaurkerDevourer, Koan-Sin Tan, kushanam, Laramie Leavitt, Li-Wen Chang, lipracer, Louis Sugy, Lu Teng, Mahmoud Abuzaina, Malcolm Slaney, Malik Shahzad Muzaffar, Marek Šuppa, Matt Conley, Michael Melesse, Milos Puzovic, mohantym, Nathan John Sircombe, Nathan Luehr, Nilesh Agarwalla, Patrice Vignola, peterjc123, Philip Turner, Rajeshwar Reddy T, Robert Kalmar, Rodrigo Formigone, Rohit Santhanam, rui, Sachin Muradi, Saduf2019, sandip, Scott Leishman, Serge Panev, Shi,Guangyong, Srinivasan Narayanamoorthy, stanley, Steven I Reeves, stevenireeves, sushreebarsa, Tamas Bela Feher, Tao He, Thomas Schmeyer, Tiago Almeida, Trevor Morris, Uday Bondhugula, Uwe L. Korn, Varghese, Jojimon, Vishnuvardhan Janapati, William Muir, William Raveane, xutianming, Yasuhiro Matsumoto, Yimei Sun, Yong Tang, Yu Feng, Yuriy Chernyshov, zhaozheng09

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.8.1

Release 2.8.1

This releases introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.7.2

Release 2.7.2

This releases introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.6.4

Release 2.6.4

This releases introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.9.0-rc2

Release 2.9.0

Breaking Changes

  • Due to security issues in TF 2.8, all boosted trees code has now been removed (after being deprecated in TF 2.8). Users should switch to TensorFlow Decision Forests.
  • Build, Compilation and Packaging
    • TensorFlow is now compiled with _GLIBCXX_USE_CXX11_ABI=1. Downstream projects that encounter std::__cxx11 or [abi:cxx11] linker errors will need to adopt this compiler option. See the GNU C++ Library docs on Dual ABI.
    • TensorFlow Python wheels now specifically conform to manylinux2014, an upgrade from manylinux2010. The minimum Pip version supporting manylinux2014 is Pip 19.3 (see pypa/manylinux. This change may affect you if you have been using TensorFlow on a very old platform equivalent to CentOS 6, as manylinux2014 targets CentOS 7 as a compatibility base. Note that TensorFlow does not officially support either platform.
    • Discussion for these changes can be found on SIG Build's TensorFlow Community Forum thread
  • The tf.keras.mixed_precision.experimental API has been removed. The non-experimental symbols under tf.keras.mixed_precision have been available since TensorFlow 2.4 and should be used instead.
    • The non-experimental API has some minor differences from the experimental API. In most cases, you only need to make three minor changes:
      • Remove the word "experimental" from tf.keras.mixed_precision symbols. E.g., replace tf.keras.mixed_precision.experimental.global_policy with tf.keras.mixed_precision.global_policy.
      • Replace tf.keras.mixed_precision.experimental.set_policy with tf.keras.mixed_precision.set_global_policy. The experimental symbol set_policy was renamed to set_global_policy in the non-experimental API.
      • Replace LossScaleOptimizer(opt, "dynamic") with LossScaleOptimizer(opt). If you pass anything other than "dynamic" to the second argument, see (1) of the next section.
    • In the following rare cases, you need to make more changes when switching to the non-experimental API:
      • If you passed anything other than "dynamic" to the loss_scale argument (the second argument) of LossScaleOptimizer:
      • If you passed a value to the loss_scale argument (the second argument) of Policy:
        • The experimental version of Policy optionally took in a tf.compat.v1.mixed_precision.LossScale in the constructor, which defaulted to a dynamic loss scale for the "mixed_float16" policy and no loss scale for other policies. In Model.compile, if the model's policy had a loss scale, the optimizer would be wrapped with a LossScaleOptimizer. With the non-experimental Policy, there is no loss scale associated with the Policy, and Model.compile wraps the optimizer with a LossScaleOptimizer if and only if the policy is a "mixed_float16" policy. If you previously passed a LossScale to the experimental Policy, consider just removing it, as the default loss scaling behavior is usually what you want. If you really want to customize the loss scaling behavior, you can wrap your optimizer with a LossScaleOptimizer before passing it to Model.compile.
      • If you use the very rarely-used function tf.keras.mixed_precision.experimental.get_layer_policy:
        • Replace tf.keras.mixed_precision.experimental.get_layer_policy(layer) with layer.dtype_policy.
  • tf.mixed_precision.experimental.LossScale and its subclasses have been removed from the TF2 namespace. This symbols were very rarely used and were only useful in TF2 for use in the now-removed tf.keras.mixed_precision.experimental API. The symbols are still available under tf.compat.v1.mixed_precision.
  • The experimental_relax_shapes heuristic for tf.function has been deprecated and replaced with reduce_retracing which encompasses broader heuristics to reduce the number of retraces (see below)

Major Features and Improvements

  • tf.keras:

    • Added tf.keras.applications.resnet_rs models. This includes the ResNetRS50, ResNetRS101, ResNetRS152, ResNetRS200, ResNetRS270, ResNetRS350 and ResNetRS420 model architectures. The ResNetRS models are based on the architecture described in Revisiting ResNets: Improved Training and Scaling Strategies
    • Added tf.keras.optimizers.experimental.Optimizer. The reworked optimizer gives more control over different phases of optimizer calls, and is easier to customize. We provide Adam, SGD, Adadelta, AdaGrad and RMSprop optimizers based on tf.keras.optimizers.experimental.Optimizer. Generally the new optimizers work in the same way as the old ones, but support new constructor arguments. In the future, the symbols tf.keras.optimizers.Optimizer/Adam/etc will point to the new optimizers, and the previous generation of optimizers will be moved to tf.keras.optimizers.legacy.Optimizer/Adam/etc.
    • Added L2 unit normalization layer tf.keras.layers.UnitNormalization.
    • Added tf.keras.regularizers.OrthogonalRegularizer, a new regularizer that encourages orthogonality between the rows (or columns) or a weight matrix.
    • Added tf.keras.layers.RandomBrightness layer for image preprocessing.
    • Added APIs for switching between interactive logging and absl logging. By default, Keras always writes the logs to stdout. However, this is not optimal in a non-interactive environment, where you don't have access to stdout, but can only view the logs. You can use tf.keras.utils.disable_interactive_logging() to write the logs to ABSL logging. You can also use tf.keras.utils.enable_interactive_logging() to change it back to stdout, or tf.keras.utils.is_interactive_logging_enabled() to check if interactive logging is enabled.
    • Changed default value for the verbose argument of Model.evaluate() and Model.predict() to "auto", which defaults to verbose=1 for most cases and defaults to verbose=2 when used with ParameterServerStrategy or with interactive logging disabled.
    • Argument jit_compile in Model.compile() now applies to Model.evaluate() and Model.predict(). Setting jit_compile=True in compile() compiles the model's training, evaluation, and inference steps to XLA. Note that jit_compile=True may not necessarily work for all models.
    • Added DTensor-related Keras APIs under tf.keras.dtensor namespace. The APIs are still classified as experimental. You are welcome to try it out. Please check the tutoral and guide on https://www.tensorflow.org/ for more details about DTensor.

  • tf.lite:

    • Added TFLite builtin op support for the following TF ops:
      • tf.math.argmin/tf.math.argmax for input data type tf.bool on CPU.
      • tf.nn.gelu op for output data type tf.float32 and quantization on CPU.
    • Add nominal support for unsigned 16-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for unsigned 16-bit integer tensor types in cast op.
    • Experimental support for lowering list_ops.tensor_list_set_item with DynamicUpdateSlice.
    • Enabled a new MLIR-based dynamic range quantization backend by default
      • The new backend is used for post-training int8 dynamic range quantization and post-training float16 quantization.
      • Set experimental_new_dynamic_range_quantizer in tf.lite.TFLiteConverter to False to disable this change
    • Native TF Lite variables are now enabled during conversion by default on all v2 TfLiteConverter entry points. experimental_enable_resource_variables on tf.lite.TFLiteConverter is now True by default and will be removed in the future.

  • tf.function:

    • Custom classes used as arguments for tf.function can now specify rules regarding when retracing needs to occur by implementing the Tracing Protocol available through tf.types.experimental.SupportsTracingProtocol.
    • TypeSpec classes (as associated with ExtensionTypes) also implement the Tracing Protocol which can be overriden if necessary.
    • The newly introduced reduce_retracing option also uses the Tracing Protocol to proactively generate generalized traces similar to experimental_relax_shapes (which has now been deprecated).

  • Unified eager and tf.function execution:

    • Eager mode can now execute each op as a tf.function, allowing for more consistent feature support in future releases.
    • It is available for immediate use.
      • See the TF_RUN_EAGER_OP_AS_FUNCTION environment variable in eager context.
      • Eager performance should be similar with this feature enabled.
        • A roughly 5us per-op overhead may be observed when running many small functions.
        • Note a known issue with GPU performance.
      • The behavior of tf.function itself is unaffected.
    • Note: This feature will be enabled by default in an upcoming version of TensorFlow.

  • tf.experimental.dtensor: Added DTensor, an extension to TensorFlow for large-scale modeling with minimal changes to user code. You are welcome to try it out, though be aware that the DTensor API is experimental and up-to backward-incompatible changes. DTensor and Keras integration is published under tf.keras.dtensor in this release (refer to the tf.keras entry). The tutoral and guide for DTensor will be published on https://www.tensorflow.org/. Please stay tuned.

Bug Fixes and Other Changes

  • tf.data:

    • Fixed bug in tf.data.experimental.parse_example_dataset when tf.io.RaggedFeatures would specify value_key but no partitions. Before the fix, setting value_key but no partitions would result in the feature key being replaced by the value key, e.g. {'value_key': <RaggedTensor>} instead of {'key': <RaggedTensor>}. Now the correct feature key will be used. This aligns the behavior of tf.data.experimental.parse_example_dataset to match the behavior of tf.io.parse_example.
    • Added a new field, filter_parallelization, to tf.data.experimental.OptimizationOptions. If it is set to True, tf.data will run Filter transformation with multiple threads. Its default value is False if not specified.

  • tf.keras:

    • Fixed bug in optimizers that prevented them from properly checkpointing slot variables when they are ShardedVariables (used for training with tf.distribute.experimental.ParameterServerStrategy).

  • tf.random:

    • Added tf.random.experimental.index_shuffle, for shuffling a sequence without materializing the sequence in memory.

  • tf.RaggedTensor:

    • Introduced tf.experimental.RowPartition, which encodes how one dimension in a RaggedTensor relates to another, into the public API.
    • Introduced tf.experimental.DynamicRaggedShape, which represents the shape of a RaggedTensor.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aaron Debattista, Abel Soares Siqueira, Abhishek Varma, Andrei Ivanov, andreii, Andrew Goodbody, apeltop, Arnab Dutta, Ashiq Imran, Banikumar Maiti (Intel Aipg), Ben Greiner, Benjamin Peterson, bhack, Christopher Bate, chunduriv, Copybara-Service, DEKHTIARJonathan, Deven Desai, Duncan Riach, Eric Kunze, Everton Constantino, Faruk D, Fredrik Knutsson, gadagashwini, Gauri1 Deshpande, gtiHibGele, Guozhong Zhuang, Islem-Esi, Ivanov Viktor, Jason Furmanek, Jason Zaman, Jim, Jinzhe Zeng, John Laxson, Jonas Eschle, Jonas Eschle 'Mayou36, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, KaurkerDevourer, Koan-Sin Tan, kushanam, Laramie Leavitt, Li-Wen Chang, lipracer, Louis Sugy, Lu Teng, Mahmoud Abuzaina, Malcolm Slaney, Malik Shahzad Muzaffar, Marek Šuppa, Matt Conley, Michael Melesse, Milos Puzovic, mohantym, Nathan John Sircombe, Nathan Luehr, Nilesh Agarwalla, Patrice Vignola, peterjc123, Philip Turner, Rajeshwar Reddy T, Robert Kalmar, Rodrigo Formigone, Rohit Santhanam, rui, Sachin Muradi, Saduf2019, sandip, Scott Leishman, Serge Panev, Shi,Guangyong, Srinivasan Narayanamoorthy, stanley, Steven I Reeves, stevenireeves, sushreebarsa, Tamas Bela Feher, Tao He, Thomas Schmeyer, Tiago Almeida, Trevor Morris, Uday Bondhugula, Uwe L. Korn, Varghese, Jojimon, Vishnuvardhan Janapati, William Muir, William Raveane, xutianming, Yasuhiro Matsumoto, Yimei Sun, Yong Tang, Yu Feng, Yuriy Chernyshov, zhaozheng09

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.9.0-rc1

Release 2.9.0

Breaking Changes

  • Due to security issues in TF 2.8, all boosted trees code has now been removed (after being deprecated in TF 2.8). Users should switch to TensorFlow Decision Forests.
  • Build, Compilation and Packaging
    • TensorFlow is now compiled with _GLIBCXX_USE_CXX11_ABI=1. Downstream projects that encounter std::__cxx11 or [abi:cxx11] linker errors will need to adopt this compiler option. See the GNU C++ Library docs on Dual ABI.
    • TensorFlow Python wheels now specifically conform to manylinux2014, an upgrade from manylinux2010. The minimum Pip version supporting manylinux2014 is Pip 19.3 (see pypa/manylinux. This change may affect you if you have been using TensorFlow on a very old platform equivalent to CentOS 6, as manylinux2014 targets CentOS 7 as a compatibility base. Note that TensorFlow does not officially support either platform.
    • Discussion for these changes can be found on SIG Build's TensorFlow Community Forum thread
  • The tf.keras.mixed_precision.experimental API has been removed. The non-experimental symbols under tf.keras.mixed_precision have been available since TensorFlow 2.4 and should be used instead.
    • The non-experimental API has some minor differences from the experimental API. In most cases, you only need to make three minor changes:
      • Remove the word "experimental" from tf.keras.mixed_precision symbols. E.g., replace tf.keras.mixed_precision.experimental.global_policy with tf.keras.mixed_precision.global_policy.
      • Replace tf.keras.mixed_precision.experimental.set_policy with tf.keras.mixed_precision.set_global_policy. The experimental symbol set_policy was renamed to set_global_policy in the non-experimental API.
      • Replace LossScaleOptimizer(opt, "dynamic") with LossScaleOptimizer(opt). If you pass anything other than "dynamic" to the second argument, see (1) of the next section.
    • In the following rare cases, you need to make more changes when switching to the non-experimental API:
      • If you passed anything other than "dynamic" to the loss_scale argument (the second argument) of LossScaleOptimizer:
      • If you passed a value to the loss_scale argument (the second argument) of Policy:
        • The experimental version of Policy optionally took in a tf.compat.v1.mixed_precision.LossScale in the constructor, which defaulted to a dynamic loss scale for the "mixed_float16" policy and no loss scale for other policies. In Model.compile, if the model's policy had a loss scale, the optimizer would be wrapped with a LossScaleOptimizer. With the non-experimental Policy, there is no loss scale associated with the Policy, and Model.compile wraps the optimizer with a LossScaleOptimizer if and only if the policy is a "mixed_float16" policy. If you previously passed a LossScale to the experimental Policy, consider just removing it, as the default loss scaling behavior is usually what you want. If you really want to customize the loss scaling behavior, you can wrap your optimizer with a LossScaleOptimizer before passing it to Model.compile.
      • If you use the very rarely-used function tf.keras.mixed_precision.experimental.get_layer_policy:
        • Replace tf.keras.mixed_precision.experimental.get_layer_policy(layer) with layer.dtype_policy.
  • tf.mixed_precision.experimental.LossScale and its subclasses have been removed from the TF2 namespace. This symbols were very rarely used and were only useful in TF2 for use in the now-removed tf.keras.mixed_precision.experimental API. The symbols are still available under tf.compat.v1.mixed_precision.
  • The experimental_relax_shapes heuristic for tf.function has been deprecated and replaced with reduce_retracing which encompasses broader heuristics to reduce the number of retraces (see below)

Major Features and Improvements

  • tf.keras:

    • Added tf.keras.applications.resnet_rs models. This includes the ResNetRS50, ResNetRS101, ResNetRS152, ResNetRS200, ResNetRS270, ResNetRS350 and ResNetRS420 model architectures. The ResNetRS models are based on the architecture described in Revisiting ResNets: Improved Training and Scaling Strategies
    • Added tf.keras.optimizers.experimental.Optimizer. The reworked optimizer gives more control over different phases of optimizer calls, and is easier to customize. We provide Adam, SGD, Adadelta, AdaGrad and RMSprop optimizers based on tf.keras.optimizers.experimental.Optimizer. Generally the new optimizers work in the same way as the old ones, but support new constructor arguments. In the future, the symbols tf.keras.optimizers.Optimizer/Adam/etc will point to the new optimizers, and the previous generation of optimizers will be moved to tf.keras.optimizers.legacy.Optimizer/Adam/etc.
    • Added L2 unit normalization layer tf.keras.layers.UnitNormalization.
    • Added tf.keras.regularizers.OrthogonalRegularizer, a new regularizer that encourages orthogonality between the rows (or columns) or a weight matrix.
    • Added tf.keras.layers.RandomBrightness layer for image preprocessing.
    • Added APIs for switching between interactive logging and absl logging. By default, Keras always writes the logs to stdout. However, this is not optimal in a non-interactive environment, where you don't have access to stdout, but can only view the logs. You can use tf.keras.utils.disable_interactive_logging() to write the logs to ABSL logging. You can also use tf.keras.utils.enable_interactive_logging() to change it back to stdout, or tf.keras.utils.is_interactive_logging_enabled() to check if interactive logging is enabled.
    • Changed default value for the verbose argument of Model.evaluate() and Model.predict() to "auto", which defaults to verbose=1 for most cases and defaults to verbose=2 when used with ParameterServerStrategy or with interactive logging disabled.
    • Argument jit_compile in Model.compile() now applies to Model.evaluate() and Model.predict(). Setting jit_compile=True in compile() compiles the model's training, evaluation, and inference steps to XLA. Note that jit_compile=True may not necessarily work for all models.
    • Added DTensor-related Keras APIs under tf.keras.dtensor namespace. The APIs are still classified as experimental. You are welcome to try it out. Please check the tutoral and guide on https://www.tensorflow.org/ for more details about DTensor.

  • tf.lite:

    • Added TFLite builtin op support for the following TF ops:
      • tf.math.argmin/tf.math.argmax for input data type tf.bool on CPU.
      • tf.nn.gelu op for output data type tf.float32 and quantization on CPU.
    • Add nominal support for unsigned 16-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for unsigned 16-bit integer tensor types in cast op.
    • Experimental support for lowering list_ops.tensor_list_set_item with DynamicUpdateSlice.
    • Enabled a new MLIR-based dynamic range quantization backend by default
      • The new backend is used for post-training int8 dynamic range quantization and post-training float16 quantization.
      • Set experimental_new_dynamic_range_quantizer in tf.lite.TFLiteConverter to False to disable this change
    • Native TF Lite variables are now enabled during conversion by default on all v2 TfLiteConverter entry points. experimental_enable_resource_variables on tf.lite.TFLiteConverter is now True by default and will be removed in the future.

  • tf.function:

    • Custom classes used as arguments for tf.function can now specify rules regarding when retracing needs to occur by implementing the Tracing Protocol available through tf.types.experimental.SupportsTracingProtocol.
    • TypeSpec classes (as associated with ExtensionTypes) also implement the Tracing Protocol which can be overriden if necessary.
    • The newly introduced reduce_retracing option also uses the Tracing Protocol to proactively generate generalized traces similar to experimental_relax_shapes (which has now been deprecated).

  • Unified eager and tf.function execution:

    • Eager mode can now execute each op as a tf.function, allowing for more consistent feature support in future releases.
    • It is available for immediate use.
      • See the TF_RUN_EAGER_OP_AS_FUNCTION environment variable in eager context.
      • Eager performance should be similar with this feature enabled.
        • A roughly 5us per-op overhead may be observed when running many small functions.
        • Note a known issue with GPU performance.
      • The behavior of tf.function itself is unaffected.
    • Note: This feature will be enabled by default in an upcoming version of TensorFlow.

  • tf.experimental.dtensor: Added DTensor, an extension to TensorFlow for large-scale modeling with minimal changes to user code. You are welcome to try it out, though be aware that the DTensor API is experimental and up-to backward-incompatible changes. DTensor and Keras integration is published under tf.keras.dtensor in this release (refer to the tf.keras entry). The tutoral and guide for DTensor will be published on https://www.tensorflow.org/. Please stay tuned.

Bug Fixes and Other Changes

  • tf.data:

    • Fixed bug in tf.data.experimental.parse_example_dataset when tf.io.RaggedFeatures would specify value_key but no partitions. Before the fix, setting value_key but no partitions would result in the feature key being replaced by the value key, e.g. {'value_key': <RaggedTensor>} instead of {'key': <RaggedTensor>}. Now the correct feature key will be used. This aligns the behavior of tf.data.experimental.parse_example_dataset to match the behavior of tf.io.parse_example.
    • Added a new field, filter_parallelization, to tf.data.experimental.OptimizationOptions. If it is set to True, tf.data will run Filter transformation with multiple threads. Its default value is False if not specified.

  • tf.keras:

    • Fixed bug in optimizers that prevented them from properly checkpointing slot variables when they are ShardedVariables (used for training with tf.distribute.experimental.ParameterServerStrategy).

  • tf.random:

    • Added tf.random.experimental.index_shuffle, for shuffling a sequence without materializing the sequence in memory.

  • tf.RaggedTensor:

    • Introduced tf.experimental.RowPartition, which encodes how one dimension in a RaggedTensor relates to another, into the public API.
    • Introduced tf.experimental.DynamicRaggedShape, which represents the shape of a RaggedTensor.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aaron Debattista, Abel Soares Siqueira, Abhishek Varma, Andrei Ivanov, andreii, Andrew Goodbody, apeltop, Arnab Dutta, Ashiq Imran, Banikumar Maiti (Intel Aipg), Ben Greiner, Benjamin Peterson, bhack, Christopher Bate, chunduriv, Copybara-Service, DEKHTIARJonathan, Deven Desai, Duncan Riach, Eric Kunze, Everton Constantino, Faruk D, Fredrik Knutsson, gadagashwini, Gauri1 Deshpande, gtiHibGele, Guozhong Zhuang, Islem-Esi, Ivanov Viktor, Jason Furmanek, Jason Zaman, Jim, Jinzhe Zeng, John Laxson, Jonas Eschle, Jonas Eschle 'Mayou36, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, KaurkerDevourer, Koan-Sin Tan, kushanam, Laramie Leavitt, Li-Wen Chang, lipracer, Louis Sugy, Lu Teng, Mahmoud Abuzaina, Malcolm Slaney, Malik Shahzad Muzaffar, Marek Šuppa, Matt Conley, Michael Melesse, Milos Puzovic, mohantym, Nathan John Sircombe, Nathan Luehr, Nilesh Agarwalla, Patrice Vignola, peterjc123, Philip Turner, Rajeshwar Reddy T, Robert Kalmar, Rodrigo Formigone, Rohit Santhanam, rui, Sachin Muradi, Saduf2019, sandip, Scott Leishman, Serge Panev, Shi,Guangyong, Srinivasan Narayanamoorthy, stanley, Steven I Reeves, stevenireeves, sushreebarsa, Tamas Bela Feher, Tao He, Thomas Schmeyer, Tiago Almeida, Trevor Morris, Uday Bondhugula, Uwe L. Korn, Varghese, Jojimon, Vishnuvardhan Janapati, William Muir, William Raveane, xutianming, Yasuhiro Matsumoto, Yimei Sun, Yong Tang, Yu Feng, Yuriy Chernyshov, zhaozheng09

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.9.0-rc0

Release 2.9.0

Breaking Changes

  • Due to security issues in TF 2.8, all boosted trees code has now been removed (after being deprecated in TF 2.8). Users should switch to TensorFlow Decision Forests.
  • Build, Compilation and Packaging
    • TensorFlow is now compiled with _GLIBCXX_USE_CXX11_ABI=1. Downstream projects that encounter std::__cxx11 or [abi:cxx11] linker errors will need to adopt this compiler option. See the GNU C++ Library docs on Dual ABI.
    • TensorFlow Python wheels now specifically conform to manylinux2014, an upgrade from manylinux2010. The minimum Pip version supporting manylinux2014 is Pip 19.3 (see pypa/manylinux. This change may affect you if you have been using TensorFlow on a very old platform equivalent to CentOS 6, as manylinux2014 targets CentOS 7 as a compatibility base. Note that TensorFlow does not officially support either platform.
    • Discussion for these changes can be found on SIG Build's TensorFlow Community Forum thread
  • The tf.keras.mixed_precision.experimental API has been removed. The non-experimental symbols under tf.keras.mixed_precision have been available since TensorFlow 2.4 and should be used instead.
    • The non-experimental API has some minor differences from the experimental API. In most cases, you only need to make three minor changes:
      • Remove the word "experimental" from tf.keras.mixed_precision symbols. E.g., replace tf.keras.mixed_precision.experimental.global_policy with tf.keras.mixed_precision.global_policy.
      • Replace tf.keras.mixed_precision.experimental.set_policy with tf.keras.mixed_precision.set_global_policy. The experimental symbol set_policy was renamed to set_global_policy in the non-experimental API.
      • Replace LossScaleOptimizer(opt, "dynamic") with LossScaleOptimizer(opt). If you pass anything other than "dynamic" to the second argument, see (1) of the next section.
    • In the following rare cases, you need to make more changes when switching to the non-experimental API:
      • If you passed anything other than "dynamic" to the loss_scale argument (the second argument) of LossScaleOptimizer:
      • If you passed a value to the loss_scale argument (the second argument) of Policy:
        • The experimental version of Policy optionally took in a tf.compat.v1.mixed_precision.LossScale in the constructor, which defaulted to a dynamic loss scale for the "mixed_float16" policy and no loss scale for other policies. In Model.compile, if the model's policy had a loss scale, the optimizer would be wrapped with a LossScaleOptimizer. With the non-experimental Policy, there is no loss scale associated with the Policy, and Model.compile wraps the optimizer with a LossScaleOptimizer if and only if the policy is a "mixed_float16" policy. If you previously passed a LossScale to the experimental Policy, consider just removing it, as the default loss scaling behavior is usually what you want. If you really want to customize the loss scaling behavior, you can wrap your optimizer with a LossScaleOptimizer before passing it to Model.compile.
      • If you use the very rarely-used function tf.keras.mixed_precision.experimental.get_layer_policy:
        • Replace tf.keras.mixed_precision.experimental.get_layer_policy(layer) with layer.dtype_policy.
  • tf.mixed_precision.experimental.LossScale and its subclasses have been removed from the TF2 namespace. This symbols were very rarely used and were only useful in TF2 for use in the now-removed tf.keras.mixed_precision.experimental API. The symbols are still available under tf.compat.v1.mixed_precision.
  • The experimental_relax_shapes heuristic for tf.function has been deprecated and replaced with reduce_retracing which encompasses broader heuristics to reduce the number of retraces (see below)

Major Features and Improvements

  • tf.keras:

    • Added tf.keras.applications.resnet_rs models. This includes the ResNetRS50, ResNetRS101, ResNetRS152, ResNetRS200, ResNetRS270, ResNetRS350 and ResNetRS420 model architectures. The ResNetRS models are based on the architecture described in Revisiting ResNets: Improved Training and Scaling Strategies
    • Added tf.keras.optimizers.experimental.Optimizer. The reworked optimizer gives more control over different phases of optimizer calls, and is easier to customize. We provide Adam, SGD, Adadelta, AdaGrad and RMSprop optimizers based on tf.keras.optimizers.experimental.Optimizer. Generally the new optimizers work in the same way as the old ones, but support new constructor arguments. In the future, the symbols tf.keras.optimizers.Optimizer/Adam/etc will point to the new optimizers, and the previous generation of optimizers will be moved to tf.keras.optimizers.legacy.Optimizer/Adam/etc.
    • Added L2 unit normalization layer tf.keras.layers.UnitNormalization.
    • Added tf.keras.regularizers.OrthogonalRegularizer, a new regularizer that encourages orthogonality between the rows (or columns) or a weight matrix.
    • Added tf.keras.layers.RandomBrightness layer for image preprocessing.
    • Added APIs for switching between interactive logging and absl logging. By default, Keras always writes the logs to stdout. However, this is not optimal in a non-interactive environment, where you don't have access to stdout, but can only view the logs. You can use tf.keras.utils.disable_interactive_logging() to write the logs to ABSL logging. You can also use tf.keras.utils.enable_interactive_logging() to change it back to stdout, or tf.keras.utils.is_interactive_logging_enabled() to check if interactive logging is enabled.
    • Changed default value for the verbose argument of Model.evaluate() and Model.predict() to "auto", which defaults to verbose=1 for most cases and defaults to verbose=2 when used with ParameterServerStrategy or with interactive logging disabled.
    • Argument jit_compile in Model.compile() now applies to Model.evaluate() and Model.predict(). Setting jit_compile=True in compile() compiles the model's training, evaluation, and inference steps to XLA. Note that jit_compile=True may not necessarily work for all models.
    • Added DTensor-related Keras APIs under tf.keras.dtensor namespace. The APIs are still classified as experimental. You are welcome to try it out. Please check the tutoral and guide on https://www.tensorflow.org/ for more details about DTensor.

  • tf.lite:

    • Added TFLite builtin op support for the following TF ops:
      • tf.math.argmin/tf.math.argmax for input data type tf.bool on CPU.
      • tf.nn.gelu op for output data type tf.float32 and quantization on CPU.
    • Add nominal support for unsigned 16-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for unsigned 16-bit integer tensor types in cast op.
    • Experimental support for lowering list_ops.tensor_list_set_item with DynamicUpdateSlice.
    • Enabled a new MLIR-based dynamic range quantization backend by default
      • The new backend is used for post-training int8 dynamic range quantization and post-training float16 quantization.
      • Set experimental_new_dynamic_range_quantizer in tf.lite.TFLiteConverter to False to disable this change
    • Native TF Lite variables are now enabled during conversion by default on all v2 TfLiteConverter entry points. experimental_enable_resource_variables on tf.lite.TFLiteConverter is now True by default and will be removed in the future.

  • tf.function:

    • Custom classes used as arguments for tf.function can now specify rules regarding when retracing needs to occur by implementing the Tracing Protocol available through tf.types.experimental.SupportsTracingProtocol.
    • TypeSpec classes (as associated with ExtensionTypes) also implement the Tracing Protocol which can be overriden if necessary.
    • The newly introduced reduce_retracing option also uses the Tracing Protocol to proactively generate generalized traces similar to experimental_relax_shapes (which has now been deprecated).

  • Unified eager and tf.function execution:

    • Eager mode can now execute each op as a tf.function, allowing for more consistent feature support in future releases.
    • It is available for immediate use.
      • See the TF_RUN_EAGER_OP_AS_FUNCTION environment variable in eager context.
      • Eager performance should be similar with this feature enabled.
        • A roughly 5us per-op overhead may be observed when running many small functions.
        • Note a known issue with GPU performance.
      • The behavior of tf.function itself is unaffected.
    • Note: This feature will be enabled by default in an upcoming version of TensorFlow.

Bug Fixes and Other Changes

  • tf.data:

    • Fixed bug in tf.data.experimental.parse_example_dataset when tf.io.RaggedFeatures would specify value_key but no partitions. Before the fix, setting value_key but no partitions would result in the feature key being replaced by the value key, e.g. {'value_key': <RaggedTensor>} instead of {'key': <RaggedTensor>}. Now the correct feature key will be used. This aligns the behavior of tf.data.experimental.parse_example_dataset to match the behavior of tf.io.parse_example.
    • Added a new field, filter_parallelization, to tf.data.experimental.OptimizationOptions. If it is set to True, tf.data will run Filter transformation with multiple threads. Its default value is False if not specified.

  • tf.keras:

    • Fixed bug in optimizers that prevented them from properly checkpointing slot variables when they are ShardedVariables (used for training with tf.distribute.experimental.ParameterServerStrategy).

  • tf.random:

    • Added tf.random.experimental.index_shuffle, for shuffling a sequence without materializing the sequence in memory.

  • tf.RaggedTensor:

    • Introduced tf.experimental.RowPartition, which encodes how one dimension in a RaggedTensor relates to another, into the public API.
    • Introduced tf.experimental.DynamicRaggedShape, which represents the shape of a RaggedTensor.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aaron Debattista, Abel Soares Siqueira, Abhishek Varma, Andrei Ivanov, andreii, Andrew Goodbody, apeltop, Arnab Dutta, Ashiq Imran, Banikumar Maiti (Intel Aipg), Ben Greiner, Benjamin Peterson, bhack, Christopher Bate, chunduriv, Copybara-Service, DEKHTIARJonathan, Deven Desai, Duncan Riach, Eric Kunze, Everton Constantino, Faruk D, Fredrik Knutsson, gadagashwini, Gauri1 Deshpande, gtiHibGele, Guozhong Zhuang, Islem-Esi, Ivanov Viktor, Jason Furmanek, Jason Zaman, Jim, Jinzhe Zeng, John Laxson, Jonas Eschle, Jonas Eschle 'Mayou36, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, KaurkerDevourer, Koan-Sin Tan, kushanam, Laramie Leavitt, Li-Wen Chang, lipracer, Louis Sugy, Lu Teng, Mahmoud Abuzaina, Malcolm Slaney, Malik Shahzad Muzaffar, Marek Šuppa, Matt Conley, Michael Melesse, Milos Puzovic, mohantym, Nathan John Sircombe, Nathan Luehr, Nilesh Agarwalla, Patrice Vignola, peterjc123, Philip Turner, Rajeshwar Reddy T, Robert Kalmar, Rodrigo Formigone, Rohit Santhanam, rui, Sachin Muradi, Saduf2019, sandip, Scott Leishman, Serge Panev, Shi,Guangyong, Srinivasan Narayanamoorthy, stanley, Steven I Reeves, stevenireeves, sushreebarsa, Tamas Bela Feher, Tao He, Thomas Schmeyer, Tiago Almeida, Trevor Morris, Uday Bondhugula, Uwe L. Korn, Varghese, Jojimon, Vishnuvardhan Janapati, William Muir, William Raveane, xutianming, Yasuhiro Matsumoto, Yimei Sun, Yong Tang, Yu Feng, Yuriy Chernyshov, zhaozheng09

- C++
Published by tensorflow-jenkins almost 4 years ago

tensorflow - TensorFlow 2.8.0

Release 2.8.0

Major Features and Improvements

  • tf.lite:

    • Added TFLite builtin op support for the following TF ops:
      • tf.raw_ops.Bucketize op on CPU.
      • tf.where op for data types tf.int32/tf.uint32/tf.int8/tf.uint8/tf.int64.
      • tf.random.normal op for output data type tf.float32 on CPU.
      • tf.random.uniform op for output data type tf.float32 on CPU.
      • tf.random.categorical op for output data type tf.int64 on CPU.

  • tensorflow.experimental.tensorrt:

    • conversion_params is now deprecated inside TrtGraphConverterV2 in favor of direct arguments: max_workspace_size_bytes, precision_mode, minimum_segment_size, maximum_cached_engines, use_calibration and allow_build_at_runtime.
    • Added a new parameter called save_gpu_specific_engines to the .save() function inside TrtGraphConverterV2. When False, the .save() function won't save any TRT engines that have been built. When True (default), the original behavior is preserved.
    • TrtGraphConverterV2 provides a new API called .summary() which outputs a summary of the inference converted by TF-TRT. It namely shows each TRTEngineOp with their input(s)' and output(s)' shape and dtype. A detailed version of the summary is available which prints additionally all the TensorFlow OPs included in each of the TRTEngineOps.

  • tf.tpu.experimental.embedding:

    • tf.tpu.experimental.embedding.FeatureConfig now takes an additional argument output_shape which can specify the shape of the output activation for the feature.
    • tf.tpu.experimental.embedding.TPUEmbedding now has the same behavior as tf.tpu.experimental.embedding.serving_embedding_lookup which can take arbitrary rank of dense and sparse tensor. For ragged tensor, though the input tensor remains to be rank 2, the activations now can be rank 2 or above by specifying the output shape in the feature config or via the build method.

  • Add tf.config.experimental.enable_op_determinism, which makes TensorFlow ops run deterministically at the cost of performance. Replaces the TF_DETERMINISTIC_OPS environmental variable, which is now deprecated. The "Bug Fixes and Other Changes" section lists more determinism-related changes.

  • (Since TF 2.7) Add PluggableDevice support to TensorFlow Profiler.

Bug Fixes and Other Changes

  • tf.data:

    • The optimization parallel_batch now becomes default if not disabled by users, which will parallelize copying of batch elements.
    • Added the ability for TensorSliceDataset to identify and handle inputs that are files. This enables creating hermetic SavedModels when using datasets created from files.
    • The optimization parallel_batch now becomes default if not disabled by users, which will parallelize copying of batch elements.
    • Added the ability for TensorSliceDataset to identify and handle inputs that are files. This enables creating hermetic SavedModels when using datasets created from files.

  • tf.lite:

    • Adds GPU Delegation support for serialization to Java API. This boosts initialization time up to 90% when OpenCL is available.
    • Deprecated Interpreter::SetNumThreads, in favor of InterpreterBuilder::SetNumThreads.

  • tf.keras:

    • Adds tf.compat.v1.keras.utils.get_or_create_layer to aid migration to TF2 by enabling tracking of nested keras models created in TF1-style, when used with the tf.compat.v1.keras.utils.track_tf1_style_variables decorator.
    • Added a tf.keras.layers.experimental.preprocessing.HashedCrossing layer which applies the hashing trick to the concatenation of crossed scalar inputs. This provides a stateless way to try adding feature crosses of integer or string data to a model.
    • Removed keras.layers.experimental.preprocessing.CategoryCrossing. Users should migrate to the HashedCrossing layer or use tf.sparse.cross/tf.ragged.cross directly.
    • Added additional standardize and split modes to TextVectorization:
      • standardize="lower" will lowercase inputs.
      • standardize="string_punctuation" will remove all puncuation.
      • split="character" will split on every unicode character.
    • Added an output_mode argument to the Discretization and Hashing layers with the same semantics as other preprocessing layers. All categorical preprocessing layers now support output_mode.
    • All preprocessing layer output will follow the compute dtype of a tf.keras.mixed_precision.Policy, unless constructed with output_mode="int" in which case output will be tf.int64. The output type of any preprocessing layer can be controlled individually by passing a dtype argument to the layer.
    • tf.random.Generator for keras initializers and all RNG code.
    • Added 3 new APIs for enable/disable/check the usage of tf.random.Generator in keras backend, which will be the new backend for all the RNG in Keras. We plan to switch on the new code path by default in tf 2.8, and the behavior change will likely to cause some breakage on user side (eg if the test is checking against some golden nubmer). These 3 APIs will allow user to disable and switch back to legacy behavior if they prefer. In future (eg TF 2.10), we expect to totally remove the legacy code path (stateful random Ops), and these 3 APIs will be removed as well.
    • tf.keras.callbacks.experimental.BackupAndRestore is now available as tf.keras.callbacks.BackupAndRestore. The experimental endpoint is deprecated and will be removed in a future release.
    • tf.keras.experimental.SidecarEvaluator is now available as tf.keras.utils.SidecarEvaluator. The experimental endpoint is deprecated and will be removed in a future release.
    • Metrics update and collection logic in default Model.train_step() is now customizable via overriding Model.compute_metrics().
    • Losses computation logic in default Model.train_step() is now customizable via overriding Model.compute_loss().
    • jit_compile added to Model.compile() on an opt-in basis to compile the model's training step with XLA. Note that jit_compile=True may not necessarily work for all models.

  • Deterministic Op Functionality:

    • Fix regression in deterministic selection of deterministic cuDNN convolution algorithms, a regression that was introduced in v2.5. Note that nondeterministic out-of-memory events while selecting algorithms could still lead to nondeterminism, although this is very unlikely. This additional, unlikely source will be eliminated in a later version.
    • Add determinsitic GPU implementations of:
      • tf.function(jit_compile=True)'s that use Scatter.
      • (since v2.7) Stateful ops used in tf.data.Dataset
      • (since v2.7) tf.convert_to_tensor when fed with (sparse) tf.IndexedSlices (because it uses tf.math.unsorted_segment_sum)
      • (since v2.7) tf.gather backprop (because tf.convert_to_tensor reduces tf.gather's (sparse) tf.IndexedSlices gradients into its dense params input)
      • (since v2.7) tf.math.segment_mean
      • (since v2.7) tf.math.segment_prod
      • (since v2.7) tf.math.segment_sum
      • (since v2.7) tf.math.unsorted_segment_mean
      • (since v2.7) tf.math.unsorted_segment_prod
      • (since v2.7) tf.math.unsorted_segment_sum
      • (since v2.7) tf.math.unsorted_segment_sqrt
      • (since v2.7) tf.nn.ctc_loss (resolved, possibly in prior release, and confirmed with tests)
      • (since v2.7)tf.nn.sparse_softmax_crossentropy_with_logits
    • (since v2.7) Run tf.scatter_nd and other related scatter functions, such as tf.tensor_scatter_nd_update, on CPU (with significant performance penalty).
    • Add determinism-unimplemented exception-throwing to the following ops. When op-determinism is expected (i.e. after tf.config.experimental.enable_op_determinism has been called), an attempt to use the specified paths through the following ops on a GPU will cause tf.errors.UnimplementedError (with an understandable message), unless otherwise specified, to be thrown.
      • FakeQuantWithMinMaxVarsGradient and FakeQuantWithMinMaxVarsPerChannelGradient
      • (since v2.7) tf.compat.v1.get_seed if the global random seed has not yet been set (via tf.random.set_seed). Throws RuntimeError from Python or InvalidArgument from C++
      • (since v2.7) tf.compat.v1.nn.fused_batch_norm backprop to offset when is_training=False
      • (since v2.7) tf.image.adjust_contrast forward
      • (since v2.7) tf.image.resize with method=ResizeMethod.NEAREST backprop
      • (since v2.7) tf.linalg.svd
      • (since v2.7) tf.math.bincount
      • (since v2.7) tf.nn.depthwise_conv2d backprop to filter when not using cuDNN convolution
      • (since v2.7) tf.nn.dilation2d gradient
      • (since v2.7) tf.nn.max_pool_with_argmax gradient
      • (since v2.7) tf.raw_ops.DebugNumericSummary and tf.raw_ops.DebugNumericSummaryV2
      • (since v2.7) tf.timestamp. Throws FailedPrecondition
      • (since v2.7) tf.Variable.scatter_add (and other scatter methods, both on ref and resource variables)
      • (since v2.7) The random-number-generating ops in the tf.random module when the global random seed has not yet been set (via tf.random.set_seed). Throws RuntimeError from Python or InvalidArgument from C++

  • TensorFlow-oneDNN no longer supports explicit use of oneDNN blocked tensor format, e.g., setting the environment variable TF_ENABLE_MKL_NATIVE_FORMAT will not have any effect.

  • TensorFlow has been validated on Windows Subsystem for Linux 2 (aka WSL 2) for both GPUs and CPUs.

  • Due to security issues (see section below), all boosted trees code has been deprecated. Users should switch to TensorFlow Decision Forests. TF's boosted trees code will be eliminated before the branch cut for TF 2.9 and will no longer be present since that release.

Security

  • Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)
  • Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)
  • Fixes a heap OOB access in Dequantize (CVE-2022-21726)
  • Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)
  • Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)
  • Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)
  • Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)
  • Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)
  • Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)
  • Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)
  • Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)
  • Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)
  • Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)
  • Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)
  • Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)
  • Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)
  • Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)
  • Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)
  • Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)
  • Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)
  • Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)
  • Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)
  • Fixes an integer overflow in TFLite array creation (CVE-2022-23558)
  • Fixes an integer overflow in TFLite (CVE-2022-23559)
  • Fixes a dangerous OOB write in TFLite (CVE-2022-23561)
  • Fixes a vulnerability leading to read and write outside of bounds in TFLite (CVE-2022-23560)
  • Fixes a set of vulnerabilities caused by using insecure temporary files (CVE-2022-23563)
  • Fixes an integer overflow in Range resulting in undefined behavior and OOM (CVE-2022-23562)
  • Fixes a vulnerability where missing validation causes tf.sparse.split to crash when axis is a tuple (CVE-2021-41206)
  • Fixes a CHECK-fail when decoding resource handles from proto (CVE-2022-23564)
  • Fixes a CHECK-fail with repeated AttrDef (CVE-2022-23565)
  • Fixes a heap OOB write in Grappler (CVE-2022-23566)
  • Fixes a CHECK-fail when decoding invalid tensors from proto (CVE-2022-23571)
  • Fixes a null-dereference when specializing tensor type (CVE-2022-23570)
  • Fixes a crash when type cannot be specialized (CVE-2022-23572)
  • Fixes a heap OOB read/write in SpecializeType (CVE-2022-23574)
  • Fixes an unitialized variable access in AssignOp (CVE-2022-23573)
  • Fixes an integer overflow in OpLevelCostEstimator::CalculateTensorSize (CVE-2022-23575)
  • Fixes an integer overflow in OpLevelCostEstimator::CalculateOutputSize (CVE-2022-23576)
  • Fixes a null dereference in GetInitOp (CVE-2022-23577)
  • Fixes a memory leak when a graph node is invalid (CVE-2022-23578)
  • Fixes an abort caused by allocating a vector that is too large (CVE-2022-23580)
  • Fixes multiple CHECK-failures during Grappler's IsSimplifiableReshape (CVE-2022-23581)
  • Fixes multiple CHECK-failures during Grappler's SafeToRemoveIdentity (CVE-2022-23579)
  • Fixes multiple CHECK-failures in TensorByteSize (CVE-2022-23582)
  • Fixes multiple CHECK-failures in binary ops due to type confusion (CVE-2022-23583)
  • Fixes a use after free in DecodePng kernel (CVE-2022-23584)
  • Fixes a memory leak in decoding PNG images (CVE-2022-23585)
  • Fixes multiple CHECK-fails in function.cc (CVE-2022-23586)
  • Fixes multiple CHECK-fails due to attempting to build a reference tensor (CVE-2022-23588)
  • Fixes an integer overflow in Grappler cost estimation of crop and resize operation (CVE-2022-23587)
  • Fixes a null pointer dereference in Grappler's IsConstant (CVE-2022-23589)
  • Fixes a CHECK failure in constant folding (CVE-2021-41197)
  • Fixes a stack overflow due to self-recursive function in GraphDef (CVE-2022-23591)
  • Fixes a heap OOB access in RunForwardTypeInference (CVE-2022-23592)
  • Fixes a crash due to erroneous StatusOr (CVE-2022-23590)
  • Fixes multiple crashes and heap OOB accesses in TFG dialect (MLIR) (CVE-2022-23594)
  • Fixes a segfault in simplifyBroadcast (MLIR) (CVE-2022-23593)
  • Fixes a null pointer dereference in BuildXlaCompilationCache (XLA) (CVE-2022-23595)
  • Updates icu to 69.1 to handle CVE-2020-10531

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Adam Lanicek, ag.ramesh, alesapin, Andrew Goodbody, annasuheyla, Ariel Elkin, Arnab Dutta, Ben Barsdell, bhack, cfRod, Chengji Yao, Christopher Bate, dan, Dan F-M, David Korczynski, DEKHTIARJonathan, dengzhiyuan, Deven Desai, Duncan Riach, Eli Osherovich, Ewout Ter Hoeven, ez2take, Faijul Amin, fo40225, Frederic Bastien, gadagashwini, Gauri1 Deshpande, Georgiy Manuilov, Guilherme De Lázari, Guozhong Zhuang, H1Gdev, homuler, Hongxu Jia, Jacky_Yin, jayfurmanek, jgehw, Jhalak Patel, Jinzhe Zeng, Johan Gunnarsson, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, Kevin Cheng, Koan-Sin Tan, Kruglov-Dmitry, Kun Lu, Lemo, Lequn Chen, long.chen, Louis Sugy, Mahmoud Abuzaina, Mao, Marius Brehler, Mark Harfouche, Martin Patz, Maxiwell S. Garcia, Meenakshi Venkataraman, Michael Melesse, Mrinal Tyagi, Måns Nilsson, Nathan John Sircombe, Nathan Luehr, Nilesh Agarwalla, Oktay Ozturk, Patrice Vignola, Pawel-Polyai, Rama Ketineni, Ramesh Sampath, Reza Rahimi, Rob Suderman, Robert Kalmar, Rohit Santhanam, Sachin Muradi, Saduf2019, Samuel Marks, Shi,Guangyong, Sidong-Wei, Srinivasan Narayanamoorthy, Srishti Srivastava, Steven I Reeves, stevenireeves, Supernovae, Tamas Bela Feher, Tao Xu, Thibaut Goetghebuer-Planchon, Thomas Schmeyer, tilakrayal, Valery Mironov, Victor Guo, Vignesh Kothapalli, Vishnuvardhan Janapati, wamuir, Wang,Quintin, William Muir, William Raveane, Yash Goel, Yimei Sun, Yong Tang, Yuduo Wu

- C++
Published by tensorflow-jenkins about 4 years ago

tensorflow - TensorFlow 2.7.1

Release 2.7.1

This releases introduces several vulnerability fixes:

  • Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)
  • Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)
  • Fixes a heap OOB access in Dequantize (CVE-2022-21726)
  • Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)
  • Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)
  • Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)
  • Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)
  • Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)
  • Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)
  • Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)
  • Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)
  • Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)
  • Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)
  • Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)
  • Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)
  • Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)
  • Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)
  • Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)
  • Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)
  • Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)
  • Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)
  • Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)
  • Fixes an integer overflow in TFLite array creation (CVE-2022-23558)
  • Fixes an integer overflow in TFLite (CVE-2022-23559)
  • Fixes a dangerous OOB write in TFLite (CVE-2022-23561)
  • Fixes a vulnerability leading to read and write outside of bounds in TFLite (CVE-2022-23560)
  • Fixes a set of vulnerabilities caused by using insecure temporary files (CVE-2022-23563)
  • Fixes an integer overflow in Range resulting in undefined behavior and OOM (CVE-2022-23562)
  • Fixes a vulnerability where missing validation causes tf.sparse.split to crash when axis is a tuple (CVE-2021-41206)
  • Fixes a CHECK-fail when decoding resource handles from proto (CVE-2022-23564)
  • Fixes a CHECK-fail with repeated AttrDef (CVE-2022-23565)
  • Fixes a heap OOB write in Grappler (CVE-2022-23566)
  • Fixes a CHECK-fail when decoding invalid tensors from proto (CVE-2022-23571)
  • Fixes a null-dereference when specializing tensor type (CVE-2022-23570)
  • Fixes a crash when type cannot be specialized (CVE-2022-23572)
  • Fixes a heap OOB read/write in SpecializeType (CVE-2022-23574)
  • Fixes an unitialized variable access in AssignOp (CVE-2022-23573)
  • Fixes an integer overflow in OpLevelCostEstimator::CalculateTensorSize (CVE-2022-23575)
  • Fixes an integer overflow in OpLevelCostEstimator::CalculateOutputSize (CVE-2022-23576)
  • Fixes a null dereference in GetInitOp (CVE-2022-23577)
  • Fixes a memory leak when a graph node is invalid (CVE-2022-23578)
  • Fixes an abort caused by allocating a vector that is too large (CVE-2022-23580)
  • Fixes multiple CHECK-failures during Grappler's IsSimplifiableReshape (CVE-2022-23581)
  • Fixes multiple CHECK-failures during Grappler's SafeToRemoveIdentity (CVE-2022-23579)
  • Fixes multiple CHECK-failures in TensorByteSize (CVE-2022-23582)
  • Fixes multiple CHECK-failures in binary ops due to type confusion (CVE-2022-23583)
  • Fixes a use after free in DecodePng kernel (CVE-2022-23584)
  • Fixes a memory leak in decoding PNG images (CVE-2022-23585)
  • Fixes multiple CHECK-fails in function.cc (CVE-2022-23586)
  • Fixes multiple CHECK-fails due to attempting to build a reference tensor (CVE-2022-23588)
  • Fixes an integer overflow in Grappler cost estimation of crop and resize operation (CVE-2022-23587)
  • Fixes a null pointer dereference in Grappler's IsConstant (CVE-2022-23589)
  • Fixes a CHECK failure in constant folding (CVE-2021-41197)
  • Fixes a stack overflow due to self-recursive function in GraphDef (CVE-2022-23591)
  • Fixes a crash due to erroneous StatusOr (CVE-2022-23590)
  • Fixes multiple crashes and heap OOB accesses in TFG dialect (MLIR) (CVE-2022-23594)
  • Fixes a null pointer dereference in BuildXlaCompilationCache (XLA) (CVE-2022-23595)
  • Updates icu to 69.1 to handle CVE-2020-10531

- C++
Published by tensorflow-jenkins about 4 years ago

tensorflow - TensorFlow 2.6.3

Release 2.6.3

This releases introduces several vulnerability fixes:

  • Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)
  • Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)
  • Fixes a heap OOB access in Dequantize (CVE-2022-21726)
  • Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)
  • Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)
  • Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)
  • Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)
  • Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)
  • Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)
  • Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)
  • Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)
  • Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)
  • Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)
  • Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)
  • Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)
  • Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)
  • Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)
  • Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)
  • Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)
  • Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)
  • Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)
  • Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)
  • Fixes an integer overflow in TFLite array creation (CVE-2022-23558)
  • Fixes an integer overflow in TFLite (CVE-2022-23559)
  • Fixes a dangerous OOB write in TFLite (CVE-2022-23561)
  • Fixes a vulnerability leading to read and write outside of bounds in TFLite (CVE-2022-23560)
  • Fixes a set of vulnerabilities caused by using insecure temporary files (CVE-2022-23563)
  • Fixes an integer overflow in Range resulting in undefined behavior and OOM (CVE-2022-23562)
  • Fixes a vulnerability where missing validation causes tf.sparse.split to crash when axis is a tuple (CVE-2021-41206)
  • Fixes a CHECK-fail when decoding resource handles from proto (CVE-2022-23564)
  • Fixes a CHECK-fail with repeated AttrDef (CVE-2022-23565)
  • Fixes a heap OOB write in Grappler (CVE-2022-23566)
  • Fixes a CHECK-fail when decoding invalid tensors from proto (CVE-2022-23571)
  • Fixes a null-dereference when specializing tensor type (CVE-2022-23570)
  • Fixes a crash when type cannot be specialized (CVE-2022-23572)
  • Fixes a heap OOB read/write in SpecializeType (CVE-2022-23574)
  • Fixes an unitialized variable access in AssignOp (CVE-2022-23573)
  • Fixes an integer overflow in OpLevelCostEstimator::CalculateTensorSize (CVE-2022-23575)
  • Fixes an integer overflow in OpLevelCostEstimator::CalculateOutputSize (CVE-2022-23576)
  • Fixes a null dereference in GetInitOp (CVE-2022-23577)
  • Fixes a memory leak when a graph node is invalid (CVE-2022-23578)
  • Fixes an abort caused by allocating a vector that is too large (CVE-2022-23580)
  • Fixes multiple CHECK-failures during Grappler's IsSimplifiableReshape (CVE-2022-23581)
  • Fixes multiple CHECK-failures during Grappler's SafeToRemoveIdentity (CVE-2022-23579)
  • Fixes multiple CHECK-failures in TensorByteSize (CVE-2022-23582)
  • Fixes multiple CHECK-failures in binary ops due to type confusion (CVE-2022-23583)
  • Fixes a use after free in DecodePng kernel (CVE-2022-23584)
  • Fixes a memory leak in decoding PNG images (CVE-2022-23585)
  • Fixes multiple CHECK-fails in function.cc (CVE-2022-23586)
  • Fixes multiple CHECK-fails due to attempting to build a reference tensor (CVE-2022-23588)
  • Fixes an integer overflow in Grappler cost estimation of crop and resize operation (CVE-2022-23587)
  • Fixes a null pointer dereference in Grappler's IsConstant (CVE-2022-23589)
  • Fixes a CHECK failure in constant folding (CVE-2021-41197)
  • Fixes a stack overflow due to self-recursive function in GraphDef (CVE-2022-23591)
  • Fixes a null pointer dereference in BuildXlaCompilationCache (XLA) (CVE-2022-23595)
  • Updates icu to 69.1 to handle CVE-2020-10531

- C++
Published by tensorflow-jenkins about 4 years ago

tensorflow - TensorFlow 2.5.3

Release 2.5.3

Note: This is the last release in the 2.5 series.

This releases introduces several vulnerability fixes:

  • Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)
  • Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)
  • Fixes a heap OOB access in Dequantize (CVE-2022-21726)
  • Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)
  • Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)
  • Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)
  • Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)
  • Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)
  • Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)
  • Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)
  • Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)
  • Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)
  • Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)
  • Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)
  • Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)
  • Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)
  • Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)
  • Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)
  • Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)
  • Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)
  • Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)
  • Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)
  • Fixes an integer overflow in TFLite array creation (CVE-2022-23558)
  • Fixes an integer overflow in TFLite (CVE-2022-23559)
  • Fixes a dangerous OOB write in TFLite (CVE-2022-23561)
  • Fixes a vulnerability leading to read and write outside of bounds in TFLite (CVE-2022-23560)
  • Fixes a set of vulnerabilities caused by using insecure temporary files (CVE-2022-23563)
  • Fixes an integer overflow in Range resulting in undefined behavior and OOM (CVE-2022-23562)
  • Fixes a vulnerability where missing validation causes tf.sparse.split to crash when axis is a tuple (CVE-2021-41206)
  • Fixes a CHECK-fail when decoding resource handles from proto (CVE-2022-23564)
  • Fixes a CHECK-fail with repeated AttrDef (CVE-2022-23565)
  • Fixes a heap OOB write in Grappler (CVE-2022-23566)
  • Fixes a CHECK-fail when decoding invalid tensors from proto (CVE-2022-23571)
  • Fixes an unitialized variable access in AssignOp (CVE-2022-23573)
  • Fixes an integer overflow in OpLevelCostEstimator::CalculateTensorSize (CVE-2022-23575)
  • Fixes an integer overflow in OpLevelCostEstimator::CalculateOutputSize (CVE-2022-23576)
  • Fixes a null dereference in GetInitOp (CVE-2022-23577)
  • Fixes a memory leak when a graph node is invalid (CVE-2022-23578)
  • Fixes an abort caused by allocating a vector that is too large (CVE-2022-23580)
  • Fixes multiple CHECK-failures during Grappler's IsSimplifiableReshape (CVE-2022-23581)
  • Fixes multiple CHECK-failures during Grappler's SafeToRemoveIdentity (CVE-2022-23579)
  • Fixes multiple CHECK-failures in TensorByteSize (CVE-2022-23582)
  • Fixes multiple CHECK-failures in binary ops due to type confusion (CVE-2022-23583)
  • Fixes a use after free in DecodePng kernel (CVE-2022-23584)
  • Fixes a memory leak in decoding PNG images (CVE-2022-23585)
  • Fixes multiple CHECK-fails in function.cc (CVE-2022-23586)
  • Fixes multiple CHECK-fails due to attempting to build a reference tensor (CVE-2022-23588)
  • Fixes an integer overflow in Grappler cost estimation of crop and resize operation (CVE-2022-23587)
  • Fixes a null pointer dereference in Grappler's IsConstant (CVE-2022-23589)
  • Fixes a CHECK failure in constant folding (CVE-2021-41197)
  • Fixes a stack overflow due to self-recursive function in GraphDef (CVE-2022-23591)
  • Updates icu to 69.1 to handle CVE-2020-10531

- C++
Published by tensorflow-jenkins about 4 years ago

tensorflow - TensorFlow 2.8.0-rc1

Release 2.8.0

Major Features and Improvements

  • tf.lite:

    • Added TFLite builtin op support for the following TF ops:
      • tf.raw_ops.Bucketize op on CPU.
      • tf.where op for data types tf.int32/tf.uint32/tf.int8/tf.uint8/tf.int64.
      • tf.random.normal op for output data type tf.float32 on CPU.
      • tf.random.uniform op for output data type tf.float32 on CPU.
      • tf.random.categorical op for output data type tf.int64 on CPU.

  • tensorflow.experimental.tensorrt:

    • conversion_params is now deprecated inside TrtGraphConverterV2 in favor of direct arguments: max_workspace_size_bytes, precision_mode, minimum_segment_size, maximum_cached_engines, use_calibration and allow_build_at_runtime.
    • Added a new parameter called save_gpu_specific_engines to the .save() function inside TrtGraphConverterV2. When False, the .save() function won't save any TRT engines that have been built. When True (default), the original behavior is preserved.
    • TrtGraphConverterV2 provides a new API called .summary() which outputs a summary of the inference converted by TF-TRT. It namely shows each TRTEngineOp with their input(s)' and output(s)' shape and dtype. A detailed version of the summary is available which prints additionally all the TensorFlow OPs included in each of the TRTEngineOps.

  • tf.tpu.experimental.embedding:

    • tf.tpu.experimental.embedding.FeatureConfig now takes an additional argument output_shape which can specify the shape of the output activation for the feature.
    • tf.tpu.experimental.embedding.TPUEmbedding now has the same behavior as tf.tpu.experimental.embedding.serving_embedding_lookup which can take arbitrary rank of dense and sparse tensor. For ragged tensor, though the input tensor remains to be rank 2, the activations now can be rank 2 or above by specifying the output shape in the feature config or via the build method.

  • Add tf.config.experimental.enable_op_determinism, which makes TensorFlow ops run deterministically at the cost of performance. Replaces the TF_DETERMINISTIC_OPS environmental variable, which is now deprecated. The "Bug Fixes and Other Changes" section lists more determinism-related changes.

  • (Since TF 2.7) Add PluggableDevice support to TensorFlow Profiler.

Bug Fixes and Other Changes

  • tf.data:

    • The optimization parallel_batch now becomes default if not disabled by users, which will parallelize copying of batch elements.
    • Added the ability for TensorSliceDataset to identify and handle inputs that are files. This enables creating hermetic SavedModels when using datasets created from files.
    • The optimization parallel_batch now becomes default if not disabled by users, which will parallelize copying of batch elements.
    • Added the ability for TensorSliceDataset to identify and handle inputs that are files. This enables creating hermetic SavedModels when using datasets created from files.

  • tf.lite:

    • Adds GPU Delegation support for serialization to Java API. This boosts initialization time up to 90% when OpenCL is available.
    • Deprecated Interpreter::SetNumThreads, in favor of InterpreterBuilder::SetNumThreads.

  • tf.keras:

    • Adds tf.compat.v1.keras.utils.get_or_create_layer to aid migration to TF2 by enabling tracking of nested keras models created in TF1-style, when used with the tf.compat.v1.keras.utils.track_tf1_style_variables decorator.
    • Added a tf.keras.layers.experimental.preprocessing.HashedCrossing layer which applies the hashing trick to the concatenation of crossed scalar inputs. This provides a stateless way to try adding feature crosses of integer or string data to a model.
    • Removed keras.layers.experimental.preprocessing.CategoryCrossing. Users should migrate to the HashedCrossing layer or use tf.sparse.cross/tf.ragged.cross directly.
    • Added additional standardize and split modes to TextVectorization:
      • standardize="lower" will lowercase inputs.
      • standardize="string_punctuation" will remove all puncuation.
      • split="character" will split on every unicode character.
    • Added an output_mode argument to the Discretization and Hashing layers with the same semantics as other preprocessing layers. All categorical preprocessing layers now support output_mode.
    • All preprocessing layer output will follow the compute dtype of a tf.keras.mixed_precision.Policy, unless constructed with output_mode="int" in which case output will be tf.int64. The output type of any preprocessing layer can be controlled individually by passing a dtype argument to the layer.
    • tf.random.Generator for keras initializers and all RNG code.
    • Added 3 new APIs for enable/disable/check the usage of tf.random.Generator in keras backend, which will be the new backend for all the RNG in Keras. We plan to switch on the new code path by default in tf 2.8, and the behavior change will likely to cause some breakage on user side (eg if the test is checking against some golden nubmer). These 3 APIs will allow user to disable and switch back to legacy behavior if they prefer. In future (eg TF 2.10), we expect to totally remove the legacy code path (stateful random Ops), and these 3 APIs will be removed as well.
    • tf.keras.callbacks.experimental.BackupAndRestore is now available as tf.keras.callbacks.BackupAndRestore. The experimental endpoint is deprecated and will be removed in a future release.
    • tf.keras.experimental.SidecarEvaluator is now available as tf.keras.utils.SidecarEvaluator. The experimental endpoint is deprecated and will be removed in a future release.
    • Metrics update and collection logic in default Model.train_step() is now customizable via overriding Model.compute_metrics().
    • Losses computation logic in default Model.train_step() is now customizable via overriding Model.compute_loss().
    • jit_compile added to Model.compile() on an opt-in basis to compile the model's training step with XLA. Note that jit_compile=True may not necessarily work for all models.

  • Deterministic Op Functionality:

    • Fix regression in deterministic selection of deterministic cuDNN convolution algorithms, a regression that was introduced in v2.5. Note that nondeterministic out-of-memory events while selecting algorithms could still lead to nondeterminism, although this is very unlikely. This additional, unlikely source will be eliminated in a later version.
    • Add determinsitic GPU implementations of:
      • tf.function(jit_compile=True)'s that use Scatter.
      • (since v2.7) Stateful ops used in tf.data.Dataset
      • (since v2.7) tf.convert_to_tensor when fed with (sparse) tf.IndexedSlices (because it uses tf.math.unsorted_segment_sum)
      • (since v2.7) tf.gather backprop (because tf.convert_to_tensor reduces tf.gather's (sparse) tf.IndexedSlices gradients into its dense params input)
      • (since v2.7) tf.math.segment_mean
      • (since v2.7) tf.math.segment_prod
      • (since v2.7) tf.math.segment_sum
      • (since v2.7) tf.math.unsorted_segment_mean
      • (since v2.7) tf.math.unsorted_segment_prod
      • (since v2.7) tf.math.unsorted_segment_sum
      • (since v2.7) tf.math.unsorted_segment_sqrt
      • (since v2.7) tf.nn.ctc_loss (resolved, possibly in prior release, and confirmed with tests)
      • (since v2.7)tf.nn.sparse_softmax_crossentropy_with_logits
    • (since v2.7) Run tf.scatter_nd and other related scatter functions, such as tf.tensor_scatter_nd_update, on CPU (with significant performance penalty).
    • Add determinism-unimplemented exception-throwing to the following ops. When op-determinism is expected (i.e. after tf.config.experimental.enable_op_determinism has been called), an attempt to use the specified paths through the following ops on a GPU will cause tf.errors.UnimplementedError (with an understandable message), unless otherwise specified, to be thrown.
      • FakeQuantWithMinMaxVarsGradient and FakeQuantWithMinMaxVarsPerChannelGradient
      • (since v2.7) tf.compat.v1.get_seed if the global random seed has not yet been set (via tf.random.set_seed). Throws RuntimeError from Python or InvalidArgument from C++
      • (since v2.7) tf.compat.v1.nn.fused_batch_norm backprop to offset when is_training=False
      • (since v2.7) tf.image.adjust_contrast forward
      • (since v2.7) tf.image.resize with method=ResizeMethod.NEAREST backprop
      • (since v2.7) tf.linalg.svd
      • (since v2.7) tf.math.bincount
      • (since v2.7) tf.nn.depthwise_conv2d backprop to filter when not using cuDNN convolution
      • (since v2.7) tf.nn.dilation2d gradient
      • (since v2.7) tf.nn.max_pool_with_argmax gradient
      • (since v2.7) tf.raw_ops.DebugNumericSummary and tf.raw_ops.DebugNumericSummaryV2
      • (since v2.7) tf.timestamp. Throws FailedPrecondition
      • (since v2.7) tf.Variable.scatter_add (and other scatter methods, both on ref and resource variables)
      • (since v2.7) The random-number-generating ops in the tf.random module when the global random seed has not yet been set (via tf.random.set_seed). Throws RuntimeError from Python or InvalidArgument from C++

  • TensorFlow-oneDNN no longer supports explicit use of oneDNN blocked tensor format, e.g., setting the environment variable TF_ENABLE_MKL_NATIVE_FORMAT will not have any effect.

  • TensorFlow has been validated on Windows Subsystem for Linux 2 (aka WSL 2) for both GPUs and CPUs.

  • Due to security issues (see section below), all boosted trees code has been deprecated. Users should switch to TensorFlow Decision Forests. TF's boosted trees code will be eliminated before the branch cut for TF 2.9 and will no longer be present since that release.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Adam Lanicek, ag.ramesh, alesapin, Andrew Goodbody, annasuheyla, Ariel Elkin, Arnab Dutta, Ben Barsdell, bhack, cfRod, Chengji Yao, Christopher Bate, dan, Dan F-M, David Korczynski, DEKHTIARJonathan, dengzhiyuan, Deven Desai, Duncan Riach, Eli Osherovich, Ewout Ter Hoeven, ez2take, Faijul Amin, fo40225, Frederic Bastien, gadagashwini, Gauri1 Deshpande, Georgiy Manuilov, Guilherme De Lázari, Guozhong Zhuang, H1Gdev, homuler, Hongxu Jia, Jacky_Yin, jayfurmanek, jgehw, Jhalak Patel, Jinzhe Zeng, Johan Gunnarsson, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, Kevin Cheng, Koan-Sin Tan, Kruglov-Dmitry, Kun Lu, Lemo, Lequn Chen, long.chen, Louis Sugy, Mahmoud Abuzaina, Mao, Marius Brehler, Mark Harfouche, Martin Patz, Maxiwell S. Garcia, Meenakshi Venkataraman, Michael Melesse, Mrinal Tyagi, Måns Nilsson, Nathan John Sircombe, Nathan Luehr, Nilesh Agarwalla, Oktay Ozturk, Patrice Vignola, Pawel-Polyai, Rama Ketineni, Ramesh Sampath, Reza Rahimi, Rob Suderman, Robert Kalmar, Rohit Santhanam, Sachin Muradi, Saduf2019, Samuel Marks, Shi,Guangyong, Sidong-Wei, Srinivasan Narayanamoorthy, Srishti Srivastava, Steven I Reeves, stevenireeves, Supernovae, Tamas Bela Feher, Tao Xu, Thibaut Goetghebuer-Planchon, Thomas Schmeyer, tilakrayal, Valery Mironov, Victor Guo, Vignesh Kothapalli, Vishnuvardhan Janapati, wamuir, Wang,Quintin, William Muir, William Raveane, Yash Goel, Yimei Sun, Yong Tang, Yuduo Wu

- C++
Published by tensorflow-jenkins about 4 years ago

tensorflow - TensorFlow 2.8.0-rc0

Release 2.8.0

Major Features and Improvements

  • tf.lite:
    • Added TFLite builtin op support for the following TF ops:
      • tf.raw_ops.Bucketize op on CPU.
      • tf.where op for data types tf.int32/tf.uint32/tf.int8/tf.uint8/tf.int64.
      • tf.random.normal op for output data type tf.float32 on CPU.
      • tf.random.uniform op for output data type tf.float32 on CPU.
      • tf.random.categorical op for output data type tf.int64 on CPU.
  • tensorflow.experimental.tensorrt:
    • conversion_params is now deprecated inside TrtGraphConverterV2 in favor of direct arguments: max_workspace_size_bytes, precision_mode, minimum_segment_size, maximum_cached_engines, use_calibration and allow_build_at_runtime.
    • Added a new parameter called save_gpu_specific_engines to the .save() function inside TrtGraphConverterV2. When False, the .save() function won't save any TRT engines that have been built. When True (default), the original behavior is preserved.
    • TrtGraphConverterV2 provides a new API called .summary() which outputs a summary of the inference converted by TF-TRT. It namely shows each TRTEngineOp with their input(s)' and output(s)' shape and dtype. A detailed version of the summary is available which prints additionally all the TensorFlow OPs included in each of the TRTEngineOPs.
  • tf.tpu.experimental.embedding:
    • tf.tpu.experimental.embedding.FeatureConfig now takes an additional argument output_shape which can specify the shape of the output activation for the feature.
    • tf.tpu.experimental.embedding.TPUEmbedding now has the same behavior as tf.tpu.experimental.embedding.serving_embedding_lookup which can take arbitrary rank of dense and sparse tensor. For ragged tensor, though the input tensor remains to be rank 2, the activations now can be rank 2 or above by specifying the output shape in the feature config or via the build method.
  • Add tf.config.experimental.enable_op_determinism, which makes TensorFlow ops run deterministically at the cost of performance. Replaces the TF_DETERMINISTIC_OPS environmental variable, which is now deprecated.
    • The "Bug Fixes and Other Changes" section lists more determinism-related changes.

Bug Fixes and Other Changes

  • tf.data:

    • The optimization parallel_batch now becomes default if not disabled by users, which will parallelize copying of batch elements.
    • Added the ability for TensorSliceDataset to identify and handle inputs that are files. This enables creating hermetic SavedModels when using datasets created from files.

  • tf.lite:

    • GPU
    • Adds GPU Delegation support for serialization to Java API. This boosts initialization time upto 90% when OpenCL is available.
    • Deprecated Interpreter::SetNumThreads, in favor of InterpreterBuilder::SetNumThreads.

  • Adds tf.compat.v1.keras.utils.get_or_create_layer to aid migration to TF2 by enabling tracking of nested keras models created in TF1-style, when used with the tf.compat.v1.keras.utils.track_tf1_style_variables decorator.

  • tf.keras:

    • Preprocessing Layers
    • Added a tf.keras.layers.experimental.preprocessing.HashedCrossing layer which applies the hashing trick to the concatenation of crossed scalar inputs. This provides a stateless way to try adding feature crosses of integer or string data to a model.
    • Removed keras.layers.experimental.preprocessing.CategoryCrossing. Users should migrate to the HashedCrossing layer or use tf.sparse.cross/tf.ragged.cross directly.
    • Added additional standardize and split modes to TextVectorization.
      • standardize="lower" will lowercase inputs.
      • standardize="string_punctuation" will remove all puncuation.
      • split="character" will split on every unicode character.
    • Added an output_mode argument to the Discretization and Hashing layers with the same semantics as other preprocessing layers. All categorical preprocessing layers now support output_mode.
    • All preprocessing layer output will follow the compute dtype of a tf.keras.mixed_precision.Policy, unless constructed with output_mode="int" in which case output will be tf.int64. The output type of any preprocessing layer can be controlled individually by passing a dtype argument to the layer.
    • tf.random.Generator for keras initializers and all RNG code.
    • Added 3 new APIs for enable/disable/check the usage of tf.random.Generator in keras backend, which will be the new backend for all the RNG in Keras. We plan to switch on the new code path by default in TF 2.8, and the behavior change will likely to cause some breakage on user side (eg. if the test is checking against some golden number). These 3 APIs will allow user to disable and switch back to legacy behavior if they prefer. In future (eg tf 2.10), we expect to totally remove the legacy code path (stateful random Ops), and these 3 APIs will be removed as well.
    • tf.keras.callbacks.experimental.BackupAndRestore is now available as tf.keras.callbacks.BackupAndRestore. The experimental endpoint is deprecated and will be removed in a future release.
    • tf.keras.experimental.SidecarEvaluator is now available as tf.keras.utils.SidecarEvaluator. The experimental endpoint is deprecated and will be removed in a future release.
    • Metrics update and collection logic in default Model.train_step() is now customizable via overriding Model.compute_metrics().
    • Losses computation logic in default Model.train_step() is now customizable via overriding Model.compute_loss().
    • jit_compile added to Model.compile() on an opt-in basis to compile the model's training step with XLA. Note that jit_compile=True may not necessarily work for all models.

  • Deterministic Op Functionality

    • Add determinsitic GPU implementations of:
    • tf.function(jit_compile=True)'s that use Scatter.
    • (since v2.7) Stateful ops used in tf.data.Dataset
    • (since v2.7) tf.convert_to_tensor when fed with (sparse) tf.IndexedSlices (because it uses tf.math.unsorted_segment_sum)
    • (since v2.7) tf.gather backprop (because tf.convert_to_tensor reduces tf.gather's (sparse) tf.IndexedSlices gradients into its dense params input)
    • (since v2.7) tf.math.segment_mean
    • (since v2.7) tf.math.segment_prod
    • (since v2.7) tf.math.segment_sum
    • (since v2.7) tf.math.unsorted_segment_mean
    • (since v2.7) tf.math.unsorted_segment_prod
    • (since v2.7) tf.math.unsorted_segment_sum
    • (since v2.7) tf.math.unsorted_segment_sqrt
    • (since v2.7) tf.nn.ctc_loss (resolved, possibly in prior release, and confirmed with tests)
    • (since v2.7)tf.nn.sparse_softmax_crossentropy_with_logits
    • (since v2.7) Run the following ops on CPU (with significant performance penalty):
    • tf.scatter_nd and other related scatter functions, such as tf.tensor_scatter_nd_update
    • Add determinism-unimplemented exception-throwing to the following ops. When op-determinism is expected (i.e. after tf.config.experimental.enable_op_determinism has been called), an attempt to use the specified paths through the following ops on a GPU will cause tf.errors.UnimplementedError (with an understandable message), unless otherwise specified, to be thrown.
    • FakeQuantWithMinMaxVarsGradient and FakeQuantWithMinMaxVarsPerChannelGradient
    • (since v2.7) tf.compat.v1.get_seed if the global random seed has not yet been set (via tf.random.set_seed). Throws RuntimeError from Python or InvalidArgument from C++
    • (since v2.7) tf.compat.v1.nn.fused_batch_norm backprop to offset when is_training=False
    • (since v2.7) tf.image.adjust_contrast forward
    • (since v2.7) tf.image.resize with method=ResizeMethod.NEAREST backprop
    • (since v2.7) tf.linalg.svd
    • (since v2.7) tf.math.bincount
    • (since v2.7) tf.nn.depthwise_conv2d backprop to filter when not using cuDNN convolution
    • (since v2.7) tf.nn.dilation2d gradient
    • (since v2.7) tf.nn.max_pool_with_argmax gradient
    • (since v2.7) tf.raw_ops.DebugNumericSummary and tf.raw_ops.DebugNumericSummaryV2
    • (since v2.7) tf.timestamp. Throws FailedPrecondition
    • (since v2.7) tf.Variable.scatter_add (and other scatter methods, both on ref and resource variables)
    • (since v2.7) The random-number-generating ops in the tf.random module when the global random seed has not yet been set (via tf.random.set_seed). Throws RuntimeError from Python or InvalidArgument from C++

  • Add tf.config.experimental.enable_op_determinism, which makes TensorFlow ops run deterministically at the cost of performance. This is equivalent to setting the previously-existing TF_DETERMINISTIC_OPS environmental variable to 1. The environmental variable is now deprecated, so the enable_op_determinism function should be used instead.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Adam Lanicek, ag.ramesh, alesapin, Andrew Goodbody, annasuheyla, Ariel Elkin, Arnab Dutta, Ben Barsdell, bhack, cfRod, Chengji Yao, Christopher Bate, dan, Dan F-M, David Korczynski, DEKHTIARJonathan, dengzhiyuan, Deven Desai, Duncan Riach, Eli Osherovich, Ewout Ter Hoeven, ez2take, Faijul Amin, fo40225, Frederic Bastien, gadagashwini, Gauri1 Deshpande, Georgiy Manuilov, Guilherme De Lázari, Guozhong Zhuang, H1Gdev, homuler, Hongxu Jia, Jacky_Yin, jayfurmanek, jgehw, Jhalak Patel, Jinzhe Zeng, Johan Gunnarsson, Jonathan Dekhtiar, Kaixi Hou, Kanvi Khanna, Kevin Cheng, Koan-Sin Tan, Kruglov-Dmitry, Kun Lu, Lemo, Lequn Chen, long.chen, Louis Sugy, Mahmoud Abuzaina, Mao, Marius Brehler, Mark Harfouche, Martin Patz, Maxiwell S. Garcia, Meenakshi Venkataraman, Michael Melesse, Mrinal Tyagi, Måns Nilsson, Nathan John Sircombe, Nathan Luehr, Nilesh Agarwalla, Oktay Ozturk, Patrice Vignola, Pawel-Polyai, Rama Ketineni, Ramesh Sampath, Reza Rahimi, Rob Suderman, Robert Kalmar, Rohit Santhanam, Sachin Muradi, Saduf2019, Samuel Marks, Shi,Guangyong, Sidong-Wei, Srinivasan Narayanamoorthy, Srishti Srivastava, Steven I Reeves, stevenireeves, Supernovae, Tamas Bela Feher, Tao Xu, Thibaut Goetghebuer-Planchon, Thomas Schmeyer, tilakrayal, Valery Mironov, Victor Guo, Vignesh Kothapalli, Vishnuvardhan Janapati, wamuir, Wang,Quintin, William Muir, William Raveane, Yash Goel, Yimei Sun, Yong Tang, Yuduo Wu

- C++
Published by tensorflow-jenkins about 4 years ago

tensorflow - TensorFlow 2.7.0

Release 2.7.0

Breaking Changes

  • tf.keras:

    • The methods Model.fit(), Model.predict(), and Model.evaluate() will no longer uprank input data of shape (batch_size,) to become (batch_size, 1). This enables Model subclasses to process scalar data in their train_step()/test_step()/predict_step() methods.
      Note that this change may break certain subclassed models. You can revert back to the previous behavior by adding upranking yourself in the train_step()/test_step()/predict_step() methods, e.g. if x.shape.rank == 1: x = tf.expand_dims(x, axis=-1). Functional models as well as Sequential models built with an explicit input shape are not affected.
    • The methods Model.to_yaml() and keras.models.model_from_yaml have been replaced to raise a RuntimeError as they can be abused to cause arbitrary code execution. It is recommended to use JSON serialization instead of YAML, or, a better alternative, serialize to H5.
    • LinearModel and WideDeepModel are moved to the tf.compat.v1.keras.models. namespace (tf.compat.v1.keras.models.LinearModel and tf.compat.v1.keras.models.WideDeepModel), and their experimental endpoints (tf.keras.experimental.models.LinearModel and tf.keras.experimental.models.WideDeepModel) are being deprecated.
    • RNG behavior change for all tf.keras.initializers classes. For any class constructed with a fixed seed, it will no longer generate same value when invoked multiple times. Instead, it will return different value, but a determinisitic sequence. This change will make the initialize behavior align between v1 and v2.

  • tf.lite:

    • Rename fields SignatureDef table in schema to maximize the parity with TF SavedModel's Signature concept.
    • Deprecate Makefile builds. Makefile users need to migrate their builds to CMake or Bazel. Please refer to the Build TensorFlow Lite with CMake and Build TensorFlow Lite for ARM boards for the migration.
    • Deprecate tflite::OpResolver::GetDelegates. The list returned by TfLite's BuiltinOpResolver::GetDelegates is now always empty. Instead, recommend using new method tflite::OpResolver::GetDelegateCreators in order to achieve lazy initialization on TfLite delegate instances.

  • TF Core:

    • tf.Graph.get_name_scope() now always returns a string, as documented. Previously, when called within name_scope("") or name_scope(None) contexts, it returned None; now it returns the empty string.
    • tensorflow/core/ir/ contains a new MLIR-based Graph dialect that is isomorphic to GraphDef and will be used to replace GraphDef-based (e.g., Grappler) optimizations.
    • Deprecated and removed attrs() function in shape inference. All attributes should be queried by name now (rather than range returned) to enable changing the underlying storage there.
    • The following Python symbols were accidentally added in earlier versions of TensorFlow and now are removed. Each symbol has a replacement that should be used instead, but note the replacement's argument names are different.
      • tf.quantize_and_dequantize_v4 (accidentally introduced in TensorFlow 2.4): Use tf.quantization.quantize_and_dequantize_v2 instead.
      • tf.batch_mat_mul_v3 (accidentally introduced in TensorFlow 2.6): Use tf.linalg.matmul instead.
      • tf.sparse_segment_sum_grad (accidentally introduced in TensorFlow 2.6): Use tf.raw_ops.SparseSegmentSumGrad instead. Directly calling this op is typically not necessary, as it is automatically used when computing the gradient of tf.sparse.segment_sum.
    • Renaming of tensorflow::int64 to int64t in numerous places (the former is an alias for the latter) which could result in needing to regenerate selective op registration headers else execution would fail with unregistered kernels error.

  • Modular File System Migration:

    • Support for S3 and HDFS file systems has been migrated to a modular file systems based approach and is now available in https://github.com/tensorflow/io. The tensorflow-io python package should be installed for S3 and HDFS support with tensorflow.

Major Features and Improvements

  • Improvements to the TensorFlow debugging experience:

    • Previously, TensorFlow error stack traces involved many internal frames, which could be challenging to read through, while not being actionable for end users. As of TF 2.7, TensorFlow filters internal frames in most errors that it raises, to keep stack traces short, readable, and focused on what's actionable for end users (their own code).

    This behavior can be disabled by calling tf.debugging.disable_traceback_filtering(), and can be re-enabled via tf.debugging.enable_traceback_filtering(). If you are debugging a TensorFlow-internal issue (e.g. to prepare a TensorFlow PR), make sure to disable traceback filtering. You can check whether this feature is currently enabled by calling tf.debugging.is_traceback_filtering_enabled().

    Note that this feature is only available with Python 3.7 or higher. * Improve the informativeness of error messages raised by Keras Layer.__call__(), by adding the full list of argument values passed to the layer in every exception.

  • Introduce the tf.compat.v1.keras.utils.track_tf1_style_variables decorator, which enables using large classes of tf1-style variablescope, `getvariable, andcompat.v1.layer`-based components from within TF2 models running with TF2 behavior enabled.

  • tf.data:

    • tf.data service now supports auto-sharding. Users specify the sharding policy with tf.data.experimental.service.ShardingPolicy enum. It can be one of OFF (equivalent to today's "parallel_epochs" mode), DYNAMIC (equivalent to today's "distributed_epoch" mode), or one of the static sharding policies: FILE, DATA, FILE_OR_DATA, or HINT (corresponding to values of tf.data.experimental.AutoShardPolicy).

      Static sharding (auto-sharding) requires the number of tf.data service workers be fixed. Users need to specify the worker addresses in tensorflow.data.experimental.DispatcherConfig.

    • tf.data.experimental.service.register_dataset now accepts optional compression argument.

  • Keras:

    • tf.keras.layers.Conv now includes a public convolution_op method. This method can be used to simplify the implementation of Conv subclasses. There are two primary ways to use this new method. The first is to use the method directly in your own call method: python class StandardizedConv2D(tf.keras.layers.Conv2D): def call(self, inputs): mean, var = tf.nn.moments(self.kernel, axes=[0, 1, 2], keepdims=True) return self.convolution_op(inputs, (self.kernel - mean) / tf.sqrt(var + 1e-10)) Alternatively, you can override convolution_op: python class StandardizedConv2D(tf.keras.Layer): def convolution_op(self, inputs, kernel): mean, var = tf.nn.moments(kernel, axes=[0, 1, 2], keepdims=True) # Author code uses std + 1e-5 return super().convolution_op(inputs, (kernel - mean) / tf.sqrt(var + 1e-10))
    • Added merge_state() method to tf.keras.metrics.Metric for use in distributed computations.
    • Added sparse and ragged options to tf.keras.layers.TextVectorization to allow for SparseTensor and RaggedTensor outputs from the layer.

  • distribute.experimental.rpc package:

    • distribute.experimental.rpc package introduces APIs to create a GRPC based server to register tf.function methods and a GRPC client to invoke remote registered methods. RPC APIs are intended for multi-client setups i.e. server and clients are started in separate binaries independently.

    • Example usage to create server: ```python server = tf.distribute.experimental.rpc.Server.create("grpc", "127.0.0.1:1234") @tf.function(inputsignature=[ tf.TensorSpec([], tf.int32), tf.TensorSpec([], dtypes.int32) ]) def _remotemultiply(a, b): return tf.math.multiply(a, b)

      server.register("multiply", remotemultiply) ```

    • Example usage to create client: python client = tf.distribute.experimental.rpc.Client.create("grpc", address) a = tf.constant(2, dtype=tf.int32) b = tf.constant(3, dtype=tf.int32) result = client.multiply(a, b)

  • tf.lite:

    • Add experimental API experimental_from_jax to support conversion from Jax models to TensorFlow Lite.
    • Support uint32 data type for cast op.
    • Add experimental quantization debugger tf.lite.QuantizationDebugger

  • Extension Types

    • Add experimental API to define new Python classes that can be handled by TensorFlow APIs. To create an extension type, simply define a Python class with tf.experimental.ExtensionType as its base, and use type annotations to specify the type for each field. E.g.: python class MaskedTensor(tf.experimental.ExtensionType): values: tf.Tensor mask: tf.Tensor The tf.ExtensionType base class works similarly to typing.NamedTuple and @dataclasses.dataclass from the standard Python library.
    • Extension types are supported by Keras, tf.data, TF-hub, SavedModel, tf.function, control flow ops, py_function, and distribution strategy.
    • Add "dispatch decorators" that can be used to override the default behavior of TensorFlow ops (such as tf.add or tf.concat) when they are applied to ExtensionType values.
    • The BatchableExtensionType API can be used to define extension types that support APIs that make use of batching, such as tf.data.Dataset and tf.map_fn.
    • For more information, see the Extension types guide.

Bug Fixes and Other Changes

  • TF Core:
    • Random number generation (RNG) system
      • Add argument alg to tf.random.stateless_* functions to explicitly select the RNG algorithm.
      • Add tf.nn.experimental.stateless_dropout, a stateless version of tf.nn.dropout.
      • tf.random.Generator now can be created inside the scope of tf.distribute.experimental.ParameterServerStrategy and tf.distribute.experimental.CentralStorageStrategy.
    • Add an experimental session config tf.experimental.disable_functional_ops_lowering which disables functional control flow op lowering optimization. This is useful when executing within a portable runtime where control flow op kernels may not be loaded due to selective registration.
    • Add a new experimental argument experimental_is_anonymous to tf.lookup.StaticHashTable.__init__ to create the table in anonymous mode. In this mode, the table resource can only be accessed via resource handles (not resource names) and will be deleted automatically when all resource handles pointing to it are gone.
  • tf.data:
    • Introduce the tf.data.experimental.at API which provides random access for input pipelines that consist of transformations that support random access. The initial set of transformations that support random access includes: tf.data.Dataset.from_tensor_slices,tf.data.Dataset.shuffle, tf.data.Dataset.batch, tf.data.Dataset.shard, tf.data.Dataset.map, and tf.data.Dataset.range.
    • Promote tf.data.Options.experimental_deterministic API to tf.data.Options.deterministic and deprecate the experimental endpoint.
    • Move autotuning options fromtf.data.Options.experimental_optimization.autotune* to a newly created tf.data.Options.autotune.* and remove support for tf.data.Options.experimental_optimization.autotune_buffers.
    • Add support for user-defined names of tf.data core Python API, which can be used to disambiguate tf.data events in TF Profiler Trace Viewer.
    • Promote tf.data.experimental.sample_from_datasets API to tf.data.Dataset.sample_from_datasets and deprecate the experimental endpoint.
    • Added TF_GPU_ALLOCATOR=cuda_malloc_async that use cudaMallocAsync from CUDA 11.2. This could become the default in the future.
  • TF SavedModel:
    • Custom gradients are now saved by default. See tf.saved_model.SaveOptions to disable this.
    • The savedmodelcli's --input_examples inputs are now restricted to python literals to avoid code injection.
  • XLA:
    • Add a new API that allows custom call functions to signal errors. The old API will be deprecated in a future release. See https://www.tensorflow.org/xla/custom_call for details.
    • XLA:GPU reductions are deterministic by default (reductions within jit_compile=True are now deterministic).
    • XLA:GPU works with Horovod (OSS contribution by Trent Lo from NVidia)
  • tf.saved_model.save:
    • When saving a model, not specifying a namespace whitelist for custom ops with a namespace will now default to allowing rather than rejecting them all.
  • Deterministic Op Functionality (enabled by setting the environment variable TF_DETERMINISTIC_OPS to "true" or "1"):
    • Add determinsitic GPU implementations of:
      • tf.math.segment_sum
      • tf.math.segment_prod
      • tf.math.segment_mean
      • tf.math.unsorted_segment_sum
      • tf.math.unsorted_segment_prod
      • tf.math.unsorted_segment_sqrt
      • tf.math.unsorted_segment_mean
      • tf.gather backprop
      • tf.convert_to_tensor when fed with (sparse) tf.IndexedSlices
      • tf.nn.sparse_softmax_crossentropy_with_logits
      • tf.nn.ctc_loss (resolved, possibly in prior release, and confirmed with tests)
      • stateful ops used in tf.data.Dataset
    • Run the following ops on CPU (with significant performance penalty):
      • tf.scatter_nd and other related scatter functions, such as tf.tensor_scatter_nd_update
    • Add determinism-unimplemented exception-throwing to the following ops. When op-determinism is expected (i.e. when the environment variable TF_DETERMINISTIC_OPS is set to "true" or "1"), an attempt to use the specified paths through the following ops on a GPU will cause tf.errors.UnimplementedError (with an understandable message), unless otherwise specified, to be thrown.
      • tf.compat.v1.nn.fused_batch_norm backprop to offset when is_training=False
      • tf.image.adjust_contrast forward
      • tf.nn.depthwise_conv2d backprop to filter when not using cuDNN convolution
      • tf.image.resize with method=ResizeMethod.NEAREST backprop
      • tf.math.bincount - TODO: confirm exception added
      • tf.raw_ops.DebugNumericSummary and tf.raw_ops.DebugNumericSummaryV2
      • tf.Variable.scatter_add (and other scatter methods, both on ref and resource variables)
      • tf.linalg.svd
      • tf.nn.dilation2d gradient
      • tf.nn.max_pool_with_argmax gradient
      • tf.timestamp. Throws FailedPrecondition
      • The random-number-generating ops in the tf.random module when the global random seed has not yet been set (via tf.random.set_seed). Throws RuntimeError from Python or InvalidArgument from C++
      • tf.compat.v1.get_seed if the global random seed has not yet been set (via tf.random.set_seed). Throws RuntimeError from Python or InvalidArgument from C++

Security

  • Fixes a code injection issue in saved_model_cli (CVE-2021-41228)
  • Fixes a vulnerability due to use of uninitialized value in Tensorflow (CVE-2021-41225)
  • Fixes a heap OOB in FusedBatchNorm kernels (CVE-2021-41223)
  • Fixes an arbitrary memory read in ImmutableConst (CVE-2021-41227)
  • Fixes a heap OOB in SparseBinCount (CVE-2021-41226)
  • Fixes a heap OOB in SparseFillEmptyRows (CVE-2021-41224)
  • Fixes a segfault due to negative splits in SplitV (CVE-2021-41222)
  • Fixes segfaults and vulnerabilities caused by accesses to invalid memory during shape inference in Cudnn* ops (CVE-2021-41221)
  • Fixes a null pointer exception when Exit node is not preceded by Enter op (CVE-2021-41217)
  • Fixes an integer division by 0 in tf.raw_ops.AllToAll (CVE-2021-41218)
  • Fixes a use after free and a memory leak in CollectiveReduceV2 (CVE-2021-41220)
  • Fixes an undefined behavior via nullptr reference binding in sparse matrix multiplication (CVE-2021-41219)
  • Fixes a heap buffer overflow in Transpose (CVE-2021-41216)
  • Prevents deadlocks arising from mutually recursive tf.function objects (CVE-2021-41213)
  • Fixes a null pointer exception in DeserializeSparse (CVE-2021-41215)
  • Fixes an undefined behavior arising from reference binding to nullptr in tf.ragged.cross (CVE-2021-41214)
  • Fixes a heap OOB read in tf.ragged.cross (CVE-2021-41212)
  • Fixes a heap OOB in shape inference for QuantizeV2 (CVE-2021-41211)
  • Fixes a heap OOB read in all tf.raw_ops.QuantizeAndDequantizeV* ops (CVE-2021-41205)
  • Fixes an FPE in ParallelConcat (CVE-2021-41207)
  • Fixes FPE issues in convolutions with zero size filters (CVE-2021-41209)
  • Fixes a heap OOB read in tf.raw_ops.SparseCountSparseOutput (CVE-2021-41210)
  • Fixes vulnerabilities caused by incomplete validation in boosted trees code (CVE-2021-41208)
  • Fixes vulnerabilities caused by incomplete validation of shapes in multiple TF ops (CVE-2021-41206)
  • Fixes a segfault produced while copying constant resource tensor (CVE-2021-41204)
  • Fixes a vulnerability caused by unitialized access in EinsumHelper::ParseEquation (CVE-2021-41201)
  • Fixes several vulnerabilities and segfaults caused by missing validation during checkpoint loading (CVE-2021-41203)
  • Fixes an overflow producing a crash in tf.range (CVE-2021-41202)
  • Fixes an overflow producing a crash in tf.image.resize when size is large (CVE-2021-41199)
  • Fixes an overflow producing a crash in tf.tile when tiling tensor is large (CVE-2021-41198)
  • Fixes a vulnerability produced due to incomplete validation in tf.summary.create_file_writer (CVE-2021-41200)
  • Fixes multiple crashes due to overflow and CHECK-fail in ops with large tensor shapes (CVE-2021-41197)
  • Fixes a crash in max_pool3d when size argument is 0 or negative (CVE-2021-41196)
  • Fixes a crash in tf.math.segment_* operations (CVE-2021-41195)
  • Updates curl to 7.78.0 to handle CVE-2021-22922, CVE-2021-22923, CVE-2021-22924, CVE-2021-22925, and CVE-2021-22926.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Abhilash Majumder, abhilash1910, AdeshChoudhar, Adrian Garcia Badaracco, Adrian Ratiu, ag.ramesh, Aleksandr Nikolaev, Alexander Bosch, Alexander Grund, Annie Tallund, Anush Elangovan, Artem Sokolovskii, azazhu, Balint Cristian, Bas Aarts, Ben Barsdell, bhack, cfRod, Cheney-Wang, Cheng Ren, Christopher Bate, collin, Danila Bespalov, David Datascientist, Deven Desai, Duncan Riach, Ehsan Kia, Ellie, Fan Du, fo40225, Frederic Bastien, fsx950223, Gauri1 Deshpande, geetachavan1, Guillaume Klein, guozhong.zhuang, helen, Håkon Sandsmark, japm48, jgehw, Jinzhe Zeng, Jonathan Dekhtiar, Kai Zhu, Kaixi Hou, Kanvi Khanna, Koan-Sin Tan, Koki Ibukuro, Kulin Seth, KumaTea, Kun-Lu, Lemo, lipracer, liuyuanqiang, Mahmoud Abuzaina, Marius Brehler, Maxiwell S. Garcia, mdfaijul, metarutaiga, Michal Szutenberg, nammbash, Neil Girdhar, Nishidha Panpaliya, Nyadla-Sys, Patrice Vignola, Peter Kasting, Philipp Hack, PINTO0309, Prateek Gupta, puneeshkhanna, Rahul Butani, Rajeshwar Reddy T, Reza Rahimi, RinozaJiffry, rmothukuru, Rohit Santhanam, Saduf2019, Samuel Marks, sclarkson, Sergii Khomenko, Sheng, Yang, Sidong-Wei, slowy07, Srinivasan Narayanamoorthy, Srishti Srivastava, stanley, Stella Alice Schlotter, Steven I Reeves, stevenireeves, svobora, Takayoshi Koizumi, Tamas Bela Feher, Thibaut Goetghebuer-Planchon, Trent Lo, Twice, Varghese, Jojimon, Vishnuvardhan Janapati, Wang Yanzhang, Wang,Quintin, William Muir, William Raveane, Yasir Modak, Yasuhiro Matsumoto, Yi Li, Yong Tang, zhaozheng09, Zhoulong Jiang, zzpmiracle

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.6.2

Release 2.6.2

This release just fixes an issue where keras, tensorflow_estimator and tensorboard were missing proper upper bounds and resulted in broken installs after Keras 2.7 release for all packages in TensorFlow ecosystem

- C++
Published by mihaimaruseac over 4 years ago

tensorflow - TensorFlow 2.6.1

Release 2.6.1

This release introduces several vulnerability fixes:

  • Fixes a code injection issue in saved_model_cli (CVE-2021-41228)
  • Fixes a vulnerability due to use of uninitialized value in Tensorflow (CVE-2021-41225)
  • Fixes a heap OOB in FusedBatchNorm kernels (CVE-2021-41223)
  • Fixes an arbitrary memory read in ImmutableConst (CVE-2021-41227)
  • Fixes a heap OOB in SparseBinCount (CVE-2021-41226)
  • Fixes a heap OOB in SparseFillEmptyRows (CVE-2021-41224)
  • Fixes a segfault due to negative splits in SplitV (CVE-2021-41222)
  • Fixes segfaults and vulnerabilities caused by accesses to invalid memory during shape inference in Cudnn* ops (CVE-2021-41221)
  • Fixes a null pointer exception when Exit node is not preceded by Enter op (CVE-2021-41217)
  • Fixes an integer division by 0 in tf.raw_ops.AllToAll (CVE-2021-41218)
  • Fixes a use after free and a memory leak in CollectiveReduceV2 (CVE-2021-41220)
  • Fixes an undefined behavior via nullptr reference binding in sparse matrix multiplication (CVE-2021-41219)
  • Fixes a heap buffer overflow in Transpose (CVE-2021-41216)
  • Prevents deadlocks arising from mutually recursive tf.function objects (CVE-2021-41213)
  • Fixes a null pointer exception in DeserializeSparse (CVE-2021-41215)
  • Fixes an undefined behavior arising from reference binding to nullptr in tf.ragged.cross (CVE-2021-41214)
  • Fixes a heap OOB read in tf.ragged.cross (CVE-2021-41212)
  • Fixes a heap OOB in shape inference for QuantizeV2 (CVE-2021-41211)
  • Fixes a heap OOB read in all tf.raw_ops.QuantizeAndDequantizeV* ops (CVE-2021-41205)
  • Fixes an FPE in ParallelConcat (CVE-2021-41207)
  • Fixes FPE issues in convolutions with zero size filters (CVE-2021-41209)
  • Fixes a heap OOB read in tf.raw_ops.SparseCountSparseOutput (CVE-2021-41210)
  • Fixes vulnerabilities caused by incomplete validation in boosted trees code (CVE-2021-41208)
  • Fixes vulnerabilities caused by incomplete validation of shapes in multiple TF ops (CVE-2021-41206)
  • Fixes a segfault produced while copying constant resource tensor (CVE-2021-41204)
  • Fixes a vulnerability caused by unitialized access in EinsumHelper::ParseEquation (CVE-2021-41201)
  • Fixes several vulnerabilities and segfaults caused by missing validation during checkpoint loading (CVE-2021-41203)
  • Fixes an overflow producing a crash in tf.range (CVE-2021-41202)
  • Fixes an overflow producing a crash in tf.image.resize when size is large (CVE-2021-41199)
  • Fixes an overflow producing a crash in tf.tile when tiling tensor is large (CVE-2021-41198)
  • Fixes a vulnerability produced due to incomplete validation in tf.summary.create_file_writer (CVE-2021-41200)
  • Fixes multiple crashes due to overflow and CHECK-fail in ops with large tensor shapes (CVE-2021-41197)
  • Fixes a crash in max_pool3d when size argument is 0 or negative (CVE-2021-41196)
  • Fixes a crash in tf.math.segment_* operations (CVE-2021-41195)
  • Updates curl to 7.78.0 to handle CVE-2021-22922, CVE-2021-22923, CVE-2021-22924, CVE-2021-22925, and CVE-2021-22926.

- C++
Published by mihaimaruseac over 4 years ago

tensorflow - TensorFlow 2.5.2

Release 2.5.2

This release introduces several vulnerability fixes:

  • Fixes a code injection issue in saved_model_cli (CVE-2021-41228)
  • Fixes a vulnerability due to use of uninitialized value in Tensorflow (CVE-2021-41225)
  • Fixes a heap OOB in FusedBatchNorm kernels (CVE-2021-41223)
  • Fixes an arbitrary memory read in ImmutableConst (CVE-2021-41227)
  • Fixes a heap OOB in SparseBinCount (CVE-2021-41226)
  • Fixes a heap OOB in SparseFillEmptyRows (CVE-2021-41224)
  • Fixes a segfault due to negative splits in SplitV (CVE-2021-41222)
  • Fixes segfaults and vulnerabilities caused by accesses to invalid memory during shape inference in Cudnn* ops (CVE-2021-41221)
  • Fixes a null pointer exception when Exit node is not preceded by Enter op (CVE-2021-41217)
  • Fixes an integer division by 0 in tf.raw_ops.AllToAll (CVE-2021-41218)
  • Fixes an undefined behavior via nullptr reference binding in sparse matrix multiplication (CVE-2021-41219)
  • Fixes a heap buffer overflow in Transpose (CVE-2021-41216)
  • Prevents deadlocks arising from mutually recursive tf.function objects (CVE-2021-41213)
  • Fixes a null pointer exception in DeserializeSparse (CVE-2021-41215)
  • Fixes an undefined behavior arising from reference binding to nullptr in tf.ragged.cross (CVE-2021-41214)
  • Fixes a heap OOB read in tf.ragged.cross (CVE-2021-41212)
  • Fixes a heap OOB read in all tf.raw_ops.QuantizeAndDequantizeV* ops (CVE-2021-41205)
  • Fixes an FPE in ParallelConcat (CVE-2021-41207)
  • Fixes FPE issues in convolutions with zero size filters (CVE-2021-41209)
  • Fixes a heap OOB read in tf.raw_ops.SparseCountSparseOutput (CVE-2021-41210)
  • Fixes vulnerabilities caused by incomplete validation in boosted trees code (CVE-2021-41208)
  • Fixes vulnerabilities caused by incomplete validation of shapes in multiple TF ops (CVE-2021-41206)
  • Fixes a segfault produced while copying constant resource tensor (CVE-2021-41204)
  • Fixes a vulnerability caused by unitialized access in EinsumHelper::ParseEquation (CVE-2021-41201)
  • Fixes several vulnerabilities and segfaults caused by missing validation during checkpoint loading (CVE-2021-41203)
  • Fixes an overflow producing a crash in tf.range (CVE-2021-41202)
  • Fixes an overflow producing a crash in tf.image.resize when size is large (CVE-2021-41199)
  • Fixes an overflow producing a crash in tf.tile when tiling tensor is large (CVE-2021-41198)
  • Fixes a vulnerability produced due to incomplete validation in tf.summary.create_file_writer (CVE-2021-41200)
  • Fixes multiple crashes due to overflow and CHECK-fail in ops with large tensor shapes (CVE-2021-41197)
  • Fixes a crash in max_pool3d when size argument is 0 or negative (CVE-2021-41196)
  • Fixes a crash in tf.math.segment_* operations (CVE-2021-41195)
  • Updates curl to 7.78.0 to handle CVE-2021-22922, CVE-2021-22923, CVE-2021-22924, CVE-2021-22925, and CVE-2021-22926.

- C++
Published by mihaimaruseac over 4 years ago

tensorflow - TensorFlow 2.4.4

Release 2.4.4

NOTE: This is the last release in the 2.4.x line

This release introduces several vulnerability fixes:

  • Fixes a code injection issue in saved_model_cli (CVE-2021-41228)
  • Fixes a vulnerability due to use of uninitialized value in Tensorflow (CVE-2021-41225)
  • Fixes a heap OOB in FusedBatchNorm kernels (CVE-2021-41223)
  • Fixes an arbitrary memory read in ImmutableConst (CVE-2021-41227)
  • Fixes a heap OOB in SparseBinCount (CVE-2021-41226)
  • Fixes a heap OOB in SparseFillEmptyRows (CVE-2021-41224)
  • Fixes a segfault due to negative splits in SplitV (CVE-2021-41222)
  • Fixes segfaults and vulnerabilities caused by accesses to invalid memory during shape inference in Cudnn* ops (CVE-2021-41221)
  • Fixes a null pointer exception when Exit node is not preceded by Enter op (CVE-2021-41217)
  • Fixes an integer division by 0 in tf.raw_ops.AllToAll (CVE-2021-41218)
  • Fixes an undefined behavior via nullptr reference binding in sparse matrix multiplication (CVE-2021-41219)
  • Fixes a heap buffer overflow in Transpose (CVE-2021-41216)
  • Prevents deadlocks arising from mutually recursive tf.function objects (CVE-2021-41213)
  • Fixes a null pointer exception in DeserializeSparse (CVE-2021-41215)
  • Fixes an undefined behavior arising from reference binding to nullptr in tf.ragged.cross (CVE-2021-41214)
  • Fixes a heap OOB read in tf.ragged.cross (CVE-2021-41212)
  • Fixes a heap OOB read in all tf.raw_ops.QuantizeAndDequantizeV* ops (CVE-2021-41205)
  • Fixes an FPE in ParallelConcat (CVE-2021-41207)
  • Fixes FPE issues in convolutions with zero size filters (CVE-2021-41209)
  • Fixes a heap OOB read in tf.raw_ops.SparseCountSparseOutput (CVE-2021-41210)
  • Fixes vulnerabilities caused by incomplete validation in boosted trees code (CVE-2021-41208)
  • Fixes vulnerabilities caused by incomplete validation of shapes in multiple TF ops (CVE-2021-41206)
  • Fixes a segfault produced while copying constant resource tensor (CVE-2021-41204)
  • Fixes a vulnerability caused by unitialized access in EinsumHelper::ParseEquation (CVE-2021-41201)
  • Fixes several vulnerabilities and segfaults caused by missing validation during checkpoint loading (CVE-2021-41203)
  • Fixes an overflow producing a crash in tf.range (CVE-2021-41202)
  • Fixes an overflow producing a crash in tf.image.resize when size is large (CVE-2021-41199)
  • Fixes an overflow producing a crash in tf.tile when tiling tensor is large (CVE-2021-41198)
  • Fixes a vulnerability produced due to incomplete validation in tf.summary.create_file_writer (CVE-2021-41200)
  • Fixes multiple crashes due to overflow and CHECK-fail in ops with large tensor shapes (CVE-2021-41197)
  • Fixes a crash in max_pool3d when size argument is 0 or negative (CVE-2021-41196)
  • Fixes a crash in tf.math.segment_* operations (CVE-2021-41195)
  • Updates curl to 7.78.0 to handle CVE-2021-22922, CVE-2021-22923, CVE-2021-22924, CVE-2021-22925, and CVE-2021-22926.

- C++
Published by mihaimaruseac over 4 years ago

tensorflow - TensorFlow 2.7.0-rc1

Release 2.7.0

Breaking Changes

  • tf.keras:

    • The methods Model.fit(), Model.predict(), and Model.evaluate() will no longer uprank input data of shape (batch_size,) to become (batch_size, 1). This enables Model subclasses to process scalar data in their train_step()/test_step()/predict_step() methods.
      Note that this change may break certain subclassed models. You can revert back to the previous behavior by adding upranking yourself in the train_step()/test_step()/predict_step() methods, e.g. if x.shape.rank == 1: x = tf.expand_dims(x, axis=-1). Functional models as well as Sequential models built with an explicit input shape are not affected.
    • The methods Model.to_yaml() and keras.models.model_from_yaml have been replaced to raise a RuntimeError as they can be abused to cause arbitrary code execution. It is recommended to use JSON serialization instead of YAML, or, a better alternative, serialize to H5.
    • LinearModel and WideDeepModel are moved to the tf.compat.v1.keras.models. namespace (tf.compat.v1.keras.models.LinearModel and tf.compat.v1.keras.models.WideDeepModel), and their experimental endpoints (tf.keras.experimental.models.LinearModel and tf.keras.experimental.models.WideDeepModel) are being deprecated.
    • RNG behavior change for all tf.keras.initializers classes. For any class constructed with a fixed seed, it will no longer generate same value when invoked multiple times. Instead, it will return different value, but a determinisitic sequence. This change will make the initialize behavior align between v1 and v2.

  • tf.lite:

    • Rename fields SignatureDef table in schema to maximize the parity with TF SavedModel's Signature concept.
    • Deprecate Makefile builds. Makefile users need to migrate their builds to CMake or Bazel. Please refer to the Build TensorFlow Lite with CMake and Build TensorFlow Lite for ARM boards for the migration.
    • Deprecate tflite::OpResolver::GetDelegates. The list returned by TfLite's BuiltinOpResolver::GetDelegates is now always empty. Instead, recommend using new method tflite::OpResolver::GetDelegateCreators in order to achieve lazy initialization on TfLite delegate instances.

  • TF Core:

    • tf.Graph.get_name_scope() now always returns a string, as documented. Previously, when called within name_scope("") or name_scope(None) contexts, it returned None; now it returns the empty string.
    • tensorflow/core/ir/ contains a new MLIR-based Graph dialect that is isomorphic to GraphDef and will be used to replace GraphDef-based (e.g., Grappler) optimizations.
    • Deprecated and removed attrs() function in shape inference. All attributes should be queried by name now (rather than range returned) to enable changing the underlying storage there.
    • The following Python symbols were accidentally added in earlier versions of TensorFlow and now are removed. Each symbol has a replacement that should be used instead, but note the replacement's argument names are different.
      • tf.quantize_and_dequantize_v4 (accidentally introduced in TensorFlow 2.4): Use tf.quantization.quantize_and_dequantize_v2 instead.
      • tf.batch_mat_mul_v3 (accidentally introduced in TensorFlow 2.6): Use tf.linalg.matmul instead.
      • tf.sparse_segment_sum_grad (accidentally introduced in TensorFlow 2.6): Use tf.raw_ops.SparseSegmentSumGrad instead. Directly calling this op is typically not necessary, as it is automatically used when computing the gradient of tf.sparse.segment_sum.
    • Renaming of tensorflow::int64 to int64t in numerous places (the former is an alias for the latter) which could result in needing to regenerate selective op registration headers else execution would fail with unregistered kernels error.

  • Modular File System Migration:

    • Support for S3 and HDFS file systems has been migrated to a modular file systems based approach and is now available in https://github.com/tensorflow/io. The tensorflow-io python package should be installed for S3 and HDFS support with tensorflow.

Major Features and Improvements

  • Improvements to the TensorFlow debugging experience:

    • Previously, TensorFlow error stack traces involved many internal frames, which could be challenging to read through, while not being actionable for end users. As of TF 2.7, TensorFlow filters internal frames in most errors that it raises, to keep stack traces short, readable, and focused on what's actionable for end users (their own code).

    This behavior can be disabled by calling tf.debugging.disable_traceback_filtering(), and can be re-enabled via tf.debugging.enable_traceback_filtering(). If you are debugging a TensorFlow-internal issue (e.g. to prepare a TensorFlow PR), make sure to disable traceback filtering. You can check whether this feature is currently enabled by calling tf.debugging.is_traceback_filtering_enabled().

    Note that this feature is only available with Python 3.7 or higher. * Improve the informativeness of error messages raised by Keras Layer.__call__(), by adding the full list of argument values passed to the layer in every exception.

  • Introduce the tf.compat.v1.keras.utils.track_tf1_style_variables decorator, which enables using large classes of tf1-style variablescope, `getvariable, andcompat.v1.layer`-based components from within TF2 models running with TF2 behavior enabled.

  • tf.data:

    • tf.data service now supports auto-sharding. Users specify the sharding policy with tf.data.experimental.service.ShardingPolicy enum. It can be one of OFF (equivalent to today's "parallel_epochs" mode), DYNAMIC (equivalent to today's "distributed_epoch" mode), or one of the static sharding policies: FILE, DATA, FILE_OR_DATA, or HINT (corresponding to values of tf.data.experimental.AutoShardPolicy).

      Static sharding (auto-sharding) requires the number of tf.data service workers be fixed. Users need to specify the worker addresses in tensorflow.data.experimental.DispatcherConfig.

    • tf.data.experimental.service.register_dataset now accepts optional compression argument.

  • Keras:

    • tf.keras.layers.Conv now includes a public convolution_op method. This method can be used to simplify the implementation of Conv subclasses. There are two primary ways to use this new method. The first is to use the method directly in your own call method: python class StandardizedConv2D(tf.keras.layers.Conv2D): def call(self, inputs): mean, var = tf.nn.moments(self.kernel, axes=[0, 1, 2], keepdims=True) return self.convolution_op(inputs, (self.kernel - mean) / tf.sqrt(var + 1e-10)) Alternatively, you can override convolution_op: python class StandardizedConv2D(tf.keras.Layer): def convolution_op(self, inputs, kernel): mean, var = tf.nn.moments(kernel, axes=[0, 1, 2], keepdims=True) # Author code uses std + 1e-5 return super().convolution_op(inputs, (kernel - mean) / tf.sqrt(var + 1e-10))
    • Added merge_state() method to tf.keras.metrics.Metric for use in distributed computations.
    • Added sparse and ragged options to tf.keras.layers.TextVectorization to allow for SparseTensor and RaggedTensor outputs from the layer.

  • distribute.experimental.rpc package:

    • distribute.experimental.rpc package introduces APIs to create a GRPC based server to register tf.function methods and a GRPC client to invoke remote registered methods. RPC APIs are intended for multi-client setups i.e. server and clients are started in separate binaries independently.

    • Example usage to create server: ```python server = tf.distribute.experimental.rpc.Server.create("grpc", "127.0.0.1:1234") @tf.function(inputsignature=[ tf.TensorSpec([], tf.int32), tf.TensorSpec([], dtypes.int32) ]) def _remotemultiply(a, b): return tf.math.multiply(a, b)

      server.register("multiply", remotemultiply) ```

    • Example usage to create client: python client = tf.distribute.experimental.rpc.Client.create("grpc", address) a = tf.constant(2, dtype=tf.int32) b = tf.constant(3, dtype=tf.int32) result = client.multiply(a, b)

  • tf.lite:

    • Add experimental API experimental_from_jax to support conversion from Jax models to TensorFlow Lite.
    • Support uint32 data type for cast op.
    • Add experimental quantization debugger tf.lite.QuantizationDebugger

  • Extension Types

    • Add experimental API to define new Python classes that can be handled by TensorFlow APIs. To create an extension type, simply define a Python class with tf.experimental.ExtensionType as its base, and use type annotations to specify the type for each field. E.g.: python class MaskedTensor(tf.experimental.ExtensionType): values: tf.Tensor mask: tf.Tensor The tf.ExtensionType base class works similarly to typing.NamedTuple and @dataclasses.dataclass from the standard Python library.
    • Extension types are supported by Keras, tf.data, TF-hub, SavedModel, tf.function, control flow ops, py_function, and distribution strategy.
    • Add "dispatch decorators" that can be used to override the default behavior of TensorFlow ops (such as tf.add or tf.concat) when they are applied to ExtensionType values.
    • The BatchableExtensionType API can be used to define extension types that support APIs that make use of batching, such as tf.data.Dataset and tf.map_fn.

Bug Fixes and Other Changes

  • TF Core:
    • Random number generation (RNG) system
      • Add argument alg to tf.random.stateless_* functions to explicitly select the RNG algorithm.
      • Add tf.nn.experimental.stateless_dropout, a stateless version of tf.nn.dropout.
      • tf.random.Generator now can be created inside the scope of tf.distribute.experimental.ParameterServerStrategy and tf.distribute.experimental.CentralStorageStrategy.
    • Add an experimental session config tf.experimental.disable_functional_ops_lowering which disables functional control flow op lowering optimization. This is useful when executing within a portable runtime where control flow op kernels may not be loaded due to selective registration.
    • Add a new experimental argument experimental_is_anonymous to tf.lookup.StaticHashTable.__init__ to create the table in anonymous mode. In this mode, the table resource can only be accessed via resource handles (not resource names) and will be deleted automatically when all resource handles pointing to it are gone.
  • tf.data:
    • Introduce the tf.data.experimental.at API which provides random access for input pipelines that consist of transformations that support random access. The initial set of transformations that support random access includes: tf.data.Dataset.from_tensor_slices,tf.data.Dataset.shuffle, tf.data.Dataset.batch, tf.data.Dataset.shard, tf.data.Dataset.map, and tf.data.Dataset.range.
    • Promote tf.data.Options.experimental_deterministic API to tf.data.Options.deterministic and deprecate the experimental endpoint.
    • Move autotuning options fromtf.data.Options.experimental_optimization.autotune* to a newly created tf.data.Options.autotune.* and remove support for tf.data.Options.experimental_optimization.autotune_buffers.
    • Add support for user-defined names of tf.data core Python API, which can be used to disambiguate tf.data events in TF Profiler Trace Viewer.
    • Promote tf.data.experimental.sample_from_datasets API to tf.data.Dataset.sample_from_datasets and deprecate the experimental endpoint.
  • TF SavedModel:
    • Custom gradients are now saved by default. See tf.saved_model.SaveOptions to disable this.
  • XLA:
    • Add a new API that allows custom call functions to signal errors. The old API will be deprecated in a future release. See https://www.tensorflow.org/xla/custom_call for details.
    • XLA:GPU reductions are deterministic by default (reductions within jit_compile=True are now deterministic).
    • XLA:GPU works with Horovod (OSS contribution by Trent Lo from NVidia)
  • tf.saved_model.save:
    • When saving a model, not specifying a namespace whitelist for custom ops with a namespace will now default to allowing rather than rejecting them all.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Abhilash Majumder, abhilash1910, AdeshChoudhar, Adrian Garcia Badaracco, Adrian Ratiu, ag.ramesh, Aleksandr Nikolaev, Alexander Bosch, Alexander Grund, Annie Tallund, Anush Elangovan, Artem Sokolovskii, azazhu, Balint Cristian, Bas Aarts, Ben Barsdell, bhack, cfRod, Cheney-Wang, Cheng Ren, Christopher Bate, collin, Danila Bespalov, David Datascientist, Deven Desai, Ehsan Kia, Ellie, Fan Du, fo40225, Frederic Bastien, fsx950223, Gauri1 Deshpande, geetachavan1, Guillaume Klein, guozhong.zhuang, helen, Håkon Sandsmark, japm48, jgehw, Jinzhe Zeng, Jonathan Dekhtiar, Kai Zhu, Kaixi Hou, Kanvi Khanna, Koan-Sin Tan, Koki Ibukuro, Kulin Seth, KumaTea, Kun-Lu, Lemo, lipracer, liuyuanqiang, Mahmoud Abuzaina, Marius Brehler, Maxiwell S. Garcia, mdfaijul, metarutaiga, Michal Szutenberg, nammbash, Neil Girdhar, Nishidha Panpaliya, Nyadla-Sys, Patrice Vignola, Peter Kasting, Philipp Hack, PINTO0309, Prateek Gupta, puneeshkhanna, Rahul Butani, Rajeshwar Reddy T, Reza Rahimi, RinozaJiffry, rmothukuru, Rohit Santhanam, Saduf2019, Samuel Marks, sclarkson, Sergii Khomenko, Sheng, Yang, Sidong-Wei, slowy07, Srinivasan Narayanamoorthy, Srishti Srivastava, stanley, Stella Alice Schlotter, Steven I Reeves, stevenireeves, svobora, Takayoshi Koizumi, Tamas Bela Feher, Thibaut Goetghebuer-Planchon, Trent Lo, Twice, Varghese, Jojimon, Vishnuvardhan Janapati, Wang Yanzhang, Wang,Quintin, William Muir, William Raveane, Yasir Modak, Yasuhiro Matsumoto, Yi Li, Yong Tang, zhaozheng09, Zhoulong Jiang, zzpmiracle

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.7.0-rc0

Release 2.7.0

Breaking Changes

  • tf.keras:

    • The methods Model.fit(), Model.predict(), and Model.evaluate() will no longer uprank input data of shape (batch_size,) to become (batch_size, 1). This enables Model subclasses to process scalar data in their train_step()/test_step()/predict_step() methods.
      Note that this change may break certain subclassed models. You can revert back to the previous behavior by adding upranking yourself in the train_step()/test_step()/predict_step() methods, e.g. if x.shape.rank == 1: x = tf.expand_dims(x, axis=-1). Functional models as well as Sequential models built with an explicit input shape are not affected.
    • The methods Model.to_yaml() and keras.models.model_from_yaml have been replaced to raise a RuntimeError as they can be abused to cause arbitrary code execution. It is recommended to use JSON serialization instead of YAML, or, a better alternative, serialize to H5.
    • LinearModel and WideDeepModel are moved to the tf.compat.v1.keras.models. namespace (tf.compat.v1.keras.models.LinearModel and tf.compat.v1.keras.models.WideDeepModel), and their experimental endpoints (tf.keras.experimental.models.LinearModel and tf.keras.experimental.models.WideDeepModel) are being deprecated.
    • RNG behavior change for all tf.keras.initializers classes. For any class constructed with a fixed seed, it will no longer generate same value when invoked multiple times. Instead, it will return different value, but a determinisitic sequence. This change will make the initialize behavior align between v1 and v2.

  • tf.lite:

    • Rename fields SignatureDef table in schema to maximize the parity with TF SavedModel's Signature concept.
    • Deprecate Makefile builds. Makefile users need to migrate their builds to CMake or Bazel. Please refer to the Build TensorFlow Lite with CMake and Build TensorFlow Lite for ARM boards for the migration.
    • Deprecate tflite::OpResolver::GetDelegates. The list returned by TfLite's BuiltinOpResolver::GetDelegates is now always empty. Instead, recommend using new method tflite::OpResolver::GetDelegateCreators in order to achieve lazy initialization on TfLite delegate instances.

  • TF Core:

    • tf.Graph.get_name_scope() now always returns a string, as documented. Previously, when called within name_scope("") or name_scope(None) contexts, it returned None; now it returns the empty string.
    • tensorflow/core/ir/ contains a new MLIR-based Graph dialect that is isomorphic to GraphDef and will be used to replace GraphDef-based (e.g., Grappler) optimizations.
    • Deprecated and removed attrs() function in shape inference. All attributes should be queried by name now (rather than range returned) to enable changing the underlying storage there.
    • The following Python symbols were accidentally added in earlier versions of TensorFlow and now are removed. Each symbol has a replacement that should be used instead, but note the replacement's argument names are different.
      • tf.quantize_and_dequantize_v4 (accidentally introduced in TensorFlow 2.4): Use tf.quantization.quantize_and_dequantize_v2 instead.
      • tf.batch_mat_mul_v3 (accidentally introduced in TensorFlow 2.6): Use tf.linalg.matmul instead.
      • tf.sparse_segment_sum_grad (accidentally introduced in TensorFlow 2.6): Use tf.raw_ops.SparseSegmentSumGrad instead. Directly calling this op is typically not necessary, as it is automatically used when computing the gradient of tf.sparse.segment_sum.
    • Renaming of tensorflow::int64 to int64t in numerous places (the former is an alias for the latter) which could result in needing to regenerate selective op registration headers else execution would fail with unregistered kernels error.

Major Features and Improvements

  • Improvements to the TensorFlow debugging experience:

    • Previously, TensorFlow error stack traces involved many internal frames, which could be challenging to read through, while not being actionable for end users. As of TF 2.7, TensorFlow filters internal frames in most errors that it raises, to keep stack traces short, readable, and focused on what's actionable for end users (their own code).

    This behavior can be disabled by calling tf.debugging.disable_traceback_filtering(), and can be re-enabled via tf.debugging.enable_traceback_filtering(). If you are debugging a TensorFlow-internal issue (e.g. to prepare a TensorFlow PR), make sure to disable traceback filtering. You can check whether this feature is currently enabled by calling tf.debugging.is_traceback_filtering_enabled().

    Note that this feature is only available with Python 3.7 or higher. * Improve the informativeness of error messages raised by Keras Layer.__call__(), by adding the full list of argument values passed to the layer in every exception.

  • Introduce the tf.compat.v1.keras.utils.track_tf1_style_variables decorator, which enables using large classes of tf1-style variablescope, `getvariable, andcompat.v1.layer`-based components from within TF2 models running with TF2 behavior enabled.

  • tf.data:

    • tf.data service now supports auto-sharding. Users specify the sharding policy with tf.data.experimental.service.ShardingPolicy enum. It can be one of OFF (equivalent to today's "parallel_epochs" mode), DYNAMIC (equivalent to today's "distributed_epoch" mode), or one of the static sharding policies: FILE, DATA, FILE_OR_DATA, or HINT (corresponding to values of tf.data.experimental.AutoShardPolicy).

      Static sharding (auto-sharding) requires the number of tf.data service workers be fixed. Users need to specify the worker addresses in tensorflow.data.experimental.DispatcherConfig.

    • tf.data.experimental.service.register_dataset now accepts optional compression argument.

  • Keras:

    • tf.keras.layers.Conv now includes a public convolution_op method. This method can be used to simplify the implementation of Conv subclasses. There are two primary ways to use this new method. The first is to use the method directly in your own call method: python class StandardizedConv2D(tf.keras.layers.Conv2D): def call(self, inputs): mean, var = tf.nn.moments(self.kernel, axes=[0, 1, 2], keepdims=True) return self.convolution_op(inputs, (self.kernel - mean) / tf.sqrt(var + 1e-10)) Alternatively, you can override convolution_op: python class StandardizedConv2D(tf.keras.Layer): def convolution_op(self, inputs, kernel): mean, var = tf.nn.moments(kernel, axes=[0, 1, 2], keepdims=True) # Author code uses std + 1e-5 return super().convolution_op(inputs, (kernel - mean) / tf.sqrt(var + 1e-10))
    • Added merge_state() method to tf.keras.metrics.Metric for use in distributed computations.
    • Added sparse and ragged options to tf.keras.layers.TextVectorization to allow for SparseTensor and RaggedTensor outputs from the layer.

  • distribute.experimental.rpc package:

    • distribute.experimental.rpc package introduces APIs to create a GRPC based server to register tf.function methods and a GRPC client to invoke remote registered methods. RPC APIs are intended for multi-client setups i.e. server and clients are started in separate binaries independently.

    • Example usage to create server: ```python server = tf.distribute.experimental.rpc.Server.create("grpc", "127.0.0.1:1234") @tf.function(inputsignature=[ tf.TensorSpec([], tf.int32), tf.TensorSpec([], dtypes.int32) ]) def _remotemultiply(a, b): return tf.math.multiply(a, b)

      server.register("multiply", remotemultiply) ```

    • Example usage to create client: python client = tf.distribute.experimental.rpc.Client.create("grpc", address) a = tf.constant(2, dtype=tf.int32) b = tf.constant(3, dtype=tf.int32) result = client.multiply(a, b)

  • tf.lite:

    • Add experimental API experimental_from_jax to support conversion from Jax models to TensorFlow Lite.
    • Support uint32 data type for cast op.
    • Add experimental quantization debugger tf.lite.QuantizationDebugger

  • Extension Types

    • Add experimental API to define new Python classes that can be handled by TensorFlow APIs. To create an extension type, simply define a Python class with tf.experimental.ExtensionType as its base, and use type annotations to specify the type for each field. E.g.: python class MaskedTensor(tf.experimental.ExtensionType): values: tf.Tensor mask: tf.Tensor The tf.ExtensionType base class works similarly to typing.NamedTuple and @dataclasses.dataclass from the standard Python library.
    • Extension types are supported by Keras, tf.data, TF-hub, SavedModel, tf.function, control flow ops, py_function, and distribution strategy.
    • Add "dispatch decorators" that can be used to override the default behavior of TensorFlow ops (such as tf.add or tf.concat) when they are applied to ExtensionType values.
    • The BatchableExtensionType API can be used to define extension types that support APIs that make use of batching, such as tf.data.Dataset and tf.map_fn.

Bug Fixes and Other Changes

  • TF Core:
    • Random number generation (RNG) system
      • Add argument alg to tf.random.stateless_* functions to explicitly select the RNG algorithm.
      • Add tf.nn.experimental.stateless_dropout, a stateless version of tf.nn.dropout.
      • tf.random.Generator now can be created inside the scope of tf.distribute.experimental.ParameterServerStrategy and tf.distribute.experimental.CentralStorageStrategy.
    • Add an experimental session config tf.experimental.disable_functional_ops_lowering which disables functional control flow op lowering optimization. This is useful when executing within a portable runtime where control flow op kernels may not be loaded due to selective registration.
    • Add a new experimental argument experimental_is_anonymous to tf.lookup.StaticHashTable.__init__ to create the table in anonymous mode. In this mode, the table resource can only be accessed via resource handles (not resource names) and will be deleted automatically when all resource handles pointing to it are gone.
  • tf.data:
    • Introduce the tf.data.experimental.at API which provides random access for input pipelines that consist of transformations that support random access. The initial set of transformations that support random access includes: tf.data.Dataset.from_tensor_slices,tf.data.Dataset.shuffle, tf.data.Dataset.batch, tf.data.Dataset.shard, tf.data.Dataset.map, and tf.data.Dataset.range.
    • Promote tf.data.Options.experimental_deterministic API to tf.data.Options.deterministic and deprecate the experimental endpoint.
    • Move autotuning options fromtf.data.Options.experimental_optimization.autotune* to a newly created tf.data.Options.autotune.* and remove support for tf.data.Options.experimental_optimization.autotune_buffers.
    • Add support for user-defined names of tf.data core Python API, which can be used to disambiguate tf.data events in TF Profiler Trace Viewer.
    • Promote tf.data.experimental.sample_from_datasets API to tf.data.Dataset.sample_from_datasets and deprecate the experimental endpoint.
  • TF SavedModel:
    • Custom gradients are now saved by default. See tf.saved_model.SaveOptions to disable this.
  • XLA:
    • Add a new API that allows custom call functions to signal errors. The old API will be deprecated in a future release. See https://www.tensorflow.org/xla/custom_call for details.
    • XLA:GPU reductions are deterministic by default (reductions within jit_compile=True are now deterministic).
    • XLA:GPU works with Horovod (OSS contribution by Trent Lo from NVidia)
  • tf.saved_model.save:
    • When saving a model, not specifying a namespace whitelist for custom ops with a namespace will now default to allowing rather than rejecting them all.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Abhilash Majumder, abhilash1910, AdeshChoudhar, Adrian Garcia Badaracco, Adrian Ratiu, ag.ramesh, Aleksandr Nikolaev, Alexander Bosch, Alexander Grund, Annie Tallund, Anush Elangovan, Artem Sokolovskii, azazhu, Balint Cristian, Bas Aarts, Ben Barsdell, bhack, cfRod, Cheney-Wang, Cheng Ren, Christopher Bate, collin, Danila Bespalov, David Datascientist, Deven Desai, Ehsan Kia, Ellie, Fan Du, fo40225, Frederic Bastien, fsx950223, Gauri1 Deshpande, geetachavan1, Guillaume Klein, guozhong.zhuang, helen, Håkon Sandsmark, japm48, jgehw, Jinzhe Zeng, Jonathan Dekhtiar, Kai Zhu, Kaixi Hou, Kanvi Khanna, Koan-Sin Tan, Koki Ibukuro, Kulin Seth, KumaTea, Kun-Lu, Lemo, lipracer, liuyuanqiang, Mahmoud Abuzaina, Marius Brehler, Maxiwell S. Garcia, mdfaijul, metarutaiga, Michal Szutenberg, nammbash, Neil Girdhar, Nishidha Panpaliya, Nyadla-Sys, Patrice Vignola, Peter Kasting, Philipp Hack, PINTO0309, Prateek Gupta, puneeshkhanna, Rahul Butani, Rajeshwar Reddy T, Reza Rahimi, RinozaJiffry, rmothukuru, Rohit Santhanam, Saduf2019, Samuel Marks, sclarkson, Sergii Khomenko, Sheng, Yang, Sidong-Wei, slowy07, Srinivasan Narayanamoorthy, Srishti Srivastava, stanley, Stella Alice Schlotter, Steven I Reeves, stevenireeves, svobora, Takayoshi Koizumi, Tamas Bela Feher, Thibaut Goetghebuer-Planchon, Trent Lo, Twice, Varghese, Jojimon, Vishnuvardhan Janapati, Wang Yanzhang, Wang,Quintin, William Muir, William Raveane, Yasuhiro Matsumoto, Yi Li, Yong Tang, zhaozheng09, Zhoulong Jiang, zzpmiracle

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.4.3

Release 2.4.3

This release introduces several vulnerability fixes:

  • Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)
  • Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)
  • Fixes a null pointer dereference in CompressElement (CVE-2021-37637)
  • Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)
  • Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)
  • Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)
  • Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)
  • Fixes a heap OOB in RaggedGather (CVE-2021-37641)
  • Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)
  • Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)
  • Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)
  • Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)
  • Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)
  • Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)
  • Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)
  • Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)
  • Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)
  • Fixes a use after free in boosted trees creation (CVE-2021-37652)
  • Fixes a division by 0 in ResourceGather (CVE-2021-37653)
  • Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)
  • Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)
  • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse (CVE-2021-37656)
  • Fixes an undefined behavior arising from reference binding to nullptr in MatrixDiagV* ops (CVE-2021-37657)
  • Fixes an undefined behavior arising from reference binding to nullptr in MatrixSetDiagV* ops (CVE-2021-37658)
  • Fixes an undefined behavior arising from reference binding to nullptr and heap OOB in binary cwise ops (CVE-2021-37659)
  • Fixes a division by 0 in inplace operations (CVE-2021-37660)
  • Fixes a crash caused by integer conversion to unsigned (CVE-2021-37661)
  • Fixes an undefined behavior arising from reference binding to nullptr in boosted trees (CVE-2021-37662)
  • Fixes a heap OOB in boosted trees (CVE-2021-37664)
  • Fixes vulnerabilities arising from incomplete validation in QuantizeV2 (CVE-2021-37663)
  • Fixes vulnerabilities arising from incomplete validation in MKL requantization (CVE-2021-37665)
  • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToVariant (CVE-2021-37666)
  • Fixes an undefined behavior arising from reference binding to nullptr in unicode encoding (CVE-2021-37667)
  • Fixes an FPE in tf.raw_ops.UnravelIndex (CVE-2021-37668)
  • Fixes a crash in NMS ops caused by integer conversion to unsigned (CVE-2021-37669)
  • Fixes a heap OOB in UpperBound and LowerBound (CVE-2021-37670)
  • Fixes an undefined behavior arising from reference binding to nullptr in map operations (CVE-2021-37671)
  • Fixes a heap OOB in SdcaOptimizerV2 (CVE-2021-37672)
  • Fixes a CHECK-fail in MapStage (CVE-2021-37673)
  • Fixes a vulnerability arising from incomplete validation in MaxPoolGrad (CVE-2021-37674)
  • Fixes an undefined behavior arising from reference binding to nullptr in shape inference (CVE-2021-37676)
  • Fixes a division by 0 in most convolution operators (CVE-2021-37675)
  • Fixes vulnerabilities arising from missing validation in shape inference for Dequantize (CVE-2021-37677)
  • Fixes an arbitrary code execution due to YAML deserialization (CVE-2021-37678)
  • Fixes a heap OOB in nested tf.map_fn with RaggedTensors (CVE-2021-37679)
  • Fixes a division by zero in TFLite (CVE-2021-37680)
  • Fixes an NPE in TFLite (CVE-2021-37681)
  • Fixes a vulnerability arising from use of unitialized value in TFLite (CVE-2021-37682)
  • Fixes an FPE in TFLite division operations (CVE-2021-37683)
  • Fixes an FPE in TFLite pooling operations (CVE-2021-37684)
  • Fixes an infinite loop in TFLite (CVE-2021-37686)
  • Fixes a heap OOB in TFLite (CVE-2021-37685)
  • Fixes a heap OOB in TFLite's Gather* implementations (CVE-2021-37687)
  • Fixes an undefined behavior arising from null pointer dereference in TFLite (CVE-2021-37688)
  • Fixes an undefined behavior arising from null pointer dereference in TFLite MLIR optimizations (CVE-2021-37689)
  • Fixes a FPE in LSH in TFLite (CVE-2021-37691)
  • Fixes a segfault on strings tensors with mismatched dimensions, arising in Go code (CVE-2021-37692)
  • Fixes a use after free and a potential segfault in shape inference functions (CVE-2021-37690)
  • Updates curl to 7.77.0 to handle CVE-2021-22876, CVE-2021-22897, CVE-2021-22898, and CVE-2021-22901.

- C++
Published by mihaimaruseac over 4 years ago

tensorflow - TensorFlow 2.3.4

Release 2.3.4

NOTE: This is the last release in the 2.3.x line

This release introduces several vulnerability fixes:

  • Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)
  • Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)
  • Fixes a null pointer dereference in CompressElement (CVE-2021-37637)
  • Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)
  • Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)
  • Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)
  • Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)
  • Fixes a heap OOB in RaggedGather (CVE-2021-37641)
  • Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)
  • Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)
  • Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)
  • Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)
  • Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)
  • Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)
  • Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)
  • Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)
  • Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)
  • Fixes a use after free in boosted trees creation (CVE-2021-37652)
  • Fixes a division by 0 in ResourceGather (CVE-2021-37653)
  • Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)
  • Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)
  • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse (CVE-2021-37656)
  • Fixes an undefined behavior arising from reference binding to nullptr in MatrixDiagV* ops (CVE-2021-37657)
  • Fixes an undefined behavior arising from reference binding to nullptr in MatrixSetDiagV* ops (CVE-2021-37658)
  • Fixes an undefined behavior arising from reference binding to nullptr and heap OOB in binary cwise ops (CVE-2021-37659)
  • Fixes a division by 0 in inplace operations (CVE-2021-37660)
  • Fixes a crash caused by integer conversion to unsigned (CVE-2021-37661)
  • Fixes an undefined behavior arising from reference binding to nullptr in boosted trees (CVE-2021-37662)
  • Fixes a heap OOB in boosted trees (CVE-2021-37664)
  • Fixes vulnerabilities arising from incomplete validation in QuantizeV2 (CVE-2021-37663)
  • Fixes vulnerabilities arising from incomplete validation in MKL requantization (CVE-2021-37665)
  • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToVariant (CVE-2021-37666)
  • Fixes an undefined behavior arising from reference binding to nullptr in unicode encoding (CVE-2021-37667)
  • Fixes an FPE in tf.raw_ops.UnravelIndex (CVE-2021-37668)
  • Fixes a crash in NMS ops caused by integer conversion to unsigned (CVE-2021-37669)
  • Fixes a heap OOB in UpperBound and LowerBound (CVE-2021-37670)
  • Fixes an undefined behavior arising from reference binding to nullptr in map operations (CVE-2021-37671)
  • Fixes a heap OOB in SdcaOptimizerV2 (CVE-2021-37672)
  • Fixes a CHECK-fail in MapStage (CVE-2021-37673)
  • Fixes a vulnerability arising from incomplete validation in MaxPoolGrad (CVE-2021-37674)
  • Fixes an undefined behavior arising from reference binding to nullptr in shape inference (CVE-2021-37676)
  • Fixes a division by 0 in most convolution operators (CVE-2021-37675)
  • Fixes vulnerabilities arising from missing validation in shape inference for Dequantize (CVE-2021-37677)
  • Fixes an arbitrary code execution due to YAML deserialization (CVE-2021-37678)
  • Fixes a heap OOB in nested tf.map_fn with RaggedTensors (CVE-2021-37679)
  • Fixes a division by zero in TFLite (CVE-2021-37680)
  • Fixes an NPE in TFLite (CVE-2021-37681)
  • Fixes a vulnerability arising from use of unitialized value in TFLite (CVE-2021-37682)
  • Fixes an FPE in TFLite division operations (CVE-2021-37683)
  • Fixes an FPE in TFLite pooling operations (CVE-2021-37684)
  • Fixes an infinite loop in TFLite (CVE-2021-37686)
  • Fixes a heap OOB in TFLite (CVE-2021-37685)
  • Fixes a heap OOB in TFLite's Gather* implementations (CVE-2021-37687)
  • Fixes an undefined behavior arising from null pointer dereference in TFLite (CVE-2021-37688)
  • Fixes an undefined behavior arising from null pointer dereference in TFLite MLIR optimizations (CVE-2021-37689)
  • Fixes a FPE in LSH in TFLite (CVE-2021-37691)
  • Fixes a segfault on strings tensors with mismatched dimensions, arising in Go code (CVE-2021-37692)
  • Fixes a use after free and a potential segfault in shape inference functions (CVE-2021-37690)
  • Updates curl to 7.77.0 to handle CVE-2021-22876, CVE-2021-22897, CVE-2021-22898, and CVE-2021-22901.

- C++
Published by mihaimaruseac over 4 years ago

tensorflow - TensorFlow 2.6.0

Release 2.6.0

Breaking Changes

  • tf.train.experimental.enable_mixed_precision_graph_rewrite is removed, as the API only works in graph mode and is not customizable. The function is still accessible under tf.compat.v1.mixed_precision.enable_mixed_precision_graph_rewrite, but it is recommended to use the Keras mixed precision API instead.

  • tf.lite:

    • Remove experimental.nn.dynamic_rnn, experimental.nn.TfLiteRNNCell and experimental.nn.TfLiteLSTMCell since they're no longersupported. It's recommended to just use keras lstm instead.

  • tf.keras:

    • Keras been split into a separate PIP package (keras), and its code has been moved to the GitHub repositorykeras-team/keras. The API endpoints for tf.keras stay unchanged, but are now backed by the keras PIP package. The existing code in tensorflow/python/keras is a staled copy and will be removed in future release (2.7). Please remove any imports to tensorflow.python.keras and replace them with public tf.keras API instead.
    • The methods Model.to_yaml() and keras.models.model_from_yaml have been replaced to raise a RuntimeError as they can be abused to cause arbitrary code execution. It is recommended to use JSON serialization instead of YAML, or, a better alternative, serialize to H5.

Known Caveats

  • TF Core:
    • A longstanding bug in tf.while_loop, which caused it to execute sequentially, even when parallel_iterations>1, has now been fixed. However, the increased parallelism may result in increased memory use. Users who experience unwanted regressions should reset their while_loop's parallel_iterations value to 1, which is consistent with prior behavior.

Major Features and Improvements

  • tf.keras:

    • Keras has been split into a separate PIP package (keras), and its code has been moved to the GitHub repository keras-team/keras. The API endpoints for tf.keras stay unchanged, but are now backed by the keras PIP package. All Keras-related PRs and issues should now be directed to the GitHub repository keras-team/keras.
    • tf.keras.utils.experimental.DatasetCreator now takes an optional tf.distribute.InputOptions for specific options when used with distribution.
    • tf.keras.experimental.SidecarEvaluator is now available for a program intended to be run on an evaluator task, which is commonly used to supplement a training cluster running with tf.distribute.experimental.ParameterServerStrategy (see `https://www.tensorflow.org/tutorials/distribute/parameterservertraining). It can also be used with single-worker training or other strategies. See docstring for more info.
    • Preprocessing layers moved from experimental to core.
      • Import paths moved from tf.keras.layers.preprocessing.experimental to tf.keras.layers.
    • Updates to Preprocessing layers API for consistency and clarity:
      • StringLookup and IntegerLookup default for mask_token changed to None. This matches the default masking behavior of Hashing and Embedding layers. To keep existing behavior, pass mask_token="" during layer creation.
      • Renamed "binary" output mode to "multi_hot" for CategoryEncoding, StringLookup, IntegerLookup, and TextVectorization. Multi-hot encoding will no longer automatically uprank rank 1 inputs, so these layers can now multi-hot encode unbatched multi-dimensional samples.
      • Added a new output mode "one_hot" for CategoryEncoding, StringLookup, IntegerLookup, which will encode each element in an input batch individually, and automatically append a new output dimension if necessary. Use this mode on rank 1 inputs for the old "binary" behavior of one-hot encoding a batch of scalars.
      • Normalization will no longer automatically uprank rank 1 inputs, allowing normalization of unbatched multi-dimensional samples.

  • tf.lite:

    • The recommended Android NDK version for building TensorFlow Lite has been changed from r18b to r19c.
    • Supports int64 for mul.
    • Supports native variable builtin ops - ReadVariable, AssignVariable.
    • Converter:
      • Experimental support for variables in TFLite. To enable through conversion, users need to set experimental_enable_resource_variables on tf.lite.TFLiteConverter to True. Note: mutable variables is only available using from_saved_model in this release, support for other methods is coming soon.
      • Old Converter (TOCO) is getting removed from next release. It's been deprecated for few releases already.

  • tf.saved_model:

    • SavedModels can now save custom gradients. Use the option tf.saved_model.SaveOption(experimental_custom_gradients=True) to enable this feature. The documentation in Advanced autodiff has been updated.
    • Object metadata has now been deprecated and no longer saved to the SavedModel.

  • TF Core:

    • Added tf.config.experimental.reset_memory_stats to reset the tracked peak memory returned by tf.config.experimental.get_memory_info.

  • tf.data:

    • Added target_workers param to data_service_ops.from_dataset_id and data_service_ops.distribute. Users can specify "AUTO", "ANY", or "LOCAL" (case insensitive). If "AUTO", tf.data service runtime decides which workers to read from. If "ANY", TF workers read from any tf.data service workers. If "LOCAL", TF workers will only read from local in-processs tf.data service workers. "AUTO" works well for most cases, while users can specify other targets. For example, "LOCAL" would help avoid RPCs and data copy if every TF worker colocates with a tf.data service worker. Currently, "AUTO" reads from any tf.data service workers to preserve existing behavior. The default value is "AUTO".

Bug Fixes and Other Changes

  • TF Core:
    • Added tf.lookup.experimental.MutableHashTable, which provides a generic mutable hash table implementation.
      • Compared to tf.lookup.experimental.DenseHashTable this offers lower overall memory usage, and a cleaner API. It does not require specifying a delete_key and empty_key that cannot be inserted into the table.
    • Added support for specifying number of subdivisions in all reduce host collective. This parallelizes work on CPU and speeds up the collective performance. Default behavior is unchanged.
    • Add an option perturb_singular to tf.linalg.tridiagonal_solve that allows solving linear systems with a numerically singular tridiagonal matrix, e.g. for use in inverse iteration.
    • Added tf.linalg.eigh_tridiagonal that computes the eigenvalues of a Hermitian tridiagonal matrix.
    • tf.constant now places its output on the current default device.
    • SavedModel
      • Added tf.saved_model.experimental.TrackableResource, which allows the creation of custom wrapper objects for resource tensors.
      • Added a SavedModel load option to allow restoring partial checkpoints into the SavedModel. See tf.saved_model.LoadOptions for details.
    • Added a new op SparseSegmentSumGrad to match the other sparse segment gradient ops and avoid an extra gather operation that was in the previous gradient implementation.
    • Added a new session config setting internal_fragmentation_fraction, which controls when the BFC Allocator needs to split an oversized chunk to satisfy an allocation request.
    • Added tf.get_current_name_scope() which returns the current full name scope string that will be prepended to op names.
  • tf.data:
    • Promoting tf.data.experimental.bucket_by_sequence_length API to tf.data.Dataset.bucket_by_sequence_length and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.get_single_element API to tf.data.Dataset.get_single_element and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.group_by_window API to tf.data.Dataset.group_by_window and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.RandomDataset API to tf.data.Dataset.random and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.scan API to tf.data.Dataset.scan and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.snapshot API to tf.data.Dataset.shapshot and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.take_while API to tf.data.Dataset.take_while and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.ThreadingOptions API to tf.data.ThreadingOptions and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.unique API to tf.data.Dataset.unique and deprecating the experimental endpoint.
    • Added stop_on_empty_dataset parameter to sample_from_datasets and choose_from_datasets. Setting stop_on_empty_dataset=True will stop sampling if it encounters an empty dataset. This preserves the sampling ratio throughout training. The prior behavior was to continue sampling, skipping over exhausted datasets, until all datasets are exhausted. By default, the original behavior (stop_on_empty_dataset=False) is preserved.
    • Removed previously deprecated tf.data statistics related APIs:
      • tf.data.Options.experimental_stats
      • tf.data.experimental.StatsAggregator
      • tf.data.experimental.StatsOptions.*
      • tf.data.experimental.bytes_produced_stats
      • tf.data.experimental.latency_stats
    • Removed the following experimental tf.data optimization APIs:
      • tf.data.experimental.MapVectorizationOptions.*
      • tf.data.experimental.OptimizationOptions.filter_with_random_uniform_fusion
      • tf.data.experimental.OptimizationOptions.hoist_random_uniform
      • tf.data.experimental.OptimizationOptions.map_vectorization * tf.data.experimental.OptimizationOptions.reorder_data_discarding_ops
  • tf.keras:
    • Fix usage of __getitem__ slicing in Keras Functional APIs when the inputs are RaggedTensor objects.
    • Add keepdims argument to all GlobalPooling layers.
    • Add include_preprocessing argument to MobileNetV3 architectures to control the inclusion of Rescaling layer in the model.
    • Add optional argument (force) to make_(train|test|predict)_funtion methods to skip the cached function and generate a new one. This is useful to regenerate in a single call the compiled training function when any .trainable attribute of any model's layer has changed.
    • Models now have a save_spec property which contains the TensorSpec specs for calling the model. This spec is automatically saved when the model is called for the first time.
  • tf.linalg:
    • Add CompositeTensor as a base class to LinearOperator.
  • tf.lite:
    • Fix mean op reference quantization rounding issue.
    • Added framework_stable BUILD target, which links in only the non-experimental TF Lite APIs.
    • Remove deprecated Java Interpreter methods:
      • modifyGraphWithDelegate - Use Interpreter.Options.addDelegate
      • setNumThreads - Use Interpreter.Options.setNumThreads
    • Add Conv3DTranspose as a builtin op.
  • tf.summary:
    • Fix tf.summary.should_record_summaries() so it correctly reflects when summaries will be written, even when tf.summary.record_if() is not n effect, by returning True tensor if default writer is present.
  • Grappler:
    • Disable default Grappler optimization timeout to make the optimization pipeline deterministic. This may lead to increased model loading time, because time spent in graph optimizations is now unbounded (was 20 minutes).
  • Deterministic Op Functionality (enabled by setting TF_DETERMINISTIC_OPS to "true" or "1"):
    • Add a deterministic GPU implementation of tf.nn.softmax_cross_entropy_with_logits. See PR 49178.
    • Add a deterministic CPU implementation of tf.image.crop_and_resize. See PR 48905.
    • Add determinism-unimplemented exception-throwing to the following ops. When op-determinism is expected, an attempt to use the specified paths through the following ops on a GPU will cause tf.errors.UnimplementedError (with an understandable message) to be thrown.
      • tf.nn.sparse_softmax_cross_entropy_with_logits forwards and/or backwards. See PR 47925.
      • tf.image.crop_and_resize gradient w.r.t. either image or boxes. See PR 48905.
      • tf.sparse.sparse_dense_matmul forwards. See PR 50355.

Security

  • Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)
  • Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)
  • Fixes a null pointer dereference in CompressElement (CVE-2021-37637)
  • Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)
  • Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)
  • Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)
  • Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)
  • Fixes a heap OOB in RaggedGather (CVE-2021-37641)
  • Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)
  • Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)
  • Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)
  • Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)
  • Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)
  • Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)
  • Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)
  • Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)
  • Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)
  • Fixes a use after free in boosted trees creation (CVE-2021-37652)
  • Fixes a division by 0 in ResourceGather (CVE-2021-37653)
  • Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)
  • Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)
  • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse (CVE-2021-37656)
  • Fixes an undefined behavior arising from reference binding to nullptr in MatrixDiagV* ops (CVE-2021-37657)
  • Fixes an undefined behavior arising from reference binding to nullptr in MatrixSetDiagV* ops (CVE-2021-37658)
  • Fixes an undefined behavior arising from reference binding to nullptr and heap OOB in binary cwise ops (CVE-2021-37659)
  • Fixes a division by 0 in inplace operations (CVE-2021-37660)
  • Fixes a crash caused by integer conversion to unsigned (CVE-2021-37661)
  • Fixes an undefined behavior arising from reference binding to nullptr in boosted trees (CVE-2021-37662)
  • Fixes a heap OOB in boosted trees (CVE-2021-37664)
  • Fixes vulnerabilities arising from incomplete validation in QuantizeV2 (CVE-2021-37663)
  • Fixes vulnerabilities arising from incomplete validation in MKL requantization (CVE-2021-37665)
  • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToVariant (CVE-2021-37666)
  • Fixes an undefined behavior arising from reference binding to nullptr in unicode encoding (CVE-2021-37667)
  • Fixes an FPE in tf.raw_ops.UnravelIndex (CVE-2021-37668)
  • Fixes a crash in NMS ops caused by integer conversion to unsigned (CVE-2021-37669)
  • Fixes a heap OOB in UpperBound and LowerBound (CVE-2021-37670)
  • Fixes an undefined behavior arising from reference binding to nullptr in map operations (CVE-2021-37671)
  • Fixes a heap OOB in SdcaOptimizerV2 (CVE-2021-37672)
  • Fixes a CHECK-fail in MapStage (CVE-2021-37673)
  • Fixes a vulnerability arising from incomplete validation in MaxPoolGrad (CVE-2021-37674)
  • Fixes an undefined behavior arising from reference binding to nullptr in shape inference (CVE-2021-37676)
  • Fixes a division by 0 in most convolution operators (CVE-2021-37675)
  • Fixes vulnerabilities arising from missing validation in shape inference for Dequantize (CVE-2021-37677)
  • Fixes an arbitrary code execution due to YAML deserialization (CVE-2021-37678)
  • Fixes a heap OOB in nested tf.map_fn with RaggedTensors (CVE-2021-37679)
  • Fixes a division by zero in TFLite (CVE-2021-37680)
  • Fixes an NPE in TFLite (CVE-2021-37681)
  • Fixes a vulnerability arising from use of unitialized value in TFLite (CVE-2021-37682)
  • Fixes an FPE in TFLite division operations (CVE-2021-37683)
  • Fixes an FPE in TFLite pooling operations (CVE-2021-37684)
  • Fixes an infinite loop in TFLite (CVE-2021-37686)
  • Fixes a heap OOB in TFLite (CVE-2021-37685)
  • Fixes a heap OOB in TFLite's Gather* implementations (CVE-2021-37687)
  • Fixes an undefined behavior arising from null pointer dereference in TFLite (CVE-2021-37688)
  • Fixes an undefined behavior arising from null pointer dereference in TFLite MLIR optimizations (CVE-2021-37689)
  • Fixes a FPE in LSH in TFLite (CVE-2021-37691)
  • Fixes a segfault on strings tensors with mismatched dimensions, arising in Go code (CVE-2021-37692)
  • Fixes a use after free and a potential segfault in shape inference functions (CVE-2021-37690)
  • Updates curl to 7.77.0 to handle CVE-2021-22876, CVE-2021-22897, CVE-2021-22898, and CVE-2021-22901.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aadhitya A, Abhilash Mahendrakar, Abhishek Varma, Abin Shahab, Adam Hillier, Aditya Kane, AdityaKane2001, ag.ramesh, Amogh Joshi, Armen Poghosov, armkevincheng, Avrosh K, Ayan Moitra, azazhu, Banikumar Maiti, Bas Aarts, bhack, Bhanu Prakash Bandaru Venkata, Billy Cao, Bohumir Zamecnik, Bradley Reece, CyanXu, Daniel Situnayake, David Pal, Ddavis-2015, DEKHTIARJonathan, Deven Desai, Duncan Riach, Edward, Eli Osherovich, Eugene Kuznetsov, europeanplaice, evelynmitchell, Evgeniy Polyakov, Felix Vollmer, Florentin Hennecker, François Chollet, Frederic Bastien, Fredrik Knutsson, Gabriele Macchi, Gaurav Shukla, Gauri1 Deshpande, geetachavan1, Georgiy Manuilov, H, Hengwen Tong, Henri Woodcock, Hiran Sarkar, Ilya Arzhannikov, Janghoo Lee, jdematos, Jens Meder, Jerry Shih, jgehw, Jim Fisher, Jingbei Li, Jiri Podivin, Joachim Gehweiler, Johannes Lade, Jonas I. Liechti, Jonas Liechti, Jonas Ohlsson, Jonathan Dekhtiar, Julian Gross, Kaixi Hou, Kevin Cheng, Koan-Sin Tan, Kulin Seth, linzewen, Liubov Batanina, luisleee, Lukas Geiger, Mahmoud Abuzaina, mathgaming, Matt Conley, Max H. Gerlach, mdfaijul, Mh Kwon, Michael Martis, Michal Szutenberg, Måns Nilsson, nammbash, Neil Girdhar, Nicholas Vadivelu, Nick Kreeger, Nirjas Jakilim, okyanusoz, Patrice Vignola, Patrik Laurell, Pedro Marques, Philipp Hack, Phillip Cloud, Piergiacomo De Marchi, Prashant Kumar, puneeshkhanna, pvarouktsis, QQ喵, Rajeshwar Reddy T, Rama Ketineni, Reza Rahimi, Robert Kalmar, rsun, Ryan Kuester, Saduf2019, Sean Morgan, Sean Moriarity, Shaochen Shi, Sheng, Yang, Shu Wang, Shuai Zhang, Soojeong, Stanley-Nod, Steven I Reeves, stevenireeves, Suraj Sudhir, Sven Mayer, Tamas Bela Feher, tashuang.zk, tcervi, Teng Lu, Thales Elero Cervi, Thibaut Goetghebuer-Planchon, Thomas Walther, Till Brychcy, Trent Lo, Uday Bondhugula, vishakha.agrawal, Vishnuvardhan Janapati, wamuir, Wenwen Ouyang, wenwu, Williard Joshua Jose, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yasir Modak, Yi Li, Yong Tang, zilinzhu, 박상준, 이장

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.5.1

Release 2.5.1

This release introduces several vulnerability fixes:

  • Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)
  • Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)
  • Fixes a null pointer dereference in CompressElement (CVE-2021-37637)
  • Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)
  • Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)
  • Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)
  • Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)
  • Fixes a heap OOB in RaggedGather (CVE-2021-37641)
  • Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)
  • Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)
  • Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)
  • Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)
  • Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)
  • Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)
  • Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)
  • Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)
  • Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)
  • Fixes a use after free in boosted trees creation (CVE-2021-37652)
  • Fixes a division by 0 in ResourceGather (CVE-2021-37653)
  • Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)
  • Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)
  • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse (CVE-2021-37656)
  • Fixes an undefined behavior arising from reference binding to nullptr in MatrixDiagV* ops (CVE-2021-37657)
  • Fixes an undefined behavior arising from reference binding to nullptr in MatrixSetDiagV* ops (CVE-2021-37658)
  • Fixes an undefined behavior arising from reference binding to nullptr and heap OOB in binary cwise ops (CVE-2021-37659)
  • Fixes a division by 0 in inplace operations (CVE-2021-37660)
  • Fixes a crash caused by integer conversion to unsigned (CVE-2021-37661)
  • Fixes an undefined behavior arising from reference binding to nullptr in boosted trees (CVE-2021-37662)
  • Fixes a heap OOB in boosted trees (CVE-2021-37664)
  • Fixes vulnerabilities arising from incomplete validation in QuantizeV2 (CVE-2021-37663)
  • Fixes vulnerabilities arising from incomplete validation in MKL requantization (CVE-2021-37665)
  • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToVariant (CVE-2021-37666)
  • Fixes an undefined behavior arising from reference binding to nullptr in unicode encoding (CVE-2021-37667)
  • Fixes an FPE in tf.raw_ops.UnravelIndex (CVE-2021-37668)
  • Fixes a crash in NMS ops caused by integer conversion to unsigned (CVE-2021-37669)
  • Fixes a heap OOB in UpperBound and LowerBound (CVE-2021-37670)
  • Fixes an undefined behavior arising from reference binding to nullptr in map operations (CVE-2021-37671)
  • Fixes a heap OOB in SdcaOptimizerV2 (CVE-2021-37672)
  • Fixes a CHECK-fail in MapStage (CVE-2021-37673)
  • Fixes a vulnerability arising from incomplete validation in MaxPoolGrad (CVE-2021-37674)
  • Fixes an undefined behavior arising from reference binding to nullptr in shape inference (CVE-2021-37676)
  • Fixes a division by 0 in most convolution operators (CVE-2021-37675)
  • Fixes vulnerabilities arising from missing validation in shape inference for Dequantize (CVE-2021-37677)
  • Fixes an arbitrary code execution due to YAML deserialization (CVE-2021-37678)
  • Fixes a heap OOB in nested tf.map_fn with RaggedTensors (CVE-2021-37679)
  • Fixes a division by zero in TFLite (CVE-2021-37680)
  • Fixes an NPE in TFLite (CVE-2021-37681)
  • Fixes a vulnerability arising from use of unitialized value in TFLite (CVE-2021-37682)
  • Fixes an FPE in TFLite division operations (CVE-2021-37683)
  • Fixes an FPE in TFLite pooling operations (CVE-2021-37684)
  • Fixes an infinite loop in TFLite (CVE-2021-37686)
  • Fixes a heap OOB in TFLite (CVE-2021-37685)
  • Fixes a heap OOB in TFLite's Gather* implementations (CVE-2021-37687)
  • Fixes an undefined behavior arising from null pointer dereference in TFLite (CVE-2021-37688)
  • Fixes an undefined behavior arising from null pointer dereference in TFLite MLIR optimizations (CVE-2021-37689)
  • Fixes a FPE in LSH in TFLite (CVE-2021-37691)
  • Fixes a segfault on strings tensors with mismatched dimensions, arising in Go code (CVE-2021-37692)
  • Fixes a use after free and a potential segfault in shape inference functions (CVE-2021-37690)
  • Updates curl to 7.77.0 to handle CVE-2021-22876, CVE-2021-22897, CVE-2021-22898, and CVE-2021-22901.

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.6.0-rc2

Release 2.6.0

Breaking Changes

  • tf.train.experimental.enable_mixed_precision_graph_rewrite is removed, as the API only works in graph mode and is not customizable. The function is still accessible under tf.compat.v1.mixed_precision.enable_mixed_precision_graph_rewrite, but it is recommended to use the Keras mixed precision API instead.

  • tf.lite:

    • Remove experimental.nn.dynamic_rnn, experimental.nn.TfLiteRNNCell and experimental.nn.TfLiteLSTMCell since they're no longersupported. It's recommended to just use keras lstm instead.

  • Keras been split into a separate PIP package (keras), and its code has been moved to the GitHub repositorykeras-team/keras. The API endpoints for tf.keras stay unchanged, but are now backed by the keras PIP package. The existing code in tensorflow/python/keras is a staled copy and will be removed in future release (2.7). Please remove any imports to tensorflow.python.keras and replace them with public tf.keras API instead.

Known Caveats

  • TF Core:
    • A longstanding bug in tf.while_loop, which caused it to execute sequentially, even when parallel_iterations>1, has now been fixed. However, the increased parallelism may result in increased memory use. Users who experience unwanted regressions should reset their while_loop's parallel_iterations value to 1, which is consistent with prior behavior.

Major Features and Improvements

  • tf.keras:

    • Keras has been split into a separate PIP package (keras), and its code has been moved to the GitHub repository keras-team/keras. The API endpoints for tf.keras stay unchanged, but are now backed by the keras PIP package. All Keras-related PRs and issues should now be directed to the GitHub repository keras-team/keras.
    • tf.keras.utils.experimental.DatasetCreator now takes an optional tf.distribute.InputOptions for specific options when used with distribution.
    • tf.keras.experimental.SidecarEvaluator is now available for a program intended to be run on an evaluator task, which is commonly used to supplement a training cluster running with tf.distribute.experimental.ParameterServerStrategy (see `https://www.tensorflow.org/tutorials/distribute/parameterservertraining). It can also be used with single-worker training or other strategies. See docstring for more info.
    • Preprocessing layers moved from experimental to core.
      • Import paths moved from tf.keras.layers.preprocessing.experimental to tf.keras.layers.
    • Updates to Preprocessing layers API for consistency and clarity:
      • StringLookup and IntegerLookup default for mask_token changed to None. This matches the default masking behavior of Hashing and Embedding layers. To keep existing behavior, pass mask_token="" during layer creation.
      • Renamed "binary" output mode to "multi_hot" for CategoryEncoding, StringLookup, IntegerLookup, and TextVectorization. Multi-hot encoding will no longer automatically uprank rank 1 inputs, so these layers can now multi-hot encode unbatched multi-dimensional samples.
      • Added a new output mode "one_hot" for CategoryEncoding, StringLookup, IntegerLookup, which will encode each element in an input batch individually, and automatically append a new output dimension if necessary. Use this mode on rank 1 inputs for the old "binary" behavior of one-hot encoding a batch of scalars.
      • Normalization will no longer automatically uprank rank 1 inputs, allowing normalization of unbatched multi-dimensional samples.

  • tf.lite:

    • The recommended Android NDK version for building TensorFlow Lite has been changed from r18b to r19c.
    • Supports int64 for mul.
    • Supports native variable builtin ops - ReadVariable, AssignVariable.
    • Converter:
      • Experimental support for variables in TFLite. To enable through conversion, users need to set experimental_enable_resource_variables on tf.lite.TFLiteConverter to True. Note: mutable variables is only available using from_saved_model in this release, support for other methods is coming soon.
      • Old Converter (TOCO) is getting removed from next release. It's been deprecated for few releases already.

  • tf.saved_model:

    • SavedModels can now save custom gradients. Use the option tf.saved_model.SaveOption(experimental_custom_gradients=True) to enable this feature. The documentation in Advanced autodiff has been updated.
    • Object metadata has now been deprecated and no longer saved to the SavedModel.

  • TF Core:

    • Added tf.config.experimental.reset_memory_stats to reset the tracked peak memory returned by tf.config.experimental.get_memory_info.

  • tf.data:

    • Added target_workers param to data_service_ops.from_dataset_id and data_service_ops.distribute. Users can specify "AUTO", "ANY", or "LOCAL" (case insensitive). If "AUTO", tf.data service runtime decides which workers to read from. If "ANY", TF workers read from any tf.data service workers. If "LOCAL", TF workers will only read from local in-processs tf.data service workers. "AUTO" works well for most cases, while users can specify other targets. For example, "LOCAL" would help avoid RPCs and data copy if every TF worker colocates with a tf.data service worker. Currently, "AUTO" reads from any tf.data service workers to preserve existing behavior. The default value is "AUTO".

Bug Fixes and Other Changes

  • TF Core:
    • Added tf.lookup.experimental.MutableHashTable, which provides a generic mutable hash table implementation.
      • Compared to tf.lookup.experimental.DenseHashTable this offers lower overall memory usage, and a cleaner API. It does not require specifying a delete_key and empty_key that cannot be inserted into the table.
    • Added support for specifying number of subdivisions in all reduce host collective. This parallelizes work on CPU and speeds up the collective performance. Default behavior is unchanged.
    • Add an option perturb_singular to tf.linalg.tridiagonal_solve that allows solving linear systems with a numerically singular tridiagonal matrix, e.g. for use in inverse iteration.
    • Added tf.linalg.eigh_tridiagonal that computes the eigenvalues of a Hermitian tridiagonal matrix.
    • tf.constant now places its output on the current default device.
    • SavedModel
      • Added tf.saved_model.experimental.TrackableResource, which allows the creation of custom wrapper objects for resource tensors.
      • Added a SavedModel load option to allow restoring partial checkpoints into the SavedModel. See tf.saved_model.LoadOptions for details.
    • Added a new op SparseSegmentSumGrad to match the other sparse segment gradient ops and avoid an extra gather operation that was in the previous gradient implementation.
    • Added a new session config setting internal_fragmentation_fraction, which controls when the BFC Allocator needs to split an oversized chunk to satisfy an allocation request.
    • Added tf.get_current_name_scope() which returns the current full name scope string that will be prepended to op names.
  • tf.data:
    • Promoting tf.data.experimental.bucket_by_sequence_length API to tf.data.Dataset.bucket_by_sequence_length and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.get_single_element API to tf.data.Dataset.get_single_element and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.group_by_window API to tf.data.Dataset.group_by_window and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.RandomDataset API to tf.data.Dataset.random and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.scan API to tf.data.Dataset.scan and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.snapshot API to tf.data.Dataset.shapshot and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.take_while API to tf.data.Dataset.take_while and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.ThreadingOptions API to tf.data.ThreadingOptions and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.unique API to tf.data.Dataset.unique and deprecating the experimental endpoint.
    • Added stop_on_empty_dataset parameter to sample_from_datasets and choose_from_datasets. Setting stop_on_empty_dataset=True will stop sampling if it encounters an empty dataset. This preserves the sampling ratio throughout training. The prior behavior was to continue sampling, skipping over exhausted datasets, until all datasets are exhausted. By default, the original behavior (stop_on_empty_dataset=False) is preserved.
    • Removed previously deprecated tf.data statistics related APIs:
      • tf.data.Options.experimental_stats
      • tf.data.experimental.StatsAggregator
      • tf.data.experimental.StatsOptions.*
      • tf.data.experimental.bytes_produced_stats
      • tf.data.experimental.latency_stats
    • Removed the following experimental tf.data optimization APIs:
      • tf.data.experimental.MapVectorizationOptions.*
      • tf.data.experimental.OptimizationOptions.filter_with_random_uniform_fusion
      • tf.data.experimental.OptimizationOptions.hoist_random_uniform
      • tf.data.experimental.OptimizationOptions.map_vectorization * tf.data.experimental.OptimizationOptions.reorder_data_discarding_ops
  • tf.keras:
    • Fix usage of __getitem__ slicing in Keras Functional APIs when the inputs are RaggedTensor objects.
    • Add keepdims argument to all GlobalPooling layers.
    • Add include_preprocessing argument to MobileNetV3 architectures to control the inclusion of Rescaling layer in the model.
    • Add optional argument (force) to make_(train|test|predict)_funtion methods to skip the cached function and generate a new one. This is useful to regenerate in a single call the compiled training function when any .trainable attribute of any model's layer has changed.
    • Models now have a save_spec property which contains the TensorSpec specs for calling the model. This spec is automatically saved when the model is called for the first time.
  • tf.linalg:
    • Add CompositeTensor as a base class to LinearOperator.
  • tf.lite:
    • Fix mean op reference quantization rounding issue.
    • Added framework_stable BUILD target, which links in only the non-experimental TF Lite APIs.
    • Remove deprecated Java Interpreter methods:
      • modifyGraphWithDelegate - Use Interpreter.Options.addDelegate
      • setNumThreads - Use Interpreter.Options.setNumThreads
    • Add Conv3DTranspose as a builtin op.
  • tf.summary:
    • Fix tf.summary.should_record_summaries() so it correctly reflects when summaries will be written, even when tf.summary.record_if() is not n effect, by returning True tensor if default writer is present.
  • Grappler:
    • Disable default Grappler optimization timeout to make the optimization pipeline deterministic. This may lead to increased model loading time, because time spent in graph optimizations is now unbounded (was 20 minutes).
  • Deterministic Op Functionality (enabled by setting TF_DETERMINISTIC_OPS to "true" or "1"):
    • Add a deterministic GPU implementation of tf.nn.softmax_cross_entropy_with_logits. See PR 49178.
    • Add a deterministic CPU implementation of tf.image.crop_and_resize. See PR 48905.
    • Add determinism-unimplemented exception-throwing to the following ops. When op-determinism is expected, an attempt to use the specified paths through the following ops on a GPU will cause tf.errors.UnimplementedError (with an understandable message) to be thrown.
      • tf.nn.sparse_softmax_cross_entropy_with_logits forwards and/or backwards. See PR 47925.
      • tf.image.crop_and_resize gradient w.r.t. either image or boxes. See PR 48905.
      • tf.sparse.sparse_dense_matmul forwards. See PR 50355.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aadhitya A, Abhilash Mahendrakar, Abhishek Varma, Abin Shahab, Adam Hillier, Aditya Kane, AdityaKane2001, ag.ramesh, Amogh Joshi, Armen Poghosov, armkevincheng, Avrosh K, Ayan Moitra, azazhu, Banikumar Maiti, Bas Aarts, bhack, Bhanu Prakash Bandaru Venkata, Billy Cao, Bohumir Zamecnik, Bradley Reece, CyanXu, Daniel Situnayake, David Pal, Ddavis-2015, DEKHTIARJonathan, Deven Desai, Duncan Riach, Edward, Eli Osherovich, Eugene Kuznetsov, europeanplaice, evelynmitchell, Evgeniy Polyakov, Felix Vollmer, Florentin Hennecker, François Chollet, Frederic Bastien, Fredrik Knutsson, Gabriele Macchi, Gaurav Shukla, Gauri1 Deshpande, geetachavan1, Georgiy Manuilov, H, Hengwen Tong, Henri Woodcock, Hiran Sarkar, Ilya Arzhannikov, Janghoo Lee, jdematos, Jens Meder, Jerry Shih, jgehw, Jim Fisher, Jingbei Li, Jiri Podivin, Joachim Gehweiler, Johannes Lade, Jonas I. Liechti, Jonas Liechti, Jonas Ohlsson, Jonathan Dekhtiar, Julian Gross, Kaixi Hou, Kevin Cheng, Koan-Sin Tan, Kulin Seth, linzewen, Liubov Batanina, luisleee, Lukas Geiger, Mahmoud Abuzaina, mathgaming, Matt Conley, Max H. Gerlach, mdfaijul, Mh Kwon, Michael Martis, Michal Szutenberg, Måns Nilsson, nammbash, Neil Girdhar, Nicholas Vadivelu, Nick Kreeger, Nirjas Jakilim, okyanusoz, Patrice Vignola, Patrik Laurell, Pedro Marques, Philipp Hack, Phillip Cloud, Piergiacomo De Marchi, Prashant Kumar, puneeshkhanna, pvarouktsis, QQ喵, Rajeshwar Reddy T, Rama Ketineni, Reza Rahimi, Robert Kalmar, rsun, Ryan Kuester, Saduf2019, Sean Morgan, Sean Moriarity, Shaochen Shi, Sheng, Yang, Shu Wang, Shuai Zhang, Soojeong, Stanley-Nod, Steven I Reeves, stevenireeves, Suraj Sudhir, Sven Mayer, Tamas Bela Feher, tashuang.zk, tcervi, Teng Lu, Thales Elero Cervi, Thibaut Goetghebuer-Planchon, Thomas Walther, Till Brychcy, Trent Lo, Uday Bondhugula, vishakha.agrawal, Vishnuvardhan Janapati, wamuir, Wenwen Ouyang, wenwu, Williard Joshua Jose, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yasir Modak, Yi Li, Yong Tang, zilinzhu, 박상준, 이장

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.6.0-rc1

Release 2.6.0

Breaking Changes

  • tf.train.experimental.enable_mixed_precision_graph_rewrite is removed, as the API only works in graph mode and is not customizable. The function is still accessible under tf.compat.v1.mixed_precision.enable_mixed_precision_graph_rewrite, but it is recommended to use the Keras mixed precision API instead.

  • tf.lite:

    • Remove experimental.nn.dynamic_rnn, experimental.nn.TfLiteRNNCell and experimental.nn.TfLiteLSTMCell since they're no longersupported. It's recommended to just use keras lstm instead.

  • Keras been split into a separate PIP package (keras), and its code has been moved to the GitHub repositorykeras-team/keras. The API endpoints for tf.keras stay unchanged, but are now backed by the keras PIP package. The existing code in tensorflow/python/keras is a staled copy and will be removed in future release (2.7). Please remove any imports to tensorflow.python.keras and replace them with public tf.keras API instead.

Known Caveats

  • TF Core:
    • A longstanding bug in tf.while_loop, which caused it to execute sequentially, even when parallel_iterations>1, has now been fixed. However, the increased parallelism may result in increased memory use. Users who experience unwanted regressions should reset their while_loop's parallel_iterations value to 1, which is consistent with prior behavior.

Major Features and Improvements

  • tf.keras:

    • Keras has been split into a separate PIP package (keras), and its code has been moved to the GitHub repository keras-team/keras. The API endpoints for tf.keras stay unchanged, but are now backed by the keras PIP package. All Keras-related PRs and issues should now be directed to the GitHub repository keras-team/keras.
    • tf.keras.utils.experimental.DatasetCreator now takes an optional tf.distribute.InputOptions for specific options when used with distribution.
    • tf.keras.experimental.SidecarEvaluator is now available for a program intended to be run on an evaluator task, which is commonly used to supplement a training cluster running with tf.distribute.experimental.ParameterServerStrategy (see `https://www.tensorflow.org/tutorials/distribute/parameterservertraining). It can also be used with single-worker training or other strategies. See docstring for more info.
    • Preprocessing layers moved from experimental to core.
      • Import paths moved from tf.keras.layers.preprocessing.experimental to tf.keras.layers.
    • Updates to Preprocessing layers API for consistency and clarity:
      • StringLookup and IntegerLookup default for mask_token changed to None. This matches the default masking behavior of Hashing and Embedding layers. To keep existing behavior, pass mask_token="" during layer creation.
      • Renamed "binary" output mode to "multi_hot" for CategoryEncoding, StringLookup, IntegerLookup, and TextVectorization. Multi-hot encoding will no longer automatically uprank rank 1 inputs, so these layers can now multi-hot encode unbatched multi-dimensional samples.
      • Added a new output mode "one_hot" for CategoryEncoding, StringLookup, IntegerLookup, which will encode each element in an input batch individually, and automatically append a new output dimension if necessary. Use this mode on rank 1 inputs for the old "binary" behavior of one-hot encoding a batch of scalars.
      • Normalization will no longer automatically uprank rank 1 inputs, allowing normalization of unbatched multi-dimensional samples.

  • tf.lite:

    • The recommended Android NDK version for building TensorFlow Lite has been changed from r18b to r19c.
    • Supports int64 for mul.
    • Supports native variable builtin ops - ReadVariable, AssignVariable.
    • Converter:
      • Experimental support for variables in TFLite. To enable through conversion, users need to set experimental_enable_resource_variables on tf.lite.TFLiteConverter to True. Note: mutable variables is only available using from_saved_model in this release, support for other methods is coming soon.
      • Old Converter (TOCO) is getting removed from next release. It's been deprecated for few releases already.

  • tf.saved_model:

    • SavedModels can now save custom gradients. Use the option tf.saved_model.SaveOption(experimental_custom_gradients=True) to enable this feature. The documentation in Advanced autodiff has been updated.
    • Object metadata has now been deprecated and no longer saved to the SavedModel.

  • TF Core:

    • Added tf.config.experimental.reset_memory_stats to reset the tracked peak memory returned by tf.config.experimental.get_memory_info.

  • tf.data:

    • Added target_workers param to data_service_ops.from_dataset_id and data_service_ops.distribute. Users can specify "AUTO", "ANY", or "LOCAL" (case insensitive). If "AUTO", tf.data service runtime decides which workers to read from. If "ANY", TF workers read from any tf.data service workers. If "LOCAL", TF workers will only read from local in-processs tf.data service workers. "AUTO" works well for most cases, while users can specify other targets. For example, "LOCAL" would help avoid RPCs and data copy if every TF worker colocates with a tf.data service worker. Currently, "AUTO" reads from any tf.data service workers to preserve existing behavior. The default value is "AUTO".

Bug Fixes and Other Changes

  • TF Core:
    • Added tf.lookup.experimental.MutableHashTable, which provides a generic mutable hash table implementation.
      • Compared to tf.lookup.experimental.DenseHashTable this offers lower overall memory usage, and a cleaner API. It does not require specifying a delete_key and empty_key that cannot be inserted into the table.
    • Added support for specifying number of subdivisions in all reduce host collective. This parallelizes work on CPU and speeds up the collective performance. Default behavior is unchanged.
    • Add an option perturb_singular to tf.linalg.tridiagonal_solve that allows solving linear systems with a numerically singular tridiagonal matrix, e.g. for use in inverse iteration.
    • Added tf.linalg.eigh_tridiagonal that computes the eigenvalues of a Hermitian tridiagonal matrix.
    • tf.constant now places its output on the current default device.
    • SavedModel
      • Added tf.saved_model.experimental.TrackableResource, which allows the creation of custom wrapper objects for resource tensors.
      • Added a SavedModel load option to allow restoring partial checkpoints into the SavedModel. See tf.saved_model.LoadOptions for details.
    • Added a new op SparseSegmentSumGrad to match the other sparse segment gradient ops and avoid an extra gather operation that was in the previous gradient implementation.
    • Added a new session config setting internal_fragmentation_fraction, which controls when the BFC Allocator needs to split an oversized chunk to satisfy an allocation request.
    • Added tf.get_current_name_scope() which returns the current full name scope string that will be prepended to op names.
  • tf.data:
    • Promoting tf.data.experimental.bucket_by_sequence_length API to tf.data.Dataset.bucket_by_sequence_length and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.get_single_element API to tf.data.Dataset.get_single_element and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.group_by_window API to tf.data.Dataset.group_by_window and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.RandomDataset API to tf.data.Dataset.random and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.scan API to tf.data.Dataset.scan and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.snapshot API to tf.data.Dataset.shapshot and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.take_while API to tf.data.Dataset.take_while and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.ThreadingOptions API to tf.data.ThreadingOptions and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.unique API to tf.data.Dataset.unique and deprecating the experimental endpoint.
    • Added stop_on_empty_dataset parameter to sample_from_datasets and choose_from_datasets. Setting stop_on_empty_dataset=True will stop sampling if it encounters an empty dataset. This preserves the sampling ratio throughout training. The prior behavior was to continue sampling, skipping over exhausted datasets, until all datasets are exhausted. By default, the original behavior (stop_on_empty_dataset=False) is preserved.
    • Removed previously deprecated tf.data statistics related APIs:
      • tf.data.Options.experimental_stats
      • tf.data.experimental.StatsAggregator
      • tf.data.experimental.StatsOptions.*
      • tf.data.experimental.bytes_produced_stats
      • tf.data.experimental.latency_stats
    • Removed the following experimental tf.data optimization APIs:
      • tf.data.experimental.MapVectorizationOptions.*
      • tf.data.experimental.OptimizationOptions.filter_with_random_uniform_fusion
      • tf.data.experimental.OptimizationOptions.hoist_random_uniform
      • tf.data.experimental.OptimizationOptions.map_vectorization * tf.data.experimental.OptimizationOptions.reorder_data_discarding_ops
  • tf.keras:
    • Fix usage of __getitem__ slicing in Keras Functional APIs when the inputs are RaggedTensor objects.
    • Add keepdims argument to all GlobalPooling layers.
    • Add include_preprocessing argument to MobileNetV3 architectures to control the inclusion of Rescaling layer in the model.
    • Add optional argument (force) to make_(train|test|predict)_funtion methods to skip the cached function and generate a new one. This is useful to regenerate in a single call the compiled training function when any .trainable attribute of any model's layer has changed.
    • Models now have a save_spec property which contains the TensorSpec specs for calling the model. This spec is automatically saved when the model is called for the first time.
  • tf.linalg:
    • Add CompositeTensor as a base class to LinearOperator.
  • tf.lite:
    • Fix mean op reference quantization rounding issue.
    • Added framework_stable BUILD target, which links in only the non-experimental TF Lite APIs.
    • Remove deprecated Java Interpreter methods:
      • modifyGraphWithDelegate - Use Interpreter.Options.addDelegate
      • setNumThreads - Use Interpreter.Options.setNumThreads
    • Add Conv3DTranspose as a builtin op.
  • tf.summary:
    • Fix tf.summary.should_record_summaries() so it correctly reflects when summaries will be written, even when tf.summary.record_if() is not n effect, by returning True tensor if default writer is present.
  • Grappler:
    • Disable default Grappler optimization timeout to make the optimization pipeline deterministic. This may lead to increased model loading time, because time spent in graph optimizations is now unbounded (was 20 minutes).
  • Deterministic Op Functionality (enabled by setting TF_DETERMINISTIC_OPS to "true" or "1"):
    • Add a deterministic GPU implementation of tf.nn.softmax_cross_entropy_with_logits. See PR 49178.
    • Add a deterministic CPU implementation of tf.image.crop_and_resize. See PR 48905.
    • Add determinism-unimplemented exception-throwing to the following ops. When op-determinism is expected, an attempt to use the specified paths through the following ops on a GPU will cause tf.errors.UnimplementedError (with an understandable message) to be thrown.
      • tf.nn.sparse_softmax_cross_entropy_with_logits forwards and/or backwards. See PR 47925)
      • tf.image.crop_and_resize gradient w.r.t. either image or boxes. See PR 48905.
      • tf.sparse.sparse_dense_matmul forwards. See PR 50355.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aadhitya A, Abhilash Mahendrakar, Abhishek Varma, Abin Shahab, Adam Hillier, Aditya Kane, AdityaKane2001, ag.ramesh, Amogh Joshi, Armen Poghosov, armkevincheng, Avrosh K, Ayan Moitra, azazhu, Banikumar Maiti, Bas Aarts, bhack, Bhanu Prakash Bandaru Venkata, Billy Cao, Bohumir Zamecnik, Bradley Reece, CyanXu, Daniel Situnayake, David Pal, Ddavis-2015, DEKHTIARJonathan, Deven Desai, Duncan Riach, Edward, Eli Osherovich, Eugene Kuznetsov, europeanplaice, evelynmitchell, Evgeniy Polyakov, Felix Vollmer, Florentin Hennecker, François Chollet, Frederic Bastien, Fredrik Knutsson, Gabriele Macchi, Gaurav Shukla, Gauri1 Deshpande, geetachavan1, Georgiy Manuilov, H, Hengwen Tong, Henri Woodcock, Hiran Sarkar, Ilya Arzhannikov, Janghoo Lee, jdematos, Jens Meder, Jerry Shih, jgehw, Jim Fisher, Jingbei Li, Jiri Podivin, Joachim Gehweiler, Johannes Lade, Jonas I. Liechti, Jonas Liechti, Jonas Ohlsson, Jonathan Dekhtiar, Julian Gross, Kaixi Hou, Kevin Cheng, Koan-Sin Tan, Kulin Seth, linzewen, Liubov Batanina, luisleee, Lukas Geiger, Mahmoud Abuzaina, mathgaming, Matt Conley, Max H. Gerlach, mdfaijul, Mh Kwon, Michael Martis, Michal Szutenberg, Måns Nilsson, nammbash, Neil Girdhar, Nicholas Vadivelu, Nick Kreeger, Nirjas Jakilim, okyanusoz, Patrice Vignola, Patrik Laurell, Pedro Marques, Philipp Hack, Phillip Cloud, Piergiacomo De Marchi, Prashant Kumar, puneeshkhanna, pvarouktsis, QQ喵, Rajeshwar Reddy T, Rama Ketineni, Reza Rahimi, Robert Kalmar, rsun, Ryan Kuester, Saduf2019, Sean Morgan, Sean Moriarity, Shaochen Shi, Sheng, Yang, Shu Wang, Shuai Zhang, Soojeong, Stanley-Nod, Steven I Reeves, stevenireeves, Suraj Sudhir, Sven Mayer, Tamas Bela Feher, tashuang.zk, tcervi, Teng Lu, Thales Elero Cervi, Thibaut Goetghebuer-Planchon, Thomas Walther, Till Brychcy, Trent Lo, Uday Bondhugula, vishakha.agrawal, Vishnuvardhan Janapati, wamuir, Wenwen Ouyang, wenwu, Williard Joshua Jose, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yasir Modak, Yi Li, Yong Tang, zilinzhu, 박상준, 이장

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.6.0-rc0

Release 2.6.0

Breaking Changes

  • tf.train.experimental.enable_mixed_precision_graph_rewrite is removed, as the API only works in graph mode and is not customizable. The function is still accessible under tf.compat.v1.mixed_precision.enable_mixed_precision_graph_rewrite, but it is recommended to use the Keras mixed precision API instead.

  • tf.lite:

    • Remove experimental.nn.dynamic_rnn, experimental.nn.TfLiteRNNCell and experimental.nn.TfLiteLSTMCell since they're no longersupported. It's recommended to just use keras lstm instead.

Known Caveats

  • TF Core:
    • A longstanding bug in tf.while_loop, which caused it to execute sequentially, even when parallel_iterations>1, has now been fixed. However, the increased parallelism may result in increased memory use. Users who experience unwanted regressions should reset their while_loop's parallel_iterations value to 1, which is consistent with prior behavior.

Major Features and Improvements

  • tf.keras:

    • Keras has been split into a separate PIP package (keras), and its code has been moved to the GitHub repository keras-team/keras. The API endpoints for tf.keras stay unchanged, but are now backed by the keras PIP package. All Keras-related PRs and issues should now be directed to the GitHub repository keras-team/keras.
    • tf.keras.utils.experimental.DatasetCreator now takes an optional tf.distribute.InputOptions for specific options when used with distribution.
    • tf.keras.experimental.SidecarEvaluator is now available for a program intended to be run on an evaluator task, which is commonly used to supplement a training cluster running with tf.distribute.experimental.ParameterServerStrategy (see `https://www.tensorflow.org/tutorials/distribute/parameterservertraining). It can also be used with single-worker training or other strategies. See docstring for more info.
    • Preprocessing layers moved from experimental to core.
      • Import paths moved from tf.keras.layers.preprocessing.experimental to tf.keras.layers.
    • Updates to Preprocessing layers API for consistency and clarity:
      • StringLookup and IntegerLookup default for mask_token changed to None. This matches the default masking behavior of Hashing and Embedding layers. To keep existing behavior, pass mask_token="" during layer creation.
      • Renamed "binary" output mode to "multi_hot" for CategoryEncoding, StringLookup, IntegerLookup, and TextVectorization. Multi-hot encoding will no longer automatically uprank rank 1 inputs, so these layers can now multi-hot encode unbatched multi-dimensional samples.
      • Added a new output mode "one_hot" for CategoryEncoding, StringLookup, IntegerLookup, which will encode each element in an input batch individually, and automatically append a new output dimension if necessary. Use this mode on rank 1 inputs for the old "binary" behavior of one-hot encoding a batch of scalars.
      • Normalization will no longer automatically uprank rank 1 inputs, allowing normalization of unbatched multi-dimensional samples.

  • tf.lite:

    • The recommended Android NDK version for building TensorFlow Lite has been changed from r18b to r19c.
    • Supports int64 for mul.
    • Supports native variable builtin ops - ReadVariable, AssignVariable.
    • Converter:
      • Experimental support for variables in TFLite. To enable through conversion, users need to set experimental_enable_resource_variables on tf.lite.TFLiteConverter to True. Note: mutable variables is only available using from_saved_model in this release, support for other methods is coming soon.
      • Old Converter (TOCO) is getting removed from next release. It's been deprecated for few releases already.

  • tf.saved_model:

    • SavedModels can now save custom gradients. Use the option tf.saved_model.SaveOption(experimental_custom_gradients=True) to enable this feature. The documentation in Advanced autodiff has been updated.
    • Object metadata has now been deprecated and no longer saved to the SavedModel.

  • TF Core:

    • Added tf.config.experimental.reset_memory_stats to reset the tracked peak memory returned by tf.config.experimental.get_memory_info.

  • tf.data:

    • Added target_workers param to data_service_ops.from_dataset_id and data_service_ops.distribute. Users can specify "AUTO", "ANY", or "LOCAL" (case insensitive). If "AUTO", tf.data service runtime decides which workers to read from. If "ANY", TF workers read from any tf.data service workers. If "LOCAL", TF workers will only read from local in-processs tf.data service workers. "AUTO" works well for most cases, while users can specify other targets. For example, "LOCAL" would help avoid RPCs and data copy if every TF worker colocates with a tf.data service worker. Currently, "AUTO" reads from any tf.data service workers to preserve existing behavior. The default value is "AUTO".

Bug Fixes and Other Changes

  • TF Core:
    • Added tf.lookup.experimental.MutableHashTable, which provides a generic mutable hash table implementation.
      • Compared to tf.lookup.experimental.DenseHashTable this offers lower overall memory usage, and a cleaner API. It does not require specifying a delete_key and empty_key that cannot be inserted into the table.
    • Added support for specifying number of subdivisions in all reduce host collective. This parallelizes work on CPU and speeds up the collective performance. Default behavior is unchanged.
    • Add an option perturb_singular to tf.linalg.tridiagonal_solve that allows solving linear systems with a numerically singular tridiagonal matrix, e.g. for use in inverse iteration.
    • Added tf.linalg.eigh_tridiagonal that computes the eigenvalues of a Hermitian tridiagonal matrix.
    • tf.constant now places its output on the current default device.
    • SavedModel
      • Added tf.saved_model.experimental.TrackableResource, which allows the creation of custom wrapper objects for resource tensors.
      • Added a SavedModel load option to allow restoring partial checkpoints into the SavedModel. See tf.saved_model.LoadOptions for details.
    • Added a new op SparseSegmentSumGrad to match the other sparse segment gradient ops and avoid an extra gather operation that was in the previous gradient implementation.
    • Added a new session config setting internal_fragmentation_fraction, which controls when the BFC Allocator needs to split an oversized chunk to satisfy an allocation request.
    • Added tf.get_current_name_scope() which returns the current full name scope string that will be prepended to op names.
  • tf.data:
    • Promoting tf.data.experimental.bucket_by_sequence_length API to tf.data.Dataset.bucket_by_sequence_length and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.get_single_element API to tf.data.Dataset.get_single_element and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.group_by_window API to tf.data.Dataset.group_by_window and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.RandomDataset API to tf.data.Dataset.random and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.scan API to tf.data.Dataset.scan and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.snapshot API to tf.data.Dataset.shapshot and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.take_while API to tf.data.Dataset.take_while and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.ThreadingOptions API to tf.data.ThreadingOptions and deprecating the experimental endpoint.
    • Promoting tf.data.experimental.unique API to tf.data.Dataset.unique and deprecating the experimental endpoint.
    • Added stop_on_empty_dataset parameter to sample_from_datasets and choose_from_datasets. Setting stop_on_empty_dataset=True will stop sampling if it encounters an empty dataset. This preserves the sampling ratio throughout training. The prior behavior was to continue sampling, skipping over exhausted datasets, until all datasets are exhausted. By default, the original behavior (stop_on_empty_dataset=False) is preserved.
    • Removed previously deprecated tf.data statistics related APIs:
      • tf.data.Options.experimental_stats
      • tf.data.experimental.StatsAggregator
      • tf.data.experimental.StatsOptions.*
      • tf.data.experimental.bytes_produced_stats
      • tf.data.experimental.latency_stats
    • Removed the following experimental tf.data optimization APIs:
      • tf.data.experimental.MapVectorizationOptions.*
      • tf.data.experimental.OptimizationOptions.filter_with_random_uniform_fusion
      • tf.data.experimental.OptimizationOptions.hoist_random_uniform
      • tf.data.experimental.OptimizationOptions.map_vectorization * tf.data.experimental.OptimizationOptions.reorder_data_discarding_ops
  • tf.keras:
    • Fix usage of __getitem__ slicing in Keras Functional APIs when the inputs are RaggedTensor objects.
    • Add keepdims argument to all GlobalPooling layers.
    • Add include_preprocessing argument to MobileNetV3 architectures to control the inclusion of Rescaling layer in the model.
    • Add optional argument (force) to make_(train|test|predict)_funtion methods to skip the cached function and generate a new one. This is useful to regenerate in a single call the compiled training function when any .trainable attribute of any model's layer has changed.
    • Models now have a save_spec property which contains the TensorSpec specs for calling the model. This spec is automatically saved when the model is called for the first time.
  • tf.linalg:
    • Add CompositeTensor as a base class to LinearOperator.
  • tf.lite:
    • Fix mean op reference quantization rounding issue.
    • Added framework_stable BUILD target, which links in only the non-experimental TF Lite APIs.
    • Remove deprecated Java Interpreter methods:
      • modifyGraphWithDelegate - Use Interpreter.Options.addDelegate
      • setNumThreads - Use Interpreter.Options.setNumThreads
    • Add Conv3DTranspose as a builtin op.
  • tf.summary:
    • Fix tf.summary.should_record_summaries() so it correctly reflects when summaries will be written, even when tf.summary.record_if() is not n effect, by returning True tensor if default writer is present.
  • Grappler:
    • Disable default Grappler optimization timeout to make the optimization pipeline deterministic. This may lead to increased model loading time, because time spent in graph optimizations is now unbounded (was 20 minutes).

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Aadhitya A, Abhilash Mahendrakar, Abhishek Varma, Abin Shahab, Adam Hillier, Aditya Kane, AdityaKane2001, ag.ramesh, Amogh Joshi, Armen Poghosov, armkevincheng, Avrosh K, Ayan Moitra, azazhu, Banikumar Maiti, Bas Aarts, bhack, Bhanu Prakash Bandaru Venkata, Billy Cao, Bohumir Zamecnik, Bradley Reece, CyanXu, Daniel Situnayake, David Pal, Ddavis-2015, DEKHTIARJonathan, Deven Desai, Duncan Riach, Edward, Eli Osherovich, Eugene Kuznetsov, europeanplaice, evelynmitchell, Evgeniy Polyakov, Felix Vollmer, Florentin Hennecker, François Chollet, Frederic Bastien, Fredrik Knutsson, Gabriele Macchi, Gaurav Shukla, Gauri1 Deshpande, geetachavan1, Georgiy Manuilov, H, Hengwen Tong, Henri Woodcock, Hiran Sarkar, Ilya Arzhannikov, Janghoo Lee, jdematos, Jens Meder, Jerry Shih, jgehw, Jim Fisher, Jingbei Li, Jiri Podivin, Joachim Gehweiler, Johannes Lade, Jonas I. Liechti, Jonas Liechti, Jonas Ohlsson, Jonathan Dekhtiar, Julian Gross, Kaixi Hou, Kevin Cheng, Koan-Sin Tan, Kulin Seth, linzewen, Liubov Batanina, luisleee, Lukas Geiger, Mahmoud Abuzaina, mathgaming, Matt Conley, Max H. Gerlach, mdfaijul, Mh Kwon, Michael Martis, Michal Szutenberg, Måns Nilsson, nammbash, Neil Girdhar, Nicholas Vadivelu, Nick Kreeger, Nirjas Jakilim, okyanusoz, Patrice Vignola, Patrik Laurell, Pedro Marques, Philipp Hack, Phillip Cloud, Piergiacomo De Marchi, Prashant Kumar, puneeshkhanna, pvarouktsis, QQ喵, Rajeshwar Reddy T, Rama Ketineni, Reza Rahimi, Robert Kalmar, rsun, Ryan Kuester, Saduf2019, Sean Morgan, Sean Moriarity, Shaochen Shi, Sheng, Yang, Shu Wang, Shuai Zhang, Soojeong, Stanley-Nod, Steven I Reeves, stevenireeves, Suraj Sudhir, Sven Mayer, Tamas Bela Feher, tashuang.zk, tcervi, Teng Lu, Thales Elero Cervi, Thibaut Goetghebuer-Planchon, Thomas Walther, Till Brychcy, Trent Lo, Uday Bondhugula, vishakha.agrawal, Vishnuvardhan Janapati, wamuir, Wenwen Ouyang, wenwu, Williard Joshua Jose, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yasir Modak, Yi Li, Yong Tang, zilinzhu, 박상준, 이장

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.4.2

Release 2.4.2

This release introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.3.3

Release 2.3.3

This release introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.2.3

Release 2.2.3

Note that this is the last patch release for the TensorFlow 2.2.x series.

This release introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.1.4

Release 2.1.4

Note that this is the last patch release for the TensorFlow 2.1.x series.

This release introduces several vulnerability fixes:

- C++
Published by tensorflow-jenkins over 4 years ago

tensorflow - TensorFlow 2.5.0

Release 2.5.0

Major Features and Improvements

  • Support for Python3.9 has been added.
  • tf.data:
    • tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.
    • tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).
    • tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.
    • Options returned by tf.data.Dataset.options() are no longer mutable.
    • tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().
  • tf.lite
    • Enabled the new MLIR-based quantization backend by default
      • The new backend is used for 8 bits full integer post-training quantization
      • The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)
      • Set experimental_new_quantizer in tf.lite.TFLiteConverter to False to disable this change
  • tf.keras
    • tf.keras.metrics.AUC now support logit predictions.
    • Enabled a new supported input type in Model.fit, tf.keras.utils.experimental.DatasetCreator, which takes a callable, dataset_fn. DatasetCreator is intended to work across all tf.distribute strategies, and is the only input type supported for Parameter Server strategy.
  • tf.distribute
    • tf.distribute.experimental.ParameterServerStrategy now supports training with Keras Model.fit when used with DatasetCreator.
    • Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.
  • TPU embedding support
    • Added profile_data_directory to EmbeddingConfigSpec in _tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.
  • PluggableDevice
  • oneAPI Deep Neural Network Library (oneDNN) CPU performance optimizations from Intel-optimized TensorFlow are now available in the official x86-64 Linux and Windows builds.
    • They are off by default. Enable them by setting the environment variable TF_ENABLE_ONEDNN_OPTS=1.
    • We do not recommend using them in GPU systems, as they have not been sufficiently tested with GPUs yet.
  • TensorFlow pip packages are now built with CUDA11.2 and cuDNN 8.1.0

Breaking Changes

  • The TF_CPP_MIN_VLOG_LEVEL environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL which correctly describes its effect.

Bug Fixes and Other Changes

  • tf.keras:

    • Preprocessing layers API consistency changes:
      • StringLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization.
      • IntegerLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization. Renamed max_values, oov_value and mask_value to max_tokens, oov_token and mask_token to align with StringLookup and TextVectorization.
      • TextVectorization default for pad_to_max_tokens switched to False.
      • CategoryEncoding no longer supports adapt, IntegerLookup now supports equivalent functionality. max_tokens argument renamed to num_tokens.
      • Discretization added num_bins argument for learning bins boundaries through calling adapt on a dataset. Renamed bins argument to bin_boundaries for specifying bins without adapt.
    • Improvements to model saving/loading:
      • model.load_weights now accepts paths to saved models.
    • Keras inputs can now be created directly from arbitrary tf.TypeSpecs.
    • Two new learning rate schedules added: tf.keras.optimizers.schedules.CosineDecay andtf.keras.optimizers.schedules.CosineDecayRestarts.

  • tf.data:

    • Exposing tf.data.experimental.ExternalStatePolicy, which can be used to control how external state should be handled during dataset serialization or iterator checkpointing.
    • Changing tf.data.experimental.save to store the type specification of the dataset elements. This avoids the need for explicitly specifying the element_spec argument of tf.data.experimental.load when loading the previously saved dataset.
    • Add .element_spec property to tf.data.DatasetSpec to access the inner spec. This can be used to extract the structure of nested datasets.
    • Add tf.data.experimental.AutoShardingPolicy.HINT which can be used to provide hints to tf.distribute-based auto-sharding as to where in the input pipeline to insert sharding transformations.
    • Make tf.data.Options persistent across tf.function and GraphDef boundaries.

  • XLA compilation:

    • tf.function(experimental_compile=True) has become a stable API, renamed tf.function(jit_compile=True).
    • XLA can now compile MirroredStrategy: the step function passed tostrategy.run can now be annoted with jit_compile=True.

  • tf.distribute:

    • Rename experimental_prefetch_to_device in tf.distribute.InputOptions to experimental_fetch_to_device to better reflect the purpose.

  • tf.lite:

    • class tflite::Subgraph:
      • Removed the tensors() method and the non-const overload of the nodes_and_registration() method, both of which were previously documented as temporary and to be removed.
        • Uses of tensors() can be replaced by calling the existing methods tensors_size() and tensor(int).
        • Uses of the non-const overload of nodes_and_registration can be replaced by calling the existing methods nodes_size() and context(), and then calling the GetNodeAndRegistration method in the TfLiteContext returned by context().
    • NNAPI
      • Removed deprecated Interpreter::UseNNAPI(bool) C++ API.
        • Use NnApiDelegate() and related delegate configuration methods directly.
      • Replaced the model cache key for models computation algorithm with one guaranteed to be stable across runs.
    • 16 bits quantization
      • Added int16x8 support for ABS, REDUCEMAX and REDUCEMIN operators.
      • Additional tests and fixes for ADD and SUB operators.
    • Added support for saved model's session initializer through TFLiteConverter.from_saved_model.
    • Added DEPTHTOSPACE support in Post training quantization.
    • Added dynamic range quantization support for the BatchMatMul op.
      • Both symmetric and asymmetric quantized input tensor are supported.
    • Add RFFT2D as builtin op. (RFFT2D also supports RFFTD.) Currently only supports float32 input.
    • Add 5D support to SLICE op.
    • TFLite Supports SingatureDef:
      • TFLiteConverter exports models with SignatureDef
      • Interpreter supports getting a list of signatures and getting callable function for a given signaturedef.
    • Add int8 support for ReshapeV2.
    • Add experimental support for optimization with sparsity.
    • Add nominal support for unsigned 32-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for static hash tables through TFLiteConverter.from_saved_model.
    • The Python TF Lite Interpreter bindings now has an option experimental_preserve_all_tensors to aid in debugging conversion.
    • Quantized x86 execution defaults to Ruy GEMM library for platforms with AVX support.
    • Deprecate tf.compat.v1.lite.experimental.get_potentially_supported_ops. Use tf.lite.TFLiteConverter directly to check whether a model is convertible.
    • Add support to select one of three different built-in op resolvers
    • Enabled post training with calibrations for models that require user provided TensorFlow Lite custom op libraries via converter.target_spec._experimental_custom_op_registerers. used in Python Interpreter API.

  • TF Core:

    • Corrected higher-order gradients of control flow constructs (tf.cond, tf.while_loop, and compositions like tf.foldl) computed with tf.GradientTape inside a tf.function.
    • Changed the default step size in gradient_checker_v2.compute_gradients to be exactly representable as a binary floating point numbers. This avoids poluting gradient approximations needlessly, which is some cases leads to false negatives in op gradient tests.
    • Added tf.config.experimental.get_memory_info, returning a dict with the current and peak memory usage. Deprecated tf.config.experimental.get_memory_usage in favor of this new function.
    • Extended tf.config.experimental.enable_tensor_float_32_execution to control Tensor-Float-32 evaluation in RNNs.
    • Added a 'experimental_payloads' field to tf.errors.OpError and its subclasses to support more detailed error reporting. This is inspired from Abseil Status payloads: https://github.com/abseil/abseil-cpp/blob/master/absl/status/status.h

  • tf.summary:

    • New tf.summary.graph allows manual write of TensorFlow graph (tf.Graph or tf.compat.v1.GraphDef) as a summary. This is not a replacement for the trace-based API.

  • Set /d2ReducedOptimizeHugeFunctions by default for Windows builds. This provides a big compile-time speedup, and effectively raises the minimum supported MSVC version to 16.4 (current: 16.8).

    • See: https://groups.google.com/a/tensorflow.org/d/topic/build/SsW98Eo7l3o/discussion

  • TensorRT

    • Removed the deprecated session_config parameter for the TF1-TRT converter TrtGraphConverter. Previously, we issued a warning when the value of the parameter is not None.
    • The TF2-TRT converter TrtGraphConverterV2 takes an object of class TrtConversionParams as a parameter. Removed three deprecated fields from this class: rewriter_config_template, is_dynamic_op, and max_batch_size. Previously, we issued a warning when the value of rewriter_config_template is not None. We issued an error when the value of is_dynamic_op is not True. We didn't use the value for max_batch_size for building TensorRT engines. Add parameters use_dynamic_shape to enable dynamic shape support. The default is to disable dynamic shape support. Add dynamic_shape_profile_strategy for selecting a dynamic shape profile strategy. The default is profile strategy is Range.
    • Issue a warning when function gettensorrtrewriter_config is used.

  • TF XLA

    • Add new enum value MLIR_BRIDGE_ROLLOUT_SAFE_MODE_ENABLED to tf.config.experimental.mlir_bridge_rollout to enable a \"safe\" mode. This runs the MLIR bridge only when an analysis of the graph only when an analysis of the graph determines that it is safe to run.
    • Add new enum value MLIR_BRIDGE_ROLLOUT_SAFE_MODE_FALLBACK_ENABLED' totf.config.experimental.mlirbridgerollout` to enable a fallback for the MLIR bridge in a \"safe\" mode. This runs the MLIR bridge in a FallbackEnabled mode when an analysis of the graph determines that the graph does not have unsupported features.

  • Deterministic Op Functionality:

    • Add determinism-unimplemented exception-throwing to the segment-sum ops. When the environment variable TF_DETERMINISTIC_OPS is set to "true" or "1" (when op-determinism is expected), an attempt to run the following ops on a GPU will throw tf.errors.UnimplementedError (with an understandable message) when data is a floating-point type, including complex types (if supported): tf.math.segment_prod, tf.math.segment_sum, tf.math.unsorted_segment_mean, tf.math.unsorted_segment_sqrt_n, tf.math.unsorted_segment_prod, tf.math.unsorted_segment_sum, and therefore also tf.convert_to_tensor when value is of type tf.IndexedSlices (such as in the back prop though tf.gather into a dense embedding). See issue 39751 which this change addresses, but does not solve. This exception-throwing behavior can be disabled by setting the environment variable TF_DISABLE_SEGMENT_REDUCTION_OP_DETERMINISM_EXCEPTIONS to "true" or "1". For more information about these changes, see the description in pull request 47772.
    • In previous versions of TensorFlow, when a GPU was available, tf.sparse.sparse_dense_matmul introduced truly random noise in the forward path for data of type tf.float32 but not for data of type tf.float64 (for which there was no GPU implementation). In this current release, GPU support for other floating-point types (tf.float16, tf.float64, tf.complex64, and tf.complex128) has been added for this op. If you were relying on the determinism of the tf.float64 CPU implementation being automatically selected because of the absence of the tf.float64 GPU implementation, you with either need to force the op to run on the CPU or use a different data type.

  • Security

  • Other

    • Added show_debug_info to mlir.convert_graph_def and mlir.convert_function.
    • Added Arm Compute Library (ACL) support to --config=mkl_aarch64 build.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Aaron S. Mondal, Abhilash Mahendrakar, Abhinav Upadhyay, Abhishek Kulkarni, Abolfazl Shahbazi, Adam Hillier, Aditya Kane, Ag Ramesh, ahmedsabie, Albert Villanova Del Moral, Aleksey Vitebskiy, Alex Hoffman, Alexander Bayandin, Alfie Edwards, Aman Kishore, Amogh Joshi, andreABbauer, Andrew Goodbody, Andrzej Pomirski, Artemiy Ryabinkov, Ashish Jha, ather, Ayan Moitra, Bairen Yi, Bart Ribbers, Bas Aarts, Behzad Abghari, Ben Arnao, Ben Barsdell, Benjamin Klimczak, bhack, Brendan Collins, Can Wang, Cheng Ren, Chris Leary, Chris Olivier, Clemens Giuliani, Cloud Han, Corey Cole, Cui, Yifeng, Cuong V. Nguyen, Daniel Moore, Dawid Wojciechowski, Ddavis-2015, Dean Wyatte, Denisa Roberts, dependabot[bot], Dmitry Volodin, Dominic Jack, Duncan Riach, dushuai, Elena Zhelezina, Eli Osherovich, Erik Smistad, ewsn1593, Felix Fent, fo40225, François Chollet, Frederic Bastien, Freedom" Koan-Sin Tan, fsx950223, ganand1, gbaned, Georgiy Manuilov, gerbauz, Guillaume Klein, Guozhong Zhuang, Harry Slatyer, Harsh188, henri, Henri Woodcock, Hiran Sarkar, Hollow Man, Håkon Sandsmark, I Wayan Dharmana, icysapphire, Ikko Ashimine, Jab Hofmeier, Jack Hessel, Jacob Valdez, Jakub Jatczak, James Bernardi, Jared Smolens, Jason Zaman, jedlimlx, Jenny Plunkett, Jens Elofsson, Jerry Shih, jgehw, Jia Fu Low, Jim Fisher, jpodivin, Julien Stephan, Jungsub Lim, Junha Park, Junhyuk So, justkw, Kaixi Hou, kashyapraval, Kasra Bigdeli, Kazuaki Ishizaki, Keith Mok, Kevin Cheng, kopytjuk, Kristian Hartikainen, ksood12345, Kulin Seth, kushanam, latyas, Lequn Chen, Leslie-Fang, Long M. Lưu, Lukas Geiger, machineko, Mahmoud Abuzaina, Manish, Mao Yunfei, Maozhou, Ge, Marcin Juszkiewicz, Marcin Owsiany, Marconi Jiang, Marcos Pereira, Maria Romanenko Vexlard, Maria Vexlard, Marius Brehler, marload, Martin Kubovčík, Matej, Mateusz Holenko, Maxiwell S. Garcia, Mazhar, mazharul, mbhuiyan, mdfaijul, Michael Gielda, Michael Kuchnik, Michal Szutenberg, Mikhail Stepanov, Milan Straka, Mitchel Humpherys, Mohamed Moselhy, Mohamed Nour Abouelseoud, Måns Bermell, Måns Nilsson, Nathan Luehr, Nico Jahn, Niroop Ammbashankar, Oceania2018, Omri Steiner, Orivej Desh, Oskar Flordal, oujiafan, Patrik Laurell, Paul B. Isaac'S, Paul Klinger, Pawel Piskorski, Pedro Marques, Phat Tran, Piotr Zierhoffer, piyushdatta, Pnikam-Cad, Prashant Kumar, Prateek Gupta, PratsBhatt, Pravin Karandikar, qqq.jq, QQ喵, Quintin, Rama Ketineni, ravikyram, Rehan Guha, rhdong, rmothukuru, Roger Cheng, Rohit Santhanam, rposts, Rsanthanam-Amd, rsun, Rsun-Bdti, Ryan Kuester, ryanking13, Saduf2019, Sami Kama, Samuel Marks, Scott Tseng, Sean Moriarity, Sergey Popov, Sergii Khomenko, Sheng, Yang, shwetaoj, Sidong-Wei, Simon Maurer, Simrit Kaur, Srini511, Srinivasan Narayanamoorthy, Stephan, Stephen Matthews, Sungmann Cho, Sunoru, Suraj Sudhir, Suraj Upadhyay, Taebum Kim, Takayoshi Koizumi, Tamas Bela Feher, Teng Lu, Thibaut Goetghebuer-Planchon, Tomwildenhain-Microsoft, Tony, Traun Leyden, Trent Lo, TVLIgnacy, Tzu-Wei Sung, vaibhav, Vignesh Kothapalli, Vikram Dattu, viktprog, Vinayaka Bandishti, Vincent Abriou, Vishakha Agrawal, Vivek Panyam, Vladimir Silyaev, Võ Văn Nghĩa, wamuir, Wang, Yanzhang, wangsiyu, Waqar Hameed, wxinix, Xiao Yang, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yair Ehrenwald, Yajush Vyas, Yasir Modak, Yimei Sun, Yong Tang, Yosshi999, youshenmebutuo, yqtianust, Yuan Tang, yuanbopeng, Yuriy Chernyshov, Yuta Fukasawa, Zachary Deane-Mayer, Zeno Gantner, Zhoulong Jiang, zhuyie, zilinzhu, 彭震东

- C++
Published by tensorflow-jenkins almost 5 years ago

tensorflow - TensorFlow 2.5.0-rc3

Release 2.5.0

  • Support for Python3.9 has been added.
  • PluggableDevice
  • tf.data:
    • tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.
    • tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).
    • tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.
    • Options returned by tf.data.Dataset.options() are no longer mutable.
    • tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().
  • tf.lite
    • Enabled the new MLIR-based quantization backend by default
      • The new backend is used for 8 bits full integer post-training quantization
      • The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)
      • Set experimental_new_quantizer in tf.lite.TFLiteConverter to False to disable this change
  • tf.keras
    • tf.keras.metrics.AUC now support logit predictions.
    • Enabled a new supported input type in Model.fit, tf.keras.utils.experimental.DatasetCreator, which takes a callable, dataset_fn. DatasetCreator is intended to work across all tf.distribute strategies, and is the only input type supported for Parameter Server strategy.
  • tf.distribute
    • Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.
    • tf.distribute.experimental.ParameterServerStrategy now supports training with Keras Model.fit when used with DatasetCreator.
  • oneAPI Deep Neural Network Library (oneDNN) CPU performance optimizations from Intel-optimized TensorFlow are now available in the official x86-64 Linux and Windows builds.
    • They are off by default. Enable them by setting the environment variable TF_ENABLE_ONEDNN_OPTS=1.
    • We do not recommend using them in GPU systems, as they have not been sufficiently tested with GPUs yet.
  • TensorFlow pip packages are now built with CUDA11.2 and cuDNN 8.1.0
  • TPU embedding support
    • Added profile_data_directory to EmbeddingConfigSpec in _tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.

Breaking Changes

  • The TF_CPP_MIN_VLOG_LEVEL environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL which correctly describes its effect.

Bug Fixes and Other Changes

  • tf.keras:

    • Preprocessing layers API consistency changes:
      • StringLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization.
      • IntegerLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization. Renamed max_values, oov_value and mask_value to max_tokens, oov_token and mask_token to align with StringLookup and TextVectorization.
      • TextVectorization default for pad_to_max_tokens switched to False.
      • CategoryEncoding no longer supports adapt, IntegerLookup now supports equivalent functionality. max_tokens argument renamed to num_tokens.
      • Discretization added num_bins argument for learning bins boundaries through calling adapt on a dataset. Renamed bins argument to bin_boundaries for specifying bins without adapt.
    • Improvements to model saving/loading:
      • model.load_weights now accepts paths to saved models.
    • Keras inputs can now be created directly from arbitrary tf.TypeSpecs.
    • Two new learning rate schedules added:tf.keras.optimizers.schedules.CosineDecay and tf.keras.optimizers.schedules.CosineDecayRestarts.

  • tf.data:

    • Exposing tf.data.experimental.ExternalStatePolicy, which can be used to control how external state should be handled during dataset serialization or iterator checkpointing.
    • Changing tf.data.experimental.save to store the type specification of the dataset elements. This avoids the need for explicitly specifying the element_spec argument of tf.data.experimental.load when loading the previously saved dataset.
    • Add .element_spec property to tf.data.DatasetSpec to access the inner spec. This can be used to extract the structure of nested datasets.
    • Add tf.data.experimental.AutoShardingPolicy.HINT which can be used to provide hints to tf.distribute-based auto-sharding as to where in the input pipeline to insert sharding transformations.
    • Make tf.data.Options persistent across tf.function and GraphDef boundaries.

  • XLA compilation:

    • tf.function(experimental_compile=True) has become a stable API, renamed tf.function(jit_compile=True).
    • XLA can now compile MirroredStrategy: the step function passed to strategy.run can now be annoted with jit_compile=True.

  • tf.distribute:

    • Rename experimental_prefetch_to_device in tf.distribute.InputOptions to experimental_fetch_to_device to better reflect the purpose.

  • tf.lite:

    • class tflite::Subgraph:
      • Removed the tensors() method and the non-const overload of the nodes_and_registration() method, both of which were previously documented as temporary and to be removed.
        • Uses of tensors() can be replaced by calling the existing methods tensors_size() and tensor(int).
        • Uses of the non-const overload of nodes_and_registration can be replaced by calling the existing methods nodes_size() and context(), and then calling the GetNodeAndRegistration method in the TfLiteContext returned by context().
    • NNAPI
      • Removed deprecated Interpreter::UseNNAPI(bool) C++ API.
        • Use NnApiDelegate() and related delegate configuration methods directly.
      • Replaced the model cache key for models computation algorithm with one guaranteed to be stable across runs.
    • 16 bits quantization
      • Added int16x8 support for ABS, REDUCEMAX and REDUCEMIN operators.
      • Additional tests and fixes for ADD and SUB operators.
    • Added support for saved model's session initializer through TFLiteConverter.from_saved_model.
    • Added DEPTHTOSPACE support in Post training quantization.
    • Added dynamic range quantization support for the BatchMatMul op.
      • Both symmetric and asymmetric quantized input tensor are supported.
    • Add RFFT2D as builtin op. (RFFT2D also supports RFFTD.) Currently only supports float32 input.
    • Add 5D support to SLICE op.
    • TFLite Supports SingatureDef:
      • TFLiteConverter exports models with SignatureDef
      • Interpreter supports getting a list of signatures and getting callable function for a given signature def.
    • Add int8 support for ReshapeV2.
    • Add experimental support for optimization with sparsity.
    • Add nominal support for unsigned 32-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for static hash tables through TFLiteConverter.from_saved_model.
    • The Python TF Lite Interpreter bindings now has an option experimental_preserve_all_tensors to aid in debugging conversion.
    • Quantized x86 execution defaults to Ruy GEMM library for platforms with AVX support.
    • Deprecate tf.compat.v1.lite.experimental.get_potentially_supported_ops. Use tf.lite.TFLiteConverter directly to check whether a model is convertible.
    • Add support to select one of three different built-in op resolvers to be
    • Enabled post training with calibrations for models that require user provided TensorFlow Lite custom op libraries via converter.target_spec._experimental_custom_op_registerers. used in Python Interpreter API.

  • TF Core:

    • Corrected higher-order gradients of control flow constructs (tf.cond, tf.while_loop, and compositions like tf.foldl) computed with tf.GradientTape inside a tf.function.
    • Changed the default step size in gradient_checker_v2.compute_gradients to be exactly representable as a binary floating point numbers. This avoids poluting gradient approximations needlessly, which is some cases leads to false negatives in op gradient tests.
    • Added tf.config.experimental.get_memory_info, returning a dict with the current and peak memory usage. Deprecated tf.config.experimental.get_memory_usage in favor of this new function.
    • Extended tf.config.experimental.enable_tensor_float_32_execution to control Tensor-Float-32 evaluation in RNNs.
    • Added a 'experimental_payloads' field to tf.errors.OpError and its subclasses to support more detailed error reporting. This is inspired from Abseil Status payloads: https://github.com/abseil/abseil-cpp/blob/master/absl/status/status.h

  • tf.summary:

    • New tf.summary.graph allows manual write of TensorFlow graph (tf.Graph or tf.compat.v1.GraphDef) as a summary. This is not a replacement for the trace-based API.

  • Set /d2ReducedOptimizeHugeFunctions by default for Windows builds. This provides a big compile-time speedup, and effectively raises the minimum supported MSVC version to 16.4 (current: 16.8).

    • See: https://groups.google.com/a/tensorflow.org/d/topic/build/SsW98Eo7l3o/discussion

  • TensorRT

    • Removed the deprecated session_config parameter for the TF1-TRT converter TrtGraphConverter. Previously, we issued a warning when the value of the parameter is not None.
    • The TF2-TRT converter TrtGraphConverterV2 takes an object of class TrtConversionParams as a parameter. Removed three deprecated fields from this class: rewriter_config_template, is_dynamic_op, and max_batch_size. Previously, we issued a warning when the value of rewriter_config_template is not None. We issued an error when the value of is_dynamic_op is not True. We didn't use the value for max_batch_size for building TensorRT engines. Add parameters use_dynamic_shape to enable dynamic shape support. The default is to disable dynamic shape support. Add dynamic_shape_profile_strategy for selecting a dynamic shape profile strategy. The default is profile strategy is Range.
    • Issue a warning when function gettensorrtrewriter_config is used.

  • TF XLA

    • Add new enum value MLIR_BRIDGE_ROLLOUT_SAFE_MODE_ENABLED to tf.config.experimental.mlir_bridge_rollout to enable a \"safe\" mode. This runs the MLIR bridge only when an analysis of the graph only when an analysis of the graph determines that it is safe to run.
    • Add new enum value 'MLIRBRIDGEROLLOUTSAFEMODEFALLBACKENABLED' to tf.config.experimental.mlir_bridge_rollout to enable a fallback for the MLIR bridge in a \"safe\" mode. This runs the MLIR bridge in a FallbackEnabled mode when an analysis of the graph determines that the graph does not have unsupported features.

  • Deterministic Op Functionality:

    • Add determinism-unimplemented exception-throwing to the segment-sum ops. When the environment variable TF_DETERMINISTIC_OPS is set to "true" or "1" (when op-determinism is expected), an attempt to run the following ops on a GPU will throw tf.errors.UnimplementedError (with an understandable message) when data is a floating-point type, including complex types (if supported): tf.math.segment_prod, tf.math.segment_sum, tf.math.unsorted_segment_mean, tf.math.unsorted_segment_sqrt_n, tf.math.unsorted_segment_prod, tf.math.unsorted_segment_sum, and therefore also tf.convert_to_tensor when value is of type tf.IndexedSlices (such as in the backprop though tf.gather into a dense embedding). See issue 39751 which this change addresses, but does not solve. This exception-throwing behavior can be disabled by setting the environment variable TF_DISABLE_SEGMENT_REDUCTION_OP_DETERMINISM_EXCEPTIONS to "true" or "1". For more information about these changes, see the description in pull request47772.
    • In previous versions of TensorFlow, when a GPU was available,tf.sparse.sparse_dense_matmul introduced truly random noise in the forward path for data of type tf.float32 but not for data of type tf.float64 (for which there was no GPU implementation). In this current release, GPU support for other floating-point types (tf.float16, tf.float64, tf.complex64, and tf.complex128) has been added for this op. If you were relying on the determinism of the tf.float64 CPU implementation being automatically selected because of the absence of the tf.float64 GPU implementation, you with either need to force the op to run on the CPU or use a different data type.

  • Other

    • Added show_debug_info to mlir.convert_graph_def and mlir.convert_function.
    • Added Arm Compute Library (ACL) support to --config=mkl_aarch64 build.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Aaron S. Mondal, Abhilash Mahendrakar, Abhinav Upadhyay, Abhishek Kulkarni, Abolfazl Shahbazi, Adam Hillier, Aditya Kane, Ag Ramesh, ahmedsabie, Albert Villanova Del Moral, Aleksey Vitebskiy, Alex Hoffman, Alexander Bayandin, Alfie Edwards, Aman Kishore, Amogh Joshi, andreABbauer, Andrew Goodbody, Andrzej Pomirski, Artemiy Ryabinkov, Ashish Jha, ather, Ayan Moitra, Bairen Yi, Bart Ribbers, Bas Aarts, Behzad Abghari, Ben Arnao, Ben Barsdell, Benjamin Klimczak, bhack, Brendan Collins, Can Wang, Cheng Ren, Chris Leary, Chris Olivier, Clemens Giuliani, Cloud Han, Corey Cole, Cui, Yifeng, Cuong V. Nguyen, Daniel Moore, Dawid Wojciechowski, Ddavis-2015, Dean Wyatte, Denisa Roberts, dependabot[bot], Dmitry Volodin, Dominic Jack, Duncan Riach, dushuai, Elena Zhelezina, Eli Osherovich, Erik Smistad, ewsn1593, Felix Fent, fo40225, François Chollet, Frederic Bastien, Freedom" Koan-Sin Tan, fsx950223, ganand1, gbaned, Georgiy Manuilov, gerbauz, Guillaume Klein, Guozhong Zhuang, Harry Slatyer, Harsh188, henri, Henri Woodcock, Hiran Sarkar, Hollow Man, Håkon Sandsmark, I Wayan Dharmana, icysapphire, Ikko Ashimine, Jab Hofmeier, Jack Hessel, Jacob Valdez, Jakub Jatczak, James Bernardi, Jared Smolens, Jason Zaman, jedlimlx, Jenny Plunkett, Jens Elofsson, Jerry Shih, jgehw, Jia Fu Low, Jim Fisher, jpodivin, Julien Stephan, Jungsub Lim, Junha Park, Junhyuk So, justkw, Kaixi Hou, kashyapraval, Kasra Bigdeli, Kazuaki Ishizaki, Keith Mok, Kevin Cheng, kopytjuk, Kristian Hartikainen, ksood12345, Kulin Seth, kushanam, latyas, Lequn Chen, Leslie-Fang, Long M. Lưu, Lukas Geiger, machineko, Mahmoud Abuzaina, Manish, Mao Yunfei, Maozhou, Ge, Marcin Juszkiewicz, Marcin Owsiany, Marconi Jiang, Marcos Pereira, Maria Romanenko Vexlard, Maria Vexlard, Marius Brehler, marload, Martin Kubovčík, Matej, Mateusz Holenko, Maxiwell S. Garcia, Mazhar, mazharul, mbhuiyan, mdfaijul, Michael Gielda, Michael Kuchnik, Michal Szutenberg, Mikhail Stepanov, Milan Straka, Mitchel Humpherys, Mohamed Moselhy, Mohamed Nour Abouelseoud, Måns Bermell, Måns Nilsson, Nathan Luehr, Nico Jahn, Niroop Ammbashankar, Oceania2018, Omri Steiner, Orivej Desh, Oskar Flordal, oujiafan, Patrik Laurell, Paul B. Isaac'S, Paul Klinger, Pawel Piskorski, Pedro Marques, Phat Tran, Piotr Zierhoffer, piyushdatta, Pnikam-Cad, Prashant Kumar, Prateek Gupta, PratsBhatt, Pravin Karandikar, qqq.jq, QQ喵, Quintin, Rama Ketineni, ravikyram, Rehan Guha, rhdong, rmothukuru, Roger Cheng, Rohit Santhanam, rposts, Rsanthanam-Amd, rsun, Rsun-Bdti, Ryan Kuester, ryanking13, Saduf2019, Sami Kama, Samuel Marks, Scott Tseng, Sean Moriarity, Sergey Popov, Sergii Khomenko, Sheng, Yang, shwetaoj, Sidong-Wei, Simon Maurer, Simrit Kaur, Srini511, Srinivasan Narayanamoorthy, Stephan, Stephen Matthews, Sungmann Cho, Sunoru, Suraj Sudhir, Suraj Upadhyay, Taebum Kim, Takayoshi Koizumi, Tamas Bela Feher, Teng Lu, Thibaut Goetghebuer-Planchon, Tomwildenhain-Microsoft, Tony, Traun Leyden, Trent Lo, TVLIgnacy, Tzu-Wei Sung, vaibhav, Vignesh Kothapalli, Vikram Dattu, viktprog, Vinayaka Bandishti, Vincent Abriou, Vishakha Agrawal, Vivek Panyam, Vladimir Silyaev, Võ Văn Nghĩa, wamuir, Wang, Yanzhang, wangsiyu, Waqar Hameed, wxinix, Xiao Yang, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yair Ehrenwald, Yajush Vyas, Yasir Modak, Yimei Sun, Yong Tang, Yosshi999, youshenmebutuo, yqtianust, Yuan Tang, yuanbopeng, Yuriy Chernyshov, Yuta Fukasawa, Zachary Deane-Mayer, Zeno Gantner, Zhoulong Jiang, zhuyie, zilinzhu, 彭震东

- C++
Published by tensorflow-jenkins almost 5 years ago

tensorflow - TensorFlow 2.5.0-rc2

Release 2.5.0

Major Features and Improvements

  • Support for Python3.9 has been added.
  • TPU embedding support
    • Added profile_data_directory to EmbeddingConfigSpec in_tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.
  • tf.keras.metrics.AUC now support logit predictions.
  • Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.
  • tf.data:
    • tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.
    • tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).
    • tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.
    • Options returned by tf.data.Dataset.options() are no longer mutable.
    • tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().
  • tf.lite
    • Enabled the new MLIR-based quantization backend by default
      • The new backend is used for 8 bits full integer post-training quantization
      • The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)
      • Set experimental_new_quantizer in tf.lite.TFLiteConverter to False to disable this change
  • tf.keras
    • Enabled a new supported input type in Model.fit, tf.keras.utils.experimental.DatasetCreator, which takes a callable, dataset_fn. DatasetCreator is intended to work across all tf.distribute strategies, and is the only input type supported for Parameter Server strategy.
  • tf.distribute
    • tf.distribute.experimental.ParameterServerStrategy now supports training with Keras Model.fit when used with DatasetCreator.
  • PluggableDevice
  • oneAPI Deep Neural Network Library (oneDNN) CPU performance optimizations from Intel-optimized TensorFlow are now available in the official x86-64 Linux and Windows builds.
    • They are off by default. Enable them by setting the environment variable TF_ENABLE_ONEDNN_OPTS=1.
    • We do not recommend using them in GPU systems, as they have not been sufficiently tested with GPUs yet.
  • TensorFlow pip packages are now built with CUDA11.2 and cuDNN 8.1.0

Breaking Changes

  • The TF_CPP_MIN_VLOG_LEVEL environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL which correctly describes its effect.

Bug Fixes and Other Changes

  • tf.keras:
    • Preprocessing layers API consistency changes:
      • StringLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization.
      • IntegerLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization. Renamed max_values, oov_value and mask_value to max_tokens, oov_token and mask_token to align with StringLookup and TextVectorization.
      • TextVectorization default for pad_to_max_tokens switched to False.
      • CategoryEncoding no longer supports adapt, IntegerLookup now supports equivalent functionality. max_tokens argument renamed to num_tokens.
      • Discretization added num_bins argument for learning bins boundaries through calling adapt on a dataset. Renamed bins argument to bin_boundaries for specifying bins without adapt.
    • Improvements to model saving/loading:
      • model.load_weights now accepts paths to saved models.
    • Keras inputs can now be created directly from arbitrary tf.TypeSpecs.
    • Two new learning rate schedules added: tf.keras.optimizers.schedules.CosineDecay and tf.keras.optimizers.schedules.CosineDecayRestarts.

  • tf.data:

    • Exposing tf.data.experimental.ExternalStatePolicy, which can be used to control how external state should be handled during dataset serialization or iterator checkpointing.
    • Changing tf.data.experimental.save to store the type specification of the dataset elements. This avoids the need for explicitly specifying the element_spec argument of tf.data.experimental.load when loading the previously saved dataset.
    • Add .element_spec property to tf.data.DatasetSpec to access the inner spec. This can be used to extract the structure of nested datasets.
    • Add tf.data.experimental.AutoShardingPolicy.HINT which can be used to provide hints to tf.distribute-based auto-sharding as to where in the input pipeline to insert sharding transformations.
    • Make tf.data.Options persistent across tf.function and GraphDef boundaries.

  • XLA compilation:

    • tf.function(experimental_compile=True) has become a stable API, renamed tf.function(jit_compile=True).
    • XLA can now compile MirroredStrategy: the step function passed to strategy.run can now be annoted with jit_compile=True.

  • tf.distribute:

    • Rename experimental_prefetch_to_device in tf.distribute.InputOptions to experimental_fetch_to_device to better reflect the purpose.

  • tf.lite:

    • class tflite::Subgraph:
      • Removed the tensors() method and the non-const overload of the nodes_and_registration() method, both of which were previously documented as temporary and to be removed.
        • Uses of tensors() can be replaced by calling the existing methods tensors_size() and tensor(int).
        • Uses of the non-const overload of nodes_and_registration can be replaced by calling the existing methods nodes_size() and context(), and then calling the GetNodeAndRegistration method in the TfLiteContext returned by context().
    • NNAPI
      • Removed deprecated Interpreter::UseNNAPI(bool) C++ API.
        • Use NnApiDelegate() and related delegate configuration methods directly.
      • Replaced the model cache key for models computation algorithm with one guaranteed to be stable across runs.
    • 16 bits quantization
      • Added int16x8 support for ABS, REDUCEMAX and REDUCEMIN operators.
      • Additional tests and fixes for ADD and SUB operators.
    • Added support for saved model's session initializer through TFLiteConverter.from_saved_model.
    • Added DEPTHTOSPACE support in Post training quantization.
    • Added dynamic range quantization support for the BatchMatMul op.
      • Both symmetric and asymmetric quantized input tensor are supported.
    • Add RFFT2D as builtin op. (RFFT2D also supports RFFTD.) Currently only supports float32 input.
    • Add 5D support to SLICE op.
    • TFLite Supports SingatureDef:
      • TFLiteConverter exports models with SignatureDef
      • Interpreter supports getting a list of signatures and getting callable function for a given signaturedef.
    • Add int8 support for ReshapeV2.
    • Add experimental support for optimization with sparsity.
    • Add nominal support for unsigned 32-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for static hash tables through TFLiteConverter.from_saved_model.
    • The Python TF Lite Interpreter bindings now has an option experimental_preserve_all_tensors to aid in debugging conversion.
    • Quantized x86 execution defaults to Ruy GEMM library for platforms with AVX support.
    • Deprecate tf.compat.v1.lite.experimental.get_potentially_supported_ops. Use tf.lite.TFLiteConverter directly to check whether a model is convertible.
    • Add support to select one of three different built-in op resolvers to be
    • Enabled post training with calibrations for models that require user provied TensorFlow Lite custom op libraries via converter.target_spec._experimental_custom_op_registerers. used in Python Interpreter API.

  • TF Core:

    • Corrected higher-order gradients of control flow constructs (tf.cond, tf.while_loop, and compositions like tf.foldl) computed with tf.GradientTape inside a tf.function.
    • Changed the default step size in gradient_checker_v2.compute_gradients to be exactly representable as a binary floating point numbers. This avoids poluting gradient approximations needlessly, which is some cases leads to false negatives in op gradient tests.
    • Added tf.config.experimental.get_memory_info, returning a dict with the current and peak memory usage. Deprecated tf.config.experimental.get_memory_usage in favor of this new function.
    • Extended tf.config.experimental.enable_tensor_float_32_execution to control Tensor-Float-32 evaluation in RNNs.
    • Added a 'experimental_payloads' field to tf.errors.OpError and its subclasses to support more detailed error reporting.This is inspired from Abseil Status payloads: https://github.com/abseil/abseil-cpp/blob/master/absl/status/status.h

  • tf.summary:

    • New tf.summary.graph allows manual write of TensorFlow graph (tf.Graph or tf.compat.v1.GraphDef) as a summary. This is not a replacement for the trace-based API.

  • Set /d2ReducedOptimizeHugeFunctions by default for Windows builds. This provides a big compile-time speedup, and effectively raises the minimum supported MSVC version to 16.4 (current: 16.8).

    • See: https://groups.google.com/a/tensorflow.org/d/topic/build/SsW98Eo7l3o/discussion

  • TensorRT

    • Removed the deprecated session_config parameter for the TF1-TRT converter TrtGraphConverter. Previously, we issued a warning when the value of the parameter is not None.
    • The TF2-TRT converter TrtGraphConverterV2 takes an object of class TrtConversionParams as a parameter. Removed three deprecated fields from this class: rewriter_config_template, is_dynamic_op, and max_batch_size. Previously, we issued a warning when the value of rewriter_config_template is not None. We issued an error when the value of is_dynamic_op is not True. We didn't use the value for max_batch_size for building TensorRT engines. Add parameters use_dynamic_shape to enable dynamic shape support. The default is to disable dynamic shape support. Add dynamic_shape_profile_strategy for selecting a dynamic shape profile strategy. The default is profile strategy is Range.
    • Issue a warning when function gettensorrtrewriter_config is used.

  • TF XLA

    • Add new enum value MLIR_BRIDGE_ROLLOUT_SAFE_MODE_ENABLED to tf.config.experimental.mlir_bridge_rollout to enable a \"safe\" mode. This runs the MLIR bridge only when an analysis of the graph only when an analysis of the graph determines that it is safe to run.
    • Add new enum value 'MLIRBRIDGEROLLOUTSAFEMODEFALLBACKENABLED' to tf.config.experimental.mlir_bridge_rollout to enable a fallback for the MLIR bridge in a \"safe\" mode. This runs the MLIR bridge in a FallbackEnabled mode when an analysis of the graph determines that the graph does not have unsupported features.

  • Deterministic Op Functionality:

    • Add determinism-unimplemented exception-throwing to the segment-sum ops. When the environment variable TF_DETERMINISTIC_OPS is set to "true" or "1" (when op-determinism is expected), an attempt to run the following ops on a GPU will throw tf.errors.UnimplementedError (with an understandable message) when data is a floating-point type, including complex types (if supported): tf.math.segment_prod, tf.math.segment_sum, tf.math.unsorted_segment_mean, tf.math.unsorted_segment_sqrt_n, tf.math.unsorted_segment_prod, tf.math.unsorted_segment_sum, and therefore also tf.convert_to_tensor when value is of type tf.IndexedSlices (such as in the backprop though tf.gather into a dense embedding). See issue 39751 which this change addresses, but does not solve. This exception-throwing behavior can be disabled by setting the environment variable TF_DISABLE_SEGMENT_REDUCTION_OP_DETERMINISM_EXCEPTIONS to "true" or "1". For more information about these changes, see the description in pull request 47772.
    • In previous versions of TensorFlow, when a GPU was available, tf.sparse.sparse_dense_matmul introduced truly random noise in the forward path for data of type tf.float32 but not for data of type tf.float64 (for which there was no GPU implementation). In this current release, GPU support for other floating-point types (tf.float16, tf.float64, tf.complex64, and tf.complex128) has been added for this op. If you were relying on the determinism of the tf.float64 CPU implementation being automatically selected because of the absence of the tf.float64 GPU implementation, you with either need to force the op to run on the CPU or use a different data type.

  • Other

    • Added show_debug_info to mlir.convert_graph_def and mlir.convert_function.
    • Added Arm Compute Library (ACL) support to --config=mkl_aarch64 build.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Aaron S. Mondal, Abhilash Mahendrakar, Abhinav Upadhyay, Abhishek Kulkarni, Abolfazl Shahbazi, Adam Hillier, Aditya Kane, Ag Ramesh, ahmedsabie, Albert Villanova Del Moral, Aleksey Vitebskiy, Alex Hoffman, Alexander Bayandin, Alfie Edwards, Aman Kishore, Amogh Joshi, andreABbauer, Andrew Goodbody, Andrzej Pomirski, Artemiy Ryabinkov, Ashish Jha, ather, Ayan Moitra, Bairen Yi, Bart Ribbers, Bas Aarts, Behzad Abghari, Ben Arnao, Ben Barsdell, Benjamin Klimczak, bhack, Brendan Collins, Can Wang, Cheng Ren, Chris Leary, Chris Olivier, Clemens Giuliani, Cloud Han, Corey Cole, Cui, Yifeng, Cuong V. Nguyen, Daniel Moore, Dawid Wojciechowski, Ddavis-2015, Dean Wyatte, Denisa Roberts, dependabot[bot], Dmitry Volodin, Dominic Jack, Duncan Riach, dushuai, Elena Zhelezina, Eli Osherovich, Erik Smistad, ewsn1593, Felix Fent, fo40225, François Chollet, Frederic Bastien, Freedom" Koan-Sin Tan, fsx950223, ganand1, gbaned, Georgiy Manuilov, gerbauz, Guillaume Klein, Guozhong Zhuang, Harry Slatyer, Harsh188, henri, Henri Woodcock, Hiran Sarkar, Hollow Man, Håkon Sandsmark, I Wayan Dharmana, icysapphire, Ikko Ashimine, Jab Hofmeier, Jack Hessel, Jacob Valdez, Jakub Jatczak, James Bernardi, Jared Smolens, Jason Zaman, jedlimlx, Jenny Plunkett, Jens Elofsson, Jerry Shih, jgehw, Jia Fu Low, Jim Fisher, jpodivin, Julien Stephan, Jungsub Lim, Junha Park, Junhyuk So, justkw, Kaixi Hou, kashyapraval, Kasra Bigdeli, Kazuaki Ishizaki, Keith Mok, Kevin Cheng, kopytjuk, Kristian Hartikainen, ksood12345, Kulin Seth, kushanam, latyas, Lequn Chen, Leslie-Fang, Long M. Lưu, Lukas Geiger, machineko, Mahmoud Abuzaina, Manish, Mao Yunfei, Maozhou, Ge, Marcin Juszkiewicz, Marcin Owsiany, Marconi Jiang, Marcos Pereira, Maria Romanenko Vexlard, Maria Vexlard, Marius Brehler, marload, Martin Kubovčík, Matej, Mateusz Holenko, Maxiwell S. Garcia, Mazhar, mazharul, mbhuiyan, mdfaijul, Michael Gielda, Michael Kuchnik, Michal Szutenberg, Mikhail Stepanov, Milan Straka, Mitchel Humpherys, Mohamed Moselhy, Mohamed Nour Abouelseoud, Måns Bermell, Måns Nilsson, Nathan Luehr, Nico Jahn, Niroop Ammbashankar, Oceania2018, Omri Steiner, Orivej Desh, Oskar Flordal, oujiafan, Patrik Laurell, Paul B. Isaac'S, Paul Klinger, Pawel Piskorski, Pedro Marques, Phat Tran, Piotr Zierhoffer, piyushdatta, Pnikam-Cad, Prashant Kumar, Prateek Gupta, PratsBhatt, Pravin Karandikar, qqq.jq, QQ喵, Quintin, Rama Ketineni, ravikyram, Rehan Guha, rhdong, rmothukuru, Roger Cheng, Rohit Santhanam, rposts, Rsanthanam-Amd, rsun, Rsun-Bdti, Ryan Kuester, ryanking13, Saduf2019, Sami Kama, Samuel Marks, Scott Tseng, Sean Moriarity, Sergey Popov, Sergii Khomenko, Sheng, Yang, shwetaoj, Sidong-Wei, Simon Maurer, Simrit Kaur, Srini511, Srinivasan Narayanamoorthy, Stephan, Stephen Matthews, Sungmann Cho, Sunoru, Suraj Sudhir, Suraj Upadhyay, Taebum Kim, Takayoshi Koizumi, Tamas Bela Feher, Teng Lu, Thibaut Goetghebuer-Planchon, Tomwildenhain-Microsoft, Tony, Traun Leyden, Trent Lo, TVLIgnacy, Tzu-Wei Sung, vaibhav, Vignesh Kothapalli, Vikram Dattu, viktprog, Vinayaka Bandishti, Vincent Abriou, Vishakha Agrawal, Vivek Panyam, Vladimir Silyaev, Võ Văn Nghĩa, wamuir, Wang, Yanzhang, wangsiyu, Waqar Hameed, wxinix, Xiao Yang, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yair Ehrenwald, Yajush Vyas, Yasir Modak, Yimei Sun, Yong Tang, Yosshi999, youshenmebutuo, yqtianust, Yuan Tang, yuanbopeng, Yuriy Chernyshov, Yuta Fukasawa, Zachary Deane-Mayer, Zeno Gantner, Zhoulong Jiang, zhuyie, zilinzhu, 彭震东

- C++
Published by tensorflow-jenkins almost 5 years ago

tensorflow - TensorFlow 2.5.0-rc1

Release 2.5.0

Major Features and Improvements

  • Support for Python3.9 has been added.
  • TPU embedding support
    • Added profile_data_directory to EmbeddingConfigSpec in _tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.
  • tf.keras.metrics.AUC now support logit predictions.
  • Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.
  • tf.data:
    • tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.
    • tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).
    • tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.
    • Options returned by tf.data.Dataset.options() are no longer mutable.
    • tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().
  • tf.lite
    • Enabled the new MLIR-based quantization backend by default
      • The new backend is used for 8 bits full integer post-training quantization
      • The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)
      • Set experimental_new_quantizer in tf.lite.TFLiteConverter to False to disable this change
  • tf.keras
    • Enabled a new supported input type in Model.fit, tf.keras.utils.experimental.DatasetCreator, which takes a callable, dataset_fn. DatasetCreator is intended to work across all tf.distribute strategies, and is the only input type supported for Parameter Server strategy.
  • tf.distribute
    • tf.distribute.experimental.ParameterServerStrategy now supports training with Keras Model.fit when used with DatasetCreator.
  • PluggableDevice

  • oneAPI Deep Neural Network Library (oneDNN) CPU performance optimizations from Intel-optimized TensorFlow are now available in the official x86-64 Linux and Windows builds.

    • They are off by default. Enable them by setting the environment variable TF_ENABLE_ONEDNN_OPTS=1.
    • We do not recommend using them in GPU systems, as they have not been sufficiently tested with GPUs yet.

  • TensorFlow pip packages are now built with CUDA11.2 and cuDNN 8.1.0

Breaking Changes

  • The TF_CPP_MIN_VLOG_LEVEL environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL which correctly describes its effect.

Bug Fixes and Other Changes

  • tf.keras:

    • Preprocessing layers API consistency changes:
      • StringLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization.
      • IntegerLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization. Renamed max_values, oov_value and mask_value to max_tokens, oov_token and mask_token to align with StringLookup and TextVectorization.
      • TextVectorization default for pad_to_max_tokens switched to False.
      • CategoryEncoding no longer supports adapt, IntegerLookup now supports equivalent functionality. max_tokens argument renamed to num_tokens.
      • Discretization added num_bins argument for learning bins boundaries through calling adapt on a dataset. Renamed bins argument to bin_boundaries for specifying bins without adapt.
    • Improvements to model saving/loading:
      • model.load_weights now accepts paths to saved models.
    • Keras inputs can now be created directly from arbitrary tf.TypeSpecs.
    • Two new learning rate schedules added: tf.keras.optimizers.schedules.CosineDecay and tf.keras.optimizers.schedules.CosineDecayRestarts.

  • tf.data:

    • Exposing tf.data.experimental.ExternalStatePolicy, which can be used to control how external state should be handled during dataset serialization or iterator checkpointing.
    • Changing tf.data.experimental.save to store the type specification of the dataset elements. This avoids the need for explicitly specifying the element_spec argument of tf.data.experimental.load when loading the previously saved dataset.
    • Add .element_spec property to tf.data.DatasetSpec to access the inner spec. This can be used to extract the structure of nested datasets.
    • Add tf.data.experimental.AutoShardingPolicy.HINT which can be used to provide hints to tf.distribute-based auto-sharding as to where in the input pipeline to insert sharding transformations.
    • Make tf.data.Options persistent across tf.function and GraphDef boundaries.

  • XLA compilation:

    • tf.function(experimental_compile=True) has become a stable API, renamed tf.function(jit_compile=True).
    • XLA can now compile MirroredStrategy: the step function passed to strategy.run can now be annoted with jit_compile=True.

  • tf.distribute:

    • Rename experimental_prefetch_to_device in tf.distribute.InputOptions to experimental_fetch_to_device to better reflect the purpose.

  • tf.lite:

    • class tflite::Subgraph:
      • Removed the tensors() method and the non-const overload of the nodes_and_registration() method, both of which were previously documented as temporary and to be removed.
        • Uses of tensors() can be replaced by calling the existing methods tensors_size() and tensor(int).
        • Uses of the non-const overload of nodes_and_registration can be replaced by calling the existing methods nodes_size() and context(), and then calling the GetNodeAndRegistration method in the TfLiteContext returned by context().
    • NNAPI
      • Removed deprecated Interpreter::UseNNAPI(bool) C++ API.
        • Use NnApiDelegate() and related delegate configuration methods directly.
      • Replaced the model cache key for models computation algorithm with one guaranteed to be stable across runs.
    • 16 bits quantization
      • Added int16x8 support for ABS, REDUCEMAX and REDUCEMIN operators.
      • Additional tests and fixes for ADD and SUB operators.
    • Added support for saved model's session initializer through TFLiteConverter.from_saved_model.
    • Added DEPTHTOSPACE support in Post training quantization.
    • Added dynamic range quantization support for the BatchMatMul op.
      • Both symmetric and asymmetric quantized input tensor are supported.
    • Add RFFT2D as builtin op. (RFFT2D also supports RFFTD.) Currently only supports float32 input.
    • Add 5D support to SLICE op.
    • TFLite Supports SingatureDef:
      • TFLiteConverter exports models with SignatureDef
      • Interpreter supports getting a list of signatures and getting callable function for a given signaturedef.
    • Add int8 support for ReshapeV2.
    • Add experimental support for optimization with sparsity.
    • Add nominal support for unsigned 32-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for static hash tables through TFLiteConverter.from_saved_model.
    • The Python TF Lite Interpreter bindings now has an option experimental_preserve_all_tensors to aid in debugging conversion.
    • Quantized x86 execution defaults to Ruy GEMM library for platforms with AVX support.
    • Deprecate tf.compat.v1.lite.experimental.get_potentially_supported_ops. Use tf.lite.TFLiteConverter directly to check whether a model is convertible.
    • Add support to select one of three different built-in op resolvers to be
    • Enabled post training with calibrations for models that require user provied TensorFlow Lite custom op libraries via converter.target_spec._experimental_custom_op_registerers. used in Python Interpreter API.

  • TF Core:

    • Corrected higher-order gradients of control flow constructs (tf.cond, tf.while_loop, and compositions like tf.foldl) computed with tf.GradientTape inside a tf.function.
    • Changed the default step size in gradient_checker_v2.compute_gradients to be exactly representable as a binary floating point numbers. This avoids poluting gradient approximations needlessly, which is some cases leads to false negatives in op gradient tests.
    • Added tf.config.experimental.get_memory_info, returning a dict with the current and peak memory usage. Deprecated tf.config.experimental.get_memory_usage in favor of this new function.
    • Extended tf.config.experimental.enable_tensor_float_32_execution to control Tensor-Float-32 evaluation in RNNs.
    • Added a 'experimental_payloads' field to tf.errors.OpError and its subclasses to support more detailed error reporting. This is inspired from Abseil Status payloads: https://github.com/abseil/abseil-cpp/blob/master/absl/status/status.h

  • tf.summary:

    • New tf.summary.graph allows manual write of TensorFlow graph (tf.Graph or tf.compat.v1.GraphDef) as a summary. This is not a replacement for the trace-based API.

  • Set /d2ReducedOptimizeHugeFunctions by default for Windows builds. This provides a big compile-time speedup, and effectively raises the minimum supported MSVC version to 16.4 (current: 16.8).

    • See: https://groups.google.com/a/tensorflow.org/d/topic/build/SsW98Eo7l3o/discussion

  • TensorRT

    • Removed the deprecated session_config parameter for the TF1-TRT converter TrtGraphConverter. Previously, we issued a warning when the value of the parameter is not None.
    • The TF2-TRT converter TrtGraphConverterV2 takes an object of class TrtConversionParams as a parameter. Removed three deprecated fields from this class: rewriter_config_template, is_dynamic_op, and max_batch_size. Previously, we issued a warning when the value of rewriter_config_template is not None. We issued an error when the value of is_dynamic_op is not True. We didn't use the value for max_batch_size for building TensorRT engines. Add parameters use_dynamic_shape to enable dynamic shape support. The default is to disable dynamic shape support. Add dynamic_shape_profile_strategy for selecting a dynamic shape profile strategy. The default is profile strategy is Range.
    • Issue a warning when function gettensorrtrewriter_config is used.

  • TF XLA

    • Add new enum value MLIR_BRIDGE_ROLLOUT_SAFE_MODE_ENABLED to tf.config.experimental.mlir_bridge_rollout to enable a \"safe\" mode. This runs the MLIR bridge only when an analysis of the graph only when an analysis of the graph determines that it is safe to run.
    • Add new enum value 'MLIRBRIDGEROLLOUTSAFEMODEFALLBACKENABLED' to tf.config.experimental.mlir_bridge_rollout to enable a fallback for the MLIR bridge in a \"safe\" mode. This runs the MLIR bridge in a FallbackEnabled mode when an analysis of the graph determines that the graph does not have unsupported features.

  • Other

    • Added show_debug_info to mlir.convert_graph_def and mlir.convert_function.
    • Added Arm Compute Library (ACL) support to --config=mkl_aarch64 build.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Aaron S. Mondal, Abhilash Mahendrakar, Abhinav Upadhyay, Abhishek Kulkarni, Abolfazl Shahbazi, Adam Hillier, Aditya Kane, Ag Ramesh, ahmedsabie, Albert Villanova Del Moral, Aleksey Vitebskiy, Alex Hoffman, Alexander Bayandin, Alfie Edwards, Aman Kishore, Amogh Joshi, andreABbauer, Andrew Goodbody, Andrzej Pomirski, Artemiy Ryabinkov, Ashish Jha, ather, Ayan Moitra, Bairen Yi, Bart Ribbers, Bas Aarts, Behzad Abghari, Ben Arnao, Ben Barsdell, Benjamin Klimczak, bhack, Brendan Collins, Can Wang, Cheng Ren, Chris Leary, Chris Olivier, Clemens Giuliani, Cloud Han, Corey Cole, Cui, Yifeng, Cuong V. Nguyen, Daniel Moore, Dawid Wojciechowski, Ddavis-2015, Dean Wyatte, Denisa Roberts, dependabot[bot], Dmitry Volodin, Dominic Jack, Duncan Riach, dushuai, Elena Zhelezina, Eli Osherovich, Erik Smistad, ewsn1593, Felix Fent, fo40225, François Chollet, Frederic Bastien, Freedom" Koan-Sin Tan, fsx950223, ganand1, gbaned, Georgiy Manuilov, gerbauz, Guillaume Klein, Guozhong Zhuang, Harry Slatyer, Harsh188, henri, Henri Woodcock, Hiran Sarkar, Hollow Man, Håkon Sandsmark, I Wayan Dharmana, icysapphire, Ikko Ashimine, Jab Hofmeier, Jack Hessel, Jacob Valdez, Jakub Jatczak, James Bernardi, Jared Smolens, Jason Zaman, jedlimlx, Jenny Plunkett, Jens Elofsson, Jerry Shih, jgehw, Jia Fu Low, Jim Fisher, jpodivin, Julien Stephan, Jungsub Lim, Junha Park, Junhyuk So, justkw, Kaixi Hou, kashyapraval, Kasra Bigdeli, Kazuaki Ishizaki, Keith Mok, Kevin Cheng, kopytjuk, Kristian Hartikainen, ksood12345, Kulin Seth, kushanam, latyas, Lequn Chen, Leslie-Fang, Long M. Lưu, Lukas Geiger, machineko, Mahmoud Abuzaina, Manish, Mao Yunfei, Maozhou, Ge, Marcin Juszkiewicz, Marcin Owsiany, Marconi Jiang, Marcos Pereira, Maria Romanenko Vexlard, Maria Vexlard, Marius Brehler, marload, Martin Kubovčík, Matej, Mateusz Holenko, Maxiwell S. Garcia, Mazhar, mazharul, mbhuiyan, mdfaijul, Michael Gielda, Michael Kuchnik, Michal Szutenberg, Mikhail Stepanov, Milan Straka, Mitchel Humpherys, Mohamed Moselhy, Mohamed Nour Abouelseoud, Måns Bermell, Måns Nilsson, Nathan Luehr, Nico Jahn, Niroop Ammbashankar, Oceania2018, Omri Steiner, Orivej Desh, Oskar Flordal, oujiafan, Patrik Laurell, Paul B. Isaac'S, Paul Klinger, Pawel Piskorski, Pedro Marques, Phat Tran, Piotr Zierhoffer, piyushdatta, Pnikam-Cad, Prashant Kumar, Prateek Gupta, PratsBhatt, Pravin Karandikar, qqq.jq, QQ喵, Quintin, Rama Ketineni, ravikyram, Rehan Guha, rhdong, rmothukuru, Roger Cheng, Rohit Santhanam, rposts, Rsanthanam-Amd, rsun, Rsun-Bdti, Ryan Kuester, ryanking13, Saduf2019, Sami Kama, Samuel Marks, Scott Tseng, Sean Moriarity, Sergey Popov, Sergii Khomenko, Sheng, Yang, shwetaoj, Sidong-Wei, Simon Maurer, Simrit Kaur, Srini511, Srinivasan Narayanamoorthy, Stephan, Stephen Matthews, Sungmann Cho, Sunoru, Suraj Sudhir, Suraj Upadhyay, Taebum Kim, Takayoshi Koizumi, Tamas Bela Feher, Teng Lu, Thibaut Goetghebuer-Planchon, Tomwildenhain-Microsoft, Tony, Traun Leyden, Trent Lo, TVLIgnacy, Tzu-Wei Sung, vaibhav, Vignesh Kothapalli, Vikram Dattu, viktprog, Vinayaka Bandishti, Vincent Abriou, Vishakha Agrawal, Vivek Panyam, Vladimir Silyaev, Võ Văn Nghĩa, wamuir, Wang, Yanzhang, wangsiyu, Waqar Hameed, wxinix, Xiao Yang, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yair Ehrenwald, Yajush Vyas, Yasir Modak, Yimei Sun, Yong Tang, Yosshi999, youshenmebutuo, yqtianust, Yuan Tang, yuanbopeng, Yuriy Chernyshov, Yuta Fukasawa, Zachary Deane-Mayer, Zeno Gantner, Zhoulong Jiang, zhuyie, zilinzhu, 彭震东

- C++
Published by tensorflow-jenkins almost 5 years ago

tensorflow - TensorFlow 2.5.0-rc0

Release 2.5.0

Major Features and Improvements

  • TPU embedding support
    • Added profile_data_directory to EmbeddingConfigSpec in _tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.
  • tf.keras.metrics.AUC now support logit predictions.
  • Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.
  • tf.data:
    • tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.
    • tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).
    • tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.
    • Options returned by tf.data.Dataset.options() are no longer mutable.
    • tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().
  • tf.lite
    • Enabled the new MLIR-based quantization backend by default
      • The new backend is used for 8 bits full integer post-training quantization
      • The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)
      • Set experimental_new_quantizer in tf.lite.TFLiteConverter to False to disable this change
  • tf.keras
    • Enabled a new supported input type in Model.fit, tf.keras.utils.experimental.DatasetCreator, which takes a callable, dataset_fn. DatasetCreator is intended to work across all tf.distribute strategies, and is the only input type supported for Parameter Server strategy.
  • tf.distribute
    • tf.distribute.experimental.ParameterServerStrategy now supports training with Keras Model.fit when used with DatasetCreator.
  • PluggableDevice

  • oneAPI Deep Neural Network Library (oneDNN) CPU performance optimizations from Intel-optimized TensorFlow are now available in the official x86-64 Linux and Windows builds.

    • They are off by default. Enable them by setting the environment variable TF_ENABLE_ONEDNN_OPTS=1.
    • We do not recommend using them in GPU systems, as they have not been sufficiently tested with GPUs yet.

  • TensorFlow pip packages are now built with CUDA11.2 and cuDNN 8.1.0

Breaking Changes

  • The TF_CPP_MIN_VLOG_LEVEL environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL which correctly describes its effect.

Bug Fixes and Other Changes

  • tf.keras:

    • Preprocessing layers API consistency changes:
      • StringLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization.
      • IntegerLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization. Renamed max_values, oov_value and mask_value to max_tokens, oov_token and mask_token to align with StringLookup and TextVectorization.
      • TextVectorization default for pad_to_max_tokens switched to False.
      • CategoryEncoding no longer supports adapt, IntegerLookup now supports equivalent functionality. max_tokens argument renamed to num_tokens.
      • Discretization added num_bins argument for learning bins boundaries through calling adapt on a dataset. Renamed bins argument to bin_boundaries for specifying bins without adapt.
    • Improvements to model saving/loading:
      • model.load_weights now accepts paths to saved models.
    • Keras inputs can now be created directly from arbitrary tf.TypeSpecs.
    • Two new learning rate schedules added: tf.keras.optimizers.schedules.CosineDecay and tf.keras.optimizers.schedules.CosineDecayRestarts.

  • tf.data:

    • Exposing tf.data.experimental.ExternalStatePolicy, which can be used to control how external state should be handled during dataset serialization or iterator checkpointing.
    • Changing tf.data.experimental.save to store the type specification of the dataset elements. This avoids the need for explicitly specifying the element_spec argument of tf.data.experimental.load when loading the previously saved dataset.
    • Add .element_spec property to tf.data.DatasetSpec to access the inner spec. This can be used to extract the structure of nested datasets.
    • Add tf.data.experimental.AutoShardingPolicy.HINT which can be used to provide hints to tf.distribute-based auto-sharding as to where in the input pipeline to insert sharding transformations.
    • Make tf.data.Options persistent across tf.function and GraphDef boundaries.

  • XLA compilation:

    • tf.function(experimental_compile=True) has become a stable API, renamed tf.function(jit_compile=True).
    • XLA can now compile MirroredStrategy: the step function passed to strategy.run can now be annoted with jit_compile=True.

  • tf.distribute:

    • Rename experimental_prefetch_to_device in tf.distribute.InputOptions to experimental_fetch_to_device to better reflect the purpose.

  • tf.lite:

    • class tflite::Subgraph:
      • Removed the tensors() method and the non-const overload of the nodes_and_registration() method, both of which were previously documented as temporary and to be removed.
        • Uses of tensors() can be replaced by calling the existing methods tensors_size() and tensor(int).
        • Uses of the non-const overload of nodes_and_registration can be replaced by calling the existing methods nodes_size() and context(), and then calling the GetNodeAndRegistration method in the TfLiteContext returned by context().
    • NNAPI
      • Removed deprecated Interpreter::UseNNAPI(bool) C++ API.
        • Use NnApiDelegate() and related delegate configuration methods directly.
      • Replaced the model cache key for models computation algorithm with one guaranteed to be stable across runs.
    • 16 bits quantization
      • Added int16x8 support for ABS, REDUCEMAX and REDUCEMIN operators.
      • Additional tests and fixes for ADD and SUB operators.
    • Added support for saved model's session initializer through TFLiteConverter.from_saved_model.
    • Added DEPTHTOSPACE support in Post training quantization.
    • Added dynamic range quantization support for the BatchMatMul op.
      • Both symmetric and asymmetric quantized input tensor are supported.
    • Add RFFT2D as builtin op. (RFFT2D also supports RFFTD.) Currently only supports float32 input.
    • Add 5D support to SLICE op.
    • TFLite Supports SingatureDef:
      • TFLiteConverter exports models with SignatureDef
      • Interpreter supports getting a list of signatures and getting callable function for a given signaturedef.
    • Add int8 support for ReshapeV2.
    • Add experimental support for optimization with sparsity.
    • Add nominal support for unsigned 32-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for static hash tables through TFLiteConverter.from_saved_model.
    • The Python TF Lite Interpreter bindings now has an option experimental_preserve_all_tensors to aid in debugging conversion.
    • Quantized x86 execution defaults to Ruy GEMM library for platforms with AVX support.
    • Deprecate tf.compat.v1.lite.experimental.get_potentially_supported_ops. Use tf.lite.TFLiteConverter directly to check whether a model is convertible.
    • Add support to select one of three different built-in op resolvers to be
    • Enabled post training with calibrations for models that require user provied TensorFlow Lite custom op libraries via converter.target_spec._experimental_custom_op_registerers. used in Python Interpreter API.

  • TF Core:

    • Corrected higher-order gradients of control flow constructs (tf.cond, tf.while_loop, and compositions like tf.foldl) computed with tf.GradientTape inside a tf.function.
    • Changed the default step size in gradient_checker_v2.compute_gradients to be exactly representable as a binary floating point numbers. This avoids poluting gradient approximations needlessly, which is some cases leads to false negatives in op gradient tests.
    • Added tf.config.experimental.get_memory_info, returning a dict with the current and peak memory usage. Deprecated tf.config.experimental.get_memory_usage in favor of this new function.
    • Extended tf.config.experimental.enable_tensor_float_32_execution to control Tensor-Float-32 evaluation in RNNs.
    • Added a 'experimental_payloads' field to tf.errors.OpError and its subclasses to support more detailed error reporting. This is inspired from Abseil Status payloads: https://github.com/abseil/abseil-cpp/blob/master/absl/status/status.h

  • tf.summary:

    • New tf.summary.graph allows manual write of TensorFlow graph (tf.Graph or tf.compat.v1.GraphDef) as a summary. This is not a replacement for the trace-based API.

  • Set /d2ReducedOptimizeHugeFunctions by default for Windows builds. This provides a big compile-time speedup, and effectively raises the minimum supported MSVC version to 16.4 (current: 16.8).

    • See: https://groups.google.com/a/tensorflow.org/d/topic/build/SsW98Eo7l3o/discussion

  • TensorRT

    • Removed the deprecated session_config parameter for the TF1-TRT converter TrtGraphConverter. Previously, we issued a warning when the value of the parameter is not None.
    • The TF2-TRT converter TrtGraphConverterV2 takes an object of class TrtConversionParams as a parameter. Removed three deprecated fields from this class: rewriter_config_template, is_dynamic_op, and max_batch_size. Previously, we issued a warning when the value of rewriter_config_template is not None. We issued an error when the value of is_dynamic_op is not True. We didn't use the value for max_batch_size for building TensorRT engines. Add parameters use_dynamic_shape to enable dynamic shape support. The default is to disable dynamic shape support. Add dynamic_shape_profile_strategy for selecting a dynamic shape profile strategy. The default is profile strategy is Range.
    • Issue a warning when function gettensorrtrewriter_config is used.

  • TF XLA

    • Add new enum value MLIR_BRIDGE_ROLLOUT_SAFE_MODE_ENABLED to tf.config.experimental.mlir_bridge_rollout to enable a \"safe\" mode. This runs the MLIR bridge only when an analysis of the graph only when an analysis of the graph determines that it is safe to run.
    • Add new enum value 'MLIRBRIDGEROLLOUTSAFEMODEFALLBACKENABLED' to tf.config.experimental.mlir_bridge_rollout to enable a fallback for the MLIR bridge in a \"safe\" mode. This runs the MLIR bridge in a FallbackEnabled mode when an analysis of the graph determines that the graph does not have unsupported features.

  • Other

    • Added show_debug_info to mlir.convert_graph_def and mlir.convert_function.
    • Added Arm Compute Library (ACL) support to --config=mkl_aarch64 build.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

8bitmp3, Aaron S. Mondal, Abhilash Mahendrakar, Abhinav Upadhyay, Abhishek Kulkarni, Abolfazl Shahbazi, Adam Hillier, Aditya Kane, Ag Ramesh, ahmedsabie, Albert Villanova Del Moral, Aleksey Vitebskiy, Alex Hoffman, Alexander Bayandin, Alfie Edwards, Aman Kishore, Amogh Joshi, andreABbauer, Andrew Goodbody, Andrzej Pomirski, Artemiy Ryabinkov, Ashish Jha, ather, Ayan Moitra, Bairen Yi, Bart Ribbers, Bas Aarts, Behzad Abghari, Ben Arnao, Ben Barsdell, Benjamin Klimczak, bhack, Brendan Collins, Can Wang, Cheng Ren, Chris Leary, Chris Olivier, Clemens Giuliani, Cloud Han, Corey Cole, Cui, Yifeng, Cuong V. Nguyen, Daniel Moore, Dawid Wojciechowski, Ddavis-2015, Dean Wyatte, Denisa Roberts, dependabot[bot], Dmitry Volodin, Dominic Jack, Duncan Riach, dushuai, Elena Zhelezina, Eli Osherovich, Erik Smistad, ewsn1593, Felix Fent, fo40225, François Chollet, Frederic Bastien, Freedom" Koan-Sin Tan, fsx950223, ganand1, gbaned, Georgiy Manuilov, gerbauz, Guillaume Klein, Guozhong Zhuang, Harry Slatyer, Harsh188, henri, Henri Woodcock, Hiran Sarkar, Hollow Man, Håkon Sandsmark, I Wayan Dharmana, icysapphire, Ikko Ashimine, Jab Hofmeier, Jack Hessel, Jacob Valdez, Jakub Jatczak, James Bernardi, Jared Smolens, Jason Zaman, jedlimlx, Jenny Plunkett, Jens Elofsson, Jerry Shih, jgehw, Jia Fu Low, Jim Fisher, jpodivin, Julien Stephan, Jungsub Lim, Junha Park, Junhyuk So, justkw, Kaixi Hou, kashyapraval, Kasra Bigdeli, Kazuaki Ishizaki, Keith Mok, Kevin Cheng, kopytjuk, Kristian Hartikainen, ksood12345, Kulin Seth, kushanam, latyas, Lequn Chen, Leslie-Fang, Long M. Lưu, Lukas Geiger, machineko, Mahmoud Abuzaina, Manish, Mao Yunfei, Maozhou, Ge, Marcin Juszkiewicz, Marcin Owsiany, Marconi Jiang, Marcos Pereira, Maria Romanenko Vexlard, Maria Vexlard, Marius Brehler, marload, Martin Kubovčík, Matej, Mateusz Holenko, Maxiwell S. Garcia, Mazhar, mazharul, mbhuiyan, mdfaijul, Michael Gielda, Michael Kuchnik, Michal Szutenberg, Mikhail Stepanov, Milan Straka, Mitchel Humpherys, Mohamed Moselhy, Mohamed Nour Abouelseoud, Måns Bermell, Måns Nilsson, Nathan Luehr, Nico Jahn, Niroop Ammbashankar, Oceania2018, Omri Steiner, Orivej Desh, Oskar Flordal, oujiafan, Patrik Laurell, Paul B. Isaac'S, Paul Klinger, Pawel Piskorski, Pedro Marques, Phat Tran, Piotr Zierhoffer, piyushdatta, Pnikam-Cad, Prashant Kumar, Prateek Gupta, PratsBhatt, Pravin Karandikar, qqq.jq, QQ喵, Quintin, Rama Ketineni, ravikyram, Rehan Guha, rhdong, rmothukuru, Roger Cheng, Rohit Santhanam, rposts, Rsanthanam-Amd, rsun, Rsun-Bdti, Ryan Kuester, ryanking13, Saduf2019, Sami Kama, Samuel Marks, Scott Tseng, Sean Moriarity, Sergey Popov, Sergii Khomenko, Sheng, Yang, shwetaoj, Sidong-Wei, Simon Maurer, Simrit Kaur, Srini511, Srinivasan Narayanamoorthy, Stephan, Stephen Matthews, Sungmann Cho, Sunoru, Suraj Sudhir, Suraj Upadhyay, Taebum Kim, Takayoshi Koizumi, Tamas Bela Feher, Teng Lu, Thibaut Goetghebuer-Planchon, Tomwildenhain-Microsoft, Tony, Traun Leyden, Trent Lo, TVLIgnacy, Tzu-Wei Sung, vaibhav, Vignesh Kothapalli, Vikram Dattu, viktprog, Vinayaka Bandishti, Vincent Abriou, Vishakha Agrawal, Vivek Panyam, Vladimir Silyaev, Võ Văn Nghĩa, wamuir, Wang, Yanzhang, wangsiyu, Waqar Hameed, wxinix, Xiao Yang, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yair Ehrenwald, Yajush Vyas, Yasir Modak, Yimei Sun, Yong Tang, Yosshi999, youshenmebutuo, yqtianust, Yuan Tang, yuanbopeng, Yuriy Chernyshov, Yuta Fukasawa, Zachary Deane-Mayer, Zeno Gantner, Zhoulong Jiang, zhuyie, zilinzhu, 彭震东

- C++
Published by tensorflow-jenkins almost 5 years ago

tensorflow - TensorFlow 2.4.1

Release 2.4.1

This release removes the AVX2 requirement from TF 2.4.0.

- C++
Published by tensorflow-jenkins about 5 years ago

tensorflow - TensorFlow 2.3.2

Release 2.3.2

Bug Fixes and Other Changes

  • Fixes an access to unitialized memory in Eigen code (CVE-2020-26266)
  • Fixes a security vulnerability caused by lack of validation in tf.raw_ops.DataFormatVecPermute and tf.raw_ops.DataFormatDimMap (CVE-2020-26267)
  • Fixes a vulnerability caused by attempting to write to immutable memory region in tf.raw_ops.ImmutableConst (CVE-2020-26268
  • Fixes a CHECK-fail in LSTM with zero-length input (CVE-2020-26270)
  • Fixes a security vulnerability caused by accessing heap data outside of bounds when loading a specially crafted SavedModel (CVE-2020-26271)
  • Solves an OOM issue on TPUs when XLA contexts use fused average updates
  • Updates libjpeg-turbo to 2.0.5 to handle CVE-2020-13790.
  • Updates junit to 4.13.1 to handle CVE-2020-15250.
  • Updates PCRE to 8.44 to handle CVE-2019-20838 and CVE-2020-14155.
  • Updates sqlite3 to 3.44.0 to keep in sync with master branch.

- C++
Published by tensorflow-jenkins about 5 years ago

tensorflow - TensorFlow 2.2.2

Release 2.2.2

Bug Fixes and Other Changes

  • Fixes an access to unitialized memory in Eigen code (CVE-2020-26266)
  • Fixes a security vulnerability caused by lack of validation in tf.raw_ops.DataFormatVecPermute and tf.raw_ops.DataFormatDimMap (CVE-2020-26267)
  • Fixes a vulnerability caused by attempting to write to immutable memory region in tf.raw_ops.ImmutableConst (CVE-2020-26268
  • Fixes a CHECK-fail in LSTM with zero-length input (CVE-2020-26270)
  • Fixes a security vulnerability caused by accessing heap data outside of bounds when loading a specially crafted SavedModel (CVE-2020-26271)
  • Prevents memory leaks in loading SavedModels that import functions
  • Updates libjpeg-turbo to 2.0.5 to handle CVE-2020-13790.
  • Updates junit to 4.13.1 to handle CVE-2020-15250.
  • Updates PCRE to 8.44 to handle CVE-2019-20838 and CVE-2020-14155.
  • Updates sqlite3 to 3.44.0 to keep in sync with master branch.

- C++
Published by tensorflow-jenkins about 5 years ago

tensorflow - TensorFlow 2.1.3

Release 2.1.3

Bug Fixes and Other Changes

  • Fixes an access to unitialized memory in Eigen code (CVE-2020-26266)
  • Fixes a security vulnerability caused by lack of validation in tf.raw_ops.DataFormatVecPermute and tf.raw_ops.DataFormatDimMap (CVE-2020-26267)
  • Fixes a vulnerability caused by attempting to write to immutable memory region in tf.raw_ops.ImmutableConst (CVE-2020-26268
  • Fixes a CHECK-fail in LSTM with zero-length input (CVE-2020-26270)
  • Fixes a security vulnerability caused by accessing heap data outside of bounds when loading a specially crafted SavedModel (CVE-2020-26271)
  • Updates libjpeg-turbo to 2.0.5 to handle CVE-2020-13790.
  • Updates junit to 4.13.1 to handle CVE-2020-15250.
  • Updates PCRE to 8.44 to handle CVE-2019-20838 and CVE-2020-14155.
  • Updates sqlite3 to 3.44.0 to keep in sync with master branch.
  • Newer ROCm versions are supported on the 2.1 branch.

- C++
Published by tensorflow-jenkins about 5 years ago

tensorflow - TensorFlow 2.0.4

Release 2.0.4

Note that this is the last patch release for the TensorFlow 2.0.x series.

Bug Fixes and Other Changes

  • Fixes an access to unitialized memory in Eigen code (CVE-2020-26266)
  • Fixes a security vulnerability caused by lack of validation in tf.raw_ops.DataFormatVecPermute and tf.raw_ops.DataFormatDimMap (CVE-2020-26267)
  • Fixes a vulnerability caused by attempting to write to immutable memory region in tf.raw_ops.ImmutableConst (CVE-2020-26268
  • Fixes a CHECK-fail in LSTM with zero-length input (CVE-2020-26270)
  • Fixes a security vulnerability caused by accessing heap data outside of bounds when loading a specially crafted SavedModel (CVE-2020-26271)
  • Updates libjpeg-turbo to 2.0.5 to handle CVE-2020-13790.
  • Updates junit to 4.13.1 to handle CVE-2020-15250.
  • Updates PCRE to 8.44 to handle CVE-2019-20838 and CVE-2020-14155.
  • Updates sqlite3 to 3.44.0 to keep in sync with master branch.

- C++
Published by tensorflow-jenkins about 5 years ago

tensorflow - TensorFlow 1.15.5

Release 1.15.5

Note that this is the last patch release for the TensorFlow 1.x series.

Bug Fixes and Other Changes

  • Fixes an access to unitialized memory in Eigen code (CVE-2020-26266)
  • Fixes a security vulnerability caused by lack of validation in tf.raw_ops.DataFormatVecPermute and tf.raw_ops.DataFormatDimMap (CVE-2020-26267)
  • Fixes a vulnerability caused by attempting to write to immutable memory region in tf.raw_ops.ImmutableConst (CVE-2020-26268
  • Fixes a CHECK-fail in LSTM with zero-length input (CVE-2020-26270)
  • Fixes a security vulnerability caused by accessing heap data outside of bounds when loading a specially crafted SavedModel (CVE-2020-26271)
  • Updates libjpeg-turbo to 2.0.5 to handle CVE-2020-13790.
  • Updates junit to 4.13.1 to handle CVE-2020-15250.
  • Updates PCRE to 8.44 to handle CVE-2019-20838 and CVE-2020-14155.
  • Updates sqlite3 to 3.44.0 to keep in sync with master branch.

- C++
Published by tensorflow-jenkins about 5 years ago

tensorflow - TensorFlow 2.4.0

Release 2.4.0

Major Features and Improvements

  • tf.distribute introduces experimental support for asynchronous training of models via the tf.distribute.experimental.ParameterServerStrategy API. Please see the tutorial to learn more.

  • MultiWorkerMirroredStrategy is now a stable API and is no longer considered experimental. Some of the major improvements involve handling peer failure and many bug fixes. Please check out the detailed tutorial on Multi-worker training with Keras.

  • Introduces experimental support for a new module named tf.experimental.numpy which is a NumPy-compatible API for writing TF programs. See the detailed guide to learn more. Additional details below.

  • Adds Support for TensorFloat-32 on Ampere based GPUs. TensorFloat-32, or TF32 for short, is a math mode for NVIDIA Ampere based GPUs and is enabled by default.

  • A major refactoring of the internals of the Keras Functional API has been completed, that should improve the reliability, stability, and performance of constructing Functional models.

  • Keras mixed precision API tf.keras.mixed_precision is no longer experimental and allows the use of 16-bit floating point formats during training, improving performance by up to 3x on GPUs and 60% on TPUs. Please see below for additional details.

  • TensorFlow Profiler now supports profiling MultiWorkerMirroredStrategy and tracing multiple workers using the sampling mode API.

  • TFLite Profiler for Android is available. See the detailed guide to learn more.

  • TensorFlow pip packages are now built with CUDA11 and cuDNN 8.0.2.

Breaking Changes

  • TF Core:

    • Certain float32 ops run in lower precsion on Ampere based GPUs, including matmuls and convolutions, due to the use of TensorFloat-32. Specifically, inputs to such ops are rounded from 23 bits of precision to 10 bits of precision. This is unlikely to cause issues in practice for deep learning models. In some cases, TensorFloat-32 is also used for complex64 ops. TensorFloat-32 can be disabled by running tf.config.experimental.enable_tensor_float_32_execution(False).
    • The byte layout for string tensors across the C-API has been updated to match TF Core/C++; i.e., a contiguous array of tensorflow::tstring/TF_TStrings.
    • C-API functions TF_StringDecode, TF_StringEncode, and TF_StringEncodedSize are no longer relevant and have been removed; see core/platform/ctstring.h for string access/modification in C.
    • tensorflow.python, tensorflow.core and tensorflow.compiler modules are now hidden. These modules are not part of TensorFlow public API.
    • tf.raw_ops.Max and tf.raw_ops.Min no longer accept inputs of type tf.complex64 or tf.complex128, because the behavior of these ops is not well defined for complex types.
    • XLA:CPU and XLA:GPU devices are no longer registered by default. Use TF_XLA_FLAGS=--tf_xla_enable_xla_devices if you really need them, but this flag will eventually be removed in subsequent releases.

  • tf.keras:

    • The steps_per_execution argument in model.compile() is no longer experimental; if you were passing experimental_steps_per_execution, rename it to steps_per_execution in your code. This argument controls the number of batches to run during each tf.function call when calling model.fit(). Running multiple batches inside a single tf.function call can greatly improve performance on TPUs or small models with a large Python overhead.
    • A major refactoring of the internals of the Keras Functional API may affect code that is relying on certain internal details:
    • Code that uses isinstance(x, tf.Tensor) instead of tf.is_tensor when checking Keras symbolic inputs/outputs should switch to using tf.is_tensor.
    • Code that is overly dependent on the exact names attached to symbolic tensors (e.g. assumes there will be ":0" at the end of the inputs, treats names as unique identifiers instead of using tensor.ref(), etc.) may break.
    • Code that uses full path for get_concrete_function to trace Keras symbolic inputs directly should switch to building matching tf.TensorSpecs directly and tracing the TensorSpec objects.
    • Code that relies on the exact number and names of the op layers that TensorFlow operations were converted into may have changed.
    • Code that uses tf.map_fn/tf.cond/tf.while_loop/control flow as op layers and happens to work before TF 2.4. These will explicitly be unsupported now. Converting these ops to Functional API op layers was unreliable before TF 2.4, and prone to erroring incomprehensibly or being silently buggy.
    • Code that directly asserts on a Keras symbolic value in cases where ops like tf.rank used to return a static or symbolic value depending on if the input had a fully static shape or not. Now these ops always return symbolic values.
    • Code already susceptible to leaking tensors outside of graphs becomes slightly more likely to do so now.
    • Code that tries directly getting gradients with respect to symbolic Keras inputs/outputs. Use GradientTape on the actual Tensors passed to the already-constructed model instead.
    • Code that requires very tricky shape manipulation via converted op layers in order to work, where the Keras symbolic shape inference proves insufficient.
    • Code that tries manually walking a tf.keras.Model layer by layer and assumes layers only ever have one positional argument. This assumption doesn't hold true before TF 2.4 either, but is more likely to cause issues now.
    • Code that manually enters keras.backend.get_graph() before building a functional model is no longer needed.
    • Start enforcing input shape assumptions when calling Functional API Keras models. This may potentially break some users, in case there is a mismatch between the shape used when creating Input objects in a Functional model, and the shape of the data passed to that model. You can fix this mismatch by either calling the model with correctly-shaped data, or by relaxing Input shape assumptions (note that you can pass shapes with None entries for axes that are meant to be dynamic). You can also disable the input checking entirely by setting model.input_spec = None.
    • Several changes have been made to tf.keras.mixed_precision.experimental. Note that it is now recommended to use the non-experimental tf.keras.mixed_precision API.
    • AutoCastVariable.dtype now refers to the actual variable dtype, not the dtype it will be casted to.
    • When mixed precision is enabled, tf.keras.layers.Embedding now outputs a float16 or bfloat16 tensor instead of a float32 tensor.
    • The property tf.keras.mixed_precision.experimental.LossScaleOptimizer.loss_scale is now a tensor, not a LossScale object. This means to get a loss scale of a LossScaleOptimizer as a tensor, you must now call opt.loss_scaleinstead of opt.loss_scale().
    • The property should_cast_variables has been removed from tf.keras.mixed_precision.experimental.Policy
    • When passing a tf.mixed_precision.experimental.DynamicLossScale to tf.keras.mixed_precision.experimental.LossScaleOptimizer, the DynamicLossScale's multiplier must be 2.
    • When passing a tf.mixed_precision.experimental.DynamicLossScale to tf.keras.mixed_precision.experimental.LossScaleOptimizer, the weights of the DynanmicLossScale are copied into the LossScaleOptimizer instead of being reused. This means modifying the weights of the DynamicLossScale will no longer affect the weights of the LossScaleOptimizer, and vice versa.
    • The global policy can no longer be set to a non-floating point policy in tf.keras.mixed_precision.experimental.set_policy
    • In Layer.call, AutoCastVariables will no longer be casted within MirroredStrategy.run or ReplicaContext.merge_call. This is because a thread local variable is used to determine whether AutoCastVariables are casted, and those two functions run with a different thread. Note this only applies if one of these two functions is called within Layer.call; if one of those two functions calls Layer.call, AutoCastVariables will still be casted.

  • tf.data:

    • tf.data.experimental.service.DispatchServer now takes a config tuple instead of individual arguments. Usages should be updated to tf.data.experimental.service.DispatchServer(dispatcher_config).
    • tf.data.experimental.service.WorkerServer now takes a config tuple instead of individual arguments. Usages should be updated to tf.data.experimental.service.WorkerServer(worker_config).

  • tf.distribute:

    • Removes tf.distribute.Strategy.experimental_make_numpy_dataset. Please use tf.data.Dataset.from_tensor_slices instead.
    • Renames experimental_hints in tf.distribute.StrategyExtended.reduce_to, tf.distribute.StrategyExtended.batch_reduce_to, tf.distribute.ReplicaContext.all_reduce to options.
    • Renames tf.distribute.experimental.CollectiveHints to tf.distribute.experimental.CommunicationOptions.
    • Renames tf.distribute.experimental.CollectiveCommunication to tf.distribute.experimental.CommunicationImplementation.
    • Renames tf.distribute.Strategy.experimental_distribute_datasets_from_function to distribute_datasets_from_function as it is no longer experimental.
    • Removes tf.distribute.Strategy.experimental_run_v2 method, which was deprecated in TF 2.2.

  • tf.lite:

    • tf.quantization.quantize_and_dequantize_v2 has been introduced, which updates the gradient definition for quantization which is outside the range to be 0. To simulate the V1 the behavior of tf.quantization.quantize_and_dequantize(...) use tf.grad_pass_through(tf.quantization.quantize_and_dequantize_v2)(...).

  • Building TensorFlow:

    • Windows platform builds: TensorFlow on Windows under MSVC is now built with --copt=/experimental:preprocessor --host_copt=/experimental:preprocessor (see .bazelrc for more details). Builds including TensorFlow may fail with unexpected syntax errors if these flags are absent. See also this thread on SIG Build.

Known Caveats

  • tf.keras.mixed_precision
    • When using mixed precision, calling RMSprop.apply_gradients or Nadam.apply_gradients outside a tf.function does not work and will raise the AttributeError "Tensor.op is meaningless when eager execution is enabled". See this issue for details and a workaround.

Bug Fixes and Other Changes

TF Core:

  • Introduces experimental support for a new module named tf.experimental.numpy, which is a NumPy-compatible API for writing TF programs. This module provides class ndarray, which mimics the ndarray class in NumPy, and wraps an immutable tf.Tensor under the hood. A subset of NumPy functions (e.g. numpy.add) are provided. Their inter-operation with TF facilities is seamless in most cases. See tensorflow/python/ops/numpy_ops/README.md for details of what operations are supported and what are the differences from NumPy.
  • tf.types.experimental.TensorLike is a new Union type that can be used as type annotation for variables representing a Tensor or a value that can be converted to Tensor by tf.convert_to_tensor.
  • Calling ops with a python constants or numpy values is now consistent with tf.converttotensor behavior. This avoids operations like tf.reshape truncating inputs such as from int64 to int32.
  • Adds tf.sparse.map_values to apply a function to the .values of SparseTensor arguments.
  • The Python bitwise operators for Tensor (__and__, __or__, __xor__ and __invert__ now support non-bool arguments and apply the corresponding bitwise ops. bool arguments continue to be supported and dispatch to logical ops. This brings them more in line with Python and NumPy behavior.
  • Adds tf.SparseTensor.with_values. This returns a new SparseTensor with the same sparsity pattern, but with new provided values. It is similar to the with_values function of RaggedTensor.
  • Adds StatelessCase op, and uses it if none of case branches has stateful ops.
  • Adds tf.config.experimental.get_memory_usage to return total memory usage of the device.
  • Adds gradients for RaggedTensorToVariant and RaggedTensorFromVariant.
  • Improve shape inference of nested function calls by supporting constant folding across Arg nodes which makes more static values available to shape inference functions.
    • tf.debugging:
  • tf.debugging.assert_shapes() now works on SparseTensors (Fixes #36268).
    • GPU
  • Adds Support for TensorFloat-32 on Ampere based GPUs. TensorFloat-32, or TF32 for short, is a math mode for NVIDIA Ampere based GPUs which causes certain float32 ops, such as matrix multiplications and convolutions, to run much faster on Ampere GPUs but with reduced precision. This reduced precision has not been found to effect convergence quality of deep learning models in practice. TensorFloat-32 is enabled by default, but can be disabled with tf.config.experimental.enable_tensor_float_32_execution.
    • tf.math:
  • Adds tf.math.erfcinv, the inverse to tf.math.erfc.
    • tf.nn:
  • tf.nn.max_pool2d now supports explicit padding.
    • tf.image:
  • Adds deterministic tf.image.stateless_random_* functions for each tf.image.random_* function. Added a new op stateless_sample_distorted_bounding_box which is a deterministic version of sample_distorted_bounding_box op. Given the same seed, these stateless functions/ops produce the same results independent of how many times the function is called, and independent of global seed settings.
  • Adds deterministic tf.image.resize backprop CUDA kernels for method=ResizeMethod.BILINEAR (the default method). Enable by setting the environment variable TF_DETERMINISTIC_OPS to "true" or "1".
    • tf.print:
  • Bug fix in tf.print() with OrderedDict where if an OrderedDict didn't have the keys sorted, the keys and values were not being printed in accordance with their correct mapping.
    • tf.train.Checkpoint:
  • Now accepts a root argument in the initialization, which generates a checkpoint with a root object. This allows users to create a Checkpoint object that is compatible with Keras model.save_weights() and model.load_weights. The checkpoint is also compatible with the checkpoint saved in the variables/ folder in the SavedModel.
  • When restoring, save_path can be a path to a SavedModel. The function will automatically find the checkpoint in the SavedModel.

tf.data:

  • Adds new tf.data.experimental.service.register_dataset and tf.data.experimental.service.from_dataset_id APIs to enable one process to register a dataset with the tf.data service, and another process to consume data from the dataset.
  • Adds support for dispatcher fault tolerance. To enable fault tolerance, configure a work_dir when running your dispatcher server and set dispatcher_fault_tolerance=True. The dispatcher will store its state to work_dir, so that on restart it can continue from its previous state after restart.
  • Adds support for sharing dataset graphs via shared filesystem instead of over RPC. This reduces load on the dispatcher, improving performance of distributing datasets. For this to work, the dispatcher's work_dir must be accessible from workers. If the worker fails to read from the work_dir, it falls back to using RPC for dataset graph transfer.
  • Adds support for a new "distributed_epoch" processing mode. This processing mode distributes a dataset across all tf.data workers, instead of having each worker process the full dataset. See the tf.data service docs to learn more.
  • Adds optional exclude_cols parameter to CsvDataset. This parameter is the complement of select_cols; at most one of these should be specified.
  • We have implemented an optimization which reorders data-discarding transformations such as take and shard to happen earlier in the dataset when it is safe to do so. The optimization can be disabled via the experimental_optimization.reorder_data_discarding_ops dataset option.
  • tf.data.Options were previously immutable and can now be overridden.
  • tf.data.Dataset.from_generator now supports Ragged and Sparse tensors with a new output_signature argument, which allows from_generator to produce any type describable by a tf.TypeSpec.
  • tf.data.experimental.AUTOTUNE is now available in the core API as tf.data.AUTOTUNE.

tf.distribute:

  • Introduces experimental support for asynchronous training of models via tf.distribute.experimental.ParameterServerStrategy:
    • Replaces the existing tf.distribute.experimental.ParameterServerStrategy symbol with a new class that is for parameter server training in TF2. Usage of the old symbol, usually with Estimator API, should be replaced with [tf.compat.v1.distribute.experimental.ParameterServerStrategy].
    • Added tf.distribute.experimental.coordinator.* namespace, including the main API ClusterCoordinator for coordinating the training cluster, the related data structure RemoteValue and PerWorkerValue.
  • MultiWorkerMirroredStrategy](https://www.tensorflow.org/api_docs/python/tf/distribute/MultiWorkerMirroredStrategy) is now a stable API and is no longer considered experimental. Some of the major improvements involve handling peer failure and many bug fixes. Please check out the detailed tutorial on Multi-worer training with Keras.
  • Adds tf.distribute.Strategy.gather and tf.distribute.ReplicaContext.all_gather APIs to support gathering dense distributed values.
  • Fixes various issues with saving a distributed model.

tf.keras:

  • Improvements from the Functional API refactoring:
    • Functional model construction does not need to maintain a global workspace graph, removing memory leaks especially when building many models or very large models.
    • Functional model construction should be ~8-10% faster on average.
    • Functional models can now contain non-symbolic values in their call inputs inside of the first positional argument.
    • Several classes of TF ops that were not reliably converted to Keras layers during functional API construction should now work, e.g. tf.image.ssim_multiscale
    • Error messages when Functional API construction goes wrong (and when ops cannot be converted to Keras layers automatically) should be clearer and easier to understand.
  • Optimizer.minimize can now accept a loss Tensor and a GradientTape as an alternative to accepting a callable loss.
  • Adds beta hyperparameter to FTRL optimizer classes (Keras and others) to match FTRL paper.
  • Optimizer.__init__ now accepts a gradient_aggregator to allow for customization of how gradients are aggregated across devices, as well as gradients_transformers to allow for custom gradient transformations (such as gradient clipping).
  • Improvements to Keras preprocessing layers:
    • TextVectorization can now accept a vocabulary list or file as an init arg.
    • Normalization can now accept mean and variance values as init args.
  • In Attention and AdditiveAttention layers, the call() method now accepts a return_attention_scores argument. When set to True, the layer returns the attention scores as an additional output argument.
  • Adds tf.metrics.log_cosh and tf.metrics.logcosh API entrypoints with the same implementation as their tf.losses equivalent.
  • For Keras model, the individual call of Model.evaluate uses no cached data for evaluation, while Model.fit uses cached data when validation_data arg is provided for better performance.
  • Adds a save_traces argument to model.save/ tf.keras.models.save_model which determines whether the SavedModel format stores the Keras model/layer call functions. The traced functions allow Keras to revive custom models and layers without the original class definition, but if this isn't required the tracing can be disabled with the added option.
  • The tf.keras.mixed_precision API is now non-experimental. The non-experimental API differs from the experimental API in several ways.
    • tf.keras.mixed_precision.Policy no longer takes in a tf.mixed_precision.experimental.LossScale in the constructor, and no longer has a LossScale associated with it. Instead, Model.compile will automatically wrap the optimizer with a LossScaleOptimizer using dynamic loss scaling if Policy.name is "mixed_float16".
    • tf.keras.mixed_precision.LossScaleOptimizer's constructor takes in different arguments. In particular, it no longer takes in a LossScale, and there is no longer a LossScale associated with the LossScaleOptimizer. Instead, LossScaleOptimizer directly implements fixed or dynamic loss scaling. See the documentation of tf.keras.mixed_precision.experimental.LossScaleOptimizer for details on the differences between the experimental LossScaleOptimizer and the new non-experimental LossScaleOptimizer.
    • tf.mixed_precision.experimental.LossScale and its subclasses are deprecated, as all of its functionality now exists within tf.keras.mixed_precision.LossScaleOptimizer

tf.lite:

  • TFLiteConverter:
    • Support optional flags inference_input_type and inference_output_type for full integer quantized models. This allows users to modify the model input and output type to integer types (tf.int8, tf.uint8) instead of defaulting to float type (tf.float32).
  • NNAPI
    • Adds NNAPI Delegation support for requantization use cases by converting the operation into a dequantize-quantize pair.
    • Removes deprecated Interpreter.setUseNNAPI(boolean) Java API. Use Interpreter.Options.setUseNNAPI instead.
    • Deprecates Interpreter::UseNNAPI(bool) C++ API. Use NnApiDelegate() and related delegate configuration methods directly.
    • Deprecates Interpreter::SetAllowFp16PrecisionForFp32(bool) C++ API. Prefer controlling this via delegate options, e.g. tflite::StatefulNnApiDelegate::Options::allow_fp16' orTfLiteGpuDelegateOptionsV2::isprecisionloss_allowed`.
  • GPU
    • GPU acceleration now supports quantized models by default
  • DynamicBuffer::AddJoinedString() will now add a separator if the first string to be joined is empty.
  • Adds support for cumulative sum (cumsum), both as builtin op and MLIR conversion.

TensorRT

  • Issues a warning when the session_config parameter for the TF1 converter is used or the rewrite_config_template field in the TF2 converter parameter object is used.

TPU Enhancements:

  • Adds support for the beta parameter of the FTRL optimizer for TPU embeddings. Users of other TensorFlow platforms can implement equivalent behavior by adjusting the l2 parameter.

XLA Support:

  • xla.experimental.compile is deprecated, use tf.function(experimental_compile=True) instead.
  • Adds tf.function.experimental_get_compiler_ir which returns compiler IR (currently 'hlo' and 'optimized_hlo') for given input for given function.

Security:

Other:

  • We have replaced uses of "whitelist" and "blacklist" with "allowlist" and "denylist" where possible. Please see this list for more context.
  • Adds tf.config.experimental.mlir_bridge_rollout which will help us rollout the new MLIR TPU bridge.
  • Adds tf.experimental.register_filesystem_plugin to load modular filesystem plugins from Python

Thanks to our Contributors

This release contains contributions from many people at Google as well as the following external contributors:

8bitmp3, aaa.jq, Abhineet Choudhary, Abolfazl Shahbazi, acxz, Adam Hillier, Adrian Garcia Badaracco, Ag Ramesh, ahmedsabie, Alan Anderson, Alexander Grund, Alexandre Lissy, Alexey Ivanov, Amedeo Cavallo, anencore94, Aniket Kumar Singh, Anthony Platanios, Ashwin Phadke, Balint Cristian, Basit Ayantunde, bbbboom, Ben Barsdell, Benjamin Chetioui, Benjamin Peterson, bhack, Bhanu Prakash Bandaru Venkata, Biagio Montaruli, Brent M. Spell, bubblebooy, bzhao, cfRod, Cheng Chen, Cheng(Kit) Chen, Chris Tessum, Christian, chuanqiw, codeadminperitiae, COTASPAR, CuiYifeng, danielknobe, danielyou0230, dannyfriar, daria, DarrenZhang01, Denisa Roberts, dependabot[bot], Deven Desai, Dmitry Volodin, Dmitry Zakharov, drebain, Duncan Riach, Eduard Feicho, Ehsan Toosi, Elena Zhelezina, emlaprise2358, Eugene Kuznetsov, Evaderan-Lab, Evgeniy Polyakov, Fausto Morales, Felix Johnny, fo40225, Frederic Bastien, Fredrik Knutsson, fsx950223, Gaurav Singh, Gauri1 Deshpande, George Grzegorz Pawelczak, gerbauz, Gianluca Baratti, Giorgio Arena, Gmc2, Guozhong Zhuang, Hannes Achleitner, Harirai, HarisWang, Harsh188, hedgehog91, Hemal Mamtora, Hideto Ueno, Hugh Ku, Ian Beauregard, Ilya Persky, jacco, Jakub Beránek, Jan Jongboom, Javier Montalt Tordera, Jens Elofsson, Jerry Shih, jerryyin, jgehw, Jinjing Zhou, jma, jmsmdy, Johan Nordström, John Poole, Jonah Kohn, Jonathan Dekhtiar, jpodivin, Jung Daun, Kai Katsumata, Kaixi Hou, Kamil Rakoczy, Kaustubh Maske Patil, Kazuaki Ishizaki, Kedar Sovani, Koan-Sin Tan, Koki Ibukuro, Krzysztof Laskowski, Kushagra Sharma, Kushan Ahmadian, Lakshay Tokas, Leicong Li, levinxo, Lukas Geiger, Maderator, Mahmoud Abuzaina, Mao Yunfei, Marius Brehler, markf, Martin Hwasser, Martin Kubovčík, Matt Conley, Matthias, mazharul, mdfaijul, Michael137, MichelBr, Mikhail Startsev, Milan Straka, Ml-0, Myung-Hyun Kim, Måns Nilsson, Nathan Luehr, ngc92, nikochiko, Niranjan Hasabnis, nyagato00, Oceania2018, Oleg Guba, Ongun Kanat, OscarVanL, Patrik Laurell, Paul Tanger, Peter Sobot, Phil Pearl, PlusPlusUltra, Poedator, Prasad Nikam, Rahul-Kamat, Rajeshwar Reddy T, redwrasse, Rickard, Robert Szczepanski, Rohan Lekhwani, Sam Holt, Sami Kama, Samuel Holt, Sandeep Giri, sboshin, Sean Settle, settle, Sharada Shiddibhavi, Shawn Presser, ShengYang1, Shi,Guangyong, Shuxiang Gao, Sicong Li, Sidong-Wei, Srihari Humbarwadi, Srinivasan Narayanamoorthy, Steenu Johnson, Steven Clarkson, stjohnso98, Tamas Bela Feher, Tamas Nyiri, Tarandeep Singh, Teng Lu, Thibaut Goetghebuer-Planchon, Tim Bradley, Tomasz Strejczek, Tongzhou Wang, Torsten Rudolf, Trent Lo, Ty Mick, Tzu-Wei Sung, Varghese, Jojimon, Vignesh Kothapalli, Vishakha Agrawal, Vividha, Vladimir Menshakov, Vladimir Silyaev, VoVAllen, Võ Văn Nghĩa, wondertx, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yair Ehrenwald, Yasir Modak, Yasuhiro Matsumoto, Yimei Sun, Yiwen Li, Yixing, Yoav Ramon, Yong Tang, Yong Wu, yuanbopeng, Yunmo Koo, Zhangqiang, Zhou Peng, ZhuBaohe, zilinzhu, zmx

- C++
Published by tensorflow-jenkins about 5 years ago

tensorflow - TensorFlow 2.4.0-rc4

Release 2.4.0

Major Features and Improvements

  • tf.distribute introduces experimental support for asynchronous training of Keras models via the tf.distribute.experimental.ParameterServerStrategy API. Please see below for additional details.

  • MultiWorkerMirroredStrategy is now a stable API and is no longer considered experimental. Some of the major improvements involve handling peer failure and many bug fixes. Please check out the detailed tutorial on Multi-worker training with Keras.

  • Introduces experimental support for a new module named tf.experimental.numpy which is a NumPy-compatible API for writing TF programs. See the detailed guide to learn more. Additional details below.

  • Adds Support for TensorFloat-32 on Ampere based GPUs. TensorFloat-32, or TF32 for short, is a math mode for NVIDIA Ampere based GPUs and is enabled by default.

  • A major refactoring of the internals of the Keras Functional API has been completed, that should improve the reliability, stability, and performance of constructing Functional models.

  • Keras mixed precision API tf.keras.mixed_precision is no longer experimental and allows the use of 16-bit floating point formats during training, improving performance by up to 3x on GPUs and 60% on TPUs. Please see below for additional details.

  • TensorFlow Profiler now supports profiling MultiWorkerMirroredStrategy and tracing multiple workers using the sampling mode API.

  • TFLite Profiler for Android is available. See the detailed guide to learn more.

  • TensorFlow pip packages are now built with CUDA11 and cuDNN 8.0.2.

Breaking Changes

  • TF Core:

    • Certain float32 ops run in lower precision on Ampere based GPUs, including matmuls and convolutions, due to the use of TensorFloat-32. Specifically, inputs to such ops are rounded from 23 bits of precision to 10 bits of precision. This is unlikely to cause issues in practice for deep learning models. In some cases, TensorFloat-32 is also used for complex64 ops. TensorFloat-32 can be disabled by running tf.config.experimental.enable_tensor_float_32_execution(False).
    • The byte layout for string tensors across the C-API has been updated to match TF Core/C++; i.e., a contiguous array of tensorflow::tstring/TF_TStrings.
    • C-API functions TF_StringDecode, TF_StringEncode, and TF_StringEncodedSize are no longer relevant and have been removed; see core/platform/ctstring.h for string access/modification in C.
    • tensorflow.python, tensorflow.core and tensorflow.compiler modules are now hidden. These modules are not part of TensorFlow public API.
    • tf.raw_ops.Max and tf.raw_ops.Min no longer accept inputs of type tf.complex64 or tf.complex128, because the behavior of these ops is not well defined for complex types.
    • XLA:CPU and XLA:GPU devices are no longer registered by default. Use TF_XLA_FLAGS=--tf_xla_enable_xla_devices if you really need them, but this flag will eventually be removed in subsequent releases.

  • tf.keras:

    • The steps_per_execution argument in compile() is no longer experimental; if you were passing experimental_steps_per_execution, rename it to steps_per_execution in your code. This argument controls the number of batches to run during each tf.function call when calling fit(). Running multiple batches inside a single tf.function call can greatly improve performance on TPUs or small models with a large Python overhead.
    • A major refactoring of the internals of the Keras Functional API may affect code that is relying on certain internal details:
    • Code that uses isinstance(x, tf.Tensor) instead of tf.is_tensor when checking Keras symbolic inputs/outputs should switch to using tf.is_tensor.
    • Code that is overly dependent on the exact names attached to symbolic tensors (e.g. assumes there will be ":0" at the end of the inputs, treats names as unique identifiers instead of using tensor.ref(), etc.)
    • Code that uses get_concrete_function to trace Keras symbolic inputs directly should switch to building matching tf.TensorSpecs directly and tracing the TensorSpec objects.
    • Code that relies on the exact number and names of the op layers that TensorFlow operations were converted into may have changed.
    • Code that uses tf.map_fn/tf.cond/tf.while_loop/control flow as op layers and happens to work before TF 2.4. These will explicitly be unsupported now. Converting these ops to Functional API op layers was unreliable before TF 2.4, and prone to erroring incomprehensibly or being silently buggy.
    • Code that directly asserts on a Keras symbolic value in cases where ops like tf.rank used to return a static or symbolic value depending on if the input had a fully static shape or not. Now these ops always return symbolic values.
    • Code already susceptible to leaking tensors outside of graphs becomes slightly more likely to do so now.
    • Code that tries directly getting gradients with respect to symbolic Keras inputs/outputs. Use GradientTape on the actual Tensors passed to the already- constructed model instead.
    • Code that requires very tricky shape manipulation via converted op layers in order to work, where the Keras symbolic shape inference proves insufficient.
    • Code that tries manually walking a tf.keras.Model layer by layer and assumes layers only ever have one positional argument. This assumption doesn't hold true before TF 2.4 either, but is more likely to cause issues now.
    • Code that manually enters keras.backend.get_graph() before building a functional model is no longer needed.
    • Start enforcing input shape assumptions when calling Functional API Keras models. This may potentially break some users, in case there is a mismatch between the shape used when creating Input objects in a Functional model, and the shape of the data passed to that model. You can fix this mismatch by either calling the model with correctly-shaped data, or by relaxing Input shape assumptions (note that you can pass shapes with None entries for axes that are meant to be dynamic). You can also disable the input checking entirely by setting model.input_spec = None.
    • Serveral changes have been made to tf.keras.mixed_precision.experimental. Note that it is now recommended to use the non-experimental tf.keras.mixed_precision API.
    • AutoCastVariable.dtype now refers to the actual variable dtype, not the dtype it will be casted to.
    • When mixed precision is enabled, tf.keras.layers.Embedding now outputs a float16 or bfloat16 tensor instead of a float32 tensor.
    • The property tf.keras.mixed_precision.experimental.LossScaleOptimizer.loss_scale is now a tensor, not a LossScale object. This means to get a loss scale of a LossScaleOptimizer as a tensor, you must now call opt.loss_scale instead of opt.loss_scale().
    • The property should_cast_variables has been removed from tf.keras.mixed_precision.experimental.Policy
    • When passing a tf.mixed_precision.experimental.DynamicLossScale to tf.keras.mixed_precision.experimental.LossScaleOptimizer, the DynamicLossScale's multiplier must be 2.
    • When passing a tf.mixed_precision.experimental.DynamicLossScale to tf.keras.mixed_precision.experimental.LossScaleOptimizer, the weights of the DynanmicLossScale are copied into the LossScaleOptimizer instead of being reused. This means modifying the weights of the DynamicLossScale will no longer affect the weights of the LossScaleOptimizer, and vice versa.
    • The global policy can no longer be set to a non-floating point policy in tf.keras.mixed_precision.experimental.set_policy
    • In Layer.call, AutoCastVariables will no longer be casted within MirroredStrategy.run or ReplicaContext.merge_call. This is because a thread local variable is used to determine whether AutoCastVariables are casted, and those two functions run with a different thread. Note this only applies if one of these two functions is called within Layer.call; if one of those two functions calls Layer.call, AutoCastVariables will still be casted.

  • tf.data:

    • tf.data.experimental.service.DispatchServer now takes a config tuple instead of individual arguments. Usages should be updated to tf.data.experimental.service.DispatchServer(dispatcher_config).
    • tf.data.experimental.service.WorkerServer now takes a config tuple instead of individual arguments. Usages should be updated to tf.data.experimental.service.WorkerServer(worker_config).

  • tf.distribute:

    • Removes tf.distribute.Strategy.experimental_make_numpy_dataset. Please use tf.data.Dataset.from_tensor_slices instead.
    • Renames experimental_hints in tf.distribute.StrategyExtended.reduce_to, tf.distribute.StrategyExtended.batch_reduce_to, tf.distribute.ReplicaContext.all_reduce to options:
    • Renames tf.distribute.experimental.CollectiveHints to tf.distribute.experimental.CommunicationOptions.
    • Renames tf.distribute.experimental.CollectiveCommunication to tf.distribute.experimental.CommunicationImplementation.
    • Renames tf.distribute.Strategy.experimental_distribute_datasets_from_function to distribute_datasets_from_function as it is no longer experimental.
    • Removes tf.distribute.Strategy.experimental_run_v2 method, which was deprecated in TF 2.2.

  • tf.lite:

    • tf.quantization.quantize_and_dequantize_v2 has been introduced, which updates the gradient definition for quantization which is outside the range to be 0. To simulate the V1 the behavior of tf.quantization.quantize_and_dequantize(...) use tf.grad_pass_through(tf.quantization.quantize_and_dequantize_v2)(...).

Bug Fixes and Other Changes

TF Core:

  • Introduces experimental support for a new module named tf.experimental.numpy, which is a NumPy-compatible API for writing TF programs. This module provides class ndarray, which mimics the ndarray class in NumPy, and wraps an immutable tf.Tensor under the hood. A subset of NumPy functions (e.g. numpy.add) are provided. Their inter-operation with TF facilities is seamless in most cases. See tensorflow/python/ops/numpy_ops/README.md for details of what operations are supported and what are the differences from NumPy.
  • tf.types.experimental.TensorLike is a new Union type that can be used as type annotation for variables representing a Tensor or a value that can be converted to Tensor by tf.convert_to_tensor.
  • Calling ops with a python constants or numpy values is now consistent with tf.converttotensor behavior. This avoids operations like tf.reshape truncating inputs such as from int64 to int32.
  • Adds tf.sparse.map_values to apply a function to the .values of SparseTensor arguments.
  • The Python bitwise operators for Tensor (__and__, __or__, __xor__ and __invert__ now support non-bool arguments and apply the corresponding bitwise ops. bool arguments continue to be supported and dispatch to logical ops. This brings them more in line with Python and NumPy behavior.
  • Adds tf.SparseTensor.with_values. This returns a new SparseTensor with the same sparsity pattern, but with new provided values. It is similar to the with_values function of RaggedTensor.
  • Adds StatelessCase op, and uses it if none of case branches has stateful ops.
  • Adds tf.config.experimental.get_memory_usage to return total memory usage of the device.
  • Adds gradients for RaggedTensorToVariant and RaggedTensorFromVariant.
  • Improve shape inference of nested function calls by supporting constant folding across Arg nodes which makes more static values available to shape inference functions.
    • tf.debugging:
  • tf.debugging.assert_shapes() now works on SparseTensors (Fixes #36268).
    • GPU
  • Adds Support for TensorFloat-32 on Ampere based GPUs. TensorFloat-32, or TF32 for short, is a math mode for NVIDIA Ampere based GPUs which causes certain float32 ops, such as matrix multiplications and convolutions, to run much faster on Ampere GPUs but with reduced precision. This reduced precision has not been found to effect convergence quality of deep learning models in practice. TensorFloat-32 is enabled by default, but can be disabled with tf.config.experimental.enable_tensor_float_32_execution.
    • tf.math:
  • Adds tf.math.erfcinv, the inverse to tf.math.erfc.
    • tf.nn:
  • tf.nn.max_pool2d now supports explicit padding.
    • tf.image:
  • Adds deterministic tf.image.stateless_random_* functions for each tf.image.random_* function. Added a new op stateless_sample_distorted_bounding_box which is a deterministic version of sample_distorted_bounding_box op. Given the same seed, these stateless functions/ops produce the same results independent of how many times the function is called, and independent of global seed settings.
  • Adds deterministic tf.image.resize backprop CUDA kernels for method=ResizeMethod.BILINEAR (the default method). Enable by setting the environment variable TF_DETERMINISTIC_OPS to "true" or "1".
    • tf.print:
  • Bug fix in tf.print() with OrderedDict where if an OrderedDict didn't have the keys sorted, the keys and values were not being printed in accordance with their correct mapping.
    • tf.train.Checkpoint:
  • Now accepts a root argument in the initialization, which generates a checkpoint with a root object. This allows users to create a Checkpoint object that is compatible with Keras model.save_weights() and model.load_weights. The checkpoint is also compatible with the checkpoint saved in the variables/ folder in the SavedModel.
  • When restoring, save_path can be a path to a SavedModel. The function will automatically find the checkpoint in the SavedModel.

tf.data:

  • Adds new tf.data.experimental.service.register_dataset and tf.data.experimental.service.from_dataset_id APIs to enable one process to register a dataset with the tf.data service, and another process to consume data from the dataset.
  • Adds support for dispatcher fault tolerance. To enable fault tolerance, configure a work_dir when running your dispatcher server and set dispatcher_fault_tolerance=True. The dispatcher will store its state to work_dir, so that on restart it can continue from its previous state after restart.
  • Adds support for sharing dataset graphs via shared filesystem instead of over RPC. This reduces load on the dispatcher, improving performance of distributing datasets. For this to work, the dispatcher's work_dir must be accessible from workers. If the worker fails to read from the work_dir, it falls back to using RPC for dataset graph transfer.
  • Adds support for a new "distributed_epoch" processing mode. This processing mode distributes a dataset across all tf.data workers, instead of having each worker process the full dataset. See the tf.data service docs to learn more.
  • Adds optional exclude_cols parameter to CsvDataset. This parameter is the complement of select_cols; at most one of these should be specified.
  • We have implemented an optimization which reorders data-discarding transformations such as take and shard to happen earlier in the dataset when it is safe to do so. The optimization can be disabled via the experimental_optimization.reorder_data_discarding_ops dataset option.
  • tf.data.Options were previously immutable and can now be overridden.
  • tf.data.Dataset.from_generator now supports Ragged and Sparse tensors with a new output_signature argument, which allows from_generator to produce any type describable by a tf.TypeSpec.
  • tf.data.experimental.AUTOTUNE is now available in the core API as tf.data.AUTOTUNE.

tf.distribute:

  • Introduces experimental support for asynchronous training of Keras models via tf.distribute.experimental.ParameterServerStrategy:
    • Replaces the existing tf.distribute.experimental.ParameterServerStrategy symbol with a new class that is for parameter server training in TF2. Usage of the old symbol, usually with Estimator API, should be replaced with [tf.compat.v1.distribute.experimental.ParameterServerStrategy].
    • Added tf.distribute.experimental.coordinator.* namespace, including the main API ClusterCoordinator for coordinating the training cluster, the related data structure RemoteValue and PerWorkerValue.
  • Adds tf.distribute.Strategy.gather and tf.distribute.ReplicaContext.all_gather APIs to support gathering dense distributed values.
  • Fixes various issues with saving a distributed model.

tf.keras:

  • Improvements from the Functional API refactoring:
    • Functional model construction does not need to maintain a global workspace graph, removing memory leaks especially when building many models or very large models.
    • Functional model construction should be ~8-10% faster on average.
    • Functional models can now contain non-symbolic values in their call inputs inside of the first positional argument.
    • Several classes of TF ops that were not reliably converted to Keras layers during functional API construction should now work, e.g. tf.image.ssim_multiscale
    • Error messages when Functional API construction goes wrong (and when ops cannot be converted to Keras layers automatically) should be clearer and easier to understand.
  • Optimizer.minimize can now accept a loss Tensor and a GradientTape as an alternative to accepting a callable loss.
  • Adds beta hyperparameter to FTRL optimizer classes (Keras and others) to match FTRL paper.
  • Optimizer.__init__ now accepts a gradient_aggregator to allow for customization of how gradients are aggregated across devices, as well as gradients_transformers to allow for custom gradient transformations (such as gradient clipping).
  • Improvements to Keras preprocessing layers:
    • TextVectorization can now accept a vocabulary list or file as an init arg.
    • Normalization can now accept mean and variance values as init args.
  • In Attention and AdditiveAttention layers, the call() method now accepts a return_attention_scores argument. When set to True, the layer returns the attention scores as an additional output argument.
  • Adds tf.metrics.log_cosh and tf.metrics.logcosh API entrypoints with the same implementation as their tf.losses equivalent.
  • For Keras model, the individual call of Model.evaluate uses no cached data for evaluation, while Model.fit uses cached data when validation_data arg is provided for better performance.
  • Adds a save_traces argument to model.save/ tf.keras.models.save_model which determines whether the SavedModel format stores the Keras model/layer call functions. The traced functions allow Keras to revive custom models and layers without the original class definition, but if this isn't required the tracing can be disabled with the added option.
  • The tf.keras.mixed_precision API is non non-experimental. The non-experimental API differs from the experimental API in several ways.
    • tf.keras.mixed_precision.Policy no longer takes in a tf.mixed_precision.experimental.LossScale in the constructor, and no longer has a LossScale associated with it. Instead, Model.compile will automatically wrap the optimizer with a LossScaleOptimizer using dynamic loss scaling if Policy.name is "mixed_float16".
    • tf.keras.mixed_precision.LossScaleOptimizer's constructor takes in different arguments. In particular, it no longer takes in a LossScale, and there is no longer a LossScale associated with the LossScaleOptimizer. Instead, LossScaleOptimizer directly implements fixed or dynamic loss scaling. See the documentation of tf.keras.mixed_precision.experimental.LossScaleOptimizer for details on the differences between the experimental LossScaleOptimizer and the new non-experimental LossScaleOptimizer.
    • tf.mixed_precision.experimental.LossScale and its subclasses are deprecated, as all of its functionality now exists within tf.keras.mixed_precision.LossScaleOptimizer

tf.lite:

  • TFLiteConverter:
    • Support optional flags inference_input_type and inference_output_type for full integer quantized models. This allows users to modify the model input and output type to integer types (tf.int8, tf.uint8) instead of defaulting to float type (tf.float32).
  • NNAPI
    • Adds NNAPI Delegation support for requantization use cases by converting the operation into a dequantize-quantize pair.
    • Removes deprecated Interpreter.setUseNNAPI(boolean) Java API. Use Interpreter.Options.setUseNNAPI instead.
    • Deprecates Interpreter::UseNNAPI(bool) C++ API. Use NnApiDelegate() and related delegate configuration methods directly.
    • Deprecates Interpreter::SetAllowFp16PrecisionForFp32(bool) C++ API. Prefer controlling this via delegate options, e.g. tflite::StatefulNnApiDelegate::Options::allow_fp16' orTfLiteGpuDelegateOptionsV2::isprecisionloss_allowed`.
  • GPU
    • GPU acceleration now supports quantized models by default
  • DynamicBuffer::AddJoinedString() will now add a separator if the first string to be joined is empty.
  • Adds support for cumulative sum (cumsum), both as builtin op and MLIR conversion.

TensorRT

  • Issues a warning when the session_config parameter for the TF1 converter is used or the rewrite_config_template field in the TF2 converter parameter object is used.

TPU Enhancements:

  • Adds support for the beta parameter of the FTRL optimizer for TPU embeddings. Users of other TensorFlow platforms can implement equivalent behavior by adjusting the l2 parameter.

XLA Support:

  • xla.experimental.compile is deprecated, use tf.function(experimental_compile=True) instead.
  • Adds tf.function.experimental_get_compiler_ir which returns compiler IR (currently 'hlo' and 'optimized_hlo') for given input for given function.

Security:

Other:

  • We have replaced uses of "whitelist" and "blacklist" with "allowlist" and "denylist" where possible. Please see this list for more context.
  • Adds tf.config.experimental.mlir_bridge_rollout which will help us rollout the new MLIR TPU bridge.
  • Adds tf.experimental.register_filesystem_plugin to load modular filesystem plugins from Python

Thanks to our Contributors

This release contains contributions from many people at Google and external contributors.

8bitmp3, aaa.jq, Abhineet Choudhary, Abolfazl Shahbazi, acxz, Adam Hillier, Adrian Garcia Badaracco, Ag Ramesh, ahmedsabie, Alan Anderson, Alexander Grund, Alexandre Lissy, Alexey Ivanov, Amedeo Cavallo, anencore94, Aniket Kumar Singh, Anthony Platanios, Ashwin Phadke, Balint Cristian, Basit Ayantunde, bbbboom, Ben Barsdell, Benjamin Chetioui, Benjamin Peterson, bhack, Bhanu Prakash Bandaru Venkata, Biagio Montaruli, Brent M. Spell, bubblebooy, bzhao, cfRod, Cheng Chen, Cheng(Kit) Chen, Chris Tessum, Christian, chuanqiw, codeadminperitiae, COTASPAR, CuiYifeng, danielknobe, danielyou0230, dannyfriar, daria, DarrenZhang01, Denisa Roberts, dependabot[bot], Deven Desai, Dmitry Volodin, Dmitry Zakharov, drebain, Duncan Riach, Eduard Feicho, Ehsan Toosi, Elena Zhelezina, emlaprise2358, Eugene Kuznetsov, Evaderan-Lab, Evgeniy Polyakov, Fausto Morales, Felix Johnny, fo40225, Frederic Bastien, Fredrik Knutsson, fsx950223, Gaurav Singh, Gauri1 Deshpande, George Grzegorz Pawelczak, gerbauz, Gianluca Baratti, Giorgio Arena, Gmc2, Guozhong Zhuang, Hannes Achleitner, Harirai, HarisWang, Harsh188, hedgehog91, Hemal Mamtora, Hideto Ueno, Hugh Ku, Ian Beauregard, Ilya Persky, jacco, Jakub Beránek, Jan Jongboom, Javier Montalt Tordera, Jens Elofsson, Jerry Shih, jerryyin, jgehw, Jinjing Zhou, jma, jmsmdy, Johan Nordström, John Poole, Jonah Kohn, Jonathan Dekhtiar, jpodivin, Jung Daun, Kai Katsumata, Kaixi Hou, Kamil Rakoczy, Kaustubh Maske Patil, Kazuaki Ishizaki, Kedar Sovani, Koan-Sin Tan, Koki Ibukuro, Krzysztof Laskowski, Kushagra Sharma, Kushan Ahmadian, Lakshay Tokas, Leicong Li, levinxo, Lukas Geiger, Maderator, Mahmoud Abuzaina, Mao Yunfei, Marius Brehler, markf, Martin Hwasser, Martin Kubovčík, Matt Conley, Matthias, mazharul, mdfaijul, Michael137, MichelBr, Mikhail Startsev, Milan Straka, Ml-0, Myung-Hyun Kim, Måns Nilsson, Nathan Luehr, ngc92, nikochiko, Niranjan Hasabnis, nyagato00, Oceania2018, Oleg Guba, Ongun Kanat, OscarVanL, Patrik Laurell, Paul Tanger, Peter Sobot, Phil Pearl, PlusPlusUltra, Poedator, Prasad Nikam, Rahul-Kamat, Rajeshwar Reddy T, redwrasse, Rickard, Robert Szczepanski, Rohan Lekhwani, Sam Holt, Sami Kama, Samuel Holt, Sandeep Giri, sboshin, Sean Settle, settle, Sharada Shiddibhavi, Shawn Presser, ShengYang1, Shi,Guangyong, Shuxiang Gao, Sicong Li, Sidong-Wei, Srihari Humbarwadi, Srinivasan Narayanamoorthy, Steenu Johnson, Steven Clarkson, stjohnso98, Tamas Bela Feher, Tamas Nyiri, Tarandeep Singh, Teng Lu, Thibaut Goetghebuer-Planchon, Tim Bradley, Tomasz Strejczek, Tongzhou Wang, Torsten Rudolf, Trent Lo, Ty Mick, Tzu-Wei Sung, Varghese, Jojimon, Vignesh Kothapalli, Vishakha Agrawal, Vividha, Vladimir Menshakov, Vladimir Silyaev, VoVAllen, Võ Văn Nghĩa, wondertx, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yair Ehrenwald, Yasir Modak, Yasuhiro Matsumoto, Yimei Sun, Yiwen Li, Yixing, Yoav Ramon, Yong Tang, Yong Wu, yuanbopeng, Yunmo Koo, Zhangqiang, Zhou Peng, ZhuBaohe, zilinzhu, zmx

- C++
Published by tensorflow-jenkins about 5 years ago

tensorflow - TensorFlow 2.4.0-rc3

Release 2.4.0

Major Features and Improvements

  • tf.distribute introduces experimental support for asynchronous training of Keras models via the tf.distribute.experimental.ParameterServerStrategy API. Please see below for additional details.

  • MultiWorkerMirroredStrategy is now a stable API and is no longer considered experimental. Some of the major improvements involve handling peer failure and many bug fixes. Please check out the detailed tutorial on Multi-worker training with Keras.

  • Introduces experimental support for a new module named tf.experimental.numpy which is a NumPy-compatible API for writing TF programs. See the detailed guide to learn more. Additional details below.

  • Adds Support for TensorFloat-32 on Ampere based GPUs. TensorFloat-32, or TF32 for short, is a math mode for NVIDIA Ampere based GPUs and is enabled by default.

  • A major refactoring of the internals of the Keras Functional API has been completed, that should improve the reliability, stability, and performance of constructing Functional models.

  • Keras mixed precision API tf.keras.mixed_precision is no longer experimental and allows the use of 16-bit floating point formats during training, improving performance by up to 3x on GPUs and 60% on TPUs. Please see below for additional details.

  • TensorFlow Profiler now supports profiling MultiWorkerMirroredStrategy and tracing multiple workers using the sampling mode API.

  • TFLite Profiler for Android is available. See the detailed guide to learn more.

  • TensorFlow pip packages are now built with CUDA11 and cuDNN 8.0.2.

Breaking Changes

  • TF Core:

    • Certain float32 ops run in lower precsion on Ampere based GPUs, including matmuls and convolutions, due to the use of TensorFloat-32. Specifically, inputs to such ops are rounded from 23 bits of precision to 10 bits of precision. This is unlikely to cause issues in practice for deep learning models. In some cases, TensorFloat-32 is also used for complex64 ops. TensorFloat-32 can be disabled by running tf.config.experimental.enable_tensor_float_32_execution(False).
    • The byte layout for string tensors across the C-API has been updated to match TF Core/C++; i.e., a contiguous array of tensorflow::tstring/TF_TStrings.
    • C-API functions TF_StringDecode, TF_StringEncode, and TF_StringEncodedSize are no longer relevant and have been removed; see core/platform/ctstring.h for string access/modification in C.
    • tensorflow.python, tensorflow.core and tensorflow.compiler modules are now hidden. These modules are not part of TensorFlow public API.
    • tf.raw_ops.Max and tf.raw_ops.Min no longer accept inputs of type tf.complex64 or tf.complex128, because the behavior of these ops is not well defined for complex types.
    • XLA:CPU and XLA:GPU devices are no longer registered by default. Use TF_XLA_FLAGS=--tf_xla_enable_xla_devices if you really need them, but this flag will eventually be removed in subsequent releases.

  • tf.keras:

    • The steps_per_execution argument in compile() is no longer experimental; if you were passing experimental_steps_per_execution, rename it to steps_per_execution in your code. This argument controls the number of batches to run during each tf.function call when calling fit(). Running multiple batches inside a single tf.function call can greatly improve performance on TPUs or small models with a large Python overhead.
    • A major refactoring of the internals of the Keras Functional API may affect code that is relying on certain internal details:
    • Code that uses isinstance(x, tf.Tensor) instead of tf.is_tensor when checking Keras symbolic inputs/outputs should switch to using tf.is_tensor.
    • Code that is overly dependent on the exact names attached to symbolic tensors (e.g. assumes there will be ":0" at the end of the inputs, treats names as unique identifiers instead of using tensor.ref(), etc.)
    • Code that uses get_concrete_function to trace Keras symbolic inputs directly should switch to building matching tf.TensorSpecs directly and tracing the TensorSpec objects.
    • Code that relies on the exact number and names of the op layers that TensorFlow operations were converted into may have changed.
    • Code that uses tf.map_fn/tf.cond/tf.while_loop/control flow as op layers and happens to work before TF 2.4. These will explicitly be unsupported now. Converting these ops to Functional API op layers was unreliable before TF 2.4, and prone to erroring incomprehensibly or being silently buggy.
    • Code that directly asserts on a Keras symbolic value in cases where ops like tf.rank used to return a static or symbolic value depending on if the input had a fully static shape or not. Now these ops always return symbolic values.
    • Code already susceptible to leaking tensors outside of graphs becomes slightly more likely to do so now.
    • Code that tries directly getting gradients with respect to symbolic Keras inputs/outputs. Use GradientTape on the actual Tensors passed to the already- constructed model instead.
    • Code that requires very tricky shape manipulation via converted op layers in order to work, where the Keras symbolic shape inference proves insufficient.
    • Code that tries manually walking a tf.keras.Model layer by layer and assumes layers only ever have one positional argument. This assumption doesn't hold true before TF 2.4 either, but is more likely to cause issues now.
    • Code that manually enters keras.backend.get_graph() before building a functional model is no longer needed.
    • Start enforcing input shape assumptions when calling Functional API Keras models. This may potentially break some users, in case there is a mismatch between the shape used when creating Input objects in a Functional model, and the shape of the data passed to that model. You can fix this mismatch by either calling the model with correctly-shaped data, or by relaxing Input shape assumptions (note that you can pass shapes with None entries for axes that are meant to be dynamic). You can also disable the input checking entirely by setting model.input_spec = None.
    • Serveral changes have been made to tf.keras.mixed_precision.experimental. Note that it is now recommended to use the non-experimental tf.keras.mixed_precision API.
    • AutoCastVariable.dtype now refers to the actual variable dtype, not the dtype it will be casted to.
    • When mixed precision is enabled, tf.keras.layers.Embedding now outputs a float16 or bfloat16 tensor instead of a float32 tensor.
    • The property tf.keras.mixed_precision.experimental.LossScaleOptimizer.loss_scale is now a tensor, not a LossScale object. This means to get a loss scale of a LossScaleOptimizer as a tensor, you must now call opt.loss_scale instead of opt.loss_scale().
    • The property should_cast_variables has been removed from tf.keras.mixed_precision.experimental.Policy
    • When passing a tf.mixed_precision.experimental.DynamicLossScale to tf.keras.mixed_precision.experimental.LossScaleOptimizer, the DynamicLossScale's multiplier must be 2.
    • When passing a tf.mixed_precision.experimental.DynamicLossScale to tf.keras.mixed_precision.experimental.LossScaleOptimizer, the weights of the DynanmicLossScale are copied into the LossScaleOptimizer instead of being reused. This means modifying the weights of the DynamicLossScale will no longer affect the weights of the LossScaleOptimizer, and vice versa.
    • The global policy can no longer be set to a non-floating point policy in tf.keras.mixed_precision.experimental.set_policy
    • In Layer.call, AutoCastVariables will no longer be casted within MirroredStrategy.run or ReplicaContext.merge_call. This is because a thread local variable is used to determine whether AutoCastVariables are casted, and those two functions run with a different thread. Note this only applies if one of these two functions is called within Layer.call; if one of those two functions calls Layer.call, AutoCastVariables will still be casted.

  • tf.data:

    • tf.data.experimental.service.DispatchServer now takes a config tuple instead of individual arguments. Usages should be updated to tf.data.experimental.service.DispatchServer(dispatcher_config).
    • tf.data.experimental.service.WorkerServer now takes a config tuple instead of individual arguments. Usages should be updated to tf.data.experimental.service.WorkerServer(worker_config).

  • tf.distribute:

    • Removes tf.distribute.Strategy.experimental_make_numpy_dataset. Please use tf.data.Dataset.from_tensor_slices instead.
    • Renames experimental_hints in tf.distribute.StrategyExtended.reduce_to, tf.distribute.StrategyExtended.batch_reduce_to, tf.distribute.ReplicaContext.all_reduce to options:
    • Renames tf.distribute.experimental.CollectiveHints to tf.distribute.experimental.CommunicationOptions.
    • Renames tf.distribute.experimental.CollectiveCommunication to tf.distribute.experimental.CommunicationImplementation.
    • Renames tf.distribute.Strategy.experimental_distribute_datasets_from_function to distribute_datasets_from_function as it is no longer experimental.
    • Removes tf.distribute.Strategy.experimental_run_v2 method, which was deprecated in TF 2.2.

  • tf.lite:

    • tf.quantization.quantize_and_dequantize_v2 has been introduced, which updates the gradient definition for quantization which is outside the range to be 0. To simulate the V1 the behavior of tf.quantization.quantize_and_dequantize(...) use tf.grad_pass_through(tf.quantization.quantize_and_dequantize_v2)(...).

Bug Fixes and Other Changes

TF Core:

  • Introduces experimental support for a new module named tf.experimental.numpy, which is a NumPy-compatible API for writing TF programs. This module provides class ndarray, which mimics the ndarray class in NumPy, and wraps an immutable tf.Tensor under the hood. A subset of NumPy functions (e.g. numpy.add) are provided. Their inter-operation with TF facilities is seamless in most cases. See tensorflow/python/ops/numpy_ops/README.md for details of what operations are supported and what are the differences from NumPy.
  • tf.types.experimental.TensorLike is a new Union type that can be used as type annotation for variables representing a Tensor or a value that can be converted to Tensor by tf.convert_to_tensor.
  • Calling ops with a python constants or numpy values is now consistent with tf.converttotensor behavior. This avoids operations like tf.reshape truncating inputs such as from int64 to int32.
  • Adds tf.sparse.map_values to apply a function to the .values of SparseTensor arguments.
  • The Python bitwise operators for Tensor (__and__, __or__, __xor__ and __invert__ now support non-bool arguments and apply the corresponding bitwise ops. bool arguments continue to be supported and dispatch to logical ops. This brings them more in line with Python and NumPy behavior.
  • Adds tf.SparseTensor.with_values. This returns a new SparseTensor with the same sparsity pattern, but with new provided values. It is similar to the with_values function of RaggedTensor.
  • Adds StatelessCase op, and uses it if none of case branches has stateful ops.
  • Adds tf.config.experimental.get_memory_usage to return total memory usage of the device.
  • Adds gradients for RaggedTensorToVariant and RaggedTensorFromVariant.
  • Improve shape inference of nested function calls by supporting constant folding across Arg nodes which makes more static values available to shape inference functions.
    • tf.debugging:
  • tf.debugging.assert_shapes() now works on SparseTensors (Fixes #36268).
    • GPU
  • Adds Support for TensorFloat-32 on Ampere based GPUs. TensorFloat-32, or TF32 for short, is a math mode for NVIDIA Ampere based GPUs which causes certain float32 ops, such as matrix multiplications and convolutions, to run much faster on Ampere GPUs but with reduced precision. This reduced precision has not been found to effect convergence quality of deep learning models in practice. TensorFloat-32 is enabled by default, but can be disabled with tf.config.experimental.enable_tensor_float_32_execution.
    • tf.math:
  • Adds tf.math.erfcinv, the inverse to tf.math.erfc.
    • tf.nn:
  • tf.nn.max_pool2d now supports explicit padding.
    • tf.image:
  • Adds deterministic tf.image.stateless_random_* functions for each tf.image.random_* function. Added a new op stateless_sample_distorted_bounding_box which is a deterministic version of sample_distorted_bounding_box op. Given the same seed, these stateless functions/ops produce the same results independent of how many times the function is called, and independent of global seed settings.
  • Adds deterministic tf.image.resize backprop CUDA kernels for method=ResizeMethod.BILINEAR (the default method). Enable by setting the environment variable TF_DETERMINISTIC_OPS to "true" or "1".
    • tf.print:
  • Bug fix in tf.print() with OrderedDict where if an OrderedDict didn't have the keys sorted, the keys and values were not being printed in accordance with their correct mapping.
    • tf.train.Checkpoint:
  • Now accepts a root argument in the initialization, which generates a checkpoint with a root object. This allows users to create a Checkpoint object that is compatible with Keras model.save_weights() and model.load_weights. The checkpoint is also compatible with the checkpoint saved in the variables/ folder in the SavedModel.
  • When restoring, save_path can be a path to a SavedModel. The function will automatically find the checkpoint in the SavedModel.

tf.data:

  • Adds new tf.data.experimental.service.register_dataset and tf.data.experimental.service.from_dataset_id APIs to enable one process to register a dataset with the tf.data service, and another process to consume data from the dataset.
  • Adds support for dispatcher fault tolerance. To enable fault tolerance, configure a work_dir when running your dispatcher server and set dispatcher_fault_tolerance=True. The dispatcher will store its state to work_dir, so that on restart it can continue from its previous state after restart.
  • Adds support for sharing dataset graphs via shared filesystem instead of over RPC. This reduces load on the dispatcher, improving performance of distributing datasets. For this to work, the dispatcher's work_dir must be accessible from workers. If the worker fails to read from the work_dir, it falls back to using RPC for dataset graph transfer.
  • Adds support for a new "distributed_epoch" processing mode. This processing mode distributes a dataset across all tf.data workers, instead of having each worker process the full dataset. See the tf.data service docs to learn more.
  • Adds optional exclude_cols parameter to CsvDataset. This parameter is the complement of select_cols; at most one of these should be specified.
  • We have implemented an optimization which reorders data-discarding transformations such as take and shard to happen earlier in the dataset when it is safe to do so. The optimization can be disabled via the experimental_optimization.reorder_data_discarding_ops dataset option.
  • tf.data.Options were previously immutable and can now be overridden.
  • tf.data.Dataset.from_generator now supports Ragged and Sparse tensors with a new output_signature argument, which allows from_generator to produce any type describable by a tf.TypeSpec.
  • tf.data.experimental.AUTOTUNE is now available in the core API as tf.data.AUTOTUNE.

tf.distribute:

  • Introduces experimental support for asynchronous training of Keras models via tf.distribute.experimental.ParameterServerStrategy:
    • Replaces the existing tf.distribute.experimental.ParameterServerStrategy symbol with a new class that is for parameter server training in TF2. Usage of the old symbol, usually with Estimator API, should be replaced with [tf.compat.v1.distribute.experimental.ParameterServerStrategy].
    • Added tf.distribute.experimental.coordinator.* namespace, including the main API ClusterCoordinator for coordinating the training cluster, the related data structure RemoteValue and PerWorkerValue.
  • Adds tf.distribute.Strategy.gather and tf.distribute.ReplicaContext.all_gather APIs to support gathering dense distributed values.
  • Fixes various issues with saving a distributed model.

tf.keras:

  • Improvements from the Functional API refactoring:
    • Functional model construction does not need to maintain a global workspace graph, removing memory leaks especially when building many models or very large models.
    • Functional model construction should be ~8-10% faster on average.
    • Functional models can now contain non-symbolic values in their call inputs inside of the first positional argument.
    • Several classes of TF ops that were not reliably converted to Keras layers during functional API construction should now work, e.g. tf.image.ssim_multiscale
    • Error messages when Functional API construction goes wrong (and when ops cannot be converted to Keras layers automatically) should be clearer and easier to understand.
  • Optimizer.minimize can now accept a loss Tensor and a GradientTape as an alternative to accepting a callable loss.
  • Adds beta hyperparameter to FTRL optimizer classes (Keras and others) to match FTRL paper.
  • Optimizer.__init__ now accepts a gradient_aggregator to allow for customization of how gradients are aggregated across devices, as well as gradients_transformers to allow for custom gradient transformations (such as gradient clipping).
  • Improvements to Keras preprocessing layers:
    • TextVectorization can now accept a vocabulary list or file as an init arg.
    • Normalization can now accept mean and variance values as init args.
  • In Attention and AdditiveAttention layers, the call() method now accepts a return_attention_scores argument. When set to True, the layer returns the attention scores as an additional output argument.
  • Adds tf.metrics.log_cosh and tf.metrics.logcosh API entrypoints with the same implementation as their tf.losses equivalent.
  • For Keras model, the individual call of Model.evaluate uses no cached data for evaluation, while Model.fit uses cached data when validation_data arg is provided for better performance.
  • Adds a save_traces argument to model.save/ tf.keras.models.save_model which determines whether the SavedModel format stores the Keras model/layer call functions. The traced functions allow Keras to revive custom models and layers without the original class definition, but if this isn't required the tracing can be disabled with the added option.
  • The tf.keras.mixed_precision API is non non-experimental. The non-experimental API differs from the experimental API in several ways.
    • tf.keras.mixed_precision.Policy no longer takes in a tf.mixed_precision.experimental.LossScale in the constructor, and no longer has a LossScale associated with it. Instead, Model.compile will automatically wrap the optimizer with a LossScaleOptimizer using dynamic loss scaling if Policy.name is "mixed_float16".
    • tf.keras.mixed_precision.LossScaleOptimizer's constructor takes in different arguments. In particular, it no longer takes in a LossScale, and there is no longer a LossScale associated with the LossScaleOptimizer. Instead, LossScaleOptimizer directly implements fixed or dynamic loss scaling. See the documentation of tf.keras.mixed_precision.experimental.LossScaleOptimizer for details on the differences between the experimental LossScaleOptimizer and the new non-experimental LossScaleOptimizer.
    • tf.mixed_precision.experimental.LossScale and its subclasses are deprecated, as all of its functionality now exists within tf.keras.mixed_precision.LossScaleOptimizer

tf.lite:

  • TFLiteConverter:
    • Support optional flags inference_input_type and inference_output_type for full integer quantized models. This allows users to modify the model input and output type to integer types (tf.int8, tf.uint8) instead of defaulting to float type (tf.float32).
  • NNAPI
    • Adds NNAPI Delegation support for requantization use cases by converting the operation into a dequantize-quantize pair.
    • Removes deprecated Interpreter.setUseNNAPI(boolean) Java API. Use Interpreter.Options.setUseNNAPI instead.
    • Deprecates Interpreter::UseNNAPI(bool) C++ API. Use NnApiDelegate() and related delegate configuration methods directly.
    • Deprecates Interpreter::SetAllowFp16PrecisionForFp32(bool) C++ API. Prefer controlling this via delegate options, e.g. tflite::StatefulNnApiDelegate::Options::allow_fp16' orTfLiteGpuDelegateOptionsV2::isprecisionloss_allowed`.
  • GPU
    • GPU acceleration now supports quantized models by default
  • DynamicBuffer::AddJoinedString() will now add a separator if the first string to be joined is empty.
  • Adds support for cumulative sum (cumsum), both as builtin op and MLIR conversion.

TensorRT

  • Issues a warning when the session_config parameter for the TF1 converter is used or the rewrite_config_template field in the TF2 converter parameter object is used.

TPU Enhancements:

  • Adds support for the beta parameter of the FTRL optimizer for TPU embeddings. Users of other TensorFlow platforms can implement equivalent behavior by adjusting the l2 parameter.

XLA Support:

  • xla.experimental.compile is deprecated, use tf.function(experimental_compile=True) instead.
  • Adds tf.function.experimental_get_compiler_ir which returns compiler IR (currently 'hlo' and 'optimized_hlo') for given input for given function.

Security:

Other:

  • We have replaced uses of "whitelist" and "blacklist" with "allowlist" and "denylist" where possible. Please see this list for more context.
  • Adds tf.config.experimental.mlir_bridge_rollout which will help us rollout the new MLIR TPU bridge.
  • Adds tf.experimental.register_filesystem_plugin to load modular filesystem plugins from Python

Thanks to our Contributors

This release contains contributions from many people at Google and external contributors.

8bitmp3, aaa.jq, Abhineet Choudhary, Abolfazl Shahbazi, acxz, Adam Hillier, Adrian Garcia Badaracco, Ag Ramesh, ahmedsabie, Alan Anderson, Alexander Grund, Alexandre Lissy, Alexey Ivanov, Amedeo Cavallo, anencore94, Aniket Kumar Singh, Anthony Platanios, Ashwin Phadke, Balint Cristian, Basit Ayantunde, bbbboom, Ben Barsdell, Benjamin Chetioui, Benjamin Peterson, bhack, Bhanu Prakash Bandaru Venkata, Biagio Montaruli, Brent M. Spell, bubblebooy, bzhao, cfRod, Cheng Chen, Cheng(Kit) Chen, Chris Tessum, Christian, chuanqiw, codeadminperitiae, COTASPAR, CuiYifeng, danielknobe, danielyou0230, dannyfriar, daria, DarrenZhang01, Denisa Roberts, dependabot[bot], Deven Desai, Dmitry Volodin, Dmitry Zakharov, drebain, Duncan Riach, Eduard Feicho, Ehsan Toosi, Elena Zhelezina, emlaprise2358, Eugene Kuznetsov, Evaderan-Lab, Evgeniy Polyakov, Fausto Morales, Felix Johnny, fo40225, Frederic Bastien, Fredrik Knutsson, fsx950223, Gaurav Singh, Gauri1 Deshpande, George Grzegorz Pawelczak, gerbauz, Gianluca Baratti, Giorgio Arena, Gmc2, Guozhong Zhuang, Hannes Achleitner, Harirai, HarisWang, Harsh188, hedgehog91, Hemal Mamtora, Hideto Ueno, Hugh Ku, Ian Beauregard, Ilya Persky, jacco, Jakub Beránek, Jan Jongboom, Javier Montalt Tordera, Jens Elofsson, Jerry Shih, jerryyin, jgehw, Jinjing Zhou, jma, jmsmdy, Johan Nordström, John Poole, Jonah Kohn, Jonathan Dekhtiar, jpodivin, Jung Daun, Kai Katsumata, Kaixi Hou, Kamil Rakoczy, Kaustubh Maske Patil, Kazuaki Ishizaki, Kedar Sovani, Koan-Sin Tan, Koki Ibukuro, Krzysztof Laskowski, Kushagra Sharma, Kushan Ahmadian, Lakshay Tokas, Leicong Li, levinxo, Lukas Geiger, Maderator, Mahmoud Abuzaina, Mao Yunfei, Marius Brehler, markf, Martin Hwasser, Martin Kubovčík, Matt Conley, Matthias, mazharul, mdfaijul, Michael137, MichelBr, Mikhail Startsev, Milan Straka, Ml-0, Myung-Hyun Kim, Måns Nilsson, Nathan Luehr, ngc92, nikochiko, Niranjan Hasabnis, nyagato00, Oceania2018, Oleg Guba, Ongun Kanat, OscarVanL, Patrik Laurell, Paul Tanger, Peter Sobot, Phil Pearl, PlusPlusUltra, Poedator, Prasad Nikam, Rahul-Kamat, Rajeshwar Reddy T, redwrasse, Rickard, Robert Szczepanski, Rohan Lekhwani, Sam Holt, Sami Kama, Samuel Holt, Sandeep Giri, sboshin, Sean Settle, settle, Sharada Shiddibhavi, Shawn Presser, ShengYang1, Shi,Guangyong, Shuxiang Gao, Sicong Li, Sidong-Wei, Srihari Humbarwadi, Srinivasan Narayanamoorthy, Steenu Johnson, Steven Clarkson, stjohnso98, Tamas Bela Feher, Tamas Nyiri, Tarandeep Singh, Teng Lu, Thibaut Goetghebuer-Planchon, Tim Bradley, Tomasz Strejczek, Tongzhou Wang, Torsten Rudolf, Trent Lo, Ty Mick, Tzu-Wei Sung, Varghese, Jojimon, Vignesh Kothapalli, Vishakha Agrawal, Vividha, Vladimir Menshakov, Vladimir Silyaev, VoVAllen, Võ Văn Nghĩa, wondertx, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yair Ehrenwald, Yasir Modak, Yasuhiro Matsumoto, Yimei Sun, Yiwen Li, Yixing, Yoav Ramon, Yong Tang, Yong Wu, yuanbopeng, Yunmo Koo, Zhangqiang, Zhou Peng, ZhuBaohe, zilinzhu, zmx

- C++
Published by tensorflow-jenkins about 5 years ago

tensorflow - TensorFlow 2.4.0-rc2

Release 2.4.0

Major Features and Improvements

  • tf.distribute introduces experimental support for asynchronous training of Keras models via the tf.distribute.experimental.ParameterServerStrategy API. Please see below for additional details.

  • MultiWorkerMirroredStrategy is now a stable API and is no longer considered experimental. Some of the major improvements involve handling peer failure and many bug fixes. Please check out the detailed tutorial on Multi-worker training with Keras.

  • Introduces experimental support for a new module named tf.experimental.numpy which is a NumPy-compatible API for writing TF programs. See the detailed guide to learn more. Additional details below.

  • Adds Support for TensorFloat-32 on Ampere based GPUs. TensorFloat-32, or TF32 for short, is a math mode for NVIDIA Ampere based GPUs and is enabled by default.

  • A major refactoring of the internals of the Keras Functional API has been completed, that should improve the reliability, stability, and performance of constructing Functional models.

  • Keras mixed precision API tf.keras.mixed_precision is no longer experimental and allows the use of 16-bit floating point formats during training, improving performance by up to 3x on GPUs and 60% on TPUs. Please see below for additional details.

  • TensorFlow Profiler now supports profiling MultiWorkerMirroredStrategy and tracing multiple workers using the sampling mode API.

  • TFLite Profiler for Android is available. See the detailed guide to learn more.

  • TensorFlow pip packages are now built with CUDA11 and cuDNN 8.0.2.

Breaking Changes

  • TF Core:

    • Certain float32 ops run in lower precsion on Ampere based GPUs, including matmuls and convolutions, due to the use of TensorFloat-32. Specifically, inputs to such ops are rounded from 23 bits of precision to 10 bits of precision. This is unlikely to cause issues in practice for deep learning models. In some cases, TensorFloat-32 is also used for complex64 ops. TensorFloat-32 can be disabled by running tf.config.experimental.enable_tensor_float_32_execution(False).
    • The byte layout for string tensors across the C-API has been updated to match TF Core/C++; i.e., a contiguous array of tensorflow::tstring/TF_TStrings.
    • C-API functions TF_StringDecode, TF_StringEncode, and TF_StringEncodedSize are no longer relevant and have been removed; see core/platform/ctstring.h for string access/modification in C.
    • tensorflow.python, tensorflow.core and tensorflow.compiler modules are now hidden. These modules are not part of TensorFlow public API.
    • tf.raw_ops.Max and tf.raw_ops.Min no longer accept inputs of type tf.complex64 or tf.complex128, because the behavior of these ops is not well defined for complex types.
    • XLA:CPU and XLA:GPU devices are no longer registered by default. Use TF_XLA_FLAGS=--tf_xla_enable_xla_devices if you really need them, but this flag will eventually be removed in subsequent releases.

  • tf.keras:

    • The steps_per_execution argument in compile() is no longer experimental; if you were passing experimental_steps_per_execution, rename it to steps_per_execution in your code. This argument controls the number of batches to run during each tf.function call when calling fit(). Running multiple batches inside a single tf.function call can greatly improve performance on TPUs or small models with a large Python overhead.
    • A major refactoring of the internals of the Keras Functional API may affect code that is relying on certain internal details:
    • Code that uses isinstance(x, tf.Tensor) instead of tf.is_tensor when checking Keras symbolic inputs/outputs should switch to using tf.is_tensor.
    • Code that is overly dependent on the exact names attached to symbolic tensors (e.g. assumes there will be ":0" at the end of the inputs, treats names as unique identifiers instead of using tensor.ref(), etc.)
    • Code that uses get_concrete_function to trace Keras symbolic inputs directly should switch to building matching tf.TensorSpecs directly and tracing the TensorSpec objects.
    • Code that relies on the exact number and names of the op layers that TensorFlow operations were converted into may have changed.
    • Code that uses tf.map_fn/tf.cond/tf.while_loop/control flow as op layers and happens to work before TF 2.4. These will explicitly be unsupported now. Converting these ops to Functional API op layers was unreliable before TF 2.4, and prone to erroring incomprehensibly or being silently buggy.
    • Code that directly asserts on a Keras symbolic value in cases where ops like tf.rank used to return a static or symbolic value depending on if the input had a fully static shape or not. Now these ops always return symbolic values.
    • Code already susceptible to leaking tensors outside of graphs becomes slightly more likely to do so now.
    • Code that tries directly getting gradients with respect to symbolic Keras inputs/outputs. Use GradientTape on the actual Tensors passed to the already- constructed model instead.
    • Code that requires very tricky shape manipulation via converted op layers in order to work, where the Keras symbolic shape inference proves insufficient.
    • Code that tries manually walking a tf.keras.Model layer by layer and assumes layers only ever have one positional argument. This assumption doesn't hold true before TF 2.4 either, but is more likely to cause issues now.
    • Code that manually enters keras.backend.get_graph() before building a functional model is no longer needed.
    • Start enforcing input shape assumptions when calling Functional API Keras models. This may potentially break some users, in case there is a mismatch between the shape used when creating Input objects in a Functional model, and the shape of the data passed to that model. You can fix this mismatch by either calling the model with correctly-shaped data, or by relaxing Input shape assumptions (note that you can pass shapes with None entries for axes that are meant to be dynamic). You can also disable the input checking entirely by setting model.input_spec = None.
    • Serveral changes have been made to tf.keras.mixed_precision.experimental. Note that it is now recommended to use the non-experimental tf.keras.mixed_precision API.
    • AutoCastVariable.dtype now refers to the actual variable dtype, not the dtype it will be casted to.
    • When mixed precision is enabled, tf.keras.layers.Embedding now outputs a float16 or bfloat16 tensor instead of a float32 tensor.
    • The property tf.keras.mixed_precision.experimental.LossScaleOptimizer.loss_scale is now a tensor, not a LossScale object. This means to get a loss scale of a LossScaleOptimizer as a tensor, you must now call opt.loss_scale instead of opt.loss_scale().
    • The property should_cast_variables has been removed from tf.keras.mixed_precision.experimental.Policy
    • When passing a tf.mixed_precision.experimental.DynamicLossScale to tf.keras.mixed_precision.experimental.LossScaleOptimizer, the DynamicLossScale's multiplier must be 2.
    • When passing a tf.mixed_precision.experimental.DynamicLossScale to tf.keras.mixed_precision.experimental.LossScaleOptimizer, the weights of the DynanmicLossScale are copied into the LossScaleOptimizer instead of being reused. This means modifying the weights of the DynamicLossScale will no longer affect the weights of the LossScaleOptimizer, and vice versa.
    • The global policy can no longer be set to a non-floating point policy in tf.keras.mixed_precision.experimental.set_policy
    • In Layer.call, AutoCastVariables will no longer be casted within MirroredStrategy.run or ReplicaContext.merge_call. This is because a thread local variable is used to determine whether AutoCastVariables are casted, and those two functions run with a different thread. Note this only applies if one of these two functions is called within Layer.call; if one of those two functions calls Layer.call, AutoCastVariables will still be casted.

  • tf.data:

    • tf.data.experimental.service.DispatchServer now takes a config tuple instead of individual arguments. Usages should be updated to tf.data.experimental.service.DispatchServer(dispatcher_config).
    • tf.data.experimental.service.WorkerServer now takes a config tuple instead of individual arguments. Usages should be updated to tf.data.experimental.service.WorkerServer(worker_config).

  • tf.distribute:

    • Removes tf.distribute.Strategy.experimental_make_numpy_dataset. Please use tf.data.Dataset.from_tensor_slices instead.
    • Renames experimental_hints in tf.distribute.StrategyExtended.reduce_to, tf.distribute.StrategyExtended.batch_reduce_to, tf.distribute.ReplicaContext.all_reduce to options:
    • Renames tf.distribute.experimental.CollectiveHints to tf.distribute.experimental.CommunicationOptions.
    • Renames tf.distribute.experimental.CollectiveCommunication to tf.distribute.experimental.CommunicationImplementation.
    • Renames tf.distribute.Strategy.experimental_distribute_datasets_from_function to distribute_datasets_from_function as it is no longer experimental.
    • Removes tf.distribute.Strategy.experimental_run_v2 method, which was deprecated in TF 2.2.

  • tf.lite:

    • tf.quantization.quantize_and_dequantize_v2 has been introduced, which updates the gradient definition for quantization which is outside the range to be 0. To simulate the V1 the behavior of tf.quantization.quantize_and_dequantize(...) use tf.grad_pass_through(tf.quantization.quantize_and_dequantize_v2)(...).

Bug Fixes and Other Changes

TF Core:

  • Introduces experimental support for a new module named tf.experimental.numpy, which is a NumPy-compatible API for writing TF programs. This module provides class ndarray, which mimics the ndarray class in NumPy, and wraps an immutable tf.Tensor under the hood. A subset of NumPy functions (e.g. numpy.add) are provided. Their inter-operation with TF facilities is seamless in most cases. See tensorflow/python/ops/numpy_ops/README.md for details of what operations are supported and what are the differences from NumPy.
  • tf.types.experimental.TensorLike is a new Union type that can be used as type annotation for variables representing a Tensor or a value that can be converted to Tensor by tf.convert_to_tensor.
  • Calling ops with a python constants or numpy values is now consistent with tf.converttotensor behavior. This avoids operations like tf.reshape truncating inputs such as from int64 to int32.
  • Adds tf.sparse.map_values to apply a function to the .values of SparseTensor arguments.
  • The Python bitwise operators for Tensor (__and__, __or__, __xor__ and __invert__ now support non-bool arguments and apply the corresponding bitwise ops. bool arguments continue to be supported and dispatch to logical ops. This brings them more in line with Python and NumPy behavior.
  • Adds tf.SparseTensor.with_values. This returns a new SparseTensor with the same sparsity pattern, but with new provided values. It is similar to the with_values function of RaggedTensor.
  • Adds StatelessCase op, and uses it if none of case branches has stateful ops.
  • Adds tf.config.experimental.get_memory_usage to return total memory usage of the device.
  • Adds gradients for RaggedTensorToVariant and RaggedTensorFromVariant.
  • Improve shape inference of nested function calls by supporting constant folding across Arg nodes which makes more static values available to shape inference functions.
    • tf.debugging:
  • tf.debugging.assert_shapes() now works on SparseTensors (Fixes #36268).
    • GPU
  • Adds Support for TensorFloat-32 on Ampere based GPUs. TensorFloat-32, or TF32 for short, is a math mode for NVIDIA Ampere based GPUs which causes certain float32 ops, such as matrix multiplications and convolutions, to run much faster on Ampere GPUs but with reduced precision. This reduced precision has not been found to effect convergence quality of deep learning models in practice. TensorFloat-32 is enabled by default, but can be disabled with tf.config.experimental.enable_tensor_float_32_execution.
    • tf.math:
  • Adds tf.math.erfcinv, the inverse to tf.math.erfc.
    • tf.nn:
  • tf.nn.max_pool2d now supports explicit padding.
    • tf.image:
  • Adds deterministic tf.image.stateless_random_* functions for each tf.image.random_* function. Added a new op stateless_sample_distorted_bounding_box which is a deterministic version of sample_distorted_bounding_box op. Given the same seed, these stateless functions/ops produce the same results independent of how many times the function is called, and independent of global seed settings.
  • Adds deterministic tf.image.resize backprop CUDA kernels for method=ResizeMethod.BILINEAR (the default method). Enable by setting the environment variable TF_DETERMINISTIC_OPS to "true" or "1".
    • tf.print:
  • Bug fix in tf.print() with OrderedDict where if an OrderedDict didn't have the keys sorted, the keys and values were not being printed in accordance with their correct mapping.
    • tf.train.Checkpoint:
  • Now accepts a root argument in the initialization, which generates a checkpoint with a root object. This allows users to create a Checkpoint object that is compatible with Keras model.save_weights() and model.load_weights. The checkpoint is also compatible with the checkpoint saved in the variables/ folder in the SavedModel.
  • When restoring, save_path can be a path to a SavedModel. The function will automatically find the checkpoint in the SavedModel.

tf.data:

  • Adds new tf.data.experimental.service.register_dataset and tf.data.experimental.service.from_dataset_id APIs to enable one process to register a dataset with the tf.data service, and another process to consume data from the dataset.
  • Adds support for dispatcher fault tolerance. To enable fault tolerance, configure a work_dir when running your dispatcher server and set dispatcher_fault_tolerance=True. The dispatcher will store its state to work_dir, so that on restart it can continue from its previous state after restart.
  • Adds support for sharing dataset graphs via shared filesystem instead of over RPC. This reduces load on the dispatcher, improving performance of distributing datasets. For this to work, the dispatcher's work_dir must be accessible from workers. If the worker fails to read from the work_dir, it falls back to using RPC for dataset graph transfer.
  • Adds support for a new "distributed_epoch" processing mode. This processing mode distributes a dataset across all tf.data workers, instead of having each worker process the full dataset. See the tf.data service docs to learn more.
  • Adds optional exclude_cols parameter to CsvDataset. This parameter is the complement of select_cols; at most one of these should be specified.
  • We have implemented an optimization which reorders data-discarding transformations such as take and shard to happen earlier in the dataset when it is safe to do so. The optimization can be disabled via the experimental_optimization.reorder_data_discarding_ops dataset option.
  • tf.data.Options were previously immutable and can now be overridden.
  • tf.data.Dataset.from_generator now supports Ragged and Sparse tensors with a new output_signature argument, which allows from_generator to produce any type describable by a tf.TypeSpec.
  • tf.data.experimental.AUTOTUNE is now available in the core API as tf.data.AUTOTUNE.

tf.distribute:

  • Introduces experimental support for asynchronous training of Keras models via tf.distribute.experimental.ParameterServerStrategy:
    • Replaces the existing tf.distribute.experimental.ParameterServerStrategy symbol with a new class that is for parameter server training in TF2. Usage of the old symbol, usually with Estimator API, should be replaced with [tf.compat.v1.distribute.experimental.ParameterServerStrategy].
    • Added tf.distribute.experimental.coordinator.* namespace, including the main API ClusterCoordinator for coordinating the training cluster, the related data structure RemoteValue and PerWorkerValue.
  • Adds tf.distribute.Strategy.gather and tf.distribute.ReplicaContext.all_gather APIs to support gathering dense distributed values.
  • Fixes various issues with saving a distributed model.

tf.keras:

  • Improvements from the Functional API refactoring:
    • Functional model construction does not need to maintain a global workspace graph, removing memory leaks especially when building many models or very large models.
    • Functional model construction should be ~8-10% faster on average.
    • Functional models can now contain non-symbolic values in their call inputs inside of the first positional argument.
    • Several classes of TF ops that were not reliably converted to Keras layers during functional API construction should now work, e.g. tf.image.ssim_multiscale
    • Error messages when Functional API construction goes wrong (and when ops cannot be converted to Keras layers automatically) should be clearer and easier to understand.
  • Optimizer.minimize can now accept a loss Tensor and a GradientTape as an alternative to accepting a callable loss.
  • Adds beta hyperparameter to FTRL optimizer classes (Keras and others) to match FTRL paper.
  • Optimizer.__init__ now accepts a gradient_aggregator to allow for customization of how gradients are aggregated across devices, as well as gradients_transformers to allow for custom gradient transformations (such as gradient clipping).
  • Improvements to Keras preprocessing layers:
    • TextVectorization can now accept a vocabulary list or file as an init arg.
    • Normalization can now accept mean and variance values as init args.
  • In Attention and AdditiveAttention layers, the call() method now accepts a return_attention_scores argument. When set to True, the layer returns the attention scores as an additional output argument.
  • Adds tf.metrics.log_cosh and tf.metrics.logcosh API entrypoints with the same implementation as their tf.losses equivalent.
  • For Keras model, the individual call of Model.evaluate uses no cached data for evaluation, while Model.fit uses cached data when validation_data arg is provided for better performance.
  • Adds a save_traces argument to model.save/ tf.keras.models.save_model which determines whether the SavedModel format stores the Keras model/layer call functions. The traced functions allow Keras to revive custom models and layers without the original class definition, but if this isn't required the tracing can be disabled with the added option.
  • The tf.keras.mixed_precision API is non non-experimental. The non-experimental API differs from the experimental API in several ways.
    • tf.keras.mixed_precision.Policy no longer takes in a tf.mixed_precision.experimental.LossScale in the constructor, and no longer has a LossScale associated with it. Instead, Model.compile will automatically wrap the optimizer with a LossScaleOptimizer using dynamic loss scaling if Policy.name is "mixed_float16".
    • tf.keras.mixed_precision.LossScaleOptimizer's constructor takes in different arguments. In particular, it no longer takes in a LossScale, and there is no longer a LossScale associated with the LossScaleOptimizer. Instead, LossScaleOptimizer directly implements fixed or dynamic loss scaling. See the documentation of tf.keras.mixed_precision.experimental.LossScaleOptimizer for details on the differences between the experimental LossScaleOptimizer and the new non-experimental LossScaleOptimizer.
    • tf.mixed_precision.experimental.LossScale and its subclasses are deprecated, as all of its functionality now exists within tf.keras.mixed_precision.LossScaleOptimizer

tf.lite:

  • TFLiteConverter:
    • Support optional flags inference_input_type and inference_output_type for full integer quantized models. This allows users to modify the model input and output type to integer types (tf.int8, tf.uint8) instead of defaulting to float type (tf.float32).
  • NNAPI
    • Adds NNAPI Delegation support for requantization use cases by converting the operation into a dequantize-quantize pair.
    • Removes deprecated Interpreter.setUseNNAPI(boolean) Java API. Use Interpreter.Options.setUseNNAPI instead.
    • Deprecates Interpreter::UseNNAPI(bool) C++ API. Use NnApiDelegate() and related delegate configuration methods directly.
    • Deprecates Interpreter::SetAllowFp16PrecisionForFp32(bool) C++ API. Prefer controlling this via delegate options, e.g. tflite::StatefulNnApiDelegate::Options::allow_fp16' orTfLiteGpuDelegateOptionsV2::isprecisionloss_allowed`.
  • GPU
    • GPU acceleration now supports quantized models by default
  • DynamicBuffer::AddJoinedString() will now add a separator if the first string to be joined is empty.
  • Adds support for cumulative sum (cumsum), both as builtin op and MLIR conversion.

TensorRT

  • Issues a warning when the session_config parameter for the TF1 converter is used or the rewrite_config_template field in the TF2 converter parameter object is used.

TPU Enhancements:

  • Adds support for the beta parameter of the FTRL optimizer for TPU embeddings. Users of other TensorFlow platforms can implement equivalent behavior by adjusting the l2 parameter.

XLA Support:

  • xla.experimental.compile is deprecated, use tf.function(experimental_compile=True) instead.
  • Adds tf.function.experimental_get_compiler_ir which returns compiler IR (currently 'hlo' and 'optimized_hlo') for given input for given function.

Security:

Other:

  • We have replaced uses of "whitelist" and "blacklist" with "allowlist" and "denylist" where possible. Please see this list for more context.
  • Adds tf.config.experimental.mlir_bridge_rollout which will help us rollout the new MLIR TPU bridge.
  • Adds tf.experimental.register_filesystem_plugin to load modular filesystem plugins from Python

Thanks to our Contributors

This release contains contributions from many people at Google and external contributors.

8bitmp3, aaa.jq, Abhineet Choudhary, Abolfazl Shahbazi, acxz, Adam Hillier, Adrian Garcia Badaracco, Ag Ramesh, ahmedsabie, Alan Anderson, Alexander Grund, Alexandre Lissy, Alexey Ivanov, Amedeo Cavallo, anencore94, Aniket Kumar Singh, Anthony Platanios, Ashwin Phadke, Balint Cristian, Basit Ayantunde, bbbboom, Ben Barsdell, Benjamin Chetioui, Benjamin Peterson, bhack, Bhanu Prakash Bandaru Venkata, Biagio Montaruli, Brent M. Spell, bubblebooy, bzhao, cfRod, Cheng Chen, Cheng(Kit) Chen, Chris Tessum, Christian, chuanqiw, codeadminperitiae, COTASPAR, CuiYifeng, danielknobe, danielyou0230, dannyfriar, daria, DarrenZhang01, Denisa Roberts, dependabot[bot], Deven Desai, Dmitry Volodin, Dmitry Zakharov, drebain, Duncan Riach, Eduard Feicho, Ehsan Toosi, Elena Zhelezina, emlaprise2358, Eugene Kuznetsov, Evaderan-Lab, Evgeniy Polyakov, Fausto Morales, Felix Johnny, fo40225, Frederic Bastien, Fredrik Knutsson, fsx950223, Gaurav Singh, Gauri1 Deshpande, George Grzegorz Pawelczak, gerbauz, Gianluca Baratti, Giorgio Arena, Gmc2, Guozhong Zhuang, Hannes Achleitner, Harirai, HarisWang, Harsh188, hedgehog91, Hemal Mamtora, Hideto Ueno, Hugh Ku, Ian Beauregard, Ilya Persky, jacco, Jakub Beránek, Jan Jongboom, Javier Montalt Tordera, Jens Elofsson, Jerry Shih, jerryyin, jgehw, Jinjing Zhou, jma, jmsmdy, Johan Nordström, John Poole, Jonah Kohn, Jonathan Dekhtiar, jpodivin, Jung Daun, Kai Katsumata, Kaixi Hou, Kamil Rakoczy, Kaustubh Maske Patil, Kazuaki Ishizaki, Kedar Sovani, Koan-Sin Tan, Koki Ibukuro, Krzysztof Laskowski, Kushagra Sharma, Kushan Ahmadian, Lakshay Tokas, Leicong Li, levinxo, Lukas Geiger, Maderator, Mahmoud Abuzaina, Mao Yunfei, Marius Brehler, markf, Martin Hwasser, Martin Kubovčík, Matt Conley, Matthias, mazharul, mdfaijul, Michael137, MichelBr, Mikhail Startsev, Milan Straka, Ml-0, Myung-Hyun Kim, Måns Nilsson, Nathan Luehr, ngc92, nikochiko, Niranjan Hasabnis, nyagato00, Oceania2018, Oleg Guba, Ongun Kanat, OscarVanL, Patrik Laurell, Paul Tanger, Peter Sobot, Phil Pearl, PlusPlusUltra, Poedator, Prasad Nikam, Rahul-Kamat, Rajeshwar Reddy T, redwrasse, Rickard, Robert Szczepanski, Rohan Lekhwani, Sam Holt, Sami Kama, Samuel Holt, Sandeep Giri, sboshin, Sean Settle, settle, Sharada Shiddibhavi, Shawn Presser, ShengYang1, Shi,Guangyong, Shuxiang Gao, Sicong Li, Sidong-Wei, Srihari Humbarwadi, Srinivasan Narayanamoorthy, Steenu Johnson, Steven Clarkson, stjohnso98, Tamas Bela Feher, Tamas Nyiri, Tarandeep Singh, Teng Lu, Thibaut Goetghebuer-Planchon, Tim Bradley, Tomasz Strejczek, Tongzhou Wang, Torsten Rudolf, Trent Lo, Ty Mick, Tzu-Wei Sung, Varghese, Jojimon, Vignesh Kothapalli, Vishakha Agrawal, Vividha, Vladimir Menshakov, Vladimir Silyaev, VoVAllen, Võ Văn Nghĩa, wondertx, xiaohong1031, Xiaoming (Jason) Cui, Xinan Jiang, Yair Ehrenwald, Yasir Modak, Yasuhiro Matsumoto, Yimei Sun, Yiwen Li, Yixing, Yoav Ramon, Yong Tang, Yong Wu, yuanbopeng, Yunmo Koo, Zhangqiang, Zhou Peng, ZhuBaohe, zilinzhu, zmx

- C++
Published by tensorflow-jenkins over 5 years ago