https://github.com/rindow/rindow-neuralnetworks - Version 2.2.3

Release Notes: Version 2.2.3

Bugfix

• Fixed a bug in the sample code for the Transformer model that caused prediction errors in CPU mode.

- PHP
Published by yuichiis 11 months ago

https://github.com/rindow/rindow-neuralnetworks - Version 2.2.2

Release Notes: Version 2.2.2

Bugfix

• Fixed a bug where user-defined functions return incorrect gradients in special cases.

- PHP
Published by yuichiis 11 months ago

https://github.com/rindow/rindow-neuralnetworks - Version 2.2.1

Release Notes: Version 2.2.1

Sample Code Problem Fixes

• Fixed dataset issues in the transformer model sample code.
• Fixed missing changes for PHP 8.4 in the rnn-attention model sample code.

Preview Functions

These may change or be removed in future releases. - split: A split function that can be used with automatic differentiation.

- PHP
Published by yuichiis 11 months ago

https://github.com/rindow/rindow-neuralnetworks - Version 2.2.0

Release Notes: Version 2.2.0

New PHP Support

• Added support for PHP 8.4.
• Compatible with PHP 8.1, 8.2, 8.3, and 8.4.

New Features

• Added example code for a basic Transformer model.
• Automatic generation and propagation of tensor masks in layers is now supported.
• Added MaskedArray type arrays.
• Added mask_zero option to Embedding, supporting automatic generation of mask tensors.
• Added mask functionality to GRU, LSTM, and SimpleRNN layers.
• Overhauled mask functionality in the Attention layer to support mask propagation.
• ArraySpec can now be used in automatic differentiation expressions.
• onesLike can now be used in automatic differentiation expressions.
• Constants can now be defined using constant.
• Added ndarray function to convert from Variable to plain NDArray.
• Added SequentialDataset for handling data that cannot be loaded into memory.
• Added modules variable for storing multiple Layers and Models.

New Layers

• MultiHeadAttention: Multi-head attention layer for Transformer models.
• Add: Adds two arrays.
• EinsumDense: Dense layer using the einsum function.
• inheritMask: Inherits mask tensors from another array.

Backward Compatibility

• Compatible with version 2.1.

Known Issues

• The runtime environment depends on the supported platforms of the Rindow-Math-Matrix extension module. In basic mode with PHP only, it works on all platforms.
• Due to issues with the rindow-math-buffer-ffi module on macOS, it does not work in Advanced mode or Accelerated mode on macOS.

- PHP
Published by yuichiis 11 months ago

https://github.com/rindow/rindow-neuralnetworks - Type definitions

Complete review of type definitions

• Type definitions were applied to variables and arguments that did not have type definitions, a legacy of php7.

Bug when error occurs

• Fixed minor bugs that occur when errors occur

- PHP
Published by yuichiis almost 2 years ago

https://github.com/rindow/rindow-neuralnetworks - Version 2.1

Summary:

• New functions added to Gradient
  • BandPart,Cast,Equal,Get,Greater,Increment,Less,NotEqual,Ones,Repeat,Reshape,Scale,Shape,Transpose,Zeros,ZeroLike
• New layer added
  • Attention (significantly changed functionality), LayerNormalization,
• Metric feature added
  • BinaryAccuracy, CategoricalAccuracy, GenericMetric, MeanNorm2Error, MeanSquaredError, ScalarMetric, SparseCategoricalAccuracy
• Added scheduling feature to Optimizer
  • ExponentialDecay, InverseTimeDecay
• Compatible with Rindow Math Matrix version 2.0.

compatibility:

• The model save format has been changed due to changes in Rindow Math Matrix. Previous models and other data may not be loaded.
• The method for specifying array ranges has changed due to changes in Rindow Math Matrix. You may need to modify previous source code.
• Due to changes in Rindow Math Matrix, the external library call method has been changed from PHP Extension to PHP FFI. FFI must be enabled and available. There is no need to enable FFI if you do not use external libraries.

- PHP
Published by yuichiis almost 2 years ago

https://github.com/rindow/rindow-neuralnetworks - Fix save weights

Fix save optimizer weights with portable mode.

- PHP
Published by yuichiis almost 2 years ago

https://github.com/rindow/rindow-neuralnetworks - Fix warning errors in samples　

Fix warning errors in samples

- PHP
Published by yuichiis about 3 years ago

https://github.com/rindow/rindow-neuralnetworks - Update for PHP8.2

Update for PHP8.2 and add some functions.

** Update for PHP8.2

• Changed so that warning error is not displayed.

** Add functions

• BackEnd::std()
• Gradient::stopGradient() - Stop partial automatic differentiation
• Gradient::reduceSum() - reduceSum function
• Gradient::clipByValue() - clip by value

- PHP
Published by yuichiis about 3 years ago

https://github.com/rindow/rindow-neuralnetworks - Minor fixes when using OpenCL

Bugfix

• Error Backend::mean() on clblast

New Feature

• Option specification for clblast backend: It is now possible to specify options with rindowclblast in the environment variable RINDOWNEURALNETWORKSBACKEND. Specify platform and device like "rindowclblast::0,1". Or specify the device type like "rindowclblast::GPU".

- PHP
Published by yuichiis over 3 years ago

https://github.com/rindow/rindow-neuralnetworks - Fix PHP8.1 warnings in progress-bar

Fix PHP8.1 warnings in progress-bar

- PHP
Published by yuichiis over 3 years ago

https://github.com/rindow/rindow-neuralnetworks - Major version upgrade

Ver 2.0 has been released

Summary

• The entire framework has been rewritten to move from Define-and-run to Define-by-run.
• The experimentally implemented model format for developers is now officially available.
• You can also learn and infer with fit() and predict() as in the previous version.

Requirement

• PHP8.0 or PHP8.1
• rindow-math-matrix 1.2.0

Strong recommend

• rindow-openblas 0.3.0
• rindow-opencl 0.1.4
• rindow-clblast 0.1.2

- PHP
Published by yuichiis over 3 years ago

https://github.com/rindow/rindow-neuralnetworks - Add Layer and Loss func

Summary

• Add New Layers
• Add New Loss function
• Add New Optimizer
• Corresponds to cloning of model objects
• Model fit method supports multiple inputs
• Supports new version of libraries

New Layers

• Gather
• Max

New Loss function

• Huber

New Optimizer

• RMSprop

Supports new version of libraries

• rindow-math-matrix release 1.1.0
• rindow-openblas release 0.3.0

- PHP
Published by yuichiis over 3 years ago

https://github.com/rindow/rindow-neuralnetworks - composer config for ver.1

Changed the version-dependent information in the composer configuration file.

And we moved CI from travis-ci to Github Actions workflow.

- PHP
Published by yuichiis over 3 years ago

https://github.com/rindow/rindow-neuralnetworks - Brushup the sample code and the error messages on layers

Brushup neural-machine-translation sample - Update samples/neural-machine-translation-with-attention.php for bug-fixed Plot::imshow

New Layer - ExpandDims layer

Extend error messages - Tokenizer - AbstractBaseLayer - Attention - Concatinate

- PHP
Published by yuichiis over 4 years ago

https://github.com/rindow/rindow-neuralnetworks - Automatic differentiation function

Supported automatic differentiation function.

Custom model

• Added support for automatic differentiation on custom models.
• There is no need to write backwards or explicitly write calculation graphs.

Dynamically automatic differentiation

• This is an experimental feature.
• Added a mode that can execute the automatic differentiation function in the style of "define by run".
• In dynamic mode, a computational graph is built each time the function is executed.
• You can also use the dedicated model "DynamicModel" to describe it like a custom model.

New class

• Builder\Gradient
• Gradient\Core\GradientTape
• Gradient\Core\Undetermined
• Gradient\Core\Variable
• Gradient\Func\Add
• Gradient\Func\Div
• Gradient\Func\Exp
• Gradient\Func\Log
• Gradient\Func\Matmul
• Gradient\Func\Mul
• Gradient\Func\Sqrt
• Gradient\Func\Square
• Gradient\Func\Sub
• Model\DynamicModel

- PHP
Published by yuichiis over 4 years ago

https://github.com/rindow/rindow-neuralnetworks - Add ImageClassiedDataset to Builder

Add ImageClassiedDataset to Builder

• Rindow\NeuralNetworks\Builder\Data

- PHP
Published by yuichiis almost 5 years ago

https://github.com/rindow/rindow-neuralnetworks - Update Data preparation features

New features - ImageClassifiedDataset class - suffle, limit, restrictedbyclass options in ClassifiedDirectoryDataset class

New sample code - basic text classification

Fix bugs - the dtype Error at No blas PHP extensions in Sequence data preparation

- PHP
Published by yuichiis almost 5 years ago

https://github.com/rindow/rindow-neuralnetworks - New features

Add new Layers

• GlobalAveragePooling1D
• GlobalAveragePooling2D
• GlobalAveragePooling3D
• GlobalMaxPooling1D
• GlobalMaxPooling2D
• GlobalMaxPooling3D

Add new data preparation features

• Data\Dataset\CSVDataset
• Data\Dataset\ClassifiedDirectoryDataset
• Data\Dataset\NDArrayDataset
• Data\Image\ImageFilter
• Data\Sequence\TextClassifiedDataset

Extends feartures

• BatchNormalization Layer supports N-Dimensions
• AbstractModel supports Dataset interface
• AbstractModel displays progress bar with fit function
• Supports saveWeightsToFile and loadWeightsFromFile so that AbstractModel can save other than Sequential model
• Change returnattensionscore interface of Attention Layer

Fix bugs

• Save Embedding Layer bug

Sample code

• Organize the contents
• File name change

- PHP
Published by yuichiis almost 5 years ago

https://github.com/rindow/rindow-neuralnetworks - Update for Rindow-math-matrix 1.0.10

Improved some points.

• Changes due to changes in the interface of rindow-math-matrix.
• Solved test problems on OpenCL
• Slightly improved speed on OpenCL.

Requires

• rindow-math-matrix 1.0.10 for later

String Recommended

• rindow-openblas 0.2.2 or later
• rindow-math-plot

OpenCL supports

• Rindow CLBlast 0.1.2 or later
• Rindow OpenCL 0.1.1 or later

- PHP
Published by yuichiis about 5 years ago

https://github.com/rindow/rindow-neuralnetworks - OpenCL / CLBlast support

Supported CLBlast (OpenCL BLAS library) as a backend. This is experimental and has been tested only on Windows.

- PHP
Published by yuichiis about 5 years ago

https://github.com/rindow/rindow-neuralnetworks - Attention Layer Support

New Features

Layers

• Attention Layer
• Concatenate Layer

Data Processing

• Data\Sequence\Preprocessor
• Data\Sequence\Tokenizer

New Sample

• seq2seq with attention for a neural machine translation.

Requires

• rindow-math-matrix 1.0.7

Strong recommend

• rindow-openblas 0.1.5

- PHP
Published by yuichiis over 5 years ago

https://github.com/rindow/rindow-neuralnetworks - Recurrent Neural Network Support

Add Recurrent Layers and related Layers

• SimpleRNN, LSTM, GRU, Embedding, RepeatVector

Move from Activation Layers to Activation Functions and New Function

• ReLU, Sigmoid, Softmax, Tanh

New Kernel Initializer functions

• glorotuniform, glorotnormal, heuniform, henormal, randomuniform, randomnormal, orthogonal

Callback Function

• AbstractCallback

Custom Model

• AbstractModel

RNN Sample

• Seq2seq for the Numeric addition

Requires

• rindow-math-matrix 1.0.6

Strong recommend

• rindow-openblas extension 0.1.4

- PHP
Published by yuichiis over 5 years ago

https://github.com/rindow/rindow-neuralnetworks - Add dataset and Speed up around the random functions

New Datasets - cifar-10 - fashion-MNIST

sample code - cifar10-conv2d-classifivation

speed up - random functions in backend - dropout layer - sequential model

Bug fix - save model bug - zero devide in sparse categorical closs entropy - invalid kernel initialize

Requires - rindow-math-matrix 1.0.5

Strong recommend - rindow-openblas 0.1.3

- PHP
Published by yuichiis over 5 years ago

https://github.com/rindow/rindow-neuralnetworks - Convolutional neural network support

Add Convolution and Pooling Layers

• Conv1D, Conv2D, Conv3D
• MaxPooling1D, MaxPooling2D, MaxPooling3D
• AveragePooling1D, AveragePooling2D, AveragePooling3D
• Flatten

Add Sample code

• mnist-conv2d-clasification.php

Requires

• rindow-math-matrix 1.0.4

Strong recommend

• rindow-openblas extension 0.1.2

- PHP
Published by yuichiis over 5 years ago

https://github.com/rindow/rindow-neuralnetworks - Support select function of LinearAlgebra

For new Math/Matrix/LinearAlgebra

This speeds up mini-batch headings.

- PHP
Published by yuichiis almost 6 years ago

https://github.com/rindow/rindow-neuralnetworks - Rename ReLU layer

Rename the class of ReLU layer

change capitalize

• Rindow\NeuralNetworks\Layer
  • Relu => ReLU

- PHP
Published by yuichiis almost 6 years ago

https://github.com/rindow/rindow-neuralnetworks - Change math function name

Modified the method of each module according to the method name of Math class of php_openblas. Unify method names on PHP side.

Backend :: dmaximum => Backend :: greater Backend :: dminimum => Backend :: less

- PHP
Published by yuichiis about 6 years ago

https://github.com/rindow/rindow-neuralnetworks - Add and organize loss functions

Modification of Loss Function

• Change the integration mechanism between Loss function and Activation function.
• Add CategoricalCrossEntropy and BinaryCrossEntropy

- PHP
Published by yuichiis about 6 years ago

https://github.com/rindow/rindow-neuralnetworks - first public release

- PHP
Published by yuichiis about 6 years ago