Fortran-TF-lib

Code and examples for directly calling Tensorflow ML models from Fortran.
For calling PyTorch from Fortran see the FTorch repository.

Contents

• Description
• Installation
• Usage
• Examples
• License
• Contributions
• Authors and Acknowledgment
• Users

Description

It is desirable be able to run machine learning (ML) models directly in Fortran. Such models are often trained in some other language (say Python) using popular frameworks (say TensorFlow) and saved. We want to run inference on this model without having to call a Python executable. To achieve this we use the existing TensorFlow C interface.

This project provides a library enabling a user to directly couple their TensorFlow models to Fortran code. We provide installation instructions for the library as well as instructions and examples for performing coupling. This library implements only enough of the TensorFlow C API to allow inference, no training.

Project status: This project is currently in pre-release with documentation and code being prepared for a first release. As such breaking changes may be made. If you are interested in using this library please get in touch.

Installation

Dependencies

To install the library requires the following to be installed on the system:

• cmake >= 3.1
• TensorFlow C API, download from https://www.tensorflow.org/install/lang_c¹
• Fortran and C compilers

¹ Note that this page sometimes does not list the latest version of the library. You can try altering the library download URLs on the page to reflect the newest version. E.g. if the URL ends ...-2.11.tar.gz try changing it to ...-2.13.tar.gz.

Library installation

To build and install the library:

1. Navigate to the location in which you wish to install the source and run:
   git clone git@github.com:Cambridge-ICCS/fortran-tf-lib.git to clone via ssh, or
   git clone https://github.com/Cambridge-ICCS/fortran-tf-lib.git to clone via https.
   
2. Navigate into the library directory by running:
   cd fortran-tf-lib/fortran-tf-lib/

3. Create a build directory and execute cmake from within it using the relevant flags:
   mkdir build cd build cmake .. -DCMAKE_BUILD_TYPE=Release It is likely that you will need to provide at least the TENSORFLOW_LOCATION flag.
   The Fortran compiler must be the same one that you are planning to compile your Fortran code with. It is advisable to use C and Fortran compilers from the same provider.

   The following CMake flags are available and can be passed as arguments through -D<Option>=<Value>: | Option | Value | Description | | ------------------------------------------------------------------------------------------------- | ---------------------------- | --------------------------------------------------------------| | CMAKE_Fortran_COMPILER | ifort / gfortran | Specify a Fortran compiler to build the library with. This should match the Fortran compiler you're using to build the code you are calling this library from. | | CMAKE_C_COMPILER | icc / gcc | Specify a C compiler to build the library with. | | TENSORFLOW_LOCATION² | </path/to/tensorflow/> | Location of TensorFlow C API installation¹. | | CMAKE_INSTALL_PREFIX | </path/to/install/lib/at/> | Location at which the library files should be installed. By default this is /usr/local. | | CMAKE_BUILD_TYPE | Release / Debug | Specifies build type. The default is Debug, use Release for production code.|

   ² This should be the absolute path to where the TensorFlow C API mentioned in step 1 has been installed. CMake will look in TENSORFLOW_LOCATION and TENSORFLOW_LOCATION/lib for the TensorFlow library libtensorflow.so.

4. Make and install the code to the chosen location with: make make install This will place the following directories at the install location:

   • CMAKE_INSTALL_PREFIX/include/ - contains mod files
   • CMAKE_INSTALL_PREFIX/lib64/ - contains cmake directory and .so files

Usage

In order to use fortran-tf users will typically need to follow these steps:

1. Save a TensorFlow model in the Keras SavedModel format.
2. Write Fortran using the fortran-tf-lib bindings to use the model from within Fortran.
3. Build and compile the code, linking against fortran-tf-lib.

1. Saving the model

The trained model needs to be exported. This can be done from within your code using the model.save functionality from within python. Note that the TensorFlow C API currently (version 2.13) only supports the Keras "v2" format so you must specify format='tf': ``` import tensorflow as tf

construct model (e.g. model=tf.keras.Model(inputs, outputs))

or load one (e.g. model=tf.keras.models.load_model('/path/to/model'))

model.save("my_model", format='tf') ```

2. Using the model from Fortran

To use the trained TensorFlow model from within Fortran we need to import the TF_Interface module and use the binding routines to load the model, construct the tensors, and run inference.

A very simple example is given below. For more detailed documentation please consult the API documentation, source code, and examples.

This minimal snippet loads a saved TensorFlow model, creates an input consisting of a 1x32 matrix (with arbitrary values), and runs the model to infer the output. If you use the model provided in the test case this code will produce the indicated output value.

```fortran program testprogram use TFTypes use TFInterface use isoc_binding implicit none

type(TFSession) :: session type(TFSessionOptions) :: sessionoptions type(TFGraph) :: graph type(TFStatus) :: stat type(TFOutput), dimension(1) :: inputtfoutput, outputtfoutput character(100) :: vers character(100), dimension(1) :: tags type(TFTensor), dimension(1) :: inputtensors, outputtensors, testtensor type(TFOperation), dimension(1) :: target_opers

real, dimension(32), target :: rawdata real, dimension(:), pointer :: outputdataptr integer(kind=cint64t), dimension(2) :: inputdims integer(kind=cint64t), dimension(2) :: outputdims type(cptr) :: rawdataptr type(cptr) :: outputcdataptr

raw_data = (/ & 0.71332126, 0.81275973, 0.66596436, 0.79570779, 0.83973302, 0.76604397, & 0.84371391, 0.92582056, 0.32038017, 0.0732005, 0.80589203, 0.75226581, & 0.81602784, 0.59698078, 0.32991729, 0.43125108, 0.4368422, 0.88550326, & 0.7131253, 0.14951148, 0.22084413, 0.70801317, 0.69433906, 0.62496564, & 0.50744999, 0.94047845, 0.18191579, 0.2599102, 0.53161889, 0.57402205, & 0.50751284, 0.65207096 & /)

inputdims = (/ 1, 32 /) outputdims = (/ 1, 1 /) tags(1) = 'serve'

! Print TensorFlow library version call TF_Version(vers) write(,)'Tensorflow version', vers

sessionoptions = TFNewSessionOptions() graph = TFNewGraph() stat = TF_NewStatus()

! Load session (also populates graph) session = TF_LoadSessionFromSavedModel(sessionoptions, '/path/to/model', tags, 1, & graph, stat)

if (TFGetCode( stat ) .ne. TFOK) then call TFMessage( stat, vers ) write(,)'woops', TFGetCode( stat ), vers call abort endif

call TF_DeleteSessionOptions(sessionoptions)

inputtfoutput(1)%oper = TFGraphOperationByName( graph, "servingdefaultinput1" ) inputtfoutput(1)%index = 0 if (.not.cassociated(inputtfoutput(1)%oper%p)) then write(,)'input not associated' stop endif

outputtfoutput(1)%oper = TFGraphOperationByName( graph, "StatefulPartitionedCall" ) outputtfoutput(1)%index = 0 if (.not.cassociated(output_tfoutput(1)%oper%p)) then write(,)'output not associated' stop endif

! Bind the input tensor rawdataptr = cloc(rawdata) inputtensors(1) = TFNewTensor( TFFLOAT, inputdims, 2, rawdataptr, int(128, kind=csizet) )

! Run inference call TFSessionRun( session, inputtfoutput, inputtensors, 1, outputtfoutput, outputtensors, 1, & targetopers, 0, stat ) if (TFGetCode( stat ) .ne. TFOK) then call TFMessage( stat, vers ) write(,) TFGetCode( stat ), vers call abort endif

! Bind output tensor call cfpointer( TFTensorData( outputtensors(1)), outputdataptr, shape(outputdataptr) ) write(,)'output data', outputdataptr(1)

if ((outputdataptr(1) - -0.479371) .gt. 1e-6) then write(,)'Output does not match, FAILED!' else write(,)'Output is correct, SUCCESS!' endif

! Clean up call TFDeleteTensor( inputtensors(1) ) call TFDeleteTensor( outputtensors(1) ) call TFDeleteGraph( graph ) call TFDeleteSession( session, stat ) call TF_DeleteStatus( stat )

end program test_program ```

Generating code with process_model

The example code above illustrates a problem with the TensorFlow C API that our Fortran wrapper cannot fix. To load a model, the library requires that the caller knows certain rather opaque model parameters beforehand. Often, the values in the example above will work for the tags parameter to TF_LoadSessionFromSavedModel. However, the values needed for TF_GraphOperationByName (in this case serving_default_input_1, etc) are more likely to be different.

To address this, we provide a Python script, process_model that will read a Keras SavedModel and output a simple Fortran module intended to provide a base for the user to start from. The appropriate values will be read from the model and hard-coded into the Fortran code.

E.g. process_model -o fortran_code.f90 my_model

3. Build the code

The code now needs to be compiled and linked against our installed library.

CMake

If our project were using cmake we would need the following in the CMakeLists.txt file to find the the tf-lib installation and link it to the executable.

This can be done by adding the following to the CMakeLists.txt file: find_package(FortranTensorFlow) target_link_libraries( <executable> PRIVATE FortranTensorFlow::fortran-tf ) message(STATUS "Building with Fortran TensorFlow coupling") and using the -DCMAKE_PREFIX=</path/to/fortran-tf-libs/lib64/cmake> flag when running cmake.

When running the generated code you may also need to add the location of the .so files to your LD_LIBRARY_PATH unless installing in a default location: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path/to/fortran-tf-libs>/lib64

Examples

Examples of how to use this library will be provided in the examples directory.
They demonstrate different functionalities and are provided with instructions to modify, build, and run as necessary.

License

Copyright © ICCS

Fortran-TF-Lib is distributed under the MIT Licence.

Contributions

Contributions and collaborations are welcome.

For bugs, feature requests, and clear suggestions for improvement please open an issue.

If you have built something upon Fortran-TF-Lib that would be useful to others, or can address an open issue, please fork the repository and open a pull request.

Code of Conduct

Everyone participating in the Fortran-TF-Lib project, and in particular in the issue tracker, pull requests, and social media activity, is expected to treat other people with respect and, more generally, to follow the guidelines articulated in the Python Community Code of Conduct.

Authors and Acknowledgment

Fortran-TF-Lib is written and maintained by the ICCS

Notable contributors to this project are:

• @SimonClifford

See Contributors for a full list.

Used by

The following projects make use of this code or derivatives in some way:

• DataWave - MiMA ML

Are we missing anyone? Let us know.