Ffilesystem: platform-independent, compiler-agnostic path manipulation and filesystem library

Platform independent (Linux, macOS, Windows, Cygwin, WSL, BSD, ...) and compiler-agnostic Ffilesystem path manipulation library. Simplicity and efficiency are focuses of Ffilesystem. Ffilesystem backend is implemented in C++17 using <string_view> for simplicity and speed. If available, C++ standard library <filesystem> is used. The C++ backend accesses the C standard library to access filesystem and system parameters. Networked file systems and FUSE (e.g. SSHFS) are supported as well as local filesystems.

Ffilesystem supports UTF-8 via C++ std::string or Fortran character as the path input / output argument types.

Ffilesystem does not throw or catch C++ exceptions itself.

Ffilesystem header ffilesystem.h can be used from C and C++ project code--see example. The C interface allows reuse of Ffilesystem functions in other code languages such as Matlab.

The optional Fortran interface is built by default. Disable Fortran by

```sh cmake -Dffilesystem_fortran=false -Bbuild

or

meson setup -Dfortran=false build ```

Ffilesystem brings full, fast filesystem functionality to Fortran.

The language standards must be at least:

• C++17 standard library STL
• (optional) Fortran 2003

Ffilesystem works with popular C++ STL and C standard library implementations including: glibc, newlib, musl, Cosmopolitan universal binaries, macOS universal binaries, BSD libc, Microsoft CRT, among others. On Linux, symbol DEFAULTSOURCE is defined if needed to enable C standard library functions. RAII std::string buffers are used for all string representations of paths including for C API and system calls, and are automatically freed. Ffilesystem std::string buffers are dynamically sized according to the actual path length.

For Windows drive letters without a slash after the colon, the path is treated as a relative path. This is how the Windows API, ComSpec, and C++ STL work.

Inspired by (and benchmarked against) Python pathlib. Important Ffilesystem functions are benchmarked to help improve performance. Advanced / conceptual development takes place in ffilesystem-concepts.

Compiler support

Ffilesystem supports compilers including:

• GCC ≥ 7 (gcc/g++, gfortran)
• LLVM Clang ≥ 9 (clang/clang++, flang or gfortran)
• Intel oneAPI ≥ 2023.1 (icx, icpx, ifx)
• Intel Classic ≥ 2021.9 (icpc, ifort)
• AMD AOCC (clang/clang++, flang)
• NVIDIA HPC SDK (nvc++, nvfortran)
• Visual Studio (C/C++)
• Cray: using Cray compilers alone (cc, CC, ftn) or using GCC or Intel backend

To help debug with older compilers, disable C++ <filesystem>:

sh cmake -Bbuild -Dffilesystem_cpp=off

libstdc++

Some systems have broken, obsolete, or incompatible libstdc++.

Intel: If Intel compiler linker errors use GCC >= 9.1. This can be done by setting environment variable CXXFLAGS to the top level GCC >= 9.1 directory. Set environment variable CXXFLAGS for Intel GCC toolchain like:

sh export CXXFLAGS=--gcc-toolchain=/opt/rh/gcc-toolset-10/root/usr/

which can be determined like:

sh scl enable gcc-toolset-10 "which g++"

Build

Ffilesystem can be built with CMake, Meson, or Fortran Package Manager (FPM).

"libffilesystem.a" is the library binary built that contains the Fortran "filesystem" module--it is the only binary you need to use in your project.

Please see the API docs for extensive list of functions/subroutines.

Use any one of these methods to build Ffilesystem. The self-tests are optional and not built by default. The tests use GoogleTest framework.

Cross-compiling generally works (e.g. for MUSL) but the optional GoogleTest self-tests may not build.

CMake

sh cmake -B build cmake --build build

Optionally, build and run the self-tests:

sh cmake -B build -Dffilesystem_BUILD_TESTING=on cmake --build build ctest --test-dir build

The default library with CMake is static; to build shared library:

sh cmake -B build -DBUILD_SHARED_LIBS=on ...

Meson

sh meson setup build meson compile -C build

Optionally, build and run the self-tests:

sh meson setup -Dtest=true build --reconfigure meson test -C build

The default library with Meson is shared; to build static library:

sh meson setup -Ddefault_library=static build meson compile -C build

---

Fortran Package Manager (FPM):

```sh fpm --cxx-flag=-std=c++17 build

c++17 is the minimum, can use newer

```

---

GNU Make:

sh make

We provide Fortran REPL "filesystemcli" and C++ REPL "fscli" for interactive testing of Ffilesystem routines.

Build options

Fortran "filesystem" module contains OPTIONAL (enabled by default) Fortran type "patht" that contains properties and methods. The "patht" type uses getter and setter procedure to access the path as a string character(:), allocatable.

```fortran use filesystem, only : path_t

type(path_t) :: p

p = path_t("my/path") !< setter

print *, "path: ", p%path() !< getter ```

The CMake and Meson scripts detect if Fortran 2003 type is available and enable path_t by default. To manually enable / disable path_t with CMake set command option cmake -DHAVE_F03TYPE=1 or cmake -DHAVE_F03TYPE=0 respectively.

---

statx() is used by default on glibc version 2.28 or newer systems to get file information. There is a runtime fallback to "stat()" if "statx()" is not available. statx() may be disabled by setting build option

```sh cmake -Dffilesystem_statx=false -Bbuild

or

meson setup -Dstatx=false build ```

Self test

The optional self-tests provide reasonable coverage of the Ffilesystem library. Several of the tests use argv[0] as a test file. We are aware of the shortcomings of argv[0] to get the executable name. We provide the function fs_exepath() to get the executable path reliably.

Usage from other projects

The example directory contains a use pattern from external projects. One can either cmake --install build ffilesystem or use CMake ExternalProject or FetchContent from the other project. To find ffilesystem in your CMake project:

cmake find_package(ffilesystem CONFIG REQUIRED)

CMake package variables ffilesystem_cpp and ffilesystem_fortran indicate whether ffilesystem was built with C++ <filesystem> and/or Fortran support.

ffilesystem.cmake would be included from the other project to find or build Ffilesystem automatically. It provides the appropriate imported targets for shared or static builds, including Windows DLL handling.

Notes

GCC 6.x and older aren't supported due to lack of C++17 string_view support.

In C++ code, we generally use the size_type of the class as a best practice--for example std::string::size_type where appropriate instead of std::size_t. We use size_t at the C interfaces for clarity and also certain internal library calls. ssize_t is used in certain non-Windows internal-only function calls.

Possible future features

• inquire if a file is encrypted or compressed, etc.
• determine if the terminal is in a VT100 compatible mode

Other C++ filesystem libraries

Ffilesystem emphasizes simplicity and reasonable performance and reliability for scientific computing, particularly on HPC systems. A highly performance-oriented C++ low-level no TOCTOU filesystem library is LLFIO. An older C++ object-oriented interface is CppFS.

Other implementations of C++ filesystem include:

• Boost.Filesystem what the stdlib filesystem is based on, often tries newer features. Boost.Filesystem source code
• ghc-filesystem for older compilers.
• deprecated <experimental/filesystem> is missing vital lexical operations.

Other Fortran filesystem libraries

Other Fortran libraries that provide interfaces to filesystems include the following. Generally they have noticeably fewer functions than Ffilesystem. They typically implement many functions in Fortran, where with Ffilesystem we implement in C++ or C++ <filesystem> if available. Ffilesystem Fortran code is optional, and is just a thin wrapper around the C++ functions.

• stdlib_os
• fortyxima
• Fortran-stdlib
• M_system focuses on interfaces to libc

features we won't add

There is no "is_musl()" function due to MUSL designers not providing a MUSL feature macro.

Disk / partition formatting is something we won't add. We do have functions to report the partition type and available capacity of the disk partition.

Windows

Like Microsoft STL, Ffilesystem is not designed for UNC "long" paths. We recommend using a UNC path to a mapped drive letter. Windows long paths are partially implemented due to limitations. Boost.Filesystem handles long paths and we may implement Boost.Filesystem as an optional backend in the future.

Enable Windows developer mode to use symbolic links if needed.

C++ filesystem discussion

Security research led to TOCTOU-related patches to the C++ filesystem library in various C++ standard library implementations noted in that discussion. Ffilesystem does NOT use remove_all, which was the TOCTOU concern addressed above.

Since the underlying C++ filesystem is not thread-safe, race conditions can occur if multiple threads are accessing the same filesystem object regardless of the code language used in the Ffilesystem library.