Releases | Open Source Science

ffilesystem - bugfixes

enhance drop_slash() and split() correctness
add os_version()
CMake: remove full configure Fortran link check as it gave false negative on some systems e.g. Cray
realpath: more posixy check
make tests more robust

- C++
Published by scivision about 1 year ago

ffilesystem - Windows corrections, add functions

Numerous internal corrections and efficiency improvements. Improve test coverage.

add isprefix(), isappexecalias(), isfifo(), space_capacity()

rename() clobber destination if it exists

correct mkdir() permissions check

Windows: detect and use correctly App Execution Aliases

which: find_all = true now returns all matches instead of default first match only

which(): on Windows use SearchPath() to find executables

Windows: is_reserved() bugfix "." is not reserved

Windows: is_symlink() distinguish between reparse points and symlinks

is_exe(): file or directory can be executable

- C++
Published by scivision about 1 year ago

ffilesystem - refinements

CI for macOS, Windows: add Flang Fortran compiler case

add functions:

fscpplang(): show C++ standard
fsclang(): show C standard
fs_rename(): rename (move) file
fsnonascii(): tell if string has non-ASCII characters
fs_libcxx(): tell libc++ version and vendor
fs_libc(): tell libc version and vendor
fsismsvc(): tell if compiler is MSVC-like (Visual Studio, oneAPI with MSVC interface)
fsisappleclang(): tell if AppleClang compiler

Improvements:

absolute: simplify code
copy_file: simplify code, improve error checking
remove: simplify code
is_reserved: cover more situations
mkdir: improve for Windows
drop_slash: drop duplicated interior slashes too
fsasposix: only C++ implementation (accessible from C, Fortran)
fsjoin, relative, proximate, withsuffix: does not normalize
fs_touch: use ofstream to simplify

Tests:

add several C++ tests, to improve test coverage across compilers
allow tests to run in parallel (cmake)

- C++
Published by scivision about 1 year ago

ffilesystem - Fallback to plain C++ if <filesystem> not available

no longer uses downloaded library CWalk for fallback
maintainability / reliability improved by plain C++17 fallback if not working

add isrosetta(), getlocalename(), fsbackend(), hardlinkcount(), isempty(), rootname(), stdin_tty()

remove fs_cpp()

read_symlink(): also work on Windows

Linux glibc: optional statx() for several functions

default to C++17

We also did work to in the future add Boost::filesystem optional backend (future work in "boost" Git branch)

- C++
Published by scivision over 1 year ago

ffilesystem - last Release with CWalk

planning to transition away from CWalk by using C++17 with fallback to C++17 w/o <filesystem>

- C++
Published by scivision over 1 year ago

ffilesystem -

Numerous upgrades and robustness improvements, including

exe_path(): also work for BSD
is_dir(): Windows needs a trailing slash on C: for it to be a directory formally
CMake: don't force build type / optimization
stem(): handle special case a/..
C++ mkdtemp(): fallback to C if mt19937 not available
which(): fixed bug that made it only work once
canonical(), resolve(): add expand_tilde option

Added functions

get_shell() to get shell that called program
fs_terminal() to get terminal emulation type
hostname() to get network hostname
fs_getpid() to get program PID
fs_realpath() as simple access to realpath() POSIX function
absolute() to get absolute path -- make_absolute() is deprecated
getownergroup() on UNIX to get file owner group
max_component() each filesystem allows maximum characters per segment
loadavg() get CPU load average UNIX for past one minute

CI: add GCC-7

- C++
Published by scivision over 1 year ago

ffilesystem - add functions, streamline configure checks

add functions

get_profile_dir() user_config_dir() get_username() getarg() get_owner()

streamline platform checks (speed up, simplify)

- C++
Published by scivision over 1 year ago

ffilesystem - use C for non C++17 filesystem functions

Generally use C99 functions except for functions directly using C++17 functions. This improves internal quality and maintenance by deduplicating code between C and C++ interfaces. We also broke up the source code into more internal source files for readability and IWYU clarity.

added functions fs_is_optimized() fs_cpu_arch() fs_is_android()

added optional "ok" argument to Fortran call touch(file, ok) and had touch() check that file is created--on Windows CI for example, touch would succeed, but the file wasn't actually created.

- C++
Published by scivision over 1 year ago

ffilesystem - numerous internal improvements, add cygwin

add Cygwin-specific "tocygpath()" "towinpath()"
denote const or constexpr where appropriate
overhaul mkdir for C (non-C++) to be more robust

- C++
Published by scivision over 1 year ago

ffilesystem - improve filesystem_type and tests and CXX standard set

oneAPI: use -fpscomp logicals to correctly handle C_BOOL. MUST NOT use -standard-semantics as it breaks linking with system libraries like MPI, even oneAPI's one oneMPI!
add more magic hex ids known
simplify/improve how C++ standard is set (default C++20, fallback C++17 minimum)
improve filesystem_type test
more correctly / succinctly detect needed C++ features

- C++
Published by scivision almost 2 years ago

ffilesystem - add filesystem_type(), devnull()

add filesystem_type() to determine filesystem / partition type e.g. NTFS, v9fs, etc.
add devnull() for path to null file e.g. nul on Windows, /dev/null otherwise
use CTAD feature macro for mkdtemp in case compiler is misisng this C++17 feature
internal efficiency fixes
use std::string instead of std::make_unique for simpler and safer code in several functions

- C++
Published by scivision almost 2 years ago

ffilesystem - bugfix, enhance tests

C:Windows: space: always calculate root internally

Numerous bugfixes, simplification, and improved test coverage.

- C++
Published by scivision almost 2 years ago

ffilesystem - internal refactor and bugfixes, optional path_t

Fortran 2003 path_t is now an option, default enabled, that tests if Fortran 2003 type is available. Almost all compilers available today are capable of type.

CMake >= 3.15 capable.

- C++
Published by scivision almost 2 years ago

ffilesystem - simplify, bugfix, speedup

Improve maintainability by breaking up filesystem.cpp and filesystem.c into multiple source files of similar types of functions.

add set_modtime(), is_alpha()

C parent() Windows: fix parent(c:) and test (C++ already OK)

C read_symlink(): error if truncation detected

test:parent: own file. parent('') == '.' this is a more generally correct output.

add [[nodiscard]] to appropriate functions.

- C++
Published by scivision almost 2 years ago

ffilesystem - Speed, add add get_modtime(), proximate_to(), fix relative_to()

add getmodtime(), proximateto()

fix relative_to()

speedup functions by removing non-essential normal()

Refactor Fortran code to single file + include files to greatly reduce duplicate code

- C++
Published by scivision almost 2 years ago

ffilesystem - add is_safe_name(), remove exe_dir(), lib_dir(), benchmarks

add issafename() to determine if a filename should be safe generally (no disallowed characters)
remove exedir(), libdir() as they are just parent(exepath(), parent(libpath())
speed up expanduser() by 4x to 10x
speed which() ~ 20%
add benchmarks for functions we wrote (vs. built into libstdc++)

- C++
Published by scivision almost 2 years ago

ffilesystem - bugfixes, add setenv()

cmake: skip non-essential checks to save config time

auto-detect fortran

bugfix: expanduser use deduped

c++:which: bugfix: handle undefined PATH

add setenv()

- C++
Published by scivision almost 2 years ago

ffilesystem - simplify macOS, better as_posix, Remove as_windows,as_cygpath

Remove as_windows, as_cygpath as the algorithm is not generally tractable (and not available in other pathlib-like packages)

Improve as_posix similar to other pathlib-like packages

macOS: remove problematic TargetConditionals.h, no need for this, use defines like countless other packages.

- C++
Published by scivision almost 2 years ago

ffilesystem - C++ Ffs class, improve internals

Numerous little improvements/bugfixes/added tests to ensure consistency and performance and robustness

C++: put many functions under Ffs:: class for namespacing.

C++ REPL: use std::mapstd::function as best practice demonstration.

- C++
Published by scivision almost 2 years ago

ffilesystem - add REPL, pathsep(), is_subdir()

added functions: pathsep() issubdir() removed macro FSMAX_PATH--use function as better code practice (C,C++ only)

numerous internal code quality and test improvements

added REPL interactive program for C++ and Fortran to allow use of most features in a user-friendly way.

- C++
Published by scivision almost 2 years ago

ffilesystem - add read_symlink(), create_symlink() return bool

create_symlink() return bool instead of int

add read_symlink()

improve test layout to help avoid duplicated test logic

- C++
Published by scivision about 2 years ago

ffilesystem - CMake >= 3.17, add Windows long2short, short2long, improvements

CMake >= 3.17 required for stability -- it is widely available

Corrected accidentally requiring C++20 compiler support (now again works if only C++17 available).

chmod_exe() is superseded by set_permissions()

Add Windows-only, C++-only long2short() short2long() for 8.3 filenames

Added is_readable(), is_writeable()

create_directories(), copy_file() no longer use system call exec()

Minor notes: * older Intel oneAPI also needs stdc++fs * app: add remove and touch * Windows non-C++: partially enable for internal dev only * cwalk 1.2.9 (for non-C++ only) * is_wsl: return -1 if cannot determine WSL * add permissions test for C++/C/Fortran

- C++
Published by scivision about 2 years ago

ffilesystem - improve packaging

Make fortran-filesystem work nicely as a child project via FetchContent and/or Git Submodule. Reenable oneAPI CI

- C++
Published by scivision about 2 years ago

ffilesystem - support BSD, add is_bsd, is_admin, which; trim trailing separator

add support for BSD for almost all functions

add functions: isbsd, isadmin, which()

trim trailing whitespace

parent() is parent directory

Note: since its introduction, chmod_exe does not work correctly on Windows, regardless of compiler

- C++
Published by scivision about 2 years ago

ffilesystem - add resolve(), is_mingw(), lang()

correct / distinguish behavior of canonical() from resolve(). resolve() makes a relative, non-existing path absolute by basing on cwd. canonical does not do that for non-existing path
is_mingw() added
dynamically test if MINGW symlink working rather than assume workaround needed
add lang() to print the C or C++ standard macro level
only test for utsname.h if non-cpp
NVHPC: remove unneeded flags
use static_assert() for clarity
error at configure time if non-cpp attempted on WIndows
allow user to set CMAKECXXSTANDARD. fscompiler() needs CXXSTANDARD >= 20, but this is an information-only print function. Of course at least CXX_STANDARD >= 17 needed.

- C++
Published by scivision over 2 years ago

ffilesystem - C++ exception handling for abnormal faults

For abnormal runtime conditions (empty path for mkdir) etc. that are a definite failure, throw / catch C++ exceptions. This considerably simplifies the code. However, for the case like Clang with Gfortran on Apple where the ABI is incompatible, don't rely on exceptions for essential functionality like checking file existence.

use CheckPIESupported so that PIE is done right
eliminate common/windows.c for conciseness
make tests skip only shaky parts (apple clang + gfortran) rather than whole test.

- C++
Published by scivision over 2 years ago

ffilesystem - add make_tempdir, improve fs_check

improve fs_check on some systems with Clang compiler so that C++ capabilities are more accurately tested
add make_tempdir() function (like mkdtemp) using c++ stdlib std::mt19937 Mersenne Twister PRNG for a unique directory name. This function is not secure. This function does not automatically remove the tempdir.

- C++
Published by scivision over 2 years ago

ffilesystem - C++17 string_view inputs, add set_cwd, homedir more robust

For performance & robustness, use C++17 std::string_view for C++ inputs generally. This is generally better than const std::string&.

add set_cwd() function which is just like chdir()

Make get_homedir, used by expanduser, canonical, etc. use backup methods if env var HOME or USERPROFILE isn't set.

as_posix() isn't implicitly used by canonical and similar anymore. The user is responsible to use appropriate escaping and file separators.

- C++
Published by scivision over 2 years ago

ffilesystem - add is_wsl(), use C++14 smart pointers

add is_wsl() to detect if running on Windows subsystem for Linux
Use C++14 smart pointers for quality, to avoid any use of "new" or "delete" that can cause memory leaks etc.
Consolidate compiler ABI testing (C++,C,Fortran) to be more useful / comprehensive and faster.

- C++
Published by scivision over 2 years ago

ffilesystem - Run path Cmake defaults

No longer overrides CMake default run paths (for shared build)

- C++
Published by scivision over 2 years ago

ffilesystem - add get_permissions, fallback no-C++ is Linux only

add get_permission() that returns a 9-character string of POSIX file permissions like for 644: "rw-r--r--" https://unix.stackexchange.com/questions/71585/convert-ls-l-output-format-to-chmod-format
macOS and Windows require C++. Linux can use C++ or non-C++ fallback functions. This is reduce maintenance of little-used option except for old HPC.
Improve test coverage

- C++
Published by scivision over 2 years ago

ffilesystem - simplify internals, Cygwin, cwalk external

Support Cygwin Simplify logic where possible put cwalk as ExternalProject move concepts to own repo

- C++
Published by scivision over 2 years ago

ffilesystem - add space_available, is_char_device, is_reserved. Update security, quality, test.

add functions:

space_available(): tells drive space available for this specified path
ischardevice(): is a path a special character device like a terminal, /dev/null
is_reserved(): for Windows, is a path reserved like NUL, COM1, etc. (always false on non-windows)

For Windows C-only internals, use COMSPEC env var instead of cmd.exe directory for create_directories() mkdir as a security best practice, inspired by Python 3.11.3 subprocess.

add testreserved for all functions to ensure predictable behavior for /dev/null, NUL, etc. With regard to fsequivalent/same_file, we treat any non-directory, non-regular-file as always unequal (like NaN or Infinity in IEEE754)

fsisunix() is now also true for macOS

copy_file(): error handling for case source file does not exist for stability/robustness

- C++
Published by scivision almost 3 years ago

ffilesystem - make ffilesystem::filesystem available via FetchContent

- C++
Published by scivision almost 3 years ago

ffilesystem - stability and quality

add -Dfallback=on default option, where if off it will fail at CMake configure time instead of fallback to C if C++ isn't working
rectify various clang-tidy lint for quality
add fs_compiler() function that tells what compiler version was used to build it
use goto to deduplicate code

- C++
Published by scivision almost 3 years ago

ffilesystem - Remove filesep(). C/C++ add null/empty tests

remove ill-defined filesep() function. Should keep filesep abstract from user anyway.
C/C++ add null/empty input tests for each function. Fix bugs thereby discovered
use _execlp for Windows to simplify

- C++
Published by scivision about 3 years ago

ffilesystem - MSVC build bugfix, add MSVC CI, improve touch/mkdir internals

fix build error for MSVC with cpp=off (uncommon use case0
add CI for MSVC
CMake package: add ffilesystemfortran parameter to indicate ffilesystem was built with Fortran (like ffilesystemcpp indicating C++ was used)
improve implementation of "touch" significantly with error checks
improve implementation of "mkdir" with error checks

- C++
Published by scivision about 3 years ago

ffilesystem - CMake/FPM lint. is_dir, is_file error catch. MinGW UX

CMake/FPM: lint/cleanup
isdir, isfile; C++: catch errors
MinGW: improve error messages and dedupe code for symlink

- C++
Published by scivision about 3 years ago

ffilesystem - code deduplication, CI improvements

Windows: deduplicate code for issymlink and createsymlink by use of internal functions
Windows: use MinGW workarounds as needed rather than for all Windows compilers
CI: simplify specifications and correct macOS tests. Use CMake workflow presets

- C++
Published by scivision about 3 years ago

ffilesystem - improve as_[posix,windows] robustness

improve as_[posix,windows] robustness by explicit internal allocation
re-add deprecated get_filename for now

- C++
Published by scivision over 3 years ago

ffilesystem - remove get_filename(), read/write_text(). work with CMake 3.11+

- C++
Published by scivision over 3 years ago

ffilesystem - add exe_dir(). package:link -ldl if needed. add tests

- C++
Published by scivision over 3 years ago

ffilesystem - add lib_dir()

libdir() works just like libpath(), but returns the directory of the shared library (lib_path() includes the filename)

- C++
Published by scivision over 3 years ago

ffilesystem - Fortran and/or C++ optional, use CWalk for some C fcns, fallback to C if C++ not working, increase C robustness

Fortran and/or C++ are now optional. That is, this can be used as a C-only library, C & Fortran or C++ & Fortran library.

The C library uses CWalk for some advanced C filesystem functions.

We now fallback to C if C++ filesystem not working

General increases in C library robustness.

- C++
Published by scivision over 3 years ago

ffilesystem - add lib_path(). rearchitecture to deduplicate interface code and C code

add lib_path() function that tells path to libfilesystem.[dll,dylib,so] if compiled as shared library

rearchitect code to deduplicate Fortran interfaces

fallback code is mostly in C now, for use from other C programs and for didactic purposes

remove Meson build system due to Meson bugs and since more probably use FPM now.

bugfix getenv to not crash if variable not found

more checks. Generally most function that return a path return empty string if something goes wrong

- C++
Published by scivision over 3 years ago

ffilesystem - improve symlink on Windows, add fallback functions, add exe_path

Windows (GCC, Visual Studio / oneAPI): improve correctness of is_symlink and create_symlink -- for fallback and regular filesystem module

fallback: add functions: isexe, chmodexe, chmodnoexe, issymlink, createsymlink

for fallback and full filesystem: add exe_path() function that returns full path to main executable (not of the library, but the main() containing or program containing binary

- C++
Published by scivision over 3 years ago

ffilesystem - Cpack correct package parameters

- C++
Published by scivision over 3 years ago

ffilesystem - allocated buffers, support more compilers

Using explicitly allocated character instead of implicitly allocated character for stability and to allow more compilers to work, including Flang, NVidia HPC SDK, AOCC AMD Optimizing Compilers and Cray standalone. Previously, Cray compiler required GCC or oneAPI backend.

- C++
Published by scivision over 3 years ago

ffilesystem - work with CMake 3.8+, cleanup scripts, improve toolchain

- C++
Published by scivision over 3 years ago

ffilesystem - more robust ABI check

since having the C, C++, and Fortran compilers work together is essential to this project, we include a more extensive configure time ABI check.

- C++
Published by scivision over 3 years ago

ffilesystem - maximum path dynamic per-computer

Instead of setting an arbitrary constant for maximum path, instead determine at compile-time the maximum path length of the system. This is done in filesystem.h for both the primary C++17 filesystem stdlib and fallback C-based filesystem API. There is an ultimate constant fallback of 256 maximum path length if no other limit can be found. This would be for systems we haven't tested with or a user hasn't contributed yet.

Used setproperty/getproperty in CMake as a general best practice.

Canonical: refactored to use separate input/output arguments for robustness internally.

as_posix() is removed as its implementation was overlapping with normal().

- C++
Published by scivision over 3 years ago

ffilesystem - bugfix: chmod_no_exe, remove redundant buffers

bugfix: chmodnoexe
remove redundant buffers and logic
fallback: add more functions
use FPM directory layout
put filesystem.h in include/

- C++
Published by scivision over 3 years ago

ffilesystem - add C example, improve efficiency, FPM layout

https://community.intel.com/t5/Intel-oneAPI-HPC-Toolkit/Regression-quot-ifx-quot-2022-1-0-20220316-with-file-quot-open/m-p/1383933#M9478

Intel 2022.1.0 20220316 introduced a new regression where open(file=) cannot use functions. As a workaround, we use a character(:), allocatable auxiliary variable. This is all internal to the package with probably zero practical performance impact.

We added a C language test for completeness.

- C++
Published by scivision almost 4 years ago

ffilesystem - add missing include, correct syntax

GCC 12 includes less in its stdlib, which revealed a missing include needed here. Corrected bitwise &,| to logical &&,||

- C++
Published by scivision almost 4 years ago

ffilesystem - enhance test for C++

put C++ test file as this helps reveal compiler backend issues such as Intel compiler with GCC 8 on Linux -- should use GCC 9, but this wasn't realized till using in user program until this new test was added.

- C++
Published by scivision almost 4 years ago

ffilesystem - improve package

package installed CMake script now adds needed flag for GCC < 9.1 tests: improve linker selection

- C++
Published by scivision almost 4 years ago

ffilesystem - add fallback_auto option

to avoid inadvertent loss of features when C++ filesystem is intended, we add a default-off fallback_auto bool.

- C++
Published by scivision almost 4 years ago

ffilesystem - restore non-C++ fallback

For compilers with broken libstdc++ or broken toolchains, we restore the last-resort fallback that doesn't use C++. This was tested with Cray PE without the GCC or Intel backends. Using Cray with GCC or Intel backends works with C++ filesystem. We provide an example of this in cray.cmake.

- C++
Published by scivision almost 4 years ago

ffilesystem - finer compiler capability tests

Some compilers e.g. Cray claim filesystem support, but their libstdc++ doesn't have filesystem. The new configure-time tests attempt to catch that problem. Cray compiler users should switch to the GCC or Intel backend so that libstdc++ is new enough for filesystem.

- C++
Published by scivision almost 4 years ago

ffilesystem - also usable from c/C++

Now the filesystem functions can also be used from C and C++ programs--see filesystem.h and the examples.

Removed unused CMake logic.

- C++
Published by scivision almost 4 years ago

ffilesystem - Optimize configure and CI

fail early in configure if compiler issue
optimize CI testing

- C++
Published by scivision almost 4 years ago

ffilesystem - Packaging for shared libs

add Rpath and MSVC options to make shared library packaging more robust, especially MSVC and oneAPI windows