Recent Releases of mat_gemini
mat_gemini - Work with Matlab R2020b and newer, and WSL
- MATLAB
Published by scivision over 3 years ago
mat_gemini - compatible with new Gemini3D packaging
Gemini3d v1.4.0 moved external packages to gemini3d/external
For MatGemini, the primary mission is to generate simulation inputs and plot simulation outputs. Running of simulations is just as easily accomplished without MatGemini, so we skip these tests "Incomplete" if there are conflicts between system libraries and Matlab's shared libraries.
- MATLAB
Published by scivision about 4 years ago
mat_gemini - robustness for run and offline setup improved
- gemini3d.run: print command being run
- gemini3d.run: use absolute path and print cmd
- fix gemini3d root finding, if more than one gemini3d on Path
- absolute_path: emit bad path if fails
- gemini3d.run: reenable MPIexec check and hint -- work with mpiexec having spaces in path
- gemini3d v1.2.1
- gemini3d.model.setup: print input summary like Fortran
- setup_homebrew: separate file, tells action
- ref_data.json is downloaded
- MATLAB
Published by scivision over 4 years ago
mat_gemini - more test coverage. Use CMake to extract .zst
- MATLAB
Published by scivision over 4 years ago
mat_gemini - build Gemini3D by default
- build Gemini3D by default for repeatability
- CI: build first to avoid build race as test runs in parallel
- MATLAB
Published by scivision over 4 years ago
mat_gemini - add neutral_BG, arbitrary env.var in config, explicit CMake
expand_envvar: work with any @envvar@ like Fortran
@scivision scivision committed on Aug 19
For repeatability, have user explicitly build with CMake. This avoids auto-rebuilds that may be unexpected.
readnml: read neutralBG
MSIS: expect /msis_version in input file for reproducability
- MATLAB
Published by scivision over 4 years ago
mat_gemini - require find.*, read.* like PyGemini
remove optional "required" argument and always error for find.* and read.* if file not found
- MATLAB
Published by scivision almost 5 years ago
mat_gemini - bugfix/enhance setup.m, default "gemini3d.run" name
- MATLAB
Published by scivision almost 5 years ago
mat_gemini - HDF5 write scalar
added functionality to matlab-stdlib used by MatGemini to write HDF5 scalars like Fortran and Python h5py do. This is necessary for Gemini v1.0.3 and newer
- MATLAB
Published by scivision almost 5 years ago
mat_gemini - use improve import system
this does require you to git submodule update --init if you already have mat_gemini
- MATLAB
Published by scivision about 5 years ago
mat_gemini - enhance .zstd
enhance metadata in/out with JSON
eqarchive instead of eqzip to avoid confusion
use SHA256 checksum and use CMake for checksum
- MATLAB
Published by scivision about 5 years ago
mat_gemini - allow .tar, .zstd reference and equilibrium archives
NOTE: now requires eqzip to be specified in config.nml if equrl is being used to download and extract. This avoids ambiguous situations.
Allow .zstd and .tar along with .zip to be used. We've found reliability issues with corrupted .zip and given broad acceptance of .zstd and faster, more compression we are using .zstd for archives for reference data and equilibrium data.
- MATLAB
Published by scivision about 5 years ago
mat_gemini - grid bugfix, allow @GEMINI_ROOT@ path replacement
MatGemini and PyGemini can now use environment variable GEMINISIMROOT to indicate the top-level directory of config.nml paths. In the config.nml, do entries like: eqdir = @GEMINISIMROOT@/tohoku2d where environment variable GEMINISIMROOT is set like ~/sims As usual, if one is testing / scripting, one can set temporary values for GEMINISIMROOT with "setenv()" in Matlab, or os.environ["GEMINISIMROOT"] in Python
- MATLAB
Published by scivision about 5 years ago
mat_gemini - bugfixes
quality improvements by more tolerant APIs
- MATLAB
Published by scivision over 5 years ago
mat_gemini - bugfix: read and plot for missing variables
more tolerant of missing variables
- MATLAB
Published by scivision over 5 years ago
mat_gemini - MSIS HDF5 2.0 API
requires Gemini3D >= 0.8.0. Uses HDF5 to interface with MSIS. optional MSIS 2.0
- MATLAB
Published by scivision over 5 years ago
mat_gemini - overhaul API gemini3d.{plot,read,write}
Make naming more descriptive like
matlab
gemini3d.VERB.NOUN
matlab
gemini3d.read.frame <= gemini3d.loadframe
and so on
- MATLAB
Published by scivision over 5 years ago
mat_gemini - Bugfixes
Numerous improvements to reliability, including:
- use JSON for config URLs instead of INI
- uniform setting of CMake policies
- add numerous error checks
- move Matlab CI from Travis-CI to Azure Pipelines
- MATLAB
Published by scivision over 5 years ago
mat_gemini - bugfix, add tests
- allow CMake to reconfigure if first configure fails, by deleting build directory
- redownload data zip file if bad
- add many tests to find and fix more corner cases
- switch more functions to return empty instead of "error" when file or variable missing.
- MATLAB
Published by scivision over 5 years ago
mat_gemini - add numerous tests, single core fallback
- add additional checks in files
- enhance linting
- fallback to single CPU core if MPIexec not working
- make selftests R2019b compatible
- MATLAB
Published by scivision over 5 years ago
mat_gemini - simplify public-facing API
- move non-public functions deeper in or private
- rename gemini3d.{run,plot} for convenience
- MATLAB
Published by scivision over 5 years ago
mat_gemini - bugfix: setup.m submodule update
There was a mistake in setup.m that caused the hdf5nc interface to be missing, since setup.m didn't detect it was missing. Now this is fixed. The errors caused before would be like
The import statement 'import hdf5nc.h5save' cannot be found or cannot be imported. Imported names must end with '.*' or be fully qualified.
- MATLAB
Published by scivision over 5 years ago
mat_gemini - add numerous tests. Fix loadframe standalone file
- add numerous self-tests
- enhance behavior of loading functions to be tolerant of missing files, which is useful for loading a single data file. This was an early feature of PyGemini now brought here.
- fix bug with loadframe() that required adding "./" in front of filename in same directory pwd
Dev note: test coverage ~ 50% due to a lot of raw data loading functions that are rarely used.
- MATLAB
Published by scivision over 5 years ago
mat_gemini - bugfixes, add examples
- few more bugfixes revealed by tests and examples
- add setup.m calls from geminirun and geminiplot to ensure user-facing scripts load hdf5nc
- add Matlab Live Script in Examples/
- MATLAB
Published by scivision over 5 years ago
mat_gemini - Explicitly specify TLS certs when needed
To help avoid issues with SSL/TLS certs when downloading reference data, when environment variable SSL_CERT_FILE is set, put this directory into weboptions(). This is necessary to have Matlab's vendored cURL actually use the option.
Reenable parallel test runner, when not on CI. Some CI do not have Parallel Toolbox setup correctly.
- MATLAB
Published by scivision over 5 years ago
mat_gemini - HDF5NC submodules, overhaul testing and linting
- make HDF5NC a Git submodule, to avoid duplication of code and enhance test/quality
- Make MatGemini testing use "class-based" Matlab unit testing for better reporting and robustness
- add linting with policies set in +gemini3d/+tests/MLint.txt
- MATLAB
Published by scivision over 5 years ago
mat_gemini - Lots of bugfixes and add Travis-CI
Travis-CI quietly released Matlab support several week ago. We added Travis-CI for this Matlab code, which helped discover and resolve numerous bugs.
- MATLAB
Published by scivision over 5 years ago
mat_gemini - numerous bugfixes
Numerous bugfixes, especially:
- fix infinite Cmake loop on msis_setup.exe autobuild
- use .empty property of various data types
- enhance plot labels
- MATLAB
Published by scivision over 5 years ago
mat_gemini - simplify arguments, bugfix
- use .job suffix for batch job scripts
- correct type argument specifications
- update test data
- MATLAB
Published by scivision over 5 years ago
mat_gemini - R2019b argument validation, plotframe, {h5,nc}save: name,value options and bugfixes
- Matlab >= R2019b argument validation is now used, along with strings. This makes this program work more similarly in Python and Matlab while boosting quality.
- plotframe is once again easy to use directly, using name,value options for less common options
- {h5,nc}save also use name,value options for less common options
- numerous bugfixes
- MATLAB
Published by scivision over 5 years ago
mat_gemini - general enhancements to package layout
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - bugfix, Matlab >= R2017b
Matlab >= R2017b is now required. This significantly cleans up workarounds for R2017a and earlier, relevant since so many file operations are necessary.
- plotglow: bugfix argument error
- plotall: to allow better debugging and usage, for interactive plotting only, user presses Enter instead of any key to proceed. Typing any letter plus Enter exits the main plotting loop.
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - bugfixes and added tweaks
- fixed milestone detection
- added config.nml:setup:sslverify .true..false. option or geminirun(...,params) option to disable SSL certificate verification on download. Normally this should not be used, but sometimes HPC is running very out of date OS that has trouble accessing internet HTTPS sites.
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - Matlab package import API
use Matlab packages import, which is a more object-oriented, Python-like way to reuse code modules. This avoids problems with user defined functions having the same name as Gemini3d functions and it not always being obvious which function was used.
Old way:
matlab
addpath('.........') % long list of paths
dat = loadframe('foo.h5') % hope it's the right loadframe
...
perturb(cfg, xg) % hope it's the right perturb
New way
matlab
dat = gemini3d.loadframe('foo.h5')
...
g3init.khi_periodic_lowres.perturb(cfg, xg)
The latter, new method is unambiguous as to which function is being used.
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - new loadframe api
simplify loadframe() call to require only one argument
remove mloc and make cfg.sourcemlat, cfg.sourcemlon
remove old readconfig, use read_config instead
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - better plotting, auto-download equilibrium, less resource usage
- make 3D plotting more functionalized and use R2020a save figure enhancements when available
- plotall: option to specify number of parallel workers
- writedata: less required inputs
- download equilibrium data if eqzip and equrl are specified
- Efield: loosen input requirements
- run: shutdown parallel pool before Gemini to save a lot of RAM
- setup_functions nml variable allows character array of function names to be used
- loadframe: tolerate partial variable set
- writegrid: auto file_format
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - use datetime
Matlab R2014b added datetime, which like Python datetime standardizes and makes simple many time manipulation tasks, avoiding proprietary functions with more variables to juggle. This breaks compatibility with current releases of GNU Octave. However, there are so many bugs and missing features even in the latest GNU Octave that this is not a practical loss. We increase reliability and greatly simplify code by using datetime.
However, this requires existing user code to be upgraded, hence the major release version.
Parallel plotting
matlab
plotall('~/sim/my_sim', 'png', [],[],1)
where the 1 or true means plot in parallel. Because large sims could use too much RAM this way, the default is to plot serially. However if you have a big RAM computer, the parallel option can speed plotting by many times (x number of CPUs).
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - general bugfixes and enhancements
general fixes
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - add milestones, parallelize test, robustness
- Use
ctest -S setup.cmaketemplate to platform-independently handle many setup issues - Use Git tags to get gemini3d except if already present
- add simulation milestone support
- run Matlab + Octave tests via CMake in parallel, or directly from Matlab.
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - comprehensive file I/O improvements and add tests
- added numerous unit and integration tests
- handle paths more reliably
- overhaul HDF5 and NetCDF4 to be more reliable across use cases
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - new API requiring Gemini >= 0.6.0
Enacted new Gemini API for unambiguous directory input. Directories in config.nml may be either absolute paths, or relative to the top-level output directory specified at the command line. To avoid future problems, developers should avoid using absolute_path() except in top-level scripts as necessary.
Numerous additional checks were added and self-tests added. Due to bugs in GNU Octave, even in the latest 5.2.0 release, Octave is not capable of writing HDF5 or NetCDF4 files. We have left full capability for Octave throughout this repo, in the hopes that the NetCDF4 bugs will be fixed.
added several utility functions of general use under matlab/fileio/ and matlab/sys/ for working with HDF5 and NetCDF4 files, as well as computing number of MPI images
added ability to rungemini() from Matlab, just like Python geminirun.
added ability to generate simulations iteratively over a directory
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - Numerous bugfixes
Bugfixes and functionalization
- MATLAB
Published by scivision almost 6 years ago
mat_gemini - Robustness
Robustness and bugfix and CI enhancements, including
- recursive checkcode linting in test_matlab.m
- make absolute_path failsoft (return relative path) if absolute path is not determinable. Some HPC have this issue but still work with relative path
- make model_setup() directly runnable as top-level / entry script
- rename xgrid.m to avoid namespace clash with improperly setup Matlab custom toolbox from other HPC project
- MATLAB
Published by scivision about 6 years ago
mat_gemini - use cmake to build msis
be more robust to build msis_setup.exe by using CMake instead of ad hoc matlab code.
- MATLAB
Published by scivision about 6 years ago
mat_gemini - moved from main repo
To avoid cluttering main gemini repo, moved all matlab scripts here.
- MATLAB
Published by scivision about 6 years ago