rt1 - RT1 v2.0

new

• HDF5 containers can now be used instead of pickle-dumps to store Fits and MultiFits objects
• RTprocess is now directly appending results to an HDF5-container during parallel processing
• colored log-messages can be used (requires colorama package)
• there's now a proper container for defdict to simplify get/set of variable definitions
• indicate fit-results in fit.plot.analyzemodel

changes

• processfunc is now private (e.g. _processfunc) -> use run_processing instead!

fixes

• several bugfixes along the way

- Python
Published by raphaelquast over 2 years ago

rt1 - RT1 v1.5.2

new

• there's now a quick and easy data-export function RTprocess.export_data()
  • extract parameters and performance-metrics from a folder of dump-files
• there's now an RTresults.dump_fits - generator that directly returns Fits-objects
• RTprocess object can now be initialized without a config-file
• option to pass a list of dump-files to rtprocess.run_finalout() added

fixes

• speedup for indexing of folders with large amounts of dump-files
• some minor logging updates

- Python
Published by raphaelquast over 4 years ago

rt1 - RT1 v1.5.1

new

• add possibility to provide numeric V & SRF functions for faster evaluation if no interaction-term is calculated
• there is now a property fit.config_name that indicates the config-name for fits that belong to a MultiFits object
• datasets provided to rtfits.Fits are now checked for sorting (and automatically sorted if necessary)

fixes

• make ncoefs a propery for "Rayleigh" and "Isotropic" phase-functions
• improved logging messages
  • don't log each successful progress step, only errors and warnings
  • don't create a single file for each error (log traceback to log-file directly)
• keep minimal info from fit.fit_output when dumping fits to disc
• improvements on caching and pre-computation steps
  • pre-evaluate fn-coefficients during finalout-generation
  • cache lambda-functions for V.p and SRF.brdf
• fix sorting of fractional contributions in rtplots.printsig0analysis()
• some updates on rtplots.polarplot() for symbolic parameter specification
• make sure copying of config-files and modules is only attempted from the main process

- Python
Published by raphaelquast almost 5 years ago

rt1 - RT1 v1.5

new

• add possibility to provide numeric V & SRF functions for faster evaluation if no interaction-term is calculated
• there is now a property fit.config_name that indicates the config-name for fits that belong to a MultiFits object
• datasets provided to rtfits.Fits are now checked for sorting (and automatically sorted if necessary)

fixes

• make ncoefs a propery for "Rayleigh" and "Isotropic" phase-functions
• improved logging messages
  • don't log each successful progress step, only errors and warnings
  • don't create a single file for each error (log traceback to log-file directly)
• keep minimal info from fit.fit_output when dumping fits to disc
• improvements on caching and pre-computation steps
  • pre-evaluate fn-coefficients during finalout-generation
  • cache lambda-functions for V.p and SRF.brdf
• fix sorting of fractional contributions in rtplots.printsig0analysis()
• some updates on rtplots.polarplot() for symbolic parameter specification
• make sure copying of config-files and modules is only attempted from the main process

- Python
Published by raphaelquast almost 5 years ago

rt1 - RT1 v1.4

new

• add possibility to call finalizer after rt1.rtprocess.run_processing() finished
  • to "clean-up" after processing is done (or the process is terminated)
• add option to save dumps in rtprocess.run_processing() even if postprocess is not called via rt1.rtprocess.run_processing(..., dump_fit=True)

Multiconfig Fits

• add possibility to specify multiple configs in a single .ini file
  • processing of multiple configs involves only a single call to read()!
  • you can specify as many configs as you like in a single .ini file!
  • to specify multiconfigs, add the config-specific changes to sections starting with [#config <NAME> <SECTION>]
    for example:

```ini [defdict] --- the default defdict ---

[#config NAME defdict] --- all specifications that differ from the default defdict --- ```

• Multiconfig fits will be handled with a rt1.rtfits.MultiFits object
  • use rt1.rtfits.MultiFits.accessor to access Fits propeties of all configs
• all processed configurations can be saved in a single .dump-file!
  • this avoids duplicating data that is shared between the configs

fixes

• fix issue in the default "domainfuncs" of rt1.general_functions.scale()
• make sure rt1.rtfits.reinit_object() only copies relevant properties
• speed up rtprocess.RTresults initialization for folders with many files

- Python
Published by raphaelquast almost 5 years ago

rt1 - v1.3

new

• logging implemented
• incorporate basic NetCDF export functionality via xarray (rt1.processing_config.postprocess_xarray())
• new rtmetrics class to provide easy access to common metrics calculation
• add option to disable postprocessing va RTprocess.run_processing(postprocess=False)
  • add option to run processing and NetCDF export in 2 separate runs
    • e.g. via run_processing() and run_finalout()
  • add dump_fit option to run_processing() so fits are dumped even if postprocess=False

changes

• copy is now an argument of the __init__ of RTprocess instead of RTprocess.run_processing()

unittests

• switch tests to github actions
  • use conda .yml file for requirements
• switch coverage reports to codecov
• reactivate stickler & flake8
• use github actions for PyPi upload

fixes

• fix processing and logging with ncpu=1
• enure that the processing_config class and additional modules are loaded from the correct folder
• fix future warnings in sympy and matplotlib
• make RTresults.load_fit() find fits faster
  • allow passing integers to load the nth result
• fix sorting in printsig0analysis()
• fix parsing error in param_dyn_dict
• use pickleable decoration of initializer function
• proper detection of submodules in setup.py
• incorporate pending changes in scipy > v1.6.3 (e.g. adjust call signature of sympy.Lambdify)

- Python
Published by raphaelquast about 5 years ago

rt1 - v1.2

changes

• the reader-function in rt1.rtprocess is now called without dict-unpacking!!
  (e.g. via reader(reader_args) instead of reader(**reader_args))
  ! this might require modifications in existing scripts !

fixes

• correct parsing of None in rt1.rtparse
• multiprocessing pool initializer also executed in single core processing (for debug)
• rt1.rtprocess now uses warnings.warn instead of print for general messages

new

• convenience-method interpolate_to_index() added to rt1.genreal_functions

- Python
Published by raphaelquast over 5 years ago

rt1 - v1.1.9

changes

• rtfits.preprocess() is now called prior to parallel-processing
  • a check if files already exist should now be done within the reader-function!
• copying of modules is shifted from the rtparse to rtprocess to avoid copying files when config is read
• the configparser now has methods get_module('modulename') as well as get_all_modules() to programmatically import modules
• all parallel-processing functions have been shifted to the dedicated rtprocess module

fixes

• corrections to the treatment of integer parameter-dynamics specified in fit.defdict
• the cache is now only cleared at the beginning of a fit.performfit() call
• correct final message in processfunc() progress-bar

new

• a dedicated class for parallel processing (rt1.rtprocess) has been introduced

  • rt1.rtprocess.RTprocess can be used to setup and run parallel jobs
  • rt1.rtprocess.RTresults can be used to provide easy-access to processing-results

• some updates on rtplots.printsig0analysis()

  • now one can select additional timeseries from both fit.res_df and fit.dataset
  • ... and the styles can be set using the printparamstyles - dict

- Python
Published by raphaelquast over 5 years ago

rt1 - v1.1.8

changes

• manual parameter dynamics are no longer automatically used if a column 'param_dyn' is provided
  ... to use manual dynamics, you now must use one of the following:
  • explicitly specify parameter-dynamics as 'manual' in fit.defdict
  • to combine a pandas datetime-offset freq with a manual-dyn use 'freq + manual'

new

• the model_definition text has been updated to include more details
• the property fit._model_definition is added to allow accessing the model_definition string
  (useful for example to write it to a file)
• RT1_configparser now provides a function .get_module(modulename) to import a specific module from a file

fixes

• fix import-location for copied modules in rtparse (now modules are imported from the copy-path)

- Python
Published by raphaelquast over 5 years ago

rt1 - v1.1.7

new

• define parameter-grouping with respect to number of observations within a group

fixes

• fix dynamic startvalue-assignment
• better implementation of rectangularize function

- Python
Published by raphaelquast almost 6 years ago

rt1 - v1.1.6

• added possibility to provide arbitrary start-values via the dataset

  • dataset -columns "parameter_start" are now used to assign start-values for the parameters

• there is now a load() function in rt1.rtfits to load fit-configurations dumped via fit.dump()

fixes

• cache mask during fit
• interpolation now works also with unsorted groupings -> now it's possible to fit interpolated seasonalities etc.
• ... some speed-ups

- Python
Published by raphaelquast almost 6 years ago

rt1 - v1.1.4

• add possibility for parsing (dtype-converted) lists in rtparse.get_process_specs()

fixes

• fix of issue with inner-derivatives in case unequal groupings are used within sub-parameters
• speedup in the initialization of the jac-assign rules
• fix issue in rtplots.analyzemodel with int_Q = True

changes

• change of folder-structure used in default parallel-processing funcs
  • put all files in sub-folders of dumpfolder (cfg, results, dumps)
• change of datetime format-specifications in .ini files
  • "fmt=" is now used as separator instead of ","

- Python
Published by raphaelquast almost 6 years ago

rt1 - v1.1.3

• fix error in index-order of res_df

- Python
Published by raphaelquast almost 6 years ago

rt1 - v1.1.2

• add version-number to dump-files
• avoid concatenation in jac-calculation
• faster way of getting _groupindex

- Python
Published by raphaelquast almost 6 years ago

rt1 - v1.1.1

• intermediate_residuals() added to rtplots
  • allows analyzing residual-development during fit-procedure with 2D or 3D plots
• fixed issues with unsorted (manual) parameter-groupings
• add possibility to use the dataset-index or the group-index in printsig0analysis
• add public method calc_model() as wrapper of _calc_model that directly returns a pandas.DataFrame
• get zenodo DOI and automatically deploy releases to pypi.org

- Python
Published by raphaelquast almost 6 years ago

rt1 - RT1 v1.0.0

First release of the RT1 module, consisting of:

• rt1.rt1: basic calculation routines for symbolic and numerical evaluation of generic models
• rt1.surface and rt1.volume: a set of flexible surface- and volume scattering phase functions
• rt1.rtfits: interface for (parallelized) nonlinear least-squares fitting
• rt1.rtplots: several nice plot-functions and interactive widgets to analyze results
• rt1.rtparse: a configparser to specify processing-routines via .ini files
• rt1.processing_config : some useful default functions for processing

- Python
Published by raphaelquast almost 6 years ago

rt1 - fourth beta release

this is (still) a beta-release, so use it with care!

• major updates of rtfits

  • parallel processing
  • configparser
  • use properties to avoid storing duplicated data
  • use caching to speed up fits

• additional useful features added

- Python
Published by raphaelquast almost 6 years ago

rt1 - third beta release

this is a beta-release, so use it with care!

• the fit.result list has been removed and replaced with individual class-attributes
• several plotfunctions added
• setter- functions for parameters are removed (from now on, no axis is added when setting a value)

- Python
Published by raphaelquast over 6 years ago

rt1 - second beta release

a second beta-release with major changes

- Python
Published by raphaelquast over 7 years ago

rt1 - first beta release

this is a first beta-release, so be careful if you use it!

- Python
Published by raphaelquast over 8 years ago