portalcasting - portalcasting v0.60.7

Store forecasts_evaluations.csv as parquet

Scientific Software - Peer-reviewed - R
Published by henrykironde 10 months ago

portalcasting - portalcasting v0.60.6

Scientific Software - Peer-reviewed - R
Published by henrykironde over 1 year ago

portalcasting - portalcasting v0.60.5

Avoids creating extra folders from the relative path when zipping

Scientific Software - Peer-reviewed - R
Published by henrykironde about 2 years ago

portalcasting - portalcasting v0.60.4

Switch to Plausible Zip the resources forecast folder

Scientific Software - Peer-reviewed - R
Published by henrykironde about 2 years ago

portalcasting - portalcasting v0.60.3

Scientific Software - Peer-reviewed - R
Published by henrykironde about 2 years ago

portalcasting - portalcasting v0.60.2

New version of portalPredictions Portal-forecasts reduces the size of the portalPredictions Portal-forecasts is a revised version of portalPredictions Portal-forecasts contains forecasts as zipped files

Scientific Software - Peer-reviewed - R
Published by henrykironde about 2 years ago

portalcasting - portalcasting v0.60.1

minor documentation edits (fixing some links and spelling errors) no code changes

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.60.0

Automating editing of dockerhub description

• Using a 3rd party action while docker gets this functionality integrated
• addresses #361

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.59.0

adding curl back into dockerfile

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.58.0

Editing documentation

• Auto templates for issues and prs

Inclduing git2r in the package

• Was removed accidentally from dockerfile causing breaks in main repo, seeing if it works to be in the package itself

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.57.0

Editing documentation

• Tightening and improving language in Title and Description fields in DESCRIPTION file

Preventing duplicate keys in control lists at the top level

Adding docker images with setup done

• including a production and a sandbox image

Testing docker images prior to pushing them

• A simple "can you run library(portalcasting)" to start, can use this entry point to expand the testing suite on the image in the future

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.55.0

Reorganizing the app pointing

• Now having a single app.R file in the top level of the directory simplifies the issue around where the files are for the app
• This replaces the global.R, ui.R, and server.R files

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.54.0

Adding sandbox and production version of update_ functions.

Adding Pat as an author

Pointing the dockerfile to the current build SHA

• Previously, we were using default settings for remotes::install_github, which actually points to HEAD, when testing the build and push for the docker action. that was fine (although not great) when we were only running that action on tagged version releases, as the main branch was typically up-to-date with the tag and such, but this isn't exact and it also means any testing of the build on a PR was actually still grabbing from main, which is not what we want.
• We now use the SHA for the specific event that triggers the build by passing it into the docker file as an ARG

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.53.0

Removing temporary patch for backwards compatibility

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.52.0

Patching evaluations issues

• The table was too big as was for github standard, so removing columns of repeated content that can be grabbed from metadata

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.51.1

minor patches to deal with pkgdown site building

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.51.0

Major updates to pkgdown site

• Shift to bootstrap 5

Include CI tests of \dontrun examples and eval=FALSE vignettes

• Because of the long run time of some of the code, we wrap most documentation examples and vignette code in a way to prevent evaluation in real-time. As a result, much of the documentation code isn't run and therefore would not break builds if it would error.
• To address this, we add two scripts in the new inst/extra_testing folder and a github action runner for each.

Added CITATION file

• Cites the JOSS paper.

Bringing the app code into the package

• Improves robustness of building the app (includes code and dependencies in the docker image, allows for unit testing app components, etc.)
• Also allows users to spin-up a local version with run_web_app() pointed to main

Using arrow to speed up reading and writing files

Evaluation figures now read from evaluations file

• Avoids computing evaluations while generating plots

Elimination of model-named functions

• The models are now implemented using cast on their fit and cast elements in their control lists
• Only models the need new functions have them (meta_tsglm and fit_runjags) for fitting
• forecast method used generally now for cast function
  • introduction of forecast methods for tsglm and runjags objects

Shifting covariates to a daily-level build initial step

• need to shift this to level daily so we can manage when a newmoon is split between historic and forecast days

Putting species under dataset in the models' controls lists

• This is more articulated and allows for finer control to help avoid fitting issues, etc.

Moving arguments into functionalities

• cast_date is not an argument anymore, just filled automatically
• dataset arguments are also being removed as possible to streamline (just pull from model controls)

Model functions are now species-level

• To facilitate a lot of downstream functionality, we're breaking up the model functions to operate on the species-level rather than the dataset-level, according to the new control lists
• Species that were failing the nb GARCH models (s or not) have been removed, since that throws a warning and then fits a Poisson version, and we are now fitting Poisson versions of everyone.

processmodeloutput replaces savecastoutput and various model processing bits

• provides a much more general way to produce a forecast that can be integrated in the system, leveraging the metadata files

casts metadata table includes new column

• species
• to facilitate backwards compatibility, filled with NA for previous tables if missing when loaded

updates to prefab models to 13 time steps forward (addressing issue 297)

• pevGARCH, nbGARCH, nbsGARCH all get past_mean set to 13
• all models set with lead_time of 13

new models

• sAutoArima
• sNaiveArima
• pGARCH
• psGARCH

new functions

• update_dir is an "update-flavored" setup function

scoring

• log and crps!

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.50.0

Update prefabmodelcontrols.yaml

• including a nicer print_name for rendering the model texts

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.49.0

Added a clearing out step to fill_data

Updated download climate forecasts to check for version pre-updates

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.48.0

Settings updates to avoid redownloading archive

• settings overwrite is now FALSE by default to manage the version match decision making
• the directory resources portalpredictions version is updated to be the correct value ("latest"; was NULL)
• the overwrite setting is temporarily removed from the file saving functions to prevent argument name confusion
• we need to have an external location of file version to compare

Fixing jags model inits values

• Model values were not facilitating good chain mixing, updated now

More transitions from moons to newmoons to align naming

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.47.0

Patching error in cast metadata output for jags models

• Multiple models had the jagslogisticcovariates name being used in the metadata output when that was not correct.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 3 years ago

portalcasting - portalcasting v0.46.0

Articulating dataset controls for species

• Changing dataset controls list input from having all for species to listing the species.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 3 years ago

portalcasting - portalcasting v0.45.0

Removing dm_controls as a separate dataset

• Following the removal of the interp datasets
• Now model controls indicate species to which their applied

New models

• jags_logistic_competition
• jags_logistic_covariates
• jags_logistic_competition_covariates

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 3 years ago

portalcasting - portalcasting v0.44.0

Major updates to JAGS models

• Modeling and tracking sigma not tau (sd, not precision)
• No max caps for density or counts (aka removing guardrails)
• No use of + 0.1 - 0.1 for logging, managing it differently
• Chain increase from 2 to 4, silent jags is now FALSE
• Removal of jags_SS wrapper, which limited adaptation of the model

Removing JAGS model vignette

• Removal of the jags_SS wrapper eliminates the core of that vignette

Adding integrated warm precip and ndvi covariates

• Allows covariates logistic model

Shift internal naming of subs to subdirectories

Shift folder naming of raw, casts to resources, forecasts

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 3 years ago

portalcasting - portalcasting v0.43.0

Trim text in model vignette

• Vignette now pulls text from the the model list, and loops the model list, rather than have the raw text in the markdown doc.
• This locates the description in a place that is accessible to other content generation (e.g., the portal forecast website)

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 3 years ago

portalcasting - portalcasting v0.42.0

Data interpolation moved from "dataset" to "within model"

• Patching, not all hit with previous release

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 3 years ago

portalcasting - portalcasting v0.41.0

Data interpolation moved from "dataset" to "within model"

• Previously, datasets included, for example all and all_interp. Now, only all exists and models interpolate as needed.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 3 years ago

portalcasting - portalcasting v0.40.0

Argument / nomenclature updates

• dfl arg to x

read_rodents default settings update

• Now pulls all datasets using prefab_datasets

Subdirectory internal naming changed to remove spaces

• model fits now back to fits and model scripts back to models

cast_evaluations file now saved

• Flattened version of the generated list of evaluations
• Rather crude options of saving or not and overwriting the whole file or not based on settings
• No file saving occurs when a single cast is evaluated

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.39.0

download_timeout now set to default of 600 for setup_production

• Allows download of larger directory archive without timeout

If there's only one model, don't ensemble

• Prevents warnings / errors

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.37.0

Building out evaluation pipeline

• starting with what is already occurring, but formalizing as such as part of an evaluate_casts and evaluate_cast pair of functions
• evaluate_casts function now works automatically to evaluate all the casts using evaluate_cast, generating the error table as it does when being used, but nothing is saved out or updated.
• there is also no filter on evaluated casts by deafult, so the output from the forecasts without observations to evaluate is a table with a single row with NaN, and then they get wrapped up into the list.
  • no errors, just noteworthy

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.36.0

rodent_species

• Now includes type argument with abbreviation, Latin, and table options
• No more most_abundant_species function, as we're not using it on the website.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.35.0

Updating model controls

• time
• response with type and link

Developing evaluate

• evaluate_casts and evaluate_cast currently just placeholders

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.34.0

Removing tmp sub

• No longer used, internal R code (e.g., tempdir) provides needed functionality
• Also removing clear_tmp and cleanup arg in settings

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.33.0

setup_production defaults to verbose = TRUE

• Facilitates fuller log

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.32.0

Relocation of prefab controls

• Moved from source code scripts to .yaml files in inst/extdata
• read_ and write_ functions for both rodent and model controls lists

Updating / rectifying terminology

• "data_set" -> "dataset"

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.31.0

tidying documentation fixing test bug

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.30.0

Settings list

• setup_dir now takes a settings argument that is a list of the arguments
• directory_settings function now quickly and cleanly collapses the settings that go into setup_dir

Generalized functionality for models and rodent data sets

• Control lists are now structured for use with do.call

Codebase formatting [work in progress]

• No longer concerned about the 80 char line limit
• Long argument lists, etc. are now formatted for quick top-to-bottom reading, via alignment on the ( and =

Removal of superfluous _path functions

• Use of base R functions is sufficient

arg_checks removed

• Internalized auto-checking relieves user of need to dictate checking

Temporary removal of "adding a model and data" vignette

• Need to update with new API
• Also need to add alt-text to all images

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.29.0

Patch bline bug

• Argument needed to be removed still

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.28.0

messageq

• Function redesigned to align with message directly argument for argument with the addition of the quiet argument.
• Now allows for multiple message arguments via ... that become pasted together

Removal of specialized message functions

• Minimize unnecessary functions

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - portalcasting v0.27.0

Simplified directory creation function pipeline

• Now just create_dir

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - v0.26.0

jags_logistic model added

• invoked as jags_logistic like jags_RW, applied to DM controls dataset.
• Building upon the jagsRW model, jagslogistic expands upon the "process model" underlying the Poisson observations.
• There are four process parameters: mu (the density of the species at the beginning of the time series) and tau (the precision (inverse variance) of the random walk, which is Gaussian on the log scale) for the starting value and r (growth rate) and K (carrying capacity) of the dynamic population. The observation model has no additional parameters.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 4 years ago

portalcasting - v0.25.4

Update Docker container now that build workflow is fixed.

Scientific Software - Peer-reviewed - R
Published by ethanwhite about 4 years ago

portalcasting - v0.25.3

Further removal of vestigial rEDM code

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 4 years ago

portalcasting - New Zenodo title

Update Zenodo title to match JOSS

Scientific Software - Peer-reviewed - R
Published by gmyenni about 4 years ago

portalcasting - Getting latest version of portalr

• tagging to update Docker image with portalr 0.3.9

Scientific Software - Peer-reviewed - R
Published by gmyenni about 4 years ago

portalcasting - portalcasting v0.25.0

Tidying for JOSS ms

• adding source and version args to check args
• software context added to ms
• overview added to readme
• updating the getting started vignette to use production to allow for historical uses

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 4 years ago

portalcasting - portalcasting v0.24.0

Speeding downloading

• Shift to default downloading archive from GitHub
• setupdir does not download archive by default, but setupproduction does
• download function being broken out into components -- work is still ongoing, but now have a separate function for each of the portaldata portalpredictions and climate forecasts
• addresses #132 #199

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 4 years ago

portalcasting - portalcasting v0.23.0

patching issue with ndvi preparation

• the ndvi data stream is not filling in with new content, resulting in NAs for the latter half of 2021
• using a forecast call to fill in the missing values as a temporary patch

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 4 years ago

portalcasting - portalcasting v0.22.0

Removal of simplexEDM and GPEDM

• Previous versions used Hao's fork of the rEDM packages, which has been deprecated and now breaks because of the updates in Rcpp
• Switching to CRAN version
• Updating rEDM to CRAN does not fix the issue, so these models cannot be used in prefab set
• Removed from the prefab control list and removed the documentation
• No longer exported from the NAMESPACE

Edit tests for ensembling and figure making

• A few edge case issues arose in test because of fixed moons
• Should be resolved through edits to test scripts without altering functions

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 4 years ago

portalcasting - portalcasting v0.21.4

adding henry to the description file

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 4 years ago

portalcasting - portalcasting v0.21.3

Add git2r to the docker container.

Scientific Software - Peer-reviewed - R
Published by ethanwhite over 4 years ago

portalcasting - portalcasting v 0.21.1

Patch "NA" plotting issue

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 5 years ago

portalcasting - portalcasting v0.21.0

Highlighting of species in plotting

Shift to GitHub actions

Rename default branch to main

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 5 years ago

portalcasting - portalcasting v0.20.0

Stops saving model fits in the portalPredictions repository

Addresses issues with covariate data

• Missing data from weather stations caused issues
• Now if there is a missing set of data for a month of covariates, the saved covariate forecasts are used

Added dm_controls and dm_controls_interp to prefab data sets

• For use in the basic single-species process models
• Note lowercase name! Using capitals in the actual name of the data set creation will cause problems because tolower gets used elsewhere!

Patches issue with check_args when using portalcasting::function_name()

Tightens testing time

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 5 years ago

portalcasting - portalcasting v0.18.3

Testing the Dockerfile integration

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.18.2

Setting the Docker build up with its own folder

Tidying README etc

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.18.1

testing dockerhub tag integration

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.18.0

Bringing the Dockerfile over from portalPredictions

• For building here.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.17.1

Addition of GPEDM (model and function)

• Gaussian processes using Empirical Dynamic Modeling

for real this time

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.17.0

Addition of GPEDM (model and function)

• Gaussian processes using Empirical Dynamic Modeling

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.16.0

Change in format for saving out model_fits and model_casts

• Saving model_fits and model_casts out as serialized .json files now instead of .rData files
• More reliable and generlized.
• Also added functions for reading them in (read_model_casts and read_model_fits).
• addresses

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.15.2

shifting to github of portalr

to address the backwards incompatibility between the CRAN and GitHub versions of portalr and the need for the newest (GH) version because of the break in portalData

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.15.1

patch bump to try zenodo build again (the zenodo build for v0.15.0 failed)

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.15.0

JAGS vignette

• Added a vignette that describes how to use the JAGS/runjags API within portalcasting.
• addresses

Pulls code for match.call.defaults into the package

• Use of it from DesignLibrary causes a problematic dependency chain with the docker image building

Patch bug in most_abundant_species

• Wasn't using the species name function, and so was pulling in the traps column, which was causing a break in plotting.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.14.0

(done retroactively from the branch where this was done)

Adds exclosure data to the prefab models

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis about 6 years ago

portalcasting - portalcasting v0.13.0

Full writing of control_files in model scripts

• Previously, the controls list for the files in the model scripts was taken from the environment in which the script was run, which opens the script to everything, which is undesirable.
• After the need to include a control list for runjags models forced an explicit writing of the list inputs, the code was available to transfer to the files control list.
• This does mean that the function calls in the scripts are now super long and explicit, but that's ok.
• To avoid super long model script lines (where event default inputs are repeated in the list functions), a function control_list_arg was made to generalize what was coded up from the runjags list for use also with the files control list. This function writes a script component that only includes arguments to the list function that are different from the formal definition.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 6 years ago

portalcasting - portalcasting v0.12.0

portalcast updates model scripts according to controls_model

• Previously, if you changed any controls of a prefab model, you had to manually re-write the models using fill_models before running portalcast.
• Using fill_models would result in hand-made scripts being overwritten, so a specific function (update_models) for updating the models was created.
• update_models by default only updates the models listed in the controls_model input, to avoid overwriting model scripts. To change this behavior and also update all of the prefab models' scripts, set update_prefab_models = TRUE. This is particularly handy when changing a global (with respect to model scripts) argument: main, quiet, verbose, or arg_checks.
• addresses

Messaging around trying to use not-complete directory improved

• Indication now made that a component of the directory is missing and suggestion is made to run create_dir.
• addresses

Patching data set bug in plotting

• There was a bug with matching the interpolated to the non interpolated data sets within the ensembling, which has been fixed.
• addresses

Updated messaging

• Moved most of the messaging into tidied functions.

Changed behavior of prep_rodents_table and prep_rodents

• Now there is no start_moon argument, and all of the data prior to end_moon are returned.
• This aligns the rodents prep functions with the other (moons, covariates) prep functions.
• Facilitates use of data prior to start_moon in forecasting models (e.g., for distributions of starting state variables).
• Requires that model functions now explicitly trim the rodents table being used. This has been added to all prefab models.

Fixed codecov targets

• Previous targets were restrictively high due to earlier near-perfect coverage.
• A codecov.yml file is now included in the repo (and ignored for the R build) which sets the target arbitrarily at the still-quite-high-but-not-restrictively-so 95%.
• It can be changed if needed in the future.

Simple EDM model added

• addresses

JAGS infrastructure added

• Using the runjags package, with extensive access to the API of run.jags via a control_runjags list (see runjags_control).
• Currently in place with a very simple random walk model.
• addresses

Prepared rodents table includes more content

• Expanded back in time to the start.
• Added effort columns (all default options in prefab_rodents_controls have effort = TRUE).

Updated adding a model and data vignette

• Added section at the end about just extending existing models to new data sets.
• addresses

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 6 years ago

portalcasting - v0.11.0

Ensembling reintroduced

• Associated with the reconfiguration of portalcasting from v0.8.1 to 0.9.0, ensembling was removed temporarily.
• A basic ensemble is reintroduced, now as an unweighted average across all selected models, allowing us to have an ensemble but not have it be tied to AIC weighting (because AIC weighting is no longer possible with the split between interpolated and non-interpolated data for model fitting).
• In a major departure from v0.8.1 and earlier, the ensemble's output is not saved like the actual models'. Rather, it is only calculated when needed on the fly.
• In plotting, it is now the default to use the ensemble for plot_cast_ts and plot_cast_point and for the ensemble to be included in plot_casts_err_lead and plot_casts_cov_RMSE.

Return of most_abundant_species

• Function used to select the most common species.
• Now uses the actual data and not the casts to determine the species.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 6 years ago

portalcasting - v0.10.0

Model evaluation and ensembling added back in

• Were removed with the updated version from 0.8.1 to 0.9.0 to allow time to develop the code with the new infrastructure.
• Model evaluation happens within the cast tab output as before.

Temporarily removed figures returned

• Associated with the evaluation.
• Plotting of error as a function of lead time for multiple species and multiple models. Now has a fall-back arrangement that works for a single species-model combination.
• Plotting RMSE and coverage within species-model combinations.

Flexing model controls to allow user-defined lists for prefab models

• For sandboxing with existing models, it is useful to be able to change a parameter in the model's controls, such as the data sets. Previously, that would require a lot of hacking around. Now, it's as simple as inputting the desired controls and flipping arg_checks = FALSE.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 6 years ago

portalcasting - v0.9.0

Major API update: increase in explicit top-level arguments

• Moved key arguments to focal top-level inputs, rather than nested within control options list. Allows full control, but with default settings working cleanly. addresses
• Restructuring of the controls lists, retained usage in situations where necessary: model construction, data set construction, file naming, climate data downloading.
• Openness for new setup functions, in particular setup_sandbox. addresses
• Simplification of model naming inputs. Just put the names in you need, only use the model_names functions when you need to (usually in coding inside of functions or for setting default argument levels). addresses

Directory tree structure simplified

• dirtree was removed
• base (both as a function and a concept) was removed. To make that structure use main = "./name"
• "PortalData" has been removed as a sub and replaced with "raw", which includes all raw versions of files (post unzipping) downloaded: Portal Data and Portal Predictions and covariate forecasts (whose saving is also new here).

Tightened messaging

• Expanded use of quiet and verbose connected throughout the pipeline.
• Additional messaging functions to reduce code clutter.
• Formatting of messages to reduce clutter and highlight the outline structure.

Download capacity generalized

• Flexible interface to downloading capacity through a url, with generalized and flexible functions for generating Zenodo API urls (for retrieving the raw data and historical predictions) and NMME API urls (for retrieving weather forecasts) to port into the download function. addresses and addresses and addresses

Changes for users adding their own models to the prefab set

• Substantial reduction in effort for users who wish to add models (i.e. anyone who is sandboxing). You can even just plunk your own R script (which could be a single line calling out to an external program if desired) without having to add any model script writing controls, and just add the name of the model to the models argument in portalcast and it will run it with everything else.
• Outlined in the updated Getting Started and Adding a Model/Data vignettes.
• Users adding models to the prefab suite should now permanently add their model's control options to the source code in model_script_controls rather than write their own control functions.
• Users adding models to the prefab suite should permanently add their model's function code to the prefab_models script (reusing and adding to the documentation in prefab_model_functions), rather than to its own script.
• Users should still add their model's name to the source code in model_names.

Relaxed model requirements

• Models are no longer forced to use interpolated data.
• Models are no longer required to output a rigidly formatted data-table. Presently, the requirement is just a list, but soon some specifications will be added to improve reliability.
• Outlined in the updated Adding a Model/Data vignette.

More organization via metadata

• Generalized cast output is now tracked using a unique id in the file name associated with the cast, which is related to a row in a metadata table, newly included here. addresses and addresses and addresses
• Additional control information (like data set setup) is sent to the model metadata and saved out.
• Directory setting up configuration information is now tracked in a dir_config.yaml file, which is pulled from to save information about what was used to create, setup, and run the particular casts.

Changes for users interested in analyzing their own data sets not in the standard data set configuration

• Users are now able to define rodent observation data sets that are not part of the standard data set ("all" and "controls", each also with interpolation of missing data) by giving the name in the data_sets argument and the controls defining the data set (used by portalr's summarize_rodent_data function) in the controls_rodents argument.
• In order to actualize this, a user will need to flip off the argument checking (the default in a sandbox setting, if using a standard or production setting, set arg_checks = FALSE in the relevant function).
• Users interested in permanently adding the treatment level to the available data sets should add the source code to the rodents_controls function, just like with the models.
• addresses
• Internal code points the pipeline to the files named via the data set inputs. The other data files are pointed to using the control_files (see file_controls) input list, which allows for some general flexibility with respect to what files the pipeline is reading in from the data subdirectory.

Split of standard data sets

• The prefab all and controls were both default being interpolated for all models because of the use of AIC for model comparison and ensemble building. That forced all models to use interpolated data.
• Starting in this version, the models are not required to have been fit in the same fashion (due to generalization of comparison and post-processing code), and so interpolation is not required if not needed, and we have split out the data to standard and interpolated versions.

Application of specific models to specific data sets now facilitated

• write_model and model_template have a data_sets argument that is used to write the code out, replacing the hard code requirement of analyzing "all" and "controls" for every model. Now, users who wish to analyze a particular data component can easily add it to the analysis pipeline.

Generalization of code terms

• Throughout the codebase, terminology has been generalized from "fcast"/"forecast"/"hindcast" to "cast" except where a clear distinction is needed (here primarily due to where the covariate values used come from).
• Nice benefits: highlights commonality between the two (see next section) and reduces code volume.
• start_newmoon is now start_moon like end_moon
• addresses

"Hindcasting" becomes more similar to "forecasting"

• In the codebase now, "hindcasting" is functionally "forecasting" with a forecast origin (end_moon) that is not the most recently occurring moon.
• Indeed, "hindcast" is nearly entirely removed from the codebase and "forecast" is nearly exclusively retained in documentation (and barely in the code itself), with both functionally being replaced with the generalized (and shorter) "cast".
• cast_type is retained in the metadata file for posterity, but functionality is more generally checked by considering end_moon and last_moon in combination, where end_moon is the forecast origin and last_moon is the most recent
• Rather than the complex machinery used to iterate through multiple forecasts ("hindcasting") that involved working backwards and skipping certain moons (which didn't need to be skipped anymore due to updated code from a while back that allows us to forecast fine even without the most recent samples yet), a simple for loop is able to manage iterating. This is also facilitated by the downloading of the raw portalPredictions repository from Zenodo and critically its retention in the "raw" subdirectory, which allows quick re-calculation of historic predictions of covariates. addresses
• cast_type has been removed as an input, it's auto determined now based on end_moon and the last moon available (if they're equal it's a "forecast", if not it's a "hindcast").

Softer handling of model failure

• Within cast, the model scripts are now sourced within a for-loop (rather than sapply) to allow for simple error catching of each script. addresses

Improved argument checking flow

• Arg checking is now considerably tighter, code-wise.
• Each argument is either recognized and given a set of attributes (from an internally defined list) or unrecognized and stated to the user that it's not being checked (to help notify anyone building in the code that there's a new argument).
• The argument's attributes define the logical checking flow through a series of pretty simple options.
• There is also now a arg_checks logical argument that goes into check_args to turn off all of the underlying code, enabling the user to go off the production restrictions that would otherwise through errors, even though they might technically work under the hood.

Substantial re-writes of the vignettes

• Done in general to update with the present version of the codebase.
• Broke the adding a model or data vignette into "working locally" and "adding to the pipeline", also added checklists and screen shots. addresses
• Reorganized the getting started vignette to an order that makes sense. addresses

Post-processing (evaluation and ensemble building) temporarily removed

• The model evaluation and ensemble building had to be recoded with the updated flexibility in cast output.
• These components are undergoing active development and are not yet ready for integration with the remainder of the codebase, which is stable and functional.
• To provide a more up-to-date version and anchor all of the code work done thus far, I am releasing v0.9.0 with the provision that these components are missing and will be added in the very near future in a new release.

Additional things

• drop_spp is now changed to species (so focus on inclusion, not exclusion). addresses
• Improved examples, also now as \donttest{}. addresses
• Tightened testing with skip_on_cran used judiciously. addresses
• No longer building the AIC-based ensemble. addresses
• Default confidence limit is now the more standard 0.95.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 6 years ago

portalcasting - portalcasting v0.8.1

hook up with zenodo and some other minor documentation edits no coding changes

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 6 years ago

portalcasting - hookup to zenodo

identical to v0.8.0 release just created to trigger the integration with zenodo

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis over 6 years ago

portalcasting - v0.8.0

plot_cov_RMSE_mod_spp now only plots the most recent -cast by default

• If cast_dates = NULL (the default), the plot only uses the most recent -cast to avoid swamping more current -casts with historic -casts.

Added specific checks for no casts returned in plot functions

• There's a bit of leeway with respect to argument validity, in particular around model names (to facilitate users making new models with new names, we don't want to hardwire a naming scheme in check_arg), so now there are checks to see if the tables returned from select_casts have any rows or not.

Handling the edge cases in model function testing

• The trimming of the data sets for model function testing (happens in the AutoArima test script) now includes addition of some dummy values for edge cases (all 0 observations and nearly-all-0 observations), which allows better coverage of testing for the -GARCH model functions in particular.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 7 years ago

portalcasting - v0.7.0

Addressing nbGARCH and nbsGARCH when even the Poisson fallback fails

• In nbGARCH and then extended into nbsGARCH, the models fall back to a Poisson distribution if the negative binomial fit fails. Previously (with only nbGARCH) the Poisson fit always succeeded in those back-ups, but now (with nbsGARCH) that sometimes isn't the case (because the predictor model is more complex) and even the Poisson fit can fail. So now for both models, if that fit fails, we follow what occurs in pevGARCH which is to use the fcast0 forecast of 0s and an arbitrarily high AIC (1e6).

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 7 years ago

portalcasting - v0.6.0

Addressing covariate forecasts in pevGARCH under hindcasting

• pevGARCH() was not set up to leverage the covariate_forecasts file.
• It's now set up with a toggle based on the cast_type in the metadata list (which has replaced the formerly named filename_suffix element) to load the covariate_forecasts file (using a new read_covariate_forecasts function) and then select the specific hindcast based on the source and date_made columns as selected by new elements in the metadata list (covariate_source and covariate_date_made).

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 7 years ago

portalcasting - v0.5.0

Adding nbsGARCH

• Model nbsGARCH has been added to the base set of models.

foy function

• foy() calculates the fraction of the year for a given date or set of dates.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 7 years ago

portalcasting - v0.4.1

Move to usage of CRAN portalr

• To aid with stability, we're now using the CRAN release of portalr

model_scripts function

• Provides a simple way to list the scripts in the models subdirectory.

Including the package version message in setup_dir and portalcast

• Including a simple message to report the version of portalcasting loaded in top level functions.

Vignette updates

• Adding plot (from pre-constructed images) to the how-to vignette.

Patching a bug in model_template

• There was a lingering old name from the argument switch over that was causing model templates to be written with a "" argument for the model model name input into save_forecast_output.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 7 years ago

portalcasting - v0.4.0

Tidied functionality for checking function arguments

• Introduction of check_args and check_arg which collaborate to check the validity of function arguments using a standardized set of requirments based on the argument names, thereby helping to unify and standardize the use of the codebase's arguments.
• This is a much more robust and generalized function than check_options_args() was.

Updated function names

• prep_rodents is now prep_rodents_list
• rodents_data is now prep_rodents
• update_rodents is now update_rodents_list
• read_data has been split out into read_all, read_controls, read_covariates, read_moons, and read_metadata
• model_path is now model_paths
• sub_path and sub_paths have been merged into sub_paths, which returns all if specific_subs is NULL
• lag_data is now lag_covariates

Updated argument (names to leverage check_args, etc.)

• In multiple functions data has been replaced with rodents to be specific.
• CI_level is now subsumed by confidence_level
• name is now subsumed by model
• set is not split into species_set and model_set
• The order of arguments in model_names is now back to model_set, add.
• The default subs_type for subdirs is now "portalcasting".
• The four model functions have a reduced set of inputs to leverage the directory tree, and the script generation is updated to match.
• Updating the cast argument to cast_to_check in cast_is_valid and removing the verbose argument from verify_cast to allow check_arg to leverage check_arg for verify_cast.

Removal of classes

• The models class has been removed.
• The subdirs class has been removed.

messageq function

• messageq function is added to tidy code around messages being printed based on the quiet arguments.

Inclusion of "wEnsemble" as an option in model_names

• Produces the prefab list with an "Ensemble" entry added, to allow for that simply without using the NULL options, which collects all model names.
• This facilitated addition of models as an argument in the evaluations plots.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 7 years ago

portalcasting - v0.3.0

Completed migration of plotting code

• plot_cast is now plot_cast_ts and is now fully vetted and tested
• plotcastts_ylab and plotcastts_xaxis provide tidied functions for producing the y label and x axis (respectively) for plot_cast_ts.
• plot_cast_point is now added to replace plot_species_forecast.
• plotcastpoint_yaxis provides tidied functionality for the y axis of plot_cast_point.
• select_most_ab_spp allows for a simple selection of the most abundant species from a -cast.
• plot_err_lead_spp_mods and plot_cov_RMSE_mod_spp now added to replace the raw code in the evaluation page.

Processing of forecasts

• read_casts (old) is now read_cast and specifically works for only one -cast.
• read_casts (new) reads in multiple -casts.
• select_cast is now select_casts and allows a more flexible selection by default.
• make_ensemble now returns a set of predictions with non-NA bounds when only one model is included (it returns that model as the ensemble).
• most_recent_cast returns the date of the most recent -cast. Can be dependent on the presence of a census.
• verify_cast and cast_is_valid replace forecast_is_valid from the repo codebase. verify_cast is a logical wrapper on cast_is_valid that facilitates a pipeline integration. cast_is_valid does the major set of checks of the cast data frame.
  
• append_observed_to_cast is provided to add the observed data to the forecasts and add columns for the raw error, in-forecast-window, and lead time as well.
• measure_cast_error allows for summarization of errors at the -cast level.

Processing of data

• most_recent_census returns the date of the most recent census.

Minor changes

• Argument order in models is reversed (add then set) and defaults in general are now NULL and NULL, but set = "prefab" within the options functions, to make it easy to run a novel model set.
• Argument order in subdirs is reversed (subs then type) and defaults in general are now NULL and NULL, but type = "portalcasting" within options functions and dirtree to make it easier to manage a single subdirectory.
• fdate argument has been replaced throughout with cast_date for generality.

Utilities

• na_conformer provides tidy functionality for converting non-character NA entries (can get read in from the data due to the "NA" species) to "NA". Works for both vectors and data frames.

Scientific Software - Peer-reviewed - R
Published by juniperlsimonis almost 7 years ago