Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of sdmTMB

sdmTMB - sdmTMB 0.7.4

Minor improvements and fixes

  • Let simulate.sdmTMB() work with binomial GLMs with size specified via weights and newdata supplied. #465

  • Fix issue with fold logic in LFO (leave-future-out) cross validation for lfo_forecast > 1. #454 Thanks to @Joseph-Barss.

  • Add update.sdmTMB() so that the mesh argument doesn't have to be specified if model is loaded in a fresh session. #461

  • Change default in get_index() etc. to bias_correct = TRUE. This is the recommended setting for final inference and speed improvements within TMB have made it more viable to include the bias correction by default. #456

  • Only retain Newton update parameters if they improve the objective function.

    455

  • Only run Newton updates if maximum absolute gradient is >= 1e-9 to save time. #455

  • Suppress nlminb() warnings by default, which can usually be ignored by the user and may be confusing. This can be controlled via sdmTMB(..., control = sdmTMBcontrol(suppress_nlminb_warnings = FALSE)). This option now mirrors tinyVAST.

  • Round time-varying AR(1) rho to 2 decimals in model printing/summary.

- R
Published by seananderson 7 months ago

sdmTMB - sdmTMB 0.7.2

New features

  • Add deviance residuals (residuals(fit, type = "deviance")) and deviance.sdmTMB() method (deviance(fit)). Proportion deviance explained can be calculated as 1 - deviance(fit) / deviance(fit_null) where fit_null is a null model, e.g., fit with formula = ~ 1 and turning off any random fields as desired (e.g., spatial = "off", spatiotemporal = "off").

  • Add observation_error argument to simulate.sdmTMB() to allow turning off observation error simulation. The intended use-case is for simulating from random effects but not adding observation error. #431

Minor improvements and fixes

  • Change the default in dharma_residuals() to test_uniformity = FALSE. Based on simulation testing, we generally do not recommend using these p-values to reject models.

  • Fix a bug introduced in version 0.7.0 where printing of the 2nd linear predictor smoother fixed effects (bs) was accidentally a copy of the 1st linear predictor smoother fixed effects.

  • Fix bug in simulation with time-varying AR(1) when using the project() function. Thanks to A. Allyn for pointing out the bug.

  • Fix reporting of converged models with sdmTMB_cv(). A recent change resulted in reporting only 1 model converged if all models converged.

  • Remove warning about old default residuals type.

  • Fix project() and simulate.sdmTMB(..., newdata = ...) when random intercepts/slopes are present. #431

  • Remove extra TMB data slots for project() and simulate.sdmTMB(..., newdata = ...) to save memory. #431

- R
Published by seananderson 8 months ago

sdmTMB - sdmTMB 0.7.0

New features

  • Add option for random slopes, or random intercepts to be passed in in lme4 style formulas, density ~ (1 | fyear) or density ~ (depth | fyear), Matches output of lme4 and glmmTMB, and summarizes output with tidy().

  • Add project() experimental function.

  • Add get_eao() to calculate effective area occupied.

  • Allow predicting on new data with t2() smoothers. #413

  • Add priors for breakpt() and logistic() parameters. #403

  • Add priors on time-varying SD parameters (sigma_V).

  • Add cAIC() for calculating conditional AIC. Theory based on https://arxiv.org/abs/2411.14185; also see https://doi.org/10.1002/ecy.4327. J.T. Thorson wrote the function code. EDF (effective degrees of freedom) will ultimately be further split (e.g., split by smoothers) and added to summary.sdmTMB(). #383 #387

  • Add EDF (effective degrees of freedom) printing to smoothers with print.sdmTMB() and summary.sdmTMB(). Set argument edf = TRUE. E.g. print(fit, edf = TRUE). #383 #387

  • At experimental function get_index_split(), which takes care of splitting a prediction grid by time, undoing the prediction and area-integration index calculations for each chunk to save memory.

  • Add newdata argument to simulate.sdmTMB(). This enables simulating on a new data frame similar to how one would predict on new data.

  • Add mle_mvn_samples argument to simulate.sdmTMB(). Defaults to "single". If "multiple", then a sample from the random effects is taken for each simulation iteration.

  • Allow for specifying only lower or upper limits. #394

  • sdmTMB_cv() gains a tidy() and print() method for output. #319

  • simulate.sdmTMB() method now has an return_tmb_report argument.

New vignettes/articles

Minor improvements and fixes

  • Add a useful error if memory error occurs on index calculation.

  • Fix bug in a check in make_mesh() around if coordinates look overly large. #427

  • Re-enable bias correction for get_cog() (get center of gravity).

  • Add check for Inf/-Inf values before fitting. #408

  • Add linear component of smoothers to tidy(). #90

  • Add time varying AR(1) correlation to tidy() and print(). #374

  • Warn if parameter limits are set with newton_loops > 0. #394

  • Fix bug in est column when predicting on new data with Poisson-link delta models with type = "link" and re_form = NA. #389

  • Fix bug in s95 parameter reporting from the tidy() method. s95 is present in the logistic threshold models. The model itself was fine but the s95 parameter was supposed to be reported by tidy() as a combination of two other parameters. This also affected the output in print()/summary().

  • Add progress bar to simulate.sdmTMB(). #346

  • Add AUC and TSS examples to cross validation vignette. #268

  • Add model (linear predictor number) argument to coef() method. Also, write documentation for ?coef.sdmTMB. #351

  • Add helpful error message if some coordinates in make_mesh() are NA. #365

  • Add informative message if fitting with an offset but predicting with offset argument left at NULL on newdata. #372

  • Fix passing of offset argument through in sdmTMB_cv(). Before it was being omitted in the prediction (i.e., set to 0). #372

  • Fig bug in exponentiate argument for tidy(). Set conf.int = TRUE as default. #353

  • Fix bug in prediction from delta_truncated_nbinom1() and delta_truncated_nbinom2() families. The positive component needs to be transformed to represent the mean of the untruncated distribution first before multiplying by the probability of a non-zero. Thanks to @tom-peatman #350

  • Add option for area to be passed in as the name of a column in the data frame to be used for area weighting. Used in get_index(), get_cog(), get_eao(), etc.

- R
Published by seananderson 11 months ago

sdmTMB - sdmTMB 0.6.0

  • Pass several arguments to DHARMa::plotQQunif().

  • Add silent option in simulate.sdmTMB(). Setting it to FALSE allows monitoring simulations from larger models.

  • Fix bug in est_non_rf1 and est_non_rf2 columns when all the following conditions were true:

    • predicting on new data
    • using a delta model
    • including IID random intercepts or time-varying coefficients See #342. Thanks to @tom-peatman for the issue report.

  • Fix delta-gamma binomial link printing for type = 'poisson-link' #340

  • Add suggestion to use an optimized BLAS library to README.

  • Add warning if it's detected that there were problems reloading (e.g., with readRDS()) a fitted model. Simultaneously revert the approach to how reloaded models are reattached.

  • Move log_ratio_mix parameter to 2nd phase with starting value of -1 instead of 0 to improve convergence.

  • Fix bugs for nbinom1() and nbinom2_mix() simulation.

  • Allow profile argument in the control list to take a character vector of parameters. This move these parameters from the outer optimization problem to the inner problem (but omits from the from the Laplace approximation). See documentation in TMB. This can considerably speed up fitting models with many fixed effects.

  • Add theoretical quantile residuals for the generalized gamma distribution. Thanks to J.C. Dunic. #333

  • Add "poisson-link" option to delta-mixture lognormal.

  • Fix bug in simulation from Poisson-link delta models.

  • Simplify the internal treatment of extra time slices (extra_time). #329 This is much less bug prone and also fixes a recently introduced bug. #335 This can slightly affect model results compared to the previous approach if extra time was used along with smoothers since the 'fake' extra data previously used was included when mgcv determined knot locations for smoothers.

- R
Published by seananderson over 1 year ago

sdmTMB - sdmTMB 0.5.0

  • Overhaul residuals vignette ('article') https://pbs-assess.github.io/sdmTMB/articles/web_only/residual-checking.html including brief intros to randomized quantile residuals, simulation-based residuals, 'one-sample' residuals, and uniform vs. Gaussian residuals.

  • Add check if prediction coordinates appear outside of fitted coordinates. #285

  • Fix memory issue with Tweedie family on large datasets. #302

  • Add experimental option to return standard normal residuals from dharma_residuals().

  • Make simulate.sdmTMB() not include extra_time elements.

  • Improved re-initialization of saved fitted model objects in new sessions.

  • Fix important bug in simulate.sdmTMB() method for delta families where the positive linear predictor was only getting simulated for observations present in the fitted data.

  • Add new "mle-mvn" type to residuals.sdmTMB() and make it the default. This is a fast option for evaluating goodness of fit that should be better than the previous default. See the details section in ?residuals.sdmTMB for details. The previous default is now called "mvn-eb" but is not recommended.

  • Bring dharma_residuals() back over from sdmTMBextra to sdmTMB. Add a new option in the type argument ("mle-mvn") that should make the simulation residuals consistent with the expected distribution. See the same new documentation in ?residuals.sdmTMB. The examples in ?dharma_residuals illustrate suggested use.

  • Fix bug in sanity() where gradient checks were missing abs() such that large negative gradients weren't getting caught. #324

  • Return offset vector in fitted object as an element. Ensure any extra time rows of data in the data element of the fitted object do not include the extra time slices.

  • Add experimental residuals option "mle-mvn" where a single approximate posterior sample of the random effects is drawn and these are combined with the MLE fixed effects to produce residuals. This may become the default option.

  • Add the generalized gamma distribution (thanks to J.T. Thorson with additional work by J.C. Dunic.) See gengamma(). This distribution is still in a testing phase and is not recommended for applied use yet. #286

  • Detect possible issue with factor(time) in formula if same column name is used for time and extra_time is specified. #320

  • Improve sanity() check output when there are NA fixed effect standard errors.

  • Set intern = FALSE within index bias correction, which seems to be considerably faster when testing with most models.

- R
Published by seananderson almost 2 years ago

sdmTMB - sdmTMB 0.4.3

  • Fix a bug likely introduced in July 2023 that caused issues when extra_time was specified. This is an important bug and models fit with extra_time between that date (if using the GitHub version) and v0.4.2.9004 (2024-02-24) should be checked against a current version of sdmTMB (v0.4.2.9005 or greater). On CRAN, this affected v0.4.0 (2023-10-20) to v0.4.2. Details:

    • The essence of the bug was that extra_time works by padding the data with a fake row of data for every extra time element (using the first row of data as the template). This is supposed to then be omitted from the likelihood so it has no impact on model fitting beyond spacing time-series processes appropriately and setting up internal structures for forecasting. Unfortunately, a bug was introduced that caused these fake data (1 per extra time element) to be included in the likelihood.

  • Issue error if time column has NAs. #298 #299

  • Fix bug in get_cog(..., format = "wide") where the time column was hardcoded to "year" by accident.

  • Poisson-link delta models now use a type argument in delta_gamma() and delta_lognormal(). delta_poisson_link_gamma() and delta_poisson_link_lognormal() are deprecated. #290

  • Delta families can now pass links that are different from the default "logit" and "log". #290

- R
Published by seananderson almost 2 years ago

sdmTMB - sdmTMB 0.4.2

  • Force rebuild of CRAN binaries to fix issue with breaking Matrix ABI change causing NaN gradient errors. #288 #287

  • Fix crash in if sdmTMB(..., do_index = TRUE) and extra_time supplied along with predict_args = list(newdata = ...) that lacked extra_time elements.

  • Allow get_index() to work with missing time elements.

  • Add the ability to pass a custom randomized quantile function qres_func to residuals.sdmTMB().

  • Add check for factor random intercept columns in newdata to avoid a crash.

    278 #280

  • Improve warnings/errors around use of do_index = TRUE and get_index() if newdata = NULL. #276

  • Fix prediction with offset when newdata is NULL but offset is specified. #274

  • Fix prediction failure when both offset and nsim are provided and model includes extra_time. #273

- R
Published by seananderson about 2 years ago

sdmTMB - sdmTMB 0.4.1

  • Fix memory issues detected by CRAN 'Additional issues' clang-UBSAN, valgrind.

  • Fix a bug predicting on new data with a specified offset and extra_time.

    270

  • Add warning around non-factor handling of the spatial_varying formula. #269

  • Add experimental set_delta_model() for plotting delta models with ggeffects::ggpredict() (GitHub version only until next CRAN version).

- R
Published by seananderson over 2 years ago

sdmTMB - sdmTMB 0.4.0

  • Move addbarriermesh() to sdmTMBextra to avoid final INLA dependency. https://github.com/pbs-assess/sdmTMBextra

  • Switch to using the new fmesher package for all mesh/SPDE calculations. INLA is no longer a dependency.

  • Switch to diagonal.penalty = FALSE in mgcv::smoothCon(). This changes the scale of the linear component of the smoother, but should result in the same model. https://github.com/glmmTMB/glmmTMB/issues/928#issuecomment-1642862066

  • Implement cross validation for delta models #239

  • Remove ELPD from cross validation output. Use sum_loglik instead. #235

  • Turn on Newton optimization by default. #182

  • print() now checks sanity() and issues a warning if there may be issues. #176

  • Poisson-link delta models and censored likelihood distributions have been made considerably more robust. #186

  • Standard errors are now available on SD parameters etc. in tidy() #240

  • Fix bug in print()/tidy() for delta-model positive model component sigmaE. A recently introduce bug was causing sigmaE for the 2nd model to be reported as the 1st model component sigma_E.

  • Add new anisotropy plotting function.

  • Add anisotropic range printing. #149 by @jdunic

- R
Published by seananderson over 2 years ago

sdmTMB - sdmTMB 0.3.0

  • Create the sdmTMBextra package to remove rstan/tmbstan helpers, which were causing memory sanitizer errors on CRAN. https://github.com/pbs-assess/sdmTMBextra

  • The following functions are affected:

    • predict.sdmTMB() now takes mcmc_samples, which is output from sdmTMBextra::extract_mcmc().
    • simulate.sdmTMB() now takes mcmc_samples, which is output from sdmTMBextra::extract_mcmc().
    • residuals.sdmTMB() now takes mcmc_samples, which is output sdmTMBextra::predict_mle_mcmc(). This only affects residuals(..., type = "mle-mcmc").

  • Move dharma_residuals() to sdmTMBextra to reduce heavy dependencies.

  • See examples in the Bayesian and residuals vignettes or in the help files for those functions within sdmTMBextra.

  • Various fixes to pass CRAN checks. #158

  • Fix memory issue highlighted by Additional issues CRAN checks. #158

  • 'offset' argument can now be a character value indicating a column name. This is the preferred way of using an offset with parallel cross validation. #165

  • Fix parallel cross validation when using an offset vector. #165

  • Add leave-future-out cross validation functionality. #156

  • Example data qcs_grid is no longer replicated by year to save package space. #158

  • Add message with tidy(fit, "ran_pars") about why SEs are NA.

  • Add anisotropy to print() #157

  • Fix predict(..., type = "response", se_fit = TRUE), which involves issuing a warning and sticking to link space. #140

- R
Published by seananderson about 3 years ago