Prepare initial models for model fitting with Hmsc-HPC

The Mod_Prep4HPC function prepares input data and initializes models for fitting with Hmsc-HPC. It performs multiple tasks, including data preparation, defining spatial block cross-validation folds, generating Gaussian Predictive Process (GPP) knots (Tikhonov et al.), initializing models, and creating HPC execution commands. The function supports parallel processing and offers the option to include or exclude phylogenetic tree data.

The Mod_PrepData function is used to prepare habitat-specific data for Hmsc models. This function processes environmental and species presence data, reads environment variables from a file, verifies paths, loads and filters species data based on habitat type and minimum presence grid cells per species, and merges various environmental layers (e.g., CHELSA Bioclimatic variables, habitat coverage, road and railway intensity, sampling efforts) into a single dataset. Processed data is saved to disk as an *.RData file.

Usage

Mod_Prep4HPC(
  Hab_Abb = NULL,
  DirName = NULL,
  MinEffortsSp = 100L,
  PresPerSpecies = 80L,
  EnvFile = ".env",
  GPP = TRUE,
  GPP_Dists = NULL,
  GPP_Save = TRUE,
  GPP_Plot = TRUE,
  MinLF = NULL,
  MaxLF = NULL,
  Alphapw = list(Prior = NULL, Min = 20, Max = 1200, Samples = 200),
  BioVars = c("bio3", "bio4", "bio11", "bio18", "bio19", "npp"),
  QuadraticVars = BioVars,
  EffortsAsPredictor = TRUE,
  RoadRailAsPredictor = TRUE,
  HabAsPredictor = TRUE,
  RiversAsPredictor = TRUE,
  NspPerGrid = 0L,
  ExcludeCult = TRUE,
  ExcludeZeroHabitat = TRUE,
  CV_NFolds = 4L,
  CV_NGrids = 20L,
  CV_NR = 2L,
  CV_NC = 2L,
  CV_Plot = TRUE,
  CV_SAC = FALSE,
  PhyloTree = TRUE,
  NoPhyloTree = FALSE,
  OverwriteRDS = TRUE,
  NCores = 8L,
  NChains = 4L,
  thin = NULL,
  samples = 1000L,
  transientFactor = 500L,
  verbose = 200L,
  SkipFitted = TRUE,
  NumArrayJobs = 210L,
  ModelCountry = NULL,
  VerboseProgress = TRUE,
  PrepSLURM = TRUE,
  MemPerCpu = NULL,
  Time = NULL,
  JobName = NULL,
  Path_Hmsc = NULL,
  CheckPython = FALSE,
  ToJSON = FALSE,
  Precision = 64L,
  ...
)

Mod_PrepData(
  Hab_Abb = NULL,
  DirName = NULL,
  MinEffortsSp = 100L,
  ExcludeCult = TRUE,
  ExcludeZeroHabitat = TRUE,
  PresPerSpecies = 80L,
  EnvFile = ".env",
  VerboseProgress = TRUE
)

Arguments

Hab_Abb

Character. Abbreviation for the habitat type (based on SynHab) for which to prepare data. Valid values are 0, 1, 2, 3, 4a, 4b, 10, 12a, 12b. If Hab_Abb = 0, data is prepared irrespective of the habitat type. For more details, see Pysek et al..

DirName

Character. Directory name, without its parents, where the models will be saved. This directory will be created.

MinEffortsSp

Integer. Minimum number of vascular plant species per grid cell (from GBIF data) required for inclusion in the models. This is to exclude grid cells with very little sampling efforts. Defaults to 100.

PresPerSpecies

Integer. The minimum number of presence grid cells for a species to be included in the analysis. The number of presence grid cells per species is calculated after discarding grid cells with low sampling efforts (MinEffortsSp) and zero percentage habitat coverage ExcludeZeroHabitat. Defaults to 80.

EnvFile

Character. Path to the environment file containing paths to data sources. Defaults to .env.

GPP

Logical. Whether to fit spatial random effect using Gaussian Predictive Process. Defaults to TRUE. If FALSE, non-spatial models will be fitted.

GPP_Dists

Integer. Spacing (in kilometers) between GPP knots, as well as the minimum allowable distance between a knot and the nearest sampling point. The knots are generated using the Mod_PrepKnots function, and this value is used for both knotDist and minKnotDist in Hmsc::constructKnots.

GPP_Save

Logical. Whether RData file. Default: TRUE.

GPP_Plot

Logical. Whether to plot the coordinates of the sampling units and the knots in a pdf file. Default: TRUE.

MinLF, MaxLF

Integer. Minimum and maximum number of latent factors to be used. Both default to NULL which means that the number of latent factors will be estimated from the data. If either is provided, the respective values will be used as arguments to Hmsc::setPriors.

Alphapw

Prior for the alpha parameter. Defaults to a list with Prior = NULL, Min = 20, Max = 1200, and Samples = 200. If Alphapw is NULL or a list with all NULL list items, the default prior will be used. If Prior is a matrix, it will be used as the prior. If Prior = NULL, the prior will be generated using Min, Max, and Samples. Min and Max are the minimum and maximum values of the alpha parameter (in kilometer). Samples is the number of samples to be used in the prior.

BioVars

Character vector. Variables from CHELSA (bioclimatic variables (bio1-bio19) and additional predictors (e.g., Net Primary Productivity, npp)) to be used in the model. By default, six ecologically relevant and minimally correlated variables are selected: c("bio3", "bio4", "bio11", "bio18", "bio19", "npp").

QuadraticVars

Character vector for variables for which quadratic terms are used. Defaults to all variables of the BioVars. If QuadraticVars is NULL, no quadratic terms will be used.

EffortsAsPredictor

Logical. Whether to include the (log₁₀) sampling efforts as predictor to the model. Default: TRUE.

RoadRailAsPredictor

Logical. Whether to include the (log₁₀) sum of road and railway intensity as predictor to the model. Default: TRUE.

HabAsPredictor

Logical. Whether to include the (log₁₀) percentage coverage of respective habitat type per grid cell as predictor to the model. Default: TRUE. Only valid if Hab_Abb not equals to 0.

RiversAsPredictor

Logical. Whether to include the (log₁₀) total length of rivers per grid cell as predictor to the model. Default: TRUE. See River_Length for more details.

NspPerGrid

Integer. Minimum number of species required for a grid cell to be included in the analysis. This filtering occurs after applying MinEffortsSp (sampling effort thresholds), PresPerSpecies (minimum species presence thresholds), and ExcludeZeroHabitat (exclude 0% habitat coverage). Default (0): Includes all grid cells. Positive value (>0): Includes only grid cells where at least NspPerGrid species are present.

ExcludeCult

Logical. Whether to exclude countries with cultivated or casual observations per species. Defaults to TRUE.

ExcludeZeroHabitat

Logical. Whether to exclude grid cells with zero percentage habitat coverage. Defaults to TRUE.

CV_NFolds

Integer. Number of cross-validation folds. Default: 4L.

CV_NGrids

Integer. For CV_Dist cross-validation strategy (see Mod_GetCV), this argument determines the size of the blocks (how many grid cells in both directions).

CV_NR, CV_NC

Integer. Number of rows and columns used in the CV_Large cross-validation strategy (see Mod_GetCV), in which the study area is divided into large blocks given the provided CV_NR and CV_NC values. Both default to 2 which means to split the study area into four large blocks at the median latitude and longitude.

CV_Plot

Logical. Indicating whether to plot the block cross-validation folds.

CV_SAC

Logical. Whether to use the spatial autocorrelation to determine the block size. Defaults to FALSE,

PhyloTree, NoPhyloTree

Logical. Whether to fit models with (PhyloTree) or without (NoPhyloTree) phylogenetic trees. Defaults are PhyloTree = TRUE and NoPhyloTree = FALSE, meaning only models with phylogenetic trees are fitted by default. At least one of PhyloTree and NoPhyloTree should be TRUE.

OverwriteRDS

Logical. Whether to overwrite previously exported RDS files for initial models. Default: TRUE.

NCores

Integer. Number of CPU cores to use for parallel processing. Default: 8.

NChains

Integer. Number of model chains. Default: 4.

thin

Integer vector. Thinning value(s) in MCMC sampling. If more than one value is provided, a separate model will be fitted at each value of thinning.

samples

Integer vector. Value(s) for the number of MCMC samples. If more than one value is provided, a separate model will be fitted at each value of number of samples. Defaults to 1000.

transientFactor

Integer. Transient multiplication factor. The value of transient will equal the multiplication of transientFactor and thin. Default: 500.

verbose

Integer. Interval at which MCMC sampling progress is reported. Default: 200.

SkipFitted

Logical. Whether to skip already fitted models. Default: TRUE.

NumArrayJobs

Integer. Number of jobs per SLURM script file. In LUMI HPC, there is a limit of 210 submitted jobs per user for the small-g partition. This argument is used to split the jobs into multiple SLURM scripts if needed. Default: 210. See LUMI documentation for more details.

ModelCountry

Character. Country or countries to filter observations by. Default: NULL, which means prepare data for the whole Europe.

VerboseProgress

Logical. Whether to print a message upon successful saving of files. Defaults to FALSE.

PrepSLURM

Logical. Whether to prepare SLURM command files. If TRUE (default), the SLURM commands will be saved to disk using the Mod_SLURM function.

MemPerCpu

Character. Memory per CPU for the SLURM job. This value will be assigned to the #SBATCH --mem-per-cpu= SLURM argument. Example: "32G" to request 32 gigabyte. Only effective if PrepSLURM = TRUE.

Time

Character. Requested time for each job in the SLURM bash arrays. Example: "01:00:00" to request an hour. Only effective if PrepSLURM = TRUE.

JobName

Character. Name of the submitted job(s) for SLURM. If NULL (Default), the job name will be prepared based on the folder path and the Hab_Abb value. Only effective if PrepSLURM = TRUE.

Path_Hmsc

Character. Directory path to Hmsc-HPC extension installation. This will be provided as the Path_Hmsc argument of the Mod_SLURM function.

CheckPython

Logical. Whether to check if the Python executable exists.

ToJSON

Logical. Whether to convert unfitted models to JSON before saving to RDS file. Default: FALSE.

Precision

Integer (either 32 or 64). Defines the floating-point precision mode for Hmsc-HPC sampling (–fp 32 or –fp 64). The default is 64, which is the default precision in Hmsc-HPC.

...

Additional parameters provided to the Mod_SLURM function.

Author

Ahmed El-Gabbas