Skip to contents

verification

verification.Rd

Run the model and get a goodness-of-fit value by comparing the simulated values against observations for the optimum parameter set found by optimisation

Usage

verification(fn="hydromod", par, ..., control=list(), 
             model.FUN=NULL, model.FUN.args=list(),
             change.type="repl", refValue=NULL )

Arguments

fn

character, name of a valid R function to be optimised or character value 'hydromod'. When fn='hydromod' the algorithm uses model.FUN and model.FUN.args to extract the values simulated by the model and to compute its corresponding goodness-of-fit function. When fn!='hydromod' the algorithm uses the value(s) returned by fn as both model output and its corresponding goodness-of-fit.

When fn='hydromod' the algorithm will optimise the model defined by model.FUN and model.args

...

OPTIONAL. Only used when fn!='hydromod' & fn!='hydromodInR'.

further arguments to be passed to fn.

par

numeric, or matrix/data.frame with the parameter sets that will be used for verification.

Parameter sets in par must be stored by row, i.e., each different row represents a different parameter set

control

a list of control parameters. See ‘Details’.

model.FUN

OPTIONAL. Only used when fn='hydromod'.

character, valid R function representing the model code to be calibrated/optimised

model.FUN.args

OPTIONAL. Only used when fn='hydromod'.

list with the arguments to be passed to model.FUN

change.type

OPTIONAL. Used only when fn='hydromodInR'. Character of length 1 or length equal to the number of parameters, indicating how the parameter sets in par are converted to the parameter values passed to model.FUN. Valid values are "repl" for replacement, "addi" for additive changes, and "mult" for multiplicative changes. A single value is recycled to all parameters. The default "repl" preserves the previous behaviour.

refValue

OPTIONAL. Used only when fn='hydromodInR' and any element of change.type is "addi" or "mult". Numeric reference value of length 1 or length equal to the number of parameters, used to compute the parameter values passed to model.FUN. Only parameters using "addi" or "mult" require and use their corresponding reference value.

Details

The control argument is a list that can supply any of the following components:

drty.in

character, path to the directory storing the input files required for PSO, i.e. ‘ParamRanges.txt’ and ‘ParamFiles.txt’.

drty.out

character, path to the directory storing the output files generated by hydroPSO.

digits

OPTIONAL. Only used when write2disk=TRUE.

numeric, number of significant digits used for writing the outputs in scientific notation

gof.name

character, ONLY used for identifying the goodness-of-fit of each model run and writing it to the Verification-logfile.txt, Verification-ParamValues.txt output files.

MinMax

character, indicates whether the optimum value for the analysed problem corresponds to the minimum or maximum of the the objective function. It is used to select the ‘best’ parameter set.

Valid values are in: c('min', 'max')

do.plots

logical, if TRUE a PNG plot with the comparison between observed and simulated values should be produced.

write2disk

logical, indicates if the output files will be written to the disk.

verbose

logical, if TRUE progress messages are printed.

parallel

character, indicates how to parallelise ‘verification’ (to be precise, only the evaluation of the objective function fn is parallelised). Valid values are:

-)none: no parallelisation is made (this is the default value)

-)multicore: DEPRECATED. The former multicore package is no longer available from CRAN. For backward compatibility, parallel="multicore" is automatically changed to parallel="parallel" on Unix-like systems and to parallel="parallelWin" on Microsoft Windows.

-)parallel: parallel computations using a ‘FORK’ cluster created with makeForkCluster. FORK clusters are available only on Unix-like operating systems, such as GNU/Linux and macOS. They are not supported on Microsoft Windows; on Windows, parallel="parallel" is automatically changed to parallel="parallelWin".

-)parallelWin: parallel computations using a ‘PSOCK’ cluster created with makeCluster. PSOCK clusters are the only parallel cluster option supported by parallel on Microsoft Windows, and they also work on Unix-like systems.

For both parallel="parallel" and parallel="parallelWin", fn.name="hydromod" and fn.name="hydromodInR" model runs are distributed one parameter set at a time using the selected cluster. Ordinary R objective functions are evaluated row-wise with the parRapply function of the parallel package.

On Unix-like systems, FORK clusters are usually faster than PSOCK clusters because workers inherit the parent R session and require less explicit object/package export. PSOCK clusters are more portable but usually have higher startup and data-transfer overhead, especially when large objects must be sent to workers.

par.nnodes

OPTIONAL. Used only when parallel!='none'

numeric, indicates the number of cores/CPUs to be used in the local multi-core machine, or the number of nodes to be used in the network cluster. By default par.nnodes is set to the amount of cores detected by detectCores.

par.pkgs

OPTIONAL. Used only when parallel='parallelWin'

list of package names (as characters) that need to be loaded on each PSOCK worker for allowing the objective function fn to be evaluated.

par.env

OPTIONAL. Used only when parallel!='none'

environment from which the objects required by the objective function fn, model.FUN, out.FUN, or gof.FUN are exported to PSOCK workers. By default, the environment from which verification is called is used.

par.export

OPTIONAL. Used only when parallel!='none'

character vector with the names of additional objects to be exported from par.env to PSOCK workers. By default, all functions in par.env are exported.

Value

A list of four elements:

gofs

numeric with the goodness-of-fit values corresponding to each one of the parameter sets provided in par

.

model.values

data.frame with the model outputs corresponding to each one of the parameter sets provided in par

.

best.param

numeric with the parameter values of the "best" paraemter set found during the verification period

.

best.gof

numeric with the goodness-of-fit of the "best" parameter set found during the verification period

Author

Mauricio Zambrano-Bigiarini, mzb.devel@gmail.com

See also

hydromod, hydromodInR

Examples

## R-external model verification with parameter metadata in 'drty.in'.
## The shell script used here is skipped on Windows.
if (.Platform$OS.type != "windows") {

  model.drty <- tempfile("verification-model-")
  drty.in    <- file.path(model.drty, "PSO.in")
  drty.out   <- tempfile("verification-out-")
  dir.create(drty.in, recursive=TRUE)

  writeLines("      0.00", file.path(model.drty, "input.txt"))

  writeLines(
    c("ParameterNmbr ParameterName MinValue MaxValue",
      "1 P1 0 10"),
    file.path(drty.in, "ParamRanges.txt")
  )

  writeLines(
    c("ParameterNmbr ParameterName Filename Row.Number Col.Start Col.End Decimals",
      "1 P1 input.txt 1 1 10 2"),
    file.path(drty.in, "ParamFiles.txt")
  )

  exe.fname <- file.path(model.drty, "run.model")
  writeLines(c("#!/bin/sh", "exit 0"), exe.fname)
  Sys.chmod(exe.fname, mode="0755")

  read_model_output <- function(file) {
    value <- as.numeric(trimws(readLines(file, warn=FALSE)[1]))
    c(value, value + 1)
  }

  sum_gof <- function(sim, obs) {
    sum(sim - obs)
  }

  out <- verification(
    fn="hydromod",
    par=matrix(c(1.25, 2.50), ncol=1),
    control=list(drty.in=drty.in, drty.out=drty.out, verbose=FALSE),
    model.FUN=hydromod,
    model.FUN.args=list(
      param.files="ParamFiles.txt",
      param.ranges="ParamRanges.txt",
      model.drty=model.drty,
      exe.fname=exe.fname,
      out.FUN=read_model_output,
      out.FUN.args=list(file=file.path(model.drty, "input.txt")),
      gof.FUN=sum_gof,
      obs=c(0, 0),
      verbose=FALSE
    )
  )

  out$gofs
}
#> [1] 3.5 6.0