Reduction of Error (RE) measures a model's predictive skill compared to assuming the long term mean. RE is used for the calibration and is mathematically identical to the Nash-Sutcliffe Efficiency (NSE) commonly used by hydrologists.

RE ranges from 1 (perfect fit) to -∞. Values greater than zero indicate that the model has some skill (performs better than assuming the mean).

Efficiency Criteria

Coefficient of Efficiency (CE) is equivalent to the Reduction of Error (RE), but applied to validation data.

CE therefore ranges from 1 (perfect skill) to -∞. Values greater than zero indicate that the model has some skill (performs better than assuming the mean).

Because CE measures skill for the hold-out set, we expect that CE < RE. However, this drop in skill should not be too large, or it points to over-fitting.

Efficiency Criteria

RMSE provides the 'typical' error and is an absolute measure of fit, meaning it is displayed in flow units.

RMSE is calculated as the square root of mean square of errors (predicted minus observed flow). A perfect model would have no errors, and therefore RMSE = 0.

Reconstruction-Statistical Background

Mean Absolute Error (MAE) calculates the average absolute (no sign) difference between predicted and observed flows. It is displayed in original flow units and a perfect fit have MAE = 0.

MAE is similar in interpretation to RMSE and is the more intuitive of the two. Where RMSE handles negative values by squaring them, MAE simply applies the absolute value ||.

Mean Error (ME) calculates the average error (predicted - observed). It is a measure of bias, if the model consistently over-predicts or under-predicts. Unbiased models have a ME near zero.

Mean Error should always be considered with other metrics because poor predictive models can have ME near zero if large positive and negative errors can cancel each other out.

Validation is typically performed by splitting the observed record into a 'training' set and a 'validation' set. A model fit using the training set is then used to predict values in the witheld valdation set to simulate prediction errors outside the observed record.

'Split Sample' validation defines these training and validation sets explicitly when fitting the model.

'K-fold' cross-validation predicts part of the sample (typically 10%, referred to as 'k') and predicts using the remaining 90%. This is repeated k times until each point is predicted without being used in training.

Leave-One-Out is a commonly used method and is an extreme version of K-fold cross-validation, where each point is predicted using a model fit with all other data.