Predictions

add_prediction_function!(model::LikelihoodModel, predictfunction::Function)

Adds a prediction function, predictfunction, to model and evaluates the predicted response variable(s) at the data points using the maximum likelihood estimate for model parameters. Modifies model in place.

Requirements for predictfunction

• A function to generate model predictions from that is paired with the loglikefunction.
• Takes three arguments, θ, data and t, in that order, where θ and data are the same as for loglikefunction and t needs to be an optional third argument.
• When t is not specified, the prediction function should be evaluated for the same time points/independent variable as the data. When t is specified, the prediction function should be evaluated for those specified time points/independent variable. A good practice is to include the time points of the data in the argument, data, so that the function can be specified as predictfunction(θ, data, t=data.t) (here data is a NamedTuple).
• The output of the function should be a 1D vector when there is a single predicted response variable or a 2D array when there are multiple predicted response variables.
• The prediction(s) for each response variable should be stored in the columns of the array. In Julia, vectors are stored column-wise, so in the case where there is only one response variable, this will already be the case.
• The number of rows of each predicted response variable should be the same length as the vector t used to evaluate the response.

check_prediction_function_exists(model::LikelihoodModel)

Checks if a prediction function is stored in model, returning true if there is. Otherwise false is returned and a warning is logged. Requirements for a prediction function can be seen in add_prediction_function!.

add_error_function!(model::LikelihoodModel, errorfunction::Function)

Adds an error function, errorfunction, to model. Modifies model in place. Model must contain a prediction function model.core.predictfunction to add the error function.

Requirements for errorfunction

• A function to generate lower and upper confidence quartiles (reference intervals) of each prediction realisation from predictfunction.
• Takes three arguments, predictions, θ, and region in that order, where predictions is the array of predictions generated by predictfunction, θ is the same as for loglikefunction and region is the highest density region to evaluate quartiles of the error at each prediction point.
• The output of the function should be a lower quartile and a upper quartile array, each with the same dimensions as predictions.

Predefined error functions available to use

• normal_error_σ_known
• normal_error_σ_estimated
• lognormal_error_σ_known
• lognormal_error_σ_estimated
• logitnormal_error_σ_known
• logitnormal_error_σ_estimated
• poisson_error

normal_error_σ_known(predictions::AbstractArray, 
    θ::AbstractVector, 
    region::Float64, 
    σ::Real)

Use a normal error model to quantify the uncertainty in the predictions of realisations, with known value of σ.

To use this function as the error model you must create a new function with only the first three arguments, which calls this function with a set value of σ.

Two equivalent examples of this specification with σ set to 1.3:

errorfunction(predictions, θ, region) = normal_error_σ_known(predictions, θ, region, 1.3)

function errorfunction(predictions, θ, region) 
    normal_error_σ_known(predictions, θ, region, 1.3)
end

normal_error_σ_estimated(predictions::AbstractArray, 
    θ::AbstractVector, 
    region::Float64, 
    σ_θindex::Int)

Use a normal error model to quantify the uncertainty in the predictions of realisations, where σ is an estimated model parameter in θ.

To use this function as the error model you must create a new function with only the first three arguments, which calls this function with the index, σ_θindex, of σ in θ.

Two equivalent examples of this specification with σ stored at the end of the θ parameter vector, which has 4 elements (length 4):

errorfunction(predictions, θ, region) = normal_error_σ_estimated(predictions, θ, region, 4)

function errorfunction(predictions, θ, region) 
    normal_error_σ_estimated(predictions, θ, region, 4)
end

lognormal_error_σ_known(predictions::AbstractArray, 
    θ::AbstractVector, 
    region::Float64, 
    σ::Real)

Use a log normal error model to quantify the uncertainty in the predictions of realisations, with known value of σ.

To use this function as the error model you must create a new function with only the first three arguments, which calls this function with a set value of σ.

Two equivalent examples of this specification with σ set to 1.3:

errorfunction(predictions, θ, region) = lognormal_error_σ_known(predictions, θ, region, 1.3)

function errorfunction(predictions, θ, region) 
    lognormal_error_σ_known(predictions, θ, region, 1.3)
end

lognormal_error_σ_estimated(predictions::AbstractArray, 
    θ::AbstractVector, 
    region::Float64, 
    σ_θindex::Int)

Use a log normal error model to quantify the uncertainty in the predictions of realisations, where σ is an estimated model parameter in θ.

To use this function as the error model you must create a new function with only the first three arguments, which calls this function with the index, σ_θindex, of σ in θ.

Two equivalent examples of this specification with σ stored at the end of the θ parameter vector, which has 4 elements (length 4):

errorfunction(predictions, θ, region) = lognormal_error_σ_estimated(predictions, θ, region, 4)

function errorfunction(predictions, θ, region) 
    lognormal_error_σ_estimated(predictions, θ, region, 4)
end

logitnormal_error_σ_known(predictions::AbstractArray, 
    θ::AbstractVector, 
    region::Float64, 
    σ::Real)

Use a logit-normal error model to quantify the uncertainty in the predictions of realisations, with known value of σ. Predictions are required to be defined ∈ (0,1).

Output is the correct highest density region for around σ < 1.43 (at all values of predictions). If predictions are closer to 0 or 1.0 than 0.5, then higher values of σ (closer to 2) will be acceptable as the distribution will still be unimodal. Otherwise, the distribution will not be unimodal and won't identify the correct high density regions.

To use this function as the error model you must create a new function with only the first three arguments, which calls this function with a set value of σ.

Two equivalent examples of this specification with σ set to 0.9:

errorfunction(predictions, θ, region) = logitnormal_error_σ_known(predictions, θ, region, 0.9)

function errorfunction(predictions, θ, region) 
    logitnormal_error_σ_known(predictions, θ, region, 0.9)
end

logitnormal_error_σ_estimated(predictions::AbstractArray, 
    θ::AbstractVector, 
    region::Float64, 
    σ_θindex::Int)

Use a logit-normal error model to quantify the uncertainty in the predictions of realisations, where σ is an estimated model parameter in θ. Predictions are required to be defined ∈ (0,1).

Output is the correct highest density region for around σ < 1.43 (at all values of predictions). If predictions are closer to 0 or 1.0 than 0.5, then higher values of σ (closer to 2) will be acceptable as the distribution will still be unimodal. Otherwise, the distribution will not be unimodal and won't identify the correct high density regions.

To use this function as the error model you must create a new function with only the first three arguments, which calls this function with the index, σ_θindex, of σ in θ.

Two equivalent examples of this specification with σ stored at the end of the θ parameter vector, which has 4 elements (length 4):

errorfunction(predictions, θ, region) = logitnormal_error_σ_estimated(predictions, θ, region, 4)

function errorfunction(predictions, θ, region) 
    logitnormal_error_σ_estimated(predictions, θ, region, 4)
end

poisson_error(predictions::AbstractArray, 
    θ::AbstractVector, 
    region::Float64)

Use a poisson error model to quantify the uncertainty in the predictions of realisations, where each prediction is the mean of the error model.

To use this function as the error model you don't need to create a new function, just specify this function directly.

generate_predictions_univariate!(model::LikelihoodModel,
    t::AbstractVector,
    proportion_to_keep::Real;
    <keyword arguments>)

Evalute and save proportion_to_keep individual predictions and their extrema from existing univariate profiles that meet the requirements of the univariate method of LikelihoodBasedProfileWiseAnalysis.desired_df_subset (see Keyword Arguments) at time points t. Modifies model in place.

Arguments

• model: a LikelihoodModel containing model information, saved profiles and predictions.
• t: a vector of time points to compute predictions at.
• proportion_to_keep: a Real number ∈ [0.0,1.0] of the proportion of individual predictions to save. Default is 1.0.

Keyword Arguments

• region: a Real number ∈ [0, 1] specifying the proportion of the density of the error model from which to evaluate the highest density region. Default is 0.95.
• confidence_levels: a vector of confidence levels. If empty, all confidence levels of univariate profiles will be considered for evaluating predictions from. Otherwise, only confidence levels in confidence_levels will be considered. Default is Float64[] (any confidence level).
• dofs: a vector of integer degrees of freedom used to define the asymptotic threshold for the extremities of a univariate profile. If empty, all degrees of freedom for univariate profiles will be considered for evaluating predictions from. Otherwise, only degrees of freedom in dofs will be considered. Default is Int[] (any degree of freedom).
• profile_types: a vector of AbstractProfileType structs. If empty, all profile types of univariate profiles are considered. Otherwise, only profiles with matching profile types will be considered. Default is AbstractProfileType[] (any profile type).
• overwrite_predictions: boolean variable specifying whether to re-evaluate and overwrite predictions for univariate profiles that have already had predictions evaluated. Set to true if predictions need to be evaluated for a new vector of time points. Default is false.
• show_progress: boolean variable specifying whether to display progress bars on the percentage of predictions evaluated and estimated time of completion. Default is model.show_progress.
• use_distributed: boolean variable specifying whether to use a normal for loop or a @distributed for loop across univariate profiles. Default is true.

Details

For each univariate profile that meets the requirements of LikelihoodBasedProfileWiseAnalysis.desired_df_subset, it uses LikelihoodBasedProfileWiseAnalysis.generate_prediction to generates the predictions for every parameter point in the profiles. The extrema of these predictions are computed (these are approximate simultaneous confidence bands for the prediction mean). The extrema and proportion_to_keep of the individual predictions are saved as a PredictionStruct in model.uni_predictions_dict, where the keys for the dictionary is the row number in model.uni_profiles_df of the corresponding profile.

Distributed Computing Implementation

If Distributed.jl is being used and use_distributed is true then the predictions from each univariate profile will be computed in parallel across Distributed.nworkers() workers.

Iteration Speed Of the Progress Meter

The time/it value is the time it takes for a prediction to be evaluated from a single point in parameter space.

generate_predictions_bivariate!(model::LikelihoodModel,
    t::AbstractVector,
    proportion_to_keep::Real;
    <keyword arguments>)

Evalute and save proportion_to_keep individual predictions and their extrema from existing bivariate profiles that meet the requirements of the bivariate method of LikelihoodBasedProfileWiseAnalysis.desired_df_subset (see Keyword Arguments) at time points t. Modifies model in place.

Arguments

• model: a LikelihoodModel containing model information, saved profiles and predictions.
• t: a vector of time points to compute predictions at.
• proportion_to_keep: a Real number ∈ [0.0,1.0] of the proportion of individual predictions to save. Default is 1.0.

Keyword Arguments

• region: a Real number ∈ [0, 1] specifying the proportion of the density of the error model from which to evaluate the highest density region. Default is 0.95.
• confidence_levels: a vector of confidence levels. If empty, all confidence levels of bivariate profiles will be considered for evaluating predictions from. Otherwise, only confidence levels in confidence_levels will be considered. Default is Float64[] (any confidence level).
• dofs: a vector of integer degrees of freedom used to define the asymptotic threshold for the boundary of a bivariate profile. If empty, all degrees of freedom for bivariate profiles will be considered for evaluating predictions from. Otherwise, only degrees of freedom in dofs will be considered. Default is Int[] (any degree of freedom).
• profile_types: a vector of AbstractProfileType structs. If empty, all profile types of bivariate profiles are considered. Otherwise, only profiles with matching profile types will be considered. Default is AbstractProfileType[] (any profile type).
• methods: a vector of AbstractBivariateMethod structs. If empty all methods used to find bivariate profiles are considered. Otherwise, only profiles with matching method types will be considered (struct arguments do not need to be the same). Default is AbstractBivariateMethod[] (any bivariate method).
• overwrite_predictions: boolean variable specifying whether to re-evaluate and overwrite predictions for bivariate profiles that have already had predictions evaluated. Set to true if predictions need to be evaluated for a new vector of time points. Default is false.
• show_progress: boolean variable specifying whether to display progress bars on the percentage of predictions evaluated and estimated time of completion. Default is model.show_progress.
• use_distributed: boolean variable specifying whether to use a normal for loop or a @distributed for loop across bivariate profiles. Default is true.

Details

For each bivariate profile that meets the requirements of LikelihoodBasedProfileWiseAnalysis.desired_df_subset, it uses LikelihoodBasedProfileWiseAnalysis.generate_prediction to generates the predictions for every parameter point in the profiles. The extrema of these predictions are computed (these are approximate simultaneous confidence bands for the prediction mean). The extrema and proportion_to_keep of the individual predictions are saved as a PredictionStruct in model.biv_predictions_dict, where the keys for the dictionary is the row number in model.biv_profiles_df of the corresponding profile.

Distributed Computing Implementation

If Distributed.jl is being used and use_distributed is true then the predictions from each bivariate profile will be computed in parallel across Distributed.nworkers() workers.

Iteration Speed Of the Progress Meter

The time/it value is the time it takes for a prediction to be evaluated from a single point in parameter space.

generate_predictions_dim_samples!(model::LikelihoodModel,
    t::AbstractVector,
    proportion_to_keep::Real;
    <keyword arguments>)

Evalute and save proportion_to_keep individual predictions and their extrema from existing dimensional samples that meet the requirements of the dimensional method of LikelihoodBasedProfileWiseAnalysis.desired_df_subset (see Keyword Arguments) at time points t. Modifies model in place.

Arguments

• model: a LikelihoodModel containing model information, saved profiles and predictions.
• t: a vector of time points to compute predictions at.
• proportion_to_keep: a Real number ∈ [0.0,1.0] of the proportion of individual predictions to save. Default is 1.0.

Keyword Arguments

• region: a Real number ∈ [0, 1] specifying the proportion of the density of the error model from which to evaluate the highest density region. Default is 0.95.
• confidence_levels: a vector of confidence levels. If empty, all confidence levels of dimensional samples will be considered for evaluating predictions from. Otherwise, only confidence levels in confidence_levels will be considered. Default is Float64[] (any confidence level).
• dofs: a vector of integer degrees of freedom used to define the asymptotic threshold for the boundary of a dimensional sample. If empty, all degrees of freedom for dimensional sample will be considered for evaluating predictions from. Otherwise, only degrees of freedom in dofs will be considered. Default is Int[] (any degree of freedom).
• sample_types: a vector of AbstractSampleType structs. If empty, all sample types used to find dimensional samples are considered. Otherwise, only samples with matching sample types will be considered. Default is AbstractSampleType[] (any sample type).
• overwrite_predictions: boolean variable specifying whether to re-evaluate and overwrite predictions for dimensional samples that have already had predictions evaluated. Set to true if predictions need to be evaluated for a new vector of time points. Default is false.
• show_progress: boolean variable specifying whether to display progress bars on the percentage of predictions evaluated and estimated time of completion. Default is model.show_progress.
• use_distributed: boolean variable specifying whether to use a normal for loop or a @distributed for loop across dimensional samples. Default is true.

Details

For each dimensional sample that meets the requirements of LikelihoodBasedProfileWiseAnalysis.desired_df_subset, it uses LikelihoodBasedProfileWiseAnalysis.generate_prediction to generates the predictions for every parameter point in the samples. The extrema of these predictions are computed (these are approximate simultaneous confidence bands for the prediction mean). The extrema and proportion_to_keep of the individual predictions are saved as a PredictionStruct in model.dim_predictions_dict, where the keys for the dictionary is the row number in model.dim_samples_df of the corresponding sample.

Distributed Computing Implementation

If Distributed.jl is being used and use_distributed is true then the predictions from each dimensional sample will be computed in parallel across Distributed.nworkers() workers.

Iteration Speed Of the Progress Meter

The time/it value is the time it takes for a prediction to be evaluated from a single point in parameter space.

get_uni_prediction_set(model::LikelihoodModel, uni_row_number::Int)

Returns the PredictionStruct struct corresponding to the profile in row uni_row_number of model.uni_profiles_df.

get_bivariate_prediction_set(model::LikelihoodModel, biv_row_number::Int)

Returns the PredictionStruct struct corresponding to the profile in row biv_row_number of model.biv_profiles_df.

get_dim_prediction_set(model::LikelihoodModel, dim_row_number::Int)

Returns the PredictionStruct struct corresponding to the profile in row dim_row_number of model.dim_samples_df.

AbstractPredictionStruct

Supertype for the predictions storage struct.

Subtypes

PredictionRealisationsStruct

PredictionStruct

PredictionStruct(predictions::Array{Real}, extrema::Array{Real}, realisations::PredictionRealisationsStruct)

Struct for containing evaluated predictions corresponding to confidence profiles.

Fields

• predictions: array of model predictions evaluated at the parameters given by a particular confidence profile parameter set. If a model has multiple response variables, it assumes that model.core.predictfunction stores the prediction for each variable in its columns. Values for each response variable are stored in the 3rd dimension (row=dim1, col=dim2, page/sheet=dim3). Each column corresponds to a column of the confidence profile parameter set.
• extrema: extrema of the predictions array.
• realisations: a PredictionRealisationsStruct struct.

Supertype Hiearachy

PredictionStruct <: AbstractPredictionStruct <: Any

PredictionRealisationsStruct(lq::Array{<:Real}, uq::Array{<:Real}, extrema::Array{<:Real})

Struct for containing evaluated lower and upper confidence quartiles of the reference tolerance sets for prediction realisations corresponding to confidence profiles.

Fields

• lq: array of the lower confidence quartile of the error model evaluated at each prediction point in a corresponding PredictionStruct.
• uq: array of the upper confidence quartile of the error model evaluated at each prediction point in a corresponding PredictionStruct.
• extrema: extrema of the lq and uq arrays.

Supertype Hiearachy

PredictionRealisationsStruct <: AbstractPredictionStruct <: Any