Univariate Profiles

univariate_confidenceintervals!(model::LikelihoodModel, 
    θs_to_profile::Vector{<:Int64}=collect(1:model.core.num_pars); 
    <keyword arguments>)

Computes likelihood-based confidence interval profiles for the provided θs_to_profile interest parameters, where θs_to_profile is a vector of Int corresponding to the parameter indexes in model.core.θnames. Saves these profiles by modifying model in place. Returns the evaluated confidence intervals in a DataFrame using get_uni_confidence_intervals.

Arguments

• model: a LikelihoodModel containing model information, saved profiles and predictions.
• θs_to_profile: vector of parameters to profile, as a vector of model parameter indexes. Default is collect(1:model.core.num_pars), or all parameters.

Keyword Arguments

• confidence_level: a number ∈ (0.0, 1.0) for the confidence level to evaluate the confidence interval. Default is 0.95 (95%).
• dof: an integer ∈ [1, model.core.num_pars] for the degrees of freedom used to define the asymptotic threshold (LikelihoodBasedProfileWiseAnalysis.get_target_loglikelihood) which defines the extremities of the univariate profile, i.e. the confidence interval. For parameter confidence intervals that are considered individually, it should be set to 1. For intervals that are considered simultaneously, it should be set to the number of intervals that are being calculated, i.e. model.core.num_pars when we wish the confidence interval for every parameter to hold simultaneously. Default is 1. Setting it to model.core.num_pars should be reasonable when making predictions for well-identified models with <10 parameters. Note: values other than 1 and model.core.num_pars may not have a clear statistical interpretation.
• profile_type: whether to use the true log-likelihood function or an ellipse approximation of the log-likelihood function centred at the MLE (with optional use of parameter bounds). Available profile types are LogLikelihood, EllipseApprox and EllipseApproxAnalytical. Default is LogLikelihood() (LogLikelihood).
• θlb_nuisance: a vector of lower bounds on nuisance parameters, require θlb_nuisance .≤ model.core.θmle. Default is model.core.θlb.
• θub_nuisance: a vector of upper bounds on nuisance parameters, require θub_nuisance .≥ model.core.θmle. Default is model.core.θub.
• use_existing_profiles: boolean variable specifying whether to use existing profiles of a parameter θi to decrease the width of the bracket used to search for the desired confidence interval using LikelihoodBasedProfileWiseAnalysis.get_interval_brackets. Existing profiles must have been calculated using the same value of dof. Default is false.
• use_ellipse_approx_analytical_start: boolean variable specifying whether to use existing profiles at confidence_level and dof of type EllipseApproxAnalytical of a parameter θi to decrease the width of the bracket used to search for the desired confidence interval. Can decrease search times significantly for LogLikelihood profile types. Default is false.
• num_points_in_interval: an integer number of points to optionally evaluate within the confidence interval for each interest parameter using get_points_in_intervals!. Points are linearly spaced in the interval and have their optimised log-likelihood value recorded. Useful for plots that visualise the confidence interval or for predictions from univariate profiles. Default is 0.
• additional_width: a Real number greater than or equal to zero. Specifies the additional width to optionally evaluate outside the confidence interval's width if num_points_in_interval is greater than 0 using get_points_in_intervals!. Half of this additional width will be placed on either side of the confidence interval. If the additional width goes outside a bound on the parameter, only up to the bound will be considered. The spacing of points in the additional width will try to match the spacing of points evaluated inside the interval. Useful for plots that visualise the confidence interval as it shows the trend of the log-likelihood profile outside the interval range. Default is 0.0.
• existing_profiles: Symbol ∈ [:ignore, :overwrite] specifying what to do if profiles already exist for a given interest parameter, confidence_level and profile_type. See below for each symbol's meanings. Default is :ignore.
• find_zero_atol: a Real number greater than zero for the absolute tolerance of the log-likelihood function value from the target value to be used when searching for confidence intervals. Default is model.find_zero_atol.
• optimizationsettings: a OptimizationSettings containing the optimisation settings used to find optimal values of nuisance parameters for a given interest parameter value. Default is missing (will use model.core.optimizationsettings).
• show_progress: boolean variable specifying whether to display progress bars on the percentage of θs_to_profile completed and estimated time of completion. Default is model.show_progress.
• use_distributed: boolean variable specifying whether to use a normal for loop or a @distributed for loop across combinations of interest parameters. Set this variable to false if Distributed.jl is not being used. Default is true.
• use_threads: boolean variable specifying, if use_distributed is false, whether to use a parallelised for loop across Threads.nthreads() threads or a non-parallel for loop within the call to get_points_in_intervals!. Default is true.

Details

By calling LikelihoodBasedProfileWiseAnalysis.univariate_confidenceinterval this function finds each side of the confidence interval using a bracketing method for interest parameters in θs_to_profile (depending on the setting for existing_profiles if these profiles already exist). Nuisance parameters of each point in univariate interest parameter space are found by maximising the log-likelihood function given by profile_type. Updates model.uni_profiles_df for each successful profile and saves their results as a UnivariateConfidenceStruct in model.uni_profiles_dict, where the keys for the dictionary is the row number in model.uni_profiles_df of the corresponding profile. model.uni_profiles_df.num_points is the number of points currently saved within the confidence interval inclusive.

Extended help

Valid bounds

The bracketing method utilised via Roots.jl's find_zero will be unlikely to converge to the true confidence interval for a given parameter if the bounds on that parameter are +/- Inf or the log-likelihood function evaluates to +/- Inf. Bounds should be set to prevent this from occurring.

Distributed Computing Implementation

If Distributed.jl is being used and use_distributed is true, then the univariate profiles of each interest parameter will be computed in parallel across Distributed.nworkers() workers. If use_distributed is false and use_threads is true then after the confidence intervals of each interest parameter have been computed, any interval points specified using num_points_in_interval will be computed in parallel across Threads.nthreads() threads for each interest parameter.

Iteration Speed Of the Progress Meter

An iteration within the progress meter is specified as one iteration per side of the confidence interval found and an additional iteration for once points within the interval have been found if num_points_in_interval > 0. This means on a per interest parameter basis, there are either two or three iterations counted in time/it calculation.

univariate_confidenceintervals!(model::LikelihoodModel, 
    θs_to_profile::Vector{<:Symbol}; 
    <keyword arguments>)

Profiles only the provided θs_to_profile interest parameters, where θs_to_profile is a vector of Symbol corresponding to the parameter symbols in model.core.θnames. Returns the evaluated confidence intervals in a DataFrame using get_uni_confidence_intervals.

univariate_confidenceintervals!(model::LikelihoodModel, 
    profile_m_random_combinations::Int; 
    <keyword arguments>)

Profiles m random interest parameters (sampling without replacement), where 0 < m ≤ model.core.num_pars. Returns the evaluated confidence intervals in a DataFrame using get_uni_confidence_intervals.

get_points_in_intervals!(model::LikelihoodModel, 
    num_points_in_interval::Int; 
    <keyword arguments>)

Evaluate and save num_points_in_interval linearly spaced points between the confidence intervals of existing univariate profiles that meet the requirements of the univariate method of LikelihoodBasedProfileWiseAnalysis.desired_df_subset (see Keyword Arguments), as well as any additional width on the sides of the interval. Modifies model in place.

Arguments

• model: a LikelihoodModel containing model information, saved profiles and predictions.
• num_points_in_interval: an integer number of points to evaluate within the confidence interval. Points are linearly spaced in the interval and have their optimised log-likelihood value recorded (standardised to 0.0 at the MLE point). Useful for plots that visualise the confidence interval or for predictions from univariate profiles.

Keyword Arguments

• additional_width: a Real number greater than or equal to zero. Specifies the additional width to optionally evaluate outside the confidence interval's width. Half of this additional width will be placed on either side of the confidence interval. If the additional width goes outside a bound on the parameter, only up to the bound will be considered. The spacing of points in the additional width will try to match the spacing of points evaluated inside the interval. Useful for plots that visualise the confidence interval as it shows the trend of the log-likelihood profile outside the interval range. Default is 0.0.
• confidence_levels: a vector of confidence levels. If empty, all confidence levels of univariate profiles will be considered for finding interval points. Otherwise, only confidence levels in confidence_levels will be considered. Default is Float64[] (any confidence level).
• dofs: a vector of integer degrees of freedom used to define the asymptotic threshold for the extremities of a univariate profile. If empty, all degrees of freedom for univariate profiles will be considered for evaluating predictions from. Otherwise, only degrees of freedom in dofs will be considered. Default is Int[] (any degree of freedom).
• profile_types: a vector of AbstractProfileType structs. If empty, all profile types of univariate profiles are considered. Otherwise, only profiles with matching profile types will be considered. Default is AbstractProfileType[] (any profile type).
• θlb_nuisance: a vector of lower bounds on nuisance parameters, require θlb_nuisance .≤ model.core.θmle. Default is model.core.θlb.
• θub_nuisance: a vector of upper bounds on nuisance parameters, require θub_nuisance .≥ model.core.θmle. Default is model.core.θub.
• not_evaluated_predictions: a boolean specifying whether to only get points in intervals of profiles that have not had predictions evaluated (true) or for all profiles (false). If false, then any existing predictions will be forgotten by the model and overwritten the next time predictions are evaluated for each profile. Default is true.
• optimizationsettings: a OptimizationSettings containing the optimisation settings used to find optimal values of nuisance parameters for a given interest parameter value. Default is missing (will use model.core.optimizationsettings).
• show_progress: boolean variable specifying whether to display progress bars on the percentage of θs_to_profile completed and estimated time of completion. Default is model.show_progress.
• use_threads: boolean variable specifying, if the number of workers for distributed computing is not greater than 1 (!Distributed.nworkers()>1), to use a parallelised for loop across Threads.nthreads() threads to evaluate the interval points. Default is false.

Details

Interval points and their corresponding log-likelihood values are stored in the interval_points field of a UnivariateConfidenceStruct. These are updated using LikelihoodBasedProfileWiseAnalysis.update_uni_dict_internal!. Nuisance parameters of each point in univariate interest parameter space are found by maximising the log-likelihood function given by the profile_type of the profile.

If get_points_in_intervals! has already been used on a univariate profile, with the same values of num_points_in_interval and additional_width, it will not be recomputed for that profile.

Parallel Computing Implementation

If Distributed.jl is being used then each set of interval points for distinct interest parameters will be computed in parallel across Distributed.nworkers() workers. If it is not being used (Distributed.nworkers() is equal to 1) and use_threads is true then the interval points of each interest parameter will be computed in parallel across Threads.nthreads() threads . It is highly recommended to set use_threads to true in that situation.

Iteration Speed Of the Progress Meter

An iteration within the progress meter is specified as the time it takes for all internal points within a univariate confidence interval to be found (as well as any outside, if additional_width is greater than zero).

get_uni_confidence_intervals(model::LikelihoodModel, uni_row_numbers::AbstractVector{<:Int})

Returns the confidence intervals corresponding to the profile in rows uni_row_numbers of model.uni_profiles_df as a DataFrame. If an entry has value NaN, that side of the confidence interval is outside the corresponding bound on the interest parameter.

get_uni_confidence_interval(model::LikelihoodModel, uni_row_number::Int)

Returns the confidence interval corresponding to the profile in row uni_row_number of model.uni_profiles_df as a vector of length two. If an entry has value NaN, that side of the confidence interval is outside the corresponding bound on the interest parameter.

get_uni_confidence_interval_points(model::LikelihoodModel, uni_row_number::Int)

Returns the interval points PointsAndLogLikelihood struct corresponding to the profile in row uni_row_number of model.uni_profiles_df.