Ellipse Functions

getMLE_hessian_and_covariance(f::Function, θmle::Vector{<:Float64})

Computes the negative hessian of function f at θmle using ForwardDiff.jl and it's pseudoinverse, returning both matrices.

Hessian identifiability is tested using LikelihoodBasedProfileWiseAnalysis.test_hessian_identifiability.

test_hessian_identifiability(Hmle::Matrix{T}, num_pars::Int) where T<:Float64

Modified R code from [14] in 'Code for Book', bassicocc.R.

The cutoff used to estimate whether a standardised eigenvalue is close enough to zero to indicate non-identifiability is 1e-12*number of parameters, which is a smaller value than used in [14] and [15] due to smaller error in the calculation of Hmle via automatic differentiation. This cutoff is meant as an indication of non-identifiability/singularity; the hessian may still be identifiable/non-singular even if a warning occurs.

analytic_ellipse_loglike(θ::Vector, θIndexes::Vector{Int}, mleTuple::@NamedTuple{θmle::Vector{T}, Γmle::Matrix{T}}) where T<:Float64

Computes the analytical value of the ellipse approximation of the profile log-likelihood function for the interest parameters in θindexes.

Will produce an equivalent result to optimising out nuisance parameters with LikelihoodBasedProfileWiseAnalysis.ellipse_loglike if there are no bounds on parameters (or not close to the values considered) and there are no issues with the invertibility of the hessian, $\mathcal{H}(\hat{\theta})$.

The analytic ellipse log-likelihood has no knowledge of lower and upper bounds on parameters. Hence profiles generated by optimising out the nuisance parameters for a given interest parameter may look different to those from the analytical profile if it enters space where a bound would be active. Pushing forward from analytical profiles may thus be infeasible if the profile has entered a space where a parameter bound should be active (e.g. a non-negativity bound).

analytic_ellipse_loglike_1D_soln(θIndex::Int, mleTuple::@NamedTuple{θmle::Vector{T}, Γmle::Matrix{T}}, targetll::T)

In order to find the asympotic confidence interval for an interest parameter at θIndex, also known as Wald intervals [2], we solve the following equation from Ref. [9, Eq. (7)].

\[\ell_c = -\frac{1}{2}(θ_i - \hat{θ_i})^2 \times Γ_{ii}(\hat{θ})^{-1}\]

\[θ_i = \hat{θ_i} + \sqrt{\frac{-2 \ell_c}{Γ_{ii}(\hat{θ})^{-1}}} \equiv \hat{θ}_i + \sqrt{-2 \ell_c \times Γ_{ii}(\hat{θ})}\]

Note: $C(\hat{θ}) = 2 \times H(\hat{θ})^{-1}$ and $Γ(\hat{θ}) = H(\hat{θ})^{-1}$, and $\ell_c = -χ^2(α, \texttt{dof})$, so the equation is equivalent to equation 7 in the above reference. $χ^2(α, \texttt{dof})$ is the $lpha$ quantile of the chi-squared distribution with dof degrees of freedom, as calculated in LikelihoodBasedProfileWiseAnalysis.get_target_loglikelihood.

ellipse_loglike(θ::Vector, mleTuple::@NamedTuple{θmle::Vector{T}, Hmle::Matrix{T}}) where T<:Float64

Returns the value of the ellipse approximation of the normalised log-likelihood function using a second-order Taylor expansion at the MLE, $\hat{θ}$.

Where: ``\hat{\ell}(θ) \approx \hat{\ell}^\mathcal{E} (\theta) = -\frac{1}{2} (\theta-\hat{\theta})' \mathcal{H}(\hat{\theta}) (\theta-\hat{\theta}) [2].

$\mathcal{H}$ is the hessian of the log-likelihood function at the MLE, as evaluated using LikelihoodBasedProfileWiseAnalysis.getMLE_hessian_and_covariance.

ellipse_like(θ::Vector{T}, mleTuple::@NamedTuple{θmle::Vector{T}, Hmle::Matrix{T}}) where T<:Float64

The approximate likelihood function (i.e. the natural base exponential of the log-likelihood function), LikelihoodBasedProfileWiseAnalysis.ellipse_loglike.