Differentiation of matrix functionals using triangular factorization

Abstract:

In various applications, it is necessary to differentiate a matrix functional $w(\textbf {A}(\textbf {x}))$, where $\textbf {A}(\textbf {x})$ is a matrix depending on a parameter vector $\textbf {x}$. Usually, the functional itself can be readily computed from a triangular factorization of $\textbf {A}(\textbf {x})$. This paper develops several methods that also use the triangular factorization to efficiently evaluate the first and second derivatives of the functional. Both the full and sparse matrix situations are considered. There are similarities between these methods and algorithmic differentiation. However, the methodology developed here is explicit, leading to new algorithms. It is shown how the methods apply to several applications where the functional is a log determinant, including spline smoothing, covariance selection and restricted maximum likelihood.