Uniqueness and stability for the solution of a nonlinear least squares problem

Abstract:

In this paper, we focus on the nonlinear least squares problem: $\min _{{\boldsymbol {x}}\in \mathbb {H}^d}\|\lvert A{\boldsymbol {x}}\rvert -{\boldsymbol {b}}\|$ where $A\in \mathbb {H}^{m\times d}$, ${\boldsymbol {b}}\in \mathbb {R}^m$ with $\mathbb {H}\in \left \{\mathbb {R},\mathbb {C}\right \}$ and consider the uniqueness and stability of solutions. This problem arises in applications such as phase retrieval and absolute value rectification neural networks. While several results have been developed to characterize the uniqueness and stability of solutions when ${\boldsymbol {b}}=\lvert A{\boldsymbol {x}}_0\rvert$ for some ${\boldsymbol {x}}_0\in \mathbb {H}^d$, no existing results address the case where ${\boldsymbol {b}}$ is arbitrary. In this paper, we investigate the uniqueness and stability of solutions for the more general case where ${\boldsymbol {b}}$ is not necessarily equal to $\lvert A{\boldsymbol {x}}_0\rvert$ for any ${\boldsymbol {x}}_0\in \mathbb {H}^d$. We prove that for any matrix $A\in \mathbb {H}^{m\times d}$, there is always a vector ${\boldsymbol {b}}\in \mathbb {R}^m$ for which the solution to the nonlinear least squares problem is not unique. However, we show that such “bad” vectors ${\boldsymbol {b}}$ are negligible in practice; specifically, if ${\boldsymbol {b}}\in \mathbb {R}_{ }^m$ does not lie in some measure zero set, then the solution is unique. Furthermore, we establish certain conditions under which the solution is guaranteed to be unique. Regarding the stability of solutions, we prove that the solution is not uniformly stable. However, if we restrict the vectors ${\boldsymbol {b}}$ to a convex set where the solution to the least squares problem is unique, then the solution becomes stable. To the best of our knowledge, our results represent the first theoretical results of the uniqueness and stability of solutions for the nonlinear least squares problem.