On the convergence of a wide range of trust region methods for unconstrained optimization

We consider trust region methods for seeking the unconstrained minimum of an objective function F( $F(\underline{x}),\underline{x}\in \mathcal{R}^{n}$, when the gradient $\underline{\nabla}F(\underline{x}),\underline{x}\in \mathcal{R}^{n}$, is available. The methods are iterative with $\underline{x}_{1}$ being given. The new vector of variables $\underline{x}_{k+1}$ is derived from a quadratic approximation to F that interpolates F( $\underline{x}$k) and $\underline{\nabla}F(\underline{x}_{k})$, where k is the iteration number. The second derivative matrix of the quadratic approximation, Bk say, can be indefinite, because the approximation is employed only if the vector of variables $\underline{x}$ satisfies $\left \| \underline{x}-\underline{x}_{k} \right \|\leqslant \Delta_{k}$, where Δk is a “trust region radius” that is adjusted automatically. Thus the approximation is useful if $\left\|\underline{\nabla}F(\underline{x}_{k}) \right \|$ is sufficiently large and if ‖Bk‖ and Δk are sufficiently small. It is proved under mild assumptions that the condition $\left\|\underline{\nabla}F(\underline{x}_{k+1}) \right \|\leqslant \varepsilon$ is achieved after a finite number of iterations, where ε is any given positive constant, and then it is usual to end the calculation. The assumptions include a Lipschitz condition on $\underline{\nabla}F$ and also F has to be bounded below. The termination property is established in a single theorem that applies to a wide range of trust region methods that force the sequence F( $\underline{x}$k), k = 1, 2, 3, …, to decrease monotonically. Any choice of each symmetric matrix Bk is allowed, provided that ‖Bk‖ is bounded above by a constant multiple of k.