| 释义 |
- References
- Further reading
Truncated Newton methods, also known as Hessian-free optimization,[1] are a family of optimization algorithms designed for optimizing non-linear functions with large numbers of independent variables. A truncated Newton method consists of repeated application of an iterative optimization algorithm to approximately solve Newton's equations, to determine an update to the function's parameters. The inner solver is truncated, i.e., run for only a limited number of iterations. It follows that, for truncated Newton methods to work, the inner solver needs to produce a good approximation in a finite number of iterations;[2] conjugate gradient has been suggested and evaluated as a candidate inner loop.{{r|martens}} Another prerequisite is good preconditioning for the inner algorithm.[3]References1. ^{{cite conference |last=Martens |first=James |title=Deep learning via Hessian-free optimization |url=http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pdf |conference=Proc. International Conference on Machine Learning |year=2010}}
2. ^{{cite journal |first=Stephen G. |last=Nash |doi=10.1016/S0377-0427(00)00426-X |title=A survey of truncated-Newton methods |journal=Journal of Computational and Applied Mathematics|volume=124 |issue=1–2 |year=2000 |pages=45–59}}
3. ^{{cite journal |first1=Stephen G. |last1=Nash |title=Preconditioning of truncated-Newton methods |journal=SIAM J. Sci. Stat. Comput. |volume=6 |issue=3 |year=1985 |pages=599–616}}
Further reading- {{cite journal |first1=L. |last1=Grippo |first2=F. |last2=Lampariello |first3=S. |last3=Lucidi |title=A Truncated Newton Method with Nonmonotone Line Search for Unconstrained Optimization |journal=J. Optimization Theory and Applications |volume=60 |issue=3 |year=1989 |citeseerx=10.1.1.455.7495}}
- {{cite journal |first1=Stephen G. |last1=Nash |first2=Jorge |last2=Nocedal |title=A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization |journal=SIAM J. Optim. |volume=1 |issue=3 |pages=358–372 |year=1991 |citeseerx=10.1.1.474.3400}}
{{optimization algorithms}}{{Isaac Newton}}{{applied-math-stub}} 1 : Optimization algorithms and methods |