Numerical Optimization (Springer Series in Operations Research and Financial Engineering)

Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems.

For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

There is a selected solutions manual for instructors for the new edition.

Conclusion: A little difficult, but well worth the time and money involved


outstanding   May 15, 2007
kelly londry (Ann Arbor, Michigan)
1 out of 2 found this review helpful

This book is a well-written, outstanding reference for anyone interested in understanding, using, and/or implementing state-of-the-art techniques in nonlinear optimization. Ample attention is paid to both constrained and unconstrained problem types, with a healthy and refreshing emphasis on trust-region strategies, and modern SQP and Interior-Point algorithms. Sufficient detail is paid to most topics while overall perspectives are well-maintained. This book is the very best of its kind for its intended audience. I strongly recommend it.


A book for understanding numerical optimization algorithms   October 5, 2006
a book reader
3 out of 5 found this review helpful

This books focuses on practical methods for continuous unconstrained
and constrained optimization. It does not cover problem
formulation. In all methods, the presentation tries to motivate the
approach using basic principles, rather than throw a mechanical
algorithm to the user. Thus the algorithms all make intuitive
sense. This is best demonstrated in the presentation of the KKT
conditions for constrained optimization. Below are a list of topics
covered.

Unconstrained optimization looks for a point with gradient 0. In
terms of search directions, most importantly are two: steepest
descent, Newton direction. Newton direction is based on a quadratic
approximation, and the direction is obtained by solving for the
gradient to be 0 using Newton method. We also know quasi-Newton and
Conjugate gradient. The control is in line search and trust region
method to make sure that for each step there is sufficient
descent. Line search modifies Hessian to make it positive definite.

Constrained optimization is based on KKT condition on Lagrangian
function. KKT just says that at the solution, the gradient of the
objective function is a linear combination of the gradients of the
active constraints. All interior point method form the KKT equation
and solve it using Newton equation method. Inequality constrains become
equality by adding slack variables and simple bounds on the slack
variables. The solver will make the solution to balance the total
reduction (because of the complementarity constraints) of all
variables, and the closeness to the boundary (one variable become 0).

The active set method tries to guess a set of active constraints,
minimize it by ignoring the reset of the constraints, try to update to
the minimizer. If this makes an inactive constraint become active, add
it into the active set. Once we are at the minimizer of the current
active set, we calculate the Lagrange Multipliers, if an inequality
active constraint's multiplier is negative, it is dropped from the
active set and the next iteration begin. Under some assumptions, the
next iteration will be able to reduce the objective function. Because
the subproblem only has equality constraint, can be solved using KKT
equation directly or null space method. For linear programming
problem, the addition of a constraint and dropping a constraint from
the active set happens at the same time. Each active set corresponds
to a basic feasible point.

There are also penalty, barrier, modified multiplier method to convert
the problem to solving a series of unconstrained problem. The
sequential quadratic programming method is to approximate the
objective function by quadratic model and use linear approximation to
the constraints. Solve the resulting QP subproblem using either active
set/interior point/direct KKT/gradient projection. The search
direction is safeguarded in line search by following the Wolfe
condition.



Numerical Optimization Review   February 14, 2008
amabonsu (Pennsylvania USA)
0 out of 3 found this review helpful

This book arrived quickly and in excellent condition. I couldn't ask for anything else.