| <?xml version="1.0"?> |
| |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| <document url="optimization.html"> |
| |
| <properties> |
| <title>The Commons Math User Guide - Optimization</title> |
| </properties> |
| |
| <body> |
| <section name="12 Optimization"> |
| <p><em>The contents of this section currently describes deprecated classes.</em> |
| Please refer to the new <a href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optim/package-summary.html">API |
| description</a>. |
| </p> |
| <p>Least squares optimizers are not in this package anymore, they have been moved |
| in a dedicated least-squares sub-package described in the <a href="./leastsquares.html">least squares</a> |
| section. |
| </p> |
| |
| <subsection name="12.1 Overview" href="overview"> |
| <p> |
| The optimization package provides algorithms to optimize (i.e. either minimize |
| or maximize) some objective or cost function. The package is split in several |
| sub-packages dedicated to different kind of functions or algorithms. |
| <ul> |
| <li>the univariate package handles univariate scalar functions,</li> |
| <li>the linear package handles multivariate vector linear functions |
| with linear constraints,</li> |
| <li>the direct package handles multivariate scalar functions |
| using direct search methods (i.e. not using derivatives),</li> |
| <li>the general package handles multivariate scalar or vector functions |
| using derivatives.</li> |
| <li>the fitting package handles curve fitting by univariate real functions</li> |
| </ul> |
| </p> |
| <p> |
| The top level optimization package provides common interfaces for the optimization |
| algorithms provided in sub-packages. The main interfaces defines defines optimizers |
| and convergence checkers. The functions that are optimized by the algorithms provided |
| by this package and its sub-packages are a subset of the one defined in the |
| <code>analysis</code> package, namely the real and vector valued functions. These |
| functions are called objective function here. When the goal is to minimize, the |
| functions are often called cost function, this name is not used in this package. |
| </p> |
| <p> |
| The type of goal, i.e. minimization or maximization, is defined by the enumerated |
| <a href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/GoalType.html"> |
| GoalType</a> which has only two values: <code>MAXIMIZE</code> and <code>MINIMIZE</code>. |
| </p> |
| <p> |
| Optimizers are the algorithms that will either minimize or maximize, the objective |
| function by changing its input variables set until an optimal set is found. There |
| are only four interfaces defining the common behavior of optimizers, one for each |
| supported type of objective function: |
| <ul> |
| <li><a href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/univariate/UnivariateOptimizer.html"> |
| UnivariateOptimizer</a> for <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/analysis/UnivariateFunction.html"> |
| univariate real functions</a></li> |
| <li><a href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/MultivariateOptimizer.html"> |
| MultivariateOptimizer</a> for <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/analysis/MultivariateFunction.html"> |
| multivariate real functions</a></li> |
| <li><a href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/DifferentiableMultivariateOptimizer.html"> |
| DifferentiableMultivariateOptimizer</a> for <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/analysis/DifferentiableMultivariateFunction.html"> |
| differentiable multivariate real functions</a></li> |
| <li><a href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/DifferentiableMultivariateVectorOptimizer.html"> |
| DifferentiableMultivariateVectorOptimizer</a> for <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/analysis/DifferentiableMultivariateVectorFunction.html"> |
| differentiable multivariate vectorial functions</a></li> |
| </ul> |
| </p> |
| |
| <p> |
| Despite there are only four types of supported optimizers, it is possible to optimize |
| a transform a <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/analysis/MultivariateVectorFunction.html"> |
| non-differentiable multivariate vectorial function</a> by converting it to a <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/analysis/MultivariateFunction.html"> |
| non-differentiable multivariate real function</a> thanks to the <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/LeastSquaresConverter.html"> |
| LeastSquaresConverter</a> helper class. The transformed function can be optimized using |
| any implementation of the <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/MultivariateOptimizer.html"> |
| MultivariateOptimizer</a> interface. |
| </p> |
| |
| <p> |
| For each of the four types of supported optimizers, there is a special implementation |
| which wraps a classical optimizer in order to add it a multi-start feature. This feature |
| call the underlying optimizer several times in sequence with different starting points |
| and returns the best optimum found or all optima if desired. This is a classical way to |
| prevent being trapped into a local extremum when looking for a global one. |
| </p> |
| </subsection> |
| <subsection name="12.2 Univariate Functions" href="univariate"> |
| <p> |
| A <a href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/univariate/UnivariateOptimizer.html"> |
| UnivariateOptimizer</a> is used to find the minimal values of a univariate real-valued |
| function <code>f</code>. |
| </p> |
| <p> |
| These algorithms usage is very similar to root-finding algorithms usage explained |
| in the analysis package. The main difference is that the <code>solve</code> methods in root |
| finding algorithms is replaced by <code>optimize</code> methods. |
| </p> |
| </subsection> |
| <subsection name="12.3 Linear Programming" href="linear"> |
| <p> |
| This package provides an implementation of George Dantzig's simplex algorithm |
| for solving linear optimization problems with linear equality and inequality |
| constraints. |
| </p> |
| </subsection> |
| <subsection name="12.4 Direct Methods" href="direct"> |
| <p> |
| Direct search methods only use cost function values, they don't |
| need derivatives and don't either try to compute approximation of |
| the derivatives. According to a 1996 paper by Margaret H. Wright |
| (<a href="http://cm.bell-labs.com/cm/cs/doc/96/4-02.ps.gz">Direct |
| Search Methods: Once Scorned, Now Respectable</a>), they are used |
| when either the computation of the derivative is impossible (noisy |
| functions, unpredictable discontinuities) or difficult (complexity, |
| computation cost). In the first cases, rather than an optimum, a |
| <em>not too bad</em> point is desired. In the latter cases, an |
| optimum is desired but cannot be reasonably found. In all cases |
| direct search methods can be useful. |
| </p> |
| <p> |
| Simplex-based direct search methods are based on comparison of |
| the cost function values at the vertices of a simplex (which is a |
| set of n+1 points in dimension n) that is updated by the algorithms |
| steps. |
| </p> |
| <p> |
| The instances can be built either in single-start or in |
| multi-start mode. Multi-start is a traditional way to try to avoid |
| being trapped in a local minimum and miss the global minimum of a |
| function. It can also be used to verify the convergence of an |
| algorithm. In multi-start mode, the <code>minimizes</code>method |
| returns the best minimum found after all starts, and the <code>etMinima</code> |
| method can be used to retrieve all minima from all starts (including the one |
| already provided by the <code>minimizes</code> method). |
| </p> |
| <p> |
| The <code>direct</code> package provides four solvers: |
| <ul> |
| <li>the classical <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/direct/NelderMeadSimplex.html"> |
| Nelder-Mead</a> method,</li> |
| <li>Virginia Torczon's <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/direct/MultiDirectionalSimplex.html"> |
| multi-directional</a> method,</li> |
| <li>Nikolaus Hansen's <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/direct/CMAESOptimizer.html"> |
| </a>Covariance Matrix Adaptation Evolution Strategy (CMA-ES),</li> |
| <li>Mike Powell's <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/direct/BOBYQAOptimizer.html"> |
| BOBYQA</a> method. |
| </li> |
| </ul> |
| </p> |
| <p> |
| The first two simplex-based methods do not handle simple bounds constraints by themselves. |
| However there are two adapters(<a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/direct/MultivariateFunctionMappingAdapter.html"> |
| MultivariateFunctionMappingAdapter</a> and <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/direct/MultivariateFunctionPenaltyAdapter.html"> |
| MultivariateFunctionPenaltyAdapter</a>) that can be used to wrap the user function in |
| such a way the wrapped function is unbounded and can be used with these optimizers, despite |
| the fact the underlying function is still bounded and will be called only with feasible |
| points that fulfill the constraints. Note however that using these adapters are only a |
| poor man solutions to simple bounds optimization constraints. Better solutions are to use an |
| optimizer that directly supports simple bounds. Some caveats of the mapping adapter |
| solution are that |
| <ul> |
| <li>behavior near the bounds may be numerically unstable as bounds are mapped from |
| infinite values,</li> |
| <li>start value is evaluated by the optimizer as an unbounded variable, |
| so it must be converted from bounded to unbounded by user,</li> |
| <li>optimum result is evaluated by the optimizer as an unbounded variable, |
| so it must be converted from unbounded to bounded by user,</li> |
| <li>convergence values are evaluated by the optimizer as unbounded variables, |
| so there will be scales differences when converted to bounded variables,</li> |
| <li>in the case of simplex based solvers, the initial simplex should be set up |
| as delta in unbounded variables.</li> |
| </ul> |
| One caveat of penalty adapter is that if start point or start simplex is outside of the allowed |
| range, only the penalty function is used, and the optimizer may converge without ever entering |
| the allowed range. |
| </p> |
| <p> |
| The last methods do handle simple bounds constraints directly, so the adapters are not needed |
| with them. |
| </p> |
| </subsection> |
| <subsection name="12.5 General Case" href="general"> |
| <p> |
| The general package deals with non-linear vectorial optimization problems when |
| the partial derivatives of the objective function are available. |
| </p> |
| <p> |
| One important class of estimation problems is weighted least |
| squares problems. They basically consist in finding the values |
| for some parameters p<sub>k</sub> such that a cost function |
| J = sum(w<sub>i</sub>(mes<sub>i</sub> - mod<sub>i</sub>)<sup>2</sup>) is |
| minimized. The various (target<sub>i</sub> - model<sub>i</sub>(p<sub>k</sub>)) |
| terms are called residuals. They represent the deviation between a set of |
| target values target<sub>i</sub> and theoretical values computed from |
| models model<sub>i</sub> depending on free parameters p<sub>k</sub>. |
| The w<sub>i</sub> factors are weights. One classical use case is when the |
| target values are experimental observations or measurements. |
| </p> |
| <p> |
| Solving a least-squares problem is finding the free parameters p<sub>k</sub> |
| of the theoretical models such that they are close to the target values, i.e. |
| when the residual are small. |
| </p> |
| <p> |
| Two optimizers are available in the general package, both devoted to least-squares |
| problems. The first one is based on the <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/general/GaussNewtonOptimizer.html"> |
| Gauss-Newton</a> method. The second one is the <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/general/LevenbergMarquardtOptimizer.html"> |
| Levenberg-Marquardt</a> method. |
| </p> |
| <p> |
| In order to solve a vectorial optimization problem, the user must provide it as |
| an object implementing the <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/analysis/DifferentiableMultivariateVectorFunction.html"> |
| DifferentiableMultivariateVectorFunction</a> interface. The object will be provided to |
| the <code>estimate</code> method of the optimizer, along with the target and weight arrays, |
| thus allowing the optimizer to compute the residuals at will. The last parameter to the |
| <code>estimate</code> method is the point from which the optimizer will start its |
| search for the optimal point. |
| </p> |
| |
| <dl> |
| <dt>Quadratic Problem Example</dt> |
| <dd> |
| |
| |
| We are looking to find the best parameters [a, b, c] for the quadratic function |
| <b><code>f(x) = a x<sup>2</sup> + b x + c</code></b>. |
| The data set below was generated using [a = 8, b = 10, c = 16]. |
| A random number between zero and one was added to each y value calculated. |
| |
| <table cellspacing="0" cellpadding="3"> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;"><b>X</b></td> |
| <td valign="bottom" align="center" style=" font-size:10pt;"><b>Y</b></td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">34.234064369</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">2</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">68.2681162306108</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">3</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">118.615899084602</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">4</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">184.138197238557</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">5</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">266.599877916276</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">6</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">364.147735251579</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">7</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">478.019226091914</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">8</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">608.140949270688</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">9</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">754.598868667148</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">10</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">916.128818085883</td> |
| </tr> |
| </table> |
| |
| <p> |
| First we need to implement the interface <a href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/analysis/DifferentiableMultivariateVectorFunction.html">DifferentiableMultivariateVectorFunction</a>. |
| This requires the implementation of the method signatures: |
| </p> |
| |
| |
| <ul> |
| <li><b>MultivariateMatrixFunction jacobian()</b></li> |
| <li><b>double[] value(double[] point)</b></li> |
| </ul> |
| |
| <p> |
| We'll tackle the implementation of the <code>MultivariateMatrixFunction jacobian()</code> method first. You may wish to familiarize yourself with what a <a href="http://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant"> Jacobian Matrix</a> is. |
| In this case the Jacobian is the partial derivative of the function with respect |
| to the parameters a, b and c. These derivatives are computed as follows: |
| <ul> |
| <li>d(ax<sup>2</sup> + bx + c)/da = x<sup>2</sup></li> |
| <li>d(ax<sup>2</sup> + bx + c)/db = x</li> |
| <li>d(ax<sup>2</sup> + bx + c)/dc = 1</li> |
| </ul> |
| </p> |
| |
| <p> |
| For a quadratic which has three variables the Jacobian Matrix will have three columns, one for each variable, and the number |
| of rows will equal the number of rows in our data set, which in this case is ten. So for example for <tt>[a = 1, b = 1, c = 1]</tt>, the Jacobian Matrix is (excluding the first column which shows the value of x): |
| </p> |
| |
| <table cellspacing="0" cellpadding="3"> |
| <tr> |
| <td valign="bottom" align="left" style=" font-size:10pt;"><b>x</b></td> |
| <td valign="bottom" align="left" style=" font-size:10pt;"><b>d(ax<sup>2</sup> + bx + c)/da</b></td> |
| <td valign="bottom" align="left" style=" font-size:10pt;"><b>d(ax<sup>2</sup> + bx + c)/db</b></td> |
| <td valign="bottom" align="left" style=" font-size:10pt;"><b>d(ax<sup>2</sup> + bx + c)/dc</b></td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">2</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">4</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">2</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">3</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">9</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">3</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">4</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">16</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">4</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">5</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">25</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">5</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">6</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">36</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">6</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">7</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">49</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">7</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">8</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">64</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">8</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">9</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">81</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">9</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| <tr> |
| <td valign="bottom" align="center" style=" font-size:10pt;">10</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">100</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">10</td> |
| <td valign="bottom" align="center" style=" font-size:10pt;">1</td> |
| </tr> |
| </table> |
| |
| <p> |
| The implementation of the <code>MultivariateMatrixFunction jacobian()</code> for this problem looks like this (The <code>x</code> |
| parameter is an ArrayList containing the independent values of the data set): |
| </p> |
| |
| <source> |
| private double[][] jacobian(double[] variables) { |
| double[][] jacobian = new double[x.size()][3]; |
| for (int i = 0; i < jacobian.length; ++i) { |
| jacobian[i][0] = x.get(i) * x.get(i); |
| jacobian[i][1] = x.get(i); |
| jacobian[i][2] = 1.0; |
| } |
| return jacobian; |
| } |
| |
| public MultivariateMatrixFunction jacobian() { |
| return new MultivariateMatrixFunction() { |
| private static final long serialVersionUID = -8673650298627399464L; |
| public double[][] value(double[] point) { |
| return jacobian(point); |
| } |
| }; |
| } |
| </source> |
| |
| <p> |
| Note that if for some reason the derivative of the objective function with respect |
| to its variables is difficult to obtain, |
| <a href="http://en.wikipedia.org/wiki/Numerical_differentiation">Numerical differentiation</a> can be used. |
| </p> |
| |
| |
| <p> |
| The implementation of the <code>double[] value(double[] point)</code> method, which returns |
| a <code>double</code> array containing the |
| values the objective function returns per given independent value |
| and the current set of variables or parameters, |
| can be seen below: |
| </p> |
| |
| <source> |
| public double[] value(double[] variables) { |
| double[] values = new double[x.size()]; |
| for (int i = 0; i < values.length; ++i) { |
| values[i] = (variables[0] * x.get(i) + variables[1]) * x.get(i) + variables[2]; |
| } |
| return values; |
| } |
| </source> |
| |
| <p> |
| Below is the the class containing all the implementation details |
| (Taken from the Apache Commons Math <b>org.apache.commons.math4.optimization.general.LevenbergMarquardtOptimizerTest</b>): |
| </p> |
| |
| <source> |
| private static class QuadraticProblem |
| implements DifferentiableMultivariateVectorFunction, Serializable { |
| |
| private static final long serialVersionUID = 7072187082052755854L; |
| private List<Double> x; |
| private List<Double> y; |
| |
| public QuadraticProblem() { |
| x = new ArrayList<Double>(); |
| y = new ArrayList<Double>(); |
| } |
| |
| public void addPoint(double x, double y) { |
| this.x.add(x); |
| this.y.add(y); |
| } |
| |
| public double[] calculateTarget() { |
| double[] target = new double[y.size()]; |
| for (int i = 0; i < y.size(); i++) { |
| target[i] = y.get(i).doubleValue(); |
| } |
| return target; |
| } |
| |
| private double[][] jacobian(double[] variables) { |
| double[][] jacobian = new double[x.size()][3]; |
| for (int i = 0; i < jacobian.length; ++i) { |
| jacobian[i][0] = x.get(i) * x.get(i); |
| jacobian[i][1] = x.get(i); |
| jacobian[i][2] = 1.0; |
| } |
| return jacobian; |
| } |
| |
| public double[] value(double[] variables) { |
| double[] values = new double[x.size()]; |
| for (int i = 0; i < values.length; ++i) { |
| values[i] = (variables[0] * x.get(i) + variables[1]) * x.get(i) + variables[2]; |
| } |
| return values; |
| } |
| |
| public MultivariateMatrixFunction jacobian() { |
| return new MultivariateMatrixFunction() { |
| private static final long serialVersionUID = -8673650298627399464L; |
| public double[][] value(double[] point) { |
| return jacobian(point); |
| } |
| }; |
| } |
| } |
| </source> |
| |
| <p> |
| The below code shows how to go about using the above class |
| and a LevenbergMarquardtOptimizer instance to produce an |
| optimal set of quadratic curve fitting parameters: |
| </p> |
| |
| <source> |
| QuadraticProblem problem = new QuadraticProblem(); |
| |
| problem.addPoint(1, 34.234064369); |
| problem.addPoint(2, 68.2681162306); |
| problem.addPoint(3, 118.6158990846); |
| problem.addPoint(4, 184.1381972386); |
| problem.addPoint(5, 266.5998779163); |
| problem.addPoint(6, 364.1477352516); |
| problem.addPoint(7, 478.0192260919); |
| problem.addPoint(8, 608.1409492707); |
| problem.addPoint(9, 754.5988686671); |
| problem.addPoint(10, 916.1288180859); |
| |
| LevenbergMarquardtOptimizer optimizer = new LevenbergMarquardtOptimizer(); |
| |
| final double[] weights = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 }; |
| |
| final double[] initialSolution = {1, 1, 1}; |
| |
| PointVectorValuePair optimum = optimizer.optimize(100, |
| problem, |
| problem.calculateTarget(), |
| weights, |
| initialSolution); |
| |
| final double[] optimalValues = optimum.getPoint(); |
| |
| System.out.println("A: " + optimalValues[0]); |
| System.out.println("B: " + optimalValues[1]); |
| System.out.println("C: " + optimalValues[2]); |
| |
| </source> |
| |
| <p> |
| If you run the above sample you will see |
| the following printed by the console: |
| </p> |
| |
| <pre> |
| A: 7.998832172372726 |
| B: 10.001841530162448 |
| C: 16.324008168386605 |
| </pre> |
| |
| </dd></dl> |
| <p> |
| In addition to least squares solving, the <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/general/NonLinearConjugateGradientOptimizer.html"> |
| NonLinearConjugateGradientOptimizer</a> class provides a non-linear conjugate gradient algorithm |
| to optimize <a |
| href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/analysis/DifferentiableMultivariateFunction.html"> |
| DifferentiableMultivariateFunction</a>. Both the Fletcher-Reeves and the Polak-Ribière |
| search direction update methods are supported. It is also possible to set up a preconditioner |
| or to change the line-search algorithm of the inner loop if desired (the default one is a Brent |
| solver). |
| </p> |
| <p> |
| The <a href="../commons-math-docs/apidocs/org/apache/commons/math4/legacy/optimization/direct/PowellOptimizer.html"> |
| PowellOptimizer</a> provides an optimization method for non-differentiable functions. |
| </p> |
| </subsection> |
| </section> |
| </body> |
| </document> |