updated documentation after the redesign of the optimization package
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@754764 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Luc Maisonobe
parent
c12aa3b936
commit
77a6b785d2
|
|
@ -55,7 +55,6 @@
|
|||
<item name="Fractions" href="/userguide/fraction.html"/>
|
||||
<item name="Transform Methods" href="/userguide/transform.html"/>
|
||||
<item name="3D Geometry" href="/userguide/geometry.html"/>
|
||||
<item name="Parametric Estimation" href="/userguide/estimation.html"/>
|
||||
<item name="Optimization" href="/userguide/optimization.html"/>
|
||||
<item name="Ordinary Differential Equations" href="/userguide/ode.html"/>
|
||||
</menu>
|
||||
|
|
|
|||
|
|
@ -39,6 +39,9 @@ The <action> type attribute can be add,update,fix,remove.
|
|||
</properties>
|
||||
<body>
|
||||
<release version="2.0" date="TBD" description="TBD">
|
||||
<action dev="luc" type="fix" issue="MATH-177" >
|
||||
Redesigned the optimization framework for a simpler yet more powerful API.
|
||||
</action>
|
||||
<action dev="luc" type="fix" issue="MATH-243" due-to="Christian Semrau">
|
||||
Fixed an error in computing gcd and lcm for some extreme values at integer
|
||||
range boundaries.
|
||||
|
|
|
|||
|
|
@ -1,364 +0,0 @@
|
|||
<?xml version="1.0"?>
|
||||
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<?xml-stylesheet type="text/xsl" href="./xdoc.xsl"?>
|
||||
<!-- $Revision: 480435 $ $Date: 2006-11-29 08:06:35 +0100 (mer., 29 nov. 2006) $ -->
|
||||
<document url="estimation.html">
|
||||
|
||||
<properties>
|
||||
<title>The Commons Math User Guide - Parametric Estimation</title>
|
||||
</properties>
|
||||
|
||||
<body>
|
||||
<section name="12 Parametric Estimation">
|
||||
<subsection name="12.1 Overview" href="overview">
|
||||
<p>
|
||||
The estimation package provides classes to fit some non-linear
|
||||
model to available observations depending on it. These
|
||||
problems are commonly called estimation problems.
|
||||
</p>
|
||||
<p>
|
||||
The estimation problems considered here are parametric
|
||||
problems where a user-provided model depends on initially
|
||||
unknown scalar parameters and several measurements made on
|
||||
values that depend on the model are available. As examples,
|
||||
one can consider the center and radius of a circle given
|
||||
points approximately lying on a ring, or a satellite orbit
|
||||
given range, range-rate and angular measurements from various
|
||||
ground stations.
|
||||
</p>
|
||||
<p>
|
||||
One important class of estimation problems is weighted least
|
||||
squares problems. They basically consist in finding the values
|
||||
for some parameters p<sub>k</sub> such that a cost function
|
||||
J = sum(w<sub>i</sub>r<sub>i</sub><sup>2</sup>) is minimized.
|
||||
The various r<sub>i</sub> terms represent the deviation
|
||||
r<sub>i</sub> = mes<sub>i</sub> - mod<sub>i</sub>
|
||||
between the measurements and the parameterized models. The
|
||||
w<sub>i</sub> factors are the measurements weights, they are often
|
||||
chosen either all equal to 1.0 or proportional to the inverse of the
|
||||
variance of the measurement type. The solver adjusts the values of
|
||||
the estimated parameters p<sub>k</sub> which are not bound (i.e. the
|
||||
free parameters). It does not touch the parameters which have been
|
||||
put in a bound state by the user.
|
||||
</p>
|
||||
<p>
|
||||
The aim of this package is similar to the aim of the
|
||||
optimization package, but the algorithms are entirely
|
||||
different as:
|
||||
<ul>
|
||||
<li>
|
||||
they need the partial derivatives of the measurements with
|
||||
respect to the free parameters
|
||||
</li>
|
||||
<li>
|
||||
they are residuals based instead of generic cost functions
|
||||
based
|
||||
</li>
|
||||
</ul>
|
||||
</p>
|
||||
</subsection>
|
||||
|
||||
<subsection name="12.2 Problem modeling" href="problem">
|
||||
<p>
|
||||
The problem modeling is the most important part for the
|
||||
user. Understanding it is the key to proper use of the
|
||||
package. One interface and two classes are provided for this
|
||||
purpose: <a href="../apidocs/org/apache/commons/math/estimation/EstimationProblem.html">
|
||||
EstimationProblem</a>, <a href="../apidocs/org/apache/commons/math/estimation/EstimatedParameter.html">
|
||||
EstimatedParameter</a> and <a href="../apidocs/org/apache/commons/math/estimation/WeightedMeasurement.html">
|
||||
WeightedMeasurement</a>.
|
||||
</p>
|
||||
<p>
|
||||
Consider the following example problem: we want to determine the
|
||||
linear trajectory of a sailing ship by performing angular and
|
||||
distance measurements from an observing spot on the shore. The
|
||||
problem model is represented by two equations:
|
||||
</p>
|
||||
<p>
|
||||
x(t) = x<sub>0</sub>+(t-t<sub>0</sub>)vx<sub>0</sub><br/>
|
||||
y(t) = y<sub>0</sub>+(t-t<sub>0</sub>)vy<sub>0</sub>
|
||||
</p>
|
||||
<p>
|
||||
These two equations depend on four parameters (x<sub>0</sub>, y<sub>0</sub>,
|
||||
vx<sub>0</sub> and vy<sub>0</sub>). We want to determine these four parameters.
|
||||
</p>
|
||||
<p>
|
||||
Assuming the observing spot is located at the origin of the coordinates
|
||||
system and that the angular measurements correspond to the angle between
|
||||
the x axis and the line of sight, the theoretical values of the angular
|
||||
measurements at t<sub>i</sub> and of the distance measurements at
|
||||
t<sub>j</sub> are modeled as follows:
|
||||
</p>
|
||||
<p>
|
||||
angle<sub>i,theo</sub> = atan2(y(t<sub>i</sub>), x(t<sub>i</sub>))<br/>
|
||||
distance<sub>j,theo</sub> = sqrt(x(t<sub>j</sub>)<sup>2</sup>+y(t<sub>j</sub>)<sup>2</sup>)
|
||||
</p>
|
||||
<p>
|
||||
The real observations generate a set of measurements values angle<sub>i,meas</sub>
|
||||
and distance<sub>j,meas</sub>.
|
||||
</p>
|
||||
<p>
|
||||
The following class diagram shows one way to solve this problem using the
|
||||
estimation package. The grey elements are already provided by the package
|
||||
whereas the purple elements are developed by the user.
|
||||
</p>
|
||||
<img src="./estimation-class-diagram.png"/>
|
||||
<p>
|
||||
The <code>TrajectoryDeterminationProblem</code> class holds the linear model
|
||||
equations x(t) and y(t). It delegate storage of the four parameters x<sub>0</sub>,
|
||||
y<sub>0</sub>, vx<sub>0</sub> and vy<sub>0</sub> and of the various measurements
|
||||
angle<sub>i,meas</sub> and distance<sub>j,meas</sub> to its base class
|
||||
<code>SimpleEstimationProblem</code>. Since the theoretical values of the measurements
|
||||
angle<sub>i,theo</sub> and distance<sub>j,theo</sub> depend on the linear model,
|
||||
the two classes <code>AngularMeasurement</code> and <code>DistanceMeasurement</code>
|
||||
are implemented as internal classes, thus having access to the equations of the
|
||||
linear model and to the parameters.
|
||||
</p>
|
||||
<p>
|
||||
Here are the various parts of the <code>TrajectoryDeterminationProblem.java</code>
|
||||
source file. This example, with an additional <code>main</code> method is
|
||||
available <a href="./TrajectoryDeterminationProblem.java">here</a>.
|
||||
</p>
|
||||
<dd>First, the general setup of the class: declarations, fields, constructor, setters and getters:
|
||||
<source>
|
||||
public class TrajectoryDeterminationProblem extends SimpleEstimationProblem {
|
||||
public TrajectoryDeterminationProblem(double t0,
|
||||
double x0Guess, double y0Guess,
|
||||
double vx0Guess, double vy0Guess) {
|
||||
this.t0 = t0;
|
||||
x0 = new EstimatedParameter( "x0", x0Guess);
|
||||
y0 = new EstimatedParameter( "y0", y0Guess);
|
||||
vx0 = new EstimatedParameter("vx0", vx0Guess);
|
||||
vy0 = new EstimatedParameter("vy0", vy0Guess);
|
||||
|
||||
// inform the base class about the parameters
|
||||
addParameter(x0);
|
||||
addParameter(y0);
|
||||
addParameter(vx0);
|
||||
addParameter(vy0);
|
||||
|
||||
}
|
||||
|
||||
public double getX0() {
|
||||
return x0.getEstimate();
|
||||
}
|
||||
|
||||
public double getY0() {
|
||||
return y0.getEstimate();
|
||||
}
|
||||
|
||||
public double getVx0() {
|
||||
return vx0.getEstimate();
|
||||
}
|
||||
|
||||
public double getVy0() {
|
||||
return vy0.getEstimate();
|
||||
}
|
||||
|
||||
public void addAngularMeasurement(double wi, double ti, double ai) {
|
||||
// let the base class handle the measurement
|
||||
addMeasurement(new AngularMeasurement(wi, ti, ai));
|
||||
}
|
||||
|
||||
public void addDistanceMeasurement(double wi, double ti, double di) {
|
||||
// let the base class handle the measurement
|
||||
addMeasurement(new DistanceMeasurement(wi, ti, di));
|
||||
}
|
||||
|
||||
public double x(double t) {
|
||||
return x0.getEstimate() + (t - t0) * vx0.getEstimate();
|
||||
}
|
||||
|
||||
public double y(double t) {
|
||||
return y0.getEstimate() + (t - t0) * vy0.getEstimate();
|
||||
}
|
||||
|
||||
// measurements internal classes go here
|
||||
|
||||
private double t0;
|
||||
private EstimatedParameter x0;
|
||||
private EstimatedParameter y0;
|
||||
private EstimatedParameter vx0;
|
||||
private EstimatedParameter vy0;
|
||||
|
||||
}
|
||||
</source>
|
||||
</dd>
|
||||
<dd>The two specialized measurements class are simple internal classes that
|
||||
implement the equation for their respective measurement type, using the
|
||||
enclosing class to get the parameters references and the linear models x(t)
|
||||
and y(t). The <code>serialVersionUID</code> static fields are present because
|
||||
the <code>WeightedMeasurement</code> class implements the
|
||||
<code>Serializable</code> interface.
|
||||
<source>
|
||||
private class AngularMeasurement extends WeightedMeasurement {
|
||||
|
||||
public AngularMeasurement(double weight, double t, double angle) {
|
||||
super(weight, angle);
|
||||
this.t = t;
|
||||
}
|
||||
|
||||
public double getTheoreticalValue() {
|
||||
return Math.atan2(y(t), x(t));
|
||||
}
|
||||
|
||||
public double getPartial(EstimatedParameter parameter) {
|
||||
double xt = x(t);
|
||||
double yt = y(t);
|
||||
double r = Math.sqrt(xt * xt + yt * yt);
|
||||
double u = yt / (r + xt);
|
||||
double c = 2 * u / (1 + u * u);
|
||||
if (parameter == x0) {
|
||||
return -c;
|
||||
} else if (parameter == vx0) {
|
||||
return -c * t;
|
||||
} else if (parameter == y0) {
|
||||
return c * xt / yt;
|
||||
} else {
|
||||
return c * t * xt / yt;
|
||||
}
|
||||
}
|
||||
|
||||
private final double t;
|
||||
private static final long serialVersionUID = -5990040582592763282L;
|
||||
|
||||
}
|
||||
</source>
|
||||
<source>
|
||||
private class DistanceMeasurement extends WeightedMeasurement {
|
||||
|
||||
public DistanceMeasurement(double weight, double t, double angle) {
|
||||
super(weight, angle);
|
||||
this.t = t;
|
||||
}
|
||||
|
||||
public double getTheoreticalValue() {
|
||||
double xt = x(t);
|
||||
double yt = y(t);
|
||||
return Math.sqrt(xt * xt + yt * yt);
|
||||
}
|
||||
|
||||
public double getPartial(EstimatedParameter parameter) {
|
||||
double xt = x(t);
|
||||
double yt = y(t);
|
||||
double r = Math.sqrt(xt * xt + yt * yt);
|
||||
if (parameter == x0) {
|
||||
return xt / r;
|
||||
} else if (parameter == vx0) {
|
||||
return xt * t / r;
|
||||
} else if (parameter == y0) {
|
||||
return yt / r;
|
||||
} else {
|
||||
return yt * t / r;
|
||||
}
|
||||
}
|
||||
|
||||
private final double t;
|
||||
private static final long serialVersionUID = 3257286197740459503L;
|
||||
|
||||
}
|
||||
</source>
|
||||
</dd>
|
||||
</subsection>
|
||||
<subsection name="12.3 Problem solving" href="solving">
|
||||
<p>
|
||||
Solving the problem is simply a matter of choosing an implementation
|
||||
of the <a href="../apidocs/org/apache/commons/math/estimation/Estimator.html">
|
||||
Estimator</a> interface and to pass the problem instance to its <code>estimate</code>
|
||||
method. Two implementations are already provided by the library: <a
|
||||
href="../apidocs/org/apache/commons/math/estimation/GaussNewtonEstimator.html">
|
||||
GaussNewtonEstimator</a> and <a
|
||||
href="../apidocs/org/apache/commons/math/estimation/LevenbergMarquardtEstimator.html">
|
||||
LevenbergMarquardtEstimator</a>. The first one implements a simple Gauss-Newton
|
||||
algorithm, which is sufficient when the starting point (initial guess) is close
|
||||
enough to the solution. The second one implements a more complex Levenberg-Marquardt
|
||||
algorithm which is more robust when the initial guess is far from the solution.
|
||||
</p>
|
||||
<p>
|
||||
The following sequence diagram explains roughly what occurs under the hood
|
||||
in the <code>estimate</code> method.
|
||||
</p>
|
||||
<img src="./estimation-sequence-diagram.png"/>
|
||||
<p>
|
||||
Basically, the estimator first retrieves the parameters and the measurements.
|
||||
The estimation loop is based on the gradient of the sum of the squares of the
|
||||
residuals, hence, the estimators get the various partial derivatives of all
|
||||
measurements with respect to all parameters. A new state hopefully globally
|
||||
reducing the residuals is estimated, and the parameters value are updated.
|
||||
This estimation loops stops when either the convergence conditions are met
|
||||
or the maximal number of iterations is exceeded.
|
||||
</p>
|
||||
</subsection>
|
||||
<subsection name="12.4 Fine tuning" href="tuning">
|
||||
<p>
|
||||
One important tuning parameter for weighted least-squares solving is the
|
||||
weight attributed to each measurement. This weights has two purposes:
|
||||
</p>
|
||||
<ul>
|
||||
<li>fixing unit problems when combining different types of measurements</li>
|
||||
<li>adjusting the influence of good or bad measurements on the solution</li>
|
||||
</ul>
|
||||
<p>
|
||||
The weight is a multiplicative factor for the <em>square</em> of the residuals.
|
||||
A common choice is to use the inverse of the variance of the measurements error
|
||||
as the weighting factor for all measurements for one type. On our sailing ship
|
||||
example, we may have a range measurements accuracy of about 1 meter and an angular
|
||||
measurements accuracy of about 0.01 degree, or 1.7 10<sup>-4</sup> radians. So we
|
||||
would use w=1.0 for distance measurements weight and w=3 10<sup>7</sup> for
|
||||
angular measurements weight. If we knew that the measurements quality is bad
|
||||
at tracking start because of measurement system warm-up delay for example, then
|
||||
we would reduce the weight for the first measurements and use for example
|
||||
w=0.1 and w=3 10<sup>6</sup> respectively, depending on the type.
|
||||
</p>
|
||||
<p>
|
||||
After a problem has been set up, it is possible to fine tune the
|
||||
way it will be solved. For example, it may appear the measurements are not
|
||||
sufficient to get some parameters with sufficient confidence due to observability
|
||||
problems. It is possible to fix some parameters in order to prevent the solver
|
||||
from changing them. This is realized by passing <code>true</code> to the
|
||||
<code>setBound</code> method of the parameter.
|
||||
</p>
|
||||
<p>
|
||||
It is also possible to ignore some measurements by passing <code>true</code> to the
|
||||
<code>setIgnored</code> method of the measurement. A typical use is to
|
||||
<ol>
|
||||
<li>
|
||||
perform a first determination with all parameters, to check each measurement
|
||||
residual after convergence (i.e. to compute the difference between the
|
||||
measurement and its theoretical value as computed from the estimated parameters),
|
||||
</li>
|
||||
<li>
|
||||
compute standard deviation for the measurements samples (one sample for each
|
||||
measurements type)
|
||||
</li>
|
||||
<li>
|
||||
ignore measurements whose residual are above some threshold (for example three
|
||||
time the standard deviation on the residuals) assuming they correspond to
|
||||
bad measurements,
|
||||
</li>
|
||||
<li>
|
||||
perform another determination on the reduced measurements set.
|
||||
</li>
|
||||
</ol>
|
||||
</p>
|
||||
</subsection>
|
||||
</section>
|
||||
</body>
|
||||
</document>
|
||||
|
|
@ -112,28 +112,20 @@
|
|||
<li><a href="geometry.html#a11.2_Vectors">11.2 Vectors</a></li>
|
||||
<li><a href="geometry.html#a11.3_Rotations">11.3 Rotations</a></li>
|
||||
</ul></li>
|
||||
<li><a href="estimation.html">12. Parametric Estimation</a>
|
||||
<li><a href="optimization.html">12. Optimization</a>
|
||||
<ul>
|
||||
<li><a href="estimation.html#a12.1_Overview">12.1 Overview</a></li>
|
||||
<li><a href="estimation.html#a12.2_Models">12.2 Models</a></li>
|
||||
<li><a href="estimation.html#a12.3_Parameters">12.3 Parameters</a></li>
|
||||
<li><a href="estimation.html#a12.4_Measurements">12.4 Measurements</a></li>
|
||||
<li><a href="estimation.html#a12.5_Solvers">12.5 Solvers</a></li>
|
||||
<li><a href="optimization.html#a12.1_Overview">12.1 Overview</a></li>
|
||||
<li><a href="analysis.html#a12.2_Univariate_Functions">12.2 Univariate Functions</a></li>
|
||||
<li><a href="analysis.html#a12.3_Linear_Programming">12.3 Linear Programming</a></li>
|
||||
<li><a href="optimization.html#a12.4_Direct_Methods">12.4 Direct Methods</a></li>
|
||||
<li><a href="analysis.html#a12.5_General_Case">12.5 General Case</a></li>
|
||||
</ul></li>
|
||||
<li><a href="optimization.html">13. Optimization</a>
|
||||
<li><a href="ode.html">13. Ordinary Differential Equations Integration</a>
|
||||
<ul>
|
||||
<li><a href="optimization.html#a13.1_Overview">13.1 Overview</a></li>
|
||||
<li><a href="analysis.html#a13.2_Univariate_Functions">13.2 Univariate Functions</a></li>
|
||||
<li><a href="analysis.html#a13.3_Linear_Programming">13.3 Linear Programming</a></li>
|
||||
<li><a href="optimization.html#a13.4_Direct_Methods">13.4 Direct Methods</a></li>
|
||||
<li><a href="analysis.html#a13.5_General_Case">13.5 General Case</a></li>
|
||||
</ul></li>
|
||||
<li><a href="ode.html">14. Ordinary Differential Equations Integration</a>
|
||||
<ul>
|
||||
<li><a href="ode.html#a14.1_Overview">14.1 Overview</a></li>
|
||||
<li><a href="ode.html#a14.2_Discrete_Events_Handling">14.2 Discrete Events Handling</a></li>
|
||||
<li><a href="ode.html#a14.3_ODE_Problems">14.3 ODE Problems</a></li>
|
||||
<li><a href="ode.html#a14.4_Integrators">14.4 Integrators</a></li>
|
||||
<li><a href="ode.html#a13.1_Overview">13.1 Overview</a></li>
|
||||
<li><a href="ode.html#a13.2_Discrete_Events_Handling">13.2 Discrete Events Handling</a></li>
|
||||
<li><a href="ode.html#a13.3_ODE_Problems">13.3 ODE Problems</a></li>
|
||||
<li><a href="ode.html#a13.4_Integrators">13.4 Integrators</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
</section>
|
||||
|
|
|
|||
|
|
@ -26,8 +26,8 @@
|
|||
</properties>
|
||||
|
||||
<body>
|
||||
<section name="14 Ordinary Differential Equations Integration">
|
||||
<subsection name="14.1 Overview" href="overview">
|
||||
<section name="13 Ordinary Differential Equations Integration">
|
||||
<subsection name="13.1 Overview" href="overview">
|
||||
<p>
|
||||
The ode package provides classes to solve Ordinary Differential Equations problems.
|
||||
</p>
|
||||
|
|
@ -105,7 +105,7 @@
|
|||
automatic guess is wrong.
|
||||
</p>
|
||||
</subsection>
|
||||
<subsection name="14.2 Discrete Events Handling" href="events">
|
||||
<subsection name="13.2 Discrete Events Handling" href="events">
|
||||
<p>
|
||||
Discrete events detection is based on switching functions. The user provides
|
||||
a simple <a href="../apidocs/org/apache/commons/math/ode/events/EventHandler.html">g(t, y)</a>
|
||||
|
|
@ -176,7 +176,7 @@ public int eventOccurred(double t, double[] y) {
|
|||
}
|
||||
</source>
|
||||
</subsection>
|
||||
<subsection name="14.3 ODE Problems" href="problems">
|
||||
<subsection name="13.3 ODE Problems" href="problems">
|
||||
<p>
|
||||
First order ODE problems are defined by implementing the
|
||||
<a href="../apidocs/org/apache/commons/math/ode/FirstOrderDifferentialEquations.html">FirstOrderDifferentialEquations</a>
|
||||
|
|
@ -196,7 +196,7 @@ public int eventOccurred(double t, double[] y) {
|
|||
that implement it are allowed to handle them as they want.
|
||||
</p>
|
||||
</subsection>
|
||||
<subsection name="14.4 Integrators" href="integrators">
|
||||
<subsection name="13.4 Integrators" href="integrators">
|
||||
<p>
|
||||
The tables below show the various integrators available for non-stiff problems.
|
||||
</p>
|
||||
|
|
|
|||
|
|
@ -26,28 +26,93 @@
|
|||
</properties>
|
||||
|
||||
<body>
|
||||
<section name="13 Optimization">
|
||||
<subsection name="13.1 Overview" href="overview">
|
||||
<section name="12 Optimization">
|
||||
<subsection name="12.1 Overview" href="overview">
|
||||
<p>
|
||||
The optimization package provides algorithms to optimize (i.e. minimize) some
|
||||
objective or cost function. The package is split in several sub-packages
|
||||
dedicated to different kind of functions or algorithms.
|
||||
The optimization package provides algorithms to optimize (i.e. either minimize
|
||||
or maximize) some objective or cost function. The package is split in several
|
||||
sub-packages dedicated to different kind of functions or algorithms.
|
||||
<ul>
|
||||
<li>the univariate package handles univariate real functions,</li>
|
||||
<li>the univariate package handles univariate scalar functions,</li>
|
||||
<li>the linear package handles multivariate vector linear functions
|
||||
with linear constraints,</li>
|
||||
<li>the direct package handles multivariate real functions using direct
|
||||
search methods (i.e. not using derivatives),</li>
|
||||
<li>the general package handles multivariate real or vector functions
|
||||
<li>the direct package handles multivariate scalar functions
|
||||
using direct search methods (i.e. not using derivatives),</li>
|
||||
<li>the general package handles multivariate scalar or vector functions
|
||||
using derivatives.</li>
|
||||
</ul>
|
||||
</p>
|
||||
</subsection>
|
||||
<subsection name="13.2 Univariate Functions" href="univariate">
|
||||
<p>
|
||||
A <a href="../apidocs/org/apache/commons/math/analysis/minimization/UnivariateRealMinimizer.html">
|
||||
org.apache.commons.math.optimization.univariate.UnivariateRealMinimizer</a>
|
||||
is used to find the minimal values of a univariate real-valued function <code>f</code>.
|
||||
The top level optimization package provides common interfaces for the optimization
|
||||
algorithms provided in sub-packages. The main interfaces defines objective functions
|
||||
and optimizers.
|
||||
</p>
|
||||
<p>
|
||||
Objective functions interfaces are intended to be implemented by
|
||||
user code to represent the problem to minimize or maximize. When the goal is to
|
||||
minimize, the objective function is often called a cost function. Objective
|
||||
functions can be either scalar or vectorial and can be either differentiable or
|
||||
not. There are four different interfaces, one for each case:
|
||||
<ul>
|
||||
<li><a href="../apidocs/org/apache/commons/math/optimization/ScalarObjectiveFunction.html">
|
||||
ScalarObjectiveFunction</a></li>
|
||||
<li><a href="../apidocs/org/apache/commons/math/optimization/ScalarDifferentiableObjectiveFunction.html">
|
||||
ScalarDifferentiableObjectiveFunction</a></li>
|
||||
<li><a href="../apidocs/org/apache/commons/math/optimization/VectorialObjectiveFunction.html">
|
||||
VectorialObjectiveFunction</a></li>
|
||||
<li><a href="../apidocs/org/apache/commons/math/optimization/VectorialDifferentiableObjectiveFunction.html">
|
||||
VectorialDifferentiableObjectiveFunction</a></li>
|
||||
</ul>
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Optimizers are the algorithms that will either minimize or maximize, the objective function
|
||||
by changing its input variables set until an optimal set is found. There are only three
|
||||
interfaces defining the common behavior of optimizers, one for each type of objective
|
||||
function except <a href="../apidocs/org/apache/commons/math/optimization/VectorialObjectiveFunction.html">
|
||||
VectorialObjectiveFunction</a>:
|
||||
<ul>
|
||||
<li><a href="../apidocs/org/apache/commons/math/optimization/ScalarOptimizer.html">
|
||||
ScalarOptimizer</a></li>
|
||||
<li><a href="../apidocs/org/apache/commons/math/optimization/ScalarDifferentiableOptimizer.html">
|
||||
ScalarDifferentiableOptimizer</a></li>
|
||||
<li><a href="../apidocs/org/apache/commons/math/optimization/VectorialDifferentiableOptimizer.html">
|
||||
VectorialDifferentiableOptimizer</a></li>
|
||||
</ul>
|
||||
</p>
|
||||
|
||||
<p>
|
||||
Despite there are only three types of supported optimizers, it is possible to optimize a
|
||||
transform a non-differentiable <a
|
||||
href="../apidocs/org/apache/commons/math/optimization/VectorialObjectiveFunction.html">
|
||||
VectorialObjectiveFunction</a> by transforming into a <a
|
||||
href="../apidocs/org/apache/commons/math/optimization/ScalarObjectiveFunction.html">
|
||||
ScalarObjectiveFunction</a> thanks to the <a
|
||||
href="../apidocs/org/apache/commons/math/optimization/LeastSquaresConverter.html">
|
||||
LeastSquaresConverter</a> helper class. The transformed function can be optimized using any
|
||||
implementation of the <a href="../apidocs/org/apache/commons/math/optimization/ScalarOptimizer.html">
|
||||
ScalarOptimizer</a> interface.
|
||||
</p>
|
||||
|
||||
<p>
|
||||
There are also three special implementations which wrap classical optimizers in order to
|
||||
add them a multi-start feature. This feature call the underlying optimizer several times
|
||||
in sequence with different starting points and returns the best optimum found or all optima
|
||||
if desired. This is a classical way to prevent being trapped into a local extremum when
|
||||
looking for a global one. The multi-start wrappers are <a
|
||||
href="../apidocs/org/apache/commons/math/optimization/MultiStartScalarOptimizer.html">
|
||||
MultiStartScalarOptimizer</a>, <a
|
||||
href="../apidocs/org/apache/commons/math/optimization/MultiStartScalarDifferentiableOptimizer.html">
|
||||
MultiStartScalarDifferentiableOptimizer</a> and <a
|
||||
href="../apidocs/org/apache/commons/math/optimization/MultiStartVectorialDifferentiableOptimizer.html">
|
||||
MultiStartVectorialDifferentiableOptimizer</a>.
|
||||
</p>
|
||||
</subsection>
|
||||
<subsection name="12.2 Univariate Functions" href="univariate">
|
||||
<p>
|
||||
A <a href="../apidocs/org/apache/commons/math/optimization/univariate/UnivariateRealMinimizer.html">
|
||||
UnivariateRealMinimizer</a> is used to find the minimal values of a univariate scalar-valued function
|
||||
<code>f</code>.
|
||||
</p>
|
||||
<p>
|
||||
Minimization algorithms usage is very similar to root-finding algorithms usage explained
|
||||
|
|
@ -55,11 +120,14 @@
|
|||
finding algorithms is replaced by <code>minimize</code> methods.
|
||||
</p>
|
||||
</subsection>
|
||||
<subsection name="13.3 Linear Programming" href="linear">
|
||||
<subsection name="12.3 Linear Programming" href="linear">
|
||||
<p>
|
||||
This package provides an implementation of George Dantzig's simplex algorithm
|
||||
for solving linear optimization problems with linear equality and inequality
|
||||
constraints.
|
||||
</p>
|
||||
</subsection>
|
||||
<subsection name="13.4 Direct Methods" href="direct">
|
||||
<subsection name="12.4 Direct Methods" href="direct">
|
||||
<p>
|
||||
Direct search methods only use cost function values, they don't
|
||||
need derivatives and don't either try to compute approximation of
|
||||
|
|
@ -97,8 +165,45 @@
|
|||
multi-directional</a> method.
|
||||
</p>
|
||||
</subsection>
|
||||
<subsection name="13.5 General Case" href="general">
|
||||
<subsection name="12.5 General Case" href="general">
|
||||
<p>
|
||||
The general package deals with non-linear vectorial optimization problems when
|
||||
the partial derivatives of the objective function are available.
|
||||
</p>
|
||||
<p>
|
||||
One important class of estimation problems is weighted least
|
||||
squares problems. They basically consist in finding the values
|
||||
for some parameters p<sub>k</sub> such that a cost function
|
||||
J = sum(w<sub>i</sub>(mes<sub>i</sub> - mod<sub>i</sub>)<sup>2</sup>) is
|
||||
minimized. The various (target<sub>i</sub> - model<sub>i</sub>(p<sub>k</sub>))
|
||||
terms are called residuals. They represent the deviation between a set of
|
||||
target values target<sub>i</sub> and theoretical values computed from
|
||||
models model<sub>i</sub> depending on free parameters p<sub>k</sub>.
|
||||
The w<sub>i</sub> factors are weights. One classical use case is when the
|
||||
target values are experimental observations or measurements.
|
||||
</p>
|
||||
<p>
|
||||
Solving a least-squares problem is finding the free parameters p<sub>k</sub>
|
||||
of the theoretical models such that they are close to the target values, i.e.
|
||||
when the residual are small.
|
||||
</p>
|
||||
<p>
|
||||
Two optimizers are available in the general package, both devoted to least-squares
|
||||
problems. The first one is based on the <a
|
||||
href="../apidocs/org/apache/commons/math/optimization/general/GaussNewtonOptimizer.html">
|
||||
Gauss-Newton</a> method. The second one is the <a
|
||||
href="../apidocs/org/apache/commons/math/optimization/general/LevenbergMarquardtOptimizer.html">
|
||||
Levenberg-Marquardt</a> method.
|
||||
</p>
|
||||
<p>
|
||||
In order to solve a vectorial optimization problem, the user must provide it as
|
||||
an object implementing the <a
|
||||
href="../apidocs/org/apache/commons/math/optimization/VectorialDifferentiableObjectiveFunction.html">
|
||||
VectorialDifferentiableObjectiveFunction</a> interface. The object will be provided to
|
||||
the <code>estimate</code> method of the optimizer, along with the target and weight arrays,
|
||||
thus allowing the optimizer to compute the residuals at will. The last parameter to the
|
||||
<code>estimate</code> method is the point from which the optimizer will start its
|
||||
search for the optimal point.
|
||||
</p>
|
||||
</subsection>
|
||||
</section>
|
||||
|
|
|
|||
Loading…
Reference in New Issue