- The residuals are the really the values of the objective function
that is being minimized.
- Forcing the value to be a real vector and the residual computed
through subtraction is an unnecessary constraint. For example
Rotation.distance() will compute a better residual than
subtracting the quaternion elements of two rotations.
- Method was not used by any optimizer.
JIRA: MATH-1102
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1571306 13f79535-47bb-0310-9956-ffa450edef68
The documentation now explicitly states that entries not iterated above
are the zero ones.
This is part of Apache Commons Math reconsidering support for sparse
linear algebra.
JIRA: MATH-875
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1570510 13f79535-47bb-0310-9956-ffa450edef68
Theoretically QR offers the best blend of speed and numerical stability.
Empirically the QR implementation is slightly faster than the Cholesky
implementation.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569907 13f79535-47bb-0310-9956-ffa450edef68
Re-factored the code in GaussNewtonOptimizer so that the decomposition
algorithm sees the Jacobian and residuals instead of the normal
equation. This lets the QR algorithm operate directly on the Jacobian
matrix, which is faster and less sensitive to numerical errors. As a
result, one test case that threw a singular matrix exception now passes
with the QR decomposition.
The refactoring also include a speed improvement when computing the
normal matrix for the LU decomposition. Since the normal matrix is
symmetric only half of is computed, which results in a factor of 2 speed
up in computing the normal matrix for problems with many more
measurements than states.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569905 13f79535-47bb-0310-9956-ffa450edef68
This is a reversal of a former decision, as we now think we should adopt
a generally accepted behavior which is ... to ignore the problems of
NaNs and infinities in sparse linear algebra entities.
JIRA: MATH-870 (which is therefore NOT fixed)
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569825 13f79535-47bb-0310-9956-ffa450edef68
Added more test cases to EvaluationTest to test the other methods of
DenseWeightedEvaluation/UnweightedEvaluation.
Fixed a bug in DenseWeightedEvaluation.computeValue() where the the
weights were not applied to the return value.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569361 13f79535-47bb-0310-9956-ffa450edef68
Previously JUnit would make the call to test a specific optimizer, and
then that method would call all of the individual test cases relevant to
that optimizer.,Now JUnit will directly call each individual test case.
The same test coverage is preserved. The GaussNewtonOptimizerTest is
split into two classes, one for each decomposition algorithm it can use.
There is a significant amount of duplicated code between
GaussNewtonOptimizerWithLUTest and GaussNewtonOptimizerWithQRTest.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569359 13f79535-47bb-0310-9956-ffa450edef68
* There are now 3 factory methods: one using the previous interfaces, one using
the new interfaces with weights, and one using the new interfaces without
weights.
* Make model(...) method public.
* Fix javadoc typo
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569355 13f79535-47bb-0310-9956-ffa450edef68
Covered all of the interfaces in the leastsquares package to use RealVector and
RealMatrix instead of double[] and double[][]. This reduced some duplicated code.
For example Evaluation.computeResiduals() was a complete duplication of
RealVector.subtract(). It also presents a consistent interface and allows data
encapsulation.
Lastly, this change enables [math] to "eat our own dog food." It enables the
linear package to be used in the implementation of the optimization algorithms.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569354 13f79535-47bb-0310-9956-ffa450edef68
Use Evaluation instead of PointVectorValuePair in the ConvergenceChecker. This
gives the checkers access to more information, such as the rms and covariances.
The change also simplified the optimizer implementations since they no longer
have to keep track of the current function value.
A method was added to LeastSquaresFactory to convert between the two types of
checkers and a method added to LeastSquaresBuilder so that it can accept either
type. I would have prefered to do this through method overloading, but
overloading doesn't play well with generics.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569353 13f79535-47bb-0310-9956-ffa450edef68
Changed the boolean useLu prarameter to an enum. This is benificial because:
1. user code is easier to read: new GNO(Decomposition.QR) instead of new GNO(false)
2. Allows other algorithms to be added in the future
3. developer code is easier to read. One less if statement in optimize()
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569352 13f79535-47bb-0310-9956-ffa450edef68
The main loop in GN could only be ended by convergence or exception. I made that
explicit with while(true) and removed an unreachable MathInternalError.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569350 13f79535-47bb-0310-9956-ffa450edef68
A new interface MultivariateJacobianFunction lets the function value and
Jacobian be evaluated at the same time. This saves the user from having to cache
the result between calls to get the value and the jacobian.
A factory method was added to create LeastSquaresProblems from the new interface.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569346 13f79535-47bb-0310-9956-ffa450edef68
The weights are no longer implicit in LeastSquaresProblem.Evaluation. They are
already included in the computed residuals and Jacobian.
GN and LM multiplied the residuals by the weights immediately, so that was easy
to remove.
Created an AbstractEvaluation class which handles the derived quantitied (cost,
rms, covariance,...) and two implementations. UnweightedEvaluation uses the
straight forward formulas. DenseWeightedEvaluation delegates to an Evaluation
and multiples the residuals and Jacobian by the square root of the weight matrix
before returning them. Allowed me to remove the reference to the full weight
matrix.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569344 13f79535-47bb-0310-9956-ffa450edef68
The GN and LM optimizers now use an immutable fields for their configuration
parameters and a fluent api. LM still has some mutable fields it uses a scratch
space.
This commit is a backport of Evan Ward github commit
https://github.com/wardev/commons-math/commit/157cdde
into the current development version.
Conflicts:
src/main/java/org/apache/commons/math3/fitting/leastsquares/GaussNewtonOptimizer.java
src/main/java/org/apache/commons/math3/fitting/leastsquares/LevenbergMarquardtOptimizer.java
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569343 13f79535-47bb-0310-9956-ffa450edef68
The least squares package now has two main interfaces LeastSquaresOptimizer and
LeastSquaresProblem, where the optimizer can be applied to the problem.
In this commit all of the test cases pass with one change (annotated) to
LevenbergMarquardt.testControlParameters(). The tests for Gauss Newton were
expanded to cover using the QR decomposition, which revealed a QR default
tolerance discrepancy.
A factory and a mutable builder were create for LeastSquaresProblem.
This commit is a backport of Evan Ward github commit
7e9a15efb5
into the current development version.
Conflicts:
src/main/java/org/apache/commons/math3/fitting/leastsquares/AbstractLeastSquaresOptimizer.java
src/main/java/org/apache/commons/math3/fitting/leastsquares/GaussNewtonOptimizer.java
src/main/java/org/apache/commons/math3/optim/AbstractOptimizer.java
src/test/java/org/apache/commons/math3/fitting/leastsquares/AbstractLeastSquaresOptimizerAbstractTest.java
src/test/java/org/apache/commons/math3/fitting/leastsquares/GaussNewtonOptimizerTest.java
src/test/java/org/apache/commons/math3/fitting/leastsquares/LevenbergMarquardtOptimizerTest.java
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1569342 13f79535-47bb-0310-9956-ffa450edef68
The previous version was based on Welzl algorithm, but it appeared this
algorithm really need some properties that hold in Euclidean spaces and
not on the sphere.
The current version is not perfect in the sense that some times the
enclosing spherical cap returned is not the smallest possible. It is
documented in the Javadoc.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1567599 13f79535-47bb-0310-9956-ffa450edef68
Luc Maisonobe
Gilles Sadowski
Thomas Neidhart