Small java application for the accuracy assessment of implementations of real
functions in Commons Math. The accuracy is assessed through comparison with reference values computed with multi-precision softwares. git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1407376 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Sebastien Brisard
parent
37791912c6
commit
b04b10ca6e
|
|
@ -0,0 +1 @@
|
|||
Main-Class: RealFunctionValidation
|
||||
|
|
@ -0,0 +1,139 @@
|
|||
/*
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
Validation of real functions
|
||||
============================
|
||||
|
||||
This document details the procedure used in Commons-Math 3 to assess the
|
||||
accuracy of the implementations of special functions. It is a two-step process
|
||||
|
||||
1. reference values are computed with a multi-precision software (for example,
|
||||
the Maxima Computer Algebra System) [1],
|
||||
2. these reference values are compared with the Commons-Math3 implementation.
|
||||
The accuracy is computed in ulps.
|
||||
|
||||
This process relies on a small Java application, called RealFunctionValidation,
|
||||
which can be found in $CM3_SRC/src/test/maxima/special, where $CM3_SRC is the
|
||||
root directory to the source of Commons-Math 3
|
||||
|
||||
|
||||
Compilation of RealFunctionValidation
|
||||
-------------------------------------
|
||||
|
||||
Change to the relevant directory
|
||||
|
||||
cd $CM3_SRC/src/test/maxima/special/RealFunctionValidation
|
||||
|
||||
Compile the source file. The jar file of Commons-Math3 should be included in
|
||||
your classpath. If it is installed in your local maven repository, the
|
||||
following command should work
|
||||
|
||||
javac -classpath $HOME/.m2/repository/org/apache/commons/commons-math3/3.1-SNAPSHOT/commons-math3-3.1-SNAPSHOT.jar RealFunctionValidation.java
|
||||
|
||||
Create a jar file
|
||||
|
||||
jar cfm RealFunctionValidation.jar Manifest.txt RealFunctionValidation*.class
|
||||
|
||||
Remove the unused *.class files
|
||||
|
||||
rm *.class
|
||||
|
||||
|
||||
Invocation of the application RealFunctionValidation
|
||||
----------------------------------------------------
|
||||
|
||||
The java application comes with a shell script, RealFunctionValidaton.sh. You
|
||||
should edit this file, and change the variables
|
||||
- CM3_JAR: full path to the Commons-Math 3 jar file,
|
||||
- APP_JAR: full path to the RealFunctionValidation application jar file.
|
||||
|
||||
Invoking this application is then very simple. For example, to validate the
|
||||
implementation of Gamma.logGamma, change to directory reference
|
||||
|
||||
cd $CM3_SRC/src/test/maxima/special/reference
|
||||
|
||||
and run the application
|
||||
|
||||
../RealFunctionValidation/RealFunctionValidation.sh logGamma.properties
|
||||
|
||||
|
||||
Syntax of the *.properties files
|
||||
--------------------------------
|
||||
|
||||
Parameters of the RealFunctionValidation application are specified through a
|
||||
standard Java properties file. The following keys must be specified in this
|
||||
file
|
||||
|
||||
- method: the fully qualified name to the function to be validated. This
|
||||
function should be static, take only primitive arguments, and return double.
|
||||
- signature: this key is necessary to discriminate functions with same name.
|
||||
The signature should be specified as in a plain java file. For example
|
||||
signature = double, int, float
|
||||
- inputFileMask: the name of the binary input file(s) containing the
|
||||
high-accuracy reference values. The format of this file is described in
|
||||
the next section. It is possible to specify multiple input files, which are
|
||||
indexed by an integer. Then this key should really be understood as a format
|
||||
string. In other words, the name of the file with index i is given by
|
||||
String.format(inputFileMask, i)
|
||||
- outputFileMask: the name of the binary output file(s) containing the
|
||||
reference values, the values computed through the specified method, and
|
||||
the error (in ulps). The format of this file is described in the next section. As for the input files, it is possible to specify multiple output files.
|
||||
- from: the first index
|
||||
- to: the last index (exclusive)
|
||||
- by: the increment
|
||||
|
||||
As an example, here is the properties file for evaluation of
|
||||
double Gamma.logGamma(double)
|
||||
|
||||
method=org.apache.commons.math3.special.Gamma.logGamma
|
||||
signature=double
|
||||
inputFileMask=logGamma-%02d.dat
|
||||
outputFileMask=logGamma-out-%02d.dat
|
||||
from=1
|
||||
to=5
|
||||
by=1
|
||||
|
||||
Format of the input and output binary files
|
||||
-------------------------------------------
|
||||
|
||||
The reference values are saved in a binary file
|
||||
- for a unary function f(x), the data is stored as follows
|
||||
x[0], f(x[0]), x[1], f(x[1]), ...
|
||||
- for a binary function f(x, y), the data is stored as follows
|
||||
x[0], y[0], f(x[0], y[0]), x[1], y[1], f(x[1], y[1]), ...
|
||||
- and similar storage pattern for a n-ary function.
|
||||
|
||||
The parameters x[i], y[i], ... can be of arbitrary (primitive) type. The return
|
||||
value f(x[i], y[i], ...) must be of type double.
|
||||
|
||||
The output files are also saved in a binary file
|
||||
- for a unary function f(x), the data is stored as follows
|
||||
x[0], reference value of f(x[0]), actual value of f(x[0], y[0]),
|
||||
error in ulps, x[1], y[1], reference value of f(x[1], y[1]), actual value of
|
||||
f(x[1], y[1]), error in ulps, ...
|
||||
- for a binary function f(x, y), the data is stored as follows
|
||||
x[0], y[0], reference value of f(x[0], y[0]), actual value of f(x[0], y[0]),
|
||||
error in ulps, x[1], y[1], reference value of f(x[1], y[1]), actual value of
|
||||
f(x[1], y[1]), error in ulps, ...
|
||||
|
||||
The application also prints on the standard output some statistics about the
|
||||
error.
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
[1] http://maxima.sourceforge.net/
|
||||
|
|
@ -0,0 +1,361 @@
|
|||
/*
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import java.io.DataInputStream;
|
||||
import java.io.DataOutputStream;
|
||||
import java.io.EOFException;
|
||||
import java.io.FileInputStream;
|
||||
import java.io.FileOutputStream;
|
||||
import java.io.IOException;
|
||||
import java.lang.reflect.InvocationTargetException;
|
||||
import java.lang.reflect.Method;
|
||||
import java.lang.reflect.Modifier;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Properties;
|
||||
|
||||
import org.apache.commons.math3.stat.descriptive.SummaryStatistics;
|
||||
import org.apache.commons.math3.util.FastMath;
|
||||
|
||||
/*
|
||||
* plot 'logGamma.dat' binary format="%double%double" endian=big u 1:2 w l
|
||||
*/
|
||||
public class RealFunctionValidation {
|
||||
|
||||
public static class MissingRequiredPropertyException
|
||||
extends IllegalArgumentException {
|
||||
|
||||
private static final long serialVersionUID = 20121017L;
|
||||
|
||||
public MissingRequiredPropertyException(final String key) {
|
||||
|
||||
super("missing required property " + key);
|
||||
}
|
||||
}
|
||||
|
||||
public static class ApplicationProperties {
|
||||
|
||||
private static final int DOT = '.';
|
||||
|
||||
private static final String METHOD_KEY = "method";
|
||||
|
||||
private static final String SIGNATURE_KEY = "signature";
|
||||
|
||||
private static final String INPUT_FILE_MASK = "inputFileMask";
|
||||
|
||||
private static final String OUTPUT_FILE_MASK = "outputFileMask";
|
||||
|
||||
private static final String FROM_KEY = "from";
|
||||
|
||||
private static final String TO_KEY = "to";
|
||||
|
||||
private static final String BY_KEY = "by";
|
||||
|
||||
final Method method;
|
||||
|
||||
final String inputFileMask;
|
||||
|
||||
final String outputFileMask;
|
||||
|
||||
final int from;
|
||||
|
||||
final int to;
|
||||
|
||||
final int by;
|
||||
|
||||
/**
|
||||
* Returns a {@link Method} with specified signature.
|
||||
*
|
||||
* @param className The fully qualified name of the class to which the
|
||||
* method belongs.
|
||||
* @param methodName The name of the method.
|
||||
* @param signature The signature of the method, as a list of parameter
|
||||
* types.
|
||||
* @return the method
|
||||
* @throws SecurityException
|
||||
* @throws ClassNotFoundException
|
||||
*/
|
||||
public static Method findStaticMethod(final String className,
|
||||
final String methodName,
|
||||
final List<Class<?>> signature)
|
||||
throws SecurityException, ClassNotFoundException {
|
||||
|
||||
final int n = signature.size();
|
||||
final Method[] methods = Class.forName(className).getMethods();
|
||||
for (Method method : methods) {
|
||||
if (method.getName().equals(methodName)) {
|
||||
final Class<?>[] parameters = method.getParameterTypes();
|
||||
boolean sameSignature = true;
|
||||
if (parameters.length == n) {
|
||||
for (int i = 0; i < n; i++) {
|
||||
sameSignature &= signature.get(i)
|
||||
.equals(parameters[i]);
|
||||
}
|
||||
if (sameSignature) {
|
||||
final int modifiers = method.getModifiers();
|
||||
if ((modifiers & Modifier.STATIC) != 0) {
|
||||
return method;
|
||||
} else {
|
||||
final String msg = "method must be static";
|
||||
throw new IllegalArgumentException(msg);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
throw new IllegalArgumentException("method not found");
|
||||
}
|
||||
|
||||
public static Class<?> parsePrimitiveType(final String type) {
|
||||
|
||||
if (type.equals("boolean")) {
|
||||
return Boolean.TYPE;
|
||||
} else if (type.equals("byte")) {
|
||||
return Byte.TYPE;
|
||||
} else if (type.equals("char")) {
|
||||
return Character.TYPE;
|
||||
} else if (type.equals("double")) {
|
||||
return Double.TYPE;
|
||||
} else if (type.equals("float")) {
|
||||
return Float.TYPE;
|
||||
} else if (type.equals("int")) {
|
||||
return Integer.TYPE;
|
||||
} else if (type.equals("long")) {
|
||||
return Long.TYPE;
|
||||
} else if (type.equals("short")) {
|
||||
return Short.TYPE;
|
||||
} else {
|
||||
final StringBuilder builder = new StringBuilder();
|
||||
builder.append(type).append(" is not a primitive type");
|
||||
throw new IllegalArgumentException(builder.toString());
|
||||
}
|
||||
}
|
||||
|
||||
private static String getPropertyAsString(final Properties properties,
|
||||
final String key) {
|
||||
|
||||
final String value = properties.getProperty(key);
|
||||
if (value == null) {
|
||||
throw new MissingRequiredPropertyException(key);
|
||||
} else {
|
||||
return value;
|
||||
}
|
||||
}
|
||||
|
||||
private static int getPropertyAsInteger(final Properties properties,
|
||||
final String key) {
|
||||
|
||||
final String value = properties.getProperty(key);
|
||||
if (value == null) {
|
||||
throw new MissingRequiredPropertyException(key);
|
||||
} else {
|
||||
return Integer.parseInt(value);
|
||||
}
|
||||
}
|
||||
|
||||
private ApplicationProperties(final String fullyQualifiedName,
|
||||
final String signature,
|
||||
final String inputFileMask,
|
||||
final String outputFileMask,
|
||||
final int from, final int to, final int by) {
|
||||
|
||||
this.inputFileMask = inputFileMask;
|
||||
this.outputFileMask = outputFileMask;
|
||||
this.from = from;
|
||||
this.to = to;
|
||||
this.by = by;
|
||||
|
||||
final String[] types = signature.split(",");
|
||||
final List<Class<?>> parameterTypes = new ArrayList<Class<?>>();
|
||||
for (String type : types) {
|
||||
parameterTypes.add(parsePrimitiveType(type.trim()));
|
||||
}
|
||||
final int index = fullyQualifiedName.lastIndexOf(DOT);
|
||||
try {
|
||||
final String className, methodName;
|
||||
className = fullyQualifiedName.substring(0, index);
|
||||
methodName = fullyQualifiedName.substring(index + 1);
|
||||
this.method = findStaticMethod(className, methodName,
|
||||
parameterTypes);
|
||||
} catch (ClassNotFoundException e) {
|
||||
throw new IllegalArgumentException(e);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
public static final ApplicationProperties create(final Properties properties) {
|
||||
|
||||
final String methodFullyQualifiedName;
|
||||
methodFullyQualifiedName = getPropertyAsString(properties,
|
||||
METHOD_KEY);
|
||||
|
||||
final String signature;
|
||||
signature = getPropertyAsString(properties, SIGNATURE_KEY);
|
||||
|
||||
final String inputFileMask;
|
||||
inputFileMask = getPropertyAsString(properties, INPUT_FILE_MASK);
|
||||
|
||||
final String outputFileMask;
|
||||
outputFileMask = getPropertyAsString(properties, OUTPUT_FILE_MASK);
|
||||
|
||||
final int from = getPropertyAsInteger(properties, FROM_KEY);
|
||||
final int to = getPropertyAsInteger(properties, TO_KEY);
|
||||
final int by = getPropertyAsInteger(properties, BY_KEY);
|
||||
|
||||
return new ApplicationProperties(methodFullyQualifiedName,
|
||||
signature, inputFileMask,
|
||||
outputFileMask, from, to, by);
|
||||
}
|
||||
};
|
||||
|
||||
public static Object readAndWritePrimitiveValue(final DataInputStream in,
|
||||
final DataOutputStream out,
|
||||
final Class<?> type)
|
||||
throws IOException {
|
||||
|
||||
if (!type.isPrimitive()) {
|
||||
throw new IllegalArgumentException("type must be primitive");
|
||||
}
|
||||
if (type.equals(Boolean.TYPE)) {
|
||||
final boolean x = in.readBoolean();
|
||||
out.writeBoolean(x);
|
||||
return Boolean.valueOf(x);
|
||||
} else if (type.equals(Byte.TYPE)) {
|
||||
final byte x = in.readByte();
|
||||
out.writeByte(x);
|
||||
return Byte.valueOf(x);
|
||||
} else if (type.equals(Character.TYPE)) {
|
||||
final char x = in.readChar();
|
||||
out.writeChar(x);
|
||||
return Character.valueOf(x);
|
||||
} else if (type.equals(Double.TYPE)) {
|
||||
final double x = in.readDouble();
|
||||
out.writeDouble(x);
|
||||
return Double.valueOf(x);
|
||||
} else if (type.equals(Float.TYPE)) {
|
||||
final float x = in.readFloat();
|
||||
out.writeFloat(x);
|
||||
return Float.valueOf(x);
|
||||
} else if (type.equals(Integer.TYPE)) {
|
||||
final int x = in.readInt();
|
||||
out.writeInt(x);
|
||||
return Integer.valueOf(x);
|
||||
} else if (type.equals(Long.TYPE)) {
|
||||
final long x = in.readLong();
|
||||
out.writeLong(x);
|
||||
return Long.valueOf(x);
|
||||
} else if (type.equals(Short.TYPE)) {
|
||||
final short x = in.readShort();
|
||||
out.writeShort(x);
|
||||
return Short.valueOf(x);
|
||||
} else {
|
||||
// This should never occur.
|
||||
throw new IllegalStateException();
|
||||
}
|
||||
}
|
||||
|
||||
public static SummaryStatistics assessAccuracy(final Method method,
|
||||
final DataInputStream in,
|
||||
final DataOutputStream out)
|
||||
throws IOException, IllegalAccessException, IllegalArgumentException,
|
||||
InvocationTargetException {
|
||||
|
||||
if (method.getReturnType() != Double.TYPE) {
|
||||
throw new IllegalArgumentException("method must return a double");
|
||||
}
|
||||
|
||||
final Class<?>[] types = method.getParameterTypes();
|
||||
for (int i = 0; i < types.length; i++) {
|
||||
if (!types[i].isPrimitive()) {
|
||||
final StringBuilder builder = new StringBuilder();
|
||||
builder.append("argument #").append(i + 1)
|
||||
.append(" of method ").append(method.getName())
|
||||
.append("must be of primitive of type");
|
||||
throw new IllegalArgumentException(builder.toString());
|
||||
}
|
||||
}
|
||||
|
||||
final SummaryStatistics stat = new SummaryStatistics();
|
||||
final Object[] parameters = new Object[types.length];
|
||||
while (true) {
|
||||
try {
|
||||
for (int i = 0; i < parameters.length; i++) {
|
||||
parameters[i] = readAndWritePrimitiveValue(in, out,
|
||||
types[i]);
|
||||
}
|
||||
final double expected = in.readDouble();
|
||||
if (FastMath.abs(expected) > 1E-16) {
|
||||
final Object value = method.invoke(null, parameters);
|
||||
final double actual = ((Double) value).doubleValue();
|
||||
final double err = FastMath.abs(actual - expected);
|
||||
final double ulps = err / FastMath.ulp(expected);
|
||||
out.writeDouble(expected);
|
||||
out.writeDouble(actual);
|
||||
out.writeDouble(ulps);
|
||||
stat.addValue(ulps);
|
||||
}
|
||||
} catch (EOFException e) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
return stat;
|
||||
}
|
||||
|
||||
public static void run(final ApplicationProperties properties)
|
||||
throws IllegalAccessException, IllegalArgumentException,
|
||||
InvocationTargetException, IOException {
|
||||
|
||||
for (int i = properties.from; i < properties.to; i += properties.by) {
|
||||
final String inputFileName;
|
||||
inputFileName = String.format(properties.inputFileMask, i);
|
||||
final String outputFileName;
|
||||
outputFileName = String.format(properties.outputFileMask, i);
|
||||
|
||||
final DataInputStream in;
|
||||
in = new DataInputStream(new FileInputStream(inputFileName));
|
||||
final DataOutputStream out;
|
||||
out = new DataOutputStream(new FileOutputStream(outputFileName));
|
||||
|
||||
final SummaryStatistics stats;
|
||||
stats = assessAccuracy(properties.method, in, out);
|
||||
|
||||
System.out.println("input file name = " + inputFileName);
|
||||
System.out.println("output file name = " + outputFileName);
|
||||
System.out.println(stats);
|
||||
}
|
||||
}
|
||||
|
||||
public static void main(final String[] args)
|
||||
throws IOException, IllegalAccessException, IllegalArgumentException,
|
||||
InvocationTargetException {
|
||||
|
||||
if (args.length == 0) {
|
||||
final String msg = "missing required properties file";
|
||||
throw new IllegalArgumentException(msg);
|
||||
}
|
||||
|
||||
final FileInputStream in = new FileInputStream(args[0]);
|
||||
final Properties properties = new Properties();
|
||||
properties.load(in);
|
||||
in.close();
|
||||
|
||||
final ApplicationProperties p;
|
||||
p = ApplicationProperties.create(properties);
|
||||
|
||||
run(p);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Location of the Commons-Math3 jar file
|
||||
CM3_JAR=$HOME/.m2/repository/org/apache/commons/commons-math3/3.1-SNAPSHOT/commons-math3-3.1-SNAPSHOT.jar
|
||||
|
||||
# Location of file RealFunctionValidation.jar
|
||||
APP_JAR=$HOME/Documents/workspace/commons-math3/src/test/maxima/special/RealFunctionValidation/RealFunctionValidation.jar
|
||||
|
||||
java -cp $CM3_JAR:$APP_JAR RealFunctionValidation logGamma.properties
|
||||
Loading…
Reference in New Issue