Apache
/
commons-math
mirror of https://github.com/apache/commons-math.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Small java application for the accuracy assessment of implementations of real 

functions in Commons Math. The accuracy is assessed through comparison with
reference values computed with multi-precision softwares.


git-svn-id: https://svn.apache.org/repos/asf/commons/proper/math/trunk@1407376 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Sebastien Brisard 2012-11-09 08:13:18 +00:00
parent 37791912c6
commit b04b10ca6e
4 changed files with 510 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
src/test/maxima/special/RealFunctionValidation/MANIFEST.txt Normal file
View File

@ -0,0 +1 @@
Main-Class: RealFunctionValidation

139
src/test/maxima/special/RealFunctionValidation/README.txt Normal file
View File

@ -0,0 +1,139 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
Validation of real functions
============================
This document details the procedure used in Commons-Math 3 to assess the
accuracy of the implementations of special functions. It is a two-step process
1. reference values are computed with a multi-precision software (for example,
the Maxima Computer Algebra System) [1],
2. these reference values are compared with the Commons-Math3 implementation.
The accuracy is computed in ulps.
This process relies on a small Java application, called RealFunctionValidation,
which can be found in $CM3_SRC/src/test/maxima/special, where $CM3_SRC is the
root directory to the source of Commons-Math 3
Compilation of RealFunctionValidation
-------------------------------------
Change to the relevant directory
cd $CM3_SRC/src/test/maxima/special/RealFunctionValidation
Compile the source file. The jar file of Commons-Math3 should be included in
your classpath. If it is installed in your local maven repository, the
following command should work
javac -classpath $HOME/.m2/repository/org/apache/commons/commons-math3/3.1-SNAPSHOT/commons-math3-3.1-SNAPSHOT.jar RealFunctionValidation.java
Create a jar file
jar cfm RealFunctionValidation.jar Manifest.txt RealFunctionValidation*.class
Remove the unused *.class files
rm *.class
Invocation of the application RealFunctionValidation
----------------------------------------------------
The java application comes with a shell script, RealFunctionValidaton.sh. You
should edit this file, and change the variables
- CM3_JAR: full path to the Commons-Math 3 jar file,
- APP_JAR: full path to the RealFunctionValidation application jar file.
Invoking this application is then very simple. For example, to validate the
implementation of Gamma.logGamma, change to directory reference
cd $CM3_SRC/src/test/maxima/special/reference
and run the application
../RealFunctionValidation/RealFunctionValidation.sh logGamma.properties
Syntax of the *.properties files
--------------------------------
Parameters of the RealFunctionValidation application are specified through a
standard Java properties file. The following keys must be specified in this
file
- method: the fully qualified name to the function to be validated. This
function should be static, take only primitive arguments, and return double.
- signature: this key is necessary to discriminate functions with same name.
The signature should be specified as in a plain java file. For example
signature = double, int, float
- inputFileMask: the name of the binary input file(s) containing the
high-accuracy reference values. The format of this file is described in
the next section. It is possible to specify multiple input files, which are
indexed by an integer. Then this key should really be understood as a format
string. In other words, the name of the file with index i is given by
String.format(inputFileMask, i)
- outputFileMask: the name of the binary output file(s) containing the
reference values, the values computed through the specified method, and
the error (in ulps). The format of this file is described in the next section. As for the input files, it is possible to specify multiple output files.
- from: the first index
- to: the last index (exclusive)
- by: the increment
As an example, here is the properties file for evaluation of
double Gamma.logGamma(double)
method=org.apache.commons.math3.special.Gamma.logGamma
signature=double
inputFileMask=logGamma-%02d.dat
outputFileMask=logGamma-out-%02d.dat
from=1
to=5
by=1
Format of the input and output binary files
-------------------------------------------
The reference values are saved in a binary file
- for a unary function f(x), the data is stored as follows
x[0], f(x[0]), x[1], f(x[1]), ...
- for a binary function f(x, y), the data is stored as follows
x[0], y[0], f(x[0], y[0]), x[1], y[1], f(x[1], y[1]), ...
- and similar storage pattern for a n-ary function.
The parameters x[i], y[i], ... can be of arbitrary (primitive) type. The return
value f(x[i], y[i], ...) must be of type double.
The output files are also saved in a binary file
- for a unary function f(x), the data is stored as follows
x[0], reference value of f(x[0]), actual value of f(x[0], y[0]),
error in ulps, x[1], y[1], reference value of f(x[1], y[1]), actual value of
f(x[1], y[1]), error in ulps, ...
- for a binary function f(x, y), the data is stored as follows
x[0], y[0], reference value of f(x[0], y[0]), actual value of f(x[0], y[0]),
error in ulps, x[1], y[1], reference value of f(x[1], y[1]), actual value of
f(x[1], y[1]), error in ulps, ...
The application also prints on the standard output some statistics about the
error.
References
----------
[1] http://maxima.sourceforge.net/

361
src/test/maxima/special/RealFunctionValidation/RealFunctionValidation.java Executable file
View File

@ -0,0 +1,361 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.EOFException;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import org.apache.commons.math3.stat.descriptive.SummaryStatistics;
import org.apache.commons.math3.util.FastMath;
/*
* plot 'logGamma.dat' binary format="%double%double" endian=big u 1:2 w l
*/
public class RealFunctionValidation {
public static class MissingRequiredPropertyException
extends IllegalArgumentException {
private static final long serialVersionUID = 20121017L;
public MissingRequiredPropertyException(final String key) {
super("missing required property " + key);
}
}
public static class ApplicationProperties {
private static final int DOT = '.';
private static final String METHOD_KEY = "method";
private static final String SIGNATURE_KEY = "signature";
private static final String INPUT_FILE_MASK = "inputFileMask";
private static final String OUTPUT_FILE_MASK = "outputFileMask";
private static final String FROM_KEY = "from";
private static final String TO_KEY = "to";
private static final String BY_KEY = "by";
final Method method;
final String inputFileMask;
final String outputFileMask;
final int from;
final int to;
final int by;
/**
* Returns a {@link Method} with specified signature.
*
* @param className The fully qualified name of the class to which the
* method belongs.
* @param methodName The name of the method.
* @param signature The signature of the method, as a list of parameter
* types.
* @return the method
* @throws SecurityException
* @throws ClassNotFoundException
*/
public static Method findStaticMethod(final String className,
final String methodName,
final List<Class<?>> signature)
throws SecurityException, ClassNotFoundException {
final int n = signature.size();
final Method[] methods = Class.forName(className).getMethods();
for (Method method : methods) {
if (method.getName().equals(methodName)) {
final Class<?>[] parameters = method.getParameterTypes();
boolean sameSignature = true;
if (parameters.length == n) {
for (int i = 0; i < n; i++) {
sameSignature &= signature.get(i)
.equals(parameters[i]);
}
if (sameSignature) {
final int modifiers = method.getModifiers();
if ((modifiers & Modifier.STATIC) != 0) {
return method;
} else {
final String msg = "method must be static";
throw new IllegalArgumentException(msg);
}
}
}
}
}
throw new IllegalArgumentException("method not found");
}
public static Class<?> parsePrimitiveType(final String type) {
if (type.equals("boolean")) {
return Boolean.TYPE;
} else if (type.equals("byte")) {
return Byte.TYPE;
} else if (type.equals("char")) {
return Character.TYPE;
} else if (type.equals("double")) {
return Double.TYPE;
} else if (type.equals("float")) {
return Float.TYPE;
} else if (type.equals("int")) {
return Integer.TYPE;
} else if (type.equals("long")) {
return Long.TYPE;
} else if (type.equals("short")) {
return Short.TYPE;
} else {
final StringBuilder builder = new StringBuilder();
builder.append(type).append(" is not a primitive type");
throw new IllegalArgumentException(builder.toString());
}
}
private static String getPropertyAsString(final Properties properties,
final String key) {
final String value = properties.getProperty(key);
if (value == null) {
throw new MissingRequiredPropertyException(key);
} else {
return value;
}
}
private static int getPropertyAsInteger(final Properties properties,
final String key) {
final String value = properties.getProperty(key);
if (value == null) {
throw new MissingRequiredPropertyException(key);
} else {
return Integer.parseInt(value);
}
}
private ApplicationProperties(final String fullyQualifiedName,
final String signature,
final String inputFileMask,
final String outputFileMask,
final int from, final int to, final int by) {
this.inputFileMask = inputFileMask;
this.outputFileMask = outputFileMask;
this.from = from;
this.to = to;
this.by = by;
final String[] types = signature.split(",");
final List<Class<?>> parameterTypes = new ArrayList<Class<?>>();
for (String type : types) {
parameterTypes.add(parsePrimitiveType(type.trim()));
}
final int index = fullyQualifiedName.lastIndexOf(DOT);
try {
final String className, methodName;
className = fullyQualifiedName.substring(0, index);
methodName = fullyQualifiedName.substring(index + 1);
this.method = findStaticMethod(className, methodName,
parameterTypes);
} catch (ClassNotFoundException e) {
throw new IllegalArgumentException(e);
}
}
public static final ApplicationProperties create(final Properties properties) {
final String methodFullyQualifiedName;
methodFullyQualifiedName = getPropertyAsString(properties,
METHOD_KEY);
final String signature;
signature = getPropertyAsString(properties, SIGNATURE_KEY);
final String inputFileMask;
inputFileMask = getPropertyAsString(properties, INPUT_FILE_MASK);
final String outputFileMask;
outputFileMask = getPropertyAsString(properties, OUTPUT_FILE_MASK);
final int from = getPropertyAsInteger(properties, FROM_KEY);
final int to = getPropertyAsInteger(properties, TO_KEY);
final int by = getPropertyAsInteger(properties, BY_KEY);
return new ApplicationProperties(methodFullyQualifiedName,
signature, inputFileMask,
outputFileMask, from, to, by);
}
};
public static Object readAndWritePrimitiveValue(final DataInputStream in,
final DataOutputStream out,
final Class<?> type)
throws IOException {
if (!type.isPrimitive()) {
throw new IllegalArgumentException("type must be primitive");
}
if (type.equals(Boolean.TYPE)) {
final boolean x = in.readBoolean();
out.writeBoolean(x);
return Boolean.valueOf(x);
} else if (type.equals(Byte.TYPE)) {
final byte x = in.readByte();
out.writeByte(x);
return Byte.valueOf(x);
} else if (type.equals(Character.TYPE)) {
final char x = in.readChar();
out.writeChar(x);
return Character.valueOf(x);
} else if (type.equals(Double.TYPE)) {
final double x = in.readDouble();
out.writeDouble(x);
return Double.valueOf(x);
} else if (type.equals(Float.TYPE)) {
final float x = in.readFloat();
out.writeFloat(x);
return Float.valueOf(x);
} else if (type.equals(Integer.TYPE)) {
final int x = in.readInt();
out.writeInt(x);
return Integer.valueOf(x);
} else if (type.equals(Long.TYPE)) {
final long x = in.readLong();
out.writeLong(x);
return Long.valueOf(x);
} else if (type.equals(Short.TYPE)) {
final short x = in.readShort();
out.writeShort(x);
return Short.valueOf(x);
} else {
// This should never occur.
throw new IllegalStateException();
}
}
public static SummaryStatistics assessAccuracy(final Method method,
final DataInputStream in,
final DataOutputStream out)
throws IOException, IllegalAccessException, IllegalArgumentException,
InvocationTargetException {
if (method.getReturnType() != Double.TYPE) {
throw new IllegalArgumentException("method must return a double");
}
final Class<?>[] types = method.getParameterTypes();
for (int i = 0; i < types.length; i++) {
if (!types[i].isPrimitive()) {
final StringBuilder builder = new StringBuilder();
builder.append("argument #").append(i + 1)
.append(" of method ").append(method.getName())
.append("must be of primitive of type");
throw new IllegalArgumentException(builder.toString());
}
}
final SummaryStatistics stat = new SummaryStatistics();
final Object[] parameters = new Object[types.length];
while (true) {
try {
for (int i = 0; i < parameters.length; i++) {
parameters[i] = readAndWritePrimitiveValue(in, out,
types[i]);
}
final double expected = in.readDouble();
if (FastMath.abs(expected) > 1E-16) {
final Object value = method.invoke(null, parameters);
final double actual = ((Double) value).doubleValue();
final double err = FastMath.abs(actual - expected);
final double ulps = err / FastMath.ulp(expected);
out.writeDouble(expected);
out.writeDouble(actual);
out.writeDouble(ulps);
stat.addValue(ulps);
}
} catch (EOFException e) {
break;
}
}
return stat;
}
public static void run(final ApplicationProperties properties)
throws IllegalAccessException, IllegalArgumentException,
InvocationTargetException, IOException {
for (int i = properties.from; i < properties.to; i += properties.by) {
final String inputFileName;
inputFileName = String.format(properties.inputFileMask, i);
final String outputFileName;
outputFileName = String.format(properties.outputFileMask, i);
final DataInputStream in;
in = new DataInputStream(new FileInputStream(inputFileName));
final DataOutputStream out;
out = new DataOutputStream(new FileOutputStream(outputFileName));
final SummaryStatistics stats;
stats = assessAccuracy(properties.method, in, out);
System.out.println("input file name = " + inputFileName);
System.out.println("output file name = " + outputFileName);
System.out.println(stats);
}
}
public static void main(final String[] args)
throws IOException, IllegalAccessException, IllegalArgumentException,
InvocationTargetException {
if (args.length == 0) {
final String msg = "missing required properties file";
throw new IllegalArgumentException(msg);
}
final FileInputStream in = new FileInputStream(args[0]);
final Properties properties = new Properties();
properties.load(in);
in.close();
final ApplicationProperties p;
p = ApplicationProperties.create(properties);
run(p);
}
}

9
src/test/maxima/special/RealFunctionValidation/RealFunctionValidation.sh Executable file
View File

@ -0,0 +1,9 @@
#!/bin/bash
# Location of the Commons-Math3 jar file
CM3_JAR=$HOME/.m2/repository/org/apache/commons/commons-math3/3.1-SNAPSHOT/commons-math3-3.1-SNAPSHOT.jar
# Location of file RealFunctionValidation.jar
APP_JAR=$HOME/Documents/workspace/commons-math3/src/test/maxima/special/RealFunctionValidation/RealFunctionValidation.jar
java -cp $CM3_JAR:$APP_JAR RealFunctionValidation logGamma.properties