Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Make Doubles aggregators use 64bits by default (#5478) 

* use 64-bit float representation for double based aggregator

Change-Id: Ia4f442037052add178f6ac68138c9d52f96c6e09

* review comments

Change-Id: I5a588f7364f236bf22f2b138e9d743bfb27c67fe
This commit is contained in:
Slim 2018-03-19 19:13:04 -07:00 committed by GitHub
parent b22455b924
commit 17c71a2a60
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 9 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
docs/content/configuration/index.md
View File

@ -417,11 +417,13 @@ JavaScript-based functionality is disabled by default. Please refer to the Druid
### Double Column storage ### Double Column storage
Druid's storage layer uses a 32-bit float representation to store columns created by the Prior to version 0.13.0 Druid's storage layer used a 32-bit float representation to store columns created by the
doubleSum, doubleMin, and doubleMax aggregators at indexing time. To instead use 64-bit floats doubleSum, doubleMin, and doubleMax aggregators at indexing time.
for these columns, please set the system-wide property `druid.indexing.doubleStorage=double`. Starting from version 0.13.0 the default will be 64-bit floats for Double columns.
This will become the default behavior in a future version of Druid. Using 64-bit representation for double column will lead to avoid precesion loss at the cost of doubling the storage size of such columns.
To keep the old format set the system-wide property `druid.indexing.doubleStorage=float`.
You can also use floatSum, floatMin and floatMax to use 32-bit float representation.
Support for 64-bit floating point columns was released in Druid 0.11.0, so if you use this feature then older versions of Druid will not be able to read your data segments.
|Property|Description|Default| |Property|Description|Default|
|--------|-----------|-------| |--------|-----------|-------|
|`druid.indexing.doubleStorage`|Set to "double" to use 64-bit double representation for double columns.|float| |`druid.indexing.doubleStorage`|Set to "float" to use 32-bit double representation for double columns.|double|

2
processing/src/main/java/io/druid/segment/column/Column.java
View File

@ -30,7 +30,7 @@ public interface Column
static boolean storeDoubleAsFloat() static boolean storeDoubleAsFloat()
{ {
String value = System.getProperty(DOUBLE_STORAGE_TYPE_PROPERTY, "float"); String value = System.getProperty(DOUBLE_STORAGE_TYPE_PROPERTY, "double");
return !StringUtils.toLowerCase(value).equals("double"); return !StringUtils.toLowerCase(value).equals("double");
} }