Commit Graph

5516 Commits

Author SHA1 Message Date
Charles Allen 55292bba13 Add more IndexMergerTests 2015-05-28 18:18:20 -07:00
Xavier Léauté d834a974ba flag to enable public IP in EC2-VPC autoscaling 2015-05-28 18:14:12 -07:00
Xavier Léauté 2fb2b6ce6b Merge pull request #1397 from druid-io/fix-index-task-slow
Make the index task use indexmerger and not indexmaker
2015-05-28 17:14:48 -07:00
Fangjin Yang 13c402d5d6 Merge pull request #1400 from metamx/groupByCheckForNullDimSpec
Add checkin GroupByQuery for null DimensionSpec in dimension list
2015-05-28 15:48:30 -07:00
Charles Allen 1ebe622c7d Add checkin GroupByQuery for null DimensionSpec in dimension list 2015-05-28 14:55:34 -07:00
fjy bb1145ef56 Make the index task use indexmerger and not indexmaker 2015-05-28 13:34:57 -07:00
Xavier Léauté f588a95f36 Merge pull request #1396 from druid-io/minewhat-master
druid production architecture diagram
2015-05-27 17:08:44 -07:00
janakg 5812b3dd5c druid production architecture diagram 2015-05-27 17:06:24 -07:00
Xavier Léauté fe286d28d1 Merge pull request #1395 from druid-io/new-logs
Additional schema validations and better logs for common extensions
2015-05-27 17:01:13 -07:00
fjy be2a35188e Additional schema validations and better logs for common extensions 2015-05-27 16:25:02 -07:00
Xavier Léauté f9c624c7db Merge pull request #1361 from mrijke/groupby-limithavingorder-unittest
GroupBy Query with Having/Limit/Orderingspec inconsistencies (UnitTest)
2015-05-27 14:49:18 -07:00
Xavier Léauté 1a3f04f0ed Merge pull request #1354 from metamx/multi-valued-dimension-compression
Enabling compression for multiValued dimension
2015-05-26 23:43:53 -07:00
Xavier Léauté 55092d225c Merge pull request #1392 from metamx/fixRoaringConciseExtractionFilter
Fix roaring extraction filter on empty values
2015-05-26 14:33:29 -07:00
Charles Allen fd64c24e43 Fix roaring extraction filter on empty values 2015-05-26 13:54:18 -07:00
nishant 81415282aa Enabling compression for multiValued dimension
Add test and refactoring

Add benchmark tests
2015-05-27 00:09:14 +05:30
Fangjin Yang 600b30566c Merge pull request #1385 from metamx/brokenDimExtractionFilter
Fix for breaking Dimension Extraction Filter on empty filter
2015-05-25 09:51:17 -07:00
Charles Allen e97d22a10a Fix Extraction Filter cast problems for empty results 2015-05-22 15:20:11 -07:00
Charles Allen e1399b7ce4 Add unit test to show breaking Dimension Extraction Filter 2015-05-22 15:02:11 -07:00
Xavier Léauté cd6df1feba Merge pull request #1384 from gbuisson/patch-1
added clj-druid to query libraries (clojure)
2015-05-22 16:16:50 -04:00
cheddar c1b1752595 Merge pull request #1383 from metamx/psql-transient
retry transient exceptions for PostgreSQL, fixes #1382
2015-05-22 13:01:53 -07:00
Xavier Léauté 75c092ccb1 Merge pull request #1375 from metamx/MetricManipulatorFnInstances
Modify MetricManipulatorFns to use instanced classes
2015-05-22 15:56:47 -04:00
Charles Allen 042653ebcb Modify MetricManipulatorFns to use instanced classes 2015-05-22 12:38:38 -07:00
Xavier Léauté 490c97b041 Merge pull request #1381 from himanshug/fix_seg_metadata_regression
Fixing the regression caused by #1172
2015-05-22 15:26:29 -04:00
Xavier Léauté 6b23e02d2b retry transient exceptions for PostgreSQL, fixes #1382 2015-05-22 14:47:27 -04:00
Himanshu Gupta 723df735e9 force eagerness of processing of SegmentMetadata queries on the processing executor by converting the Sequence into List 2015-05-22 13:46:26 -05:00
Himanshu Gupta 5852b64852 adding UT for SegmentMetadata bySegment query which catches following regression caused by commit 55ebf0cfdf
it fails when we issue the SegmentMetadataQuery by setting {"bySegment" : true} in context with exception -
java.lang.ClassCastException: io.druid.query.Result cannot be cast to io.druid.query.metadata.metadata.SegmentAnalysis
at io.druid.query.metadata.SegmentMetadataQueryQueryToolChest$4.compare(SegmentMetadataQueryQueryToolChest.java:222) ~[druid-processing-0.7.3-SNAPSHOT.jar:0.7.3-SNAPSHOT]
at com.google.common.collect.NullsFirstOrdering.compare(NullsFirstOrdering.java:44) ~[guava-16.0.1.jar:?]
at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:46) ~[java-util-0.27.0.jar:?]
at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:42) ~[java-util-0.27.0.jar:?]
at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649) ~[?:1.7.0_80]
2015-05-22 13:45:54 -05:00
Himanshu Gupta da0cc32bc8 Revert commit 55ebf0cfdf
which caused following regression
 it fails when we issue the SegmentMetadataQuery by setting {"bySegment" : true} in context with exception -
java.lang.ClassCastException: io.druid.query.Result cannot be cast to io.druid.query.metadata.metadata.SegmentAnalysis
at io.druid.query.metadata.SegmentMetadataQueryQueryToolChest$4.compare(SegmentMetadataQueryQueryToolChest.java:222) ~[druid-processing-0.7.3-SNAPSHOT.jar:0.7.3-SNAPSHOT]
at com.google.common.collect.NullsFirstOrdering.compare(NullsFirstOrdering.java:44) ~[guava-16.0.1.jar:?]
at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:46) ~[java-util-0.27.0.jar:?]
at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:42) ~[java-util-0.27.0.jar:?]
at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649) ~[?:1.7.0_80]
2015-05-22 13:39:34 -05:00
gbuisson 9e90dc0055 added clj-druid to query libraries (clojure) 2015-05-22 17:49:31 +02:00
Xavier Léauté 4466e77b25 Merge pull request #1371 from guobingkun/unit_test
Unit test for IndexGeneratorJob
2015-05-22 10:34:24 -04:00
Xavier Léauté 5ad5d7d18b Merge pull request #1379 from flowbehappy/fix-hadoop-ha
bug fix: hdfs task log and indexing task not work properly with Hadoop HA
2015-05-22 09:14:50 -04:00
cheddar fae86e83ad Merge pull request #1367 from metamx/hadoopTaskAbstraction
Abstractify HadoopTask
2015-05-21 13:25:51 -07:00
flow 07659f30ab bug fix: hdfs task log and indexing task not work properly with Hadoop HA 2015-05-21 20:49:42 +08:00
Maarten Rijke 82da479464 Fix for GroupBy with Having+Limit+Orderspec
* Inverted function arguments to compose postProcFn for GroupBy queries
    with havingspec + limitspec.
  * Replaced query.getLimitSpec() with null in GroupByQueryToolChest's
    mergeGroupByResults
  * Added unittest to verify functionality
2015-05-19 18:35:48 +02:00
Bingkun Guo b46aff12ae Unit test for IndexGeneratorJob 2015-05-18 12:31:16 -05:00
Xavier Léauté e6d22565ad Merge pull request #1368 from himanshug/fix_seg_metadata_complex_column
in segment metadata query return actual complex data type name
2015-05-15 19:52:16 -07:00
Himanshu Gupta 2fd3e9e8e5 return size = 0 in ColumnAnalysis if its unknown
that is if complex agg did not implement inputSizeFn() so
that segment metadata query shows atleast some information.
also instead of COMPLEX, return type of data stored.
2015-05-15 20:11:56 -05:00
Xavier Léauté 21ba859f33 Merge pull request #1366 from metamx/futurizeSQLMetadataSegmentManager
Properly shutdown SQLMetadataSegmentManager and SQLMetadataRuleManager
2015-05-15 17:33:25 -07:00
Fangjin Yang 1d33f46ece Merge pull request #1373 from metamx/segmentMetadataQueryDocClarification
Modify SegmentMetadataQuery.md to refer to flat data instead of `TSV`
2015-05-15 17:27:26 -06:00
Charles Allen 051c3ccede Add futures to SQLMetadataSegmentManager and SQLMetadataRuleManager 2015-05-15 16:08:22 -07:00
Charles Allen 03df582fe4 Modify SegmentMetadataQuery.md to refer to flat data instead of `TSV` 2015-05-15 13:49:51 -07:00
Xavier Léauté 87924c7c66 Merge pull request #1369 from druid-io/add-toc
Add missing toc entry for data ingestion
2015-05-15 10:18:00 -07:00
fjy 614e79a0da Add missing toc entry 2015-05-15 11:16:51 -06:00
Charles Allen 29ba05c04f Abstractify HadoopTask
* Add `invokeForeignLoader` to commonize the way tasks are attempted to be launched in a foreign class loader
* Add `buildClassLoader` to accomplish the common tasks for hadoop jobs when building a ClassLoader
2015-05-14 17:04:43 -07:00
Xavier Léauté 3c3db7229c Merge pull request #1355 from himanshug/long_max_min_aggregators
Long max/min aggregators
2015-05-13 12:08:11 -07:00
Himanshu Gupta 4b907a93c5 updating the aggregators doc 2015-05-13 09:25:57 -05:00
Himanshu Gupta cebb550796 additional UTs for [DoubleMax/DoubleMin] aggregation 2015-05-13 09:25:41 -05:00
Himanshu Gupta d0ec945129 adding aliases doubleMax and doubleMin for max and min respectively
renamed all [Max/Min]*.java to [DoubleMax/DoubleMin]*.java and created [Max/Min]AggregatorFactory.java which can be removed when we dont need the min/max aggregator type backward compatibility
2015-05-13 09:25:41 -05:00
Himanshu Gupta 2de38f7d29 UTs for long[Max/Min] aggregation 2015-05-13 09:25:22 -05:00
Himanshu Gupta 00436f93e2 long max/min aggregators implementation 2015-05-13 09:25:22 -05:00
Xavier Léauté ad286cc02e Merge pull request #1357 from druid-io/oh8
update pom to 0.8
2015-05-11 22:04:03 -07:00