Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,029 Commits 113 Branches 505 Tags 731 MiB
Commit Graph

657 Commits

Author SHA1 Message Date
Xavier Léauté 603152eb18 fix compilation with Java 8 2014-08-14 11:37:47 -07:00
fjy 18d3acd3a8 fix all manners of brokenness from nulls and empty strings 2014-08-14 11:11:57 -07:00
nishantmonu51 0b42ca4c75 return correct valueType 2014-08-13 18:34:00 +05:30
nishantmonu51 6f60a3f604 make valueType configurable 2014-08-13 14:37:57 +05:30
nishantmonu51 03d7ec04be make tests pass 2014-08-13 10:55:20 +05:30
nishantmonu51 1b0a72751b Add support for LongColumn 2014-08-13 08:52:36 +05:30
fjy 1adec23126 some better null handling for MV 2014-08-12 16:40:02 -07:00
Xavier Léauté b448deeca0 fix compilation with Java 8 2014-08-12 14:02:08 -07:00
Xavier Léauté 161c2f0fff Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	services/pom.xml
 2014-08-12 14:00:21 -07:00
fjy d4014c6a4e Merge pull request #675 from metamx/slimmer-column-cache 
reduce object count overhead of column cache
 2014-08-12 14:16:33 -06:00
fjy 578e0b4fb6 add more logging for groupby caching 2014-08-12 12:35:39 -07:00
nishantmonu51 f36d030ef2 fix NPE while indexing 2014-08-13 00:54:31 +05:30
nishantmonu51 c6712739dc merge changes from druid-0.7.x 2014-08-12 15:47:42 +05:30
Xavier Léauté a1bcff49c5 disable column cache by default 2014-08-11 18:10:33 -07:00
Xavier Léauté 69f4c2677f formatting 2014-08-11 18:10:19 -07:00
Xavier Léauté da399724eb add approximate object overhead 2014-08-11 17:07:10 -07:00
Xavier Léauté aecf4491be reduce object count overhead of column cache 2014-08-11 15:25:14 -07:00
fjy b740987381 Merge pull request #616 from metamx/jisoo 
add retry feature on broker side; modify QueryRunner inferface and tests
 2014-08-11 15:36:16 -06:00
fjy 2ecd55bae5 Merge branch 'druid-0.7.x' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
 2014-08-11 12:47:38 -07:00
fjy 70b86c5551 Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	services/pom.xml
 2014-08-11 12:35:47 -07:00
fjy 8998700760 prepare for next release 2014-08-08 16:19:45 -07:00
Xavier Léauté d198959c60 fix test harness 2014-08-08 14:05:10 -07:00
fjy ab5c529ad6 Merge branch 'druid-0.7.x' into new-init 2014-08-08 13:45:37 -07:00
fjy 4af6a05378 Merge pull request #632 from metamx/lz4-compression 
Change default column compression from LZF to LZ4
 2014-08-08 14:24:00 -06:00
fjy f3970bb1d8 Merge branch 'master' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
 2014-08-08 12:44:29 -07:00
Gian Merlino 889c85c64f Fix OrderByColumnSpec cacheKey. 2014-08-07 06:53:34 -07:00
Xavier Léauté 6fa611c262 refactor FixedSizeCompressedObjectStrategy 2014-08-06 16:13:39 -07:00
Xavier Léauté 991e1828b0 make LZ4 the default compression strategy 
- LZ4 is now hardwired to be the default strategy
- Rework tests to test all available compression strategies
 2014-08-06 15:50:07 -07:00
fjy 7624bb5ea3 Merge branch 'master' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
	processing/src/main/java/io/druid/guice/PropertiesModule.java
	server/src/main/java/io/druid/initialization/Initialization.java
 2014-08-06 15:47:22 -07:00
fjy 4d4c6b891e address cr 2014-08-05 15:58:42 -07:00
fjy d73492bf0f add missing final 2014-08-05 15:51:56 -07:00
fjy 8097e04938 fix naming 2014-08-05 15:48:30 -07:00
fjy 3a78960e65 Fix how metrics are emitted from servermanager 2014-08-05 15:47:30 -07:00
fjy 66225d698f fix stupidness with spatial indexing 2014-08-05 15:07:56 -07:00
jisookim0513 6f58cd1156 fix merge conflcts; change test files 2014-08-04 17:45:56 -07:00
fjy 37f3cb38db fix to string error 2014-07-31 17:23:56 -07:00
fjy a5bc6fc706 fix to string error 2014-07-31 17:08:08 -07:00
nishantmonu51 32b9290723 fix compilation 2014-07-31 16:20:58 +05:30
nishantmonu51 637bd35785 merge changes from druid-0.7.x 2014-07-31 16:07:22 +05:30
xvrl d21e377a20 Merge pull request #651 from metamx/rewrite-async 
rewrite router async logic
 2014-07-29 13:56:01 -07:00
fjy 2ad1bd3f44 add back metrics 2014-07-29 10:22:55 -07:00
Xavier Léauté df7a39a682 fix index out of bounds 2014-07-28 09:59:21 -07:00
fjy 45e443bedb a few slight tweaks and bumping up curator version 2014-07-25 16:07:40 -07:00
fjy 8ef380b3f9 Merge pull request #638 from metamx/rejigger-cache 
Rejiggering the cache
 2014-07-17 15:19:23 -06:00
fjy bc650a1c80 Merge pull request #627 from metamx/druid-firehose 
Functionality to ingest a Druid segment and change the schema
 2014-07-17 13:41:16 -06:00
nishantmonu51 e59c9ebdbc minor fixes 
fix IndexOutOfBoundsException
fix ingestFirehose
 2014-07-17 17:24:57 +05:30
fjy b70a6b1061 address code review comments 2014-07-16 13:52:06 -07:00
fjy 2f90bfcf9a rejiggering the cache for groupBy and select 2014-07-15 12:12:33 -07:00
nishantmonu51 f5f05e3a9b Sync changes from branch new-ingestion PR #599 
Sync and Resolve Conflicts
 2014-07-11 16:15:10 +05:30
jisookim0513 c4397bd65a fixed merge conflicts 2014-07-07 11:26:11 -07:00
Xavier Léauté c40a315c81 initial support for LZ4 compression 2014-07-07 11:01:08 -07:00
fjy d37527ca79 make histos work with groupby maybe 2014-07-01 16:46:58 -07:00
fjy 92bc340099 Merge branch 'master' into more-tests 2014-06-30 18:40:00 -07:00
fjy afa73a2d3a fix one of the TODOs and add test case 2014-06-30 17:29:19 -07:00
Xavier Léauté 1499ee6458 add toString method for JavascriptDimExtractionFn 2014-06-25 10:47:57 -06:00
jisookim0513 cc8a4d0847 revert changes to RetryQueryRunnerConfig 2014-06-20 13:49:33 -07:00
jisookim0513 5e7dfaf298 change retry tests 2014-06-20 13:45:04 -07:00
jisookim0513 3418291333 add a test for retrying multiple times 2014-06-20 13:15:46 -07:00
jisookim0513 1cb3fc250e change missigSegments to a static variable 2014-06-19 18:29:00 -07:00
jisookim0513 0244172ccb change missigSegments to a static variable 2014-06-19 18:22:36 -07:00
jisookim0513 4e2b5b743f add more unit tests 2014-06-19 18:18:58 -07:00
jisookim0513 bdb35e2d7e fix retry logic and change the default value of retry to 0 2014-06-19 17:50:50 -07:00
Xavier Léauté 0602a45200 add column config test (disabled by default) 2014-06-19 14:48:51 -07:00
fjy 4c40e71e54 address cr 2014-06-19 14:48:46 -07:00
fjy a870fe5cbe inject column config 2014-06-19 14:47:57 -07:00
jisookim0513 ff980091c5 delete an unused flag in IncrementalIndexSegment 2014-06-19 14:43:58 -07:00
Xavier Léauté c2e2391e3a fix thread local initial value 2014-06-19 14:43:03 -07:00
Xavier Léauté 09346b0a3c make column cache configurable 2014-06-19 14:43:03 -07:00
Xavier Léauté 8840af5977 make GenericIndexed release cache resources on close 2014-06-19 14:42:08 -07:00
jisookim0513 c4048d06dd delete an unused flag in QueryableIndexSegment 2014-06-19 14:42:04 -07:00
jisookim0513 35e080bbc1 fix concurrency issue with the map; introduce new exception; add incomplete retry query runner test 2014-06-19 14:14:54 -07:00
Xavier Léauté be25d67894 add comments 2014-06-19 14:12:40 -07:00
Xavier Léauté 618afd9976 optimize single value lookup 2014-06-19 14:12:39 -07:00
Xavier Léauté 7a351dfde3 optimize lookups by caching bytebuffer copy 2014-06-19 14:12:39 -07:00
Xavier Léauté be92c322e3 make ObjectStrategy cacheable 2014-06-19 14:12:39 -07:00
jisookim0513 8515a11787 change the signature of run() 2014-06-18 18:05:09 -07:00
jisookim0513 f4b1dc032b change the way to use RetryQueryRunnerConfig 2014-06-18 17:15:04 -07:00
jisookim0513 00856f0fec add retry feature on broker side; modify QueryRunner inferface and tests 2014-06-18 16:55:33 -07:00
fjy 0bc1915067 Merge pull request #578 from metamx/new-guava 
Update guava, java-util, and druid-api
 2014-06-18 14:23:32 -06:00
Xavier Léauté 9b03f5b13a fix timeout test race condition 2014-06-17 09:00:07 -07:00
Gian Merlino 3db07d2a55 Make TimeBoundaryQuery buildResult no longer static. 2014-06-16 16:54:31 -07:00
fjy ce61e60ee8 all manners of new tests 2014-06-16 16:19:26 -07:00
fjy c2149f8a00 add more tests for time boundary and caching 2014-06-16 15:32:28 -07:00
fjy a71f7152a5 fix broken UT 2014-06-16 14:53:26 -07:00
fjy 5aa46d08a8 address cr 2014-06-16 14:24:36 -07:00
fjy 9ca4f564ee address cr 2014-06-16 14:20:16 -07:00
fjy 08c88e8fb7 address cr 2014-06-16 14:07:10 -07:00
fjy d2cf7d3f0a address cr 2014-06-16 14:00:01 -07:00
fjy d4a47fe6e8 enable tb to return just max or min time 2014-06-16 13:45:32 -07:00
nishantmonu51 2d03888a2c Merge branch 'master' into offheap-incremental-index 
Conflicts:
	processing/src/main/java/io/druid/query/groupby/GroupByQueryHelper.java

processing/src/main/java/io/druid/query/groupby/GroupByQueryQueryToolChe
st.java

processing/src/main/java/io/druid/segment/incremental/IncrementalIndex.j
ava
 2014-06-16 20:43:17 +05:30
nishantmonu51 0629be622c remove unnecessary changes & fix index closing subquery 2014-06-16 18:50:49 +05:30
nishantmonu51 a9c09ec8a7 fix compilation 2014-06-16 17:55:43 +05:30
fjy d5d2873d3c Merge pull request #593 from metamx/fix-groupyb 
Support all standard Druid functions in GroupBy and nested groupBy
 2014-06-13 15:37:55 -06:00
Xavier Léauté 93f97a5d86 fix bysegment-query-runner 2014-06-13 13:20:14 -07:00
fjy 8246d42715 Merge branch 'master' into new-guava 2014-06-13 11:44:05 -07:00
Xavier Léauté 294a9ba78f fix same race condition in other test 2014-06-13 11:42:19 -07:00
fjy 332ba3ef20 fix bad merge 2014-06-13 11:36:36 -07:00
fjy 4cc0353be2 Merge branch 'master' of github.com:metamx/druid into new-guava 2014-06-13 11:30:07 -07:00
fjy 5bd4ad3f0f address code review 2014-06-13 11:29:49 -07:00
Xavier Léauté 4c4047165f fix query cancellation test race condition 2014-06-13 11:28:48 -07:00
fjy a63cda3281 Merge branch 'master' into new-guava 
Conflicts:
	server/src/main/java/io/druid/server/QueryResource.java
 2014-06-13 10:08:10 -07:00
nishantmonu51 49d3bd94da Merge branch 'master' into offheap-incremental-index 2014-06-13 15:39:34 +05:30
nishantmonu51 0ddaf4c307 fix race while index closing and querying 2014-06-13 15:36:57 +05:30
Xavier Léauté 8f7c82d351 avoid closing yielder prematurely 2014-06-12 16:00:06 -07:00
Xavier Léauté bfa3caa589 js/cardinality/hyperUnique happy groupBy family 2014-06-12 15:19:01 -07:00
Xavier Léauté fd47efd59e fix cardinality combining factory 2014-06-12 15:19:01 -07:00
Xavier Léauté 150480bdf9 complexmetricserde workaround for groupby 2014-06-12 15:19:01 -07:00
fjy e84bcca40f rename method 2014-06-12 13:36:40 -07:00
fjy 4a4cccf10e Merge branch 'master' into fix-groupyb 
Conflicts:
	processing/src/main/java/io/druid/query/timeseries/TimeseriesQueryEngine.java
 2014-06-12 13:24:16 -07:00
fjy 403962fe0b add some failing tests for groupBy 2014-06-12 11:36:53 -07:00
Xavier Léauté 22e368e79b add dimension/column selector to incremental index 2014-06-12 11:25:22 -07:00
nishantmonu51 025814cfff Merge branch 'master' into offheap-incremental-index 
Conflicts:
	processing/src/main/java/io/druid/query/GroupByParallelQueryRunner.java

processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactor
y.java

processing/src/main/java/io/druid/segment/incremental/IncrementalIndexSt
orageAdapter.java
 2014-06-12 20:02:27 +05:30
nishantmonu51 a7e19ad892 configure buffer sizes 2014-06-12 19:32:37 +05:30
fjy cde86d81e2 make having and limitspec work for nested queries 2014-06-10 15:50:13 -07:00
Xavier Léauté a541029b41 segmentMetadata query cancellation 2014-06-10 11:51:07 -07:00
Xavier Léauté 6550cb1776 groupBy query cancellation 2014-06-10 11:51:06 -07:00
Xavier Léauté 8f7fd93491 add comments 2014-06-10 10:36:39 -07:00
Xavier Léauté b84884ab76 remove methods used only for testing 2014-06-10 10:36:39 -07:00
Xavier Léauté c08002aa4d interrupt queries on incremental indexer 2014-06-10 10:36:39 -07:00
Xavier Léauté 99c9a2cf05 make sure to close yielder in MetricsEmittingQueryRunner 2014-06-10 10:36:39 -07:00
Xavier Léauté 97d5455f3a properly kill timed out queries 2014-06-10 10:36:38 -07:00
Xavier Léauté d0f9c438f8 proper query cancellation tests 2014-06-10 10:36:38 -07:00
Xavier Léauté 855c66c9ad less stack traces when cancelling queries 2014-06-10 10:36:38 -07:00
Xavier Léauté d2c729adec return proper json errors to propagate query cancellation and timeout 2014-06-10 10:36:38 -07:00
Xavier Léauté a56a655eae proper query exceptions and add support for query timeout 2014-06-10 10:36:38 -07:00
Xavier Léauté 1183c68ce4 formatting 2014-06-10 10:36:38 -07:00
Xavier Léauté 4f1e157639 pass through errors from computes in DirectDruidClient 2014-06-10 10:36:38 -07:00
Xavier Léauté 32f6243be0 proper closing of resources in case of query cancellation 2014-06-10 10:36:38 -07:00
Xavier Léauté 1be85af320 handle query interruption at cursor level 2014-06-10 10:36:37 -07:00
Xavier Léauté d6f38827db initial query cancellation commit 2014-06-10 10:36:28 -07:00
fjy 5955ecf032 fix bugs wtih js and card aggs 2014-06-09 22:31:30 -07:00
fjy 187b2beb47 cleaner way of implementing things 2014-06-09 22:27:33 -07:00
fjy 32b65d9c5b remove commented out code 2014-06-09 18:26:37 -07:00
fjy 6adc3011fa fix post aggs with group by in #472 2014-06-09 18:25:10 -07:00
nishantmonu51 bf09f979bd fix segment metadataquery 2014-06-05 20:35:57 +05:30
nishantmonu51 5bdc4a761a close IncrementalIndex properly and free up buffer 2014-06-05 19:41:43 +05:30
nishantmonu51 01e8a713b6 unit tests passing with offheap-indexing 2014-06-05 17:42:53 +05:30
fjy 77ec4df797 update guava, java-util, and druid-api 2014-06-03 13:43:38 -07:00
fjy 7be93a770a make all firehoses work with tasks, add a lot more documentation about configuration 2014-05-28 16:33:59 -07:00
fjy c56f3fd1af Merge pull request #565 from metamx/add-hll-test 
test for correct HLL bytebuffer folding
 2014-05-28 12:12:34 -06:00
xvrl 18685542ec Merge pull request #568 from metamx/fix-topn 
Fix inverted lexicographic topns and topns with dim extraction dimension specs
 2014-05-28 10:47:10 -07:00
fjy 642897dca3 fix inverted logic 2014-05-28 10:43:03 -07:00
fjy 2ee1defce9 better handle dim extraction regex cases 2014-05-28 10:27:00 -07:00
fjy 8bcf4bc60e address code review comments 2014-05-27 19:47:13 -07:00
fjy 9f62589d71 fix bug with previousStop not being respected for inverse lexi topN 2014-05-27 19:39:55 -07:00
fjy b9015cee34 fix brokeness with dim extraction dim specs 2014-05-27 17:25:00 -07:00
fjy 07a95a0e05 Fix inverted lexicographic topNs 2014-05-27 17:02:28 -07:00
Xavier Léauté d423ba1b27 minor optimizations 2014-05-27 11:03:12 -07:00
Xavier Léauté 5844c45956 fix issue when offset changes folding maxOverflow 2014-05-27 11:03:12 -07:00
Xavier Léauté c273298ad4 don't discard current maxOverflow if offset is high enough 2014-05-27 11:03:12 -07:00
Xavier Léauté d96d82252d test fold permutations with various maxOverflow / offsets 2014-05-27 11:03:12 -07:00
Xavier Léauté df3fb07e51 test for correct bytebuffer folding 2014-05-22 17:48:19 -07:00
fjy 2906544c30 add a context to force 2 pass algorithm for topNs 2014-05-20 10:42:37 -07:00
Xavier Léauté 9ec7c71e0f fix compilation error with updated druid-api 2014-05-19 14:06:23 -07:00
fjy 77289a4148 Merge pull request #531 from metamx/listenable-prioritized-executorservice 
make prioritized executor service listenable
 2014-05-16 16:59:04 -06:00
Xavier Léauté 3223d46c1a simplify chained sequence operations 2014-05-16 13:49:40 -07:00
Xavier Léauté 07b2a0efa6 replace iterables with sequences, fixes #128 2014-05-16 13:22:53 -07:00
Xavier Léauté d8282d0126 add JavascriptAggregatorBenchmark 2014-05-16 10:41:54 -07:00
Gian Merlino 4fbd6c7c36 JavaScriptAggregator comment explaining why primitive wrapping is anti-helpful. 2014-05-16 09:47:59 -07:00
Gian Merlino 918ee8741a More JS aggregator tests. 2014-05-15 23:08:56 -07:00
Gian Merlino d389fbc961 More consistent ObjectColumnSelectors. JS aggregator tests for nulls. 2014-05-15 23:02:44 -07:00
Gian Merlino cb028f490a JavaScriptAggregator: Context.newArray makes nicer arrays than Context.javaToJS. 2014-05-15 20:02:36 -07:00
Gian Merlino c2137e8416 ObjectColumnSelector/JS aggregator support for multi-valued columns. 2014-05-15 19:34:22 -07:00
nishantmonu51 ce3d461e7d add header 2014-05-16 04:30:58 +05:30
nishantmonu51 e06b29edb9 add sanity unit tests 2014-05-16 03:15:35 +05:30
nishantmonu51 ddc36c89cc union query fix and a broken test 2014-05-15 23:10:40 +05:30
Xavier Léauté 0912a2a09a use Guava's ForwardingExecutorService instead of DelegatingExecutorService 2014-05-09 23:24:45 -07:00
Xavier Léauté 4280b4e8fa make MetricsEmittingExecutorService listenable 2014-05-09 23:24:38 -07:00
fjy 2709a8b0b6 revert changes to hyperUnique 2014-05-07 12:33:42 -07:00
fjy 9d35490cdf Revert "Merge pull request #495 from metamx/dim-extraction-null-values" 
This reverts commit 76437379d7, reversing
changes made to 79e6d4eb56.

Conflicts:
	pom.xml
 2014-05-07 12:32:51 -07:00
Xavier Léauté 4fa6de4c6c support null dimension values in HyperUniquesSerde 2014-05-07 11:07:48 -07:00
fjy b6fb4245aa Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDriverConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfigBuilder.java
	pom.xml
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/firehose/EventReceiverFirehoseFactory.java
 2014-05-06 14:32:51 -07:00
fjy 76437379d7 Merge pull request #495 from metamx/dim-extraction-null-values 
Support null values in DimExtractionFn in topN and groupBy
 2014-05-06 14:17:55 -06:00
fjy 79e6d4eb56 Merge pull request #528 from metamx/union-query-source 
Union query source
 2014-05-06 14:13:33 -06:00
nishantmonu51 66450ca6ed review comments 
remove toShortString
minor other changes
 2014-05-07 00:54:17 +05:30
fjy a747ed0011 fix broken UT 2014-05-05 16:13:01 -07:00
fjy b1bd6eb0f2 Merge pull request #512 from metamx/cardinality 
Cardinality aggregator
 2014-05-05 17:10:24 -06:00
fjy 8af927e2a7 Merge pull request #516 from metamx/alphanumeric-topn 
Alpha-numeric sorting for topN
 2014-05-05 17:09:24 -06:00
Xavier Léauté ea318f01ee allow forcing all tasks to be prioritized 2014-05-05 11:42:30 -07:00
nishantmonu51 f2725994b7 refactor method 2014-05-05 19:57:32 +05:30
nishantmonu51 477e01daf7 review comments 2014-05-02 15:21:06 +05:30
Xavier Léauté dab30395d5 make prioritized executor service listenable 2014-05-01 15:51:59 -07:00
nishantmonu51 1b6137504c Add toString for better logging 2014-05-01 21:32:27 +05:30
nishantmonu51 92ebe69fd9 remove unwanted loggings 2014-05-01 21:16:42 +05:30
nishantmonu51 1d67e39949 Fixes 
fix NPE in select query, calculating cache key
remove unwanted logging
fix ds in topNQuery
 2014-05-01 21:04:47 +05:30
nishantmonu51 728a606d32 Add support for union queries 2014-05-01 01:54:52 +05:30
Xavier Léauté bb62a44e24 alphanumeric topN 2014-04-28 09:42:43 -07:00
fjy e30accbbf5 reducing bloat 2014-04-25 15:25:54 -07:00
Xavier Léauté 5ae848ab14 fix formatting 2014-04-25 15:09:47 -07:00
Xavier Léauté c1e6f6d83e formatting changes 2014-04-25 14:54:17 -07:00
Xavier Léauté de5b1749bc minor changes to address code review 2014-04-25 14:54:17 -07:00
fjy 76e0a48527 Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DbUpdaterJob.java
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/io/druid/indexing/common/task/HadoopIndexTask.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumber.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-04-25 14:03:28 -07:00
Xavier Léauté 3c9b8dc999 cardinality aggregator 2014-04-24 22:08:25 -07:00
fjy c4cefb667e prepare for next release 2014-04-24 11:45:25 -07:00
fjy 90e600bad6 Merge pull request #511 from metamx/fix-hll-buffer-and-swap 
HLL: avoid direct buffers + fix swap target buffer
 2014-04-24 12:03:54 -06:00
Xavier Léauté 30d1a5db75 HLL: avoid direct buffers + fix swap target buffer 2014-04-24 10:49:53 -07:00
fjy f1d705cb73 proper toStrings of filters 2014-04-22 14:46:19 -07:00
fjy fcc1366712 remove test main 2014-04-22 12:54:16 -07:00
fjy 8fd39c63d5 Better handle null adapters and NPEs in the CQE 2014-04-22 12:47:53 -07:00
xvrl d17abedae3 Merge pull request #499 from metamx/fix-pull-from-cache 
Fix pull from cache
 2014-04-22 11:19:57 -07:00
Xavier Léauté 3bac366013 handle NaN / Infinity values 2014-04-22 10:42:44 -07:00
nishantmonu51 4721ba3f1e add test + small fix for other test 2014-04-22 23:03:48 +05:30
nishantmonu51 65c71a40bb fix more issues and add test 
1) Add test for caching and calculation of dependent post-aggs for
TimeSeries and TopN
2) Fix more NPEs while calculating dependent postAggs, passing
baseHolder was wrong as it does not contain post-aggs during computing
dependent ones.
 2014-04-22 22:05:17 +05:30
nishantmonu51 3fb42251d9 calculate postAggs while pulling from cache 2014-04-22 11:25:21 +05:30
nishantmonu51 7bafb71841 Revert "fix npe in topNBinaryFn with post aggs" 
This reverts commit bbedde3418.
 2014-04-22 11:10:55 +05:30
fjy c44cdba65e Merge pull request #496 from metamx/fix-topN-postagg-npe 
fix npe in topNBinaryFn with post aggs
 2014-04-21 23:22:07 -06:00
Xavier Léauté bbedde3418 fix npe in topNBinaryFn with post aggs 2014-04-21 22:20:41 -07:00
fjy 07aab28325 clean up topnbinaryfn 2014-04-21 20:30:18 -07:00
Xavier Léauté 48590862cb support for null values in group-by 2014-04-21 17:58:44 -07:00
Xavier Léauté 44bd4339b5 support for null values in DimExtractionFn 2014-04-21 15:16:56 -07:00
nishantmonu51 f20d3b2895 merge changes from master 2014-04-21 21:43:44 +05:30
nishantmonu51 e8b993b765 fix by segment results 
1) fix class cast in by segment results
2) exclude old netty version coming from curator
 2014-04-21 21:26:34 +05:30
fjy e3fbbff237 clean up code 2014-04-18 17:41:47 -07:00
Xavier Léauté a0c8d9d413 restore previous behavior of not optimizing Timeseries post-aggregators 2014-04-18 16:45:06 -07:00
Xavier Léauté 597e55ab50 don't override context when not finalizing 2014-04-18 16:26:21 -07:00
nishantmonu51 68e8cda2da fix post aggregator 2014-04-19 04:24:07 +05:30
fjy c00fb1d08e Merge pull request #445 from metamx/hadoop-version-update 
Hadoop version update
 2014-04-17 19:16:44 -06:00
fjy 399efe7dc4 make uniques work for groupby 2014-04-17 14:41:17 -07:00
nishantmonu51 2b5fec1dcf review comments 2014-04-16 17:12:27 +05:30
nishantmonu51 6da2272076 fix NPE in aggregatrFirstTopnAlgo 2014-04-16 09:12:34 +05:30
fjy 228be3acb0 Merge pull request #467 from metamx/optimize-postAgg-calculation 
Optimize post agg calculation
 2014-04-15 15:18:01 -06:00
nishantmonu51 36f6f8ab5a Add testOutOfOrderPruneDependentPostAgg 2014-04-16 00:49:56 +05:30
nishantmonu51 6217c67027 complete javadoc, add comment, use Lists.reverse 2014-04-15 22:55:40 +05:30
nishantmonu51 3eb8160b51 review comments 2014-04-15 21:06:30 +05:30
nishantmonu51 d66ae5ac10 fix dependent PostAggregators 2014-04-15 13:37:31 +05:30
fjy a46c4592b4 a few cleanups 2014-04-14 09:53:29 -07:00
fjy 7a8b9c2160 fix select serde 2014-04-10 15:35:55 -07:00
Xavier Léauté be30d450d7 support dimension extraction functions in group by 2014-04-09 17:17:01 -07:00
nishantmonu51 ecfa6bd1b1 fix casing and add comment 2014-04-08 17:26:51 +05:30
nishantmonu51 22c3296aa3 Merge branch 'master' into optimize-postAgg-calculation 
Conflicts:
	server/src/test/java/io/druid/client/CachingClusteredClientTest.java
 2014-04-08 03:35:56 +05:30
nishantmonu51 c322f44863 time series post Aggregation improved 
move post aggregation to finalise results
 2014-04-08 03:17:15 +05:30
fjy 6a26f7684f fix context on CQE 2014-04-07 14:36:13 -07:00
nishantmonu51 4bb36dd453 fix tests 2014-04-08 02:37:57 +05:30
fjy bae0fdf936 add context serde tests and deprecate backwards compatible methods 2014-04-07 13:18:55 -07:00
fjy 93349b5d07 fix broken unit tests 2014-04-07 10:56:38 -07:00
nishantmonu51 db35009acd Compute postage only when required 2014-04-07 23:17:25 +05:30
fjy 98c3faa767 cleanup 2014-04-06 09:23:59 -07:00
fjy 1267fbb7f5 fix context to be backwards compat 2014-04-06 09:20:58 -07:00
Hagen Rother e0e43deaec fix a NPE leading to broker 500 response 2014-04-04 13:07:52 +02:00
fjy 9ebdf16ed6 make last commit a final var 2014-03-27 17:56:25 -07:00
fjy cb1798c520 fix bug with reference closing that I swore we fixed 2014-03-27 17:55:03 -07:00
Xavier Léauté 0be29df8ae make hll tests deterministic 2014-03-27 16:15:32 -07:00
Xavier Léauté 5f0ada925d add HLL maxOverflow tests 2014-03-27 16:01:40 -07:00
Xavier Léauté 03c51518bc don't assume bytebuffer limit is set 2014-03-27 15:58:34 -07:00
Xavier Léauté 11f08b3895 benchmark more closely matches actual implementation 2014-03-26 22:18:26 -07:00
Xavier Léauté c92fd6f45d folding into direct buffers is faster 2014-03-26 17:13:41 -07:00
Xavier Léauté 83ac1d323e one more final variable for good measure 2014-03-26 16:50:24 -07:00
Xavier Léauté 2580a9c7f1 mark long-running tests as @Ignore + docs 2014-03-26 16:49:55 -07:00
Xavier Léauté f42b850d60 clean up + header 2014-03-26 09:31:45 -07:00
nishantmonu51 a9a6682a0e Upgrade to Hadoop 2.3.0 2014-03-26 11:43:31 +05:30
Xavier Léauté 178a9aed40 make it clear mergeAndStoreByteRegister has no side effects 2014-03-25 17:07:09 -07:00
Xavier Léauté a0c38f574b check should be safe to move out of loop 2014-03-25 17:07:09 -07:00
Xavier Léauté 46a5409207 minor hll optimizations 2014-03-25 17:07:09 -07:00
Xavier Léauté 5eaadfffa9 no need for volatile references 
we make no guarantees wrt threadsafety in this class
 2014-03-25 17:07:09 -07:00
Xavier Léauté 7dc0e9f1a9 remove HLL version specific code outside of HyperLogLogCollector 2014-03-25 17:07:09 -07:00
Xavier Léauté cd340aa314 hll benchmark 2014-03-25 17:07:09 -07:00
fjy e832dc7a8c Merge branch 'master' of github.com:metamx/druid into better-context 2014-03-21 10:46:36 -07:00
nishantmonu51 dc0e7f404a fix serialisation of segmentMetaDataQuery 2014-03-21 14:11:26 +05:30
fjy 7f8211ecc7 extend context to use map string object instead of map string string 2014-03-20 13:41:45 -07:00
nishantmonu51 32ad26210d Fix backwards compatibilty constant post aggregator 2014-03-18 20:34:49 +05:30
fjy 1d1ec1d855 clean up some defaults and prepare for next release 2014-03-17 21:41:34 -07:00
Xavier Léauté 7e97073aeb Set queryId in MetricsEmittingQueryRunner 
- queryId is now user8 instead of user10, since it was already used
before
 2014-03-17 13:25:47 -07:00
fjy 2adcf07f5f Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DetermineHashedPartitionsJob.java
	indexing-service/src/main/java/io/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/test/java/io/druid/indexing/common/task/TaskSerdeTest.java
	processing/src/test/java/io/druid/segment/TestIndex.java
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-03-17 10:59:31 -07:00
fjy f525361c3d fix broken tests and prepare for next release 2014-03-14 15:05:07 -07:00
fjy f0fdfbb2f7 fix broken constant post agg 2014-03-14 14:54:52 -07:00
fjy f00ffe4789 fix broken tests 2014-03-13 11:33:34 -07:00
Xavier Léauté de55098898 add comment 2014-03-12 10:47:23 -07:00
Xavier Léauté 2ef710191f make sure constants are not null 2014-03-12 10:40:02 -07:00
fjy 42618e6fcf a collection of small bug fixes 2014-03-12 10:34:24 -07:00
fjy ae2dc6506c fix broken granularity 2014-03-10 15:03:43 -07:00
fjy 36b54f7e8f fix all problems with topNs 2014-03-10 13:22:47 -07:00
Xavier Léauté b81bd6c959 add test for single value index top-n 2014-03-10 13:10:45 -07:00
fjy 6ce5266aca make stuff work 2014-03-10 12:19:58 -07:00
fjy 7b0d2f45bc Merge branch 'fix-iisa' of github.com:metamx/druid into fix-iisa 2014-03-10 12:02:03 -07:00
fjy 2db73998b9 add fix for II 2014-03-10 12:01:49 -07:00
Xavier Léauté adf60d1548 test reset sanity on out of order values 2014-03-10 11:54:07 -07:00
Yuval Oren 995002fcb3 Fix legacy data source serialization 2014-03-06 18:05:24 -08:00
Yuval Oren 0b0648266c Fix DataSource serialization 2014-03-06 15:28:20 -08:00
Yuval Oren 5601c51ae6 finalize variables and optimize imports 2014-03-06 15:03:43 -08:00
Yuval Oren be1ef3a161 Cleaned up equals() and hashCode(), per code review 2014-03-06 14:35:56 -08:00
Yuval Oren 4b63645802 Merge branch 'master' into subquery 
Conflicts:
	processing/src/main/java/io/druid/query/BaseQuery.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryQueryToolChest.java
	processing/src/test/java/io/druid/query/QueryRunnerTestHelper.java
	processing/src/test/java/io/druid/query/topn/TopNQueryRunnerTest.java
 2014-03-06 13:33:59 -08:00
fjy 5826d9594c add headers 2014-03-05 14:29:00 -08:00
fjy 08138688e4 port hyperunique to open source 2014-03-05 14:19:38 -08:00
fjy 0e8e527662 Merge branch 'master' into new-schema 2014-03-04 14:42:16 -08:00
fjy 5aba6141d8 Merge pull request #414 from metamx/incremental-index-synchronization-fix 
Add test and fix one more issue caught by test
 2014-03-04 15:41:26 -07:00
nishantmonu51 4f06dd5a7e Add test and fix one more issue caught by test 2014-03-05 01:09:33 +05:30
fjy 0feddc3831 Merge pull request #411 from metamx/small-fixes-docs-searchquery-limit 
Small fixes
 2014-03-04 10:30:27 -07:00
fjy 4983210907 Merge pull request #412 from pgkiran/filepeon-cleanup 
Temp filePeon files cleanup
 2014-03-04 10:29:52 -07:00
nishantmonu51 eb7bc28f41 fix synchronization 
getDimvalues can also add to the dimension, protect it properly
 2014-03-04 11:58:43 +05:30
fjy c4c4d80336 make local testing pass 2014-03-03 14:52:43 -08:00
fjy 46b9ac78e7 Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	pom.xml
	publications/whitepaper/druid.pdf
	publications/whitepaper/druid.tex
 2014-03-03 14:48:15 -08:00
Kiran Patchigolla 2dea60251b Temp filePeon files cleanup 
Related to https://groups.google.com/forum/#!topic/druid-development/TFUNas5ubyc
cleanup() method is added but still some files are not cleaned up. Additionally setting deleteOnExit on these temp files
 2014-03-03 13:33:47 -08:00
nishantmonu51 eedd6a1b9e fix #362 
fix #362
 2014-02-28 12:38:39 +05:30
nishantmonu51 0701b1534d review comments queryID changes, add wait time for realtime node queries 2014-02-27 14:39:25 +05:30
nishantmonu51 b480d8f543 review comments 2014-02-25 03:09:44 +05:30
nishantmonu51 373501a06d add queryID to query context and request_logs 2014-02-24 16:11:33 +05:30
fjy 7b0b90a860 Merge branch 'master' of github.com:metamx/druid into new-schema 2014-02-20 16:12:31 -08:00
fjy ab83cba790 Merge pull request #393 from metamx/group-by-improvements 
Group by memory usage improvements
 2014-02-19 19:52:37 -07:00
fjy 20cac8c506 not compiling yet but close 2014-02-19 15:54:27 -08:00
nishantmonu51 5be5cadd7b minor refactoring 2014-02-19 21:55:18 +05:30
fjy 4b7c76762d unit tests passingn at this point, finished rt port maybe 2014-02-18 15:14:38 -08:00
fjy 1a8f9f2e12 prepare for next release 2014-02-14 15:07:44 -08:00
fjy ef648b85f5 fix broken priority queries 2014-02-14 15:05:30 -08:00
fjy a8c4362d72 rejiggering druid api 2014-02-14 12:57:52 -08:00
nishantmonu51 74966eb2ef add copyright 2014-02-12 01:00:38 +05:30
nishantmonu51 3302cff6db groupBy improvements 2014-02-12 00:51:10 +05:30
nishantmonu51 533a263fbd initial working version 2014-02-10 22:32:26 +05:30
Yuval Oren bd11309593 Merge branch 'master' into subquery 2014-02-06 15:36:29 -08:00
fjy 0f6af72ea4 Merge branch 'master' into new-schema 2014-02-06 12:46:13 -08:00
fjy 9756b18f9b sort complex metrics with groupby 2014-02-06 11:26:05 -08:00
Yuval Oren 98956cdadb Extra null value check. Not sure why this is needed in the subquery implementation and not regular queries. Bug elsewhere? 2014-02-03 15:53:10 -08:00
fjy 14d0e54327 first commit 2014-02-03 14:15:03 -08:00
Yuval Oren c88cb2af16 Fix NPE when subquery returns empty result 2014-01-24 15:58:47 -08:00
Yuval Oren f2c1f798b9 Fix subquery merge issues 2014-01-24 15:03:48 -08:00
Yuval Oren cd719eab3c Merge branch 'master' into subquery 2014-01-24 14:42:41 -08:00
fjy c93f896092 Merge branch 'master' into select 
Conflicts:
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-01-20 13:26:51 -08:00
Yuval Oren 4c8495662f Subquery support for GroupBy queries 2014-01-15 15:38:57 -08:00
Xavier Léauté 022423e200 fix column closing 2014-01-14 15:01:46 -08:00
fjy 2aabc6c4e6 Merge branch 'master' into select 
Conflicts:
	processing/src/main/java/io/druid/query/Query.java
	server/src/main/java/io/druid/guice/QueryRunnerFactoryModule.java
	server/src/main/java/io/druid/guice/QueryToolChestModule.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-01-08 16:31:19 -08:00
fjy 0301dde671 remove unnecessary doc line 2014-01-08 15:56:29 -08:00
fjy 50a6839693 inital commit of topNs 2014-01-08 15:51:03 -08:00
Gian Merlino 3b2833d55d JavaScriptAggregatorFactory: Handle missing columns by passing down null args 2013-12-16 21:05:14 -08:00
fjy 5ecd909f7c reduce NPEs in CQE 2013-12-16 14:27:01 -08:00
Xavier Léauté bb69f8adcd fix JS dim extraction null handling + test case 2013-12-09 18:36:14 -08:00
fjy 1fb5fc6707 tests 2013-12-09 14:59:44 -08:00
fjy 414c17c8b2 dont need to allocate so much mem 2013-12-06 16:13:30 -08:00
fjy 5ba05a46f6 fix nulls in realtime persist 2013-12-05 17:59:02 -08:00
fjy b78919c729 fix NPEs 2013-12-05 16:49:28 -08:00
fjy ae5a4ff54c add versions to incremental index segment 2013-12-05 16:47:28 -08:00
fjy 241c80ef24 working pagination maybe 2013-12-05 13:37:44 -08:00
fjy 31c7ed639d first commit 2013-12-04 13:53:08 -08:00
fjy de0d6e2264 Merge pull request #308 from metamx/javascript-dimextractionfn 
Javascript DimExtractionFn
 2013-11-27 11:47:07 -08:00
Xavier Léauté 5c77884684 Address code review 
- more tests
- fix license
- add missing getter
 2013-11-25 17:14:10 -08:00
Xavier Léauté 51c0393455 typo 2013-11-22 16:33:16 -08:00
Xavier Léauté 41df3e1f11 implement javascript dimextractionfn 2013-11-22 16:33:16 -08:00
Xavier Léauté d0fe70a21f replace deprecated calls to getJsonFactory and createJsonParser 2013-11-20 17:14:23 -08:00
fjy 0587a9ecd0 Merge pull request #289 from alberts/master 
Fix compile error in Eclipse 4.3.
 2013-11-13 13:15:55 -08:00
Xavier Léauté 66181be1c6 make things final and fix copy-paste errors 2013-11-07 15:25:47 -08:00
Xavier Léauté 074589ffce JavaScript post-aggregator 2013-11-07 15:07:14 -08:00
Albert Strasheim 1589909b9d Fix compile error in Eclipse 4.3. 
The method dimEquals(String, String) is ambiguous for the type DimFilters.
 2013-11-04 21:23:13 -08:00
Xavier Léauté a86b02635a remove DimensionSelectorFactory 2013-10-23 22:02:06 -07:00
daggerrz 522feb65d5 Use druid-api 0.1.2-SNAPSHOT which provides Row.getRaw() 2013-10-20 22:39:13 -04:00
cheddar c47fe202c7 Fix HadoopDruidIndexer to work with the new way of things 
There are multiple and sundry changes in here.

First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.

In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module.  I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.

In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api.  They are all API-level things, however, so they really belong in druid-api instead.

Lastly, I removed the druid-realtime module and put it all in druid-server.
 2013-10-09 15:15:44 -05:00
fjy ac330f72bb first set of changes to standarize the naming convention we use in druid 2013-10-03 16:36:48 -07:00
fjy bc8db7daa5 1) make chat handler resource work again 
2) add more default configs
3) make examples work again
 2013-10-02 14:22:39 -07:00
fjy ed9e0cf9f6 add a local firehose for indexing local files 2013-09-30 16:03:26 -07:00
fjy a79ad7bab4 make dynamic master resource configuration work again 2013-09-27 15:00:40 -07:00
cheddar 46631bf409 Port fix for issue where IncrementalIndex doesn't properly filter for "null" values from 0.5.x to master 2013-09-18 18:18:53 -05:00
cheddar e019dddaaa Reapply Fix for NoSuchElementException that happens in IncrementalIndex when some columns are null 2013-09-18 17:54:49 -05:00
cheddar dc728b5721 Update versions of poms and make IncrementalIndexStorageAdapterTest compile 2013-09-18 17:01:03 -05:00