Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,093 Commits 113 Branches 504 Tags 724 MiB
Commit Graph

667 Commits

Author SHA1 Message Date
Xavier Léauté de5b1749bc minor changes to address code review 2014-04-25 14:54:17 -07:00
fjy 76e0a48527 Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DbUpdaterJob.java
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/io/druid/indexing/common/task/HadoopIndexTask.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumber.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-04-25 14:03:28 -07:00
Xavier Léauté 3c9b8dc999 cardinality aggregator 2014-04-24 22:08:25 -07:00
fjy c4cefb667e prepare for next release 2014-04-24 11:45:25 -07:00
fjy 90e600bad6 Merge pull request #511 from metamx/fix-hll-buffer-and-swap 
HLL: avoid direct buffers + fix swap target buffer
 2014-04-24 12:03:54 -06:00
Xavier Léauté 30d1a5db75 HLL: avoid direct buffers + fix swap target buffer 2014-04-24 10:49:53 -07:00
fjy f1d705cb73 proper toStrings of filters 2014-04-22 14:46:19 -07:00
fjy fcc1366712 remove test main 2014-04-22 12:54:16 -07:00
fjy 8fd39c63d5 Better handle null adapters and NPEs in the CQE 2014-04-22 12:47:53 -07:00
xvrl d17abedae3 Merge pull request #499 from metamx/fix-pull-from-cache 
Fix pull from cache
 2014-04-22 11:19:57 -07:00
Xavier Léauté 3bac366013 handle NaN / Infinity values 2014-04-22 10:42:44 -07:00
nishantmonu51 4721ba3f1e add test + small fix for other test 2014-04-22 23:03:48 +05:30
nishantmonu51 65c71a40bb fix more issues and add test 
1) Add test for caching and calculation of dependent post-aggs for
TimeSeries and TopN
2) Fix more NPEs while calculating dependent postAggs, passing
baseHolder was wrong as it does not contain post-aggs during computing
dependent ones.
 2014-04-22 22:05:17 +05:30
nishantmonu51 3fb42251d9 calculate postAggs while pulling from cache 2014-04-22 11:25:21 +05:30
nishantmonu51 7bafb71841 Revert "fix npe in topNBinaryFn with post aggs" 
This reverts commit bbedde3418.
 2014-04-22 11:10:55 +05:30
fjy c44cdba65e Merge pull request #496 from metamx/fix-topN-postagg-npe 
fix npe in topNBinaryFn with post aggs
 2014-04-21 23:22:07 -06:00
Xavier Léauté bbedde3418 fix npe in topNBinaryFn with post aggs 2014-04-21 22:20:41 -07:00
fjy 07aab28325 clean up topnbinaryfn 2014-04-21 20:30:18 -07:00
Xavier Léauté 48590862cb support for null values in group-by 2014-04-21 17:58:44 -07:00
Xavier Léauté 44bd4339b5 support for null values in DimExtractionFn 2014-04-21 15:16:56 -07:00
nishantmonu51 f20d3b2895 merge changes from master 2014-04-21 21:43:44 +05:30
nishantmonu51 e8b993b765 fix by segment results 
1) fix class cast in by segment results
2) exclude old netty version coming from curator
 2014-04-21 21:26:34 +05:30
fjy e3fbbff237 clean up code 2014-04-18 17:41:47 -07:00
Xavier Léauté a0c8d9d413 restore previous behavior of not optimizing Timeseries post-aggregators 2014-04-18 16:45:06 -07:00
Xavier Léauté 597e55ab50 don't override context when not finalizing 2014-04-18 16:26:21 -07:00
nishantmonu51 68e8cda2da fix post aggregator 2014-04-19 04:24:07 +05:30
fjy c00fb1d08e Merge pull request #445 from metamx/hadoop-version-update 
Hadoop version update
 2014-04-17 19:16:44 -06:00
fjy 399efe7dc4 make uniques work for groupby 2014-04-17 14:41:17 -07:00
nishantmonu51 2b5fec1dcf review comments 2014-04-16 17:12:27 +05:30
nishantmonu51 6da2272076 fix NPE in aggregatrFirstTopnAlgo 2014-04-16 09:12:34 +05:30
fjy 228be3acb0 Merge pull request #467 from metamx/optimize-postAgg-calculation 
Optimize post agg calculation
 2014-04-15 15:18:01 -06:00
nishantmonu51 36f6f8ab5a Add testOutOfOrderPruneDependentPostAgg 2014-04-16 00:49:56 +05:30
nishantmonu51 6217c67027 complete javadoc, add comment, use Lists.reverse 2014-04-15 22:55:40 +05:30
nishantmonu51 3eb8160b51 review comments 2014-04-15 21:06:30 +05:30
nishantmonu51 d66ae5ac10 fix dependent PostAggregators 2014-04-15 13:37:31 +05:30
fjy a46c4592b4 a few cleanups 2014-04-14 09:53:29 -07:00
fjy 7a8b9c2160 fix select serde 2014-04-10 15:35:55 -07:00
Xavier Léauté be30d450d7 support dimension extraction functions in group by 2014-04-09 17:17:01 -07:00
nishantmonu51 ecfa6bd1b1 fix casing and add comment 2014-04-08 17:26:51 +05:30
nishantmonu51 22c3296aa3 Merge branch 'master' into optimize-postAgg-calculation 
Conflicts:
	server/src/test/java/io/druid/client/CachingClusteredClientTest.java
 2014-04-08 03:35:56 +05:30
nishantmonu51 c322f44863 time series post Aggregation improved 
move post aggregation to finalise results
 2014-04-08 03:17:15 +05:30
fjy 6a26f7684f fix context on CQE 2014-04-07 14:36:13 -07:00
nishantmonu51 4bb36dd453 fix tests 2014-04-08 02:37:57 +05:30
fjy bae0fdf936 add context serde tests and deprecate backwards compatible methods 2014-04-07 13:18:55 -07:00
fjy 93349b5d07 fix broken unit tests 2014-04-07 10:56:38 -07:00
nishantmonu51 db35009acd Compute postage only when required 2014-04-07 23:17:25 +05:30
fjy 98c3faa767 cleanup 2014-04-06 09:23:59 -07:00
fjy 1267fbb7f5 fix context to be backwards compat 2014-04-06 09:20:58 -07:00
Hagen Rother e0e43deaec fix a NPE leading to broker 500 response 2014-04-04 13:07:52 +02:00
fjy 9ebdf16ed6 make last commit a final var 2014-03-27 17:56:25 -07:00
fjy cb1798c520 fix bug with reference closing that I swore we fixed 2014-03-27 17:55:03 -07:00
Xavier Léauté 0be29df8ae make hll tests deterministic 2014-03-27 16:15:32 -07:00
Xavier Léauté 5f0ada925d add HLL maxOverflow tests 2014-03-27 16:01:40 -07:00
Xavier Léauté 03c51518bc don't assume bytebuffer limit is set 2014-03-27 15:58:34 -07:00
Xavier Léauté 11f08b3895 benchmark more closely matches actual implementation 2014-03-26 22:18:26 -07:00
Xavier Léauté c92fd6f45d folding into direct buffers is faster 2014-03-26 17:13:41 -07:00
Xavier Léauté 83ac1d323e one more final variable for good measure 2014-03-26 16:50:24 -07:00
Xavier Léauté 2580a9c7f1 mark long-running tests as @Ignore + docs 2014-03-26 16:49:55 -07:00
Xavier Léauté f42b850d60 clean up + header 2014-03-26 09:31:45 -07:00
nishantmonu51 a9a6682a0e Upgrade to Hadoop 2.3.0 2014-03-26 11:43:31 +05:30
Xavier Léauté 178a9aed40 make it clear mergeAndStoreByteRegister has no side effects 2014-03-25 17:07:09 -07:00
Xavier Léauté a0c38f574b check should be safe to move out of loop 2014-03-25 17:07:09 -07:00
Xavier Léauté 46a5409207 minor hll optimizations 2014-03-25 17:07:09 -07:00
Xavier Léauté 5eaadfffa9 no need for volatile references 
we make no guarantees wrt threadsafety in this class
 2014-03-25 17:07:09 -07:00
Xavier Léauté 7dc0e9f1a9 remove HLL version specific code outside of HyperLogLogCollector 2014-03-25 17:07:09 -07:00
Xavier Léauté cd340aa314 hll benchmark 2014-03-25 17:07:09 -07:00
fjy e832dc7a8c Merge branch 'master' of github.com:metamx/druid into better-context 2014-03-21 10:46:36 -07:00
nishantmonu51 dc0e7f404a fix serialisation of segmentMetaDataQuery 2014-03-21 14:11:26 +05:30
fjy 7f8211ecc7 extend context to use map string object instead of map string string 2014-03-20 13:41:45 -07:00
nishantmonu51 32ad26210d Fix backwards compatibilty constant post aggregator 2014-03-18 20:34:49 +05:30
fjy 1d1ec1d855 clean up some defaults and prepare for next release 2014-03-17 21:41:34 -07:00
Xavier Léauté 7e97073aeb Set queryId in MetricsEmittingQueryRunner 
- queryId is now user8 instead of user10, since it was already used
before
 2014-03-17 13:25:47 -07:00
fjy 2adcf07f5f Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DetermineHashedPartitionsJob.java
	indexing-service/src/main/java/io/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/test/java/io/druid/indexing/common/task/TaskSerdeTest.java
	processing/src/test/java/io/druid/segment/TestIndex.java
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-03-17 10:59:31 -07:00
fjy f525361c3d fix broken tests and prepare for next release 2014-03-14 15:05:07 -07:00
fjy f0fdfbb2f7 fix broken constant post agg 2014-03-14 14:54:52 -07:00
fjy f00ffe4789 fix broken tests 2014-03-13 11:33:34 -07:00
Xavier Léauté de55098898 add comment 2014-03-12 10:47:23 -07:00
Xavier Léauté 2ef710191f make sure constants are not null 2014-03-12 10:40:02 -07:00
fjy 42618e6fcf a collection of small bug fixes 2014-03-12 10:34:24 -07:00
fjy ae2dc6506c fix broken granularity 2014-03-10 15:03:43 -07:00
fjy 36b54f7e8f fix all problems with topNs 2014-03-10 13:22:47 -07:00
Xavier Léauté b81bd6c959 add test for single value index top-n 2014-03-10 13:10:45 -07:00
fjy 6ce5266aca make stuff work 2014-03-10 12:19:58 -07:00
fjy 7b0d2f45bc Merge branch 'fix-iisa' of github.com:metamx/druid into fix-iisa 2014-03-10 12:02:03 -07:00
fjy 2db73998b9 add fix for II 2014-03-10 12:01:49 -07:00
Xavier Léauté adf60d1548 test reset sanity on out of order values 2014-03-10 11:54:07 -07:00
Yuval Oren 995002fcb3 Fix legacy data source serialization 2014-03-06 18:05:24 -08:00
Yuval Oren 0b0648266c Fix DataSource serialization 2014-03-06 15:28:20 -08:00
Yuval Oren 5601c51ae6 finalize variables and optimize imports 2014-03-06 15:03:43 -08:00
Yuval Oren be1ef3a161 Cleaned up equals() and hashCode(), per code review 2014-03-06 14:35:56 -08:00
Yuval Oren 4b63645802 Merge branch 'master' into subquery 
Conflicts:
	processing/src/main/java/io/druid/query/BaseQuery.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryQueryToolChest.java
	processing/src/test/java/io/druid/query/QueryRunnerTestHelper.java
	processing/src/test/java/io/druid/query/topn/TopNQueryRunnerTest.java
 2014-03-06 13:33:59 -08:00
fjy 5826d9594c add headers 2014-03-05 14:29:00 -08:00
fjy 08138688e4 port hyperunique to open source 2014-03-05 14:19:38 -08:00
fjy 0e8e527662 Merge branch 'master' into new-schema 2014-03-04 14:42:16 -08:00
fjy 5aba6141d8 Merge pull request #414 from metamx/incremental-index-synchronization-fix 
Add test and fix one more issue caught by test
 2014-03-04 15:41:26 -07:00
nishantmonu51 4f06dd5a7e Add test and fix one more issue caught by test 2014-03-05 01:09:33 +05:30
fjy 0feddc3831 Merge pull request #411 from metamx/small-fixes-docs-searchquery-limit 
Small fixes
 2014-03-04 10:30:27 -07:00
fjy 4983210907 Merge pull request #412 from pgkiran/filepeon-cleanup 
Temp filePeon files cleanup
 2014-03-04 10:29:52 -07:00
nishantmonu51 eb7bc28f41 fix synchronization 
getDimvalues can also add to the dimension, protect it properly
 2014-03-04 11:58:43 +05:30
fjy c4c4d80336 make local testing pass 2014-03-03 14:52:43 -08:00
fjy 46b9ac78e7 Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	pom.xml
	publications/whitepaper/druid.pdf
	publications/whitepaper/druid.tex
 2014-03-03 14:48:15 -08:00
Kiran Patchigolla 2dea60251b Temp filePeon files cleanup 
Related to https://groups.google.com/forum/#!topic/druid-development/TFUNas5ubyc
cleanup() method is added but still some files are not cleaned up. Additionally setting deleteOnExit on these temp files
 2014-03-03 13:33:47 -08:00
nishantmonu51 eedd6a1b9e fix #362 
fix #362
 2014-02-28 12:38:39 +05:30
nishantmonu51 0701b1534d review comments queryID changes, add wait time for realtime node queries 2014-02-27 14:39:25 +05:30
nishantmonu51 b480d8f543 review comments 2014-02-25 03:09:44 +05:30
nishantmonu51 373501a06d add queryID to query context and request_logs 2014-02-24 16:11:33 +05:30
fjy 7b0b90a860 Merge branch 'master' of github.com:metamx/druid into new-schema 2014-02-20 16:12:31 -08:00
fjy ab83cba790 Merge pull request #393 from metamx/group-by-improvements 
Group by memory usage improvements
 2014-02-19 19:52:37 -07:00
fjy 20cac8c506 not compiling yet but close 2014-02-19 15:54:27 -08:00
nishantmonu51 5be5cadd7b minor refactoring 2014-02-19 21:55:18 +05:30
fjy 4b7c76762d unit tests passingn at this point, finished rt port maybe 2014-02-18 15:14:38 -08:00
fjy 1a8f9f2e12 prepare for next release 2014-02-14 15:07:44 -08:00
fjy ef648b85f5 fix broken priority queries 2014-02-14 15:05:30 -08:00
fjy a8c4362d72 rejiggering druid api 2014-02-14 12:57:52 -08:00
nishantmonu51 74966eb2ef add copyright 2014-02-12 01:00:38 +05:30
nishantmonu51 3302cff6db groupBy improvements 2014-02-12 00:51:10 +05:30
nishantmonu51 533a263fbd initial working version 2014-02-10 22:32:26 +05:30
Yuval Oren bd11309593 Merge branch 'master' into subquery 2014-02-06 15:36:29 -08:00
fjy 0f6af72ea4 Merge branch 'master' into new-schema 2014-02-06 12:46:13 -08:00
fjy 9756b18f9b sort complex metrics with groupby 2014-02-06 11:26:05 -08:00
Yuval Oren 98956cdadb Extra null value check. Not sure why this is needed in the subquery implementation and not regular queries. Bug elsewhere? 2014-02-03 15:53:10 -08:00
fjy 14d0e54327 first commit 2014-02-03 14:15:03 -08:00
Yuval Oren c88cb2af16 Fix NPE when subquery returns empty result 2014-01-24 15:58:47 -08:00
Yuval Oren f2c1f798b9 Fix subquery merge issues 2014-01-24 15:03:48 -08:00
Yuval Oren cd719eab3c Merge branch 'master' into subquery 2014-01-24 14:42:41 -08:00
fjy c93f896092 Merge branch 'master' into select 
Conflicts:
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-01-20 13:26:51 -08:00
Yuval Oren 4c8495662f Subquery support for GroupBy queries 2014-01-15 15:38:57 -08:00
Xavier Léauté 022423e200 fix column closing 2014-01-14 15:01:46 -08:00
fjy 2aabc6c4e6 Merge branch 'master' into select 
Conflicts:
	processing/src/main/java/io/druid/query/Query.java
	server/src/main/java/io/druid/guice/QueryRunnerFactoryModule.java
	server/src/main/java/io/druid/guice/QueryToolChestModule.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-01-08 16:31:19 -08:00
fjy 0301dde671 remove unnecessary doc line 2014-01-08 15:56:29 -08:00
fjy 50a6839693 inital commit of topNs 2014-01-08 15:51:03 -08:00
Gian Merlino 3b2833d55d JavaScriptAggregatorFactory: Handle missing columns by passing down null args 2013-12-16 21:05:14 -08:00
fjy 5ecd909f7c reduce NPEs in CQE 2013-12-16 14:27:01 -08:00
Xavier Léauté bb69f8adcd fix JS dim extraction null handling + test case 2013-12-09 18:36:14 -08:00
fjy 1fb5fc6707 tests 2013-12-09 14:59:44 -08:00
fjy 414c17c8b2 dont need to allocate so much mem 2013-12-06 16:13:30 -08:00
fjy 5ba05a46f6 fix nulls in realtime persist 2013-12-05 17:59:02 -08:00
fjy b78919c729 fix NPEs 2013-12-05 16:49:28 -08:00
fjy ae5a4ff54c add versions to incremental index segment 2013-12-05 16:47:28 -08:00
fjy 241c80ef24 working pagination maybe 2013-12-05 13:37:44 -08:00
fjy 31c7ed639d first commit 2013-12-04 13:53:08 -08:00
fjy de0d6e2264 Merge pull request #308 from metamx/javascript-dimextractionfn 
Javascript DimExtractionFn
 2013-11-27 11:47:07 -08:00
Xavier Léauté 5c77884684 Address code review 
- more tests
- fix license
- add missing getter
 2013-11-25 17:14:10 -08:00
Xavier Léauté 51c0393455 typo 2013-11-22 16:33:16 -08:00
Xavier Léauté 41df3e1f11 implement javascript dimextractionfn 2013-11-22 16:33:16 -08:00
Xavier Léauté d0fe70a21f replace deprecated calls to getJsonFactory and createJsonParser 2013-11-20 17:14:23 -08:00
fjy 0587a9ecd0 Merge pull request #289 from alberts/master 
Fix compile error in Eclipse 4.3.
 2013-11-13 13:15:55 -08:00
Xavier Léauté 66181be1c6 make things final and fix copy-paste errors 2013-11-07 15:25:47 -08:00
Xavier Léauté 074589ffce JavaScript post-aggregator 2013-11-07 15:07:14 -08:00
Albert Strasheim 1589909b9d Fix compile error in Eclipse 4.3. 
The method dimEquals(String, String) is ambiguous for the type DimFilters.
 2013-11-04 21:23:13 -08:00
Xavier Léauté a86b02635a remove DimensionSelectorFactory 2013-10-23 22:02:06 -07:00
daggerrz 522feb65d5 Use druid-api 0.1.2-SNAPSHOT which provides Row.getRaw() 2013-10-20 22:39:13 -04:00
cheddar c47fe202c7 Fix HadoopDruidIndexer to work with the new way of things 
There are multiple and sundry changes in here.

First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.

In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module.  I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.

In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api.  They are all API-level things, however, so they really belong in druid-api instead.

Lastly, I removed the druid-realtime module and put it all in druid-server.
 2013-10-09 15:15:44 -05:00
fjy ac330f72bb first set of changes to standarize the naming convention we use in druid 2013-10-03 16:36:48 -07:00
fjy bc8db7daa5 1) make chat handler resource work again 
2) add more default configs
3) make examples work again
 2013-10-02 14:22:39 -07:00
fjy ed9e0cf9f6 add a local firehose for indexing local files 2013-09-30 16:03:26 -07:00
fjy a79ad7bab4 make dynamic master resource configuration work again 2013-09-27 15:00:40 -07:00
cheddar 46631bf409 Port fix for issue where IncrementalIndex doesn't properly filter for "null" values from 0.5.x to master 2013-09-18 18:18:53 -05:00
cheddar e019dddaaa Reapply Fix for NoSuchElementException that happens in IncrementalIndex when some columns are null 2013-09-18 17:54:49 -05:00
cheddar dc728b5721 Update versions of poms and make IncrementalIndexStorageAdapterTest compile 2013-09-18 17:01:03 -05:00
cheddar 6bfe2310bb Merge branch 'master' into guice 
Conflicts:
	client/pom.xml
	client/src/main/java/com/metamx/druid/query/segment/LegacySegmentSpec.java
	indexing-common/pom.xml
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolboxFactory.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskQueueTest.java
	realtime/src/main/java/com/metamx/druid/realtime/RealtimeNode.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/IrcFirehoseFactory.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/WikipediaIrcDecoder.java
	realtime/src/main/java/com/metamx/druid/realtime/plumber/RealtimePlumberSchool.java
	server/src/main/java/com/metamx/druid/BaseServerNode.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/index/v1/IncrementalIndexStorageAdapter.java
	server/src/main/java/com/metamx/druid/query/group/GroupByQueryEngine.java
 2013-09-18 16:46:54 -05:00
cheddar dadd9bc5ce 1) Rename MetricSelector(Factory) to ColumnSelector(Factory) 
2) Hard coded plugin as a first pass, won't actually work for anyone else.
 2013-09-04 11:19:46 -05:00
cheddar 740c70ed06 1) It's good when unit tests actually pass... 2013-08-31 17:27:15 -05:00
cheddar 3c39f90c89 1) Move Firehose interface and dependencies to druid-api 
2) Move DataSegment* interfaces and dependencies to druid-api
 2013-08-31 16:43:28 -05:00
cheddar 5ab671050e No more com.metamx.druid, it is now all io.druid! 2013-08-30 19:42:12 -05:00
cheddar bd0756e360 More stuff moved, things still compiling and tests still passing. Yay! 2013-08-30 18:58:35 -05:00
cheddar 56e2b956d0 OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.! 2013-08-30 18:21:04 -05:00