Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,489 Commits 113 Branches 505 Tags 731 MiB
Commit Graph

245 Commits

Author SHA1 Message Date
cheddar 57c31656d1 1) Eliminate PhoneBook 2013-04-26 19:19:53 -05:00
cheddar 43d630c098 1) Replace InventoryManagers with Curator-based Inventory Managers 
2) Replace ZkSegmentAnnouncer with CuratorSegmentAnnouncer
3) Adjust LoadQueuePeons to operate using Curator
4) Remove ZkPhoneBook

Things left to do:
1) Unit tests
2) Remove references to zkclient library
3) Remove references to PhoneBook interface
 2013-04-26 19:19:23 -05:00
fjy e01a78515e many more fixes and unit tests 2013-04-25 19:01:45 -07:00
fjy 9b9c516e30 more tests 2013-04-22 18:39:41 -07:00
fjy 6c4e844f97 semi working spatial 2013-04-19 16:56:56 -07:00
Eric Tschetter 55648c47a7 1) Adjust the GroupByQuery to also be able to merge results on the local node. Fixes #116 
2) Make the GroupByQuery operate in a multi-threaded fashion by default (this is configurable via druid.query.groupBy.singleThreaded).  Fixes #96
3) Fix up some post aggregation computation stuff.  I believe #72 is fixed
4) Fix case sensitivity issue with post aggregations on GroupBy queries
 2013-04-09 19:13:08 -07:00
René Scheibe a6dc29cf85 cleanup - removed extra empty lines 
- there should not be any adjacent empty lines
 2013-04-07 14:32:09 +02:00
Nelson Ray b1f2186c1f use reservoir sampling in pickSegmentToMove to avoid IndexOutOfBoundsException when a segment gets dropped mid-run 2013-04-02 12:22:11 -07:00
Fangjin Yang 0730976c80 fix broken UT 2013-04-01 17:16:19 -07:00
Fangjin Yang 258f18471e nodes will not try and download segments during startup 2013-04-01 16:55:03 -07:00
xvrl c420fe3b56 fix output timestamps in groupby queries with granularity "all". 2013-03-23 23:42:43 -07:00
xvrl c0397aa67c add test case for granularities not aligned with segment boundaries 2013-03-21 10:38:06 -07:00
Eric Tschetter 1c3ef48f34 1) Adjust the Config stuff that WorkerSetupManager was using to be reusable 
2) Use new ConfigManager to pull out a whitelist for the Master
 2013-03-07 17:45:57 -06:00
Nelson Ray e67a8757ba Merge remote-tracking branch 'origin/master' into new_balancer 
Conflicts:
	server/src/main/java/com/metamx/druid/master/DruidMaster.java
 2013-02-26 14:18:07 -08:00
Gian Merlino 49700dffe9 Merge branch 'master' into task-stuff 
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/common/TaskToolbox.java
	merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
	merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
	merger/src/main/java/com/metamx/druid/merger/common/task/MergeTask.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
	merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
 2013-02-25 12:18:31 -08:00
Eric Tschetter dc3459d3f9 1) Initial commit of refactorings on top of housejester's to simplify the zipping and simplify the creation of other methods of loading data 2013-02-21 15:47:01 -06:00
Nelson Ray 728e208f49 merge master 2013-02-15 11:39:22 -08:00
xvrl 05d347c304 Merge remote-tracking branch 'origin/master' into speedup-timeboundary 
Conflicts:
	client/pom.xml
	common/pom.xml
	druid-services/pom.xml
	examples/pom.xml
	examples/rand/pom.xml
	examples/twitter/pom.xml
	index-common/pom.xml
	indexer/pom.xml
	merger/pom.xml
	pom.xml
	realtime/pom.xml
	server/pom.xml
 2013-02-15 11:25:47 -08:00
Eric Tschetter 5ee7c0c264 1) Remove stray import of java.util.List 2013-02-15 12:15:53 -06:00
Gian Merlino 548c901c0b Additional fix for columns with cardinality 0 2013-02-14 23:45:02 -08:00
Gian Merlino 22d1f2a0c0 IndexMergerTest: Empty column test 2013-02-14 21:11:31 -08:00
Gian Merlino 18daa19fc9 Merge branch 'master' into task-stuff 
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/common/TaskHolder.java
	merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
	merger/src/main/java/com/metamx/druid/merger/common/task/AbstractTask.java
	merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
	merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
	merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
	merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/TaskContext.java
	merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
	merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
	merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
	server/src/main/java/com/metamx/druid/master/DruidMaster.java
 2013-02-14 13:10:29 -08:00
Nelson Ray 754b065c54 merge master 2013-02-14 09:08:25 -08:00
Eric Tschetter c5b78e75cf 1) Move up to Jackson 2.1 
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
 2013-02-13 18:51:32 -08:00
Eric Tschetter a0b159fed7 1) Move up to Jackson 2.1 
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
 2013-02-12 20:58:17 -06:00
Nelson Ray fcbac96193 revert change to findNewSegmentHome: now return single ServerHolder 2013-02-12 10:47:30 -08:00
xvrl e9f546c4ca add default implementations to QueryToolChest methods 2013-02-11 16:51:40 -08:00
Nelson Ray 695b78b19c make findNewSegmentHome return a priority queue so assignment can assign replicants to other servers 2013-02-11 15:28:17 -08:00
xvrl e10025c841 filter out unneeded segments for TimeBoundary query 2013-02-11 13:21:37 -08:00
Nelson Ray 1b9764fffb add binary version to datasegments for balancer test 2013-02-11 11:36:45 -08:00
Nelson Ray e12da51057 merge master 2013-02-11 11:28:50 -08:00
Fangjin Yang 7540dd58ca fix broken UT 2013-02-06 16:52:24 -08:00
fjy e2862fe1fb Merge pull request #79 from metamx/bug_fixes 
Throttle replicants that are created by rules
 2013-02-06 16:50:04 -08:00
Fangjin Yang 84baef0462 fix some bugs from last commit 2013-02-06 13:57:56 -08:00
Eric Tschetter 34b4383934 1) Adjust DataSegment to have a binaryVersion field that indicates the expected binary version of the segment files 
2) Eliminate race condition from RemoteTaskRunnerTest.testAlreadyExecutedTask()
 2013-02-01 18:59:33 -06:00
Fangjin Yang f50c214514 Merge branch 'master' into task-stuff 2013-01-31 09:59:52 -08:00
Nelson Ray 73f774e768 make masterbalancertest more stringent 2013-01-29 18:17:38 -08:00
Nelson Ray a1ce4b294e simplify balancer test 2013-01-29 18:16:49 -08:00
Nelson Ray 034d652569 make balancer tests take into account loadqueue status 2013-01-29 17:13:01 -08:00
cheddar 9a22e46eca Merge pull request #71 from metamx/queryrunner-onerow-test 
Bug fixes for incremental index storage adapter and one row indexes
 2013-01-29 13:41:15 -08:00
cheddar a7abd2ca86 Merge pull request #70 from metamx/timezone-tests 
More time zone tests
 2013-01-29 13:40:04 -08:00
Fangjin Yang 41ca33c734 Merge branch 'master' of github.com:metamx/druid into task-stuff 2013-01-29 12:57:56 -08:00
Gian Merlino 61569f62be TimeseriesQueryRunner tests with small incremental indexes 2013-01-29 12:54:52 -08:00
Eric Tschetter da914b835e 1) Add caching for SegmentMetadataQuery 2013-01-29 13:04:57 -06:00
xvrl 27e09987c6 timezone test 2013-01-29 10:38:22 -08:00
Eric Tschetter 08d3253f3d Merge branch 'master' of github.com:metamx/druid 2013-01-29 11:29:42 -06:00
Eric Tschetter f2cce28297 1) Make SegmentMetadataQuery work 2013-01-29 11:29:26 -06:00
Eric Tschetter 7439a2e820 1) Make tests pass 2013-01-28 16:42:51 -06:00
xvrl c0822325e7 proper groupby tests 2013-01-28 16:41:03 -06:00
xvrl 5e1cac6d9f timezone test 2013-01-24 17:57:10 -08:00
xvrl 55ae4c87dd timezone support in groupby query 2013-01-23 08:51:02 -08:00
xvrl 35058786d9 match query interval to granularity for this test 2013-01-23 08:50:43 -08:00
xvrl 86a6d112e3 proper groupby tests 2013-01-22 16:54:14 -08:00
Fangjin Yang 272d737517 cleaning up some interactions with RTR and workers 2013-01-22 16:21:38 -08:00
xvrl f05c050c53 add test for timezone 2013-01-21 15:49:39 -08:00
Nelson Ray 94b72e8878 replace param BalancerCostAnalyzer getter with a factory 2013-01-21 15:32:29 -08:00
Nelson Ray 2d7113b263 huge simplification of balancing code 2013-01-21 14:28:25 -08:00
Eric Tschetter c8cb96b006 1) Remove vast majority of usages of IndexIO.mapDir() and deprecated it. IndexIO.loadIndex() is the new IndexIO.mapDir() 
2) Fix bug with IndexMerger and null columns
3) Add QueryableIndexIndexableAdapter so that QueryableIndexes can be merged
4) Adjust twitter example to have multiple values for each hash tag
5) Adjusted GroupByQueryEngine to just drop dimensions that don't exist instead of throwing an NPE
 2013-01-16 17:10:33 -06:00
Nelson Ray 5f40dc6d9c put BalancerCostAnalyzer into params 2013-01-10 10:27:43 -08:00
Nelson Ray 6fda5330fd finish making MAX_SEGMENTS_TO_MOVE a config item 2013-01-07 16:56:55 -08:00
Nelson Ray a7553776fa make sampling of segments uniformly at random and add unit test 2013-01-07 12:51:51 -08:00
Eric Tschetter b184c46ae0 1) Introduce idea of value types and generic columns to the serialization format 
2) Create SegmentLoader interface for the loading of segments!
3) Setup serialization of new format and conversion function inside IndexIO
4) Make some adjustments to the twitter demo to make it make a bit more sense based on some email feedback from someone kicking the tires.
 2013-01-04 18:12:49 -06:00
Nelson Ray 402ddbf261 update 2013-01-04 11:20:06 -08:00
Nelson Ray 117cac2795 update 2013-01-03 14:12:10 -08:00
xvrl a9b8d8f1ce fix end of month bug in PeriodDropRule as well 2013-01-02 16:10:23 -08:00
xvrl 0543322c56 tested timestamps should be deterministic 2013-01-02 14:31:51 -08:00
xvrl 19c3fd3363 make reference timestamp explicit for time based rules 2013-01-02 14:27:34 -08:00
Nelson Ray 72758e1345 update 2012-12-31 13:55:48 -08:00
Nelson Ray 3eda9dc5f2 working balancercostanalyzer 2012-12-31 11:08:37 -08:00
Nelson Ray 366216f9f3 initial cost analyzer commit 2012-12-28 16:30:21 -08:00
Fangjin Yang 585a812f6e bug fixes again in load and drop 2012-12-15 00:43:16 -08:00
Fangjin Yang 6a11e99681 bug fixes for dropping segments and logging 2012-12-14 21:27:18 -08:00
Fangjin Yang 74a977504e bug fix for dropping segments in master 2012-12-14 17:06:03 -08:00
Fangjin Yang 8c7fd3bc49 more specific checks in rule test 2012-12-14 09:12:02 -08:00
Fangjin Yang aa69ed5251 alert when no rules match; exceptions in rules no longer block 2012-12-14 09:05:25 -08:00
Fangjin Yang 4a1abfc5dc gui for rule configuratin; introducing jquery ui to master console 2012-12-05 15:39:59 -08:00
Fangjin Yang 5b8bd412c0 caching for rules; get rules from cache 2012-12-03 16:13:20 -08:00
Fangjin Yang 5822f4f5f7 refactor master to run rules before cleaning up; more master stats; general improvements 2012-12-03 14:43:04 -08:00
Fangjin Yang 91cd9e9536 fix broken UT 2012-11-30 13:17:14 -08:00
Fangjin Yang aa8fb7b9a3 abstract the assign and drop logic; cleanups everywhere; http endpoints for getting and assigning rules 2012-11-30 11:28:10 -08:00
Fangjin Yang 2e5e1ce989 first commit of tiers for compute nodes; working UT at this point 2012-11-28 17:37:08 -08:00
Eric Tschetter 09a31e5995 Tests should build. 2012-11-28 12:35:24 -06:00
Gian Merlino 5902ccd0cb DruidMasterSegmentMerger: Clearer method names and more tests 2012-11-27 15:46:03 -08:00
Eric Tschetter 06b8e4230c 1) Fix PostAggregations with GroupBy (Fixes #24) 
2) Add GroupByTimeseriesQueryRunnerTest which wraps a GroupByQueryRunner to look like a TimeseriesQueryRunner and leverages the tests there to help verify behavior (#8)
 2012-11-21 12:51:43 -06:00
Fangjin Yang 4131063049 Merge branch 'master' into fjy 2012-11-19 15:07:00 -08:00
Eric Tschetter 701cc9562b 1) Adjust the StorageAdapters to lowercase names of metrics and dimensions before looking them up. 
2) Add some docs to InputRow/Row to indicate that column names passed into the methods are *always* lowercase and that the rows need to act accordingly. (fixes #29, or at least clarifies the behavior...)
 2012-11-19 17:01:17 -06:00
Gian Merlino 16835a1f84 Master: Add segment limit to merge selection algo 
If we attempt to merge too many segments at once, we risk creating an
oversized indexing task.
 2012-11-19 14:46:45 -08:00
Fangjin Yang 3ddfdfc702 add missing header 2012-11-08 10:45:21 -08:00
Fangjin Yang bff5a7d786 fix some merge issues 2012-11-08 10:43:39 -08:00
Fangjin Yang 499263386c timeseries for druid + unit tests 2012-11-05 11:38:03 -08:00
Eric Tschetter f07b15db1f Undo massive import re-ordering 2012-11-01 18:38:27 -05:00
Eric Tschetter 8dd6f5c059 1) Refactor the BrokerMain into a BrokerMain and a BrokerNode to allow for extensions 
2) Rename ServerMain to ComputeMain
3) Refactor ComputeMain to ComputeMain and ComputeNode to allow for extensions
4) Remove "TheSizeAdjuster", was old legacy stuff that's not relevant anymore
5) Fix bug with registering different IndexIO handlers
6) Adjust it so that when a query type is unknown, it returns an error message instead of just emitting an alert and returning nothing
7) Create super classes for the various *Node classes that allow for code sharing on pieces that are common
 2012-11-01 18:20:38 -05:00
Ian Brandt 0d24df7628 Removed unused imports. 2012-10-30 22:22:46 -07:00
Eric Tschetter 27999caca0 1) Create LICENSE 
2) Attach copyright and notice of license to files
 2012-10-24 05:09:47 -04:00
Eric Tschetter 9d41599967 Initial commit of OSS Druid Code 2012-10-24 03:39:51 -04:00