cheddar
ee1e73cfa1
1) Make it compile again after the merge
2013-08-27 14:36:01 -05:00
cheddar
5fa944dd26
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
pom.xml
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/http/MasterServletModule.java
server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
2013-08-27 14:27:32 -05:00
cheddar
269997dc94
1) ExecutorNode is working, except for the running of the task. Need to adjust it to be able to run a task and then everything will be wonderful
2013-08-26 18:08:41 -05:00
cheddar
ba5ed4c9d4
1) Fix bug with NPE thrown when requesting a dimension on a GroupBy query that is not dictionary encoded
2013-08-14 17:34:11 -07:00
cheddar
c0624d9591
Merge branch 'master' of github.com:metamx/druid
2013-08-14 10:04:04 -07:00
cheddar
a7ef5b8b70
1) Fix bug with SingleSegmentLoader.StorageLocation keeping track of its storage size incorrectly. Add unit test ftw.
2013-08-14 10:03:47 -07:00
cheddar
4791d0466d
Merge pull request #198 from metamx/loadBalancer
...
Improve runtime of segment distribution algorithm
2013-08-13 18:47:43 -07:00
cheddar
c2305a82f9
Merge pull request #209 from zanox/kafka-protobuf
...
Kafka protobuf
2013-08-13 18:38:27 -07:00
cheddar
145e08682c
1) Add check in ServerManagerTest to make sure that the Segment has been "checked out" before the factory ever sees it.
...
2) Some code readability changes to ReferenceCountingSegment
2013-08-13 17:01:24 -07:00
cheddar
28c7e78a91
1) Fix ReferenceCountingSegmentTest to actually work again
2013-08-13 14:28:44 -07:00
fjy
41347c82bb
fix broken server manager tests
2013-08-13 12:07:45 -07:00
fjy
070d822345
fix broken UT
2013-08-13 12:01:24 -07:00
fjy
13e522c8a7
update the server manager unit tests to check for the case where a segment may be accessed in toYield calls before an increment occurs
2013-08-13 11:53:41 -07:00
Jan Rudert
3f32ef7bbc
Merge branch 'master' of github.com:metamx/druid into kafka-protobuf
2013-08-12 17:42:40 +02:00
jan
55267ca0e4
removed @author tags
2013-08-11 13:44:13 +02:00
Dhruv Parthasarathy
01db89ac6e
fixed issue with web console. Cleaned up code
2013-08-09 00:33:40 -07:00
Dhruv Parthasarathy
fc5e15f72b
merged changes with master
2013-08-08 18:11:33 -07:00
Dhruv Parthasarathy
abe347efd3
merged with master
2013-08-08 17:21:57 -07:00
Dhruv Parthasarathy
8a8ea5d028
removed test annotation from profiler
2013-08-08 16:38:24 -07:00
Dhruv Parthasarathy
eed9e1b373
merged with master
2013-08-08 16:14:06 -07:00
Dhruv Parthasarathy
3a298681e4
fixed formatting
2013-08-07 10:59:54 -07:00
Dhruv Parthasarathy
c4f0e4d96d
fixed with Erics feedback
2013-08-07 10:53:37 -07:00
fjy
5f292143e0
disable merging segments with linear shard spec
2013-08-06 16:17:49 -07:00
Dhruv Parthasarathy
f31ee552d0
fixed with fj comments
2013-08-06 15:03:07 -07:00
Dhruv Parthasarathy
6124ecd9cb
fied some tests
2013-08-06 14:26:08 -07:00
cheddar
eee1efdcb5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/client/DruidServerConfig.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/index/ChatHandlerProvider.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/TaskMasterLifecycle.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
2013-08-06 13:33:31 -07:00
cheddar
9f71d42314
1) Broker works with Guice
...
2) Extract ServerViewModule to handle various ServerView thingies
3) Extract QueryToolChestModule and QueryRunnerFactoryModule to reuse code for configuration of Query stuff
4) Extract QueryJettyServerInitializer to reuse between Historical and Broker nodes
5) Remove ClientMain, BrokerMain and BrokerNode
2013-08-06 12:01:52 -07:00
Dhruv Parthasarathy
ebba2d66a3
results good
2013-08-06 11:46:18 -07:00
Dhruv Parthasarathy
e40d5b55ef
testing speed for one run
2013-08-06 11:40:49 -07:00
Dhruv Parthasarathy
575dfe3679
replication working too
2013-08-05 14:51:58 -07:00
Dhruv Parthasarathy
540c4885d7
fixes to make profiler work
2013-08-05 12:36:43 -07:00
cheddar
2361e0112a
Make it all compile again...
2013-08-02 10:14:46 -07:00
Dhruv Parthasarathy
6f6a335a40
adding profiling for 55k servers
2013-08-01 23:38:44 -07:00
Dhruv Parthasarathy
9f17e8fc43
all working
2013-08-01 21:01:02 -07:00
Dhruv Parthasarathy
696b04020b
wrote regression test
2013-08-01 18:28:48 -07:00
fjy
584ccac833
move scanning of workers and tasks into RTR start, simplify bootstrap, make tests better
2013-08-01 17:50:05 -07:00
cheddar
9e78bb38f5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/QueryableNode.java
client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
pom.xml
server/src/main/java/com/metamx/druid/coordination/ServerManager.java
server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
server/src/main/java/com/metamx/druid/http/ComputeNode.java
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-08-01 16:42:47 -07:00
cheddar
019bb5d453
1) Another whole bunch of changes to annotate things and create Modules and bind stuff. But OMFG, the compute node actually appears to be working!
...
2) The compute node works with Guice
3) The compute node fires up with Guice and appears to work
4) Did I mention that the compute node, now called historical node, fires up with Guice and appears to work?
2013-08-01 15:28:08 -07:00
cheddar
fba2bc9623
Merge pull request #204 from metamx/close-handle
...
add the ability to unmap mmapped files
2013-08-01 14:09:46 -07:00
Dhruv Parthasarathy
9b340d9f6c
simplified console code. Added Builder. Fixed urls
2013-08-01 14:06:22 -07:00
fjy
8f39db7804
more data to exception msgs
2013-08-01 14:04:27 -07:00
fjy
6d54ae9f81
more unit tests to test for multiple closes in different threads
2013-08-01 13:51:56 -07:00
fjy
e92e0c35a4
put all concurrency test back into server manager test, introduce better concurrency for reference counting
2013-08-01 13:13:03 -07:00
fjy
b86258cc14
fix broken master UT based on changes that were discarded as part of a pull req
2013-08-01 10:44:39 -07:00
fjy
2f8351b314
fix broken ut
2013-08-01 09:41:36 -07:00
cheddar
f4fc8b2f6e
1) A whole boondoggle of changes to Guicify the instantiation of things for the Historical nodes
...
2) Add GuiceInjectableValues that supports @JacksonInject resolution via Guice
3) Refactor RequestLogger stuff and setup polymorphic deserialization method of creating objects
2013-07-31 18:36:47 -07:00
fjy
e2b5cd6067
Merge branch 'master' of github.com:metamx/druid into worker-resource
...
Conflicts:
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
2013-07-31 15:23:13 -07:00
fjy
f76540c16c
move reference counting classes out of server manager and more tests for concurrency
2013-07-31 15:14:39 -07:00
fjy
5d1fe50703
additional ut for reference counting
2013-07-30 17:55:57 -07:00
fjy
404b84823e
reference count to make sure we arent accessing any instances of a closed mmapped segment
2013-07-30 17:36:56 -07:00
cheddar
843b6650a2
Merge pull request #206 from metamx/peontimeout
...
Add a timeout in LoadQueuePeon to fail an assign if it takes too long
2013-07-30 15:08:46 -07:00
fjy
35d8a82879
move the close method to the segment object
2013-07-30 12:24:01 -07:00
cheddar
66adef5d64
Merge pull request #179 from zanox/master
...
Fix path used when storing segments to HDFS
2013-07-30 11:13:29 -07:00
Dhruv Parthasarathy
4e4d582e07
formatted files
2013-07-29 18:10:08 -07:00
Dhruv Parthasarathy
b0d9889c83
added web console for changing dynamicConfigs
2013-07-29 17:49:09 -07:00
fjy
091dce11c8
Add a timeout in LQP to fail an assign if it takes too long
2013-07-29 15:47:05 -07:00
Dhruv Parthasarathy
31ffed466c
first set of changes
2013-07-29 11:39:24 -07:00
Dhruv Parthasarathy
ba4a9c28a7
removed unecessary line
2013-07-26 10:52:50 -07:00
Dhruv Parthasarathy
1e4f64b119
redid testing
2013-07-26 10:50:06 -07:00
fjy
217894d44a
Merge branch 'master' into worker-resource
...
Conflicts:
pom.xml
2013-07-25 16:18:42 -07:00
Dhruv Parthasarathy
8e4fba431c
moved profiler
2013-07-25 14:37:03 -07:00
Dhruv Parthasarathy
34a9525966
added tests for random sampler and changed abstract class to interface
2013-07-25 14:14:16 -07:00
Dhruv Parthasarathy
a03dcc6429
cleaned up comments
2013-07-24 15:34:08 -07:00
Dhruv Parthasarathy
bcb3e089d8
profiler added, commented out initialTotalCost normalization and normalizedInitialCost
2013-07-24 14:57:02 -07:00
fjy
980b09d903
fix broken test
2013-07-23 10:19:26 -07:00
fjy
ea7c7d1d66
Merge branch 'master' into batch-zk
...
Conflicts:
client/src/main/java/com/metamx/druid/QueryableNode.java
2013-07-23 10:14:26 -07:00
fjy
3f97ac9253
add read to batch zk
2013-07-22 18:01:13 -07:00
cheddar
7504f3ab65
1) Adjust the Announcer to cleanup parent nodes that it creates. This stops it from leaving little turdlings lying around on zookeeper as nodes exit
...
2) Rename *CuratorDataSegmentAnnouncer because neither of them depend directly on Curator.
2013-07-22 16:31:01 -07:00
Jan Rudert
18ec081855
moved getHdfsStorageDir() to DataSegmentPusherUtil, extended test
2013-07-08 08:39:08 +02:00
Jan Rudert
6f06d701d9
Building a correct segment path for HDFS
2013-07-04 16:00:20 +02:00
fjy
d3eb491155
Merge branch 'batch-zk' into worker-resource
...
Conflicts:
pom.xml
server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
2013-07-02 16:24:59 -07:00
fjy
7219ed15d3
fix according to code review
2013-07-02 15:56:12 -07:00
fjy
a6787ab201
fix up configs
2013-06-27 16:40:17 -07:00
fjy
caa68e101a
first commit; things working right now
2013-06-19 15:56:45 -07:00
cheddar
06f7e7e665
1) Fix DatabaseSegmentManager so that Unit Test passes
2013-06-10 14:13:28 -07:00
fjy
3312d9b802
fix merging problem with linear shard specs
2013-06-10 14:07:36 -07:00
cheddar
2f56c24259
1) Inject IndexingServiceClient
...
2) Switch all the DBI references to IDBI
2013-06-07 17:37:33 -07:00
cheddar
9df458a065
1) Initial commit of conversion to using Guice modules for bootstrapping. Things don't actually completely work yet.
2013-06-06 15:43:15 -07:00
fjy
42cc87a294
Merge branch 'master' into refactor-indexing
...
Conflicts:
indexing-service/src/main/java/com/metamx/druid/indexing/common/task/IndexTask.java
pom.xml
2013-05-31 17:28:59 -07:00
fjy
1fb8c9db7f
fix spatial bugs and more tests
2013-05-22 13:35:12 -07:00
fjy
6e5b893e6d
make spatial filters work
2013-05-21 17:10:44 -07:00
fjy
c6ad0753f0
bug fixes
2013-05-16 16:02:20 -07:00
fjy
803e8ff69e
add configurable spatial dimensions to hadoop indexer
2013-05-14 17:38:05 -07:00
fjy
5af188f18d
Merge branch 'master' into spatial
2013-05-14 16:10:18 -07:00
cheddar
fe5627e119
1) Left some log lines in the tests... That's no good.
2013-05-14 16:59:34 -05:00
cheddar
8bd19e9d04
1) Take in g9yuayon's pull request, fix merge conflicts and formatting
2013-05-14 16:55:43 -05:00
fjy
986bb2482d
Merge branch 'master' into spatial
2013-05-08 10:37:38 -07:00
fjy
43d3c53806
fix order by double sum agg
2013-05-08 10:14:40 -07:00
fjy
8902986069
Merge branch 'master' into spatial
2013-05-03 23:09:30 -07:00
fjy
344a2b5d24
numerous bug fixes and improvements according to code review
2013-05-03 23:09:05 -07:00
cheddar
665f3ef0fd
1) Unit tests for order by and limit stuff for groupBy queries
...
2) Adjustments to order by and limit stuff in light of unit tests-n-stuff
2013-05-03 17:47:18 -05:00
fjy
d57141f46e
checking stuff in but stuff not really working yet
2013-05-03 10:35:26 -07:00
fjy
63ccb24ef6
Merge branch 'master' into spatial
...
Conflicts:
pom.xml
server/src/main/java/com/metamx/druid/http/InfoResource.java
2013-05-01 10:12:04 -07:00
cheddar
405eed7c60
Better name for ServerInventoryThingie
2013-04-29 11:48:42 -05:00
cheddar
99305b29c8
Fixorz unit tests!
2013-04-26 19:19:57 -05:00
cheddar
de5c8c9d64
It's better when things compile
2013-04-26 19:19:56 -05:00
cheddar
67ce1b6f26
1) Adjust all of the uses of zookeeper paths to be based on ZkPathsConfig
2013-04-26 19:19:56 -05:00
cheddar
3e94204c44
1) Switch from com.netflix.curator to org.apache.curator
2013-04-26 19:19:55 -05:00
cheddar
b8ba9138ff
1) Fix bug with Master not starting up without an indexer specified
...
2) Fix bug with PotentiallyGzippedCompressionProvider only catching ZipExceptions and not IOException (java 6 throws IO, java 7 throws Zip)
3) Create DruidServerMetadata and use that instead of DruidServer to represent the current server
2013-04-26 19:19:55 -05:00
cheddar
f71b941a1a
1) Refactor the announcement of segments to all exist inside the DataSegmentAnnouncer
...
2) Adjust the ExecutorNodes to expose the correct nodeType given the refactorings
2013-04-26 19:19:54 -05:00