Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,489 Commits 113 Branches 505 Tags 728 MiB
Commit Graph

1153 Commits

Author SHA1 Message Date
fjy 9bbaade8f9 [maven-release-plugin] prepare for next development iteration 2013-11-13 10:37:12 -08:00
fjy 4dffef698e [maven-release-plugin] prepare release druid-0.6.12 2013-11-13 10:37:08 -08:00
fjy 12a188d8d6 [maven-release-plugin] prepare for next development iteration 2013-11-12 15:20:54 -08:00
fjy 63d9f7362a [maven-release-plugin] prepare release druid-0.6.11 2013-11-12 15:20:47 -08:00
fjy 51aa7a2284 doesnt work yet 2013-11-11 14:04:03 -08:00
fjy 0681fb614d [maven-release-plugin] prepare for next development iteration 2013-11-08 15:56:10 -08:00
fjy de5984ccea [maven-release-plugin] prepare release druid-0.6.10 2013-11-08 15:56:06 -08:00
fjy d212b3fc43 [maven-release-plugin] prepare for next development iteration 2013-11-08 15:14:46 -08:00
fjy 4cd4578539 [maven-release-plugin] prepare release druid-0.6.9 2013-11-08 15:14:41 -08:00
fjy 822a30658d fix broken http post emitter and prepare for next release 2013-11-08 14:01:16 -08:00
fjy fd4fb78e1f [maven-release-plugin] prepare for next development iteration 2013-11-08 11:34:12 -08:00
fjy f8d5d127a4 [maven-release-plugin] prepare release druid-0.6.8 2013-11-08 11:34:09 -08:00
fjy 90240e6f1e [maven-release-plugin] prepare for next development iteration 2013-11-07 23:29:02 -08:00
fjy c3d6233bc4 [maven-release-plugin] prepare release druid-0.6.7 2013-11-07 23:28:58 -08:00
fjy d70bcc5657 repair broken pom 2013-11-07 23:27:06 -08:00
fjy fafd9c8d27 [maven-release-plugin] prepare for next development iteration 2013-11-07 18:20:19 -08:00
fjy f945416538 [maven-release-plugin] prepare release druid-0.6.5 2013-11-07 18:20:16 -08:00
fjy ed328aca6e [maven-release-plugin] prepare for next development iteration 2013-11-07 18:03:39 -08:00
fjy e34f1a829b [maven-release-plugin] prepare release druid-0.6.4 2013-11-07 18:03:36 -08:00
fjy 009646ed56 [maven-release-plugin] prepare for next development iteration 2013-11-07 17:40:12 -08:00
fjy 4cde47c9d1 [maven-release-plugin] prepare release druid-0.6.3 2013-11-07 17:40:06 -08:00
fjy e6a83c0339 [maven-release-plugin] prepare for next development iteration 2013-11-07 17:05:52 -08:00
fjy 2ac09b798d [maven-release-plugin] prepare release druid-0.6.2 2013-11-07 17:05:48 -08:00
Xavier Léauté 1c87a00713 remove servlet-api deps, duplicates jetty dependency 2013-11-07 16:34:50 -08:00
Xavier Léauté 4fe52320d7 remove duplicate dependencies 2013-11-07 16:13:38 -08:00
fjy 6181114acd Merge pull request #288 from metamx/abandon-rt-segments 
Realtime: Abandon segments that fail to push when shutting down
 2013-11-07 15:53:49 -08:00
fjy ee15fd9f85 [maven-release-plugin] prepare for next development iteration 2013-11-05 15:42:37 -08:00
fjy 713015199a [maven-release-plugin] prepare release druid-0.6.1 2013-11-05 15:42:33 -08:00
Xavier Léauté 76732c25f8 increase timeout for slow machines and provide more descriptive error message 2013-11-04 14:07:43 -08:00
Gian Merlino d8d3a6958e Realtime: Abandon segments that fail to push 2013-11-02 08:57:58 -07:00
Xavier Léauté dce6e7e0eb hack to support remote repository authentication 2013-11-01 09:33:07 -07:00
fjy 5d8f1d11ec fix broken UT as a result of new druid api changes 2013-10-22 10:44:28 -07:00
daggerrz 522feb65d5 Use druid-api 0.1.2-SNAPSHOT which provides Row.getRaw() 2013-10-20 22:39:13 -04:00
fjy a684885839 [maven-release-plugin] prepare for next development iteration 2013-10-18 15:48:16 -07:00
fjy 88dcfe9a94 [maven-release-plugin] prepare release druid-0.6.0 2013-10-18 15:48:12 -07:00
fjy 2dc716bf7e fix bug and make it actually possible to load extensions 2013-10-16 11:59:01 -07:00
fjy 9796a40b92 port docs over to 0.6 and a bunch of misc fixes 2013-10-11 18:38:53 -07:00
fjy a9a723bd11 clean up poms, add a new loading your own data tutorial, add new validation, clean up logs 2013-10-09 15:42:39 -07:00
cheddar c47fe202c7 Fix HadoopDruidIndexer to work with the new way of things 
There are multiple and sundry changes in here.

First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.

In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module.  I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.

In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api.  They are all API-level things, however, so they really belong in druid-api instead.

Lastly, I removed the druid-realtime module and put it all in druid-server.
 2013-10-09 15:15:44 -05:00
fjy af1dbe6eab fix docs for 0.6 part 1 of many 2013-10-07 14:47:04 -07:00
fjy 703b674800 add availability zone info to autoscaling 2013-10-07 12:16:50 -07:00
fjy ac330f72bb first set of changes to standarize the naming convention we use in druid 2013-10-03 16:36:48 -07:00
fjy bc8db7daa5 1) make chat handler resource work again 
2) add more default configs
3) make examples work again
 2013-10-02 14:22:39 -07:00
Gian Merlino 62eda5020c ShardSpec: Remove isInChunk(Map<String, String>) 2013-10-01 12:50:08 -07:00
fjy 4a7a28a7f6 Merge branch 'master' into fix-config 2013-09-27 17:17:46 -07:00
Gian Merlino dc5dab8747 Fixes for property conversion, firehose registration, and the indexing service 2013-09-27 17:09:59 -07:00
fjy a79ad7bab4 make dynamic master resource configuration work again 2013-09-27 15:00:40 -07:00
fjy e404295c1f make indexing service work 2013-09-26 17:44:21 -07:00
fjy 8bc56daa66 fix things up according to code review comments 2013-09-26 11:35:45 -07:00
fjy 0461e4688f Merge branch 'master' into guice-hi 2013-09-26 11:05:03 -07:00
fjy 87259321b6 port hadoop druid indexer to new guice framework 2013-09-26 11:04:42 -07:00
Gian Merlino 45e22d98f1 Fix S3DataSegmentPuller retry bug 2013-09-25 07:56:45 -07:00
fjy 19276f6bad fix spacing issues and other code review comments 2013-09-24 14:34:09 -07:00
fjy be5bb7f2eb fix lifecycle startup/stop ordering problem with discovery module and make druid able to load local extensions 2013-09-24 14:20:56 -07:00
fjy 15843c3978 refactor how server service discovery is done 2013-09-24 10:36:26 -07:00
cheddar cc632cdb30 Make Jetty Server lazy 
JettyServerModule had been eagerly instantiating the Server object, which was causing things that didn't care about an HTTP interface to all of a sudden require host and port parameters.  The change makes the JettyServerModule only setup the bindings without eagerly instantiating the Jetty Server.  Each cli needs to register the Server class with the Lifecycle in order to make it actually get used.
 2013-09-23 16:02:24 -05:00
cheddar 98c663e75c Enable usage of SysMonitor to verify disk locations via DruidSysMonitor 2013-09-20 18:33:14 -05:00
cheddar b13ae3f405 Merge branch 'master' into fix-extensionization 
Conflicts:
	services/src/main/java/io/druid/cli/CliBroker.java
	services/src/main/java/io/druid/cli/CliCoordinator.java
 2013-09-20 17:49:13 -05:00
cheddar 5712b29c8c Fix issues with bindings and handling extensions 
The way the Guice bindings were setup previously, each process only had bindings
for the things it cared about.  This became problematic when adding extension modules
that bound everything that they could possibly need expecting that the processes would
only instantiate what they actually do need.  Guice tries to fail-fast and verifies that all
 bindings exist before it does anything, which is a problem because the extension bind
 some objects that don't necessarily have all of their dependencies bound in all processes.

The fix for this is to build a single Injector with all bindings in it and let each of the
 processes only load the things that they care about.  This also requires the use of
 Module overrides and other such interesting things, which are node done.

 In doing the fix, I also swapped out the way that the DataSegmentPusher/Puller stuff is bound, as well as made the Cassandra stuff fail if its settings are not provided.  This all of a sudden made all of the things require Cassandra's settings, so I migrated the Cassandra deep storage stuff into its own module.

 In doing these changes, I also discovered that some properties weren't properly converting for the ConvertProperties command (specifically, the properties related to data segment loading and pushing), so I fixed that.
 2013-09-20 17:45:01 -05:00
fjy f26e13c61a fix broken http endpoints 2013-09-19 17:33:58 -07:00
fjy 8d7af40313 Merge branch 'guice' of github.com:metamx/druid into guice 2013-09-18 15:59:29 -07:00
fjy b3bfcb7792 workaround for broker to plug in extensions 2013-09-18 15:59:05 -07:00
cheddar dc728b5721 Update versions of poms and make IncrementalIndexStorageAdapterTest compile 2013-09-18 17:01:03 -05:00
cheddar 6bfe2310bb Merge branch 'master' into guice 
Conflicts:
	client/pom.xml
	client/src/main/java/com/metamx/druid/query/segment/LegacySegmentSpec.java
	indexing-common/pom.xml
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolboxFactory.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskQueueTest.java
	realtime/src/main/java/com/metamx/druid/realtime/RealtimeNode.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/IrcFirehoseFactory.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/WikipediaIrcDecoder.java
	realtime/src/main/java/com/metamx/druid/realtime/plumber/RealtimePlumberSchool.java
	server/src/main/java/com/metamx/druid/BaseServerNode.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/index/v1/IncrementalIndexStorageAdapter.java
	server/src/main/java/com/metamx/druid/query/group/GroupByQueryEngine.java
 2013-09-18 16:46:54 -05:00
fjy e551afef5d fix broken master console 2013-09-18 09:41:59 -07:00
fjy 9b4c3756ed fix exception catching logic of moving segments 2013-09-15 10:27:37 -07:00
Gian Merlino 163fa14e62 [maven-release-plugin] prepare for next development iteration 2013-09-13 19:02:46 -07:00
Gian Merlino a20f3fc6dd [maven-release-plugin] prepare release druid-0.5.58 2013-09-13 19:02:40 -07:00
fjy d0b0adbb68 [maven-release-plugin] prepare for next development iteration 2013-09-13 18:57:25 -07:00
fjy 23c5fdb107 [maven-release-plugin] prepare release druid-0.5.57 2013-09-13 18:57:20 -07:00
fjy b0a09ba1f1 [maven-release-plugin] prepare for next development iteration 2013-09-13 18:45:20 -07:00
fjy b17f5c403b [maven-release-plugin] prepare release druid-0.5.56 2013-09-13 18:45:14 -07:00
Gian Merlino 22a99c7589 [maven-release-plugin] prepare for next development iteration 2013-09-13 17:59:03 -07:00
Gian Merlino 542d564453 [maven-release-plugin] prepare release druid-0.5.55 2013-09-13 17:58:56 -07:00
fjy 1ec098c010 use num processing threads for realtime query execution 2013-09-13 13:54:41 -07:00
cheddar e0738b3b22 1) Add tool to convert properties from the old settings to the new ones. 2013-09-12 17:59:23 -05:00
cheddar 044e43d231 1) Remove last TODO related to segment lifetime. Don't think we care about that anymore. 2013-09-12 14:17:11 -05:00
cheddar a001c1d32a 1) Remove TODO from ZkCoordinator. Always load the cache, 'cause I say so. 2013-09-12 13:02:40 -05:00
cheddar 2ea09e2d70 1) Clean up specification of StorageLocations 2013-09-12 12:56:56 -05:00
cheddar a2dcc45a8e 1) Remove SingleSegmentLoader and replace with OmniSegmentLoader 2013-09-12 11:47:03 -05:00
cheddar b8bd19e87c whitespace 2013-09-12 10:13:04 -05:00
cheddar 3e4a4f5566 1) Adjustments to allow for the addition of new Runnables via modules. Modules should implement CliCommandCreator and add their stuff there. 2013-09-11 17:41:20 -05:00
cheddar 6c9a107356 1) remove duplicate package initialization.initialization 2013-09-09 17:02:57 -05:00
fjy e03c623467 [maven-release-plugin] prepare for next development iteration 2013-09-05 13:48:21 -07:00
fjy 552d0c061a [maven-release-plugin] prepare release druid-0.5.54 2013-09-05 13:48:11 -07:00
fjy 5a64e2a64a [maven-release-plugin] prepare for next development iteration 2013-09-04 14:24:04 -07:00
fjy 523355d00b [maven-release-plugin] prepare release druid-0.5.53 2013-09-04 14:23:56 -07:00
fjy 567168330b [maven-release-plugin] prepare for next development iteration 2013-09-04 14:06:33 -07:00
fjy 39bda32b81 [maven-release-plugin] prepare release druid-0.5.52 2013-09-04 14:06:25 -07:00
fjy f08b8287a4 [maven-release-plugin] prepare for next development iteration 2013-09-04 13:22:19 -07:00
fjy cf9038abb4 [maven-release-plugin] prepare release druid-0.5.51 2013-09-04 13:22:11 -07:00
cheddar 491e4965ae Merge branch 'master' of github.com:metamx/druid 2013-09-04 14:43:55 -05:00
cheddar 7e4719fe51 1) Add unit test for NoSuchElementException as seen by Pablo. 
2) Fix unit test
 2013-09-04 14:43:42 -05:00
cheddar dadd9bc5ce 1) Rename MetricSelector(Factory) to ColumnSelector(Factory) 
2) Hard coded plugin as a first pass, won't actually work for anyone else.
 2013-09-04 11:19:46 -05:00
cheddar 740c70ed06 1) It's good when unit tests actually pass... 2013-08-31 17:27:15 -05:00
cheddar 3c39f90c89 1) Move Firehose interface and dependencies to druid-api 
2) Move DataSegment* interfaces and dependencies to druid-api
 2013-08-31 16:43:28 -05:00
cheddar 5ab671050e No more com.metamx.druid, it is now all io.druid! 2013-08-30 19:42:12 -05:00
cheddar bd0756e360 More stuff moved, things still compiling and tests still passing. Yay! 2013-08-30 18:58:35 -05:00
cheddar 56e2b956d0 OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.! 2013-08-30 18:21:04 -05:00
cheddar 2e26a4d8b9 1) Add back segment-related interfaces and classes to new druid-segment module 2013-08-29 17:23:16 -05:00
cheddar 386cc79420 [maven-release-plugin] prepare for next development iteration 2013-08-29 17:20:10 -05:00
cheddar 83e5e5434e [maven-release-plugin] prepare release druid-0.5.50 2013-08-29 17:20:05 -05:00
cheddar 846b3c9b94 1) Spell better 2013-08-29 16:51:05 -05:00
cheddar ca2c56d827 1) Remove PostAggregator and put it in druid-api 
2) Remove io.druid files from client
 2013-08-29 16:50:09 -05:00
cheddar 3c1b0a8a02 1) Make it compile after merging with master 2013-08-29 16:47:17 -05:00
cheddar cb90ed05b0 Revert the previous commit. After going down this path, I realized that extracting things enough to allow Queries to be extended without depending on Druid proper was going to lead down a very nasty path. So, I've decided against that. Extending queries will require a tight dependency on Druid proper. 2013-08-29 16:45:03 -05:00
cheddar 2a46086e20 1) Didn't remove the io.druid files from client. Remove those and make sure things compile 
2) Switch DefaultObjectMapper to CommonObjectMapper
3) Create new DefaultObjectMapper in client that has Query stuff registered on it by default
 2013-08-29 15:25:36 -05:00
cheddar 5ad023cd3f Merge branch 'master' into guice 2013-08-29 13:59:59 -05:00
cheddar d9a8340dc7 [maven-release-plugin] prepare for next development iteration 2013-08-29 13:29:54 -05:00
cheddar 5118e1da12 [maven-release-plugin] prepare release druid-0.5.49 2013-08-29 13:29:49 -05:00
cheddar df2465eb99 1) Add unit test for groupBy query with dimension that doesn't exist 
2) Fix group by query to work with new unit test, fixes #232
 2013-08-29 11:48:34 -05:00
cheddar 15ec8e7054 1) Remove the Searchable interface 2013-08-28 18:51:19 -05:00
cheddar c02d887cfe 1) Rewrite SearchQueryRunner to not require StorageAdapter to be "Searchable" 
2) Extract SearchQueryRunner out of SearchQueryRunnerFactory
3) Extract ColumnSelectorBitmapIndexSelector out to make it reusable
 2013-08-28 18:50:40 -05:00
cheddar 1f37e962f6 1) Delete the files that have moved to druid-api and get them through the druid-api dependency. Still need to fix Searchable on StorageAdapter, but other than that, everything should compile. 2013-08-28 16:18:40 -05:00
cheddar 9c30ced5ea 1) Move various "api" classes to io.druid packages and make sure things compile and stuff 2013-08-28 15:51:02 -05:00
cheddar fc60158273 1) What is an .Rhistory file and why is it checked in? It's not! (anymore) 2013-08-27 14:37:12 -05:00
cheddar ee1e73cfa1 1) Make it compile again after the merge 2013-08-27 14:36:01 -05:00
cheddar 8097450d8c Some things that didn't get committed with the merge for some reason!? 2013-08-27 14:29:03 -05:00
cheddar 5fa944dd26 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
	client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
	client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
	client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
	pom.xml
	server/src/main/java/com/metamx/druid/http/MasterMain.java
	server/src/main/java/com/metamx/druid/http/MasterServletModule.java
	server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
	server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
	server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
 2013-08-27 14:27:32 -05:00
cheddar 3617ac17fc 1) Eliminate ExecutorMain and have it run using the new Main! 2013-08-27 14:11:05 -05:00
cheddar 269997dc94 1) ExecutorNode is working, except for the running of the task. Need to adjust it to be able to run a task and then everything will be wonderful 2013-08-26 18:08:41 -05:00
cheddar 55dbda2046 1) Worker appears to be running! It's also now known as the MiddleManager 2013-08-23 17:59:48 -05:00
cheddar b897c2cb22 1) IndexCoordinator appears to work as the CliOverlord now, yay! 2013-08-23 14:11:34 -05:00
cheddar 344ec1ec2f [maven-release-plugin] prepare for next development iteration 2013-08-23 13:20:09 -05:00
cheddar 5b0de6eb5c [maven-release-plugin] prepare release druid-0.5.48 2013-08-23 13:20:02 -05:00
fjy 6a8c160740 update code according to code review 2013-08-22 10:46:05 -07:00
fjy 3f51ef63a4 Merge branch 'guice' into guice-rt 2013-08-22 10:20:53 -07:00
fjy 4102dbd1e0 [maven-release-plugin] prepare for next development iteration 2013-08-21 21:50:55 -07:00
fjy 0daa2a95fe [maven-release-plugin] prepare release druid-0.5.47 2013-08-21 21:50:46 -07:00
fjy b1d0f989ec [maven-release-plugin] prepare for next development iteration 2013-08-21 20:28:15 -07:00
fjy ab467bf42a [maven-release-plugin] prepare release druid-0.5.46 2013-08-21 20:28:06 -07:00
fjy 1037a48a8e [maven-release-plugin] prepare for next development iteration 2013-08-21 18:55:49 -07:00
fjy 6b9ec95f01 [maven-release-plugin] prepare release druid-0.5.45 2013-08-21 18:55:40 -07:00
fjy ed666d9d5f [maven-release-plugin] prepare for next development iteration 2013-08-20 19:37:52 -07:00
fjy 66dcfe8c3e [maven-release-plugin] prepare release druid-0.5.44 2013-08-20 19:37:44 -07:00
fjy baedacf970 [maven-release-plugin] prepare for next development iteration 2013-08-20 19:29:05 -07:00
fjy a4eb025ba7 [maven-release-plugin] prepare release druid-0.5.43 2013-08-20 19:28:56 -07:00
Gian Merlino a3793eeba4 [maven-release-plugin] prepare for next development iteration 2013-08-20 17:58:28 -07:00
Gian Merlino b792bf2ba5 [maven-release-plugin] prepare release druid-0.5.42 2013-08-20 17:58:22 -07:00
fjy 47ec240703 [maven-release-plugin] prepare for next development iteration 2013-08-15 13:47:57 -07:00
fjy 33e2fb3e21 [maven-release-plugin] prepare release druid-0.5.41 2013-08-15 13:47:48 -07:00
fjy 1fb6107a37 fix the case where RTR does not clean up a completed task on startup 2013-08-15 13:09:02 -07:00
cheddar 0c5a906a1b Merge branch 'master' of github.com:metamx/druid 2013-08-14 17:34:22 -07:00
cheddar ba5ed4c9d4 1) Fix bug with NPE thrown when requesting a dimension on a GroupBy query that is not dictionary encoded 2013-08-14 17:34:11 -07:00
Gian Merlino a0fcd02b3c [maven-release-plugin] prepare for next development iteration 2013-08-14 17:03:40 -07:00
Gian Merlino b6c445c726 [maven-release-plugin] prepare release druid-0.5.40 2013-08-14 17:03:28 -07:00
fjy 36d02ab943 [maven-release-plugin] prepare for next development iteration 2013-08-14 14:05:49 -07:00
fjy 8c841e90a4 [maven-release-plugin] prepare release druid-0.5.39 2013-08-14 14:05:39 -07:00
fjy fc1018bc3f revert removal of master loadstatus 2013-08-14 14:02:47 -07:00
fjy 761a411b16 [maven-release-plugin] prepare for next development iteration 2013-08-14 10:13:50 -07:00
fjy 4dd727d974 [maven-release-plugin] prepare release druid-0.5.38 2013-08-14 10:13:34 -07:00
cheddar c0624d9591 Merge branch 'master' of github.com:metamx/druid 2013-08-14 10:04:04 -07:00
cheddar a7ef5b8b70 1) Fix bug with SingleSegmentLoader.StorageLocation keeping track of its storage size incorrectly. Add unit test ftw. 2013-08-14 10:03:47 -07:00
cheddar 7ebe053ac1 Fix bug with segment loader counting thingie 2013-08-14 09:37:22 -07:00
cheddar 4791d0466d Merge pull request #198 from metamx/loadBalancer 
Improve runtime of segment distribution algorithm
 2013-08-13 18:47:43 -07:00
cheddar c2305a82f9 Merge pull request #209 from zanox/kafka-protobuf 
Kafka protobuf
 2013-08-13 18:38:27 -07:00
fjy 9870cab855 [maven-release-plugin] prepare for next development iteration 2013-08-13 17:21:56 -07:00
fjy e4c998cfef [maven-release-plugin] prepare release druid-0.5.37 2013-08-13 17:21:46 -07:00
fjy 85ee8bb267 port realtime to guice 2013-08-13 17:08:45 -07:00
cheddar 145e08682c 1) Add check in ServerManagerTest to make sure that the Segment has been "checked out" before the factory ever sees it. 
2) Some code readability changes to ReferenceCountingSegment
 2013-08-13 17:01:24 -07:00
cheddar a5855fb749 1) Fix ServerManager to make sure that it "checks out" the Segment before it starts processing 2013-08-13 14:51:14 -07:00
cheddar 28c7e78a91 1) Fix ReferenceCountingSegmentTest to actually work again 2013-08-13 14:28:44 -07:00
fjy 41347c82bb fix broken server manager tests 2013-08-13 12:07:45 -07:00
fjy 070d822345 fix broken UT 2013-08-13 12:01:24 -07:00
fjy 08318af7f7 Merge branch 'master' of github.com:metamx/druid 2013-08-13 11:54:32 -07:00
fjy 13e522c8a7 update the server manager unit tests to check for the case where a segment may be accessed in toYield calls before an increment occurs 2013-08-13 11:53:41 -07:00
Gian Merlino 246bd0fbaa [maven-release-plugin] prepare for next development iteration 2013-08-13 07:27:24 -07:00
Gian Merlino a2e9441693 [maven-release-plugin] prepare release druid-0.5.36 2013-08-13 07:27:18 -07:00
fjy d4fc069e01 [maven-release-plugin] prepare for next development iteration 2013-08-12 18:58:14 -07:00
fjy dd8c96c734 [maven-release-plugin] prepare release druid-0.5.35 2013-08-12 18:58:06 -07:00
cheddar bb18b162ab 1) Include new version of java-util to fix issue with resources not getting closed when an exception is thrown sometimes 
2) Whitespace changes to GroupByQueryEngine
 2013-08-12 17:57:30 -07:00
Gian Merlino 8d7a4f4493 Retries for S3TaskLogs, S3DataSegmentPusher 2013-08-12 14:27:34 -07:00
fjy 1f4e0ea40c [maven-release-plugin] prepare for next development iteration 2013-08-12 13:21:29 -07:00
fjy 1c2ad4f08c [maven-release-plugin] prepare release druid-0.5.34 2013-08-12 13:21:21 -07:00
fjy a39eb65edb change summary of segment failures to one per line 2013-08-12 13:15:35 -07:00
fjy dd20950f8a adding missing continue 2013-08-12 13:07:29 -07:00
fjy 2ec2957207 clean up the error reporting code according to code review 2013-08-12 13:04:48 -07:00
fjy 0c360c05c2 better handling of exceptions during startup 2013-08-12 12:56:59 -07:00
fjy 2f728f3478 fix zk coord to not fail during startup exceptions 2013-08-12 11:33:42 -07:00
Jan Rudert 3f32ef7bbc Merge branch 'master' of github.com:metamx/druid into kafka-protobuf 2013-08-12 17:42:40 +02:00
jan 55267ca0e4 removed @author tags 2013-08-11 13:44:13 +02:00
Dhruv Parthasarathy 01db89ac6e fixed issue with web console. Cleaned up code 2013-08-09 00:33:40 -07:00
fjy 68a3f1ab79 [maven-release-plugin] prepare for next development iteration 2013-08-08 22:15:04 -07:00
fjy ce5679554d [maven-release-plugin] prepare release druid-0.5.33 2013-08-08 22:14:55 -07:00
fjy a190269faa fix druid master segments getting stuck bug 2013-08-08 22:06:41 -07:00
Dhruv Parthasarathy fc5e15f72b merged changes with master 2013-08-08 18:11:33 -07:00
Dhruv Parthasarathy abe347efd3 merged with master 2013-08-08 17:21:57 -07:00
Dhruv Parthasarathy 764863d6f5 made volatile. fixed parens 2013-08-08 16:40:10 -07:00
Dhruv Parthasarathy 8a8ea5d028 removed test annotation from profiler 2013-08-08 16:38:24 -07:00
Dhruv Parthasarathy 01b35ec819 all tests passed. Ready for merge 2013-08-08 16:23:26 -07:00
Dhruv Parthasarathy eed9e1b373 merged with master 2013-08-08 16:14:06 -07:00
Dhruv Parthasarathy 0b24ea710c fixed paths 2013-08-08 16:10:31 -07:00
Dhruv Parthasarathy 3a298681e4 fixed formatting 2013-08-07 10:59:54 -07:00
Dhruv Parthasarathy c4f0e4d96d fixed with Erics feedback 2013-08-07 10:53:37 -07:00
fjy ddc2e87c4c [maven-release-plugin] prepare for next development iteration 2013-08-06 21:21:28 -07:00
fjy ffbcd18cdc [maven-release-plugin] prepare release druid-0.5.32 2013-08-06 21:21:18 -07:00
fjy 4617d37e28 [maven-release-plugin] prepare for next development iteration 2013-08-06 21:08:02 -07:00
fjy 53b248dc28 [maven-release-plugin] prepare release druid-0.5.31 2013-08-06 21:07:52 -07:00
Gian Merlino 99a3de1597 [maven-release-plugin] prepare for next development iteration 2013-08-06 17:41:58 -07:00
Gian Merlino c0ccfd66ea [maven-release-plugin] prepare release druid-0.5.30 2013-08-06 17:41:51 -07:00
fjy 930b089f7f [maven-release-plugin] prepare for next development iteration 2013-08-06 16:20:37 -07:00
fjy dd5f5ae49b [maven-release-plugin] prepare release druid-0.5.29 2013-08-06 16:20:29 -07:00
fjy 5f292143e0 disable merging segments with linear shard spec 2013-08-06 16:17:49 -07:00
Dhruv Parthasarathy f31ee552d0 fixed with fj comments 2013-08-06 15:03:07 -07:00
Dhruv Parthasarathy 0d5b222f8f removed SlickGrid :( 2013-08-06 14:30:35 -07:00
Dhruv Parthasarathy 6124ecd9cb fied some tests 2013-08-06 14:26:08 -07:00
cheddar eee1efdcb5 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/client/DruidServerConfig.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/index/ChatHandlerProvider.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/TaskMasterLifecycle.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
 2013-08-06 13:33:31 -07:00
cheddar 51f648ec11 1) Rename QueryToolChestModule back 2013-08-06 12:05:56 -07:00
cheddar 69f7831d82 1) Rename QueryToolChestModule because there was a case sensitivity issue, grrr 2013-08-06 12:04:04 -07:00
cheddar 9f71d42314 1) Broker works with Guice 
2) Extract ServerViewModule to handle various ServerView thingies
3) Extract QueryToolChestModule and QueryRunnerFactoryModule to reuse code for configuration of Query stuff
4) Extract QueryJettyServerInitializer to reuse between Historical and Broker nodes
5) Remove ClientMain, BrokerMain and BrokerNode
 2013-08-06 12:01:52 -07:00
Dhruv Parthasarathy ebba2d66a3 results good 2013-08-06 11:46:18 -07:00
Dhruv Parthasarathy e40d5b55ef testing speed for one run 2013-08-06 11:40:49 -07:00
Gian Merlino ba847bff32 [maven-release-plugin] prepare for next development iteration 2013-08-05 19:58:40 -07:00
Gian Merlino 7101d9ad2e [maven-release-plugin] prepare release druid-0.5.28 2013-08-05 19:58:34 -07:00
fjy db695f49c2 [maven-release-plugin] prepare for next development iteration 2013-08-05 18:26:20 -07:00
fjy c22e5b7ee0 [maven-release-plugin] prepare release druid-0.5.27 2013-08-05 18:26:10 -07:00
fjy 8ff0b53df4 [maven-release-plugin] prepare for next development iteration 2013-08-05 18:00:46 -07:00
fjy 1d3729906d [maven-release-plugin] prepare release druid-0.5.26 2013-08-05 18:00:36 -07:00
fjy 90e082463b [maven-release-plugin] prepare for next development iteration 2013-08-05 16:10:19 -07:00
fjy eb276d34d7 [maven-release-plugin] prepare release druid-0.5.25 2013-08-05 16:10:11 -07:00
Dhruv Parthasarathy 575dfe3679 replication working too 2013-08-05 14:51:58 -07:00
fjy a95d9c46e2 [maven-release-plugin] prepare for next development iteration 2013-08-05 14:47:35 -07:00
fjy 8e1f4d3256 [maven-release-plugin] prepare release druid-0.5.24 2013-08-05 14:47:26 -07:00
Dhruv Parthasarathy 1045aac4b8 finished profiling here 2013-08-05 13:53:40 -07:00
cheddar d64bc2f762 1) Eliminate Old mains() that aren't needed anymore. 2013-08-05 13:33:43 -07:00
Dhruv Parthasarathy 540c4885d7 fixes to make profiler work 2013-08-05 12:36:43 -07:00
fjy fbb1211cbc [maven-release-plugin] prepare for next development iteration 2013-08-05 11:44:36 -07:00
fjy 22a90d36b7 [maven-release-plugin] prepare release druid-0.5.23 2013-08-05 11:44:28 -07:00
Jan Rudert 7195637a41 use same hadoop version as druid master (0.20.2) 2013-08-05 09:16:57 +02:00
Dhruv Parthasarathy 1c19fb4a02 changed number of replicants 2013-08-02 17:41:47 -07:00
cheddar d66af7625c 1) Eliminate the need for SingleSegmentLoader 
2) Setup the configuration glue for ServerInventoryViews and DataSegmentAnnouncer
3) Make processes run and work again!
 2013-08-02 17:05:01 -07:00
cheddar 2361e0112a Make it all compile again... 2013-08-02 10:14:46 -07:00
Jan Rudert 37ab5ef28b Merge branch 'master' of github.com:metamx/druid into HEAD 2013-08-02 09:23:19 +02:00
Dhruv Parthasarathy 6f6a335a40 adding profiling for 55k servers 2013-08-01 23:38:44 -07:00
Dhruv Parthasarathy 9f17e8fc43 all working 2013-08-01 21:01:02 -07:00
Dhruv Parthasarathy 696b04020b wrote regression test 2013-08-01 18:28:48 -07:00
fjy 584ccac833 move scanning of workers and tasks into RTR start, simplify bootstrap, make tests better 2013-08-01 17:50:05 -07:00
Dhruv Parthasarathy 6f28eac995 fixed problem of balancer ignoring home server 2013-08-01 17:28:32 -07:00
Dhruv Parthasarathy cd3b62e123 fix to runtime params 2013-08-01 17:01:50 -07:00
cheddar 9e78bb38f5 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/QueryableNode.java
	client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
	client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
	client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
	client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
	indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
	pom.xml
	server/src/main/java/com/metamx/druid/coordination/ServerManager.java
	server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
	server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
	server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/http/MasterMain.java
	server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
	server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
	server/src/main/java/com/metamx/druid/master/DruidMaster.java
 2013-08-01 16:42:47 -07:00
cheddar 019bb5d453 1) Another whole bunch of changes to annotate things and create Modules and bind stuff. But OMFG, the compute node actually appears to be working! 
2) The compute node works with Guice
3) The compute node fires up with Guice and appears to work
4) Did I mention that the compute node, now called historical node, fires up with Guice and appears to work?
 2013-08-01 15:28:08 -07:00
cheddar fba2bc9623 Merge pull request #204 from metamx/close-handle 
add the ability to unmap mmapped files
 2013-08-01 14:09:46 -07:00
Dhruv Parthasarathy 9b340d9f6c simplified console code. Added Builder. Fixed urls 2013-08-01 14:06:22 -07:00
fjy 8f39db7804 more data to exception msgs 2013-08-01 14:04:27 -07:00
fjy 6d54ae9f81 more unit tests to test for multiple closes in different threads 2013-08-01 13:51:56 -07:00
fjy dc169d9fcd hide decrement and update close logic to call decrement 2013-08-01 13:24:56 -07:00
fjy e92e0c35a4 put all concurrency test back into server manager test, introduce better concurrency for reference counting 2013-08-01 13:13:03 -07:00
fjy b86258cc14 fix broken master UT based on changes that were discarded as part of a pull req 2013-08-01 10:44:39 -07:00
fjy 2f8351b314 fix broken ut 2013-08-01 09:41:36 -07:00
Jan Rudert 73d5ff4b94 Merge branch 'master' of github.com:metamx/druid into HEAD 2013-08-01 18:00:29 +02:00