Commit Graph

508 Commits

Author SHA1 Message Date
fjy a684885839 [maven-release-plugin] prepare for next development iteration 2013-10-18 15:48:16 -07:00
fjy 88dcfe9a94 [maven-release-plugin] prepare release druid-0.6.0 2013-10-18 15:48:12 -07:00
fjy 5bca36247f fix more docs 2013-10-18 15:37:43 -07:00
fjy 9796a40b92 port docs over to 0.6 and a bunch of misc fixes 2013-10-11 18:38:53 -07:00
fjy 560981acbf a ton of fixes to docs 2013-10-10 15:05:01 -07:00
fjy a9a723bd11 clean up poms, add a new loading your own data tutorial, add new validation, clean up logs 2013-10-09 15:42:39 -07:00
fjy 3364b9ae5e Merge pull request #262 from metamx/fix-hadoop
Fix HadoopDruidIndexer to work with the new way of things
2013-10-09 13:27:49 -07:00
cheddar c47fe202c7 Fix HadoopDruidIndexer to work with the new way of things
There are multiple and sundry changes in here.

First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.

In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module.  I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.

In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api.  They are all API-level things, however, so they really belong in druid-api instead.

Lastly, I removed the druid-realtime module and put it all in druid-server.
2013-10-09 15:15:44 -05:00
Russell Jurney 806bc0f58c Changed webstream example to second indexGranularity. 2013-10-09 12:23:00 -07:00
Russell Jurney 4e30a14438 Reverted minute/second change. 2013-10-08 13:27:39 -07:00
Russell Jurney caa3c663f5 Changed wikipedia example to second indexGranularity. 2013-10-08 13:05:38 -07:00
fjy af1dbe6eab fix docs for 0.6 part 1 of many 2013-10-07 14:47:04 -07:00
fjy cdbbf445fd remove ec2 startup files; they are obsoleted by whirr 2013-10-06 14:22:44 -07:00
fjy ac330f72bb first set of changes to standarize the naming convention we use in druid 2013-10-03 16:36:48 -07:00
fjy bf157ea8ef fix the example assembly and have better error logging for rabbitmq 2013-10-03 10:54:25 -07:00
fjy bc8db7daa5 1) make chat handler resource work again
2) add more default configs
3) make examples work again
2013-10-02 14:22:39 -07:00
cheddar 5712b29c8c Fix issues with bindings and handling extensions
The way the Guice bindings were setup previously, each process only had bindings
for the things it cared about.  This became problematic when adding extension modules
that bound everything that they could possibly need expecting that the processes would
only instantiate what they actually do need.  Guice tries to fail-fast and verifies that all
 bindings exist before it does anything, which is a problem because the extension bind
 some objects that don't necessarily have all of their dependencies bound in all processes.

The fix for this is to build a single Injector with all bindings in it and let each of the
 processes only load the things that they care about.  This also requires the use of
 Module overrides and other such interesting things, which are node done.

 In doing the fix, I also swapped out the way that the DataSegmentPusher/Puller stuff is bound, as well as made the Cassandra stuff fail if its settings are not provided.  This all of a sudden made all of the things require Cassandra's settings, so I migrated the Cassandra deep storage stuff into its own module.

 In doing these changes, I also discovered that some properties weren't properly converting for the ConvertProperties command (specifically, the properties related to data segment loading and pushing), so I fixed that.
2013-09-20 17:45:01 -05:00
cheddar dc728b5721 Update versions of poms and make IncrementalIndexStorageAdapterTest compile 2013-09-18 17:01:03 -05:00
cheddar 6bfe2310bb Merge branch 'master' into guice
Conflicts:
	client/pom.xml
	client/src/main/java/com/metamx/druid/query/segment/LegacySegmentSpec.java
	indexing-common/pom.xml
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolboxFactory.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskQueueTest.java
	realtime/src/main/java/com/metamx/druid/realtime/RealtimeNode.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/IrcFirehoseFactory.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/WikipediaIrcDecoder.java
	realtime/src/main/java/com/metamx/druid/realtime/plumber/RealtimePlumberSchool.java
	server/src/main/java/com/metamx/druid/BaseServerNode.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/index/v1/IncrementalIndexStorageAdapter.java
	server/src/main/java/com/metamx/druid/query/group/GroupByQueryEngine.java
2013-09-18 16:46:54 -05:00
fjy cabae7993d port over multi threaded realtime and also fix broken realtime nodes that can't start up 2013-09-16 16:03:47 -07:00
Gian Merlino 163fa14e62 [maven-release-plugin] prepare for next development iteration 2013-09-13 19:02:46 -07:00
Gian Merlino a20f3fc6dd [maven-release-plugin] prepare release druid-0.5.58 2013-09-13 19:02:40 -07:00
fjy d0b0adbb68 [maven-release-plugin] prepare for next development iteration 2013-09-13 18:57:25 -07:00
fjy 23c5fdb107 [maven-release-plugin] prepare release druid-0.5.57 2013-09-13 18:57:20 -07:00
fjy b0a09ba1f1 [maven-release-plugin] prepare for next development iteration 2013-09-13 18:45:20 -07:00
fjy b17f5c403b [maven-release-plugin] prepare release druid-0.5.56 2013-09-13 18:45:14 -07:00
Gian Merlino 22a99c7589 [maven-release-plugin] prepare for next development iteration 2013-09-13 17:59:03 -07:00
Gian Merlino 542d564453 [maven-release-plugin] prepare release druid-0.5.55 2013-09-13 17:58:56 -07:00
fjy 87dc839fb2 fix example properties for real time node 2013-09-13 14:36:42 -07:00
fjy e03c623467 [maven-release-plugin] prepare for next development iteration 2013-09-05 13:48:21 -07:00
fjy 552d0c061a [maven-release-plugin] prepare release druid-0.5.54 2013-09-05 13:48:11 -07:00
fjy 5a64e2a64a [maven-release-plugin] prepare for next development iteration 2013-09-04 14:24:04 -07:00
fjy 523355d00b [maven-release-plugin] prepare release druid-0.5.53 2013-09-04 14:23:56 -07:00
fjy 898f937f3c also update wikipedia example query.body 2013-09-04 14:20:25 -07:00
fjy 567168330b [maven-release-plugin] prepare for next development iteration 2013-09-04 14:06:33 -07:00
fjy 39bda32b81 [maven-release-plugin] prepare release druid-0.5.52 2013-09-04 14:06:25 -07:00
fjy 9f391c2bdd update example wiki realtime spec to match publically avaiable segments 2013-09-04 13:59:23 -07:00
fjy f08b8287a4 [maven-release-plugin] prepare for next development iteration 2013-09-04 13:22:19 -07:00
fjy cf9038abb4 [maven-release-plugin] prepare release druid-0.5.51 2013-09-04 13:22:11 -07:00
fjy 301e1896e7 Merge pull request #236 from rjurney/master
Edited assembly.xml to exclude the selfcontained jars, and edited path o...
2013-09-04 13:03:15 -07:00
Russell Jurney a11eae4144 Edited assembly.xml to exclude the selfcontained jars, and edited path of run_example_server to use the lib/ directory instead of selfcontained jars, thus reducing the size of the assembly tarball from 150MB to 50MB. 2013-09-04 12:53:38 -07:00
cheddar 3c39f90c89 1) Move Firehose interface and dependencies to druid-api
2) Move DataSegment* interfaces and dependencies to druid-api
2013-08-31 16:43:28 -05:00
cheddar 5ab671050e No more com.metamx.druid, it is now all io.druid! 2013-08-30 19:42:12 -05:00
cheddar bd0756e360 More stuff moved, things still compiling and tests still passing. Yay! 2013-08-30 18:58:35 -05:00
cheddar 56e2b956d0 OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.! 2013-08-30 18:21:04 -05:00
cheddar 386cc79420 [maven-release-plugin] prepare for next development iteration 2013-08-29 17:20:10 -05:00
cheddar 83e5e5434e [maven-release-plugin] prepare release druid-0.5.50 2013-08-29 17:20:05 -05:00
Russell Jurney 3e9d2d497d Changed port number for realtime server from 8080 to 8083 to make the exaxmples work with the new realtime config. 2013-08-29 15:14:09 -07:00
cheddar 5ad023cd3f Merge branch 'master' into guice 2013-08-29 13:59:59 -05:00
cheddar d9a8340dc7 [maven-release-plugin] prepare for next development iteration 2013-08-29 13:29:54 -05:00