189b3e2b9b
Revert "Merge branch 'determine-partitions-improvements'"
...
This reverts commit 7ad228ceb59c55e2b779
2014-02-14 12:47:34 -08:00
48d0c37f98
documentation for random partition spec
2014-02-05 15:30:44 +05:30
bacc72415f
correct locking and partitionsSpec
2014-02-05 03:17:47 +05:30
569452121e
fix partitioner for loca mode
2014-01-31 21:59:17 +05:30
82b748ad43
review comments
2014-01-31 20:19:33 +05:30
97e5d68635
determine intervals working with determine partitions
2014-01-31 19:04:52 +05:30
5fd76067cd
remove logging and use new determine partition job
2014-01-31 13:51:38 +05:30
7ca87d59df
Determine partitions using cardinality
2014-01-31 00:49:11 +05:30
f898c29e20
fix batch indexing and prepare for next release
2014-01-17 15:52:04 -08:00
3b17c4c03c
a whole bunch of docs and fixes
2014-01-13 18:01:56 -08:00
1ecc94cfb6
another attempt at index task
2014-01-10 17:56:22 -08:00
52746b8ea6
fix hadoop intake's parser exception catching (was too specific)
2013-12-19 07:04:47 +01:00
a1c09df17f
make the hadoop index task work again
2013-10-16 09:45:17 -07:00
c47fe202c7
Fix HadoopDruidIndexer to work with the new way of things
...
There are multiple and sundry changes in here.
First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.
In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module. I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.
In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api. They are all API-level things, however, so they really belong in druid-api instead.
Lastly, I removed the druid-realtime module and put it all in druid-server.
2013-10-09 15:15:44 -05:00
a79ad7bab4
make dynamic master resource configuration work again
2013-09-27 15:00:40 -07:00
8bc56daa66
fix things up according to code review comments
2013-09-26 11:35:45 -07:00
87259321b6
port hadoop druid indexer to new guice framework
2013-09-26 11:04:42 -07:00
3c39f90c89
1) Move Firehose interface and dependencies to druid-api
...
2) Move DataSegment* interfaces and dependencies to druid-api
2013-08-31 16:43:28 -05:00
5ab671050e
No more com.metamx.druid, it is now all io.druid!
2013-08-30 19:42:12 -05:00
bd0756e360
More stuff moved, things still compiling and tests still passing. Yay!
2013-08-30 18:58:35 -05:00
56e2b956d0
OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.!
2013-08-30 18:21:04 -05:00
2a46086e20
1) Didn't remove the io.druid files from client. Remove those and make sure things compile
...
2) Switch DefaultObjectMapper to CommonObjectMapper
3) Create new DefaultObjectMapper in client that has Query stuff registered on it by default
2013-08-29 15:25:36 -05:00
9c30ced5ea
1) Move various "api" classes to io.druid packages and make sure things compile and stuff
2013-08-28 15:51:02 -05:00
5fa944dd26
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
pom.xml
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/http/MasterServletModule.java
server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
2013-08-27 14:27:32 -05:00
d11d0a8284
fix according to code review
2013-08-22 10:49:46 -07:00
778fd0f10e
Fix persist of empty indexes in index generator job
2013-08-22 10:16:43 -07:00
eee1efdcb5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/client/DruidServerConfig.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/index/ChatHandlerProvider.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/TaskMasterLifecycle.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
2013-08-06 13:33:31 -07:00
3c808b15c3
1) Fix HadoopDruidIndexerConfigTest to actually verify the current correct behavior.
2013-08-05 11:37:20 -07:00
2b71505421
1) Fix HadoopDruidIndexerConfig to no longer replace ":" with "_" on the segmentOutputDir. The segmentOutputDir is user-supplied so they should have the ability to just not set a bad directory.
2013-08-05 11:22:26 -07:00
2361e0112a
Make it all compile again...
2013-08-02 10:14:46 -07:00
9e78bb38f5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/QueryableNode.java
client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
pom.xml
server/src/main/java/com/metamx/druid/coordination/ServerManager.java
server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
server/src/main/java/com/metamx/druid/http/ComputeNode.java
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-08-01 16:42:47 -07:00
ad087a7a22
correct segment path for hadoop indexer
2013-07-10 09:21:45 +02:00
2f56c24259
1) Inject IndexingServiceClient
...
2) Switch all the DBI references to IDBI
2013-06-07 17:37:33 -07:00
f68df7ab69
1) Make tests work and continue trying to make the DruidMaster start up with just Guice
2013-06-07 12:01:46 -07:00
42cc87a294
Merge branch 'master' into refactor-indexing
...
Conflicts:
indexing-service/src/main/java/com/metamx/druid/indexing/common/task/IndexTask.java
pom.xml
2013-05-31 17:28:59 -07:00
08d84001ba
Merge branch 'master' into refactor-indexing
2013-05-16 16:03:29 -07:00
26e0eb62cb
merge and other refactorings
2013-05-15 17:28:08 -07:00
fjy
nishantmonu51
Hagen Rother
cheddar
Jan Rudert