6181114acd
Merge pull request #288 from metamx/abandon-rt-segments
...
Realtime: Abandon segments that fail to push when shutting down
2013-11-07 15:53:49 -08:00
ee15fd9f85
[maven-release-plugin] prepare for next development iteration
2013-11-05 15:42:37 -08:00
713015199a
[maven-release-plugin] prepare release druid-0.6.1
2013-11-05 15:42:33 -08:00
76732c25f8
increase timeout for slow machines and provide more descriptive error message
2013-11-04 14:07:43 -08:00
d8d3a6958e
Realtime: Abandon segments that fail to push
2013-11-02 08:57:58 -07:00
dce6e7e0eb
hack to support remote repository authentication
2013-11-01 09:33:07 -07:00
5d8f1d11ec
fix broken UT as a result of new druid api changes
2013-10-22 10:44:28 -07:00
522feb65d5
Use druid-api 0.1.2-SNAPSHOT which provides Row.getRaw()
2013-10-20 22:39:13 -04:00
a684885839
[maven-release-plugin] prepare for next development iteration
2013-10-18 15:48:16 -07:00
88dcfe9a94
[maven-release-plugin] prepare release druid-0.6.0
2013-10-18 15:48:12 -07:00
2dc716bf7e
fix bug and make it actually possible to load extensions
2013-10-16 11:59:01 -07:00
9796a40b92
port docs over to 0.6 and a bunch of misc fixes
2013-10-11 18:38:53 -07:00
a9a723bd11
clean up poms, add a new loading your own data tutorial, add new validation, clean up logs
2013-10-09 15:42:39 -07:00
c47fe202c7
Fix HadoopDruidIndexer to work with the new way of things
...
There are multiple and sundry changes in here.
First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.
In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module. I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.
In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api. They are all API-level things, however, so they really belong in druid-api instead.
Lastly, I removed the druid-realtime module and put it all in druid-server.
2013-10-09 15:15:44 -05:00
af1dbe6eab
fix docs for 0.6 part 1 of many
2013-10-07 14:47:04 -07:00
703b674800
add availability zone info to autoscaling
2013-10-07 12:16:50 -07:00
ac330f72bb
first set of changes to standarize the naming convention we use in druid
2013-10-03 16:36:48 -07:00
bc8db7daa5
1) make chat handler resource work again
...
2) add more default configs
3) make examples work again
2013-10-02 14:22:39 -07:00
62eda5020c
ShardSpec: Remove isInChunk(Map<String, String>)
2013-10-01 12:50:08 -07:00
4a7a28a7f6
Merge branch 'master' into fix-config
2013-09-27 17:17:46 -07:00
dc5dab8747
Fixes for property conversion, firehose registration, and the indexing service
2013-09-27 17:09:59 -07:00
a79ad7bab4
make dynamic master resource configuration work again
2013-09-27 15:00:40 -07:00
e404295c1f
make indexing service work
2013-09-26 17:44:21 -07:00
8bc56daa66
fix things up according to code review comments
2013-09-26 11:35:45 -07:00
0461e4688f
Merge branch 'master' into guice-hi
2013-09-26 11:05:03 -07:00
87259321b6
port hadoop druid indexer to new guice framework
2013-09-26 11:04:42 -07:00
45e22d98f1
Fix S3DataSegmentPuller retry bug
2013-09-25 07:56:45 -07:00
19276f6bad
fix spacing issues and other code review comments
2013-09-24 14:34:09 -07:00
be5bb7f2eb
fix lifecycle startup/stop ordering problem with discovery module and make druid able to load local extensions
2013-09-24 14:20:56 -07:00
15843c3978
refactor how server service discovery is done
2013-09-24 10:36:26 -07:00
cc632cdb30
Make Jetty Server lazy
...
JettyServerModule had been eagerly instantiating the Server object, which was causing things that didn't care about an HTTP interface to all of a sudden require host and port parameters. The change makes the JettyServerModule only setup the bindings without eagerly instantiating the Jetty Server. Each cli needs to register the Server class with the Lifecycle in order to make it actually get used.
2013-09-23 16:02:24 -05:00
98c663e75c
Enable usage of SysMonitor to verify disk locations via DruidSysMonitor
2013-09-20 18:33:14 -05:00
b13ae3f405
Merge branch 'master' into fix-extensionization
...
Conflicts:
services/src/main/java/io/druid/cli/CliBroker.java
services/src/main/java/io/druid/cli/CliCoordinator.java
2013-09-20 17:49:13 -05:00
5712b29c8c
Fix issues with bindings and handling extensions
...
The way the Guice bindings were setup previously, each process only had bindings
for the things it cared about. This became problematic when adding extension modules
that bound everything that they could possibly need expecting that the processes would
only instantiate what they actually do need. Guice tries to fail-fast and verifies that all
bindings exist before it does anything, which is a problem because the extension bind
some objects that don't necessarily have all of their dependencies bound in all processes.
The fix for this is to build a single Injector with all bindings in it and let each of the
processes only load the things that they care about. This also requires the use of
Module overrides and other such interesting things, which are node done.
In doing the fix, I also swapped out the way that the DataSegmentPusher/Puller stuff is bound, as well as made the Cassandra stuff fail if its settings are not provided. This all of a sudden made all of the things require Cassandra's settings, so I migrated the Cassandra deep storage stuff into its own module.
In doing these changes, I also discovered that some properties weren't properly converting for the ConvertProperties command (specifically, the properties related to data segment loading and pushing), so I fixed that.
2013-09-20 17:45:01 -05:00
f26e13c61a
fix broken http endpoints
2013-09-19 17:33:58 -07:00
8d7af40313
Merge branch 'guice' of github.com:metamx/druid into guice
2013-09-18 15:59:29 -07:00
b3bfcb7792
workaround for broker to plug in extensions
2013-09-18 15:59:05 -07:00
dc728b5721
Update versions of poms and make IncrementalIndexStorageAdapterTest compile
2013-09-18 17:01:03 -05:00
6bfe2310bb
Merge branch 'master' into guice
...
Conflicts:
client/pom.xml
client/src/main/java/com/metamx/druid/query/segment/LegacySegmentSpec.java
indexing-common/pom.xml
indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolboxFactory.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/task/RealtimeIndexTask.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskQueueTest.java
realtime/src/main/java/com/metamx/druid/realtime/RealtimeNode.java
realtime/src/main/java/com/metamx/druid/realtime/firehose/IrcFirehoseFactory.java
realtime/src/main/java/com/metamx/druid/realtime/firehose/WikipediaIrcDecoder.java
realtime/src/main/java/com/metamx/druid/realtime/plumber/RealtimePlumberSchool.java
server/src/main/java/com/metamx/druid/BaseServerNode.java
server/src/main/java/com/metamx/druid/http/ComputeNode.java
server/src/main/java/com/metamx/druid/index/v1/IncrementalIndexStorageAdapter.java
server/src/main/java/com/metamx/druid/query/group/GroupByQueryEngine.java
2013-09-18 16:46:54 -05:00
e551afef5d
fix broken master console
2013-09-18 09:41:59 -07:00
9b4c3756ed
fix exception catching logic of moving segments
2013-09-15 10:27:37 -07:00
163fa14e62
[maven-release-plugin] prepare for next development iteration
2013-09-13 19:02:46 -07:00
a20f3fc6dd
[maven-release-plugin] prepare release druid-0.5.58
2013-09-13 19:02:40 -07:00
d0b0adbb68
[maven-release-plugin] prepare for next development iteration
2013-09-13 18:57:25 -07:00
23c5fdb107
[maven-release-plugin] prepare release druid-0.5.57
2013-09-13 18:57:20 -07:00
b0a09ba1f1
[maven-release-plugin] prepare for next development iteration
2013-09-13 18:45:20 -07:00
b17f5c403b
[maven-release-plugin] prepare release druid-0.5.56
2013-09-13 18:45:14 -07:00
22a99c7589
[maven-release-plugin] prepare for next development iteration
2013-09-13 17:59:03 -07:00
542d564453
[maven-release-plugin] prepare release druid-0.5.55
2013-09-13 17:58:56 -07:00
1ec098c010
use num processing threads for realtime query execution
2013-09-13 13:54:41 -07:00
e0738b3b22
1) Add tool to convert properties from the old settings to the new ones.
2013-09-12 17:59:23 -05:00
044e43d231
1) Remove last TODO related to segment lifetime. Don't think we care about that anymore.
2013-09-12 14:17:11 -05:00
a001c1d32a
1) Remove TODO from ZkCoordinator. Always load the cache, 'cause I say so.
2013-09-12 13:02:40 -05:00
2ea09e2d70
1) Clean up specification of StorageLocations
2013-09-12 12:56:56 -05:00
a2dcc45a8e
1) Remove SingleSegmentLoader and replace with OmniSegmentLoader
2013-09-12 11:47:03 -05:00
b8bd19e87c
whitespace
2013-09-12 10:13:04 -05:00
3e4a4f5566
1) Adjustments to allow for the addition of new Runnables via modules. Modules should implement CliCommandCreator and add their stuff there.
2013-09-11 17:41:20 -05:00
6c9a107356
1) remove duplicate package initialization.initialization
2013-09-09 17:02:57 -05:00
e03c623467
[maven-release-plugin] prepare for next development iteration
2013-09-05 13:48:21 -07:00
552d0c061a
[maven-release-plugin] prepare release druid-0.5.54
2013-09-05 13:48:11 -07:00
5a64e2a64a
[maven-release-plugin] prepare for next development iteration
2013-09-04 14:24:04 -07:00
523355d00b
[maven-release-plugin] prepare release druid-0.5.53
2013-09-04 14:23:56 -07:00
567168330b
[maven-release-plugin] prepare for next development iteration
2013-09-04 14:06:33 -07:00
39bda32b81
[maven-release-plugin] prepare release druid-0.5.52
2013-09-04 14:06:25 -07:00
f08b8287a4
[maven-release-plugin] prepare for next development iteration
2013-09-04 13:22:19 -07:00
cf9038abb4
[maven-release-plugin] prepare release druid-0.5.51
2013-09-04 13:22:11 -07:00
491e4965ae
Merge branch 'master' of github.com:metamx/druid
2013-09-04 14:43:55 -05:00
7e4719fe51
1) Add unit test for NoSuchElementException as seen by Pablo.
...
2) Fix unit test
2013-09-04 14:43:42 -05:00
dadd9bc5ce
1) Rename MetricSelector(Factory) to ColumnSelector(Factory)
...
2) Hard coded plugin as a first pass, won't actually work for anyone else.
2013-09-04 11:19:46 -05:00
740c70ed06
1) It's good when unit tests actually pass...
2013-08-31 17:27:15 -05:00
3c39f90c89
1) Move Firehose interface and dependencies to druid-api
...
2) Move DataSegment* interfaces and dependencies to druid-api
2013-08-31 16:43:28 -05:00
5ab671050e
No more com.metamx.druid, it is now all io.druid!
2013-08-30 19:42:12 -05:00
bd0756e360
More stuff moved, things still compiling and tests still passing. Yay!
2013-08-30 18:58:35 -05:00
56e2b956d0
OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.!
2013-08-30 18:21:04 -05:00
2e26a4d8b9
1) Add back segment-related interfaces and classes to new druid-segment module
2013-08-29 17:23:16 -05:00
386cc79420
[maven-release-plugin] prepare for next development iteration
2013-08-29 17:20:10 -05:00
83e5e5434e
[maven-release-plugin] prepare release druid-0.5.50
2013-08-29 17:20:05 -05:00
846b3c9b94
1) Spell better
2013-08-29 16:51:05 -05:00
ca2c56d827
1) Remove PostAggregator and put it in druid-api
...
2) Remove io.druid files from client
2013-08-29 16:50:09 -05:00
3c1b0a8a02
1) Make it compile after merging with master
2013-08-29 16:47:17 -05:00
cb90ed05b0
Revert the previous commit. After going down this path, I realized that extracting things enough to allow Queries to be extended without depending on Druid proper was going to lead down a very nasty path. So, I've decided against that. Extending queries will require a tight dependency on Druid proper.
2013-08-29 16:45:03 -05:00
2a46086e20
1) Didn't remove the io.druid files from client. Remove those and make sure things compile
...
2) Switch DefaultObjectMapper to CommonObjectMapper
3) Create new DefaultObjectMapper in client that has Query stuff registered on it by default
2013-08-29 15:25:36 -05:00
5ad023cd3f
Merge branch 'master' into guice
2013-08-29 13:59:59 -05:00
d9a8340dc7
[maven-release-plugin] prepare for next development iteration
2013-08-29 13:29:54 -05:00
5118e1da12
[maven-release-plugin] prepare release druid-0.5.49
2013-08-29 13:29:49 -05:00
df2465eb99
1) Add unit test for groupBy query with dimension that doesn't exist
...
2) Fix group by query to work with new unit test, fixes #232
2013-08-29 11:48:34 -05:00
15ec8e7054
1) Remove the Searchable interface
2013-08-28 18:51:19 -05:00
c02d887cfe
1) Rewrite SearchQueryRunner to not require StorageAdapter to be "Searchable"
...
2) Extract SearchQueryRunner out of SearchQueryRunnerFactory
3) Extract ColumnSelectorBitmapIndexSelector out to make it reusable
2013-08-28 18:50:40 -05:00
1f37e962f6
1) Delete the files that have moved to druid-api and get them through the druid-api dependency. Still need to fix Searchable on StorageAdapter, but other than that, everything should compile.
2013-08-28 16:18:40 -05:00
9c30ced5ea
1) Move various "api" classes to io.druid packages and make sure things compile and stuff
2013-08-28 15:51:02 -05:00
fc60158273
1) What is an .Rhistory file and why is it checked in? It's not! (anymore)
2013-08-27 14:37:12 -05:00
ee1e73cfa1
1) Make it compile again after the merge
2013-08-27 14:36:01 -05:00
8097450d8c
Some things that didn't get committed with the merge for some reason!?
2013-08-27 14:29:03 -05:00
5fa944dd26
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
pom.xml
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/http/MasterServletModule.java
server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
2013-08-27 14:27:32 -05:00
3617ac17fc
1) Eliminate ExecutorMain and have it run using the new Main!
2013-08-27 14:11:05 -05:00
269997dc94
1) ExecutorNode is working, except for the running of the task. Need to adjust it to be able to run a task and then everything will be wonderful
2013-08-26 18:08:41 -05:00
55dbda2046
1) Worker appears to be running! It's also now known as the MiddleManager
2013-08-23 17:59:48 -05:00
b897c2cb22
1) IndexCoordinator appears to work as the CliOverlord now, yay!
2013-08-23 14:11:34 -05:00
344ec1ec2f
[maven-release-plugin] prepare for next development iteration
2013-08-23 13:20:09 -05:00
5b0de6eb5c
[maven-release-plugin] prepare release druid-0.5.48
2013-08-23 13:20:02 -05:00
fjy
fjy
Xavier Léauté
Gian Merlino
daggerrz
cheddar