Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,489 Commits 113 Branches 505 Tags 728 MiB
Commit Graph

657 Commits

Author SHA1 Message Date
Himadri Singh cd1e7cc65d Updated test 2013-12-19 12:01:18 +05:30
Himadri Singh b1a9ecc1cf StatusResourceTest 
misses testing reloading of modules currently
 2013-12-19 11:42:22 +05:30
Himadri Singh c58fbfe40c minor fixes 2013-12-19 10:49:38 +05:30
fjy 9b8134c73e Merge branch 'master' into az 2013-12-18 15:44:43 -08:00
Ray Sayre c969b37bac Remove unused payload column frosub-select that fails in postgres. Use boolean true/false for 'used' column so that queries run in postres 2013-12-18 13:01:36 -08:00
fjy 27bba22e6f make rules assignable across tiers 2013-12-16 15:54:42 -08:00
fjy e309f48357 Merge branch 'master' into az 
Conflicts:
	server/src/main/java/io/druid/server/coordinator/DruidCoordinator.java
 2013-12-16 14:57:54 -08:00
Xavier Léauté ac2ca0e46c separate move and archive tasks 2013-12-16 14:00:55 -08:00
Xavier Léauté a417cd5df2 add archive task 2013-12-16 13:59:15 -08:00
fjy 84233238b1 Merge branch 'master' of github.com:metamx/druid 2013-12-16 12:01:55 -08:00
fjy 0998d835b0 fix redirection issues with master and overlord 2013-12-16 12:01:52 -08:00
fjy b917ada54d Merge pull request #328 from metamx/coordinator-leader-discovery 
Coordinator: Link service announcement to leadership
 2013-12-16 12:01:24 -08:00
Gian Merlino 7759ed4517 Coordinator: Link service announcement to leadership 2013-12-16 11:28:28 -08:00
Gian Merlino d2451fa37b Coordinator: Stop databaseRuleManager in stopBeingLeader 2013-12-16 11:12:20 -08:00
Gian Merlino 079331493a ZkCoordinator: Make addSegment, addSegments idempotent 2013-12-16 10:56:14 -08:00
Himadri Singh 3752122a7b implementing comments from review 2013-12-16 15:16:28 +05:30
fjy 52cdb20f10 add better messaging and error handling 2013-12-13 15:01:07 -08:00
fjy ca7335f45f things dont work 2013-12-13 13:35:22 -08:00
Himadri Singh 65facc935d Do not load extensions on each call 2013-12-13 13:05:57 +05:30
Gian Merlino dbe93034d0 StatusResource: Remove extension versions 
They cause services to create a brand-new startup injector and reload all
modules whenever /status is requested. Not sure how bad this is, but I think
we're only supposed to have one startup injector so there may be a better
approach out there.
 2013-12-12 17:23:15 -08:00
Gian Merlino b2ea216bdf HttpClientModule: Default http timeout to PT5M 2013-12-11 15:05:16 -08:00
Gian Merlino f3cfd1d781 Introduce FileTaskLogs, and move TaskLogs module from server to indexing-service 2013-12-10 17:39:43 -08:00
fjy a262d8bf11 fix curator shutdown problem in curator inv manager test 2013-12-09 18:56:30 -08:00
fjy 8aba4ad3d2 Merge pull request #311 from metamx/versions 
Add versions to static page
 2013-12-09 17:18:16 -08:00
Himadri Singh b5f6dbc32f Code refactoring, one place! 2013-12-10 02:09:41 +05:30
Himadri Singh e6b915f1e7 druid-api version not required 2013-12-10 00:05:56 +05:30
fjy d9762ac9bd fix shutdown 2013-12-06 11:17:19 -08:00
fjy 5fe0a8f2fb fix casting 2013-12-06 11:16:27 -08:00
fjy 1fcbbc8262 fix logging 2013-12-06 11:14:39 -08:00
fjy 2c5562b6ff fix wrong shard spec 2013-12-06 10:29:22 -08:00
fjy f8ff9296bc fix logs 2013-12-06 10:26:35 -08:00
fjy 767663af92 fix how flushing plumber flushes 2013-12-06 10:25:50 -08:00
fjy 932d688022 remove unneccessary merge n persist start 2013-12-06 10:15:18 -08:00
fjy 90aa9eca47 more logs 2013-12-06 09:57:07 -08:00
fjy 986223a49d more logs 2013-12-06 09:56:47 -08:00
fjy c4b990bb27 more logging 2013-12-06 09:56:23 -08:00
fjy e7b6b5c2af fix persist with flushing plumber 2013-12-06 09:51:43 -08:00
fjy 5ba05a46f6 fix nulls in realtime persist 2013-12-05 17:59:02 -08:00
fjy 6f079bcc8f fix segment id for ii 2013-12-05 17:01:55 -08:00
fjy ae5a4ff54c add versions to incremental index segment 2013-12-05 16:47:28 -08:00
fjy 60d72a0fd3 add flushing plumber 2013-12-05 16:24:54 -08:00
fjy 241c80ef24 working pagination maybe 2013-12-05 13:37:44 -08:00
fjy 31c7ed639d first commit 2013-12-04 13:53:08 -08:00
Himadri Singh a2c8388742 No need of druid-server.version 2013-12-04 10:21:36 +05:30
fjy 929b2336a9 fix non deterministic behaviour in test 2013-12-02 17:02:24 -08:00
fjy 05f72601c4 fix request time metric and prepare for next release 2013-12-02 14:20:59 -08:00
Himadri Singh 95fafe02a7 server versions with loaded extensions versions 2013-11-27 12:21:35 +05:30
Himadri Singh abf417a1c4 version of druid-server 2013-11-26 17:47:23 +05:30
Stefán Freyr Stefánsson 386c81f0d9 Merge remote-tracking branch 'upstream/master' into rabbitmq-module 2013-11-25 22:44:25 +00:00
Himadri Singh 659493c4d2 Add versions to static page 2013-11-24 20:27:34 +05:30
Stefán Freyr Stefánsson 9eb4a7d9df Added commons-cli dependency. 2013-11-22 10:44:16 +00:00
Stefán Freyr Stefánsson 71598ee60e Moving RabbitMQ stuff to a module. 2013-11-21 18:41:06 +00:00
Xavier Léauté 3134affac9 fix NPE in DirectDruidClient 2013-11-20 17:15:26 -08:00
Xavier Léauté d0fe70a21f replace deprecated calls to getJsonFactory and createJsonParser 2013-11-20 17:14:23 -08:00
fjy c60e75a6b4 Merge branch 'master' of github.com:metamx/druid into query-resource 2013-11-20 10:02:51 -08:00
fjy e7167ae00a add new query resource 2013-11-20 10:02:43 -08:00
Stefán Freyr Stefánsson 6aafba4393 Merge remote-tracking branch 'upstream/master' into rabbitmq-lyra 2013-11-20 16:25:53 +00:00
Stefán Freyr Stefánsson f644c8ea66 Ugly quick'n'dirty way of getting lyra to work. 2013-11-19 10:18:00 +00:00
Xavier Léauté 1e9d72d95a add total segment size/count metric 2013-11-18 13:51:06 -08:00
fjy f66fe3cbbb another better default 2013-11-18 13:16:47 -08:00
fjy c5361cc100 fix serde problem in CCC and better defaults for jetty threads 2013-11-18 13:14:18 -08:00
fjy 60dbdcebac we need to explicitly set the connector thread pool based on the default number of threads in our config to accomodate how jetty 8 works 2013-11-16 11:09:42 -08:00
fjy 51aa7a2284 doesnt work yet 2013-11-11 14:04:03 -08:00
fjy 822a30658d fix broken http post emitter and prepare for next release 2013-11-08 14:01:16 -08:00
fjy 6181114acd Merge pull request #288 from metamx/abandon-rt-segments 
Realtime: Abandon segments that fail to push when shutting down
 2013-11-07 15:53:49 -08:00
Xavier Léauté 76732c25f8 increase timeout for slow machines and provide more descriptive error message 2013-11-04 14:07:43 -08:00
Gian Merlino d8d3a6958e Realtime: Abandon segments that fail to push 2013-11-02 08:57:58 -07:00
Xavier Léauté dce6e7e0eb hack to support remote repository authentication 2013-11-01 09:33:07 -07:00
fjy 5d8f1d11ec fix broken UT as a result of new druid api changes 2013-10-22 10:44:28 -07:00
daggerrz 522feb65d5 Use druid-api 0.1.2-SNAPSHOT which provides Row.getRaw() 2013-10-20 22:39:13 -04:00
fjy 2dc716bf7e fix bug and make it actually possible to load extensions 2013-10-16 11:59:01 -07:00
fjy 9796a40b92 port docs over to 0.6 and a bunch of misc fixes 2013-10-11 18:38:53 -07:00
cheddar c47fe202c7 Fix HadoopDruidIndexer to work with the new way of things 
There are multiple and sundry changes in here.

First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.

In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module.  I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.

In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api.  They are all API-level things, however, so they really belong in druid-api instead.

Lastly, I removed the druid-realtime module and put it all in druid-server.
 2013-10-09 15:15:44 -05:00
fjy af1dbe6eab fix docs for 0.6 part 1 of many 2013-10-07 14:47:04 -07:00
fjy 703b674800 add availability zone info to autoscaling 2013-10-07 12:16:50 -07:00
fjy ac330f72bb first set of changes to standarize the naming convention we use in druid 2013-10-03 16:36:48 -07:00
fjy bc8db7daa5 1) make chat handler resource work again 
2) add more default configs
3) make examples work again
 2013-10-02 14:22:39 -07:00
Gian Merlino 62eda5020c ShardSpec: Remove isInChunk(Map<String, String>) 2013-10-01 12:50:08 -07:00
fjy 4a7a28a7f6 Merge branch 'master' into fix-config 2013-09-27 17:17:46 -07:00
Gian Merlino dc5dab8747 Fixes for property conversion, firehose registration, and the indexing service 2013-09-27 17:09:59 -07:00
fjy a79ad7bab4 make dynamic master resource configuration work again 2013-09-27 15:00:40 -07:00
fjy e404295c1f make indexing service work 2013-09-26 17:44:21 -07:00
fjy 8bc56daa66 fix things up according to code review comments 2013-09-26 11:35:45 -07:00
fjy 0461e4688f Merge branch 'master' into guice-hi 2013-09-26 11:05:03 -07:00
fjy 87259321b6 port hadoop druid indexer to new guice framework 2013-09-26 11:04:42 -07:00
Gian Merlino 45e22d98f1 Fix S3DataSegmentPuller retry bug 2013-09-25 07:56:45 -07:00
fjy 19276f6bad fix spacing issues and other code review comments 2013-09-24 14:34:09 -07:00
fjy be5bb7f2eb fix lifecycle startup/stop ordering problem with discovery module and make druid able to load local extensions 2013-09-24 14:20:56 -07:00
fjy 15843c3978 refactor how server service discovery is done 2013-09-24 10:36:26 -07:00
cheddar cc632cdb30 Make Jetty Server lazy 
JettyServerModule had been eagerly instantiating the Server object, which was causing things that didn't care about an HTTP interface to all of a sudden require host and port parameters.  The change makes the JettyServerModule only setup the bindings without eagerly instantiating the Jetty Server.  Each cli needs to register the Server class with the Lifecycle in order to make it actually get used.
 2013-09-23 16:02:24 -05:00
cheddar 98c663e75c Enable usage of SysMonitor to verify disk locations via DruidSysMonitor 2013-09-20 18:33:14 -05:00
cheddar b13ae3f405 Merge branch 'master' into fix-extensionization 
Conflicts:
	services/src/main/java/io/druid/cli/CliBroker.java
	services/src/main/java/io/druid/cli/CliCoordinator.java
 2013-09-20 17:49:13 -05:00
cheddar 5712b29c8c Fix issues with bindings and handling extensions 
The way the Guice bindings were setup previously, each process only had bindings
for the things it cared about.  This became problematic when adding extension modules
that bound everything that they could possibly need expecting that the processes would
only instantiate what they actually do need.  Guice tries to fail-fast and verifies that all
 bindings exist before it does anything, which is a problem because the extension bind
 some objects that don't necessarily have all of their dependencies bound in all processes.

The fix for this is to build a single Injector with all bindings in it and let each of the
 processes only load the things that they care about.  This also requires the use of
 Module overrides and other such interesting things, which are node done.

 In doing the fix, I also swapped out the way that the DataSegmentPusher/Puller stuff is bound, as well as made the Cassandra stuff fail if its settings are not provided.  This all of a sudden made all of the things require Cassandra's settings, so I migrated the Cassandra deep storage stuff into its own module.

 In doing these changes, I also discovered that some properties weren't properly converting for the ConvertProperties command (specifically, the properties related to data segment loading and pushing), so I fixed that.
 2013-09-20 17:45:01 -05:00
fjy f26e13c61a fix broken http endpoints 2013-09-19 17:33:58 -07:00
fjy b3bfcb7792 workaround for broker to plug in extensions 2013-09-18 15:59:05 -07:00
fjy e551afef5d fix broken master console 2013-09-18 09:41:59 -07:00
fjy 9b4c3756ed fix exception catching logic of moving segments 2013-09-15 10:27:37 -07:00
cheddar e0738b3b22 1) Add tool to convert properties from the old settings to the new ones. 2013-09-12 17:59:23 -05:00
cheddar 044e43d231 1) Remove last TODO related to segment lifetime. Don't think we care about that anymore. 2013-09-12 14:17:11 -05:00
cheddar a001c1d32a 1) Remove TODO from ZkCoordinator. Always load the cache, 'cause I say so. 2013-09-12 13:02:40 -05:00