Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,519 Commits 111 Branches 503 Tags 726 MiB
Commit Graph

308 Commits

Author SHA1 Message Date
fjy 88c592ccaa add more meaningful request logging 2014-04-10 10:37:25 -07:00
nishantmonu51 ea8b2dffd5 Add chatHandlerResource 2014-04-09 14:25:18 +05:30
fjy 7db756f819 prepare for next release 2014-04-08 16:25:59 -07:00
fjy a4df23f092 prepare for next release 2014-04-08 11:42:59 -07:00
nishantmonu51 c6f4b344cb add bindings for chat handler provider 2014-04-08 11:56:23 +05:30
fjy 4ef2bbbfc1 prepare for next release 2014-04-07 14:37:02 -07:00
fjy 2e9579a146 prepare for next release 2014-04-07 12:43:30 -07:00
fjy 1843316db6 commonalize event receiver firehose 2014-04-03 20:46:36 -07:00
fjy 5d8d506fe8 fix commented out filters 2014-04-03 17:06:28 -07:00
fjy d9062d91fa fix endpoint bugs and more docs 2014-04-03 17:01:33 -07:00
fjy b0bf08554a fix broken build 2014-04-02 22:22:42 -07:00
fjy e53d107472 revert broken cache 2014-04-02 22:12:22 -07:00
fjy ea9f31bbe0 learn to math and prepare for next release 2014-04-02 21:31:17 -07:00
fjy f185d215e1 fix jetty default thread pool size for jetty 9 and prepare for next release 2014-04-02 21:01:07 -07:00
fjy fc94b847c3 prepare for next release 2014-04-02 17:31:19 -07:00
fjy 7547e989f0 Merge pull request #456 from metamx/processing-threads-default 
more sensible defaults
 2014-04-02 18:25:43 -06:00
Xavier Léauté 2eb9493010 fix documentation link 2014-04-02 15:27:49 -07:00
fjy 1866cbad1b remove unneeded comment 2014-04-01 18:15:51 -07:00
fjy eaec0e3ab2 cleanup 2014-04-01 17:56:25 -07:00
fjy 6952eb7878 asynch servlet for query forwarding 2014-04-01 17:51:28 -07:00
nishantmonu51 06b53f02a8 upgrade default version in CluHadoopIndexer and merge from master 2014-03-27 03:37:45 +05:30
fjy 5169249ed5 Merge pull request #429 from metamx/compute-populate-cache 
Enable historical node to populate cache
 2014-03-25 12:03:26 -06:00
fjy eec07ad45f Merge branch 'master' of github.com:metamx/druid 
Conflicts:
	services/src/main/java/io/druid/cli/CliHadoopIndexer.java
 2014-03-25 10:04:14 -07:00
fjy b6dd2fae64 update some comments on standalone hadoop indexer 2014-03-25 10:03:40 -07:00
nishantmonu51 fe7a184096 review comments 2014-03-25 17:02:36 +05:30
nishantmonu51 0ba72ffb2b merge from master 2014-03-25 15:43:05 +05:30
nishantmonu51 6a34c09e05 Merge branch 'master' into compute-populate-cache 2014-03-25 13:23:00 +05:30
fjy 5ebb2d27e3 fix hadoop 2014-03-24 18:43:31 -07:00
fjy 75ec8bed9a address code review comments 2014-03-24 14:53:30 -07:00
fjy fcd7522596 Merge branch 'master' into router-node 2014-03-24 13:43:52 -07:00
fjy 55169175de prepare for next release 2014-03-21 11:05:54 -07:00
nishantmonu51 69e9b7a6c5 merge changes from master 2014-03-20 20:23:58 +05:30
fjy ab2524ba0a add router node 2014-03-18 13:44:29 -07:00
fjy 804a4051fa prepare for next rel 2014-03-18 08:10:59 -07:00
fjy 1d1ec1d855 clean up some defaults and prepare for next release 2014-03-17 21:41:34 -07:00
fjy 2adcf07f5f Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DetermineHashedPartitionsJob.java
	indexing-service/src/main/java/io/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/test/java/io/druid/indexing/common/task/TaskSerdeTest.java
	processing/src/test/java/io/druid/segment/TestIndex.java
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-03-17 10:59:31 -07:00
nishantmonu51 468e8e0b31 Memcache changes 2014-03-17 18:54:30 +05:30
fjy f1da044cfa fix another breaking endpoint in druid coordinator 2014-03-14 15:15:08 -07:00
fjy f525361c3d fix broken tests and prepare for next release 2014-03-14 15:05:07 -07:00
fjy f0fdfbb2f7 fix broken constant post agg 2014-03-14 14:54:52 -07:00
fjy b3fb2fea9c prepare for next release 2014-03-14 10:57:43 -07:00
nishantmonu51 04f3d0a13a fix cli hadoop indexer 
* run determine partitions job using CLI
 2014-03-11 16:36:19 +05:30
fjy ae2dc6506c fix broken granularity 2014-03-10 15:03:43 -07:00
fjy 243af486cf prepare for next release 2014-03-10 13:24:53 -07:00
fjy e1fac64298 prepare for next release 2014-03-06 12:53:05 -08:00
fjy 7b0b90a860 Merge branch 'master' of github.com:metamx/druid into new-schema 2014-02-20 16:12:31 -08:00
fjy 5d2367f0fd unit tests pass at this point 2014-02-20 15:52:12 -08:00
fjy ab83cba790 Merge pull request #393 from metamx/group-by-improvements 
Group by memory usage improvements
 2014-02-19 19:52:37 -07:00
fjy fb93448e9d prepare for next release because my last one failzored out of bad internetz 2014-02-14 15:22:27 -08:00
fjy 1a8f9f2e12 prepare for next release 2014-02-14 15:07:44 -08:00
fjy a986166b9c prepare for next release 2014-02-12 20:01:05 -08:00
nishantmonu51 3302cff6db groupBy improvements 2014-02-12 00:51:10 +05:30
fjy 2497bd2e85 prepare for next release 2014-02-07 13:04:34 -08:00
fjy c8b350270f prepare for next release 2014-02-07 11:54:56 -08:00
fjy 8edf0cf136 more backwards compatability fix 2014-01-26 21:27:34 +08:00
fjy 158a725985 fix backwards compat issues 2014-01-26 20:07:09 +08:00
fjy f3e1cbc423 fix according to code review 2014-01-21 16:26:28 -08:00
fjy e2ea2ec7fd Merge branch 'master' into az 
Conflicts:
	server/src/main/java/io/druid/server/coordinator/DruidCoordinator.java
	server/src/main/java/io/druid/server/http/CoordinatorResource.java
	server/src/test/java/io/druid/client/DirectDruidClientTest.java
 2014-01-20 15:10:13 -08:00
fjy 215584891e Merge branch 'master' into moar-endpoints 2014-01-20 13:06:45 -08:00
fjy 6404967994 fixes 2014-01-20 13:05:05 -08:00
fjy 8d01a46d02 actually register resources 2014-01-17 18:01:09 -08:00
fjy f898c29e20 fix batch indexing and prepare for next release 2014-01-17 15:52:04 -08:00
fjy 0ceda034cd fix dependency issue and prepare for next release 2014-01-14 17:34:02 -08:00
fjy 187bb84b45 prepare for next release 2014-01-14 16:07:05 -08:00
fjy 9184906c39 prepare for next release 2014-01-14 09:54:42 -08:00
fjy e8d61ffe99 prepare for next release 2014-01-13 18:02:24 -08:00
fjy f123a14373 Merge branch 'master' into az 
Conflicts:
	docs/content/Realtime.md
	server/src/main/java/io/druid/client/selector/ServerSelector.java
 2014-01-09 19:09:38 -08:00
Xavier Léauté 632d6b0a78 make server selection configurable, add back random server selection 2014-01-09 14:37:33 -08:00
fjy f44509b530 queries working at this point, tests pass, still need more tests and fix rules interface 2014-01-07 16:24:53 -08:00
fjy e11952f3b6 bridge is kinda working for mastery stuff right now, tests pass 2014-01-06 17:01:50 -08:00
Xavier Léauté 95d92915bf more tests for Initialization + dependency cleanup 2013-12-30 16:31:10 -08:00
Gian Merlino bbe1b82347 Update versions 2013-12-20 12:26:27 -08:00
Gian Merlino 4bc8ff4d6a Update versions 2013-12-20 11:19:36 -08:00
fjy fba6caf7fd Merge pull request #326 from metamx/versions 
StatusResource
 2013-12-18 22:44:50 -08:00
Gian Merlino fc4dd51194 Update versions 2013-12-18 15:33:25 -08:00
fjy d112b2d211 prepare for next release 2013-12-17 17:47:04 -08:00
fjy 0051877f84 fix status endpoint 2013-12-17 17:46:16 -08:00
fjy 07875dd30b prepare for next release 2013-12-17 16:28:11 -08:00
fjy 69f33b0acd remove commented line 2013-12-17 16:24:15 -08:00
fjy bbb2754a7c fix redirects in druid 2013-12-17 16:22:36 -08:00
fjy f2241c8885 prepare for next release 2013-12-16 16:16:00 -08:00
Xavier Léauté ac2ca0e46c separate move and archive tasks 2013-12-16 14:00:55 -08:00
Xavier Léauté a417cd5df2 add archive task 2013-12-16 13:59:15 -08:00
fjy 84233238b1 Merge branch 'master' of github.com:metamx/druid 2013-12-16 12:01:55 -08:00
fjy 0998d835b0 fix redirection issues with master and overlord 2013-12-16 12:01:52 -08:00
Gian Merlino 7759ed4517 Coordinator: Link service announcement to leadership 2013-12-16 11:28:28 -08:00
Himadri Singh 3752122a7b implementing comments from review 2013-12-16 15:16:28 +05:30
fjy 87b83bceb1 fix task storage config serde and prepare for next release 2013-12-13 16:55:22 -08:00
fjy 01f9c1df31 fix broken task storage config and prepare for next release 2013-12-13 16:45:32 -08:00
fjy 9a5fd6b0e9 prepare for next release 2013-12-13 16:18:34 -08:00
Gian Merlino 600dc7546f Configurability of recency threshold 2013-12-13 16:02:54 -08:00
fjy 407f8caf94 remove server monitor from list of default monitors 2013-12-13 09:04:32 -08:00
fjy 25ebba2c38 fix default start delay of indexing service and prepare for next release 2013-12-12 18:51:25 -08:00
fjy 19eafe0e96 prepare for next release 2013-12-12 17:26:50 -08:00
fjy a58f90a689 prepare for next release 2013-12-12 14:51:14 -08:00
Gian Merlino 70c153592f CliPeon: Fix local mode 2013-12-12 14:22:57 -08:00
Gian Merlino c4b8c8bc6f Rework indexing service internals to hopefully be more reliable. 
The TaskQueue directly manages the TaskRunner. The main management loop runs
periodically and checks that the runner is doing reasonable things. If not, it
attempts to adjust the runner. The management loop also runs on-demand when a
task is added to keep task assignment relatively low latency. The TaskConsumer
is no longer necessary and so it no longer exists.

Task interval locks are handled differently. Instead of some tasks acquiring
locks at runtime and some tasks having implicit fixed lock intervals, all tasks
ask for locks explicitly. This occurs either in "isReady" (which runs on the
overlord) or in "run" (which runs on the peon).

Other changes:
- The TaskQueue is attached to the leader lifecycle, instead of global
- The TaskLockbox is able to sync itself from storage and is no longer
  bootstrapped by the TaskQueue.
- RemoteTaskRunner does not clean up zk paths until asked to. This will
  prevent deletion of statuses that have not yet been committed.
- Added retries on DbTaskStorage operations.
- Removed SpawnTasksAction (no more subtasks)
- Removed obsolete EventReceiverFirehose configs
- Removed obsolete OldOverlordResource
- Removed TaskStorageQueryAdapter methods related to subtasks
 2013-12-11 15:05:16 -08:00
Gian Merlino f3cfd1d781 Introduce FileTaskLogs, and move TaskLogs module from server to indexing-service 2013-12-10 17:39:43 -08:00
fjy 6cc0860657 fix s3 module problems and prepare for next release 2013-12-09 20:05:55 -08:00
fjy ab1f02b2bc fix version brokenness and prepare for next release 2013-12-09 18:38:22 -08:00
fjy 976affb3af prepare for next release 2013-12-09 18:28:31 -08:00
fjy 303f6ff334 fix worker config setup problems 2013-12-09 18:25:29 -08:00
fjy 8aba4ad3d2 Merge pull request #311 from metamx/versions 
Add versions to static page
 2013-12-09 17:18:16 -08:00
Himadri Singh b5f6dbc32f Code refactoring, one place! 2013-12-10 02:09:41 +05:30
Himadri Singh 8117fff4e2 Some minor fixes 2013-12-10 00:39:10 +05:30
fjy fbad1e2d7a fix convert properties with infoPath and prepare for next release 2013-12-04 18:55:35 -08:00
Himadri Singh 32200b9b41 removing debug line. 2013-12-04 11:00:47 +05:30
Himadri Singh 47278ad109 Merge relevant changes from 'druid-version-info' 2013-12-04 10:21:19 +05:30
fjy 05f72601c4 fix request time metric and prepare for next release 2013-12-02 14:20:59 -08:00
fjy ddb6e83f17 fix realtime default and prepare for next deploy 2013-11-25 17:53:17 -08:00
fjy fbb65988bd make query resource eagerly instantiate and prepare for next release 2013-11-20 11:58:25 -08:00
fjy c234dafee1 prepare for next release 2013-11-20 10:43:37 -08:00
fjy 84a6262e04 prepare for next release 2013-11-20 10:27:09 -08:00
fjy e7167ae00a add new query resource 2013-11-20 10:02:43 -08:00
fjy 984e24c1bb fix broken links in docs, prepare for next release 2013-11-19 09:57:56 -08:00
fjy 67a93ccd4d fix broken validation query config port 2013-11-18 15:53:49 -08:00
fjy 1820110818 clean up docs and fix server monitor double emitting metrics 2013-11-18 15:37:30 -08:00
fjy a86d3af17e fix docs 2013-11-14 18:29:27 -08:00
fjy 346cf0e04c fix out of order in urls for hadoop classpath 2013-11-14 18:18:39 -08:00
fjy 5d00e4075e prepare for next release 2013-11-14 15:06:26 -08:00
fjy 6b41681424 fix ordering of urls, ugh, need to write more tests 2013-11-14 15:06:07 -08:00
fjy 23ebca6d32 more hadoop dependency hell, getting the right urls to hadoop is hard 2013-11-14 14:29:42 -08:00
fjy 61367c6b13 prepare for next release 2013-11-14 13:16:38 -08:00
fjy c51eed060f actually use a class loader pull out dependencies from extension modules 2013-11-14 11:52:21 -08:00
fjy fcf0c6ce06 change the classpath ordering for batch processing and prepare for next release 2013-11-14 10:52:51 -08:00
fjy 60300bdad0 fix doc version 2013-11-13 13:34:57 -08:00
fjy c1009f0872 prepare for next release 2013-11-13 11:36:50 -08:00
fjy d0f990129a fix docs 2013-11-13 10:59:03 -08:00
fjy 9fe9ca3163 the emitter library is broken and also prepare for next release 2013-11-08 15:54:16 -08:00
fjy 822a30658d fix broken http post emitter and prepare for next release 2013-11-08 14:01:16 -08:00
fjy 64b93bf448 fix broken autoscaling and prepare for next release 2013-11-08 11:29:42 -08:00
fjy d70bcc5657 repair broken pom 2013-11-07 23:27:06 -08:00
fjy a0eedf7342 fix broken docs 2013-11-07 23:11:30 -08:00
fjy d1c3176bf5 update docs ver 2013-11-07 19:47:36 -08:00
fjy a049b42674 fix an issue with task tables not getting created automatically and prepare for next release 2013-11-07 18:01:35 -08:00
fjy d49554185a fix broken default table names 2013-11-07 17:38:00 -08:00
fjy 6b573c76f1 more fixes 2013-11-07 14:40:45 -08:00
fjy 33561031a2 update docs version to match next release 2013-11-05 15:30:41 -08:00
Xavier Léauté dce6e7e0eb hack to support remote repository authentication 2013-11-01 09:33:07 -07:00
Xavier Léauté ef88f37bbe add tools pull-deps to cli 2013-10-30 18:16:27 -07:00
fjy 21235ae533 fix configuration conversion bugs and docs and hdfs configs 2013-10-29 17:34:24 -04:00
Gian Merlino 2e9c46867f Fixes for indexing service. 
- Create IndexingServiceFirehoseModule so firehoses can be loaded by all mains
- Fix implicit lock acquisition in AbstractTask
 2013-10-18 11:14:33 -07:00
fjy a1c09df17f make the hadoop index task work again 2013-10-16 09:45:17 -07:00
fjy 9796a40b92 port docs over to 0.6 and a bunch of misc fixes 2013-10-11 18:38:53 -07:00
fjy a9a723bd11 clean up poms, add a new loading your own data tutorial, add new validation, clean up logs 2013-10-09 15:42:39 -07:00
fjy 4e509d1d09 Merge branch 'master' into is-docs 2013-10-09 14:05:10 -07:00
cheddar c47fe202c7 Fix HadoopDruidIndexer to work with the new way of things 
There are multiple and sundry changes in here.

First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.

In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module.  I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.

In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api.  They are all API-level things, however, so they really belong in druid-api instead.

Lastly, I removed the druid-realtime module and put it all in druid-server.
 2013-10-09 15:15:44 -05:00
fjy 4ec4b8e024 rewrite indexing service docs 2013-10-08 16:34:58 -07:00
fjy ac330f72bb first set of changes to standarize the naming convention we use in druid 2013-10-03 16:36:48 -07:00
fjy bf157ea8ef fix the example assembly and have better error logging for rabbitmq 2013-10-03 10:54:25 -07:00