Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,961 Commits 113 Branches 504 Tags 726 MiB
Commit Graph

1260 Commits

Author SHA1 Message Date
jisookim0513 a5d02b9030 second attempt to fix merge-conflicts 2014-09-24 15:59:00 -07:00
jisookim0513 3bf39cc9f8 attempted to fix merge-conflicts 2014-09-24 15:55:42 -07:00
Xavier Léauté 77c9afe92f try to make javadoc happy 2014-09-23 18:53:29 -07:00
Xavier Léauté d02fd97ea3 fix comment 2014-09-23 16:25:46 -07:00
Xavier Léauté 8603d2fd3e keep default behavior 2014-09-23 16:23:02 -07:00
Xavier Léauté 05d4f71ddc fix background annoucing race condition 2014-09-23 15:04:48 -07:00
Xavier Léauté 12449481e3 parallelize segment loading 2014-09-22 16:33:25 -07:00
nishantmonu51 f51ab84386 merge changes from druid-0.7.x 2014-09-22 23:48:45 +05:30
nishantmonu51 443e5788fb make OffheapIncrementalIndex tuneable 2014-09-22 19:26:10 +05:30
jisookim0513 273205f217 initial attempt for abstraction; druid cluster works with Derby as a default 2014-09-19 17:39:59 -07:00
nishantmonu51 8eb6466487 revert buffer size and add back rowFlushBoundary 2014-09-19 23:06:04 +05:30
Xavier Léauté 32b6135f3d background announce segment cache 2014-09-18 12:09:20 -07:00
Xavier Léauté d9a0d403b3 default zk compressed + batch announcements in 0.7 2014-09-16 14:54:45 -07:00
fjy 8b387034a3 Merge pull request #741 from metamx/serverview-initialization 
initialize segment view before announcing broker
 2014-09-16 15:35:09 -06:00
fjy 1dbb6fd4b0 Merge pull request #736 from metamx/fix-synchronized 
fix synchronized on field getting updated
 2014-09-16 11:06:35 -06:00
Xavier Léauté 762485a0c9 remove unnecessary transient 2014-09-16 09:59:25 -07:00
Xavier Léauté f1e3518f37 initialize segment view before announcing broker 2014-09-16 00:07:31 -07:00
xvrl a73670ee7b Merge pull request #738 from metamx/dim-ex 
Have better dimension exclusions
 2014-09-15 17:14:29 -07:00
fjy 77dbac3739 fix import 2014-09-15 17:10:34 -07:00
fjy f2d6360d4d cleanup 2014-09-15 17:08:45 -07:00
fjy 34ae426ffa address cr 2014-09-15 17:08:02 -07:00
fjy 1ab2662772 have better dimension exclusions 2014-09-15 16:52:07 -07:00
Xavier Léauté 02892233ac fix synchronized on field getting updated 2014-09-15 13:56:31 -07:00
fjy aa28bc0613 address cr 2014-09-15 13:14:23 -07:00
Xavier Léauté 0712c666cb null check for something that cannot be null 2014-09-15 13:00:06 -07:00
Xavier Léauté c8b8e3f6e9 negating compare is bad 2014-09-15 13:00:06 -07:00
Xavier Léauté 137ad50bf1 classes that should be static 2014-09-15 13:00:06 -07:00
Xavier Léauté e57e2d97ba make constants final 2014-09-15 12:53:40 -07:00
Xavier Léauté d4795ce927 fix missing charsets 2014-09-15 12:53:40 -07:00
Xavier Léauté cfa92e8217 fix incorrect nullable annotations 2014-09-15 12:13:52 -07:00
fjy cbf5076961 still propagate log 2014-09-12 11:43:42 -07:00
fjy e62ed40d86 make some logs not emit 2014-09-11 16:49:18 -07:00
fjy 469ccbbe5e Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/query/FinalizeResultsQueryRunner.java
	processing/src/main/java/io/druid/query/UnionQueryRunner.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactory.java
	processing/src/main/java/io/druid/query/topn/TopNQueryEngine.java
	processing/src/main/java/io/druid/query/topn/TopNQueryRunnerFactory.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/test/java/io/druid/server/initialization/JettyTest.java
	services/pom.xml
 2014-09-11 16:20:50 -07:00
fjy fec7b43fcb make making v9 segments something completely configurable 2014-09-10 15:28:30 -07:00
Xavier Léauté d6f6a59ae2 fix for missing context when embedding broker 2014-09-09 14:25:56 -07:00
Tarek Rached 609887887e updated env when we are local 2014-09-09 13:29:42 -07:00
Tarek Rached ae2a09ab5e added new console 2014-09-09 12:44:57 -07:00
Tarek Rached 47b52aaae1 moved old console to old-console 2014-09-09 12:44:30 -07:00
Xavier Léauté 508e982190 Merge remote-tracking branch 'origin/master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/main/java/io/druid/server/ClientQuerySegmentWalker.java
	services/pom.xml
 2014-08-30 22:42:36 -07:00
Xavier Léauté 58ab759fc6 remove unused imports 2014-08-29 14:03:47 -07:00
Xavier Léauté ac05836833 make Java 8 javadoc happy 2014-08-29 13:58:50 -07:00
Xavier Léauté 047f793cb7 query post-processing operators 
- Allows post-processing of query results
- Example timewarp post-processing operator
 2014-08-25 16:55:46 -07:00
Gian Merlino 92b0611afb Merge pull request #693 from metamx/handle-arithmetic-ex 
Handle arithmetic ex - additional commit for PR #692
 2014-08-20 23:24:06 -07:00
nishantmonu51 fe03476c53 truncate segmentGranularity 2014-08-21 11:40:30 +05:30
Gian Merlino 5970b96dc5 Merge pull request #692 from metamx/handle-arithmetic-ex 
Handle arithmetic exception for overlow
 2014-08-20 23:09:43 -07:00
nishantmonu51 8c9d639947 review comment - better way to handle it 2014-08-21 11:38:24 +05:30
nishantmonu51 87b83e7bdb better comment 2014-08-21 09:47:23 +05:30
nishantmonu51 c48c4ac0f6 handle arithmeticEx 2014-08-21 09:44:55 +05:30
nishantmonu51 67f4bbae74 fixes from review comments 
fix sync of aggs,
fix NPE in sink.isFull,
RealtimeTuningConfig lower the bufferSize to 256m
 2014-08-21 09:24:14 +05:30
nishantmonu51 79ff993b31 increase default buffer size to 512m 2014-08-20 22:15:06 +05:30
nishantmonu51 33354cf7fe replace maxRowsInMemory with BufferSize 2014-08-20 20:59:44 +05:30
nishantmonu51 60906c3244 Revert "make valueType configurable" 
This reverts commit 6f60a3f604.
 2014-08-20 11:55:26 +05:30
fjy 77e514688a Merge branch 'druid-0.7.x' into offheap-incremental-index 2014-08-18 11:14:19 -07:00
Xavier Léauté 1fd30ab588 default service/host/port for all nodes 2014-08-15 17:14:05 -07:00
fjy 4f0f198a00 add more logging 2014-08-14 14:43:54 -07:00
fjy c36ffd1e55 address cr 2014-08-14 14:39:08 -07:00
fjy 6ba0a0df8b address cr 2014-08-14 13:56:36 -07:00
fjy 96d5d4a59a fix some rt cleanup and failure edge cases 2014-08-14 13:38:32 -07:00
nishantmonu51 6f60a3f604 make valueType configurable 2014-08-13 14:37:57 +05:30
nishantmonu51 1b0a72751b Add support for LongColumn 2014-08-13 08:52:36 +05:30
nishantmonu51 c6712739dc merge changes from druid-0.7.x 2014-08-12 15:47:42 +05:30
fjy b740987381 Merge pull request #616 from metamx/jisoo 
add retry feature on broker side; modify QueryRunner inferface and tests
 2014-08-11 15:36:16 -06:00
fjy 2ecd55bae5 Merge branch 'druid-0.7.x' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
 2014-08-11 12:47:38 -07:00
fjy 0e1a8f6f02 concurrency my old foe, we meet again 2014-08-09 19:33:07 -07:00
fjy 8998700760 prepare for next release 2014-08-08 16:19:45 -07:00
fjy e5896f4863 fix yet another bug with LQP 2014-08-08 15:50:57 -07:00
fjy 9a371f3490 Merge pull request #674 from metamx/fix-arbitrary-granularity 
fix missing queryGranularity in ArbitraryGranularitySpec
 2014-08-08 16:12:24 -06:00
Xavier Léauté 3826b1d976 fix missing queryGranularity in ArbitraryGranularitySpec 2014-08-08 15:11:35 -07:00
fjy 68827ff88b fix another bug in LQP with concurrency 2014-08-08 15:06:03 -07:00
fjy 48b99e43fc fix LQP register callback 2014-08-08 14:34:54 -07:00
fjy f3970bb1d8 Merge branch 'master' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
 2014-08-08 12:44:29 -07:00
xvrl 038f2de496 Merge pull request #672 from metamx/router-strat 
Add more config to router strategies; fix bad test
 2014-08-08 11:37:35 -07:00
fjy 9803be7529 Merge pull request #673 from metamx/rt-plumber-merge-behavior 
Change merged-segment-removing behavior in RealtimePlumber.
 2014-08-08 12:24:23 -06:00
Gian Merlino 179170224d Fix conditional. 2014-08-08 11:20:36 -07:00
Gian Merlino f547049929 Change merged-segment-removing behavior in RealtimePlumber. 
Old: Merged segments are deleted upon successful push, or on IOException.
New: Merged segments are deleted on any Exception, but NOT successful push.

Deleting a merged segment means that the next merge-and-push run will try to
push it again. So we want that to happen if there was any sort of Exception.
We *don't* want it to happen if the merge-and-push was successful, since in
that case, we are just waiting for historical nodes to load the segment. It
might take a while, but there's no reason to re-push while waiting.
 2014-08-08 10:59:16 -07:00
fjy a855db2170 make things immutable 2014-08-08 10:53:10 -07:00
fjy d2c7a67a0c clean up some redundant code 2014-08-08 10:52:05 -07:00
fjy f1409f751a fix bad defaults 2014-08-08 10:46:36 -07:00
fjy b3eb4cc5da fix defaults 2014-08-08 10:42:58 -07:00
fjy 627447c6f8 add more config to router strategies; fix bad test 2014-08-08 10:41:11 -07:00
fjy 08c7aabb34 fix deadlock condition that can exist in the coordinator 2014-08-07 15:55:25 -07:00
fjy 91ebe45b4e support both rejectionPolicy and rejectionPolicyFactory in serde 2014-08-07 10:06:27 -07:00
fjy 7624bb5ea3 Merge branch 'master' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
	processing/src/main/java/io/druid/guice/PropertiesModule.java
	server/src/main/java/io/druid/initialization/Initialization.java
 2014-08-06 15:47:22 -07:00
Xavier Léauté 87fc354237 rename CleanupAvailable -> CleanupUnneeded 2014-08-06 14:29:00 -07:00
Xavier Léauté 70660ad295 separate cleanup of overshadowed vs. unavailable segments 2014-08-06 14:06:52 -07:00
fjy 8b6f030ad5 Merge pull request #661 from metamx/drop-remove-stuff 
Coordinator drop/remove related stuff.
 2014-08-06 14:28:23 -06:00
Gian Merlino 6e33cd4e0a Coordinator drop/remove related stuff. 
- DruidCoordinatorCleanup should wait for availableSegments to populate
  before dropping any segments.
- Clarify that millisToWaitBeforeDeleting applies to "removing" rather than
  "dropping" segments.
- LoadRule shouldn't need to wait for the deletionWaitTime before dropping
  excess replicas.
 2014-08-06 12:52:09 -07:00
fjy 4d4c6b891e address cr 2014-08-05 15:58:42 -07:00
fjy 8097e04938 fix naming 2014-08-05 15:48:30 -07:00
fjy 3a78960e65 Fix how metrics are emitted from servermanager 2014-08-05 15:47:30 -07:00
fjy 66225d698f fix stupidness with spatial indexing 2014-08-05 15:07:56 -07:00
Gian Merlino c58a316437 Need more synchronizing in DruidDataSource. 2014-08-05 13:32:53 -07:00
jisookim0513 6f58cd1156 fix merge conflcts; change test files 2014-08-04 17:45:56 -07:00
fjy 290982433b Merge pull request #657 from metamx/only-uncached-query-metrics 
Only emit query metrics for uncached queries.
 2014-08-04 18:01:18 -06:00
Gian Merlino 6c8222788b Only emit query metrics for uncached queries. 2014-08-04 16:42:38 -07:00
fjy 872a699142 another log fix 2014-07-31 17:23:40 -07:00
fjy 93afc26d92 another log fix 2014-07-31 15:36:01 -07:00
fjy 1c4d8c55dc finally track down and fix the node exists problem with not deleting noop requests 2014-07-31 13:56:29 -07:00
nishantmonu51 637bd35785 merge changes from druid-0.7.x 2014-07-31 16:07:22 +05:30
fjy ecbc9eda48 revert breakage to chat handlers 2014-07-29 21:55:05 -07:00
fjy 3920300900 fix router bug 2014-07-29 21:44:07 -07:00
fjy f577e1e74b fix guava version and firehose 2014-07-29 17:23:32 -07:00
fjy 43d4e5418a Merge pull request #653 from metamx/override-static-resource 
allow overriding coordinator static resource
 2014-07-29 17:15:01 -06:00
fjy 1586d60ec2 fix bugs in log printing 2014-07-29 16:14:29 -07:00
Xavier Léauté 05a09d2550 allow overriding coordinator static resource 2014-07-29 15:13:57 -07:00
fjy 1ac84fcd00 some more logging cleanup for zk coordination 2014-07-29 14:07:19 -07:00
xvrl d21e377a20 Merge pull request #651 from metamx/rewrite-async 
rewrite router async logic
 2014-07-29 13:56:01 -07:00
Xavier Léauté 25a5a0fd33 fix unnecessary error messages 2014-07-29 13:54:36 -07:00
fjy 088e2c5af3 address cr 2014-07-29 13:36:28 -07:00
fjy b09517319d remove debug code 2014-07-29 13:23:12 -07:00
fjy 4a417cbcfc clean up some logging in Zk coordinator 2014-07-29 11:49:06 -07:00
fjy 41db4b1063 actually use the metrics emitting listener 2014-07-29 10:28:51 -07:00
fjy 2ad1bd3f44 add back metrics 2014-07-29 10:22:55 -07:00
fjy 382cde9cf2 less verbose logging for LQP 2014-07-29 09:51:53 -07:00
fjy 088c2386dc rewrite router async logic 2014-07-28 16:32:15 -07:00
fjy 82d623ada7 fix compilation error 2014-07-25 16:35:55 -07:00
Xavier Léauté 863929d550 unnecessary arguments 2014-07-25 16:26:44 -07:00
Xavier Léauté df59d2acb4 remove debug code 2014-07-25 16:25:47 -07:00
Xavier Léauté 2d4f1889e8 completely async request proxying + jetty update 2014-07-25 16:22:43 -07:00
fjy fc95a79abd add query priorities to metric emission 2014-07-25 16:10:55 -07:00
fjy 45e443bedb a few slight tweaks and bumping up curator version 2014-07-25 16:07:40 -07:00
fjy bb5c5b3ce7 fix config with broker select and add docs 2014-07-25 10:50:41 -07:00
Xavier Léauté 85f739797f remove debug println 2014-07-24 16:49:09 -07:00
Xavier Léauté 03efbb0e02 default query timeout to server idle timeout 2014-07-22 16:59:27 -07:00
Xavier Léauté 00f086665c default async timeout to server idle timeout 2014-07-22 16:55:21 -07:00
fjy 09e2f13ffc adding missing header 2014-07-22 10:12:47 -07:00
fjy 8f2fc595ee allow router to override selection based on config 2014-07-22 10:12:02 -07:00
nishantmonu51 4ce12470a1 Add way to skip determine partitions for index task 
Add a way to skip determinePartitions for IndexTask by manually
specifying numShards.
 2014-07-18 18:52:15 +05:30
fjy 291f4c00ae Merge branch 'master' of github.com:metamx/druid into cleanup-ingest 2014-07-17 20:24:59 -07:00
Xavier Léauté 95a09c04bf log errors in asyncForwardingServlet 2014-07-17 16:22:30 -07:00
fjy 76f096e667 Merge branch 'master' of github.com:metamx/druid into cleanup-ingest 2014-07-17 14:38:54 -07:00
fjy 8ef380b3f9 Merge pull request #638 from metamx/rejigger-cache 
Rejiggering the cache
 2014-07-17 15:19:23 -06:00
fjy ef325f15ff update pom and fix broken UT 2014-07-17 13:47:59 -07:00
fjy ba978d8b79 some minor cleanups to ingest firehose 2014-07-17 13:05:59 -07:00
fjy bc650a1c80 Merge pull request #627 from metamx/druid-firehose 
Functionality to ingest a Druid segment and change the schema
 2014-07-17 13:41:16 -06:00
fjy b70a6b1061 address code review comments 2014-07-16 13:52:06 -07:00
fjy de8cb55260 Merge branch 'master' into rejigger-cache 
Conflicts:
	pom.xml
 2014-07-16 13:37:20 -07:00
fjy bdfeccd092 doc and timeout fix 2014-07-15 15:57:58 -07:00
fjy 2f90bfcf9a rejiggering the cache for groupBy and select 2014-07-15 12:12:33 -07:00
nishantmonu51 f5f05e3a9b Sync changes from branch new-ingestion PR #599 
Sync and Resolve Conflicts
 2014-07-11 16:15:10 +05:30
nishantmonu51 a12688bc8b fix partitionNum calculation & remove extra sorting 2014-07-10 12:21:27 +05:30
nishantmonu51 fa43049240 review comments & pom changes 2014-07-10 11:48:46 +05:30
nishantmonu51 a1a5e4254f CombiningFirehoseFactory 
can be used to ingest data from multiple sources.
 2014-07-08 21:14:48 +05:30
nishantmonu51 36fc85736c Add ShardSpec Lookup 
Optimize choosing shardSpec for Hash Partitions
 2014-07-08 18:01:31 +05:30
jisookim0513 c4397bd65a fixed merge conflicts 2014-07-07 11:26:11 -07:00
fjy c1cd9ff2ce fix broken rename of rejection policies 2014-07-02 23:41:37 -06:00
fjy 255504727a Merge pull request #615 from metamx/push-bootstrapped-sinks 
push sinks after bootstrap. fix #570
 2014-06-20 11:45:19 -06:00
fjy b978eb33f1 minor fix for cleanup of historical cache file 2014-06-20 10:42:03 -07:00
nishantmonu51 a4003966b2 push sinks after bootstrap. fix #570 2014-06-20 21:45:30 +05:30
xvrl 12ee17c4d2 Merge pull request #614 from metamx/delay-drop 
add an optional delay for how long it takes to drop a segment, fixes all...
 2014-06-19 22:38:26 -07:00
jisookim0513 0244172ccb change missigSegments to a static variable 2014-06-19 18:22:36 -07:00
jisookim0513 4e2b5b743f add more unit tests 2014-06-19 18:18:58 -07:00
jisookim0513 bdb35e2d7e fix retry logic and change the default value of retry to 0 2014-06-19 17:50:50 -07:00
fjy fc36bfbc6d remove pointless exception 2014-06-19 16:49:50 -07:00
fjy 2aab2a0cd9 address cr 2014-06-19 16:48:43 -07:00
fjy b313601a3b add an optional delay for how long it takes to drop a segment, fixes all manners of inconsistent issues 2014-06-19 16:32:38 -07:00
fjy 4c40e71e54 address cr 2014-06-19 14:48:46 -07:00
fjy a870fe5cbe inject column config 2014-06-19 14:47:57 -07:00
Xavier Léauté 09346b0a3c make column cache configurable 2014-06-19 14:43:03 -07:00
jisookim0513 8e85097999 fix header so it passes the entire context 2014-06-19 14:38:11 -07:00
jisookim0513 35e080bbc1 fix concurrency issue with the map; introduce new exception; add incomplete retry query runner test 2014-06-19 14:14:54 -07:00
jisookim0513 8515a11787 change the signature of run() 2014-06-18 18:05:09 -07:00
jisookim0513 f4b1dc032b change the way to use RetryQueryRunnerConfig 2014-06-18 17:15:04 -07:00
jisookim0513 00856f0fec add retry feature on broker side; modify QueryRunner inferface and tests 2014-06-18 16:55:33 -07:00
fjy 0bc1915067 Merge pull request #578 from metamx/new-guava 
Update guava, java-util, and druid-api
 2014-06-18 14:23:32 -06:00
fjy 3dbf475784 fix linux test failures 2014-06-17 11:05:36 -07:00
Xavier Léauté 4a835a9f0f fix initialization test static state dependency 2014-06-17 10:07:20 -07:00
nishantmonu51 0629be622c remove unnecessary changes & fix index closing subquery 2014-06-16 18:50:49 +05:30
fjy 332ba3ef20 fix bad merge 2014-06-13 11:36:36 -07:00
fjy 4cc0353be2 Merge branch 'master' of github.com:metamx/druid into new-guava 2014-06-13 11:30:07 -07:00
fjy 5bd4ad3f0f address code review 2014-06-13 11:29:49 -07:00
fjy a63cda3281 Merge branch 'master' into new-guava 
Conflicts:
	server/src/main/java/io/druid/server/QueryResource.java
 2014-06-13 10:08:10 -07:00
nishantmonu51 49d3bd94da Merge branch 'master' into offheap-incremental-index 2014-06-13 15:39:34 +05:30
nishantmonu51 0ddaf4c307 fix race while index closing and querying 2014-06-13 15:36:57 +05:30
Xavier Léauté b2419b1530 fix missing return statement 2014-06-12 17:09:21 -07:00
Xavier Léauté 8f7c82d351 avoid closing yielder prematurely 2014-06-12 16:00:06 -07:00
nishantmonu51 025814cfff Merge branch 'master' into offheap-incremental-index 
Conflicts:
	processing/src/main/java/io/druid/query/GroupByParallelQueryRunner.java

processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactor
y.java

processing/src/main/java/io/druid/segment/incremental/IncrementalIndexSt
orageAdapter.java
 2014-06-12 20:02:27 +05:30
nishantmonu51 a7e19ad892 configure buffer sizes 2014-06-12 19:32:37 +05:30
Xavier Léauté 1fb9b21cf0 async servlet delete support + cleanup 2014-06-10 10:36:39 -07:00
Xavier Léauté b84884ab76 remove methods used only for testing 2014-06-10 10:36:39 -07:00
Xavier Léauté 855c66c9ad less stack traces when cancelling queries 2014-06-10 10:36:38 -07:00
Xavier Léauté d01f272a7a forward cancellation in direct druid client 2014-06-10 10:36:38 -07:00
Xavier Léauté d2c729adec return proper json errors to propagate query cancellation and timeout 2014-06-10 10:36:38 -07:00
Xavier Léauté def62c74f8 properly remove completed queries from query manager 2014-06-10 10:36:38 -07:00
Xavier Léauté 4f1e157639 pass through errors from computes in DirectDruidClient 2014-06-10 10:36:38 -07:00
Xavier Léauté d6f38827db initial query cancellation commit 2014-06-10 10:36:28 -07:00
fjy 5380f14905 add header 2014-06-09 16:13:57 -07:00
fjy a3e86f7094 Merge branch 'master' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
 2014-06-09 16:01:53 -07:00
fjy 7233fce50c refactor examples to use global runtime 2014-06-09 15:59:44 -07:00
nishantmonu51 5ef660f6d1 improve mem usage for getLoadStatus 2014-06-06 00:23:34 +05:30
nishantmonu51 ab31300513 remove duplicate code 2014-06-05 23:16:38 +05:30
nishantmonu51 03e1fb6f3f reduce memory usage for end points 
do not copy entire segments when not needed,
reduces memory pressure on the coordinator when no. of segments is
large enough.
 2014-06-05 23:14:51 +05:30
nishantmonu51 5bdc4a761a close IncrementalIndex properly and free up buffer 2014-06-05 19:41:43 +05:30
nishantmonu51 6265613bb9 Merge branch 'master' into offheap-incremental-index 2014-06-05 17:42:57 +05:30
nishantmonu51 01e8a713b6 unit tests passing with offheap-indexing 2014-06-05 17:42:53 +05:30
Xavier Léauté 6e23ea457b fix memory issues in FilteredServerView 2014-06-04 23:54:37 -07:00
fjy 77ec4df797 update guava, java-util, and druid-api 2014-06-03 13:43:38 -07:00
fjy 8340a1b0a1 Properties module can now take multiple properties files 2014-06-03 10:34:38 -07:00
fjy f7c4d6a24a fix bug with loadqueue endpoint 2014-06-03 10:28:22 -07:00
fjy abe26a7f2e change default merge size to 500mb instead of 1g 2014-06-02 12:48:57 -07:00
fjy 49f03b808d fix default value for max segments to merge 2014-06-02 10:20:13 -07:00
Xavier Léauté ea7eaba735 require default predicate in ServerInventoryViews 2014-05-29 17:09:34 -07:00
Xavier Léauté f32aee7804 make it explicit it's a concurrent map 2014-05-29 17:01:45 -07:00
Xavier Léauté f30d58ad31 filter out segment callbacks not related to given filter 2014-05-29 15:42:57 -07:00
Xavier Léauté d98a10a7d8 FilteredServerView 2014-05-29 15:42:57 -07:00
fjy 7be93a770a make all firehoses work with tasks, add a lot more documentation about configuration 2014-05-28 16:33:59 -07:00
Xavier Léauté 347815cf27 preserve segment order when pulling from cache 2014-05-27 13:56:58 -07:00
fjy f0c40709cd fix coordinator dynamic configs 2014-05-20 13:20:43 -07:00
fjy 37dca4d437 make sure the dynamic config has at least 1 thread 2014-05-20 12:45:35 -07:00
Xavier Léauté 2786c68411 fix file casing 2014-05-19 20:22:13 -07:00
Gian Merlino a7760af65a Merge pull request #553 from metamx/snapshot-coordinator 
Snapshot coordinator state before running anything
 2014-05-19 18:24:19 -07:00
fjy 1addea4dbe more code review comments 2014-05-19 17:52:34 -07:00
fjy 1b6e816c28 Merge pull request #544 from metamx/java-8-fix-tests 
Make tests pass with Java 8
 2014-05-19 18:11:52 -06:00
fjy 6126dd4231 fix more code review comments 2014-05-19 17:04:26 -07:00
fjy 7879f38ccc fix according to code review 2014-05-19 13:35:23 -07:00
fjy 5ee3f5c9d3 snapshot coordinator state before running 2014-05-18 22:32:20 -07:00
fjy 68618ffc75 fix another code review comment 2014-05-16 15:38:46 -07:00
fjy 389319dd6e fix for code review comments 2014-05-16 15:32:58 -07:00
fjy 09ad32c5c5 fix race condition with merge and persist and sink adding 
Conflicts:
	indexing-service/src/main/java/io/druid/indexing/common/task/IndexTask.java
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumber.java
 2014-05-16 15:21:09 -07:00
fjy 43853a1d9e Merge pull request #545 from metamx/union-query-fix 
Union query fix
 2014-05-15 17:17:46 -06:00
fjy 01d0234b46 Merge branch 'master' of github.com:metamx/druid 2014-05-15 14:23:26 -07:00
fjy 6098cff37a clean up event receiver firehose factory 2014-05-15 14:23:13 -07:00
nishantmonu51 ddc36c89cc union query fix and a broken test 2014-05-15 23:10:40 +05:30
fjy 793bc3539b Merge pull request #538 from metamx/fix-jetty-exception-handling 
fix jetty threads getting stuck
 2014-05-15 10:55:46 -06:00
Xavier Léauté 088c60cf3d always use ByteCountingLRUMap.remove() to remove 2014-05-14 22:48:41 -07:00
fjy c57a18d6b6 Merge branch 'master' of github.com:metamx/druid 2014-05-12 11:38:55 -07:00
fjy 32eee959c3 fix backwards compat issue 2014-05-12 11:38:46 -07:00
nishantmonu51 26569f4b7f fix jetty threads getting stuck 
Fix Jetty threads getting stuck on exceptions
 2014-05-10 02:12:19 +05:30
nishantmonu51 6be6337ab0 remove initial delay from polling 
remove initial delay from polling in coordinator, this will help in the
cases when the pollDuration is set to higher than coordinator
startDelay.
 2014-05-09 23:33:00 +05:30
fjy e7da1bb9ce fix injection problems with server select strategies 2014-05-07 10:13:05 -07:00
fjy 1100d2f2a1 rename configs to make a bit more sense 2014-05-06 14:52:50 -07:00
fjy b6fb4245aa Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDriverConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfigBuilder.java
	pom.xml
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/firehose/EventReceiverFirehoseFactory.java
 2014-05-06 14:32:51 -07:00
fjy ad54f150ec fix broken UT with bad api version 2014-05-06 13:51:22 -07:00
fjy 0ee2323875 Merge pull request #399 from metamx/multithread-costbalancer 
Multithread costbalancer
 2014-05-06 14:18:07 -06:00
fjy 79e6d4eb56 Merge pull request #528 from metamx/union-query-source 
Union query source
 2014-05-06 14:13:33 -06:00
nishantmonu51 66450ca6ed review comments 
remove toShortString
minor other changes
 2014-05-07 00:54:17 +05:30
fjy 544c0f08f6 Merge pull request #459 from metamx/move-firehose 
Move the event receiver firehose such that standalone realtime nodes can use it
 2014-05-05 17:09:43 -06:00
fjy 42c2fcb6b4 Merge pull request #518 from metamx/server-select 
Smarter server selection at the broker level
 2014-05-05 17:09:15 -06:00
fjy 9615c5e428 Merge pull request #519 from metamx/more-endpoints 
Add more endpoints, fixes #517
 2014-05-05 17:09:02 -06:00
nishantmonu51 f2725994b7 refactor method 2014-05-05 19:57:32 +05:30
Himadri Singh 0b9191b876 limit on balancer compute threads 
max being no. of processors - 1
 2014-05-05 17:26:44 +05:30
nishantmonu51 477e01daf7 review comments 2014-05-02 15:21:06 +05:30
nishantmonu51 728a606d32 Add support for union queries 2014-05-01 01:54:52 +05:30
fjy 46eedd7a95 Merge pull request #526 from metamx/fix-race-ingestion 
Fix FileNotFound in realtime nodes
 2014-04-30 14:10:41 -06:00
nishantmonu51 faa74ce2cb hasSwapped properly check 
protect check for hasSwapped.
 2014-05-01 01:38:03 +05:30
nishantmonu51 ff56573910 Fix race 
Fix race in ingestion that can cause FNF,
Properly check for partitionNum
shutdown persistExecutor
 2014-05-01 01:34:58 +05:30
fjy 3fd651c235 fix the rule again, update docs, and prepare for next release 2014-04-30 10:43:30 -07:00
fjy 159c3361b8 Merge pull request #524 from metamx/fix-router2 
Small fixes to router
 2014-04-30 11:21:45 -06:00
fjy 68242950f7 make coordinator routing smarter and period load rule explicit 2014-04-30 10:17:40 -07:00
fjy 9bc6436176 Add more endpoints, fixes #517 2014-04-28 15:55:03 -07:00
fjy e90eb4b83a add custom tier selector 2014-04-28 15:31:15 -07:00
fjy 3c3daf452e Smarter server selection at the router level 2014-04-28 13:58:10 -07:00
fjy e30accbbf5 reducing bloat 2014-04-25 15:25:54 -07:00
fjy a33c2f7880 clean up firehose module and introduce parsers module 2014-04-25 14:20:02 -07:00
fjy 171d20d52d Merge branch 'move-firehose' of github.com:metamx/druid into move-firehose 2014-04-25 14:13:19 -07:00
fjy eef034ca7e Merge branch 'master' of github.com:metamx/druid into move-firehose 2014-04-25 14:13:08 -07:00
fjy 76e0a48527 Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DbUpdaterJob.java
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/io/druid/indexing/common/task/HadoopIndexTask.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumber.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-04-25 14:03:28 -07:00
Xavier Léauté 5d105e7598 fix stupid cache config bug 2014-04-24 17:13:04 -07:00
fjy 7f0319a8ae fix NPE 2014-04-23 16:51:12 -07:00
fjy 2d1f33e59f Merge pull request #500 from metamx/batch-ingestion-fixes 
Batch ingestion fixes
 2014-04-22 17:59:24 -06:00
nishantmonu51 0d8c1ffe54 review comments and add partitioner 2014-04-23 03:30:30 +05:30
fjy 22fba22522 remove whitespace 2014-04-22 13:07:12 -07:00
fjy d1b035a8b9 use emitting logger for query resource 2014-04-22 12:59:24 -07:00
fjy cde93d810c fix server metrics to avoid NaN and Infinity 2014-04-22 10:26:46 -07:00
fjy e3fbbff237 clean up code 2014-04-18 17:41:47 -07:00
nishantmonu51 68e8cda2da fix post aggregator 2014-04-19 04:24:07 +05:30
fjy a520a2dde0 handle null request uris 2014-04-17 18:13:03 -07:00
Xavier Léauté 6a52112034 let broker populate cache lazily 2014-04-17 14:40:05 -07:00
fjy c6274762b7 throw ISE when no servers found at all 2014-04-16 10:27:34 -07:00
fjy 6306e87a96 avoid copying bytes 2014-04-16 10:14:15 -07:00
fjy 51e2da9d8d use a separate annotation for the router, less confusing 2014-04-16 10:08:58 -07:00
fjy 24a0268036 cleanup code in router and add forwarding for get requests 2014-04-16 09:36:18 -07:00
fjy 508cf6a4f7 Merge branch 'master' of github.com:metamx/druid 2014-04-15 14:23:58 -07:00
fjy 9b06b6aa58 misc fixes for router 2014-04-15 14:23:54 -07:00
nishantmonu51 cd1c1d2615 fix casing issue when parsing event 2014-04-15 23:32:54 +05:30
fjy 6f1f21cfd6 Merge pull request #476 from metamx/non-blocking-persist-n-merge 
Non blocking persist n merge
 2014-04-15 10:58:47 -06:00
fjy f6de38a87c Merge pull request #480 from metamx/compute-cache-monitor 
enable cache monitor on computes by default
 2014-04-15 10:51:54 -06:00
nishantmonu51 3172383e56 modify comment 2014-04-15 12:55:40 +05:30
Xavier Léauté 16bffe0588 default cache size to zero 2014-04-14 16:23:54 -07:00
fjy a46c4592b4 a few cleanups 2014-04-14 09:53:29 -07:00
nishantmonu51 595ad7d21d rename thread 2014-04-11 12:34:17 +05:30
nishantmonu51 db60353cac persist-n-merge in separate pool 
persist-n-merge in separate pool, do not block intermediate persists
 2014-04-11 12:18:11 +05:30
fjy 0904436d2f actually emit query stats 2014-04-10 12:54:27 -07:00
fjy 6742223c26 fix another missing log 2014-04-10 11:26:53 -07:00
fjy 8e143adadf address code review comments 2014-04-10 11:13:11 -07:00
fjy 88c592ccaa add more meaningful request logging 2014-04-10 10:37:25 -07:00
Xavier Léauté 69a8723f18 fix npe in CachingQueryRunner 2014-04-09 17:04:42 -07:00
xvrl c05f169171 Merge pull request #458 from metamx/stream-cache 
use and populate cache at compute node level
 2014-04-09 10:42:26 -07:00
fjy e867c93422 turn down too much logging; gp 2014-04-09 09:55:12 -07:00
nishantmonu51 ea8b2dffd5 Add chatHandlerResource 2014-04-09 14:25:18 +05:30
Xavier Léauté 19d6aec396 address code review comments 2014-04-08 17:21:01 -07:00
Xavier Léauté 60b95dabad CachePopulatingQueryRunner -> CachingQueryRunner 
Populate and use cache at compute node level
 2014-04-08 17:14:17 -07:00
Xavier Léauté 3e6ffbd5ac populate cache on sequence close 2014-04-08 17:14:17 -07:00
fjy dd8e9e2930 fix concurrent modification exceptions in local cache 2014-04-08 16:23:24 -07:00
nishantmonu51 2c8682a1b6 avoid using Math.abs 
review comment on - c6f5311
To avoid overflow issues and unnecessary calls to Math.abs.
 2014-04-08 21:58:54 +05:30
nishantmonu51 12991a959a actually fix it 2014-04-08 20:08:21 +05:30
nishantmonu51 c6f531168f fix hash based partition - boundary case Integer.MIN 2014-04-08 20:07:22 +05:30
fjy f5dc75ef82 Merge pull request #464 from metamx/fix-context 
Fix context to be backwards compatible with String values
 2014-04-07 11:54:17 -06:00
fjy dda09633be make query id a bit more readable 2014-04-07 10:53:43 -07:00
fjy 40c14c9a3c fix case with no request params 2014-04-06 12:43:40 -07:00