Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,750 Commits 111 Branches 503 Tags 725 MiB
Commit Graph

2594 Commits

Author SHA1 Message Date
Gian Merlino 23c993c9e7 Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports. 
- Add druid.indexer.server.maxChatRequests, which sets up a QoSFilter on the main Jetty server.
- Deprecate druid.indexer.runner.separateIngestionEndpoint
- Deprecate druid.indexer.server.chathandler.*
 2016-02-19 13:36:09 -08:00
Gian Merlino e0c049c0b0 Make startup properties logging optional. 
Off by default, but enabled in the example config files. See also #2452.
 2016-02-12 14:12:16 -08:00
Fangjin Yang 1430bc2c88 Merge pull request #2276 from harshjain2/feature-2021 
Fix for issue 2021.
 2016-02-10 17:04:45 -08:00
Gian Merlino fa92b77f5a Harmonize znode writing code in RTR and Worker. 
- Throw most exceptions rather than suppressing them, which should help
  detect problems. Continue suppressing exceptions that make sense to
  suppress.
- Handle payload length checks consistently, and improve error message.
- Remove unused WorkerCuratorCoordinator.announceTaskAnnouncement method.
- Max znode length should be int, not long.
- Add tests.
 2016-02-10 14:52:00 -08:00
Harsh Jain a3eb863c8e Fix for issue 2021 2016-02-10 22:19:12 +05:30
Himanshu Gupta b40c342cd1 make Global stupid pool cache size configurable 2016-02-05 14:18:06 -06:00
Parag Jain 9002548eeb increase test time out and general clean up 2016-02-03 13:26:37 -06:00
Charles Allen 5111fd52f2 Add check for log4j-core in Log4jShutterDownerModule 2016-02-02 15:56:48 -08:00
Himanshu dc89cdd0f9 Merge pull request #2336 from himanshug/fix_2331 
limit size of X-Druid-Response-Context header to 7K
 2016-02-02 12:06:59 -06:00
navis.ryu c03918f89a AsyncQueryForwardingServletTest#testDeleteBroadcast sometimes fails by port conflict 2016-01-29 19:28:58 +09:00
Himanshu Gupta f6b4dbd697 bug fix and unit tests for DruidCoordinatorSegmentKiller 2016-01-28 14:10:17 -06:00
Himanshu Gupta ab3edfa8fc moving DruidCoordinatorSegmentKiller class out of DruidCoordinator 2016-01-28 14:03:56 -06:00
Nishant 3880f54b87 Merge pull request #2332 from himanshug/configurable_partial 
make populateUncoveredIntervals a configuration in query context
 2016-01-28 10:34:35 +05:30
Himanshu Gupta a7bde8f4da limit size of X-Druid-Response-Context header to 7K due to https://github.com/druid-io/druid/issues/2331 2016-01-27 15:18:08 -06:00
Xavier Léauté 5a3642bb93 Merge pull request #2247 from metamx/pedanticBuild 
Enable strict building in travis
 2016-01-27 10:27:03 -08:00
Charles Allen 508734c8b0 Long constant reformatting in tests `l` --> `L` 2016-01-27 08:59:19 -08:00
Nishant fd6bf3fe22 Use interval comparator instead of bucketMonthComparator 
fix when two segments have same interval

review comments
 2016-01-27 17:35:43 +05:30
Himanshu Gupta 3719b6e3c8 make populateUncoveredIntervals a configuration in query context 2016-01-26 15:13:45 -06:00
Harsh Jain 41730b96d4 Fix for issue 2021 2016-01-25 02:48:22 +05:30
Himanshu 7a6109f0ca Merge pull request #2321 from gianm/pending-index 
Replace two-column index on pendingSegments table with one-column index.
 2016-01-22 13:38:15 -06:00
Gian Merlino 0bd9bff075 Replace two-column index on pendingSegments table with one-column index. 
Fixes #2319.
 2016-01-22 10:50:21 -08:00
Himank Chaudhary 1a5d4e714c Adding custom mapper for json processing exception to return bad request instead of 500 2016-01-22 09:48:52 -08:00
Fangjin Yang 04d3054353 Merge pull request #2303 from CHOIJAEHONG1/localfirehouse-basedir-npe 
Throw an IAE when baseDir is null in LocalFireHose
 2016-01-21 07:58:52 -08:00
Nishant dcb7830330 Merge pull request #984 from drcrallen/thread-priority-rebase 
Use thread priorities. (aka set `nice` values for background-like tasks)
 2016-01-21 15:02:34 +05:30
Charles Allen 2e1d6aaf3d Use thread priorities. (aka set `nice` values for background-like tasks) 
* Defaults the thread priority to java.util.Thread.NORM_PRIORITY in io.druid.indexing.common.task.AbstractTask
 * Each exec service has its own Task Factory which is assigned a priority for spawned task. Therefore each priority class has a unique exec service
 * Added priority to tasks as taskPriority in the task context. <0 means low, 0 means take default, >0 means high. It is up to any particular implementation to determine how to handle these numbers
 * Add options to ForkingTaskRunner
    * Add "-XX:+UseThreadPriorities" default option
    * Add "-XX:ThreadPriorityPolicy=42" default option
 * AbstractTask - Removed unneded @JsonIgnore on priority
 * Added priority to RealtimePlumber executors. All sub-executors (non query runners) get Thread.MIN_PRIORITY
 * Add persistThreadPriority and mergeThreadPriority to realtime tuning config
 2016-01-20 14:00:31 -08:00
Nishant 61aca6f9cc remove wrong checks 
sink never have null hydrants and hydrants never have null adapters
 2016-01-20 23:43:53 +05:30
Jaehong Choi 7132428bba throw IAE when baseDir is null in LocalFireHose 2016-01-21 01:27:32 +09:00
Nishant 59ea186af7 fix reference counting for segments 2016-01-20 17:24:21 +05:30
jon-wei 747343e621 Preserve dimension order across indexes during ingestion 2016-01-19 13:34:11 -08:00
Jonathan Wei df2906a91c Merge pull request #2290 from gianm/index-merger-v9-stuff 
Respect buildV9Directly in PlumberSchools, so it works on standalone realtime.
 2016-01-19 13:04:00 -08:00
Fangjin Yang 0c31f007fc Merge pull request #1728 from himanshug/aggregators_in_segment_metadata 
Store AggregatorFactory[] in segment metadata
 2016-01-19 12:55:49 -08:00
Himanshu fe841fd961 Merge pull request #2118 from guobingkun/fix_segment_loading 
Fix loading segment for historical
 2016-01-19 14:25:48 -06:00
Himanshu Gupta a99aef29a1 adding aggregators to segment metadata 2016-01-19 14:23:39 -06:00
Gian Merlino 1dcf22edb7 Respect buildV9Directly in PlumberSchools, so it works on standalone realtime nodes. 
Also parameterize some tests to run with/without buildV9Directly:

- IndexGeneratorJobTest
- RealtimeIndexTaskTest
- RealtimePlumberSchoolTest
 2016-01-19 12:15:06 -08:00
Bingkun Guo c4ad50f92c Fix loading segment for historical 
Historical will drop a segment that shouldn't be dropped in the following scenario:
Historical node tried to load segmentA, but failed with SegmentLoadingException,
then ZkCoordinator called removeSegment(segmentA, blah) to schedule a runnable that would drop segmentA by deleting its files. Now, before that runnable executed, another LOAD request was sent to this historical, this time historical actually succeeded on loading segmentA and announced it. But later on, the scheduled drop-of-segment runnable started executing and removed the segment files, while historical is still announcing segmentA.
 2016-01-19 10:29:49 -06:00
Himanshu Gupta 164b0aad7a removing Map<String,Object> segmentMetadata from methods in Index[Maker/Merger] and using Metadata class 
instead of a Map to store segment metadata
 2016-01-18 22:03:46 -06:00
Himanshu Gupta 637d2605e7 kill unwanted parent directories when a segment is deleted from LocalDataSegmentKiller 2016-01-18 21:51:04 -06:00
Harsh Jain 71f1cd5e34 Fix for issue 2021 2016-01-17 16:10:04 +05:30
Fangjin Yang f6a1a4ae20 Merge pull request #2138 from KurtYoung/feature-build-v9 
build v9 directly
 2016-01-16 13:35:46 -06:00
Kurt Young 82ff98c2bf add config for build v9 directly and update docs 2016-01-16 11:26:34 +08:00
Fangjin Yang 2e54553a8f Merge pull request #1990 from himanshug/schedule_kill_task 
support periodic hard delete of segments
 2016-01-15 15:22:33 -06:00
David Lim 7c65880e55 Merge pull request #2270 from rasahner/warnOfChatHandlerNoop 
if chathandler is noop, log using warn instead of info
 2016-01-15 13:29:27 -07:00
Harsh Jain 6ec6835b5d Fix for issue 2021. 2016-01-16 00:58:33 +05:30
David Lim 34cd8f8c72 Merge pull request #2258 from fjy/acl-zk 
acl for zookeeper is added
 2016-01-15 10:27:08 -07:00
Fangjin Yang a54c726726 Merge pull request #2266 from anubhgup/fix-announce 
Fix loss in segment announcements when segments do not fit in zNode
 2016-01-14 17:10:43 -08:00
Xavier Léauté dc1a62c3d9 Merge pull request #2248 from metamx/druidNodeHashEquals 
Add hashCode and equals to DruidNode
 2016-01-14 16:04:58 -08:00
Anubhav Gupta 6d09ab839f Fix for loss in segment announcements when segments do not fit in the znodes during compress mode. 
Added unit test (from Navis).
 2016-01-14 14:44:23 -08:00
Robin 7361cd173f if chathandler is noop, log using warn instead of info 2016-01-14 10:12:42 -06:00
Nikita Geer 1908d63162 acl for zookeeper is added 2016-01-13 14:56:05 -08:00
navis.ryu 18479bb757 time-descending result of timeseries queries 2016-01-13 12:23:01 +09:00