Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,057 Commits 326 Branches 371 Tags
Commit Graph

770 Commits

Author SHA1 Message Date
Karthik Kambatla
798ab51289 YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha) 
(cherry picked from commit e7257acd8a7adb74d81cd1d009d4a99f023ed844)
 2014-12-30 19:55:50 -08:00
Jian He
cde5bfe3ec YARN-2987. Fixed ClientRMService#getQueueInfo to check against queue and app ACLs. Contributed by Varun Saxena 
(cherry picked from commit e2351c7ae24cea9b217af4174512d279c55e8efd)
 2014-12-30 17:16:05 -08:00
Jian He
143e48c25a YARN-2493. Added node-labels page on RM web UI. Contributed by Wangda Tan 
(cherry picked from commit b7442bf92eb6e1ae64a0f9644ffc2eee4597aad5)
 2014-12-30 16:49:59 -08:00
Zhijie Shen
e5cb52ec1f YARN-2938. Fixed new findbugs warnings in hadoop-yarn-resourcemanager and hadoop-yarn-applicationhistoryservice. Contributed by Varun Saxena. 
(cherry picked from commit 241d3b3a50c6af92f023d8b2c24598f4813f4674)
 2014-12-29 10:07:54 -08:00
Jian He
ca0349b87a YARN-2992. ZKRMStateStore crashes due to session expiry. Contributed by Karthik Kambatla 
(cherry picked from commit 1454efe5d4fe4214ec5ef9142d55dbeca7dab953)
 2014-12-26 18:00:32 -08:00
Junping Du
fc3e810a19 YARN-2993. Several fixes (missing acl check, error log msg ...) and some refinement in AdminService. (Contributed by Yi Liu) 
(cherry picked from commit 40ee4bff65b2bfdabfd16ee7d9be3382a0476565)
 2014-12-26 07:01:44 -08:00
Jian He
60530a6c4f YARN-2946. Fixed potential deadlock in RMStateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 4f18018b7abb405f38d269da096ec9fd80c81d9a)
 2014-12-23 22:15:28 -08:00
Jian He
e3a54b1b8b YARN-2837. Support TimeLine server to recover delegation token when restarting. Contributed by Zhijie Shen 
(cherry picked from commit 149512a83743715bb100feb6747b7e88301018ab)
 2014-12-23 18:26:37 -08:00
Jian He
8b398a66ca YARN-2340. Fixed NPE when queue is stopped during RM restart. Contributed by Rohith Sharmaks 
(cherry picked from commit 0d89859b51157078cc504ac81dc8aa75ce6b1782)
 2014-12-22 21:55:06 -08:00
Jian He
411836b74c YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan 
(cherry picked from commit fdf042dfffa4d2474e3cac86cfb8fe9ee4648beb)
 2014-12-22 16:52:15 -08:00
Junping Du
6f252f7f36 YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du) 
(cherry picked from commit a696fbb001b946ae75f3b8e962839c2fd3decfa1)
 2014-12-22 03:08:25 -08:00
Karthik Kambatla
2abec14ec6 YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha) 
(cherry picked from commit 24ee9e3431d27811530ffa01d8d241133fd643fe)
 2014-12-20 12:20:29 -08:00
Jian He
9180d11b3b YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 808cba3821d5bc4267f69d14220757f01cd55715)
 2014-12-19 16:56:55 -08:00
Karthik Kambatla
8ee40a1580 YARN-2738. [YARN-2574] Add FairReservationSystem for FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit a22ffc318801698e86cd0e316b4824015f2486ac)
 2014-12-19 15:38:27 -08:00
Jason Lowe
173664d70f YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie). Contributed by Jian He 
(cherry picked from commit 0402bada1989258ecbfdc437cb339322a1f55a97)
 2014-12-18 23:31:11 +00:00
Junping Du
11e4ec52a1 YARN-2972. DelegationTokenRenewer thread pool never expands. Contributed by Jason Lowe 
(cherry picked from commit 2b4b0e8847048850206f091c6870a02e08cfe836)
 2014-12-17 17:41:21 -08:00
Karthik Kambatla
2d832ad2eb YARN-2964. FSLeafQueue#assignContainer - document the reason for using both write and read locks. (Tsuyoshi Ozawa via kasha) 
(cherry picked from commit f2d150ea1205b77a75c347ace667b4cd060aaf40)
 2014-12-17 16:24:17 -08:00
Steve Loughran
a858d726c8 YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel) 2014-12-12 17:10:54 +00:00
Devaraj K
10ff600b35 YARN-2243. Order of arguments for Preconditions.checkNotNull() is wrong in 
SchedulerApplicationAttempt ctor. Contributed by Devaraj K.

(cherry picked from commit bda748ac3abf30f6cd4c0e22c80c73396abc59fb)
 2014-12-12 12:37:13 +05:30
Jian He
0f47e4e745 YARN-2930. Fixed TestRMRestart#testRMRestartRecoveringNodeLabelManager intermittent failure. Contributed by Wangda Tan 
(cherry picked from commit 2ed90a57fdd31d194b4a690df68b158ed9743dba)
 2014-12-09 16:48:46 -08:00
Karthik Kambatla
1986ea8dd2 YARN-2910. FSLeafQueue can throw ConcurrentModificationException. (Wilfred Spiegelenburg via kasha) 
(cherry picked from commit a2e07a54561a57a83b943628ebbc53ed5ba52718)
 2014-12-09 14:00:52 -08:00
Jian He
4cc0abe4fe YARN-2869. CapacityScheduler should trim sub queue names when parse configuration. Contributed by Wangda Tan 
(cherry picked from commit e69af836f34f16fba565ab112c9bf0d367675b16)
 2014-12-05 17:34:15 -08:00
Jason Lowe
b72fb6c774 YARN-2056. Disable preemption at Queue level. Contributed by Eric Payne 
(cherry picked from commit 4b130821995a3cfe20c71e38e0f63294085c0491)
 2014-12-05 21:08:14 +00:00
Jian He
1d1e7682c9 YARN-2301. Improved yarn container command. Contributed by Naganarasimha G R 
(cherry picked from commit 258623ff8bb1a1057ae3501d4f20982d5a59ea34)
 2014-12-04 12:53:52 -08:00
Jian He
58c971164c YARN-2880. Added a test to make sure node labels will be recovered if RM restart is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit 73fbb3c66b0d90abee49c766ee9d2f05517cb9de)
 2014-12-03 17:17:40 -08:00
Karthik Kambatla
25be97808b YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps. (Naganarasimha G R via kasha) 
(cherry picked from commit 799353e2c7db5af6e40e3521439b5c8a3c5c6a51)
 2014-12-03 13:45:03 -08:00
Jian He
d6f3d4893d YARN-2894. Fixed a bug regarding application view acl when RM fails over. Contributed by Rohith Sharmaks 
(cherry picked from commit 392c3aaea8e8f156b76e418157fa347256283c56)
 2014-12-02 17:17:16 -08:00
Jian He
5067ac098b YARN-2136. Changed RMStateStore to ignore store opearations when fenced. Contributed by Varun Saxena 
(cherry picked from commit 52bcefca8bb13d3757009f1f08203e7dca3b1e16)
 2014-12-02 11:02:00 -08:00
Jian He
d208c90148 YARN-2765. Added leveldb-based implementation for RMStateStore. Contributed by Jason Lowe 
(cherry picked from commit a7fba0bc28764e0fb36c335ea60cc58079fe007f)
 2014-12-01 16:45:17 -08:00
Junping Du
c7bd22974a YARN-2907. SchedulerNode#toString should print all resource detail instead of only memory. (Contributed by Rohith) 
(cherry picked from commit c732ed760e115192652280cd4b5799d49e62db7e)
 2014-12-01 05:41:56 -08:00
Jian He
ae35b0e14d YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users. Contributed by Jason Lowe 
(cherry picked from commit 8a7ca13b13c0c3f008a6490cc96d4d48a051d1f7)
 2014-11-25 16:18:02 -08:00
Jian He
2863056530 YARN-2404. Removed ApplicationAttemptState and ApplicationState class in RMStateStore. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 5805a81efbc024024d8172489dfdc6cf77879416)
 2014-11-25 12:51:03 -08:00
Zhijie Shen
57d62d4ded YARN-2697. Remove useless RMAuthenticationHandler. Contributed by Haosong Huang. 
(cherry picked from commit e37a4ff0c1712a1cb80e0412ec53a5d10b8d30f9)
 2014-11-24 15:06:44 -08:00
Sandy Ryza
3e2e8eac22 YARN-2669. FairScheduler: queue names shouldn't allow periods (Wei Yan via Sandy Ryza) 2014-11-21 16:08:22 -08:00
Karthik Kambatla
e9db0aa35c YARN-2604. Scheduler should consider max-allocation-* in conjunction with the largest node. (Robert Kanter via kasha) 
(cherry picked from commit 3114d4731dcca7cb6c16aaa7c7a6550b7dd7dccb)
 2014-11-21 10:42:47 -08:00
Karthik Kambatla
7ba5bb0c5b YARN-2315. FairScheduler: Set current capacity in addition to capacity. (Zhihai Xu via kasha) 
(cherry picked from commit a9a0cc3679432774154a07d3157ffa0a43e0bf01)
 2014-11-19 20:19:54 -08:00
Karthik Kambatla
564deb75a8 YARN-2802. ClusterMetrics to include AM launch and register delays. (Zhihai Xu via kasha) 
(cherry picked from commit f9f8e750edbe6db453f0a845e2ed49ede66e0e8a)
 2014-11-19 19:50:35 -08:00
Jian He
db31ef7e7f YARN-2865. Fixed RM to always create a new RMContext when transtions from StandBy to Active. Contributed by Rohith Sharmaks 
(cherry picked from commit 9cb8b75ba57f18639492bfa3b7e7c11c00bb3d3b)
 2014-11-19 19:49:44 -08:00
Karthik Kambatla
8df4c04c3a YARN-2690. [YARN-2574] Make ReservationSystem and its dependent classes independent of Scheduler type. (Anubhav Dhoot via kasha) 
(cherry picked from commit 2fce6d61412843f0447f60cfe02326f769edae25)
 2014-11-17 16:46:18 -08:00
Jason Lowe
242fd0e39a YARN-2414. RM web UI: app page will crash if app is failed before any attempt has been created. Contributed by Wangda Tan 
(cherry picked from commit 81c9d17af84ed87b9ded7057cb726a3855ddd32d)
 2014-11-17 21:16:54 +00:00
Jian He
21054c5d14 YARN-2432. RMStateStore should process the pending events before close. Contributed by Varun Saxena 
(cherry picked from commit 9b860661a5754e829bd8c34ef092757421589bbe)
 2014-11-14 17:25:21 -08:00
Sandy Ryza
c4c77669f0 YARN-2811. In Fair Scheduler, reservation fulfillments shouldn't ignore max share (Siqi Li via Sandy Ryza) 2014-11-14 15:20:37 -08:00
Jian He
beb184ac58 YARN-2856. Fixed RMAppImpl to handle ATTEMPT_KILLED event at ACCEPTED state on app recovery. Contributed by Rohith Sharmaks 
(cherry picked from commit d005404ef7211fe96ce1801ed267a249568540fd)
 2014-11-13 15:34:56 -08:00
Vinod Kumar Vavilapalli
ed3e5cb164 YARN-2853. Fixed a bug in ResourceManager causing apps to hang when the user kill request races with ApplicationMaster finish. Contributed by Jian He. 
(cherry picked from commit 3651fe1b089851b38be351c00a9899817166bf3e)
 2014-11-13 08:13:57 -08:00
Jason Lowe
6984d899e3 YARN-2780. Log aggregated resource allocation in rm-appsummary.log. Contributed by Eric Payne 
(cherry picked from commit f8aefa5e9c8c6d2817205b5ed8d914db31f56ae7)
 2014-11-12 17:02:30 +00:00
Arun C. Murthy
adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Vinod Kumar Vavilapalli
d76fc94b21 YARN-2834. Fixed ResourceManager to ignore token-renewal failures on recovery consistent with the (somewhat incorrect) behaviour in the non-recovery case. Contributed by Jian He. 
(cherry picked from commit e76faebc9589654e83c8244ef9aff88391e56b80)
 2014-11-09 18:57:48 -08:00
Arun C. Murthy
175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 
(cherry picked from commit 43cd07b408c6613d2c9aa89203cfa3110d830538)
 2014-11-09 15:03:59 -08:00
Zhijie Shen
1880a5a7c3 YARN-2505. Supported get/add/remove/change labels in RM REST API. Contributed by Craig Welch. 
(cherry picked from commit 9a4e0d343e9e891c10ef6682e7b2231a59e69ade)
 2014-11-07 20:39:53 -08:00
Vinod Kumar Vavilapalli
b865a95f06 YARN-2826. Fixed user-groups mappings' refresh bug caused by YARN-2826. Contributed by Wangda Tan. 
(cherry picked from commit df36edf751202db00d8f43103d7120ec56d70a04)
 2014-11-07 19:44:52 -08:00