Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,254 Commits 326 Branches 371 Tags
Commit Graph

796 Commits

Author SHA1 Message Date
Zhijie Shen
3ab3caa9ea YARN-2543. Made resource usage be published to the timeline server too. Contributed by Naganarasimha G R. 
(cherry picked from commit e5b56e2af6b48cbc8501f624ae4b4f9d13eccacf)
 2015-02-03 17:35:40 -08:00
Wangda Tan
205e15c4a3 YARN-3075. NodeLabelsManager implementation to retrieve label to node mapping (Varun Saxena via wangda) 
(cherry picked from commit 5bd984691b3e3df5d8aac910295138996e7a5839)
 2015-02-03 12:55:25 -08:00
Jian He
c0b1311a93 YARN-3098. Created common QueueCapacities class in Capacity Scheduler to track capacities-by-labels of queues. Contributed by Wangda Tan 
(cherry picked from commit 21d80b3dd90a8e33e51701887c8d9369ed4ab17d)
 2015-02-03 11:43:40 -08:00
Jason Lowe
3828fb5fab YARN-3085. Application summary should include the application type. Contributed by Rohith 
(cherry picked from commit a761bf8726cb6285a505f1773ea3403eb985a65f)
 2015-02-03 14:57:25 +00:00
Jian He
4941b8f30d YARN-3077. Fixed RM to create zk root path recursively. Contributed by Chun Chen 
(cherry picked from commit 054a947989d6ccbe54a803ca96dcebeba8328367)
 2015-01-30 17:35:26 -08:00
Jian He
cabf97ae4f YARN-3099. Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 
(cherry picked from commit 86358221fc85a7743052a0b4c1647353508bf308)
 2015-01-30 15:16:22 -08:00
Akira Ajisaka
1081b4c4a7 HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka) 
(cherry picked from commit 342efa110af9989829b8a8caa699658a2ba01607)
 2015-01-30 02:50:46 +09:00
Wangda Tan
58d7d1efc6 YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) 
(cherry picked from commit 7882bc0f1433ae73361cab4207eb0c15abee4586)
 2015-01-28 21:56:17 -08:00
Wangda Tan
4718af4352 YARN-2932. Add entry for preemptable status (enabled/disabled) to scheduler web UI and queue initialize/refresh logging. (Eric Payne via wangda) 
(cherry picked from commit 18741adf97f4fda5f8743318b59c440928e51297)
 2015-01-27 15:37:02 -08:00
Jian He
61b4116b4b YARN-3092. Created a common ResourceUsage class to track labeled resource usages in Capacity Scheduler. Contributed by Wangda Tan 
(cherry picked from commit 6f9fe76918bbc79109653edc6cde85df05148ba3)
 2015-01-26 15:38:27 -08:00
Tsuyoshi Ozawa
25f2cf8a4e YARN-2800. Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature. Contributed by Wangda Tan. 
(cherry picked from commit 24aa462673d392fed859f8088acf9679ae62a129)
 2015-01-23 20:37:26 +09:00
Wangda Tan
ef6fc24dfb YARN-2933. Capacity Scheduler preemption policy should only consider capacity without labels temporarily. Contributed by Mayank Bansal 
(cherry picked from commit 0a2d3e717d9c42090a32ff177991a222a1e34132)
 2015-01-19 16:57:17 -08:00
Junping Du
7cddec31d7 YARN-3064. TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout. (Contributed by Jian He) 
(cherry picked from commit 5d1cca34fa14706586f8bbe7f2a2264757fdcc11)
 2015-01-16 00:10:36 -08:00
Jian He
ed5a0d0aa8 YARN-2861. Fixed Timeline DT secret manager to not reuse RM's configs. Contributed by Zhijie Shen 
(cherry picked from commit 9e33116d1d8944a393937337b3963e192b9c74d1)
 2015-01-15 11:08:19 -08:00
Jian He
4931600030 YARN-2637. Fixed max-am-resource-percent calculation in CapacityScheduler when activating applications. Contributed by Craig Welch 
(cherry picked from commit c53420f58364b11fbda1dace7679d45534533382)
 2015-01-13 17:32:36 -08:00
Robert Kanter
13cdcf28e0 YARN-2643. Don't create a new DominantResourceCalculator on every FairScheduler.allocate call. (kasha via rkanter) 
(cherry picked from commit 51881535e659940b1b332d0c5952ee1f9958cc7f)
 2015-01-12 17:51:45 -08:00
Robert Kanter
e5059b91bb YARN-3027. Scheduler should use totalAvailable resource from node instead of availableResource for maxAllocation. (adhoot via rkanter) 
(cherry picked from commit ae7bf31fe1c63f323ba5271e50fd0e4425a7510f)
 2015-01-12 10:48:45 -08:00
Jian He
36b3dcaab2 YARN-3014. Replaces labels on a host should update all NM's labels on that host. Contributed by Wangda Tan 
(cherry picked from commit a2604062681230442eefae79815db5d77ccc4c2e)
 2015-01-09 17:50:18 -08:00
Zhijie Shen
d2fbba790a YARN-2996. Improved synchronization and I/O operations of FS- and Mem- RMStateStore. Contributed by Yi Liu. 
(cherry picked from commit dc2eaa26b20cfbbcdd5784bb8761d08a42f29605)
 2015-01-08 09:49:27 -08:00
Jian He
d3b26454e4 YARN-3010. Fixed findbugs warning in AbstractYarnScheduler. Contributed by Yi Liu 
(cherry picked from commit e13a484a2be64fb781c5eca5ae7056cbe194ac5e)
 2015-01-07 14:12:54 -08:00
Zhijie Shen
8a81312062 YARN-2427. Added the API of moving apps between queues in RM web services. Contributed by Varun Vasudev. 
(cherry picked from commit 60103fca04dc713183e4ec9e12f961642e7d1001)
 2015-01-06 14:39:44 -08:00
Jian He
c61e8a7bfa YARN-2978. Fixed potential NPE while getting queue info. Contributed by Varun Saxena 
(cherry picked from commit dd57c2047bfd21910acc38c98153eedf1db75169)
 2015-01-06 14:16:43 -08:00
Karthik Kambatla
7adffad2bb YARN-2881. [YARN-2574] Implement PlanFollower for FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit 0c4b11267717eb451fa6ed4c586317f2db32fbd5)
 2015-01-06 04:42:55 +05:30
Karthik Kambatla
d070597a0e YARN-2360. Fair Scheduler: Display dynamic fair share for queues on the scheduler page. (Ashwin Shankar and Wei Yan via kasha) 
(cherry picked from commit 270a271f53f52a1f33e63a0520ae6032536dd4c6)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-01-06 03:28:48 +05:30
Zhijie Shen
c6cf748985 YARN-2958. Made RMStateStore not update the last sequence number when updating the delegation token. Contributed by Varun Saxena. 
(cherry picked from commit 562a701945be3a672f9cb5a52cc6db2c1589ba2b)
 2015-01-05 13:35:09 -08:00
Tsuyoshi Ozawa
c116743bdd YARN-2922. ConcurrentModificationException in CapacityScheduler's LeafQueue. Contributed by Rohith Sharmaks. 
(cherry picked from commit ddc5be48fc35868abf7f59088f747c636e76a42a)
 2015-01-05 00:11:01 +09:00
Karthik Kambatla
798ab51289 YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha) 
(cherry picked from commit e7257acd8a7adb74d81cd1d009d4a99f023ed844)
 2014-12-30 19:55:50 -08:00
Jian He
cde5bfe3ec YARN-2987. Fixed ClientRMService#getQueueInfo to check against queue and app ACLs. Contributed by Varun Saxena 
(cherry picked from commit e2351c7ae24cea9b217af4174512d279c55e8efd)
 2014-12-30 17:16:05 -08:00
Jian He
143e48c25a YARN-2493. Added node-labels page on RM web UI. Contributed by Wangda Tan 
(cherry picked from commit b7442bf92eb6e1ae64a0f9644ffc2eee4597aad5)
 2014-12-30 16:49:59 -08:00
Zhijie Shen
e5cb52ec1f YARN-2938. Fixed new findbugs warnings in hadoop-yarn-resourcemanager and hadoop-yarn-applicationhistoryservice. Contributed by Varun Saxena. 
(cherry picked from commit 241d3b3a50c6af92f023d8b2c24598f4813f4674)
 2014-12-29 10:07:54 -08:00
Jian He
ca0349b87a YARN-2992. ZKRMStateStore crashes due to session expiry. Contributed by Karthik Kambatla 
(cherry picked from commit 1454efe5d4fe4214ec5ef9142d55dbeca7dab953)
 2014-12-26 18:00:32 -08:00
Junping Du
fc3e810a19 YARN-2993. Several fixes (missing acl check, error log msg ...) and some refinement in AdminService. (Contributed by Yi Liu) 
(cherry picked from commit 40ee4bff65b2bfdabfd16ee7d9be3382a0476565)
 2014-12-26 07:01:44 -08:00
Jian He
60530a6c4f YARN-2946. Fixed potential deadlock in RMStateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 4f18018b7abb405f38d269da096ec9fd80c81d9a)
 2014-12-23 22:15:28 -08:00
Jian He
e3a54b1b8b YARN-2837. Support TimeLine server to recover delegation token when restarting. Contributed by Zhijie Shen 
(cherry picked from commit 149512a83743715bb100feb6747b7e88301018ab)
 2014-12-23 18:26:37 -08:00
Jian He
8b398a66ca YARN-2340. Fixed NPE when queue is stopped during RM restart. Contributed by Rohith Sharmaks 
(cherry picked from commit 0d89859b51157078cc504ac81dc8aa75ce6b1782)
 2014-12-22 21:55:06 -08:00
Jian He
411836b74c YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan 
(cherry picked from commit fdf042dfffa4d2474e3cac86cfb8fe9ee4648beb)
 2014-12-22 16:52:15 -08:00
Junping Du
6f252f7f36 YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du) 
(cherry picked from commit a696fbb001b946ae75f3b8e962839c2fd3decfa1)
 2014-12-22 03:08:25 -08:00
Karthik Kambatla
2abec14ec6 YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha) 
(cherry picked from commit 24ee9e3431d27811530ffa01d8d241133fd643fe)
 2014-12-20 12:20:29 -08:00
Jian He
9180d11b3b YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 808cba3821d5bc4267f69d14220757f01cd55715)
 2014-12-19 16:56:55 -08:00
Karthik Kambatla
8ee40a1580 YARN-2738. [YARN-2574] Add FairReservationSystem for FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit a22ffc318801698e86cd0e316b4824015f2486ac)
 2014-12-19 15:38:27 -08:00
Jason Lowe
173664d70f YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie). Contributed by Jian He 
(cherry picked from commit 0402bada1989258ecbfdc437cb339322a1f55a97)
 2014-12-18 23:31:11 +00:00
Junping Du
11e4ec52a1 YARN-2972. DelegationTokenRenewer thread pool never expands. Contributed by Jason Lowe 
(cherry picked from commit 2b4b0e8847048850206f091c6870a02e08cfe836)
 2014-12-17 17:41:21 -08:00
Karthik Kambatla
2d832ad2eb YARN-2964. FSLeafQueue#assignContainer - document the reason for using both write and read locks. (Tsuyoshi Ozawa via kasha) 
(cherry picked from commit f2d150ea1205b77a75c347ace667b4cd060aaf40)
 2014-12-17 16:24:17 -08:00
Steve Loughran
a858d726c8 YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel) 2014-12-12 17:10:54 +00:00
Devaraj K
10ff600b35 YARN-2243. Order of arguments for Preconditions.checkNotNull() is wrong in 
SchedulerApplicationAttempt ctor. Contributed by Devaraj K.

(cherry picked from commit bda748ac3abf30f6cd4c0e22c80c73396abc59fb)
 2014-12-12 12:37:13 +05:30
Jian He
0f47e4e745 YARN-2930. Fixed TestRMRestart#testRMRestartRecoveringNodeLabelManager intermittent failure. Contributed by Wangda Tan 
(cherry picked from commit 2ed90a57fdd31d194b4a690df68b158ed9743dba)
 2014-12-09 16:48:46 -08:00
Karthik Kambatla
1986ea8dd2 YARN-2910. FSLeafQueue can throw ConcurrentModificationException. (Wilfred Spiegelenburg via kasha) 
(cherry picked from commit a2e07a54561a57a83b943628ebbc53ed5ba52718)
 2014-12-09 14:00:52 -08:00
Jian He
4cc0abe4fe YARN-2869. CapacityScheduler should trim sub queue names when parse configuration. Contributed by Wangda Tan 
(cherry picked from commit e69af836f34f16fba565ab112c9bf0d367675b16)
 2014-12-05 17:34:15 -08:00
Jason Lowe
b72fb6c774 YARN-2056. Disable preemption at Queue level. Contributed by Eric Payne 
(cherry picked from commit 4b130821995a3cfe20c71e38e0f63294085c0491)
 2014-12-05 21:08:14 +00:00
Jian He
1d1e7682c9 YARN-2301. Improved yarn container command. Contributed by Naganarasimha G R 
(cherry picked from commit 258623ff8bb1a1057ae3501d4f20982d5a59ea34)
 2014-12-04 12:53:52 -08:00