Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,944 Commits 326 Branches 371 Tags
Commit Graph

1207 Commits

Author SHA1 Message Date
Wangda Tan
8ee632caa7 YARN-2740. Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled. (Naganarasimha G R via wangda) 
(cherry picked from commit db1b674b50ddecf2774f4092d677c412722bdcb1)
 2015-04-27 16:42:36 -07:00
Jian He
9bf09b334d YARN-2498. Respect labels in preemption policy of capacity scheduler for inter-queue preemption. Contributed by Wangda Tan 
(cherry picked from commit d497f6ea2be559aa31ed76f37ae949dbfabe2a51)
 2015-04-24 17:03:57 -07:00
Tsuyoshi Ozawa
3884948d6c YARN-3406. Display count of running containers in the RM's Web UI. Contributed by Ryu Kobayashi. 
(cherry picked from commit 4a3dabd94fc3dddd9d7604b826065c23859d565f)
 2015-04-25 07:17:42 +09:00
Jian He
0583c27fb1 YARN-3387. Previous AM's container completed status couldn't pass to current AM if AM and RM restarted during the same time. Contributed by Sandflee 
(cherry picked from commit d03dcb9635dbd79a45d229d1cab5fd28e5e49f49)
 2015-04-24 12:14:17 -07:00
Xuan
c184466939 YARN-3511. Add errors and warnings page to ATS. Contributed by Varun Vasudev 
(cherry picked from commit eee9facbbae52cb62dfca01b8bbe676b8e289863)
 2015-04-24 09:43:35 -07:00
Vinod Kumar Vavilapalli
0ec6e7e7f5 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 
(cherry picked from commit f5fe35e297ed4a00a1ba93d090207ef67cebcc9d)
 2015-04-23 11:25:30 -07:00
Wangda Tan
9692bcc2c8 YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) 
(cherry picked from commit 395205444e8a9ae6fc86f0a441e98486a775511a)
 2015-04-23 10:48:00 -07:00
tgraves
1cd2fcf25d YARN-3434. Interaction between reservations and userlimit can result in significant ULF violation 
(cherry picked from commit 189a63a719c63b67a1783a280bfc2f72dcb55277)
 2015-04-23 14:49:24 +00:00
Steve Loughran
63ccb9167e YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly. (Xuan Gong via stevel) 2015-04-23 09:22:23 +01:00
Junping Du
8de6045842 New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K 
(cherry picked from commit fad9d7e85b1ba0934ab592daa9d3c9550b2bb501)
 2015-04-22 10:08:16 -07:00
Jian He
dc4698bb33 YARN-3494. Expose AM resource limit and usage in CS QueueMetrics. Contributed by Rohith Sharmaks 
(cherry picked from commit bdd90110e6904b59746812d9a093924a65e72280)
 2015-04-21 20:34:30 -07:00
Wangda Tan
8a9599fb9e YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) 2015-04-21 17:47:27 -07:00
Tsuyoshi Ozawa
0b87ae27aa YARN-3495. Confusing log generated by FairScheduler. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 105afd54779852c518b978101f23526143e234a5)
 2015-04-22 05:48:16 +09:00
Wangda Tan
d0ea982e64 YARN-3463. Integrate OrderingPolicy Framework with CapacityScheduler. (Craig Welch via wangda) 
(cherry picked from commit 44872b76fcc0ddfbc7b0a4e54eef50fe8708e0f5)
 2015-04-20 17:13:01 -07:00
Jian He
6f70f1b1af YARN-3136. Fixed a synchronization problem of AbstractYarnScheduler#getTransferredContainers. Contributed by Sunil G 
(cherry picked from commit 497c86b485b1bb8a2eba52308646d8e1ee76bce3)
 2015-04-18 12:47:30 -07:00
Wangda Tan
3316cd4357 YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) 
(cherry picked from commit f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)
 2015-04-17 17:23:00 -07:00
Wangda Tan
e7cbecddc3 YARN-3493. RM fails to come up with error "Failed to load/recover state" when mem settings are changed. (Jian He via wangda) 
(cherry picked from commit f65eeb412d140a3808bcf99344a9f3a965918f70)
 2015-04-17 17:15:16 -07:00
Jian He
9827da95bc YARN-3451. Display attempt start time and elapsed time on the web UI. Contributed by Rohith Sharmaks 
(cherry picked from commit 6779467ab6fcc6a02d0e8c80b138cc9df1aa831e)
 2015-04-17 15:57:36 -07:00
Jian He
57eb07d34b YARN-2696. Queue sorting in CapacityScheduler should consider node label. Contributed by Wangda Tan 
(cherry picked from commit d573f09fb93dbb711d504620af5d73840ea063a6)
 2015-04-17 13:37:34 -07:00
Junping Du
b244701fad YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong. 
(cherry picked from commit 1db355a875c3ecc40a244045c6812e00c8d36ef1)
 2015-04-17 13:20:27 -07:00
Jian He
42cf8e0554 YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang 
(cherry picked from commit bb6dde68f19be1885a9e7f7949316a03825b6f3e)
 2015-04-16 19:44:32 -07:00
Akira Ajisaka
51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
 2015-04-16 10:06:03 -05:00
Jian He
6fed2c2a79 YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan 
(cherry picked from commit 1b89a3e173f8e905074ed6714a7be5c003c0e2c4)
 2015-04-15 14:03:29 -07:00
Tsuyoshi Ozawa
e977247f3e YARN-3326. Support RESTful API for getLabelsToNodes. Contributed by Naganarasimha G R. 
(cherry picked from commit e48cedc663b8a26fd62140c8e2907f9b4edd9785)
 2015-04-15 14:04:24 -05:00
Wangda Tan
89a7c98436 YARN-3318. Create Initial OrderingPolicy Framework and FifoOrderingPolicy. (Craig Welch via wangda) 
(cherry picked from commit 5004e753322084e42dfda4be1d2db66677f86a1e)
 2015-04-15 09:57:52 -07:00
Jian He
9ebbf1bfce YARN-3361. CapacityScheduler side changes to support non-exclusive node labels. Contributed by Wangda Tan 
(cherry picked from commit 0fefda645bca935b87b6bb8ca63e6f18340d59f5)
 2015-04-14 11:46:35 -07:00
Jian He
81bbee6852 YARN-3266. RMContext#inactiveNodes should have NodeId as map key. Contributed by Chengbing Liu 
(cherry picked from commit b46ee1e7a31007985b88072d9af3d97c33a261a7)
 2015-04-14 11:31:42 -07:00
Jian He
d58f5c8894 YARN-3472. Fixed possible leak in DelegationTokenRenewer#allTokens. Contributed by Rohith Sharmaks 
(cherry picked from commit a1afbc48b53f6bdbd30dc8eb56a7621d49c5d6db)
 2015-04-13 14:07:51 -07:00
Jian He
32b01263d3 YARN-3382. Some of UserMetricsInfo metrics are incorrectly set to root queue metrics. Contributed by Rohit Agarwal 
(cherry picked from commit 944a16579fdb54c1c420162ef830ef2ac5451c69)
 2015-04-13 10:54:15 -07:00
Karthik Kambatla
ad99d268e5 YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) 
(cherry picked from commit e516706b896743e47e2852be81944eb5613e3e76)
 2015-04-10 11:21:30 -07:00
Jian He
d3daf9665c YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev 2015-04-10 10:40:31 -07:00
Junping Du
0891de0712 YARN-1376. NM need to notify the log aggregation status to RM through Node heartbeat. Contributed by Xuan Gong. 
(cherry picked from commit 92431c961741747b5d6442f4025016d48d9a6863)
 2015-04-10 08:58:07 -07:00
Xuan
f5b49160d9 YARN-3293. Track and display capacity scheduler health metrics in web 
UI. Contributed by Varun Vasudev

(cherry picked from commit afa5d4715a3aea2a6e93380b014c7bb8f0880383)
 2015-04-09 23:40:02 -07:00
Vinod Kumar Vavilapalli
1ff3fd33ed YARN-3055. Fixed ResourceManager's DelegationTokenRenewer to not stop token renewal of applications part of a bigger workflow. Contributed by Daryn Sharp. 
(cherry picked from commit 9c5911294e0ba71aefe4763731b0e780cde9d0ca)
 2015-04-09 13:10:06 -07:00
Wangda Tan
73d9e4c77b YARN-3466. Fix RM nodes web page to sort by node HTTP-address, #containers and node-label column. (Jason Lowe via wangda) 
(cherry picked from commit 1885141e90837252934192040a40047c7adbc1b5)
 2015-04-09 10:36:10 -07:00
Xuan
0522d6970d YARN-3294. Allow dumping of Capacity Scheduler debug logs via web UI for 
a fixed time period. Contributed by Varun Vasudev

(cherry picked from commit d27e9241e8676a0edb2d35453cac5f9495fcd605)
 2015-04-07 09:54:29 -07:00
Robert Kanter
354d2c75cf YARN-2429. TestAMRMTokens.testTokenExpiry fails Intermittently with error message:Invalid AMRMToken (zxu via rkanter) 
(cherry picked from commit 99b08a748e7b00a58b63330b353902a6da6aae27)
 2015-04-06 14:11:50 -07:00
Tsuyoshi Ozawa
d2b2d76cce TestFairScheduler.testContinuousScheduling fails Intermittently. Contributed by Zhihai Xu. 
(cherry picked from commit 53959e69f7ef4442324627fbfb159012968a2092)
 2015-04-06 20:19:47 +09:00
Xuan
1eb6d74967 YARN-3435. AM container to be allocated Appattempt AM container shown as 
null. Contributed by Bibin A Chundatt

(cherry picked from commit 96d72118f5c81aa4e0dca0dc0241fbf1a3fff4d2)
 2015-04-05 00:43:22 -07:00
Wangda Tan
405aead879 YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda) 
(cherry picked from commit bad070fe15a642cc6f3a165612fbd272187e03cb)
 2015-04-02 17:23:44 -07:00
Sandy Ryza
6286cfd711 YARN-3415. Non-AM containers can be counted towards amResourceUsage of a fairscheduler queue (Zhihai Xu via Sandy Ryza) 2015-04-02 13:57:53 -07:00
Zhijie Shen
6d0bebc912 YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong. 
(cherry picked from commit 8366a36ad356e6318b8ce6c5c96e201149f811bd)
 2015-04-01 13:49:00 -07:00
Xuan
e26b6e55e9 YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev

(cherry picked from commit 4728bdfa15809db4b8b235faa286c65de4a48cf6)
 2015-04-01 04:20:37 -07:00
Karthik Kambatla
85c420a4d4 YARN-3412. RM tests should use MockRM where possible. (kasha) 
(cherry picked from commit 79f7f2aabfd7a69722748850f4d3b1ff54af7556)
 2015-03-31 09:22:16 -07:00
Tsuyoshi Ozawa
99b8255693 YARN-3258. FairScheduler: Need to add more logging to investigate allocations. Contributed by Anubhav Dhoot. 
(cherry picked from commit b5a22e983832d4843b5df1d07858988e8bbf37e3)
 2015-03-31 17:43:07 +09:00
Wangda Tan
cba4ed1678 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 
(cherry picked from commit 2a945d24f7de1a7ae6e7bd6636188ce3b55c7f52)
 2015-03-30 12:05:54 -07:00
Wangda Tan
29629020c1 YARN-3383. AdminService should use warn instead of info to log exception when operation fails. (Li Lu via wangda) 
(cherry picked from commit 97a7277a2d696474b5c8e2d814c8291d4bde246e)
 2015-03-24 10:39:45 -07:00
Karthik Kambatla
4e0c48703e YARN-2868. FairScheduler: Metric for latency to allocate first container for an application. (Ray Chiang via kasha) 
(cherry picked from commit 972f1f1ab94a26ec446a272ad030fe13f03ed442)
 2015-03-23 14:10:23 -07:00
Karthik Kambatla
75591e4131 YARN-3241. FairScheduler handles invalid queue names inconsistently. (Zhihai Xu via kasha) 
(cherry picked from commit 2bc097cd14692e6ceb06bff959f28531534eb307)
 2015-03-23 13:24:22 -07:00
cnauroth
342c525eaa YARN-3336. FileSystem memory leak in DelegationTokenRenewer. 
(cherry picked from commit 6ca1f12024fd7cec7b01df0f039ca59f3f365dc1)
 2015-03-23 10:46:06 -07:00