Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,038 Commits 326 Branches 371 Tags
Commit Graph

1339 Commits

Author SHA1 Message Date
Xuan
a0445eae5c YARN-3582. NPE in WebAppProxyServlet. Contributed by Jian He 
(cherry picked from commit a583a40693f5c56c40b39fd12cfa0bb7174fc526)
 2015-05-05 21:43:35 -07:00
Jian He
eb0c6d2ee7 YARN-3343. Increased TestCapacitySchedulerNodeLabelUpdate#testNodeUpdate timeout. Contributed by Rohith Sharmaks 
(cherry picked from commit e4c3b52c896291012f869ebc0a21e85e643fadd1)
 2015-05-05 11:34:11 -07:00
Xuan
02cdcaec58 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA

(cherry picked from commit b7dd3a4f04f712b7594c4e6e7ce50fd314f7c342)
 2015-05-05 10:45:18 -07:00
Junping Du
6ea007ba91 YARN-3396. Handle URISyntaxException in ResourceLocalizationService. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 38102420621308f5ba91cdeb6a18a63aa5acf640)
 2015-05-05 10:19:16 -07:00
Jason Lowe
0f30913ea8 YARN-3552. RM Web UI shows -1 running containers for completed apps. Contributed by Rohith 
(cherry picked from commit 9356cf8676fd18f78655e8a6f2e6c946997dbd40)
 2015-05-05 13:43:57 +00:00
Jian He
f71c49c5e3 YARN-2725. Added test cases of retrying creating znode in ZKRMStateStore. Contributed by Tsuyoshi Ozawa 
(cherry picked from commit d701acc9c67adc578ba18035edde1166eedaae98)
 2015-05-04 16:17:01 -07:00
Wangda Tan
4e77ee2f6f YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner (Devaraj K via wangda) 
(cherry picked from commit 71f4de220c74bf2c90630bd0442979d92380d304)
 2015-05-04 16:02:57 -07:00
Allen Wittenauer
02e650248d YARN-2980. Move health check script related functionality to hadoop-common (Varun Saxena via aw) 
(cherry picked from commit d4ac6822e1c5dfac504ced48f10ab57a55b49e93)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2015-05-04 16:00:20 -07:00
Jason Lowe
99b49a8181 YARN-3097. Logging of resource recovery on NM restart has redundancies. Contributed by Eric Payne 
(cherry picked from commit 8f65c793f2930bfd16885a2ab188a9970b754974)
 2015-05-04 10:32:33 -05:00
Gera Shegalov
9325dd863c YARN-2893. AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream. (Zhihai Xu via gera) 
(cherry picked from commit f8204e241d9271497defd4d42646fb89c61cefe3)
 2015-05-01 19:04:48 -07:00
Robert Kanter
32b3b8e2ef YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter) 
(cherry picked from commit ac7d152901e29b1f444507fe4e421eb6e1402b5a)
 2015-05-01 16:39:59 -07:00
Vinod Kumar Vavilapalli
90f6e8c90f YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. 
(cherry picked from commit 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)
 2015-04-30 21:43:22 -07:00
Zhijie Shen
c9ee316045 YARN-3544. Got back AM logs link on the RM web UI for a completed app. Contributed by Xuan Gong. 
(cherry picked from commit 21bf2cdcb77f69abc906e6cd401a8fb221f250e9)
 2015-04-30 13:06:46 -07:00
Wangda Tan
100b41add6 YARN-3564. Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable fails randomly. (Jian He via wangda) 
(cherry picked from commit e2e8f771183df798e926abc97116316a05b19c9a)
 2015-04-30 11:06:47 -07:00
Jian He
a7a5737bc4 YARN-3533. Test: Fix launchAM in MockRM to wait for attempt to be scheduled. Contributed by Anubhav Dhoot 
(cherry picked from commit 4c1af156aef4f3bb1d9823d5980c59b12007dc77)
 2015-04-29 14:51:04 -07:00
tgraves
2e13183f60 YARN-3517. RM web ui for dumping scheduler logs should be for admins only (Varun Vasudev via tgraves) 
(cherry picked from commit 2e215484bd05cd5e3b7a81d3558c6879a05dd2d2)
 2015-04-29 21:27:16 +00:00
Karthik Kambatla
4cc38df7de YARN-3485. FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies. (kasha) 
(cherry picked from commit 8f82970e0c247b37b2bf33aa21f6a39afa07efde)
 2015-04-28 21:03:12 -07:00
Wangda Tan
8ee632caa7 YARN-2740. Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled. (Naganarasimha G R via wangda) 
(cherry picked from commit db1b674b50ddecf2774f4092d677c412722bdcb1)
 2015-04-27 16:42:36 -07:00
Xuan
0e8af401d7 YARN-3530. ATS throws exception on trying to filter results without 
otherinfo. Contributed by zhijie shen

(cherry picked from commit 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55)
 2015-04-27 10:37:40 -07:00
Karthik Kambatla
4045c41afe YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) 
(cherry picked from commit 47279c3228185548ed09c36579b420225e4894f5)
 2015-04-26 09:14:12 -07:00
Jian He
9bf09b334d YARN-2498. Respect labels in preemption policy of capacity scheduler for inter-queue preemption. Contributed by Wangda Tan 
(cherry picked from commit d497f6ea2be559aa31ed76f37ae949dbfabe2a51)
 2015-04-24 17:03:57 -07:00
Tsuyoshi Ozawa
3884948d6c YARN-3406. Display count of running containers in the RM's Web UI. Contributed by Ryu Kobayashi. 
(cherry picked from commit 4a3dabd94fc3dddd9d7604b826065c23859d565f)
 2015-04-25 07:17:42 +09:00
Jason Lowe
cf4154676b YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked. Contributed by Brahma Reddy Battula 
(cherry picked from commit 5e093f0d400f82f67d9b2d24253c79e4a5abacf9)
 2015-04-24 22:04:03 +00:00
Jian He
0583c27fb1 YARN-3387. Previous AM's container completed status couldn't pass to current AM if AM and RM restarted during the same time. Contributed by Sandflee 
(cherry picked from commit d03dcb9635dbd79a45d229d1cab5fd28e5e49f49)
 2015-04-24 12:14:17 -07:00
Xuan
c184466939 YARN-3511. Add errors and warnings page to ATS. Contributed by Varun Vasudev 
(cherry picked from commit eee9facbbae52cb62dfca01b8bbe676b8e289863)
 2015-04-24 09:43:35 -07:00
Xuan
8f6053ae51 YARN-3516. killing ContainerLocalizer action doesn't take effect when 
private localizer receives FETCH_FAILURE status. Contributed by zhihai
xu

(cherry picked from commit 0b3f8957a87ada1a275c9904b211fdbdcefafb02)
 2015-04-23 16:41:52 -07:00
Vinod Kumar Vavilapalli
0ec6e7e7f5 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 
(cherry picked from commit f5fe35e297ed4a00a1ba93d090207ef67cebcc9d)
 2015-04-23 11:25:30 -07:00
Wangda Tan
9692bcc2c8 YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) 
(cherry picked from commit 395205444e8a9ae6fc86f0a441e98486a775511a)
 2015-04-23 10:48:00 -07:00
tgraves
1cd2fcf25d YARN-3434. Interaction between reservations and userlimit can result in significant ULF violation 
(cherry picked from commit 189a63a719c63b67a1783a280bfc2f72dcb55277)
 2015-04-23 14:49:24 +00:00
Steve Loughran
63ccb9167e YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly. (Xuan Gong via stevel) 2015-04-23 09:22:23 +01:00
Vinod Kumar Vavilapalli
04783b0402 YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers Contributed by Sidharta Seethana. 
(cherry picked from commit a100be685cc4521e9949589948219231aa5d2733)
 2015-04-22 17:36:55 -07:00
Junping Du
8de6045842 New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K 
(cherry picked from commit fad9d7e85b1ba0934ab592daa9d3c9550b2bb501)
 2015-04-22 10:08:16 -07:00
Jian He
613a783380 YARN-3503. Expose disk utilization percentage and bad local and log dir counts in NM metrics. Contributed by Varun Vasudev 
(cherry picked from commit 674c7ef64916fabbe59c8d6cdd50ca19cf7ddb7c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2015-04-21 21:06:06 -07:00
Jian He
dc4698bb33 YARN-3494. Expose AM resource limit and usage in CS QueueMetrics. Contributed by Rohith Sharmaks 
(cherry picked from commit bdd90110e6904b59746812d9a093924a65e72280)
 2015-04-21 20:34:30 -07:00
Wangda Tan
8a9599fb9e YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) 2015-04-21 17:47:27 -07:00
Tsuyoshi Ozawa
0b87ae27aa YARN-3495. Confusing log generated by FairScheduler. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 105afd54779852c518b978101f23526143e234a5)
 2015-04-22 05:48:16 +09:00
Wangda Tan
d0ea982e64 YARN-3463. Integrate OrderingPolicy Framework with CapacityScheduler. (Craig Welch via wangda) 
(cherry picked from commit 44872b76fcc0ddfbc7b0a4e54eef50fe8708e0f5)
 2015-04-20 17:13:01 -07:00
Jian He
6f70f1b1af YARN-3136. Fixed a synchronization problem of AbstractYarnScheduler#getTransferredContainers. Contributed by Sunil G 
(cherry picked from commit 497c86b485b1bb8a2eba52308646d8e1ee76bce3)
 2015-04-18 12:47:30 -07:00
Wangda Tan
3316cd4357 YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) 
(cherry picked from commit f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)
 2015-04-17 17:23:00 -07:00
Wangda Tan
e7cbecddc3 YARN-3493. RM fails to come up with error "Failed to load/recover state" when mem settings are changed. (Jian He via wangda) 
(cherry picked from commit f65eeb412d140a3808bcf99344a9f3a965918f70)
 2015-04-17 17:15:16 -07:00
Jian He
9827da95bc YARN-3451. Display attempt start time and elapsed time on the web UI. Contributed by Rohith Sharmaks 
(cherry picked from commit 6779467ab6fcc6a02d0e8c80b138cc9df1aa831e)
 2015-04-17 15:57:36 -07:00
Jian He
57eb07d34b YARN-2696. Queue sorting in CapacityScheduler should consider node label. Contributed by Wangda Tan 
(cherry picked from commit d573f09fb93dbb711d504620af5d73840ea063a6)
 2015-04-17 13:37:34 -07:00
Junping Du
b244701fad YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong. 
(cherry picked from commit 1db355a875c3ecc40a244045c6812e00c8d36ef1)
 2015-04-17 13:20:27 -07:00
Jian He
42cf8e0554 YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang 
(cherry picked from commit bb6dde68f19be1885a9e7f7949316a03825b6f3e)
 2015-04-16 19:44:32 -07:00
Akira Ajisaka
51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
 2015-04-16 10:06:03 -05:00
Jian He
6fed2c2a79 YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan 
(cherry picked from commit 1b89a3e173f8e905074ed6714a7be5c003c0e2c4)
 2015-04-15 14:03:29 -07:00
Jian He
58f99b740a YARN-3404. Display queue name on application page. Contributed by Ryu Kobayashi 
(cherry picked from commit b2e6cf607f1712d103520ca6b3ff21ecc07cd265)
 2015-04-15 13:53:24 -07:00
Tsuyoshi Ozawa
e977247f3e YARN-3326. Support RESTful API for getLabelsToNodes. Contributed by Naganarasimha G R. 
(cherry picked from commit e48cedc663b8a26fd62140c8e2907f9b4edd9785)
 2015-04-15 14:04:24 -05:00
Wangda Tan
89a7c98436 YARN-3318. Create Initial OrderingPolicy Framework and FifoOrderingPolicy. (Craig Welch via wangda) 
(cherry picked from commit 5004e753322084e42dfda4be1d2db66677f86a1e)
 2015-04-15 09:57:52 -07:00
Harsh J
01af29106a YARN-3462. Patches applied for YARN-2424 are inconsistent between trunk and branch-2. Contributed by Naganarasimha G R. 2015-04-15 09:41:43 +05:30