Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,172 Commits 326 Branches 371 Tags
Commit Graph

3906 Commits

Author SHA1 Message Date
Rohith Sharma K S
53033c69f3 YARN-7190. Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath. Contributed by Varun Saxena. 
(cherry picked from commit 09d996fdd429a85822a06ab87f0e2322d0d7ca68)
 2017-12-15 22:17:43 +05:30
Weiwei Yang
1011bfd220 YARN-7642. Add test case to verify context update after container promotion or demotion with or without auto update. Contributed by Weiwei Yang. 
(cherry picked from commit 89b6c482c1720a7f7ac86ce022c403825a086fa0)
 2017-12-15 12:55:03 +08:00
Sunil G
9ee1649cec YARN-7119. Support multiple resource types in rmadmin updateNodeResource command. Contributed by Manikandan R. 
(cherry picked from commit 2564b4d07f63de142fe1909d61904831c108667c)
 2017-12-14 16:12:43 +05:30
Sunil G
6d814ae068 YARN-7383. Node resource is not parsed correctly for resource names containing dot. Contributed by Gergely Novák. 
(cherry picked from commit 10fc8d2a7dc975ce70de63f88a2674389314f197)
 2017-12-13 22:01:28 +05:30
Weiwei Yang
415c69b36f YARN-7647. NM print inappropriate error log when node-labels is enabled. Contributed by Yang Wang. 2017-12-13 13:18:54 +08:00
Jason Lowe
472c7edc62 YARN-7595. Container launching code suppresses close exceptions after writes. Contributed by Jim Brennan 
(cherry picked from commit 2abab1d7c53e64c160384fd5a3ac4cd8ffa57af4)
 2017-12-12 16:07:47 -06:00
Weiwei Yang
3dc1ea6050 YARN-7608. Incorrect sTarget column causing DataTable warning on RM application and scheduler web page. Contributed by Gergely Novák. 2017-12-11 10:33:46 +08:00
Wangda Tan
41796ea60a YARN-7591. NPE in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) 
Change-Id: I67e086d6d35c3166ec0f6305490ea6527bd6d799
 2017-12-08 21:43:28 -08:00
Weiwei Yang
27c7a1f221 YARN-7607. Remove the trailing duplicated timestamp in container diagnostics message. Contributed by Weiwei Yang. 2017-12-07 17:36:31 +08:00
Robert Kanter
cdf305c337 YARN-6483. addendum to skip failing test (asuresh via rkanter) 2017-12-06 12:10:59 -08:00
Weiwei Yang
8bfd30b921 YARN-7610. Extend Distributed Shell to support launching job with opportunistic containers. Contributed by Weiwei Yang. 
(cherry picked from commit 40b0045ebe0752cd3d1d09be00acbabdea983799)
 2017-12-06 19:10:34 +08:00
Weiwei Yang
48af34571f YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang. 
(cherry picked from commit 05c347fe51c01494ed8110f8f116a01c90205f13)
 2017-12-06 12:24:00 +08:00
Wangda Tan
5cb1b97905 YARN-7381. Enable the configuration: yarn.nodemanager.log-container-debug-info.enabled by default in yarn-default.xml. (Xuan Gong via wangda) 
Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87
(cherry picked from commit 0311cf05358cd75388f48f048c44fba52ec90f00)
 2017-12-05 13:10:15 -08:00
Sunil G
f034668b91 YARN-6907. Node information page in the old web UI should report resource types. Contributed by Gergely Novák. 
(cherry picked from commit 81f6e46b2fb54659a08864677ad14f80fe4e452d)
 2017-12-04 12:19:27 +05:30
Arun Suresh
6f123aae41 YARN-7587. Skip dispatching opportunistic containers to nodes whose queue is already full. (Weiwei Yang via asuresh) 
(cherry picked from commit 37ca4169508c3003dbe9044fefd37eb8cd8c0503)
 2017-12-03 22:23:46 -08:00
Jason Lowe
5db3ca39dc YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim Brennan 
(cherry picked from commit 60f95fb719f00a718b484c36a823ec5aa8b099f4)
 2017-12-01 15:48:33 -06:00
Jason Lowe
f222b9d362 YARN-7589. TestPBImplRecords fails with NullPointerException. Contributed by Daniel Templeton 
(cherry picked from commit 25df5054216a6a76d09d9c49984f8075ebc6a197)
 2017-12-01 15:38:57 -06:00
Junping Du
58d849b858 YARN-7558. yarn logs command fails to get logs for running containers if UI authentication is enabled. Contributed by Xuan Gong. 
(cherry picked from commit a409425986fc128bb54f656b05373201545f7213)
 2017-11-30 13:49:00 -08:00
Jason Lowe
f072caa0bc YARN-7495. Improve robustness of the AggregatedLogDeletionService. Contributed by Jonathan Eagles 
(cherry picked from commit 5cfaee2e6db8b2ac55708de0968ff5539ee3bd76)
 2017-11-30 12:40:45 -06:00
Daniel Templeton
40372be7bb YARN-7541. Node updates don't update the maximum cluster capability for resources other than CPU and memory 
(cherry picked from commit 8498d287cd3beddcf8fe19625227e09982ec4be2)
 2017-11-30 09:26:25 -08:00
Jason Lowe
a4f1e3036d YARN-6647. RM can crash during transitionToStandby due to InterruptedException. Contributed by Bibin A Chundatt 
(cherry picked from commit a2c7a73e33045ce42cce19aacbe45c0421a61994)
 2017-11-28 11:19:43 -06:00
Yufei Gu
c0ab3c8407 YARN-7363. ContainerLocalizer don't have a valid log4j config in case of Linux container executor. (Contributed by Yufei Gu) 
(cherry picked from commit d8923cdbf1567aee10a54f144fef734d1465ebed)
 2017-11-27 14:33:13 -08:00
Yufei Gu
f335d509d3 YARN-7290. Method canContainerBePreempted can return true when it shouldn't. (Contributed by Steven Rand) 
(cherry picked from commit 2bde3aedf139368fc71f053d8dd6580b498ff46d)
 2017-11-24 23:39:03 -08:00
Wangda Tan
6e038165a6 YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) 
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
(cherry picked from commit 834e91ee91d22d74866afbf6252107e969bf8370)
 2017-11-23 20:02:34 -08:00
Arun Suresh
5884b8a80a YARN-6483. Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM. (Juan Rodriguez Hortala via asuresh) 
(cherry picked from commit b46ca7e73b8bac3fdbff0b13afe009308078acf2)
 2017-11-22 19:19:52 -08:00
Yufei Gu
b85eb9bb53 YARN-7524. Remove unused FairSchedulerEventLog. (Contributed by Wilfred Spiegelenburg) 
(cherry picked from commit 4cc9479dae2bfb7d14d29b55d103eea9fa35a586)
 2017-11-22 14:19:36 -08:00
yufei
717634e45c YARN-7513. Remove the scheduler lock in FSAppAttempt.getWeight() (Contributed by Wilfred Spiegelenburg) 
(cherry picked from commit 03c311eae3ad591630a452921172a4406dbda181)
 2017-11-21 10:35:06 -08:00
Haibo Chen
b419e8aa25 YARN-7531. ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType(). 2017-11-21 09:12:27 -08:00
Wangda Tan
adc5157a3c YARN-7527. Over-allocate node resource in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) 
Change-Id: I51ae6c2ab7a3d1febdd7d8d0519b63a13295ac7d
(cherry picked from commit 0d781dd03b979d65de94978071b2faa55005b34a)
 2017-11-20 13:14:17 -08:00
bibinchundatt
fbcadfe3c4 YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 
(cherry picked from commit b5b81a4f086126974c75dab9b54aea20a0c1a48f)
 2017-11-18 20:01:01 +05:30
bibinchundatt
ae8d1b3c34 YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 
(cherry picked from commit 96e6a993609d170b4d2c70b2dba8b4530093606f)
 2017-11-18 19:11:40 +05:30
Varun Vasudev
6681e7238b YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
 2017-11-17 12:05:19 +05:30
Haibo Chen
9bbe7eb226 YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. (Yufei Gu via Haibo Chen) 
(cherry picked from commit 28d0fcbef40930ca5652c0e9a5d777910f3ad3c4)
 2017-11-16 10:49:40 -08:00
Sunil G
c3fb49667a YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-16 22:35:22 +05:30
Jason Lowe
9780d58a63 YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)
 2017-11-15 12:32:56 -06:00
Daniel Templeton
570bc80901 YARN-7414. FairScheduler#getAppWeight() should be moved into FSAppAttempt#getWeight() 
(Contributed by Soumabrata Chakraborty via Daniel Templeton)

(cherry picked from commit b246c547490dd94271806ca4caf1e5f199f0fb09)
 2017-11-15 10:06:17 -08:00
Daniel Templeton
5536fa1e37 YARN-6953. Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() 
(Contributed by Manikandan R via Daniel Templeton)

(cherry picked from commit e094eb74b9e7d8c3c6f1990445d248b062cc230b)
 2017-11-15 10:06:11 -08:00
Andrew Wang
1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Junping Du
ef212b8550 YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 
(cherry picked from commit e14f03dfbf078de63126a1e882261081b9ec6778)
 2017-11-13 15:28:21 -08:00
Wangda Tan
7b3cd10131 YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2017-11-13 14:45:40 -08:00
Daniel Templeton
c2f479c248 YARN-7369. Improve the resource types docs 2017-11-13 11:08:58 -08:00
Daniel Templeton
5b55a74bad YARN-7442. [YARN-7069] Limit format of resource type name (Contributed by Wangda Tan via Daniel Templeton) 
(cherry picked from commit 2c6213a44280f5b3950167131293ff83f48ff56f)
 2017-11-13 11:03:56 -08:00
Subru Krishnan
37ae818660 YARN-7475. Fix Container log link in new YARN UI. (Sunil G via Subru). 
(cherry picked from commit 3c5b46c2edd69bb238d635ae61ff91656dec23df)
(cherry picked from commit 3e26077848ed1d7461576116a9ae841d38aa3ef1)
 2017-11-12 09:54:55 -08:00
bibinchundatt
bfbf94197f YARN-7406. Moving logging APIs over to slf4j in hadoop-yarn-api. Contributed by Yeliang Cang. 
(cherry picked from commit 2c2b7a3672e0744ce6a77a117cedefba04fed603)
 2017-11-11 10:38:47 +05:30
Sean Mackrory
abaf90792e YARN-7465. start-yarn.sh fails to start ResourceManagers unless running as root. 
(cherry picked from commit 1883a0024949f7946264a4d7649b03fd1881567a)
 2017-11-09 14:49:54 -07:00
Konstantinos Karanasos
e55dc1700d YARN-7437. Rename PlacementSet and SchedulingPlacementSet. (Wangda Tan via kkaranasos) 
(cherry picked from commit ac4d2b1081d8836a21bc70e77f4e6cd2071a9949)
 2017-11-09 13:07:10 -08:00
Robert Kanter
82abc7224f YARN-7386. Duplicate Strings in various places in Yarn memory (misha@cloudera.com via rkanter) 
(cherry picked from commit a2c150a7369cc629bbfaa2dfa3a8495b6f9c42e2)
 2017-11-09 12:13:09 -08:00
Daniel Templeton
933a09e886 YARN-7143. FileNotFound handling in ResourceUtils is inconsistent 
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
(cherry picked from commit db82a41d94872cea4d0c1bb1336916cebc2faeec)
 2017-11-09 11:59:16 -08:00
Wangda Tan
51d3693a9c YARN-7205. Log improvements for the ResourceUtils. (Sunil G via wangda) 
Change-Id: I0f5b7a7f68ec5d3e1d52211f83fdd089bc0bfd37
(cherry picked from commit 8bcc49e6771ca75f012211e27870a421b19233e7)
 2017-11-09 11:59:16 -08:00
Haibo Chen
91a7f8d246 YARN-7388. TestAMRestart should be scheduler agnostic. 
(cherry picked from commit a1382a18dff8a70aa25240d6fbba6e22832a7679)
 2017-11-09 10:50:49 -08:00