Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,212 Commits 326 Branches 371 Tags
Commit Graph

4139 Commits

Author SHA1 Message Date
bibinchundatt
ae8d1b3c34 YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 
(cherry picked from commit 96e6a993609d170b4d2c70b2dba8b4530093606f)
 2017-11-18 19:11:40 +05:30
Varun Vasudev
6681e7238b YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
 2017-11-17 12:05:19 +05:30
Haibo Chen
9bbe7eb226 YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. (Yufei Gu via Haibo Chen) 
(cherry picked from commit 28d0fcbef40930ca5652c0e9a5d777910f3ad3c4)
 2017-11-16 10:49:40 -08:00
Sunil G
c3fb49667a YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-16 22:35:22 +05:30
Jason Lowe
9780d58a63 YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)
 2017-11-15 12:32:56 -06:00
Daniel Templeton
570bc80901 YARN-7414. FairScheduler#getAppWeight() should be moved into FSAppAttempt#getWeight() 
(Contributed by Soumabrata Chakraborty via Daniel Templeton)

(cherry picked from commit b246c547490dd94271806ca4caf1e5f199f0fb09)
 2017-11-15 10:06:17 -08:00
Daniel Templeton
5536fa1e37 YARN-6953. Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() 
(Contributed by Manikandan R via Daniel Templeton)

(cherry picked from commit e094eb74b9e7d8c3c6f1990445d248b062cc230b)
 2017-11-15 10:06:11 -08:00
Andrew Wang
1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Junping Du
ef212b8550 YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 
(cherry picked from commit e14f03dfbf078de63126a1e882261081b9ec6778)
 2017-11-13 15:28:21 -08:00
Wangda Tan
7b3cd10131 YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2017-11-13 14:45:40 -08:00
Daniel Templeton
c2f479c248 YARN-7369. Improve the resource types docs 2017-11-13 11:08:58 -08:00
Daniel Templeton
5b55a74bad YARN-7442. [YARN-7069] Limit format of resource type name (Contributed by Wangda Tan via Daniel Templeton) 
(cherry picked from commit 2c6213a44280f5b3950167131293ff83f48ff56f)
 2017-11-13 11:03:56 -08:00
Subru Krishnan
37ae818660 YARN-7475. Fix Container log link in new YARN UI. (Sunil G via Subru). 
(cherry picked from commit 3c5b46c2edd69bb238d635ae61ff91656dec23df)
(cherry picked from commit 3e26077848ed1d7461576116a9ae841d38aa3ef1)
 2017-11-12 09:54:55 -08:00
bibinchundatt
bfbf94197f YARN-7406. Moving logging APIs over to slf4j in hadoop-yarn-api. Contributed by Yeliang Cang. 
(cherry picked from commit 2c2b7a3672e0744ce6a77a117cedefba04fed603)
 2017-11-11 10:38:47 +05:30
Sean Mackrory
abaf90792e YARN-7465. start-yarn.sh fails to start ResourceManagers unless running as root. 
(cherry picked from commit 1883a0024949f7946264a4d7649b03fd1881567a)
 2017-11-09 14:49:54 -07:00
Konstantinos Karanasos
e55dc1700d YARN-7437. Rename PlacementSet and SchedulingPlacementSet. (Wangda Tan via kkaranasos) 
(cherry picked from commit ac4d2b1081d8836a21bc70e77f4e6cd2071a9949)
 2017-11-09 13:07:10 -08:00
Robert Kanter
82abc7224f YARN-7386. Duplicate Strings in various places in Yarn memory (misha@cloudera.com via rkanter) 
(cherry picked from commit a2c150a7369cc629bbfaa2dfa3a8495b6f9c42e2)
 2017-11-09 12:13:09 -08:00
Daniel Templeton
933a09e886 YARN-7143. FileNotFound handling in ResourceUtils is inconsistent 
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
(cherry picked from commit db82a41d94872cea4d0c1bb1336916cebc2faeec)
 2017-11-09 11:59:16 -08:00
Wangda Tan
51d3693a9c YARN-7205. Log improvements for the ResourceUtils. (Sunil G via wangda) 
Change-Id: I0f5b7a7f68ec5d3e1d52211f83fdd089bc0bfd37
(cherry picked from commit 8bcc49e6771ca75f012211e27870a421b19233e7)
 2017-11-09 11:59:16 -08:00
Haibo Chen
91a7f8d246 YARN-7388. TestAMRestart should be scheduler agnostic. 
(cherry picked from commit a1382a18dff8a70aa25240d6fbba6e22832a7679)
 2017-11-09 10:50:49 -08:00
bibinchundatt
4370b70e63 YARN-7454. RMAppAttemptMetrics#getAggregateResourceUsage can NPE due to double lookup. Contributed by Jason Lowe. 
(cherry picked from commit 0a72c2f56c37063609de72eef1f74632890c048a)
 2017-11-09 21:08:47 +05:30
Daniel Templeton
9bcc9e3f4d YARN-7458. TestContainerManagerSecurity is still flakey 
(Contributed by Robert Kanter via Daniel Templeton)

Change-Id: Ibb1975ad086c3a33f8af0b4f8b9a13c3cdca3f7d
(cherry picked from commit 49b4c0b334e5472dbbf71b042a6a6b1d4b2ce3b7)
 2017-11-08 17:32:34 -08:00
Daniel Templeton
d2b16e3e83 YARN-7166. Container REST endpoints should report resource types 
Change-Id: If9c2fe58d4cf758bb6b6cf363dc01f35f8720987
(cherry picked from commit 0de10680b7e5a9dfc85173bcfd338fd3656aa57f)
 2017-11-08 16:51:20 -08:00
Arun Suresh
91e726c07f YARN-7453. Fix issue where RM fails to switch to active after first successful start. (Rohith Sharma K S via asuresh) 
(cherry picked from commit a9c70b0e84dab0c41e480a0dc0cb1a22efdc64ee)
 2017-11-08 08:01:54 -08:00
Daniel Templeton
4e7b5824b4 YARN-7401. Reduce lock contention in ClusterNodeTracker#getClusterCapacity() 
(cherry picked from commit 8db9d61ac2e04888cb228b29fe54b41c730cf0e6)
 2017-11-07 14:58:56 -08:00
Haibo Chen
91e95dc484 YARN-7360. TestRM.testNMTokenSentForNormalContainer() should be scheduler agnostic. 
(cherry picked from commit 8f214dc4f8423250947a3f0027f70b9ab402ab62)
 2017-11-06 15:54:14 -08:00
Wangda Tan
df27824b80 YARN-7410. Cleanup FixedValueResource to avoid dependency to ResourceUtils. (wangda) 2017-11-06 15:26:49 -08:00
Wangda Tan
170b6a48c4 YARN-7307. Allow client/AM update supported resource types via YARN APIs. (Sunil G via wangda) 2017-11-06 10:01:10 -08:00
bibinchundatt
e49509f024 Add containerId to Localizer failed logs. Contributed by Prabhu Joseph 
(cherry picked from commit dcd99c4b9a7a15649dc9823ac060ce23c6a31056)
 2017-11-06 23:07:16 +05:30
Inigo Goiri
3e70fd9af0 YARN-7434. Router getApps REST invocation fails with multiple RMs. Contributed by Inigo Goiri. 
(cherry picked from commit 6e2259264ad9525eeec2a14055d53236711659d7)
 2017-11-02 21:30:20 -07:00
Subru Krishnan
1b6de3fac5 YARN-7432. Fix DominantResourceFairnessPolicy serializable findbugs issues. (Daniel Templeton via Subru). 
(cherry picked from commit ad0fff2b419e8fe8bb5c9a7e19b79dec16cbd96f)
 2017-11-02 17:40:03 -07:00
Eric Payne
3d923d66e8 YARN-7370: Preemption properties should be refreshable. Contrubted by Gergely Novák. 
(cherry picked from commit e6ec02001fc4eed9eb51c8653d8f931135e49eda)
 2017-11-02 12:48:07 -05:00
Wangda Tan
fb442e9a7c YARN-7364. Queue dash board in new YARN UI has incorrect values. (Sunil G via wangda) 
Change-Id: Ia5336f407147d2985a61b5f9f1e69b5b35f398a4
(cherry picked from commit 752f000e0f521ca7be173d6ca338da4585284fd0)
 2017-11-02 09:50:57 -07:00
Jason Lowe
b7dee1f060 YARN-7286. Add support for docker to have no capabilities. Contributed by Eric Badger 
(cherry picked from commit d00b6f7c1ff2d7569ae9efdc6823ebcfb86ef2d4)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestDockerContainerRuntime.java
 2017-11-02 09:47:26 -05:00
Rohith Sharma K S
9efe9bd3d8 addendum patch for YARN-7289. 
(cherry picked from commit 940ffe3f9ce74286a8863e3743faf88a33c817a3)
 2017-11-02 14:02:48 +05:30
Eric Yang
be7cfe57fc YARN-7412. Fix unit test for docker mount check on ubuntu. (Contributed by Eric Badger) 2017-11-01 18:45:24 -04:00
Junping Du
552f74e6b7 YARN-7400. Incorrect log preview displayed in jobhistory server ui. Contributed by Xuan Gong. 
(cherry picked from commit d57dba99428cbe3f3dfcec834d79f709e7529ef9)
 2017-11-01 15:02:50 -07:00
Inigo Goiri
08c3efa4a0 YARN-7276. Federation Router Web Service fixes. Contributed by Inigo Goiri. 
(cherry picked from commit 8be5707067509b78bde5fcf60072ae988d5a9f32)
 2017-11-01 13:42:12 -07:00
Inigo Goiri
8d41875219 YARN-7276 addendum to add timeline service depencies. Contributed by Inigo Goiri. 
(cherry picked from commit 70f1a9470cd7a08e7b48df99e16e57ed6f2c13dc)
 2017-11-01 13:28:50 -07:00
Daniel Templeton
c025c9ac62 YARN-7336. Unsafe cast from long to int Resource.hashCode() method 
(Contributed by Miklos Szegedi via Daniel Templeton)

(cherry picked from commit d64736d58965722b71d6eade578b6c4c266e6448)
 2017-10-31 15:06:18 -07:00
Daniel Templeton
7689d4781a YARN-6927. Add support for individual resource types requests in MapReduce 
(Contributed by Gergo Repas via Daniel Templeton)

(cherry picked from commit 9a7e81083801a57d6bb96584988415cbef67460d)
 2017-10-31 15:06:18 -07:00
Daniel Templeton
77401022a7 YARN-7374. Improve performance of DRF comparisons for resource types in fair scheduler 
(cherry picked from commit 9711b78998ca3a1f7734058a78c7baddd130ce0f)
 2017-10-31 15:06:18 -07:00
Daniel Templeton
ab93bf5b00 YARN-7397. Reduce lock contention in FairScheduler#getAppWeight() 
(cherry picked from commit e62bbbca7adafa0e050212e99c41c95a844700ff)
 2017-10-31 15:06:18 -07:00
Daniel Templeton
c818c85238 YARN-6984. DominantResourceCalculator.isAnyMajorResourceZero() should test all resources 
(Contributed by Sunil G via Daniel Templeton)

(cherry picked from commit 679f99b1455a1fcd785aef8ddc705f63cf0e6518)
 2017-10-31 15:06:18 -07:00
Daniel Templeton
68585156dc YARN-7270 addendum: Reapplied changes after YARN-3926 backports 2017-10-31 15:06:18 -07:00
Wangda Tan
6f41843906 YARN-7172. ResourceCalculator.fitsIn() should not take a cluster resource parameter. (Sen Zhao via wangda) 
Change-Id: Icc3670c9381ce7591ca69ec12da5aa52d3612d34
(cherry picked from commit e81596d06d226f1cfa44b2390ce3095ed4dee621)
 2017-10-31 15:06:18 -07:00
Yufei Gu
f30f78a736 YARN-6612. Update fair scheduler policies to be aware of resource types. (Contributed by Daniel Templeton via Yufei Gu) 
(cherry picked from commit 09b476e6dabe8039a41dde7930c8a9c0d14bb750)
 2017-10-31 15:06:18 -07:00
Sunil G
74030d808c YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan. 
(cherry picked from commit da0b6a354bf6f6bf37ca5a05a4a8eece09aa4893)
 2017-10-31 15:06:17 -07:00
Daniel Templeton
0db9ddb728 YARN-7136. Additional Performance Improvement for Resource Profile Feature 
(Contributed by Wangda Tan via Daniel Templeton)

(cherry picked from commit bf2b687412f9a830ec4834477ccf25dbe76fddcd)
 2017-10-31 15:06:17 -07:00
Wangda Tan
335e3a6334 YARN-7056. [Partial backport] Document Resource Profiles feature. (Sunil G via wangda) 
Change-Id: I5f5d52cdf7aba4ac09684aab573e6916dbad183e
(cherry picked from commit 6b88cd1901a65b977fed759e322dcd75a2cd33b6)
 2017-10-31 15:06:17 -07:00