Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,387 Commits 326 Branches 371 Tags
Commit Graph

3137 Commits

Author SHA1 Message Date
Akira Ajisaka
98c3544e94
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout. 
(cherry picked from commit f8bed5e9a7d1ece127fb3da123bbfc26ada0016f)
 2017-07-31 11:10:10 +09:00
Arun Suresh
7e643130f2 YARN-6870. Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker. (Brook Zhou via asuresh) 
(cherry picked from commit 890e14c02a612c772cecd5dff2411060efd418a3)
 2017-07-28 16:33:42 -07:00
Yufei Gu
e29d1c75ea YARN-6864. FSPreemptionThread cleanup for readability. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit 9902be72cbf7a170caa5cb1f13c227d881a39064)
 2017-07-27 23:21:30 -07:00
Varun Saxena
05bbbcbdbe Addendum for YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit 11ece0bda1f6e5dd9d0f828b7c29acacf6087baa)
 2017-07-28 00:33:41 +05:30
Varun Saxena
0b342a644a YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit f66fd11e514fb326fd1f37a88b444a5276f0947b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
 2017-07-28 00:33:37 +05:30
Sunil G
08d5f3acf6 Addendum patch for YARN-6102. 2017-07-26 18:40:52 +05:30
Daniel Templeton
00f4a80e8f YARN-6307. Refactor FairShareComparator#compare (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit f81a4efb8c40f99a9a6b7b42d3b6eeedf43eb27a)
 2017-07-25 13:02:00 -07:00
Akira Ajisaka
22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 
(cherry picked from commit cca51e916b7387ea358688e8f8188ead948fbdcc)
 2017-07-25 15:25:36 +09:00
Akira Ajisaka
09241feb13
YARN-6150. TestContainerManagerSecurity tests for Yarn Server are flakey. Contributed by Daniel Sturman and Ray Chiang. 
(cherry picked from commit 218b1b33ffe83cf2e330a2aa90685d0c14547a3d)
 2017-07-25 15:12:47 +09:00
Daniel Templeton
9b6e787230 YARN-6845. Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue. 
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit 10583625c1c803fc243adf6479cb9435af7e72da)
 2017-07-24 13:45:09 -07:00
Daniel Templeton
eb1751670a YARN-6779. DominantResourceFairnessPolicy.DominantResourceFairnessComparator.calculateShares() should be @VisibleForTesting 
(Contributed by Yeliang Cang via Daniel Templeton)

(cherry picked from commit bb30bd3771442df253cbe55c448379580bd5ad07)
 2017-07-24 12:15:35 -07:00
Daniel Templeton
6b8e353c65 YARN-6844. AMRMClientImpl.checkNodeLabelExpression() has wrong error message 
(Contributed by Manikandan R via Daniel Templeton)

(cherry picked from commit 4c40cd451cbdbce5d2b94ad0e7e3cc991c3439c5)
 2017-07-24 12:06:50 -07:00
Sunil G
a657472b42 YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S. 2017-07-24 20:57:56 +05:30
Jason Lowe
db7c5636b6 YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 
(cherry picked from commit 24853bf32a045b8f029fb136edca2af03836c8d5)
 2017-07-24 10:22:31 -05:00
Jason Lowe
e76e7b9f0b YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)
 2017-07-20 11:06:39 -05:00
Wangda Tan
5070c9ba39 YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda) 
Change-Id: Iaf9bb7e5ed3aa1300abdccf87ae6fcbddbd25e3e
 2017-07-19 15:25:21 -07:00
Rohith Sharma K S
f806e8627b YARN-6819. Application report fails if app rejected due to nodesize. Contributed by Bibin A Chundatt. 
(cherry picked from commit 845c4e52bdd579a24df5dbba7477b0ebf2fa16f1)
 2017-07-19 11:16:17 +05:30
Yufei Gu
d04bb465d2 YARN-6778. In ResourceWeights, weights and setWeights() should be final. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit daaf530fce4b91cf9f568b9b0c5e8b20e6774134)
 2017-07-18 16:40:13 -07:00
Sunil G
487b0a3d7f Addendum patch for YARN-5731 
(cherry picked from commit 0b7afc060c2024a882bd1934d0f722bfca731742)
 2017-07-18 13:46:43 +05:30
Sunil G
9b1abb448d YARN-6714. IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by Tao Yang. 2017-07-17 13:22:04 +05:30
Sunil G
d462e48336 YARN-6280. Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API. Contributed by Lantao Jin. 2017-07-17 12:25:41 +05:30
Arun Suresh
0101973dbf YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 2017-07-14 13:57:22 -07:00
Jason Lowe
9eb93c6f3f YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt 
(cherry picked from commit a5ae5ac50e97cf829c41dcf01655cd9bd4d36a00)
 2017-07-14 15:03:48 -05:00
bibinchundatt
af80d10e54 YARN-6759. Fix TestRMRestart.testRMRestartWaitForPreviousAMToFinish failure. Contributed by Naganarasimha G R 
(cherry picked from commit 75c0220b4494dd4424a0c531e0bf0a763748dc62)
 2017-07-14 14:05:38 +05:30
Yufei Gu
b2a34fe8d6 YARN-6769. Make schedulables without demand less needy in FairSharePolicy#compare. (Yunfan Zhou via Yufei Gu) 
(cherry picked from commit 4a574e9a84f2e997038452b22f2ad2a2d42e8ac8)
 2017-07-13 23:11:27 -07:00
Sunil G
fd6935709f YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. Contributed by Wangda Tan. 2017-07-14 08:45:17 +05:30
Sunil G
f5cdee6bfd YARN-6792. Incorrect XML convertion in NodeIDsInfo and LabelsToNodesInfo. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 228ddaa31d812533b862576445494bc2cd8a2884)
 2017-07-14 08:08:32 +05:30
Jason Lowe
a77fb561ef YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe 
(cherry picked from commit ebc048cc055d0f7d1b85bc0b6f56cd15673e837d)
 2017-07-13 17:49:55 -05:00
Jason Lowe
6ce2b0e4d4 YARN-6654. RollingLevelDBTimelineStore backwards incompatible after fst upgrade. Contributed by Jonathan Eagles 
(cherry picked from commit 5f1ee72b0ebf0330417b7c0115083bc851923be4)
 2017-07-13 17:28:49 -05:00
Sunil G
d83e8712f6 YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne. 
(cherry picked from commit ca13b224b2feb9c44de861da9cbba8dd2a12cb35)
 2017-07-13 14:50:28 -05:00
Wangda Tan
cbb5f601f1 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda) 
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
(cherry picked from commit c583ab02c730be0a63d974039a78f2dc67dc2db6)
 2017-07-13 10:44:07 -05:00
Xuan
edc0aa95f1 YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong) 2017-07-12 15:45:51 -07:00
Jason Lowe
401b191a3c YARN-6797. TimelineWriter does not fully consume the POST response. Contributed by Jason Lowe 
(cherry picked from commit 655110393bf90a7dac5d85f831ea225071f5f3c2)
 2017-07-12 15:54:17 -05:00
Wangda Tan
f2d440b3b3 YARN-5889. Improve and refactor user-limit calculation in Capacity Scheduler. (Sunil G via wangda) 
(cherry picked from commit 5fb723bb77722d41df6959eee23e1b0cfeb5584e)
 2017-07-12 10:06:02 -05:00
Eric Payne
e6cdf770ca Revert "YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G)" 
This reverts commit eda4ac07c1835031aca7e27cc673f1c5913813bb.
Commit eda4ac07c1835031aca7e27cc673f1c5913813bb was a separate patch from trunk rather than a cherry-pick. I will cherryp-ick dependencies and then cherry-pick the trunk commit for YARN-2113.
 2017-07-12 08:56:40 -05:00
Akira Ajisaka
3419381e98
YARN-6809. Fix typo in ResourceManagerHA.md. Contributed by Yeliang Cang. 
(cherry picked from commit b628d0da5196faa5abf4c95dc7e1bb4eb67f6707)
 2017-07-12 19:39:03 +09:00
Yufei Gu
20a2770d72 YARN-6764. Simplify the logic in FairScheduler#attemptScheduling. Contributed by Yufei Gu. 
(cherry picked from commit 3de47ab5ea5cb75805a03010dc72e11b0cf6c173)
 2017-07-09 16:11:26 -07:00
Naganarasimha
5f603306d8 YARN-6770. A small mistake in the example of TimelineClient. Contributed by Jinjiang Ling. 
(cherry picked from commit ba5b056e8a151fad9d7573c6e62cffc40cde7da3)
 2017-07-09 21:31:53 +05:30
Naganarasimha
b08ef44d9e YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundatt. 
(cherry picked from commit 626d730bfc4c1f6ea54e8a246a0a676cdb74937f)
 2017-07-09 17:27:50 +05:30
Daniel Templeton
fcef28e53b YARN-6410. FSContext.scheduler should be final (Contributed by Yeliang Cang via Daniel Templeton) 
(cherry picked from commit 9d278804425f0118b590d302985408e0794b88bb)
 2017-07-09 19:38:37 +09:00
Daniel Templeton
a6d194d720 YARN-6746. SchedulerUtils.checkResourceRequestMatchingNodePartition() is dead code 
(Contributed by Deepti Sawhney via Daniel Templeton)

(cherry picked from commit c5a0c38d8e1752031d1dd8bdd4a7a3894cf775cb)
 2017-07-09 19:31:20 +09:00
Daniel Templeton
edcfae8258 YARN-6752. Display reserved resources in web UI per application 
(Contributed by Abdullah Yousufi via Daniel Templeton)

(cherry picked from commit 06159858868a9cbeefc31bc6be5ae48a59cc8a6c)
 2017-07-09 19:19:07 +09:00
Eric Payne
eda4ac07c1 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G) 2017-07-06 10:43:00 -05:00
Jason Lowe
53c2441c10 YARN-6708. Nodemanager container crash after ext3 folder limit. Contributed by Bibin A Chundatt 
(cherry picked from commit 7576a688ea84aed7206321b1f03594e43a5f216e)
 2017-07-06 09:44:02 -05:00
Naganarasimha
87c81cfad3 YARN-6681. Eliminate double-copy of child queues in canAssignToThisQueue. Contributed by Daryn Sharp. 
(cherry picked from commit fa1aaee87b0141a0255b5f8e5fd8e8f49d7efe86)
 2017-07-01 12:32:01 +05:30
Daniel Templeton
93c5421a43 YARN-6751. Display reserved resources in web UI per queue 
(Contributed by Abdullah Yousufi via Daniel Templeton)

(cherry picked from commit ec975197799417a1d5727dedc395fe6c15c30eb2)
 2017-06-29 16:57:23 -07:00
Daniel Templeton
15242786b6 YARN-6743. yarn.resourcemanager.zk-max-znode-size.bytes description needs spaces in yarn-default.xml 
(Contributed by Lori Loberg via Daniel Templeton)

(cherry picked from commit 25d891a784304fcf02f57bc7984c31af45003553)
 2017-06-28 17:17:56 -07:00
Junping Du
c5f43feb7c YARN-5311. Document graceful decommission CLI and usage. Contributed by Elek, Marton. 
(cherry picked from commit 4e3eebc943835077e3dd0df9e0b9239ae604cb89)
 2017-06-28 15:32:49 -07:00
Naganarasimha
a5ae7c0cae YARN-6467. CSQueueMetrics needs to update the current metrics for default partition only. Contributed by Manikandan R. 2017-06-29 00:09:30 +05:30
Jason Lowe
44b3c6ffaa YARN-6738. LevelDBCacheTimelineStore should reuse ObjectMapper instances. Contributed by Zoltan Haindrich 
(cherry picked from commit 63ce1593c5b78eb172773e7498d9c321debe81e8)
 2017-06-27 17:13:49 -05:00