Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,016 Commits 326 Branches 371 Tags
Commit Graph

1362 Commits

Author SHA1 Message Date
Karthik Kambatla
a064375e68 YARN-5616. Clean up WeightAdjuster. (Yufei Gu via kasha) 
(cherry picked from commit c0e492e50fa98d423c8a61c94d2d1f9553558b6d)
 2016-09-06 22:40:49 -07:00
Karthik Kambatla
a5748eec7b YARN-5264. Store all queue-specific information in FSQueue. (Yufei Gu via kasha) 
(cherry picked from commit f6ea9be5473ab66798b0536317d2f32c5348eb57)
 2016-09-02 15:02:20 -07:00
Ray Chiang
4ee1729cb0 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 2016-09-02 14:57:05 -07:00
Varun Vasudev
1222433729 YARN-5555. Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested. Contributed by Eric Payne. 
(cherry picked from commit 05f5c0f631680cffc36a79550c351620615445db)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java
 2016-09-02 16:16:31 +05:30
Karthik Kambatla
736dd1646d YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. (Robert Kanter via kasha) 
(cherry picked from commit 74f4bae45597f4794e99e33309130ddff647b21f)
 2016-09-01 14:44:41 -07:00
Arun Suresh
b279f42d79 YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b7b6141ed88359652cf22e8973c0661)
 2016-08-30 16:33:31 -07:00
Subru Krishnan
3ad7cb3e1c YARN-5327. API changes required to support recurring reservations in the YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru). 
(cherry picked from commit b930dc3ec06afa479a249490976e3e127d201706)
 2016-08-26 17:01:37 -07:00
Junping Du
af1502fcab YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 
(cherry picked from commit 9ef632f3b0b0e0808116cd1c7482a205b7ebef7d)
 2016-08-26 09:05:33 -07:00
Naganarasimha
026027166e YARN-3940. Application moveToQueue should check NodeLabel permission. Contributed by Bibin A Chundatt 
(cherry picked from commit 46e02ab719d06f2708c0a61e8011b8f261235193)
 2016-08-26 20:23:01 +05:30
Naganarasimha
ee3358402a YARN-5564. Fix typo in RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE. Contributed by Ray Chiang 
(cherry picked from commit 27c3b86252386c9c064a6420b3c650644cbb9ef3)
 2016-08-26 08:53:42 +05:30
Rohith Sharma K S
a19859c3f5 YARN-5544. TestNodeBlacklistingOnAMFailures fails on trunk. Contributed by Sunil G. 
(cherry picked from commit 0d5997d2b98eb89e72828dfcd78f02aa4e7e1e67)
 2016-08-23 14:39:19 +05:30
Wangda Tan
c7d782d2f6 YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda) 
(cherry picked from commit 444b2ea7afebf9f6c3d356154b71abfd0ea95b23)
 2016-08-19 16:47:03 -07:00
Varun Saxena
928cccb9f9 YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena) 2016-08-19 17:32:23 +05:30
Varun Saxena
a746775e9a Revert "YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena)" 
This reverts commit 613048a8a24fe08554d2fcdd8b5bc57f7aadaf2f.
 2016-08-19 16:40:48 +05:30
Varun Saxena
613048a8a2 YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena) 2016-08-19 15:18:42 +05:30
Junping Du
d464483bf7 YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. 2016-08-18 07:23:29 -07:00
Karthik Kambatla
07d5ab16df YARN-4702. FairScheduler: Allow setting maxResources for ad hoc queues. (Daniel Templeton via kasha) 
(cherry picked from commit 20f0eb871c57cc4c5a6d19aae0e3745b6175509b)
 2016-08-17 17:54:28 -07:00
Varun Saxena
354afbb9df YARN-5521. Fix random failure of TestCapacityScheduler#testKillAllAppsInQueue (sandflee via Varun Saxena) 2016-08-16 00:04:40 +05:30
Varun Saxena
3c7cf5cd77 YARN-5491. Fix random failure of TestCapacityScheduler#testCSQueueBlocked (Bibin A Chundatt via Varun Saxena) 2016-08-15 03:56:31 +05:30
Varun Saxena
918a0f12fa YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena) 2016-08-13 00:19:00 +05:30
Naganarasimha
b63d3dcac9 YARN-4833. For Queue AccessControlException client retries multiple times on both RM. Contributed by Bibin A Chundatt 2016-08-12 20:16:15 +05:30
Rohith Sharma K S
0937715f7f YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C. 
(cherry picked from commit 5199db387d59f7233a0e52ac298df31e8ed8af20)
 2016-08-11 11:51:56 +05:30
Jason Lowe
2b31f6fb2c YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C 
(cherry picked from commit 5c95bb315ba605b3bed77966a99a63854234e2c5)
 2016-08-10 18:27:25 +00:00
Jason Lowe
3d401206cf YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee 2016-08-10 18:06:56 +00:00
Karthik Kambatla
447ba47640 YARN-5343. TestContinuousScheduling#testSortedNodes fails intermittently. (Yufei Gu via kasha) 
(cherry picked from commit 7992c0b42ceb10fd3ca6c4ced4f59b8e8998e046)
 2016-08-09 16:51:27 -07:00
Arun Suresh
5f7edb79d1 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 
(cherry picked from commit 82c9e061017c32e633e0b0cbb7978749a6df4fb2)
 2016-08-09 00:46:08 -07:00
Naganarasimha
991823482f YARN-4624. NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI. Contributed by Brahma Reddy Battula 
(cherry picked from commit d81b8163b4e5c0466a6af6e1068f512c5fd24a61)
 2016-08-06 07:04:30 +05:30
Wangda Tan
a73e118437 YARN-4888. Changes in scheduler to identify resource-requests explicitly by allocation-id. (Subru Krishnan via wangda) 
(cherry picked from commit 3f100d76ff5df020dbb8ecd4f5b4f9736a0a8270)
 2016-08-05 10:44:07 -07:00
Rohith Sharma K S
69da77c357 YARN-5333. Some recovered apps are put into default queue when RM HA. Contributed by Jun Gong. 
(cherry picked from commit d9a354c2f39274b2810144d1ae133201e44e3bfc)
 2016-08-05 21:37:05 +05:30
Jason Lowe
9a0ac56a5c YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla 
(cherry picked from commit 4d92aefd35d4517d9435d81bafdec0d77905a7a0)
 2016-08-03 19:00:06 +00:00
Arun Suresh
f197378f81 YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit e5766b1dbee02ae0ef89618e172f3fb227af19e8)
 2016-07-31 11:49:09 -07:00
Subru Krishnan
2f4df2b851 YARN-5203.Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API. Contributed by Ellen Hui. 
(cherry picked from commit 4e756d72719ec3c6d64a1e3daccbc0b8e8de998c)
 2016-07-28 16:04:30 -07:00
Wangda Tan
77e0b6d1bf YARN-5195. RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. (sandflee via wangda) 
(cherry picked from commit d62e121ffc0239e7feccc1e23ece92c5fac685f6)
 2016-07-26 21:36:37 -07:00
Wangda Tan
55fa19d576 YARN-5342. Improve non-exclusive node partition resource allocation in Capacity Scheduler. (Sunil G via wangda) 
(cherry picked from commit 49969b16cdba0f251b9f8bf3d8df9906e38b5c61)
 2016-07-26 21:16:12 -07:00
Arun Suresh
df509a8066 YARN-5392. Replace use of Priority in the Scheduling infrastructure with an opaque ShedulerRequestKey. (asuresh and subru) 
(cherry picked from commit 5aace38b748ba71aaadd2c4d64eba8dc1f816828)
 2016-07-26 14:58:54 -07:00
Rohith Sharma K S
eff65463de YARN-5092. TestRMDelegationTokens fails intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 557a245d83c358211f51b3ab1950d707a70679d9)
 2016-07-21 12:49:06 +05:30
Akira Ajisaka
2bf5c4eb1a YARN-4883. Make consistent operation name in AdminService. Contributed by Kai Sasaki. 
(cherry picked from commit c63afdbe14df7359841aa4b213252a8a275e8c4b)
 2016-07-20 16:51:44 -07:00
Varun Saxena
2153c9f04c YARN-4996. Make TestNMReconnect.testCompareRMNodeAfterReconnect() scheduler agnostic (Kai Sasaki via Varun Saxena) 2016-07-19 16:07:56 +05:30
Ray Chiang
66a501be5e YARN-5272. Handle queue names consistently in FairScheduler. (Wilfred Spiegelenburg via rchiang) 
(cherry picked from commit 2df1b1e72b18422606e38a9fbe311988637cb9a5)
 2016-07-15 14:39:28 -07:00
Arun Suresh
9d31c0b53f YARN-5181. ClusterNodeTracker: add method to get list of nodes matching a specific resourceName. (kasha via asuresh) 
(cherry picked from commit e905a42a2c6c88210b5c2120d7daf0bb3739cd1d)
 2016-07-15 14:38:26 -07:00
Wangda Tan
e34e1aa4fe YARN-4484. Available Resource calculation for a queue is not correct when used with labels. (Sunil G via wangda) 
(cherry picked from commit 24db9167f16ba643a186624b33a6b9b80020f476)
 2016-07-15 11:41:35 -07:00
Rohith Sharma K S
694b4abbc0 YARN-5362. TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail. Contributed by sandflee. 
(cherry picked from commit d6d41e820ac7b3ba73f5e4ea1ee72715dc1ffc9f)
 2016-07-13 19:15:41 +05:30
Varun Saxena
dc3e077301 YARN-5353. ResourceManager can leak delegation tokens when they are shared across apps. (Jason Lowe via Varun Saxena). 2016-07-13 07:46:00 +05:30
Jason Lowe
1f96baf4ad YARN-5317. testAMRestartNotLostContainerCompleteMsg may fail. Contributed by sandflee 
(cherry picked from commit 10b704c5946afe7bfd4a6be40192ce7ca745d817)
 2016-07-12 20:38:26 +00:00
Jian He
e94e6be772 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:37:52 -07:00
Varun Saxena
5ef27b65b9 YARN-5037. Fix random failure of TestRMRestart#testQueueMetricsOnRMRestart (sandflee via Varun Saxena). 2016-07-10 22:13:53 +05:30
Varun Saxena
f515678b67 YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena). 2016-07-09 01:43:09 +05:30
Varun Saxena
550abb53e5 YARN-5297. Avoid printing a stack trace when recovering an app after the RM restarts. (Junping Du via Varun Saxena). 2016-07-09 00:14:51 +05:30
Junping Du
5c0386d535 YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee. 
(cherry picked from commit 30ee57ceb1e80c30ea3adfe7736d4d4c7d5c8386)
 2016-07-08 04:15:47 -07:00
Wangda Tan
e246cf74c9 YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda) 2016-07-06 16:42:35 -07:00