Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,992 Commits 326 Branches 371 Tags
Commit Graph

1851 Commits

Author SHA1 Message Date
Junping Du
f252e1fe28 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da2781da97f8c76fe1262c4d99b9c37f)
 2017-08-28 18:27:38 -07:00
Sunil G
80570737a8 YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04e0ef46f4628b006f2430ad63cac97e)
 2017-08-28 12:27:29 +05:30
Jason Lowe
ea5bb5898f YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:32:29 -05:00
Jason Lowe
82957ce0d0 YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169dc3da9df0158ba6f44a9bc8576e217)
 2017-08-25 09:20:50 -05:00
Yufei Gu
389b659c94 YARN-7049. FSAppAttempt preemption related fields have confusing names. (Contributed by Karthik Kambatla via Yufei Gu) 
(cherry picked from commit 9e2699ac2c99d8df85191dd7fbf9468b00f5b5aa)
 2017-08-24 17:54:00 -07:00
Junping Du
c98917f798 YARN-7090. testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured. Contributed by Wangda Tan. 
(cherry picked from commit 652dd434d96cd1a1fe25cd8c636b1859c29f462b)
 2017-08-23 18:07:07 -07:00
Wangda Tan
245b49332d YARN-6251. Do async container release to prevent deadlock during container updates. (Arun Suresh via wangda) 
Change-Id: I6c67d20c5dd4d22752830ebf0ed2340824976ecb
(cherry picked from commit f49843a9888ad8fe5c1bb4c16bfb5217d693009d)
 2017-08-23 10:35:37 -07:00
Subru Krishnan
a45ffdcddc YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru). 
(cherry picked from commit 4249172e1419acdb2b69ae3db43dc59da2aa2e03)
 2017-08-22 19:21:35 -07:00
Jason Lowe
a7893033fe YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 
(cherry picked from commit 657dd59cc840bce3426170baf68f9d46842f4f53)
 2017-08-22 13:17:31 -05:00
Jason Lowe
fc7df25a11 YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 
(cherry picked from commit 3efcd51c3b3eb667d83e08b500bb7a7ea559fabe)
 2017-08-22 12:59:47 -05:00
Arun Suresh
f8f193c8e8 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 8410d862d3a72740f461ef91dddb5325955e1ca5)
 2017-08-20 08:31:17 -07:00
Yufei Gu
4bd417a056 YARN-6969. Clean up unused code in class FairSchedulerQueueInfo. (Larry Lo via Yufei Gu) 
(cherry picked from commit 8991f0baec62625c45144e2544066195800ab95b)
 2017-08-18 14:45:13 -07:00
Daniel Templeton
78f9dca2cb YARN-6952. Enable scheduling monitor in FS. (Yufei Gu via Daniel Templeton) 2017-08-18 14:40:30 -07:00
bibinchundatt
ecf1a029bb YARN-7007. NPE in RM while using YarnClient.getApplications(). Contributed by Lingfeng Su. 
(cherry picked from commit e05fa3451db343c0d22496b332910874b6be5b7f)
 2017-08-18 20:31:26 +05:30
Naganarasimha
fa3094e436 YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung. 
(cherry picked from commit 0a3aa40fe7878c939dbf4e6b43466595159ff930)
 2017-08-17 10:42:00 +05:30
bibinchundatt
67f9968407 YARN-6741. Deleting all children of a Parent Queue on refresh throws exception. Contributed by Naganarasimha G R. 2017-08-16 14:52:09 +05:30
Akira Ajisaka
a28a3dc8f2
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka. 
(cherry picked from commit 588c190afd49bdbd5708f7805bf6c68f09fee142)
 2017-08-16 14:07:08 +09:00
Subru Krishnan
ad46a90872 HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru). 
(cherry picked from commit 75dd866bfb8b63cb9f13179d4365b05c48e0907d)
 2017-08-15 16:54:48 -07:00
Arun Suresh
7b22df3da6 YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 4d7be1d8575e9254c59d41460960708e3718503a)
 2017-08-14 19:57:34 -07:00
Jason Lowe
56d0e757c7 YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894169425bff054364a1ab4c48b347fb9)
 2017-08-14 15:34:50 -05:00
Jason Lowe
e5ba40cc98 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d6f364a82dd32485d9ef6d987380ce3)
 2017-08-14 14:41:38 -05:00
Subru Krishnan
a70efb6138 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-14 11:03:50 -07:00
Carlo Curino
394573780b YARN-6687. Validate that the duration of the periodic reservation is less than the periodicity. (subru via curino) 
(cherry picked from commit 28d97b79b69bb2be02d9320105e155eeed6f9e78)
 2017-08-14 11:01:56 -07:00
Jian He
630ada1b4e YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang 2017-08-14 10:56:39 -07:00
Daniel Templeton
7d3cea232d Revert "YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator" 
This reverts commit ce0cdc50d9700118139e7907dede274d2db93fda.
 2017-08-11 20:52:07 -07:00
Daniel Templeton
ce0cdc50d9 YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator 
(Contributed by Larry Lo via Daniel Templeton)

(cherry picked from commit 0996acde6c325667aa19ae0740eb6b40bf4a682a)
 2017-08-11 14:51:42 -07:00
Daniel Templeton
11e3673748 YARN-6967. Limit application attempt's diagnostic message size thoroughly 
(Contributed by Chengbing Liu via Daniel Templeton)

(cherry picked from commit 65364defb4a633ca20b39ebc38cd9c0db63a5835)
 2017-08-11 14:31:11 -07:00
Daniel Templeton
2566e5fcf7 YARN-6884. AllocationFileLoaderService.loadQueue() has an if without braces 
(Contributed by weiyuan via Daniel Templeton)

(cherry picked from commit c7680d4cc4d9302a5b5efcf2467bd32ecea99585)
 2017-08-11 14:23:17 -07:00
Subru Krishnan
e9ec1e8f87 YARN-6897. Refactoring RMWebServices by moving some util methods to RMWebAppUtil. (Giovanni Matteo Fumarola via Subru). 2017-08-08 14:01:22 -07:00
Daniel Templeton
c5b3d80a4b YARN-6879. TestLeafQueue.testDRFUserLimits() has commented out code 
(Contributed by Angela Wang via Daniel Templeton)

(cherry picked from commit e0c24145d2c2a7d2cf10864fb4800cb1dcbc2977)
 2017-08-08 13:36:49 -07:00
Wangda Tan
e18415a350 YARN-4161. Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration. (Wei Yan via wangda) 
Change-Id: Ic441ae4e0bf72e7232411eb54243ec143d5fd0d3
(cherry picked from commit adb84f34db7e1cdcd72aa8e3deb464c48da9e353)
 2017-08-07 11:34:45 -07:00
Yufei Gu
b9549e108e YARN-6802. Add max AM resource and AM resource usage to leaf queue view in FairScheduler WebUI. (YunFan Zhou via Yufei Gu) 2017-08-04 17:23:44 -07:00
Carlo Curino
7654b52167 YARN-6634. [API] Refactor ResourceManager WebServices to make API explicit (Contributed by Giovanni Matteo Fumarola via curino). 
(Edited cherry-pick of 547cb53e8ae2f19b55f9de7670b91b009d8f3bdd).
 2017-08-04 17:04:20 -07:00
Arun Suresh
4cfd248a72 YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh) 
(cherry picked from commit 077fcf6a96e420e7f36350931722b8603d010cf1)
 2017-08-04 16:50:21 -07:00
Arun Suresh
043b7d133e YARN-6776. Refactor ApplicaitonMasterService to move actual processing logic to a separate class. (asuresh) 
(cherry picked from commit 5496a34c0cb2b1a83cfa6b0aba5a77b05ff2d8f0)
 2017-08-04 16:43:35 -07:00
Arun Suresh
2d62af6545 YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh) 
(cherry picked from commit aaf106fde35ec97e2e2ea4d7a67434038c4273ac)
 2017-08-04 16:19:44 -07:00
Arun Suresh
88d6adf419 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 35dc7829236f92054d5ce6ea78d3a44ca6c8f3d3)
 2017-08-03 21:18:43 -07:00
Yufei Gu
b6729a7be5 YARN-6895. [FairScheduler] Preemption reservation may cause regular reservation leaks. (Miklos Szegedi via Yufei Gu) 2017-08-03 16:56:15 -07:00
Yufei Gu
ce609d06c0 YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 13:52:58 -07:00
Sunil G
82ec700241 YARN-6678. Handle IllegalStateException in Async Scheduling mode of CapacityScheduler. Contributed by Tao Yang. 
(cherry picked from commit f64cfeaf61ec65a465decdd8215f567d4e6677a9)
 2017-08-03 19:28:16 +05:30
Eric Payne
223229b99c YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 8ce8672b6b551dacb9467924fc70f88790f5891f)
 2017-08-02 12:58:07 -05:00
Jian He
6ee0fe70c7 YARN-6872. [Addendum patch] Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 2017-08-02 00:03:59 -07:00
Jian He
4006cc266b YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 2017-08-01 10:00:04 -07:00
Yufei Gu
e29d1c75ea YARN-6864. FSPreemptionThread cleanup for readability. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit 9902be72cbf7a170caa5cb1f13c227d881a39064)
 2017-07-27 23:21:30 -07:00
Varun Saxena
05bbbcbdbe Addendum for YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit 11ece0bda1f6e5dd9d0f828b7c29acacf6087baa)
 2017-07-28 00:33:41 +05:30
Varun Saxena
0b342a644a YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit f66fd11e514fb326fd1f37a88b444a5276f0947b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
 2017-07-28 00:33:37 +05:30
Sunil G
08d5f3acf6 Addendum patch for YARN-6102. 2017-07-26 18:40:52 +05:30
Daniel Templeton
00f4a80e8f YARN-6307. Refactor FairShareComparator#compare (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit f81a4efb8c40f99a9a6b7b42d3b6eeedf43eb27a)
 2017-07-25 13:02:00 -07:00
Akira Ajisaka
22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 
(cherry picked from commit cca51e916b7387ea358688e8f8188ead948fbdcc)
 2017-07-25 15:25:36 +09:00
Daniel Templeton
9b6e787230 YARN-6845. Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue. 
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit 10583625c1c803fc243adf6479cb9435af7e72da)
 2017-07-24 13:45:09 -07:00