Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,159 Commits 326 Branches 371 Tags
Commit Graph

1789 Commits

Author SHA1 Message Date
Yufei Gu
1d40bc225a YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu) 
(cherry picked from commit 82c5dd1d508292ed88eda0f5356776437ba67d4c)
 2017-09-12 16:14:31 -07:00
Wangda Tan
10803bf08d YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda) 
Change-Id: Ia7d61e0d9df1e703bc983a31e6856e84a5a0521c
 2017-09-11 20:56:17 -07:00
Sunil G
23dc6ef8b0 YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S. 2017-09-11 10:34:59 +05:30
Subru Krishnan
2c051dbe9e YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit fa6137501c1499ae33f6e0e2adc31671a7e782dc)
 2017-09-08 14:44:46 -07:00
Chris Douglas
2f8825588e YARN-5164. Use plan RLE to improve CapacityOverTimePolicy efficiency 
(cherry picked from commit d383bfdcd40c2315197fa5f85c3e5cb3bb83167f)
 2017-09-08 14:44:34 -07:00
Arun Suresh
2b09cec218 YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 1f53ae79728065417c6a99eb6fcc8d3a080ab4cc)
 2017-09-08 09:26:50 -07:00
Sunil G
a39829c456 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 19:20:52 +05:30
Sunil G
ef342455cb YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne. 
(cherry picked from commit c1386a90d7ff65b57be00a7245de82c31abb4665)
 2017-09-07 14:47:07 -05:00
Subru Krishnan
5ccdd83923 YARN-5328. Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 
(cherry picked from commit b6e7d1369690eaf50ce9ea7968f91a72ecb74de0)
 2017-09-06 16:46:49 -07:00
Wei-Chiu Chuang
92d9ad7761 Revert "Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem." 
This reverts commit 80516b3de79e9aac4dba1374638257b4611f199e.
 2017-09-05 14:52:43 -07:00
Subru Krishnan
80516b3de7 Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 
(cherry picked from commit 7996eca7dcfaa1bdf970e32022274f2699bef8a1)
 2017-09-01 15:17:27 -07:00
Jason Lowe
57b5b186ce YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne 
(cherry picked from commit 99a7f5d451fb91ef956c926f891283f70f0ada1c)
 2017-09-01 11:10:40 -05:00
Daniel Templeton
004231dc08 YARN-6780. ResourceWeights.toString() cleanup (Contributed by weiyuan via Daniel Templeton) 
(cherry picked from commit d4417dae4f38b58d41517a0263484da73ed41b75)
 2017-08-31 09:38:31 -07:00
Sunil G
e8b4427259 YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan. 
(cherry picked from commit f9e0cc8cdca387bf2811c07beccae4af07362952)
 2017-08-31 18:43:17 +05:30
Rohith Sharma K S
877d96b83d YARN-6982. Potential issue on setting AMContainerSpec#tokenConf to null before app is completed. Contributed by Manikandan R. 
(cherry picked from commit 4cae120c619811006b26b9a95680a98732572af6)
 2017-08-30 10:50:57 +05:30
Jason Lowe
c930aa3ff9 YARN-5816. TestDelegationTokenRenewer#testCancelWithMultipleAppSubmissions is still flakey. Contributed by Robert Kanter 2017-08-29 10:02:44 -05:00
Junping Du
f252e1fe28 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da2781da97f8c76fe1262c4d99b9c37f)
 2017-08-28 18:27:38 -07:00
Sunil G
80570737a8 YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04e0ef46f4628b006f2430ad63cac97e)
 2017-08-28 12:27:29 +05:30
Jason Lowe
ea5bb5898f YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:32:29 -05:00
Jason Lowe
82957ce0d0 YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169dc3da9df0158ba6f44a9bc8576e217)
 2017-08-25 09:20:50 -05:00
Yufei Gu
389b659c94 YARN-7049. FSAppAttempt preemption related fields have confusing names. (Contributed by Karthik Kambatla via Yufei Gu) 
(cherry picked from commit 9e2699ac2c99d8df85191dd7fbf9468b00f5b5aa)
 2017-08-24 17:54:00 -07:00
Junping Du
c98917f798 YARN-7090. testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured. Contributed by Wangda Tan. 
(cherry picked from commit 652dd434d96cd1a1fe25cd8c636b1859c29f462b)
 2017-08-23 18:07:07 -07:00
Wangda Tan
245b49332d YARN-6251. Do async container release to prevent deadlock during container updates. (Arun Suresh via wangda) 
Change-Id: I6c67d20c5dd4d22752830ebf0ed2340824976ecb
(cherry picked from commit f49843a9888ad8fe5c1bb4c16bfb5217d693009d)
 2017-08-23 10:35:37 -07:00
Subru Krishnan
a45ffdcddc YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru). 
(cherry picked from commit 4249172e1419acdb2b69ae3db43dc59da2aa2e03)
 2017-08-22 19:21:35 -07:00
Jason Lowe
a7893033fe YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 
(cherry picked from commit 657dd59cc840bce3426170baf68f9d46842f4f53)
 2017-08-22 13:17:31 -05:00
Jason Lowe
fc7df25a11 YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 
(cherry picked from commit 3efcd51c3b3eb667d83e08b500bb7a7ea559fabe)
 2017-08-22 12:59:47 -05:00
Arun Suresh
f8f193c8e8 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 8410d862d3a72740f461ef91dddb5325955e1ca5)
 2017-08-20 08:31:17 -07:00
Yufei Gu
4bd417a056 YARN-6969. Clean up unused code in class FairSchedulerQueueInfo. (Larry Lo via Yufei Gu) 
(cherry picked from commit 8991f0baec62625c45144e2544066195800ab95b)
 2017-08-18 14:45:13 -07:00
Daniel Templeton
78f9dca2cb YARN-6952. Enable scheduling monitor in FS. (Yufei Gu via Daniel Templeton) 2017-08-18 14:40:30 -07:00
bibinchundatt
ecf1a029bb YARN-7007. NPE in RM while using YarnClient.getApplications(). Contributed by Lingfeng Su. 
(cherry picked from commit e05fa3451db343c0d22496b332910874b6be5b7f)
 2017-08-18 20:31:26 +05:30
Naganarasimha
fa3094e436 YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung. 
(cherry picked from commit 0a3aa40fe7878c939dbf4e6b43466595159ff930)
 2017-08-17 10:42:00 +05:30
bibinchundatt
67f9968407 YARN-6741. Deleting all children of a Parent Queue on refresh throws exception. Contributed by Naganarasimha G R. 2017-08-16 14:52:09 +05:30
Akira Ajisaka
a28a3dc8f2
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka. 
(cherry picked from commit 588c190afd49bdbd5708f7805bf6c68f09fee142)
 2017-08-16 14:07:08 +09:00
Subru Krishnan
ad46a90872 HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru). 
(cherry picked from commit 75dd866bfb8b63cb9f13179d4365b05c48e0907d)
 2017-08-15 16:54:48 -07:00
Arun Suresh
7b22df3da6 YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 4d7be1d8575e9254c59d41460960708e3718503a)
 2017-08-14 19:57:34 -07:00
Jason Lowe
56d0e757c7 YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894169425bff054364a1ab4c48b347fb9)
 2017-08-14 15:34:50 -05:00
Jason Lowe
e5ba40cc98 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d6f364a82dd32485d9ef6d987380ce3)
 2017-08-14 14:41:38 -05:00
Subru Krishnan
a70efb6138 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-14 11:03:50 -07:00
Carlo Curino
394573780b YARN-6687. Validate that the duration of the periodic reservation is less than the periodicity. (subru via curino) 
(cherry picked from commit 28d97b79b69bb2be02d9320105e155eeed6f9e78)
 2017-08-14 11:01:56 -07:00
Jian He
630ada1b4e YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang 2017-08-14 10:56:39 -07:00
Daniel Templeton
7d3cea232d Revert "YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator" 
This reverts commit ce0cdc50d9700118139e7907dede274d2db93fda.
 2017-08-11 20:52:07 -07:00
Daniel Templeton
ce0cdc50d9 YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator 
(Contributed by Larry Lo via Daniel Templeton)

(cherry picked from commit 0996acde6c325667aa19ae0740eb6b40bf4a682a)
 2017-08-11 14:51:42 -07:00
Daniel Templeton
11e3673748 YARN-6967. Limit application attempt's diagnostic message size thoroughly 
(Contributed by Chengbing Liu via Daniel Templeton)

(cherry picked from commit 65364defb4a633ca20b39ebc38cd9c0db63a5835)
 2017-08-11 14:31:11 -07:00
Daniel Templeton
2566e5fcf7 YARN-6884. AllocationFileLoaderService.loadQueue() has an if without braces 
(Contributed by weiyuan via Daniel Templeton)

(cherry picked from commit c7680d4cc4d9302a5b5efcf2467bd32ecea99585)
 2017-08-11 14:23:17 -07:00
Subru Krishnan
e9ec1e8f87 YARN-6897. Refactoring RMWebServices by moving some util methods to RMWebAppUtil. (Giovanni Matteo Fumarola via Subru). 2017-08-08 14:01:22 -07:00
Daniel Templeton
c5b3d80a4b YARN-6879. TestLeafQueue.testDRFUserLimits() has commented out code 
(Contributed by Angela Wang via Daniel Templeton)

(cherry picked from commit e0c24145d2c2a7d2cf10864fb4800cb1dcbc2977)
 2017-08-08 13:36:49 -07:00
Wangda Tan
e18415a350 YARN-4161. Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration. (Wei Yan via wangda) 
Change-Id: Ic441ae4e0bf72e7232411eb54243ec143d5fd0d3
(cherry picked from commit adb84f34db7e1cdcd72aa8e3deb464c48da9e353)
 2017-08-07 11:34:45 -07:00
Yufei Gu
b9549e108e YARN-6802. Add max AM resource and AM resource usage to leaf queue view in FairScheduler WebUI. (YunFan Zhou via Yufei Gu) 2017-08-04 17:23:44 -07:00
Carlo Curino
7654b52167 YARN-6634. [API] Refactor ResourceManager WebServices to make API explicit (Contributed by Giovanni Matteo Fumarola via curino). 
(Edited cherry-pick of 547cb53e8ae2f19b55f9de7670b91b009d8f3bdd).
 2017-08-04 17:04:20 -07:00
Arun Suresh
4cfd248a72 YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh) 
(cherry picked from commit 077fcf6a96e420e7f36350931722b8603d010cf1)
 2017-08-04 16:50:21 -07:00