Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,506 Commits 326 Branches 371 Tags
Commit Graph

1255 Commits

Author SHA1 Message Date
Rohith Sharma K S
71a3b01d51 YARN-5544. TestNodeBlacklistingOnAMFailures fails on trunk. Contributed by Sunil G. 
(cherry picked from commit 0d5997d2b98eb89e72828dfcd78f02aa4e7e1e67)
 2016-08-23 14:40:05 +05:30
Wangda Tan
9bf420b3bc YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-08-19 16:55:34 -07:00
Wangda Tan
3bf2e16f76 YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda) 
(cherry picked from commit 444b2ea7afebf9f6c3d356154b71abfd0ea95b23)
(cherry picked from commit c7d782d2f6e61103f7a009c48b11dcd64e07c2ce)
 2016-08-19 16:54:03 -07:00
Varun Vasudev
5b49bcaee3 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 
(cherry picked from commit 308d63f382e1992ea2b8ccf3130edaaa751c644d)
(cherry picked from commit 99966b1912320326fb63be1ce6d4222d736ca365)
 2016-08-18 12:33:52 -07:00
Karthik Kambatla
42a88f9c58 YARN-3446. FairScheduler headroom calculation should exclude nodes in the blacklist. (Zhihai Xu via kasha) 
(cherry picked from commit 9d04f26d4c42170ee3dab2f6fb09a94bbf72fc65)
(cherry picked from commit f0923819c35b8c499a9bcd0c6c78511f3a400fef)
 2016-08-18 12:30:39 -07:00
Varun Saxena
ca139a3f87 YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena) 2016-08-13 00:27:17 +05:30
Rohith Sharma K S
f7cc0af611 YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C. 
(cherry picked from commit 5199db387d59f7233a0e52ac298df31e8ed8af20)
 2016-08-11 11:52:44 +05:30
Jason Lowe
69217ba813 YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C 
(cherry picked from commit 5c95bb315ba605b3bed77966a99a63854234e2c5)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java
 2016-08-10 19:15:12 +00:00
Jason Lowe
e1845faeda YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee 
(cherry picked from commit 3d401206cf1eca5360f7f53165732a73d644a1fb)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2016-08-10 18:09:11 +00:00
Rohith Sharma K S
3b4d6ece1a YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat. Contributed by Rohith Sharma K S 
(cherry picked from commit 26b5e6116f392b6be91dca57968259f87554ce33)
 2016-08-09 22:19:17 +00:00
Naganarasimha
e6a994badf YARN-4624. NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI. Contributed by Brahma Reddy Battula 
(cherry picked from commit d81b8163b4e5c0466a6af6e1068f512c5fd24a61)
(cherry picked from commit 991823482f6687be681be69931e8a4646bc68a69)
 2016-08-06 08:19:56 +05:30
Jason Lowe
470ea3da1f YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit c01bee010832ca31d8e60e5461181cdf05140602)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-08-05 20:10:04 +00:00
Jason Lowe
aca7eea611 YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla 2016-08-03 19:01:14 +00:00
Wangda Tan
c322e749d6 YARN-5342. Improve non-exclusive node partition resource allocation in Capacity Scheduler. (Sunil G via wangda) 2016-08-03 11:29:12 -07:00
Rohith Sharma K S
7234fce25b YARN-5092. TestRMDelegationTokens fails intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 557a245d83c358211f51b3ab1950d707a70679d9)
 2016-07-21 12:49:52 +05:30
Akira Ajisaka
66f3056090 YARN-4883. Make consistent operation name in AdminService. Contributed by Kai Sasaki. 
(cherry picked from commit c63afdbe14df7359841aa4b213252a8a275e8c4b)
(cherry picked from commit 2bf5c4eb1a181ad9d4b337388d9fff19773fa2c1)
 2016-07-20 16:52:22 -07:00
Wangda Tan
343633a6e8 YARN-4484. Available Resource calculation for a queue is not correct when used with labels. (Sunil G via wangda) 
(cherry picked from commit 24db9167f16ba643a186624b33a6b9b80020f476)
(cherry picked from commit e34e1aa4fe0a0826439227175fc3321f840dddd4)
 2016-07-15 11:46:17 -07:00
Varun Saxena
f87866446d YARN-5353. ResourceManager can leak delegation tokens when they are shared across apps. (Jason Lowe via Varun Saxena). 2016-07-13 07:40:10 +05:30
Jason Lowe
0a94c6d9da YARN-5317. testAMRestartNotLostContainerCompleteMsg may fail. Contributed by sandflee 
(cherry picked from commit 10b704c5946afe7bfd4a6be40192ce7ca745d817)
 2016-07-12 20:39:09 +00:00
Jian He
23eb3c7ceb YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:38:35 -07:00
Varun Saxena
f685973e21 YARN-5037. Fix random failure of TestRMRestart#testQueueMetricsOnRMRestart (sandflee via Varun Saxena). 2016-07-11 09:39:06 +05:30
Varun Saxena
845daee463 YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena). 2016-07-09 01:54:22 +05:30
Junping Du
8645e4ac3b YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee. 
(cherry picked from commit 30ee57ceb1e80c30ea3adfe7736d4d4c7d5c8386)
(cherry picked from commit 5c0386d535c9e9b68f5b72cc8a44c634b64a5825)
 2016-07-08 06:06:56 -07:00
Varun Saxena
22ca536b0c YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena). 2016-07-06 09:31:48 +05:30
Jian He
4437e6f336 YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee 
(cherry picked from commit c35a5a7a8d85b42498e6981a6b1f09f2bdd56459)
 2016-07-01 14:29:47 -07:00
Rohith Sharma K S
33e6986ec9 YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe. 2016-06-21 10:12:01 +05:30
Wangda Tan
d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Wangda Tan
19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Arun Suresh
51a16b1ff0 YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh) 
(cherry picked from commit 3a154f75ed85d864b3ffd35818992418f2b6aa59)
(cherry picked from commit d3655f41230fd5a5f3bdb0124d5754459e593e82)
 2016-06-06 21:19:48 -07:00
Arun Suresh
934bd8989b YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh) 
(cherry picked from commit 7a9b7372a1a917c7b5e1beca7e13c0419e3dbfef)
(cherry picked from commit f0a869b52a4e4ad7e02143a7e703700a4f4b1f88)
 2016-06-06 21:08:52 -07:00
Ming Ma
ec4f9a14f9 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5)
(cherry picked from commit 74e2b5efa26f27027fed212b4b2108f0e95587fb)
 2016-06-06 14:49:43 -07:00
Arun Suresh
6963d50e2f YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh) 
(cherry picked from commit db54670e83a84c1d7deff2c225725687cf9e5f14)
(cherry picked from commit 8ebd89dcfb358cd973a1e71ea4429d0abb58b536)
 2016-06-03 14:52:45 -07:00
Vinod Kumar Vavilapalli
c87b9c1471 YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He. 
Made one minor edit for branch-2 patch.

(cherry picked from commit f10ebc67f57a4a2e3cc916c41154ab9b6a4635c9)
(cherry picked from commit 029888871fb13da75872ab345eb03d8f4e502ad4)
 2016-06-03 13:15:44 -07:00
Jian He
01a3f7899c YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan 
(cherry picked from commit 097baaaebae021c47bb7d69aa1ff1a2440df5166)
 2016-06-03 10:50:12 -07:00
Wangda Tan
2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f75c796413ce860b1859a31cc5292ca.
 2016-05-31 22:16:53 -07:00
Varun Vasudev
457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Karthik Kambatla
427e3f995e YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha) 
(cherry picked from commit 04ded558b03ee0fbf68a611cf1f25508b4447e44)
(cherry picked from commit 59335b4d7a969378cb765b000ba1e13dabc44a3a)
 2016-05-26 16:54:38 -07:00
Carlo Curino
7982933c09 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e63d7c53e601be530021d6d5a15ab7f)
(cherry picked from commit c656977961e2ba0f9dfd349ed59bf1d0d41c57f5)
 2016-05-25 17:02:22 -07:00
Naganarasimha
943cefc81e YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt 2016-05-25 06:27:44 +08:00
Naganarasimha
abab31b8f2 YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 
(cherry picked from commit b4078bd17b41fbfff5a5c0bca5bf903a327826a7)
 2016-05-24 08:19:26 +08:00
Jason Lowe
0bfad9675e YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 
(cherry picked from commit ac954486c5102b8fbbc4229a0d3a512bcc7013c0)
 2016-05-23 15:56:33 +00:00
Rohith Sharma K S
f102c4f6c8 YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-20 08:53:21 +05:30
Jian He
61fa9256c5 YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca536e7deac50976b8a8774450fe089f)
 2016-05-19 13:02:24 -07:00
Rohith Sharma K S
726c1f14b8 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2016-05-19 10:50:32 +05:30
Eric Payne
a552932b38 YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger. 
(cherry picked from commit 1217c8f6b49e3afd8ca008ffddcf6615f0accfc5)
 2016-05-16 20:44:41 +00:00
Karthik Kambatla
a9b0041abc YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 
(cherry picked from commit 4b4e4c6ba83bc5c41d7bb69bb2483bcfe894a260)
(cherry picked from commit f96b2b6fac84f565d4d0cf78c927f0b116364cfc)
 2016-05-11 17:40:57 -07:00
Junping Du
6d1300880f YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 
(cherry picked from commit 39f2bac38b111f90d3402229201cdb4315f5d4f5)
(cherry picked from commit 9dc3683d87912d9b56a69557360c19a5896d0e51)
 2016-05-11 09:40:51 -07:00
Naganarasimha
b47259caf0 YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 2750fb900fc9c468570d051cb19e3a379a898c38)
 2016-05-08 23:06:15 +05:30
Yongjun Zhang
5ae5b869aa YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang) 
(cherry picked from commit 47c41e7ac7e6b905a58550f8899f629c1cf8b138)
 2016-05-06 22:43:44 -07:00
Jason Lowe
da24fe95f8 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 
(cherry picked from commit b2ed6ae73197990a950ce71ece80c0f23221c384)
 2016-05-06 23:00:53 +00:00