Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,333 Commits 326 Branches 371 Tags
Commit Graph

2568 Commits

Author SHA1 Message Date
Junping Du
25ea7cc373 Fix OOM issue in DistributedShell. Contributed by Jian He. 
(cherry picked from commit 9bdb5bebea1183ec2f697ee3e55392df4fe697bb)
(cherry picked from commit 7db796562c0a512bb626c207b6ed0169e3bbe92f)
 2016-07-09 05:12:21 -07:00
Varun Saxena
845daee463 YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena). 2016-07-09 01:54:22 +05:30
Junping Du
8645e4ac3b YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee. 
(cherry picked from commit 30ee57ceb1e80c30ea3adfe7736d4d4c7d5c8386)
(cherry picked from commit 5c0386d535c9e9b68f5b72cc8a44c634b64a5825)
 2016-07-08 06:06:56 -07:00
Vinod Kumar Vavilapalli
e4a62a233d YARN-5314. Fixed a ConcurrentModificationException in ATS v1.5 EntityGroupFSTimelineStore. Contributed by Li Lu. 
(cherry picked from commit 673e5e02feba9171498a518c06ae70639c5f8854)
 2016-07-07 22:42:19 -07:00
Varun Saxena
22ca536b0c YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena). 2016-07-06 09:31:48 +05:30
Vinod Kumar Vavilapalli
4ea87cb38c YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du. 
(cherry picked from commit ce9c006430d13a28bc1ca57c5c70cc1b7cba1692)
 2016-07-05 17:12:37 -07:00
Jian He
4437e6f336 YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee 
(cherry picked from commit c35a5a7a8d85b42498e6981a6b1f09f2bdd56459)
 2016-07-01 14:29:47 -07:00
Rohith Sharma K S
33e6986ec9 YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe. 2016-06-21 10:12:01 +05:30
Wangda Tan
93b406b66d YARN-4844. Addendum patch, revert few changes to public and evolving APIs to avoid breaking downstream projects as much as possible. 
(cherry picked from commit 7eb524ddee4af80c87b134070945977bf73e950f)
 2016-06-16 11:10:46 -07:00
Junping Du
810470508b YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-16 07:18:36 -07:00
Wangda Tan
d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Wangda Tan
9436548152 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:56:18 -07:00
Junping Du
2be48e7d15 YARN-5199. Close LogReader in NMWebServices#getLogs. Contributed by Xuan Gong. 2016-06-09 12:29:25 -07:00
Xuan
11b4d1e486 Revert "YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong." 
This reverts commit 22ac37615a933f9cee8cf19ad0182586a037b690.
 2016-06-08 11:23:12 -07:00
Vinod Kumar Vavilapalli
593d2c66c4 YARN-5080. Addendum fix to the original patch to fix YARN logs CLI. Contributed by Xuan Gong 
(cherry picked from commit 5a43583c0bbb9650ea6a9f48d9544ec3ec24b580)
 2016-06-08 09:52:37 -07:00
Jason Lowe
fb31aaef63 YARN-5206. RegistrySecurity includes id:pass in exception text if considered invalid. Contributed by Steve Loughran 
(cherry picked from commit 8c8a377cac10b086a7ff37ee366b79e6b04d2738)
 2016-06-08 14:13:48 +00:00
Wangda Tan
19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Arun Suresh
51a16b1ff0 YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh) 
(cherry picked from commit 3a154f75ed85d864b3ffd35818992418f2b6aa59)
(cherry picked from commit d3655f41230fd5a5f3bdb0124d5754459e593e82)
 2016-06-06 21:19:48 -07:00
Arun Suresh
934bd8989b YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh) 
(cherry picked from commit 7a9b7372a1a917c7b5e1beca7e13c0419e3dbfef)
(cherry picked from commit f0a869b52a4e4ad7e02143a7e703700a4f4b1f88)
 2016-06-06 21:08:52 -07:00
Ming Ma
ec4f9a14f9 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5)
(cherry picked from commit 74e2b5efa26f27027fed212b4b2108f0e95587fb)
 2016-06-06 14:49:43 -07:00
Arun Suresh
6963d50e2f YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh) 
(cherry picked from commit db54670e83a84c1d7deff2c225725687cf9e5f14)
(cherry picked from commit 8ebd89dcfb358cd973a1e71ea4429d0abb58b536)
 2016-06-03 14:52:45 -07:00
Vinod Kumar Vavilapalli
c87b9c1471 YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He. 
Made one minor edit for branch-2 patch.

(cherry picked from commit f10ebc67f57a4a2e3cc916c41154ab9b6a4635c9)
(cherry picked from commit 029888871fb13da75872ab345eb03d8f4e502ad4)
 2016-06-03 13:15:44 -07:00
Jian He
3c2bd19fa5 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29794f8e61bebe03b2a7ca4b6743ec92)
 2016-06-03 11:11:49 -07:00
Jian He
01a3f7899c YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan 
(cherry picked from commit 097baaaebae021c47bb7d69aa1ff1a2440df5166)
 2016-06-03 10:50:12 -07:00
Wangda Tan
2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f75c796413ce860b1859a31cc5292ca.
 2016-05-31 22:16:53 -07:00
Varun Vasudev
457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Karthik Kambatla
e5b1fb2a91 YARN-5132. Exclude generated protobuf sources from YARN Javadoc build. (Subru Krishnan via kasha) 
(cherry picked from commit f5ff05cc8a67c125717261392c996136ba66785b)
(cherry picked from commit b3f5337bb3f773777aa4cca822318b64fb2465f5)
 2016-05-28 08:12:56 -07:00
Akira Ajisaka
01e7c2bbe4 YARN-5166. javadoc:javadoc goal fails on hadoop-yarn-client. Contributed by Andras Bokor. 
(cherry picked from commit e4022debf717083ab9192164af9978500035d1be)
(cherry picked from commit ec811909e41cdea84a0d9c9254ed7c756cbae37b)
 2016-05-27 23:16:00 +09:00
Junping Du
6b41fceb69 YARN-4987. Read cache concurrency issue between read and evict in EntityGroupFS timeline store. Contributed by Li Lu. 
(cherry picked from commit 705286ccaeea36941d97ec1c1700746b74264924)
(cherry picked from commit dea79a7dd66775788e73f962a1c68fe261684799)
 2016-05-27 07:03:18 -07:00
Karthik Kambatla
427e3f995e YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha) 
(cherry picked from commit 04ded558b03ee0fbf68a611cf1f25508b4447e44)
(cherry picked from commit 59335b4d7a969378cb765b000ba1e13dabc44a3a)
 2016-05-26 16:54:38 -07:00
Carlo Curino
7982933c09 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e63d7c53e601be530021d6d5a15ab7f)
(cherry picked from commit c656977961e2ba0f9dfd349ed59bf1d0d41c57f5)
 2016-05-25 17:02:22 -07:00
Jason Lowe
fe10caee8d YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong 
(cherry picked from commit 1ba31fe9e906dbd093afd4b254216601967a4a7b)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2016-05-25 21:37:31 +00:00
Karthik Kambatla
b491d24bb6 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha) 
(cherry picked from commit 77d5ce9069b9ad64673f04edac9b1b1beddfd403)
(cherry picked from commit 0ac8c098e877e56f3da8e66e456e989060790116)
 2016-05-25 10:53:01 -07:00
Varun Vasudev
35456bb7c9 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit f440a9d8c4a177bc5062d21d4b4bc4d9b2944344)
(cherry picked from commit 36f2ae0692d73a865a5c0c520d1346b6d4498c25)
 2016-05-25 19:03:33 +05:30
Naganarasimha
943cefc81e YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt 2016-05-25 06:27:44 +08:00
Naganarasimha
abab31b8f2 YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 
(cherry picked from commit b4078bd17b41fbfff5a5c0bca5bf903a327826a7)
 2016-05-24 08:19:26 +08:00
Jason Lowe
0bfad9675e YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 
(cherry picked from commit ac954486c5102b8fbbc4229a0d3a512bcc7013c0)
 2016-05-23 15:56:33 +00:00
Jason Lowe
8e404b4321 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 
(cherry picked from commit d1df0266cf4e9ff0ec70813c156556ca4e74f791)
 2016-05-23 15:17:26 +00:00
Jian He
dbde956bf5 YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi 
(cherry picked from commit d364ceac85622e99133b3eb3becef0c8188e6f89)
(cherry picked from commit 6e0522ee1b46b262d031eb7eddd1e7e1aa3ca736)
 2016-05-20 11:36:40 -07:00
Rohith Sharma K S
f102c4f6c8 YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-20 08:53:21 +05:30
Jian He
61fa9256c5 YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca536e7deac50976b8a8774450fe089f)
 2016-05-19 13:02:24 -07:00
Junping Du
c8843cac03 YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong. 
(cherry picked from commit 141873ca7d68619e3458712b17aed57fec8fd7c8)
(cherry picked from commit 416274b53eefb76cf9d3dc2992d3e22b304c321e)
 2016-05-19 09:54:03 -07:00
Steve Loughran
7a001ae19b HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:47:46 +01:00
Rohith Sharma K S
726c1f14b8 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2016-05-19 10:50:32 +05:30
Naganarasimha
0fb6a05577 YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt 
(cherry picked from commit f04c81c9ce93512bc714531a7731debbe6b794ce)
 2016-05-19 01:40:06 +08:00
Eric Payne
a552932b38 YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger. 
(cherry picked from commit 1217c8f6b49e3afd8ca008ffddcf6615f0accfc5)
 2016-05-16 20:44:41 +00:00
Jason Lowe
70faa87ccf YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 
(cherry picked from commit 81effb7dcde2b31423438d6f1b8b8204d4ca05b3)
 2016-05-16 15:43:42 +00:00
Xuan
4eeec4e35b YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or 
RUNNING AM. Contributed by Xuan Gong

(cherry picked from commit 7be53b65f95dae89c760cde354b42f15f5ee3b5f)
(cherry picked from commit 3fe4107909393dea91adac9fda16c45814e01648)
 2016-05-14 12:22:33 -07:00
Karthik Kambatla
a9b0041abc YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 
(cherry picked from commit 4b4e4c6ba83bc5c41d7bb69bb2483bcfe894a260)
(cherry picked from commit f96b2b6fac84f565d4d0cf78c927f0b116364cfc)
 2016-05-11 17:40:57 -07:00
Junping Du
6d1300880f YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 
(cherry picked from commit 39f2bac38b111f90d3402229201cdb4315f5d4f5)
(cherry picked from commit 9dc3683d87912d9b56a69557360c19a5896d0e51)
 2016-05-11 09:40:51 -07:00