Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,274 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2586 Commits

Author SHA1 Message Date
Junping Du 25ea7cc373 Fix OOM issue in DistributedShell. Contributed by Jian He. 
(cherry picked from commit 9bdb5bebea)
(cherry picked from commit 7db796562c)
 2016-07-09 05:12:21 -07:00
Varun Saxena 845daee463 YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena). 2016-07-09 01:54:22 +05:30
Junping Du 8645e4ac3b YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee. 
(cherry picked from commit 30ee57ceb1)
(cherry picked from commit 5c0386d535)
 2016-07-08 06:06:56 -07:00
Vinod Kumar Vavilapalli e4a62a233d YARN-5314. Fixed a ConcurrentModificationException in ATS v1.5 EntityGroupFSTimelineStore. Contributed by Li Lu. 
(cherry picked from commit 673e5e02fe)
 2016-07-07 22:42:19 -07:00
Varun Saxena 22ca536b0c YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena). 2016-07-06 09:31:48 +05:30
Vinod Kumar Vavilapalli 4ea87cb38c YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du. 
(cherry picked from commit ce9c006430)
 2016-07-05 17:12:37 -07:00
Jian He 4437e6f336 YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee 
(cherry picked from commit c35a5a7a8d)
 2016-07-01 14:29:47 -07:00
Rohith Sharma K S 33e6986ec9 YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe. 2016-06-21 10:12:01 +05:30
Wangda Tan 93b406b66d YARN-4844. Addendum patch, revert few changes to public and evolving APIs to avoid breaking downstream projects as much as possible. 
(cherry picked from commit 7eb524ddee)
 2016-06-16 11:10:46 -07:00
Junping Du 810470508b YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-16 07:18:36 -07:00
Wangda Tan d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Wangda Tan 9436548152 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:56:18 -07:00
Junping Du 2be48e7d15 YARN-5199. Close LogReader in NMWebServices#getLogs. Contributed by Xuan Gong. 2016-06-09 12:29:25 -07:00
Xuan 11b4d1e486 Revert "YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong." 
This reverts commit 22ac37615a.
 2016-06-08 11:23:12 -07:00
Vinod Kumar Vavilapalli 593d2c66c4 YARN-5080. Addendum fix to the original patch to fix YARN logs CLI. Contributed by Xuan Gong 
(cherry picked from commit 5a43583c0b)
 2016-06-08 09:52:37 -07:00
Jason Lowe fb31aaef63 YARN-5206. RegistrySecurity includes id:pass in exception text if considered invalid. Contributed by Steve Loughran 
(cherry picked from commit 8c8a377cac)
 2016-06-08 14:13:48 +00:00
Wangda Tan 19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Arun Suresh 51a16b1ff0 YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh) 
(cherry picked from commit 3a154f75ed)
(cherry picked from commit d3655f4123)
 2016-06-06 21:19:48 -07:00
Arun Suresh 934bd8989b YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh) 
(cherry picked from commit 7a9b7372a1)
(cherry picked from commit f0a869b52a)
 2016-06-06 21:08:52 -07:00
Ming Ma ec4f9a14f9 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010)
(cherry picked from commit 74e2b5efa2)
 2016-06-06 14:49:43 -07:00
Arun Suresh 6963d50e2f YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh) 
(cherry picked from commit db54670e83)
(cherry picked from commit 8ebd89dcfb)
 2016-06-03 14:52:45 -07:00
Vinod Kumar Vavilapalli c87b9c1471 YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He. 
Made one minor edit for branch-2 patch.

(cherry picked from commit f10ebc67f5)
(cherry picked from commit 029888871f)
 2016-06-03 13:15:44 -07:00
Jian He 3c2bd19fa5 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29)
 2016-06-03 11:11:49 -07:00
Jian He 01a3f7899c YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan 
(cherry picked from commit 097baaaeba)
 2016-06-03 10:50:12 -07:00
Wangda Tan 2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f.
 2016-05-31 22:16:53 -07:00
Varun Vasudev 457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Karthik Kambatla e5b1fb2a91 YARN-5132. Exclude generated protobuf sources from YARN Javadoc build. (Subru Krishnan via kasha) 
(cherry picked from commit f5ff05cc8a)
(cherry picked from commit b3f5337bb3)
 2016-05-28 08:12:56 -07:00
Akira Ajisaka 01e7c2bbe4 YARN-5166. javadoc:javadoc goal fails on hadoop-yarn-client. Contributed by Andras Bokor. 
(cherry picked from commit e4022debf7)
(cherry picked from commit ec811909e4)
 2016-05-27 23:16:00 +09:00
Junping Du 6b41fceb69 YARN-4987. Read cache concurrency issue between read and evict in EntityGroupFS timeline store. Contributed by Li Lu. 
(cherry picked from commit 705286ccae)
(cherry picked from commit dea79a7dd6)
 2016-05-27 07:03:18 -07:00
Karthik Kambatla 427e3f995e YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha) 
(cherry picked from commit 04ded558b0)
(cherry picked from commit 59335b4d7a)
 2016-05-26 16:54:38 -07:00
Carlo Curino 7982933c09 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e)
(cherry picked from commit c656977961)
 2016-05-25 17:02:22 -07:00
Jason Lowe fe10caee8d YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong 
(cherry picked from commit 1ba31fe9e9)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2016-05-25 21:37:31 +00:00
Karthik Kambatla b491d24bb6 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha) 
(cherry picked from commit 77d5ce9069)
(cherry picked from commit 0ac8c098e8)
 2016-05-25 10:53:01 -07:00
Varun Vasudev 35456bb7c9 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit f440a9d8c4a177bc5062d21d4b4bc4d9b2944344)
(cherry picked from commit 36f2ae0692)
 2016-05-25 19:03:33 +05:30
Naganarasimha 943cefc81e YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt 2016-05-25 06:27:44 +08:00
Naganarasimha abab31b8f2 YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 
(cherry picked from commit b4078bd17b)
 2016-05-24 08:19:26 +08:00
Jason Lowe 0bfad9675e YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 
(cherry picked from commit ac954486c5)
 2016-05-23 15:56:33 +00:00
Jason Lowe 8e404b4321 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 
(cherry picked from commit d1df0266cf)
 2016-05-23 15:17:26 +00:00
Jian He dbde956bf5 YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi 
(cherry picked from commit d364ceac85)
(cherry picked from commit 6e0522ee1b)
 2016-05-20 11:36:40 -07:00
Rohith Sharma K S f102c4f6c8 YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-20 08:53:21 +05:30
Jian He 61fa9256c5 YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca)
 2016-05-19 13:02:24 -07:00
Junping Du c8843cac03 YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong. 
(cherry picked from commit 141873ca7d)
(cherry picked from commit 416274b53e)
 2016-05-19 09:54:03 -07:00
Steve Loughran 7a001ae19b HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:47:46 +01:00
Rohith Sharma K S 726c1f14b8 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2016-05-19 10:50:32 +05:30
Naganarasimha 0fb6a05577 YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt 
(cherry picked from commit f04c81c9ce)
 2016-05-19 01:40:06 +08:00
Eric Payne a552932b38 YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger. 
(cherry picked from commit 1217c8f6b4)
 2016-05-16 20:44:41 +00:00
Jason Lowe 70faa87ccf YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 
(cherry picked from commit 81effb7dcd)
 2016-05-16 15:43:42 +00:00
Xuan 4eeec4e35b YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or 
RUNNING AM. Contributed by Xuan Gong

(cherry picked from commit 7be53b65f9)
(cherry picked from commit 3fe4107909)
 2016-05-14 12:22:33 -07:00
Karthik Kambatla a9b0041abc YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 
(cherry picked from commit 4b4e4c6ba8)
(cherry picked from commit f96b2b6fac)
 2016-05-11 17:40:57 -07:00
Junping Du 6d1300880f YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 
(cherry picked from commit 39f2bac38b)
(cherry picked from commit 9dc3683d87)
 2016-05-11 09:40:51 -07:00
Naganarasimha b47259caf0 YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 2750fb900f)
 2016-05-08 23:06:15 +05:30
Yongjun Zhang 5ae5b869aa YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang) 
(cherry picked from commit 47c41e7ac7)
 2016-05-06 22:43:44 -07:00
Jason Lowe da24fe95f8 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 
(cherry picked from commit b2ed6ae731)
 2016-05-06 23:00:53 +00:00
Karthik Kambatla 367e757d8d YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt 2016-05-06 14:28:24 +05:30
Wangda Tan 3620d0e623 YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 
(cherry picked from commit 7bd418e48c)
 2016-05-05 10:08:37 -07:00
Jason Lowe b452ecf4a0 YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit ee86cef2fe)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2016-05-05 14:40:35 +00:00
Junping Du 22ac37615a YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 
(cherry picked from commit 3cf223166d452a0f58f92676837a9edb8ddc1139)
(cherry picked from commit c79dc07dc193904f2586a5d64ea2f4e56d2396b8)
 2016-05-04 09:49:08 -07:00
Jason Lowe 8013529746 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 
(cherry picked from commit c6b4839168)
 2016-05-03 22:15:31 +00:00
Junping Du 4f1c198881 YARN-4851. Metric improvements for ATS v1.5 storage components. Li Lu via junping_du. 
(cherry picked from commit 06413da72e)
(cherry picked from commit 47f67ae447)
 2016-05-03 04:59:57 -07:00
rohithsharmaks 9ea567da42 YARN-4109. Exception on RM scheduler page loading with labels. (Mohammad Shahid Khan via rohithsharmaks) 
(cherry picked from commit 8c180a13c8)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2016-05-03 10:31:32 +05:30
Robert Kanter 3c3d003402 Remove parent's env vars from child processes 
(cherry picked from commit ac8fb579c6)
 2016-04-29 15:30:06 -07:00
Jian He 5ba79d77fb YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501b)
 2016-04-28 21:54:53 -07:00
Jian He e62162db46 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 
(cherry picked from commit dd80042c42)
 2016-04-28 21:36:23 -07:00
Li Lu 76b4ec32e6 YARN-4956. findbug issue on LevelDBCacheTimelineStore. (Zhiyuan Yang via gtcarrera9) 
(cherry picked from commit f16722d2ef)
 2016-04-27 10:58:50 -07:00
Wangda Tan 358b54d063 YARN-3215. Respect labels in CapacityScheduler when computing headroom. (Naganarasimha G R via wangda) 2016-04-22 15:59:49 -07:00
Eric Payne b726ae0876 YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda 
(cherry picked from commit 3dce486d88)
 2016-04-21 22:01:07 +00:00
Vinod Kumar Vavilapalli 92e45b8321 Fixed TimelineClient to retry SocketTimeoutException too. Contributed by Xuan Gong. 
(cherry picked from commit 477003730e)
 2016-04-18 11:51:21 -07:00
Xuan 84ab382939 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012)
(cherry picked from commit 94e4d349b0)
 2016-04-16 19:40:46 -07:00
Arun Suresh aa457912ca YARN-4468. Document the general ReservationSystem functionality, and the REST API. (subru and carlo via asuresh) 
(cherry picked from commit 8460a323ce)
 2016-04-15 17:24:44 -07:00
Jason Lowe d389afb3da YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 
(cherry picked from commit 69f3d428d5)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
 2016-04-15 20:45:20 +00:00
Naganarasimha 3e223d5dc0 YARN-4909. Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter. Contributed by Bibin A Chundatt 
(cherry picked from commit fdbafbc9e5)
 2016-04-16 00:00:03 +05:30
Jason Lowe 9b5c5bd42f YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 3150ae8108)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/recovery/NMLeveldbStateStoreService.java
 2016-04-14 19:40:10 +00:00
Xuan 3127b86492 YARN-4886. Add HDFS caller context for EntityGroupFSTimelineStore. Contributed by Li Lu 
(cherry picked from commit e0cb426758)
(cherry picked from commit 5bc64dafc3)
 2016-04-13 10:40:04 -07:00
Junping Du c85a63aba1 YARN-4928. Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon. Contributed by Gergely Novák. 
(cherry picked from commit 08ddb3ac6d)
(cherry picked from commit a3f8491410)
 2016-04-13 09:05:09 -07:00
Rohith Sharma K S 9d4dedd65f YARN-4794. Deadlock in NMClientImpl. Contributed by Jian He. 
(cherry picked from commit ff722bbbdf)
 2016-04-12 10:05:35 +05:30
Vinod Kumar Vavilapalli 3589b9e10e YARN-4168. Fixed a failing test TestLogAggregationService.testLocalFileDeletionOnDiskFull. Contributed by Takashi Ohnishi. 
(cherry picked from commit 44bbc50d91)
 2016-04-11 12:13:37 -07:00
Jason Lowe 3cf7856878 Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" 
This reverts commit 9d3c51eb5a.
 2016-04-11 15:57:56 +00:00
Akira Ajisaka 13be7a849d YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit 1ff27f9d12)
(cherry picked from commit 8cf6630fc6)
 2016-04-11 14:55:54 +09:00
Akira Ajisaka 35fd4d4124 YARN-4938. MiniYarnCluster should not request transitionToActive to RM on non-HA environment. Contributed by Eric Badger. 
(cherry picked from commit 1b78b2ba17)
(cherry picked from commit 88556294e2)
 2016-04-11 01:33:02 +09:00
Wangda Tan 67ee302723 YARN-4826. Document configuration of ReservationSystem for CapacityScheduler. (Subru Krishnan via wangda) 
(cherry picked from commit fcb3fcd4c6)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md

(cherry picked from commit dac2021f87)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
 2016-04-08 16:33:55 -07:00
Jian He 4cf0b1932f YARN-4740. AM may not receive the container complete msg when it restarts. Contributed by Jun Gong 2016-04-08 11:21:34 -07:00
Karthik Kambatla ddb1407980 YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha) 
(cherry picked from commit e82f961a39)
(cherry picked from commit 2b97a50eec)
 2016-04-07 17:35:06 -07:00
Wangda Tan b7c83d4883 YARN-4699. Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node. (Sunil G via wangda) 
(cherry picked from commit 21eb428448)
 2016-04-05 16:30:56 -07:00
Masatake Iwasaki deef54b261 YARN-4917. Fix typos in documentation of Capacity Scheduler. (Takashi Ohnishi via iwasakims) 
(cherry picked from commit 500e5a5952)
 2016-04-06 04:01:48 +09:00
Masatake Iwasaki 90adda5da5 YARN-4915. Fix typo in YARN Secure Containers documentation (Takashi Ohnishi via iwasakims) 
(cherry picked from commit 30206346cf)
 2016-04-06 03:47:53 +09:00
Junping Du 3df56a97df Small addendum to YARN-4893 to fix build failure. 2016-04-05 08:41:48 -07:00
Junping Du 9c2dc1a194 YARN-4893. Fix some intermittent test failures in TestRMAdminService. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6be28bcc46)
(cherry picked from commit eeff2e35f8)
 2016-04-05 08:41:48 -07:00
Jason Lowe 9d3c51eb5a YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit 1cbcd4a491)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2016-04-05 13:47:05 +00:00
Li Lu 3574b4559c YARN-4706. UI Hosting Configuration in TimelineServer doc is broken. (Akira AJISAKA via gtcarrera9) 
(cherry picked from commit f61de41736)
 2016-04-04 14:43:37 -07:00
naganarasimha 9bd089ac64 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 5092c94195)
 2016-04-04 18:16:23 +05:30
Allen Wittenauer 608223b51b YARN-4850. test-fair-scheduler.xml isn't valid xml (Yufei Gu via aw) 
(cherry picked from commit b1394d6307)
(cherry picked from commit 92a3dbe44f)
 2016-04-01 16:58:00 -07:00
Arun Suresh 82d1f858ee Missing file for YARN-4895. 
(cherry picked from commit 5686caa9fc)
 2016-04-01 15:58:54 -07:00
Arun Suresh 9def4d4d39 YARN-4895. Add subtractFrom method to ResourceUtilization class. Contributed by Konstantinos Karanasos. 
(cherry picked from commit 82621e38a0)
 2016-04-01 15:00:30 -07:00
Wangda Tan 6d454a5d6e YARN-4634. Scheduler UI/Metrics need to consider cases like non-queue label mappings. (Sunil G via wangda) 
(cherry picked from commit 12b11e2e68)
 2016-03-31 14:36:34 -07:00
Sangjin Lee 5e94e539e0 YARN-4183. Clarify the behavior of timeline service config properties (Naganarasimha G R via sjlee) 
(cherry picked from commit 6d67420dbc)
 2016-03-31 10:50:18 -07:00
Xuan d5ea2462c2 YARN-4863. AHS Security login should be in serviceInit() instead of serviceStart(). Contributed by Junping Du 
(cherry picked from commit 80182809ae)
(cherry picked from commit ffe01e05cd)
 2016-03-28 22:21:31 -07:00
Jason Lowe 35f9cfda61 YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 
(cherry picked from commit 948b758070)
 2016-03-28 23:03:16 +00:00
Jian He 7c81e374da YARN-4117. End to end unit test with mini YARN cluster for AMRMProxy Service. Contributed by Giovanni Matteo Fumarola 2016-03-28 13:23:53 -07:00
Jian He 0c84f9aee2 YARN-998. Keep NM resource updated through dynamic resource config for RM/NM restart. Contributed by Junping Du 
(cherry picked from commit c7d843af3b)
 2016-03-28 11:28:59 -07:00
Arun Suresh 1a194f8751 YARN-4823. Refactor the nested reservation id field in listReservation to simple string field. (subru via asuresh) 
(cherry picked from commit 00bebb7e58)
 2016-03-25 15:57:11 -07:00
Jason Lowe e06446be74 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong 
(cherry picked from commit e8fc81f9c8)
 2016-03-25 20:20:01 +00:00
Arun Suresh 945e3cd55c YARN-4687. Document Reservation ACLs (Contributed by Sean Po) 
(cherry picked from commit 2e1d0ff4e9)
 2016-03-24 10:42:07 -07:00
Arun Suresh 2399e1e46a YARN-4683. Document the List Reservations REST API. (Contributed by Sean Po) 
(cherry picked from commit d820975c1a)
 2016-03-24 10:32:13 -07:00
Arun Suresh 3c2b610b68 YARN-4825. Remove redundant code in ClientRMService::listReservations. (subru via asuresh) 
(cherry picked from commit d82e797b65)
 2016-03-24 10:02:17 -07:00
Junping Du 782db43d18 YARN-4820. ResourceManager web redirects in HA mode drops query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit 19b645c938)
(cherry picked from commit c722262c75)
 2016-03-23 19:39:09 -07:00
Junping Du 762c7d4361 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong. 
(cherry picked from commit af1d125f9c)
(cherry picked from commit 76602161c0)
 2016-03-23 09:02:56 -07:00
Eric Payne 878e1cfc77 YARN-4686. MiniYARNCluster.start() returns before cluster is completely started. Contributed by Eric Badger. 
(cherry picked from commit 92b7e0d413)
 2016-03-18 17:19:06 +00:00
Junping Du 52ce763a54 YARN-4785. inconsistent value type of the type field for LeafQueueInfo in response of RM REST API. 
(cherry picked from commit ca8106d2dd)
(cherry picked from commit 66257613b4)
 2016-03-17 09:30:43 -07:00
Xuan d21bee170b Revert "YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9)" 
This reverts commit 3a0b695441.

(cherry picked from commit a888b5bc53)
(cherry picked from commit 398dbde538)
 2016-03-15 22:12:05 -07:00
Li Lu 8640a9c45f YARN-4817. Change Log Level to DEBUG for putDomain call in ATS 1.5. (Xuan Gong via gtcarrera9) 
(cherry-picked from commit 5176a69868)
 2016-03-15 11:49:09 -07:00
Li Lu 0ebc947140 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9) 
(cherry picked from commit 3a0b695441)
 2016-03-15 11:43:39 -07:00
Junping Du 4f5b496b8a YARN-4545. Allow YARN distributed shell to use ATS v1.5 APIs. Li Lu via junping_du 
(cherry picked from commit f291d82cd4)
(cherry picked from commit 3d5ac829da)
 2016-03-14 08:34:08 -07:00
Li Lu 92e86ff0cb YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. 
This commit amends commit d49cfb3504 with a missed test file.

(cherry picked from commit 017d2c127b)
 2016-03-10 13:06:02 -08:00
Li Lu 8e89bb9d38 YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. (Steve Loughran via gtcarrera9) 
(cherry-picked from commit d49cfb3504)
 2016-03-10 10:58:51 -08:00
Allen Wittenauer 8b4a8ff1b2 Revert "HADOOP-12366. expose calculated paths (aw)" 
This reverts commit 81a7ad45c9.
 2016-03-09 09:59:33 -08:00
Wangda Tan 06088c4c67 YARN-4465. SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled. (Bibin A Chundatt via wangda) 
(cherry picked from commit 0233d4e0ee)
 2016-03-08 14:31:40 -08:00
Akira Ajisaka 2acd4c3bd2 HADOOP-12860. Expand section "Data Encryption on HTTP" in SecureMode documentation. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit f86850b544)
(cherry picked from commit 0586386472)
 2016-03-08 14:30:01 +09:00
Jason Lowe 0d734cea44 YARN-4760. proxy redirect to history server uses wrong URL. Contributed by Eric Badger 
(cherry picked from commit 4163e36c2b)
 2016-03-07 15:58:01 +00:00
Jason Lowe 53ec7c9243 YARN-4744. Too many signal to container failure in case of LCE. Contributed by Sidharta Seethana 
(cherry picked from commit 059caf9989)
 2016-03-07 15:48:06 +00:00
Zhihai Xu 1a99d5ab97 YARN-4761. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations on fair scheduler. Contributed by Sangjin Lee 
(cherry picked from commit e1ccc9622b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
 2016-03-06 20:03:55 -08:00
Masatake Iwasaki 9d551a3adf HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 
(cherry picked from commit cbd31328a6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
	hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md

(cherry picked from commit 722182fdfb)
 2016-03-04 14:13:30 +09:00
Jian He c50f79d838 YARN-4671. There is no need to acquire CS lock when completing a container. Contributed by Meng Ding 2016-03-01 13:15:37 -08:00
Jian He 475a277e60 YARN-4748. ApplicationHistoryManagerOnTimelineStore should not swallow exceptions on generateApplicationReport. Contributed by Li Lu 
(cherry picked from commit d93c22ec27)
 2016-02-29 18:19:49 -08:00
Haohui Mai 69b195d619 HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-02-29 14:15:25 -08:00
Jason Lowe da9f00ac8b YARN-4723. NodesListManager$UnknownNodeId ClassCastException. Contributed by Kuhu Shukla 
(cherry picked from commit 6b0f813e89)
 2016-02-26 20:26:31 +00:00
Ming Ma 6f5ca1b293 YARN-4720. Skip unnecessary NN operations in log aggregation. (Jun Gong via mingma) 
(cherry picked from commit 7f3139e54d)
 2016-02-26 08:49:51 -08:00
Li Lu b22f4db3a0 YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via 
gtcarrera9)

(cherry picked from commit 9e0f7b8b69)
 2016-02-24 14:08:28 -08:00
Sangjin Lee 6b59e59865 YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee) 
(cherry picked from commit 553b591ba0)
 2016-02-24 09:32:54 -08:00
Jason Lowe 6f3f5a8c38 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 
(cherry picked from commit d284e187b8)
 2016-02-23 20:51:41 +00:00
Junping Du fe43f57c2a YARN-3223. Resource update during NM graceful decommission. Contributed by Brook Zhou. 
(cherry picked from commit 9ed17f181d)
(cherry picked from commit e3ce0ffdc3)
 2016-02-23 03:41:27 -08:00
Varun Vasudev 5f68f640a5 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 
(cherry picked from commit 140cb5d745)
(cherry picked from commit 2c218ca8a8)
 2016-02-23 12:33:11 +05:30
Zhe Zhang 6d4a4e785b HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh. 
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
 2016-02-22 14:15:11 -08:00
Junping Du 606672a5b1 YARN-4386. refreshNodesGracefully() should send recommission event to active RMNodes only. Contributed by Kuhu Shukla. 
(cherry picked from commit 3fab88540f)
(cherry picked from commit 7402448387)
 2016-02-22 07:39:52 -08:00
Akira Ajisaka 8ddf5822f3 YARN-4707. Remove the extra char (>) from SecureContainer.md. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5e7d4d51f8)
(cherry picked from commit 34635f9e37)
 2016-02-22 17:18:17 +09:00
Rohith Sharma K S f18d092bda YARN-4654. Yarn node label CLI should parse "=" correctly when trying to remove all labels on a node. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit c1afac3a98)
 2016-02-18 14:21:55 +05:30
Sangjin Lee 3a8c7ffeb4 YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when possible (Ming Ma via sjlee) 
(cherry picked from commit 7de70680fe)
 2016-02-17 21:03:07 -08:00
Jian He e25d82bbc2 YARN-4653. Document YARN security model from the perspective of Application Developers. Contributed by Steve Loughran 2016-02-14 17:15:18 +08:00
Steve Loughran 706351afb8 YARN-4682. AMRM client to log when AMRM token updated. (Prabhu Joseph via stevel) 2016-02-12 10:58:09 +00:00
Arun Suresh 379c2f4c05 YARN-2575. Create separate ACLs for Reservation create/update/delete/list ops (Sean Po via asuresh) 2016-02-11 15:54:45 -08:00
Jian He fc06ae38ca YARN-4138. Roll back container resource allocation after resource increase token expires. Contributed by Meng Ding 2016-02-11 10:42:45 +08:00
Jian He 145add1aec YARN-4519. Potential deadlock of CapacityScheduler between decrease container and assign containers. Contributed Meng Ding 2016-02-11 10:30:02 +08:00
= 9220bb649b YARN-4420. Add REST API for List Reservations. (Sean Po via curino) 
(cherry picked from commit fc9eb4577c)
 2016-02-10 14:41:38 -08:00
Arun Suresh 685aeeb1a7 YARN-4360. Improve GreedyReservationAgent to support "early" allocations, and performance improvements (curino via asuresh) 
(cherry picked from commit 5cf5c41a89)
 2016-02-10 09:13:31 -08:00
Devaraj K 35651bab99 YARN-4667. RM Admin CLI for refreshNodesResources throws NPE when nothing 
is configured. Contributed by Naganarasimha G R.

(cherry picked from commit 565af873d5)
 2016-02-08 15:04:15 +05:30
Devaraj K aeea77ce14 YARN-4100. Add Documentation for Distributed and Delegated-Centralized 
Node Labels feature. Contributed by Naganarasimha G R.

(cherry picked from commit db144eb1c5)
 2016-02-02 12:08:56 +05:30
Wangda Tan c487453b91 YARN-4340. Add list API to reservation system. (Sean Po via wangda) 
(cherry picked from commit 9875325d5c)
 2016-02-02 10:23:25 +08:00
Jason Lowe 9862879e27 YARN-3102. Decommisioned Nodes not listed in Web UI. Contributed by Kuhu Shukla 
(cherry picked from commit ed55950164)
 2016-02-01 23:17:02 +00:00
Jason Lowe a9916057e3 YARN-4428. Redirect RM page to AHS page when AHS turned on and RM page is not available. Contributed by Chang Li 
(cherry picked from commit 772ea7b41b)
 2016-01-29 21:55:38 +00:00
Jian He e45799a61b YARN-4617. LeafQueue#pendingOrderingPolicy should always use fixed ordering policy instead of using same as active applications ordering policy. Contributed by Rohith Sharma K S 
(cherry picked from commit f4a57d4a53)
 2016-01-29 12:23:02 -08:00
Devaraj K eb2366eda1 YARN-4411. RMAppAttemptImpl#createApplicationAttemptReport throws 
IllegalArgumentException. Contributed by Bibin A Chundatt and yarntime.

(cherry picked from commit a277bdc9ed)
 2016-01-29 13:53:29 +05:30
Rohith Sharma K S d28b5b0c5c YARN-4219. addendum patch to fix javadoc errors 
(cherry picked from commit 09d831c95b)
 2016-01-29 11:53:51 +05:30
Xuan 353518f54c YARN-4219. New levelDB cache storage for timeline v1.5. Contributed by 
Li Lu

(cherry picked from commit 9fab22b366)
(cherry picked from commit 2f00d8d3a7)
 2016-01-28 14:37:27 -08:00
Jason Lowe 290b7db656 YARN-4643. Container recovery is broken with delegating container runtime. Contributed by Sidharta Seethana 
(cherry picked from commit 61382ff8fa)
 2016-01-28 19:01:28 +00:00
Jian He 389a7fb33f YARN-4520. Finished app info is unnecessarily persisted in NM state-store if container is acquired but not lunched on this node. Contributed by sandflee 2016-01-25 15:36:59 -08:00
Jason Lowe 435b6ae82b YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 
(cherry picked from commit 50bd067e1d)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-25 16:11:59 +00:00
Jason Lowe a025b7b7ea Revert "YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R." 
This reverts commit 92de57bdca.

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-25 16:03:27 +00:00
Akira Ajisaka a7e84dad41 YARN-4592. Remove unused GetContainerStatus proto. Contributed by Chang Li. 
(cherry picked from commit 56a0c17508)
(cherry picked from commit 02d4a1fce3)
 2016-01-25 15:20:57 +09:00
rohithsharmaks 80928af293 YARN-4614. Fix random failure in TestApplicationPriority#testApplicationPriorityAllocationWithChangeInPriority. (Sunil G via rohithsharmaks) 
(cherry picked from commit 99829eb221)
 2016-01-23 07:59:08 +05:30
Jason Lowe c1953e5e91 YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 
(cherry picked from commit 46e5ea81e0)
 2016-01-22 21:56:37 +00:00
Jason Lowe e2762f035f YARN-4371. "yarn application -kill" should take multiple application ids. Contributed by Sunil G 
(cherry picked from commit e91e8b711c)
 2016-01-22 21:37:41 +00:00
Akira Ajisaka ecba5f2ed3 YARN-4605. Spelling mistake in the help message of "yarn applicationattempt" command. Contributed by Weiwei Yang. 
(cherry picked from commit 8f58f742ae)
(cherry picked from commit ddc22bbcb1)
 2016-01-22 19:47:57 +09:00
Jason Lowe d856f82b73 YARN-4610. Reservations continue looking for one app causes other apps to starve. Contributed by Jason Lowe 
(cherry picked from commit 468a53b22f)
 2016-01-21 18:34:03 +00:00
Akira Ajisaka a2d73b185f YARN-4608. Redundant code statement in WritingYarnApplications. Contributed by Kai Sasaki. 
(cherry picked from commit d3224e2d6274daf73a103c0dbe21ba69add7ca26)
(cherry picked from commit 01cd6dce63f2ac21a52f6f2539cb222e895044eb)
 2016-01-22 02:15:16 +09:00
Jason Lowe 15e84fd626 YARN-4492. Add documentation for preemption supported in Capacity scheduler. Contributed by Naganarasimha G R 
(cherry picked from commit f858ac2426)
 2016-01-21 15:40:30 +00:00
Colin Patrick Mccabe de5175d216 HADOOP-12712. Fix some cmake plugin and native build warnings (cmccabe) 
(cherry picked from commit b2c155f810)

 Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/CompileMojo.java
	hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/TestMojo.java
	hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/util/Exec.java
 2016-01-21 16:04:05 +05:30
Wangda Tan ada560bc4e YARN-4557. Fix improper Queues sorting in PartitionedQueueComparator when accessible-node-labels=*. (Naganarasimha G R via wangda) 
(cherry picked from commit c9fe0371a8ab40c0d3437fadd32788b320788a7a)
 2016-01-21 11:19:09 +08:00
Wangda Tan 5dc2e78c97 HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 
(cherry picked from commit 89d1fd5dac)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-01-19 21:31:35 +08:00
Jian He 914a8fff56 YARN-4565. Fix a bug that leads to AM resource limit not hornored when sizeBasedWeight enabled for FairOrderingPolicy. Contributed by Wangda Tan 2016-01-18 21:05:21 -08:00
Wangda Tan 2a2ea38a9b YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit a44ce3f14f)
 2016-01-19 09:33:52 +08:00
Wangda Tan bacb38aaf3 Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" 
This reverts commit 3fe5728563.

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 150f5ae034)
 2016-01-19 09:33:47 +08:00
Jian He cdef283cd0 YARN-4596. SystemMetricPublisher should not swallow error messages from TimelineClient#putEntities. Contributed by Li Lu 2016-01-18 16:59:24 -08:00
rohithsharmaks c70322d1cf YARN-4265 Addendum patch to fix compilation issue caused by project version number 2016-01-18 16:38:52 +05:30
Wangda Tan 1cb0ad00bb YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit 805a9ed85e)
(cherry picked from commit 6dab78c71a)
 2016-01-18 17:08:42 +08:00
Wangda Tan 2ab34fa89f YARN-4524. Cleanup AppSchedulingInfo. (Karthik Kambatla via wangda) 2016-01-18 17:00:15 +08:00
Wangda Tan ca7ac65453 YARN-4304. AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics. (Sunil G via wangda) 
(cherry picked from commit b08ecf5c75)
 2016-01-18 11:34:26 +08:00
Wangda Tan 7bb65e3308 YARN-4538. QueueMetrics pending cores and memory metrics wrong. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9523648d57)
 2016-01-18 11:18:07 +08:00
Junping Du a04f29ca90 YARN-4265. Provide new timeline plugin storage to support fine-grained entity caching. Contributed by Li Lu and Jason Lowe 
(cherry picked from commit 02f597c5db)
(cherry picked from commit 4a30a44b11)
 2016-01-17 17:44:17 -08:00
rohithsharmaks 9baeae9368 YARN-4389. Allow application to enable or disable am blacklisting. (Sunil G via rohithsharmaks) 
(cherry picked from commit d3c08cde68)
 2016-01-15 22:48:43 +05:30
Junping Du a03532e04e YARN-4581. AHS writer thread leak makes RM crash while RM is recovering. Contributed by sandflee. 
(cherry picked from commit fc6d3a3b23)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit e09222306f)
 2016-01-15 08:53:48 -08:00
Akira Ajisaka 4417c3b6d2 YARN-4534. Remove the redundant symbol in yarn rmadmin help msg. Contributed by Lin Yiqun. 
(cherry picked from commit 62b0d61513)
(cherry picked from commit ffca0ebfff)
 2016-01-14 14:00:18 +09:00
Jason Lowe a8292161fc YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1)
 2016-01-12 15:57:44 +00:00
Wangda Tan 00fca06b43 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9e792da014)
 2016-01-12 12:56:12 +08:00
Akira Ajisaka 3fced13eae HADOOP-12584. Disable browsing the static directory in HttpServer2. Contributed by Robert Kanter. 
(cherry picked from commit 9c89bcd04212543ae279d34938ec2ad319e5ba6d)
(cherry picked from commit 6f80be98758ff5d53a192f86ada6cc4062fcdef8)
 2016-01-12 13:35:03 +09:00
Jian He 8172921f6d Missing file for YARN-4580 2016-01-11 17:00:17 -08:00
Jian He e73bf71e10 YARN-4537. Pull out priority comparison from fifocomparator and use compound comparator for FifoOrdering policy. Contributed by Rohith Sharma K S 
(cherry picked from commit b8942be888)
 2016-01-11 16:45:48 -08:00
Jian He 17d29b515f YARN-4479. Change CS LeafQueue pendingOrderingPolicy to hornor recovered apps. Contributed by Rohith Sharma K S 
(cherry picked from commit 109e528ef5)
 2016-01-08 15:52:03 -08:00
Akira Ajisaka 852033ca62 YARN-4544. All the log messages about rolling monitoring interval are shown with WARN level. Contributed by Takashi Ohnishi. 
(cherry picked from commit 52b77577c4)
(cherry picked from commit 3905435d3d)
 2016-01-08 01:13:44 +09:00
Junping Du 49ba43e05b YARN-4546. ResourceManager crash due to scheduling opportunity overflow. Contributed by Jason Lowe. 
(cherry picked from commit c1462a67ff)
(cherry picked from commit 1cc001db4c)
 2016-01-06 05:52:21 -08:00
Junping Du a5eb8bb8f5 YARN-1382. Remove unusableRMNodesConcurrentSet (never used) in NodeListManager to get rid of memory leak. Contributed by Rohith Sharma K S. 
(cherry picked from commit 223ce323bb)
(cherry picked from commit 401d1e6e8a)
 2015-12-30 07:53:55 -08:00
Jian He 9b11e7bd6d YARN-4098. Document ApplicationPriority feature. Contributed by Rohith Sharma K S 2015-12-29 16:52:30 -08:00
Wangda Tan c7fe0bac77 YARN-4315. NaN in Queue percentage for cluster apps page. (Bibin A Chundatt via wangda) 
(cherry picked from commit 561abb9fee)
(cherry picked from commit 3efbde5378)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-29 13:31:43 -08:00
Junping Du ce74b17028 YARN-4400. AsyncDispatcher.waitForDrained should be final. Contributed by Daniel Templeton. 
(cherry picked from commit bb5df272b9)
(cherry picked from commit dbc4c79b44)
 2015-12-23 08:56:37 -08:00
Junping Du 630b637ff3 YARN-4234. New put APIs in TimelineClient for ats v1.5. Contributed by Xuan Gong. 
(cherry picked from commit 882f2f0464)
(cherry picked from commit 6e97a3c968)
 2015-12-23 05:31:16 -08:00
Arun Suresh 546178a5de YARN-4477. FairScheduler: Handle condition which can result in an infinite loop in attemptScheduling. (Tao Jie via asuresh) 
(cherry picked from commit e88422df45)
 2015-12-21 22:42:42 -08:00
cnauroth eb5f46e34f YARN-3458. CPU resource monitoring in Windows. Contributed by Inigo Goiri. 
(cherry picked from commit 114b590955)
(cherry picked from commit fdfcffb3d1)
 2015-12-21 16:59:24 -08:00
Wangda Tan e507ed1f90 YARN-4290. Add -showDetails option to YARN Nodes CLI to print all nodes reports information. (Sunil G via wangda) 
(cherry picked from commit a0de702851)
 2015-12-21 11:38:15 -08:00
Wangda Tan 01c813d210 YARN-4454. NM to nodelabel mapping going wrong after RM restart. (Bibin A Chundatt via wangda) 
(cherry picked from commit bc038b382c)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-21 11:31:54 -08:00
Uma Mahesh 6f30919336 YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh) 
(cherry picked from commit 0f82b5d878)
 2015-12-19 23:28:51 -08:00
Wangda Tan b74391ce14 YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue. (Naganarasimha G R via wangda) 
(cherry picked from commit 9b856d9787)
(cherry picked from commit 5060fd1840)
 2015-12-18 16:34:45 -08:00
Wangda Tan c4b1386b4e YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda) 
(cherry picked from commit 7faa406f27)
(cherry picked from commit 2f5435acbd)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:45 -08:00
Wangda Tan 9ed9eb265f YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83)
(cherry picked from commit 9f4a6ec663)
 2015-12-18 16:34:45 -08:00
Zhihai Xu e4f638d815 YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time. Contributed by Lin Yiqun 
(cherry picked from commit 2aaed10327)
(cherry picked from commit be50df687b)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:45 -08:00
Wangda Tan 1d1c6c1f8d YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda) 
(cherry picked from commit 07b0fb996a)
(cherry picked from commit c915a658c8)
 2015-12-18 16:34:44 -08:00
Wangda Tan c7e10d70de YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 
(cherry picked from commit dfcbbddb09)
(cherry picked from commit 9f979107c2)
 2015-12-18 16:34:44 -08:00
Wangda Tan 8ee83765f4 YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda) 
(cherry picked from commit 6cb0af3c39)
(cherry picked from commit a1a723fdff)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:44 -08:00
Xuan 41986d5373 YARN-3623-Addendum: Improve the description for Timeline Service Version configuration in yarn-default.xml 
(cherry picked from commit 21daa6c68a)
(cherry picked from commit 455c6f39c5)
 2015-12-18 16:34:43 -08:00
Wangda Tan a4ba6c8525 YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 
(cherry picked from commit 7e4715186d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 7013f9d6cd)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:43 -08:00
Jonathan Eagles c4ae007fd4 YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles) 
(cherry picked from commit 4ff973f96a)
(cherry picked from commit 7c604c3d06)
 2015-12-18 16:34:43 -08:00
Xuan 46cb47c03e YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)
(cherry picked from commit 7decba83ef)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:43 -08:00
Jian He 12772f5a45 Add missing file for YARN-4419 
(cherry picked from commit 8a97ca4b00)
 2015-12-18 16:34:42 -08:00
Jian He f355645ca4 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 
(cherry picked from commit 755dda8dd8)

(cherry picked from commit be404e1680)
 2015-12-18 16:34:42 -08:00
Wangda Tan 15b3ff1971 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda) 
(cherry picked from commit a2c3bfc8c1)
(cherry picked from commit 51e448d295)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:33:53 -08:00
Jian He 9c22910570 YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 
(cherry picked from commit 48cbde9083)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:33:53 -08:00
Junping Du 8812be3dc1 YARN-3226. UI changes for decommissioning node. Contributed by Sunil G. 
(cherry picked from commit 1de56b0448)
(cherry picked from commit 6d2914a697)
 2015-12-17 15:22:37 -08:00
Junping Du 92de57bdca YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 
(cherry picked from commit 50bd067e1d)
(cherry picked from commit 1a2ef845b5)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java
(cherry picked from commit 0f9dd48842)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-12-17 06:16:08 -08:00
Jason Lowe d939c23cc1 YARN-4461. Redundant nodeLocalityDelay log in LeafQueue. Contributed by Eric Payne 
(cherry picked from commit 91828fef6b)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-12-16 23:24:51 +00:00
Siddharth Seth 7415870d13 YARN-4207. Add a non-judgemental YARN app completion status. Contributed by Rich Haase. 
(cherry picked from commit 0f708d465f)
(cherry picked from commit 433212ef87)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-16 11:39:37 -08:00
Xuan f22d0046de YARN-4439. Clarify NMContainerStatus#toString method. Contributed by 
Jian He

(cherry picked from commit d8a45425eb)
(cherry picked from commit 992f65dd8b)
 2015-12-14 17:16:07 -08:00
Jian He bc7603e393 YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception. Contributed by Brahma Reddy Battula 2015-12-14 15:06:13 -08:00
Jian He 16f7ac48d4 YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du 2015-12-14 13:52:33 -08:00
Arun Suresh c2e788a7b5 YARN-4358 addendum patch to fix javadoc error 
(cherry picked from commit 7fb212e5e6)
 2015-12-12 22:26:35 -08:00
Junping Du 5adf7d851c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
(cherry picked from commit 0b5e76e9ff)
 2015-12-10 02:58:28 -08:00
Akira Ajisaka 59c892683f YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang. 
(cherry picked from commit 16dc65811d4e0e7ab61c61295149848fe53ddd89)
(cherry picked from commit 925bb94d87db7e916ed380d4512225e796f9e03d)
 2015-12-09 16:33:12 +09:00
rohithsharmaks 5bc33c5ae4 YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition. (Daniel Templeton via rohithsharmaks) 
(cherry picked from commit a5e2e1ecb0)
 2015-12-09 11:35:02 +05:30
rohithsharmaks 6a6bbc51c4 YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM. (Junpin Du via rohithsharmaks) 2015-12-09 11:06:31 +05:30
Chris Douglas 3d100163de YARN-4248. Followup patch adding asf-licence exclusions for json test files 
(cherry picked from commit 9f50e13d5d)
(cherry picked from commit bfe4796802)
 2015-12-08 12:08:53 -08:00
= 3205fe3da8 YARN-4248. REST API for submit/update/delete Reservations. (curino) 
(cherry picked from commit c25a635459)
(cherry picked from commit 4ac1564418)
 2015-12-07 13:46:37 -08:00
rohithsharmaks 03e615ba55 YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena via rohithsharmaks) 
(cherry picked from commit 01a641bc44)
 2015-12-07 15:20:50 +05:30
Arun Suresh db69c5f744 YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh) 
(cherry picked from commit 742632e346)
 2015-12-05 21:35:32 -08:00
Junping Du 6af5255b02 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter. 
(cherry picked from commit 62e9348bc1)
(cherry picked from commit e76ba91fa5)
 2015-12-03 06:39:07 -08:00
Allen Wittenauer 81a7ad45c9 HADOOP-12366. expose calculated paths (aw) 2015-12-01 08:15:57 -08:00
Tsuyoshi Ozawa d76b523b02 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83)
 2015-11-26 01:10:41 +09:00
Wangda Tan 7b9f7eaf5c YARN-4384. updateNodeResource CLI should not accept negative values for resource. (Junping Du via wangda) 
(cherry picked from commit 23c625ec57)
 2015-11-24 16:36:30 -08:00
Jason Lowe 3a2f5d6329 YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla 
(cherry picked from commit f5acf94eca)
 2015-11-24 23:47:54 +00:00
Jason Lowe 9e54433c6c YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 
(cherry picked from commit 4ac6799d4a)
 2015-11-24 22:36:19 +00:00
Tsuyoshi Ozawa 9e458c3e9e YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang. 
(cherry picked from commit 28dfe721b8)
 2015-11-24 22:10:31 +09:00
Akira Ajisaka ac0ddc4fec YARN-4298. Fix findbugs warnings in hadoop-yarn-common. Contributed by Sunil G. 
(cherry picked from commit 8c0133f3d4)
 2015-11-24 18:40:56 +09:00
Karthik Kambatla 49ed955c91 YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 
(cherry picked from commit 52948bb20b)
 2015-11-24 13:52:29 +05:30
Jian He b339a4b8d6 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 
(cherry picked from commit 8676a118a1)
 2015-11-23 17:20:24 -08:00
Jason Lowe 9d303edbfb YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 
(cherry picked from commit d36b6e045f)
 2015-11-23 20:31:50 +00:00
Arun Suresh cbbdbe3cdd YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh) 
(cherry picked from commit da1016365a)
 2015-11-21 10:01:10 -08:00
Wangda Tan cdca460ad4 YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda) 
(cherry picked from commit 2346fa3141)
 2015-11-20 15:56:17 -08:00
Jason Lowe f3bca8bc1a YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 
(cherry picked from commit 060cdcbe5d)
 2015-11-20 23:13:27 +00:00
Xuan be65d319fa YARN-2859.addendum: fix the remaining issue from the previous patch 
(cherry picked from commit f114e728da)
 2015-11-19 10:09:48 -08:00
Steve Loughran a0b3e43a6d YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable. (stevel) 2015-11-18 21:41:24 +00:00
Arun Suresh d8e60080e9 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh) 
(cherry picked from commit 6a61928fb7)
 2015-11-17 15:51:59 -08:00
Jian He 2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Junping Du 1d107d8058 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b)
 2015-11-15 04:44:59 -08:00
Wangda Tan 875aec3177 YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda) 
(cherry picked from commit 7ff280fca9)
 2015-11-12 11:50:41 -08:00
Wangda Tan 13f8c7093b YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda) 
(cherry picked from commit 7f55a18071)
 2015-11-12 11:24:04 -08:00