Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,431 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2829 Commits

Author SHA1 Message Date
Eric Payne 73563bc9b6 YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe. 2017-04-26 14:33:27 -05:00
Junping Du 0de6f28020 YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6ed9d36224)
(cherry picked from commit 9289f4527d)
 2017-04-17 15:59:00 -07:00
Jason Lowe a61ec7c374 YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger 
(cherry picked from commit 4168805502)
 2017-04-14 17:18:08 -05:00
Eric Payne d27f74d915 YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger 
(cherry picked from commit 157a9f3104)
 2017-04-14 11:54:30 -05:00
Junping Du c43ee00587 YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen. 
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
(cherry picked from commit 9d9087a67a)
(cherry picked from commit d77cada182)
 2017-04-12 14:17:04 -07:00
Junping Du dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Jason Lowe c5b1a8910d YARN-6461. TestRMAdminCLI has very low test timeouts. Contributed by Eric Badger 
(cherry picked from commit 90d97372ed)
 2017-04-11 09:18:56 -05:00
Jason Lowe 6e41aec109 YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 2017-04-06 16:26:24 -05:00
Varun Saxena bf2c63ce46 YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena) 
(cherry picked from commit 7d963c477a)
 2017-04-06 02:09:32 +05:30
Jason Lowe ab231f5e1c YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang 2017-04-05 12:36:26 -05:00
Karthik Kambatla 9c1cf63a2d YARN-6436. TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low. (Eric Badger via kasha) 
(cherry picked from commit 9cc04b4708)
 2017-04-05 10:04:36 -05:00
Varun Saxena 4fb72699ff YARN-6420. RM startup failure due to wrong order in nodelabel editlog (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit 19b89c4c7b)
 2017-04-05 09:43:36 +05:30
Eric Payne 800d632caa YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe 
(cherry picked from commit 318bfb01bc)
 2017-03-31 12:37:15 -05:00
Varun Saxena a2020024e2 YARN-6352. Header injections are possible in application proxy servlet (Naganarasimha G R via Varun Saxena) 
(cherry picked from commit 3fe7d36e72)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
 2017-03-30 04:06:54 +05:30
Varun Saxena 5e751f3367 YARN-5368. Memory leak in timeline server (Jonathan Eagles via Varun Saxena) 
(cherry picked from commit 01aca54a22)
 2017-03-29 02:01:53 +05:30
Jason Lowe b1f0bde5dc YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter 
(cherry picked from commit bea8f8190f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRM.java
 2017-03-28 09:54:55 -05:00
Wangda Tan b218676b15 YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda) 
(cherry picked from commit cd014d57aa)
 2017-03-27 14:05:54 -07:00
Junping Du 86800b30a9 Add COMMON/MAPREDUCE/YARN jdiff of 2.8.0 release. 
(cherry picked from commit 6df029db36)
(cherry picked from commit 3ad4d37363)
 2017-03-25 14:34:55 -07:00
Jason Lowe 6c8df1208a YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi 
(cherry picked from commit 4a8e304502)
 2017-03-17 14:08:14 -05:00
Jason Lowe 5d86ce7b0d YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee 
(cherry picked from commit a16ba4296e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/Container.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/MockContainer.java
 2017-03-16 09:36:34 -05:00
Jason Lowe dd7c688a94 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 
(cherry picked from commit 9649c27864)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/resource/TestResources.java
 2017-03-10 13:09:20 -06:00
Jason Lowe 5c14eb6f18 YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen 
(cherry picked from commit deb9f56946)
 2017-03-10 11:09:39 -06:00
Jason Lowe ffc00c57d4 YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne 
(cherry picked from commit d7762a5511)
 2017-03-08 16:50:26 -06:00
Jian He 2e8e51ad40 HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-08 10:49:42 -08:00
Jian He d13a5ddc16 YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong 2017-03-03 15:24:40 -08:00
Jason Lowe 6c48440627 YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen 
(cherry picked from commit e58fc76030)
 2017-03-03 08:24:20 -06:00
Karthik Kambatla 1d156c8758 YARN-6175. FairScheduler: Negative vcore for resource needed to preempt. (Yufei Gu via kasha) 2017-02-23 14:29:30 -08:00
Wangda Tan 920847edce YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 
(cherry picked from commit 0566b30875dfc61dbbba4cde0d42e15b116e85b8)
 2017-02-22 11:21:57 -08:00
Li Lu 83a5896180 YARN-6177. Yarn client should exit with an informative error message if an incompatible Jersey library is used at client. Contributed by Weiwei Yang. 
(cherry picked from commit 5d339c46f5)
 2017-02-16 13:50:22 -08:00
Rohith Sharma K S da944fe0f4 YARN-6200. Reverting since the same functionality achieved by YARN-1623. 
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit f102c4f6c8.
 2017-02-16 17:54:37 +05:30
Karthik Kambatla 233252aa9a YARN-3933. FairScheduler: Multiple calls to completedContainer are not safe. (Shiwei Guo and Miklos Szegedi via kasha) 
(cherry picked from commit 646c6d6509)
(cherry picked from commit 30ff5bff1a)
 2017-02-13 11:47:10 -08:00
Karthik Kambatla 5d0ec2e24d YARN-6151. FS preemption does not consider child queues over fairshare if the parent is under. (Yufei Gu via kasha) 2017-02-09 20:23:50 -08:00
Jason Lowe af65f8bb59 YARN-6137. Yarn client implicitly invoke ATS client which accesses HDFS. Contributed by Li Lu 
(cherry picked from commit 37b4acf7ce)
 2017-02-08 15:01:10 -06:00
Sunil G 959db38662 YARN-6031. Application recovery has failed when node label feature is turned off during RM recovery. Contributed by Ying Zhang. 2017-02-08 16:03:45 +05:30
Chris Nauroth cf6c143c0f YARN-5456. container-executor support for FreeBSD, NetBSD, and others if conf path is absolute. Contributed by Allen Wittenauer. 
(cherry picked from commit b913677365)
(cherry picked from commit c5203e170ddd68fc8532792878c37d4762068472)
 2017-01-19 20:51:38 +05:30
Chris Nauroth 3668da2c18 YARN-5121. fix some container-executor portability issues. Contributed by Allen Wittenauer. 
(cherry picked from commit ef501b1a0b)
(cherry picked from commit 384803d09ac45886e74a0501f4b419a2b756c20c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-01-19 20:51:38 +05:30
Naganarasimha e459092a08 YARN-6072. RM unable to start in secure mode. Contributed by Ajith S. 2017-01-12 10:14:39 +05:30
Junping Du dbac88baa9 YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe. 2017-01-10 18:06:18 -08:00
Junping Du 7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Varun Saxena fedcce2f4c YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena) 
(cherry picked from commit f59e36b4ce)
 2017-01-09 11:27:48 +05:30
Junping Du 689d19575f YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong. 
(cherry picked from commit 736f54b727)
 2017-01-05 20:37:56 +00:00
Jason Lowe 2adce1b361 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit d0162f2040)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/util/WebAppUtils.java
 2017-01-05 20:28:26 +00:00
Jian He 42d1b38989 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2017-01-04 15:43:06 -08:00
Jian He ca66bedeff YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 
(cherry picked from commit 264c06a438)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
 2017-01-04 15:09:14 -08:00
Junping Du 7aa95bcdd5 Addendum patch to YARN-3866: fix incompatible API change. 2017-01-04 00:05:09 -08:00
Wangda Tan ebafe075d6 YARN-6029. CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by one thread and LeafQueue#assignContainers is releasing excessive reserved container is called by another thread. (Tao Yang via wangda) 2017-01-03 16:21:46 -08:00
Jian He d7b80f1b02 YARN-5709. Cleanup leader election configs and pluggability. Contributed by Karthik Kambatla 2016-12-29 16:24:17 -08:00
Wangda Tan 1e886e7520 YARN-4567. Fix javadoc of ResourceManager.java failing on Java 8. (stevel) 2016-12-29 16:05:47 -08:00
Sunil G e253633009 YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 
(cherry picked from commit f6715b26b6)
 2016-12-28 16:06:13 +05:30
Akira Ajisaka 8e4479dda3 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539ea)
(cherry picked from commit 522bc98026)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestHedgingRequestRMFailoverProxyProvider.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
 2016-12-23 04:15:39 +09:00
Xuan e0401e0781 YARN-4559. Make leader elector and zk store share the same curator 
client. Contributed by Jian He

(cherry picked from commit 890a2ebd1a)
(cherry picked from commit be3322792f)
 2016-12-20 16:15:59 -08:00
Xuan 120f3a0ff9 YARN-4438. Implement RM leader election with curator. Contributed by Jian He 
(cherry picked from commit 89022f8d4b)
(cherry picked from commit 2cbbf76c3d)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-12-13 13:30:15 -08:00
Wangda Tan 6a18ae849f YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda) 
(cherry-picked from commit 90dd3a8148)
 2016-12-12 23:13:08 +00:00
Karthik Kambatla a0fa1f8386 YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha) 
(cherry picked from commit aedd5c4c1b)
 2016-12-10 09:05:33 -08:00
Jian He 40367c8da3 YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 
(cherry picked from commit bb62e05925)
 2016-12-09 20:33:54 +00:00
Naganarasimha 5b44b0c9ed YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 
(cherry picked from commit f3b8ff54ab)
 2016-12-08 07:37:12 +05:30
Junping Du 879caf688b YARN-5184. Fix up incompatible changes introduced on ContainerStatus and NodeReport. Contributed by Sangjin Lee. 2016-12-06 14:48:36 -08:00
Jian He 802a1fb2fc YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 
(cherry picked from commit 43ebff2e35)
 2016-12-05 11:40:46 -08:00
Wangda Tan a87abf9b69 YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda) 
(cherry picked from commit fdc46bfb37)
 2016-12-02 22:53:19 +00:00
Wangda Tan f9ef3e3719 YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda) 2016-12-02 21:17:18 +00:00
Jason Lowe e6394e3a35 YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria 
(cherry picked from commit 6b17d449bf)
 2016-12-02 17:00:05 +00:00
Jason Lowe 74503ed6b4 YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger 
(cherry picked from commit 009452bb6d)
 2016-11-21 16:40:00 +00:00
Jian He f1e2e7354c YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan 2016-11-19 01:57:11 +00:00
Wangda Tan c34072a890 YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan) 
(cherry picked from commit 7e8c9beb41)
 2016-11-18 16:13:54 +00:00
Wei-Chiu Chuang 20c90e13b8 YARN-5271. ATS client doesn't work with Jersey 2 on the classpath. Contributed by Weiwei Yang. 
(cherry picked from commit 09520cb439)
(cherry picked from commit c5a3ec493b)
 2016-11-17 22:28:37 -06:00
Jason Lowe cb0fccad19 YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang 
(cherry picked from commit 59bfcbf357)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
 2016-11-16 22:43:56 +00:00
Xiao Chen fc4c0595be YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák. 
(cherry picked from commit d5c396ccb361e1a0cf4307ce1d1b37fae0eec250)
 2016-11-15 13:57:43 -08:00
Naganarasimha 5c13bb5008 YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung. 
(cherry picked from commit 7ffb9943b8)
 2016-11-15 15:52:05 +05:30
Naganarasimha 59a3d9a576 Reverting it because of YARN-5287, blocker issue has been reported. 
Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang."

This reverts commit 16c8fd9dca.
 2016-11-13 17:43:01 +05:30
Eric Payne 1a6a5af44a YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 
(cherry picked from commit dd5b9dabf9)
 2016-11-10 23:32:17 +00:00
Naganarasimha 108f09a763 YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 
(cherry picked from commit 8a2998c08c)
 2016-11-10 05:24:12 +05:30
Naganarasimha 5f386967a1 YARN-5720. Update document for "rmadmin -replaceLabelOnNode". Contributred by Tao Jie 
(cherry picked from commit 0e75496049)
 2016-11-08 10:37:34 +05:30
Varun Vasudev 3603e525b1 YARN-4884. Fix missing documentation about rmadmin command regarding node labels. Contributed by Kai Sasaki. 
(cherry picked from commit f1b8f6b2c1)
 2016-11-08 10:22:59 +05:30
Sunil ee5dd2c388 YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributed by Bibin A Chundatt 2016-11-05 13:06:31 +05:30
Jason Lowe 4cb4e4c0a1 YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 
(cherry picked from commit 6bb741ff0e)
 2016-11-04 22:25:35 +00:00
Rohith Sharma K S c89d462761 YARN-5608. TestAMRMClient.setup() fails with ArrayOutOfBoundsException. Contributed by Daniel Templeton. 
(cherry picked from commit f9557127b3)
 2016-11-04 13:52:11 +00:00
Naganarasimha 605e9bf3bf YARN-5697. Use CliParser to parse options in RMAdminCLI. Contributed by Tao Jie. 2016-11-03 10:31:30 +05:30
Varun Saxena 8602e2cb0f YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:43:28 +05:30
Jason Lowe 4a70bd86da YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 
(cherry picked from commit 76893a4100)
 2016-11-01 20:24:44 +00:00
Naganarasimha b8ae7a8ee4 YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt. 
(cherry picked from commit e4023f0d81)
 2016-10-31 19:09:53 +05:30
Varun Saxena 367cb029d4 YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-30 11:50:54 +05:30
Jason Lowe 93165c1ade YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 2016-10-28 17:44:56 +00:00
Jason Lowe 9cc7ab4e96 YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 
(cherry picked from commit 1b79c417dc)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
 2016-10-28 16:11:59 +00:00
Jason Lowe 83bb428e6c YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 
(cherry picked from commit 7146359bfd)
 2016-10-28 15:50:22 +00:00
Jason Lowe 558e53b10b HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da0)
 2016-10-28 15:06:57 +00:00
Carlo Curino b32f14d806 YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 
(cherry picked from commit 892a8348fc)
(cherry picked from commit 58ac40b552)
 2016-10-27 15:23:04 -07:00
Subru Krishnan 55ba220721 YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 
(cherry picked from commit 79ae78dcbe)
(cherry picked from commit 4274600b95)
 2016-10-27 15:19:01 -07:00
Jason Lowe d21dc8aa6c YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li 
(cherry picked from commit 7e3c327d31)
 2016-10-27 20:45:18 +00:00
Naganarasimha 3b9df97158 YARN-3848. TestNodeLabelContainerAllocation is timing out. Contributed by Varun Saxena 
(cherry picked from commit 6c8830992c)
 2016-10-27 17:16:34 +05:30
Jian He dab611e605 Revert YARN-4126. RM should not issue delegation tokens in unsecure mode. 
(cherry picked from commit bb8214cab8)
 2016-10-23 16:56:12 -07:00
Rohith Sharma K S 34023ca418 YARN-4763. RMApps Page crashes with NPE. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 19ee185907)
 2016-10-19 17:40:42 +00:00
Varun Saxena 28d75482ff YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-10-14 23:36:39 +05:30
Varun Vasudev d1841b4750 YARN-5704. Provide config knobs to control enabling/disabling new/work in progress features in container-executor. Contributed by Sidharta Seethana. 2016-10-09 13:22:01 +05:30
Varun Vasudev 50409404bf YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 8ed2e060e8)
(cherry picked from commit 78919f8c34)
 2016-10-09 11:53:03 +05:30
Junping Du 2ff6ffe0ef YARN-5659. getPathFromYarnURL should use standard methods. Contributed by Sergey Shelukhin. 
(cherry picked from commit 459a4833a9)
(cherry picked from commit 8bee319130)
 2016-10-07 07:51:01 -07:00
Rohith Sharma K S 319b101b7e YARN-5101. YARN_APPLICATION_UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed order. Contributed by Sunil G. 
(cherry picked from commit 4d2f380d78)
 2016-10-06 20:44:39 +05:30
Varun Saxena 3656b84085 YARN-5491. Fix random failure of TestCapacityScheduler#testCSQueueBlocked (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit d677b68c25)
 2016-10-04 20:57:01 +00:00
Naganarasimha 90443140a5 YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 2016-10-03 13:29:18 -04:00
Jason Lowe 515727f46f YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant. (Akihiro Suda via rohithsharmaks) 
(cherry picked from commit ac68666803)
 2016-10-03 16:06:17 +00:00
Wangda Tan 69f91d8c48 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc)
(cherry picked from commit 269401dc83)
 2016-09-27 11:46:49 -07:00
Rohith Sharma K S 4f6facb90d YARN-5631. Missing refreshClusterMaxPriority usage in rmadmin help message. Contributed by Kai Sasaki. 2016-09-27 23:07:43 +05:30
Naganarasimha 63ca9f5441 YARN-5660. Wrong audit constants are used in Get/Put of priority in RMWebService. Contributed by Rohith Sharma K S 
(cherry picked from commit ebf528c443)
 2016-09-27 16:46:52 +05:30
Akira Ajisaka ece3ca0cba YARN-5663. Small refactor in ZKRMStateStore. Contributed by Oleksii Dymytrov. 
(cherry picked from commit 14a696f369)
(cherry picked from commit 74f2df16a9)
 2016-09-26 15:01:39 +09:00
Naganarasimha 4c2b20ca3e YARN-3877. YarnClientImpl.submitApplication swallows exceptions. Contributed by Varun Saxena 
(cherry picked from commit e4e72db5f9)
 2016-09-25 17:42:08 +05:30
Naganarasimha 41169b7c2b YARN-5664. Fix Yarn documentation to link to correct versions. Contributed by Xiao Chen 
(cherry picked from commit 4d16d2d72f)
 2016-09-24 23:24:56 +05:30
Naganarasimha fa2025316d YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 22:42:28 +05:30
Kihwal Lee 9269225522 YARN-5622. TestYarnCLI.testGetContainers fails due to mismatched date formats. Contributed by Eric Badger. 
(cherry picked from commit 6e849cb658)
 2016-09-23 14:15:31 -05:00
Varun Saxena 42f6f68c78 YARN-5539. TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out (Junping Du via Varun Saxena) 2016-09-23 13:34:18 +05:30
Wangda Tan 6f6e62ad86 YARN-4591. YARN Web UIs should provide a robots.txt. (Sidharta Seethana via wangda) 
(cherry picked from commit 5a58bfee30)
 2016-09-20 17:29:27 -07:00
Jason Lowe 4782390eb1 YARN-5655. TestContainerManagerSecurity#testNMTokens is asserting. Contributed by Robert Kanter 
(cherry picked from commit c6d1d742e7)
 2016-09-20 14:17:26 +00:00
Jason Lowe 564d9e6101 YARN-5540. Scheduler spends too much time looking at empty priorities. Contributed by Jason Lowe 2016-09-19 20:34:46 +00:00
Naganarasimha a1cc90bca1 YARN-4232. TopCLI console support for HA mode. Contributed by Bibin A Chundatt 
(cherry picked from commit ade7c2bc9c)
 2016-09-17 10:01:11 +05:30
Jason Lowe 1ac8206a85 YARN-5630. NM fails to start after downgrade from 2.8 to 2.7. Contributed by Jason Lowe 
(cherry picked from commit e793309735)
 2016-09-13 14:43:55 +00:00
Varun Vasudev d5ea508ca2 YARN-5555. Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested. Contributed by Eric Payne. 2016-09-10 18:28:38 +00:00
Jason Lowe 1742022390 YARN-5632. UPDATE_EXECUTION_TYPE causes UpdateContainerRequestPBImpl to throw. Contributed by Jason Lowe 2016-09-09 22:40:17 +00:00
Karthik Kambatla 7a27b2a82f YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. Addendum to fix test flakiness. (Robert Kanter via kasha) 2016-09-08 21:11:20 -07:00
Karthik Kambatla f9016dfec3 YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. (Robert Kanter via kasha) 2016-09-06 16:23:06 -07:00
Ray Chiang 02272a6a77 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 
(cherry picked from commit 4ee1729cb0)
 2016-09-02 15:00:24 -07:00
Arun Suresh 979b29a03c YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b)
(cherry picked from commit b279f42d79)
 2016-08-31 20:06:49 -07:00
Zhe Zhang 765139e4da YARN-5550. TestYarnCLI#testGetContainers should format according to CONTAINER_PATTERN. Contributed by Jonathan Hung. 
(cherry picked from commit ed6ff5cd78)
(cherry picked from commit 3ad4f0823c)
 2016-08-29 14:51:40 -07:00
Jason Lowe 09e585e30a YARN-5560. Clean up bad exception catching practices in TestYarnClient. Contributed by Sean Po 
(cherry picked from commit 4cbe61407d)
 2016-08-29 16:20:04 +00:00
Naganarasimha b952ce854b YARN-3940. Application moveToQueue should check NodeLabel permission. Contributed by Bibin A Chundatt 
(cherry picked from commit 46e02ab719)
 2016-08-26 20:24:37 +05:30
Jason Lowe 8657dece8b YARN-5389. TestYarnClient#testReservationDelete fails. Contributed by Sean Po 
(cherry picked from commit 4037b22b7c)
 2016-08-25 15:11:47 +00:00
Rohith Sharma K S 71a3b01d51 YARN-5544. TestNodeBlacklistingOnAMFailures fails on trunk. Contributed by Sunil G. 
(cherry picked from commit 0d5997d2b9)
 2016-08-23 14:40:05 +05:30
Wangda Tan 2e9d84d24c YARN-4685. Disable AM blacklisting by default to mitigate situations that application get hanged. (Rohith Sharma K S via wangda) 
(cherry picked from commit 2da32a6ef9)
(cherry picked from commit 3e511f39c2)
 2016-08-19 16:55:40 -07:00
Wangda Tan 9bf420b3bc YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-08-19 16:55:34 -07:00
Wangda Tan 3bf2e16f76 YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda) 
(cherry picked from commit 444b2ea7af)
(cherry picked from commit c7d782d2f6)
 2016-08-19 16:54:03 -07:00
Varun Vasudev 5b49bcaee3 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 
(cherry picked from commit 308d63f382)
(cherry picked from commit 99966b1912)
 2016-08-18 12:33:52 -07:00
Karthik Kambatla 42a88f9c58 YARN-3446. FairScheduler headroom calculation should exclude nodes in the blacklist. (Zhihai Xu via kasha) 
(cherry picked from commit 9d04f26d4c)
(cherry picked from commit f0923819c3)
 2016-08-18 12:30:39 -07:00
Jason Lowe 74831973cb YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit e0b14f26f5)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-08-15 19:45:45 +00:00
Junping Du 1a38dd9cee YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 
(cherry picked from commit 0005816743)
 2016-08-15 19:18:36 +00:00
Varun Saxena ca139a3f87 YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena) 2016-08-13 00:27:17 +05:30
Rohith Sharma K S f7cc0af611 YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C. 
(cherry picked from commit 5199db387d)
 2016-08-11 11:52:44 +05:30
Jason Lowe 69217ba813 YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C 
(cherry picked from commit 5c95bb315b)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java
 2016-08-10 19:15:12 +00:00
Jason Lowe e1845faeda YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee 
(cherry picked from commit 3d401206cf)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2016-08-10 18:09:11 +00:00
Rohith Sharma K S 3b4d6ece1a YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat. Contributed by Rohith Sharma K S 
(cherry picked from commit 26b5e6116f)
 2016-08-09 22:19:17 +00:00
Naganarasimha 16c8fd9dca YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang. 2016-08-09 17:34:48 +05:30
Naganarasimha e6a994badf YARN-4624. NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI. Contributed by Brahma Reddy Battula 
(cherry picked from commit d81b8163b4)
(cherry picked from commit 991823482f)
 2016-08-06 08:19:56 +05:30
Jason Lowe 470ea3da1f YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit c01bee0108)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-08-05 20:10:04 +00:00
Jason Lowe eef24f5d93 YARN-5469. Increase timeout of TestAmFilter.testFilter. Contributed by Eric Badger 
(cherry picked from commit db4a61dc61)
 2016-08-03 19:54:00 +00:00
Jason Lowe 5883718eea YARN-4717. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 
(cherry picked from commit 22ca176dfe)
 2016-08-03 19:33:58 +00:00
Jason Lowe c972498b74 YARN-5462. TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently. Contributed by Eric Badger 
(cherry picked from commit db646540f0)
 2016-08-03 19:19:47 +00:00
Jason Lowe aca7eea611 YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla 2016-08-03 19:01:14 +00:00
Wangda Tan c322e749d6 YARN-5342. Improve non-exclusive node partition resource allocation in Capacity Scheduler. (Sunil G via wangda) 2016-08-03 11:29:12 -07:00
Junping Du bba06f4526 YARN-5434. Add -client|server argument for graceful decommmission. Contributed by Robert Kanter. 
(cherry picked from commit 95f2b98597)
(cherry picked from commit 47dd871f96)
 2016-07-29 10:31:17 -07:00
Jason Lowe 95fb2fe394 YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S 
(cherry picked from commit a1890c32c5)
 2016-07-28 21:49:21 +00:00
Junping Du 08358c8d4e YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu. 
(cherry picked from commit 7f3c306e2e)
(cherry picked from commit 932bf79244)
 2016-07-28 06:56:00 -07:00
mattf-horton c96cb3fd48 HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley. 2016-07-21 16:01:05 -07:00
Rohith Sharma K S 7234fce25b YARN-5092. TestRMDelegationTokens fails intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 557a245d83)
 2016-07-21 12:49:52 +05:30
Rohith Sharma K S cdcd131d5f YARN-1126. Add validation of users input nodes-states options to nodes CLI. Contributed by Wei Yan. 
(cherry picked from commit be34b2a8fd)
 2016-07-21 12:25:14 +05:30