Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,009 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1934 Commits

Author SHA1 Message Date
Rohith Sharma K S da944fe0f4 YARN-6200. Reverting since the same functionality achieved by YARN-1623. 
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit f102c4f6c8.
 2017-02-16 17:54:37 +05:30
Karthik Kambatla 233252aa9a YARN-3933. FairScheduler: Multiple calls to completedContainer are not safe. (Shiwei Guo and Miklos Szegedi via kasha) 
(cherry picked from commit 646c6d6509)
(cherry picked from commit 30ff5bff1a)
 2017-02-13 11:47:10 -08:00
Karthik Kambatla 5d0ec2e24d YARN-6151. FS preemption does not consider child queues over fairshare if the parent is under. (Yufei Gu via kasha) 2017-02-09 20:23:50 -08:00
Sunil G 959db38662 YARN-6031. Application recovery has failed when node label feature is turned off during RM recovery. Contributed by Ying Zhang. 2017-02-08 16:03:45 +05:30
Chris Nauroth cf6c143c0f YARN-5456. container-executor support for FreeBSD, NetBSD, and others if conf path is absolute. Contributed by Allen Wittenauer. 
(cherry picked from commit b913677365)
(cherry picked from commit c5203e170ddd68fc8532792878c37d4762068472)
 2017-01-19 20:51:38 +05:30
Chris Nauroth 3668da2c18 YARN-5121. fix some container-executor portability issues. Contributed by Allen Wittenauer. 
(cherry picked from commit ef501b1a0b)
(cherry picked from commit 384803d09ac45886e74a0501f4b419a2b756c20c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-01-19 20:51:38 +05:30
Naganarasimha e459092a08 YARN-6072. RM unable to start in secure mode. Contributed by Ajith S. 2017-01-12 10:14:39 +05:30
Junping Du dbac88baa9 YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe. 2017-01-10 18:06:18 -08:00
Junping Du 7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Varun Saxena fedcce2f4c YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena) 
(cherry picked from commit f59e36b4ce)
 2017-01-09 11:27:48 +05:30
Junping Du 689d19575f YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong. 
(cherry picked from commit 736f54b727)
 2017-01-05 20:37:56 +00:00
Jason Lowe 2adce1b361 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit d0162f2040)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/util/WebAppUtils.java
 2017-01-05 20:28:26 +00:00
Jian He 42d1b38989 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2017-01-04 15:43:06 -08:00
Jian He ca66bedeff YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 
(cherry picked from commit 264c06a438)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
 2017-01-04 15:09:14 -08:00
Junping Du 7aa95bcdd5 Addendum patch to YARN-3866: fix incompatible API change. 2017-01-04 00:05:09 -08:00
Wangda Tan ebafe075d6 YARN-6029. CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by one thread and LeafQueue#assignContainers is releasing excessive reserved container is called by another thread. (Tao Yang via wangda) 2017-01-03 16:21:46 -08:00
Jian He d7b80f1b02 YARN-5709. Cleanup leader election configs and pluggability. Contributed by Karthik Kambatla 2016-12-29 16:24:17 -08:00
Wangda Tan 1e886e7520 YARN-4567. Fix javadoc of ResourceManager.java failing on Java 8. (stevel) 2016-12-29 16:05:47 -08:00
Sunil G e253633009 YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 
(cherry picked from commit f6715b26b6)
 2016-12-28 16:06:13 +05:30
Akira Ajisaka 8e4479dda3 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539ea)
(cherry picked from commit 522bc98026)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestHedgingRequestRMFailoverProxyProvider.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
 2016-12-23 04:15:39 +09:00
Xuan e0401e0781 YARN-4559. Make leader elector and zk store share the same curator 
client. Contributed by Jian He

(cherry picked from commit 890a2ebd1a)
(cherry picked from commit be3322792f)
 2016-12-20 16:15:59 -08:00
Xuan 120f3a0ff9 YARN-4438. Implement RM leader election with curator. Contributed by Jian He 
(cherry picked from commit 89022f8d4b)
(cherry picked from commit 2cbbf76c3d)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-12-13 13:30:15 -08:00
Wangda Tan 6a18ae849f YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda) 
(cherry-picked from commit 90dd3a8148)
 2016-12-12 23:13:08 +00:00
Karthik Kambatla a0fa1f8386 YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha) 
(cherry picked from commit aedd5c4c1b)
 2016-12-10 09:05:33 -08:00
Jian He 40367c8da3 YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 
(cherry picked from commit bb62e05925)
 2016-12-09 20:33:54 +00:00
Naganarasimha 5b44b0c9ed YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 
(cherry picked from commit f3b8ff54ab)
 2016-12-08 07:37:12 +05:30
Jian He 802a1fb2fc YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 
(cherry picked from commit 43ebff2e35)
 2016-12-05 11:40:46 -08:00
Wangda Tan a87abf9b69 YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda) 
(cherry picked from commit fdc46bfb37)
 2016-12-02 22:53:19 +00:00
Wangda Tan f9ef3e3719 YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda) 2016-12-02 21:17:18 +00:00
Jason Lowe 74503ed6b4 YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger 
(cherry picked from commit 009452bb6d)
 2016-11-21 16:40:00 +00:00
Jian He f1e2e7354c YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan 2016-11-19 01:57:11 +00:00
Wangda Tan c34072a890 YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan) 
(cherry picked from commit 7e8c9beb41)
 2016-11-18 16:13:54 +00:00
Jason Lowe cb0fccad19 YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang 
(cherry picked from commit 59bfcbf357)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
 2016-11-16 22:43:56 +00:00
Xiao Chen fc4c0595be YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák. 
(cherry picked from commit d5c396ccb361e1a0cf4307ce1d1b37fae0eec250)
 2016-11-15 13:57:43 -08:00
Naganarasimha 5c13bb5008 YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung. 
(cherry picked from commit 7ffb9943b8)
 2016-11-15 15:52:05 +05:30
Naganarasimha 59a3d9a576 Reverting it because of YARN-5287, blocker issue has been reported. 
Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang."

This reverts commit 16c8fd9dca.
 2016-11-13 17:43:01 +05:30
Eric Payne 1a6a5af44a YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 
(cherry picked from commit dd5b9dabf9)
 2016-11-10 23:32:17 +00:00
Naganarasimha 108f09a763 YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 
(cherry picked from commit 8a2998c08c)
 2016-11-10 05:24:12 +05:30
Sunil ee5dd2c388 YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributed by Bibin A Chundatt 2016-11-05 13:06:31 +05:30
Jason Lowe 4cb4e4c0a1 YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 
(cherry picked from commit 6bb741ff0e)
 2016-11-04 22:25:35 +00:00
Varun Saxena 8602e2cb0f YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:43:28 +05:30
Jason Lowe 4a70bd86da YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 
(cherry picked from commit 76893a4100)
 2016-11-01 20:24:44 +00:00
Naganarasimha b8ae7a8ee4 YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt. 
(cherry picked from commit e4023f0d81)
 2016-10-31 19:09:53 +05:30
Varun Saxena 367cb029d4 YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-30 11:50:54 +05:30
Jason Lowe 93165c1ade YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 2016-10-28 17:44:56 +00:00
Jason Lowe 9cc7ab4e96 YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 
(cherry picked from commit 1b79c417dc)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
 2016-10-28 16:11:59 +00:00
Jason Lowe 83bb428e6c YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 
(cherry picked from commit 7146359bfd)
 2016-10-28 15:50:22 +00:00
Jason Lowe 558e53b10b HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da0)
 2016-10-28 15:06:57 +00:00
Carlo Curino b32f14d806 YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 
(cherry picked from commit 892a8348fc)
(cherry picked from commit 58ac40b552)
 2016-10-27 15:23:04 -07:00
Subru Krishnan 55ba220721 YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 
(cherry picked from commit 79ae78dcbe)
(cherry picked from commit 4274600b95)
 2016-10-27 15:19:01 -07:00