Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,028 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2705 Commits

Author SHA1 Message Date
Jian He d13a5ddc16 YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong 2017-03-03 15:24:40 -08:00
Jason Lowe 6c48440627 YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen 
(cherry picked from commit e58fc76030)
 2017-03-03 08:24:20 -06:00
Karthik Kambatla 1d156c8758 YARN-6175. FairScheduler: Negative vcore for resource needed to preempt. (Yufei Gu via kasha) 2017-02-23 14:29:30 -08:00
Wangda Tan 920847edce YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 
(cherry picked from commit 0566b30875dfc61dbbba4cde0d42e15b116e85b8)
 2017-02-22 11:21:57 -08:00
Li Lu 83a5896180 YARN-6177. Yarn client should exit with an informative error message if an incompatible Jersey library is used at client. Contributed by Weiwei Yang. 
(cherry picked from commit 5d339c46f5)
 2017-02-16 13:50:22 -08:00
Rohith Sharma K S da944fe0f4 YARN-6200. Reverting since the same functionality achieved by YARN-1623. 
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit f102c4f6c8.
 2017-02-16 17:54:37 +05:30
Karthik Kambatla 233252aa9a YARN-3933. FairScheduler: Multiple calls to completedContainer are not safe. (Shiwei Guo and Miklos Szegedi via kasha) 
(cherry picked from commit 646c6d6509)
(cherry picked from commit 30ff5bff1a)
 2017-02-13 11:47:10 -08:00
Karthik Kambatla 5d0ec2e24d YARN-6151. FS preemption does not consider child queues over fairshare if the parent is under. (Yufei Gu via kasha) 2017-02-09 20:23:50 -08:00
Jason Lowe af65f8bb59 YARN-6137. Yarn client implicitly invoke ATS client which accesses HDFS. Contributed by Li Lu 
(cherry picked from commit 37b4acf7ce)
 2017-02-08 15:01:10 -06:00
Sunil G 959db38662 YARN-6031. Application recovery has failed when node label feature is turned off during RM recovery. Contributed by Ying Zhang. 2017-02-08 16:03:45 +05:30
Chris Nauroth cf6c143c0f YARN-5456. container-executor support for FreeBSD, NetBSD, and others if conf path is absolute. Contributed by Allen Wittenauer. 
(cherry picked from commit b913677365)
(cherry picked from commit c5203e170ddd68fc8532792878c37d4762068472)
 2017-01-19 20:51:38 +05:30
Chris Nauroth 3668da2c18 YARN-5121. fix some container-executor portability issues. Contributed by Allen Wittenauer. 
(cherry picked from commit ef501b1a0b)
(cherry picked from commit 384803d09ac45886e74a0501f4b419a2b756c20c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-01-19 20:51:38 +05:30
Naganarasimha e459092a08 YARN-6072. RM unable to start in secure mode. Contributed by Ajith S. 2017-01-12 10:14:39 +05:30
Junping Du dbac88baa9 YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe. 2017-01-10 18:06:18 -08:00
Junping Du 7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Varun Saxena fedcce2f4c YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena) 
(cherry picked from commit f59e36b4ce)
 2017-01-09 11:27:48 +05:30
Junping Du 689d19575f YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong. 
(cherry picked from commit 736f54b727)
 2017-01-05 20:37:56 +00:00
Jason Lowe 2adce1b361 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit d0162f2040)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/util/WebAppUtils.java
 2017-01-05 20:28:26 +00:00
Jian He 42d1b38989 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2017-01-04 15:43:06 -08:00
Jian He ca66bedeff YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 
(cherry picked from commit 264c06a438)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
 2017-01-04 15:09:14 -08:00
Junping Du 7aa95bcdd5 Addendum patch to YARN-3866: fix incompatible API change. 2017-01-04 00:05:09 -08:00
Wangda Tan ebafe075d6 YARN-6029. CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by one thread and LeafQueue#assignContainers is releasing excessive reserved container is called by another thread. (Tao Yang via wangda) 2017-01-03 16:21:46 -08:00
Jian He d7b80f1b02 YARN-5709. Cleanup leader election configs and pluggability. Contributed by Karthik Kambatla 2016-12-29 16:24:17 -08:00
Wangda Tan 1e886e7520 YARN-4567. Fix javadoc of ResourceManager.java failing on Java 8. (stevel) 2016-12-29 16:05:47 -08:00
Sunil G e253633009 YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 
(cherry picked from commit f6715b26b6)
 2016-12-28 16:06:13 +05:30
Akira Ajisaka 8e4479dda3 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539ea)
(cherry picked from commit 522bc98026)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestHedgingRequestRMFailoverProxyProvider.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
 2016-12-23 04:15:39 +09:00
Xuan e0401e0781 YARN-4559. Make leader elector and zk store share the same curator 
client. Contributed by Jian He

(cherry picked from commit 890a2ebd1a)
(cherry picked from commit be3322792f)
 2016-12-20 16:15:59 -08:00
Xuan 120f3a0ff9 YARN-4438. Implement RM leader election with curator. Contributed by Jian He 
(cherry picked from commit 89022f8d4b)
(cherry picked from commit 2cbbf76c3d)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-12-13 13:30:15 -08:00
Wangda Tan 6a18ae849f YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda) 
(cherry-picked from commit 90dd3a8148)
 2016-12-12 23:13:08 +00:00
Karthik Kambatla a0fa1f8386 YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha) 
(cherry picked from commit aedd5c4c1b)
 2016-12-10 09:05:33 -08:00
Jian He 40367c8da3 YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 
(cherry picked from commit bb62e05925)
 2016-12-09 20:33:54 +00:00
Naganarasimha 5b44b0c9ed YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 
(cherry picked from commit f3b8ff54ab)
 2016-12-08 07:37:12 +05:30
Junping Du 879caf688b YARN-5184. Fix up incompatible changes introduced on ContainerStatus and NodeReport. Contributed by Sangjin Lee. 2016-12-06 14:48:36 -08:00
Jian He 802a1fb2fc YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 
(cherry picked from commit 43ebff2e35)
 2016-12-05 11:40:46 -08:00
Wangda Tan a87abf9b69 YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda) 
(cherry picked from commit fdc46bfb37)
 2016-12-02 22:53:19 +00:00
Wangda Tan f9ef3e3719 YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda) 2016-12-02 21:17:18 +00:00
Jason Lowe e6394e3a35 YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria 
(cherry picked from commit 6b17d449bf)
 2016-12-02 17:00:05 +00:00
Jason Lowe 74503ed6b4 YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger 
(cherry picked from commit 009452bb6d)
 2016-11-21 16:40:00 +00:00
Jian He f1e2e7354c YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan 2016-11-19 01:57:11 +00:00
Wangda Tan c34072a890 YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan) 
(cherry picked from commit 7e8c9beb41)
 2016-11-18 16:13:54 +00:00
Wei-Chiu Chuang 20c90e13b8 YARN-5271. ATS client doesn't work with Jersey 2 on the classpath. Contributed by Weiwei Yang. 
(cherry picked from commit 09520cb439)
(cherry picked from commit c5a3ec493b)
 2016-11-17 22:28:37 -06:00
Jason Lowe cb0fccad19 YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang 
(cherry picked from commit 59bfcbf357)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
 2016-11-16 22:43:56 +00:00
Xiao Chen fc4c0595be YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák. 
(cherry picked from commit d5c396ccb361e1a0cf4307ce1d1b37fae0eec250)
 2016-11-15 13:57:43 -08:00
Naganarasimha 5c13bb5008 YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung. 
(cherry picked from commit 7ffb9943b8)
 2016-11-15 15:52:05 +05:30
Naganarasimha 59a3d9a576 Reverting it because of YARN-5287, blocker issue has been reported. 
Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang."

This reverts commit 16c8fd9dca.
 2016-11-13 17:43:01 +05:30
Eric Payne 1a6a5af44a YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 
(cherry picked from commit dd5b9dabf9)
 2016-11-10 23:32:17 +00:00
Naganarasimha 108f09a763 YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 
(cherry picked from commit 8a2998c08c)
 2016-11-10 05:24:12 +05:30
Naganarasimha 5f386967a1 YARN-5720. Update document for "rmadmin -replaceLabelOnNode". Contributred by Tao Jie 
(cherry picked from commit 0e75496049)
 2016-11-08 10:37:34 +05:30
Varun Vasudev 3603e525b1 YARN-4884. Fix missing documentation about rmadmin command regarding node labels. Contributed by Kai Sasaki. 
(cherry picked from commit f1b8f6b2c1)
 2016-11-08 10:22:59 +05:30
Sunil ee5dd2c388 YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributed by Bibin A Chundatt 2016-11-05 13:06:31 +05:30