Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,967 Commits 326 Branches 371 Tags
Commit Graph

2690 Commits

Author SHA1 Message Date
Varun Saxena
fedcce2f4c YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena) 
(cherry picked from commit f59e36b4ce71d3019ab91b136b6d7646316954e7)
 2017-01-09 11:27:48 +05:30
Junping Du
689d19575f YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong. 
(cherry picked from commit 736f54b727c3f0ecc8fb9a594f2281c240c89cb8)
 2017-01-05 20:37:56 +00:00
Jason Lowe
2adce1b361 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit d0162f2040a4d98fbac51527dfc9447888cb32ab)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/util/WebAppUtils.java
 2017-01-05 20:28:26 +00:00
Jian He
42d1b38989 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2017-01-04 15:43:06 -08:00
Jian He
ca66bedeff YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 
(cherry picked from commit 264c06a4381695002d218f688c85f448d6362ec8)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
 2017-01-04 15:09:14 -08:00
Junping Du
7aa95bcdd5 Addendum patch to YARN-3866: fix incompatible API change. 2017-01-04 00:05:09 -08:00
Wangda Tan
ebafe075d6 YARN-6029. CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by one thread and LeafQueue#assignContainers is releasing excessive reserved container is called by another thread. (Tao Yang via wangda) 2017-01-03 16:21:46 -08:00
Jian He
d7b80f1b02 YARN-5709. Cleanup leader election configs and pluggability. Contributed by Karthik Kambatla 2016-12-29 16:24:17 -08:00
Wangda Tan
1e886e7520 YARN-4567. Fix javadoc of ResourceManager.java failing on Java 8. (stevel) 2016-12-29 16:05:47 -08:00
Sunil G
e253633009 YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 
(cherry picked from commit f6715b26b65fc127b35368d0460b786ece88b5b8)
 2016-12-28 16:06:13 +05:30
Akira Ajisaka
8e4479dda3 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539eaf7745ec8690f9281726fb4cdcdbe94)
(cherry picked from commit 522bc98026c6a4a96bbf5246d9d0fbed23401eb6)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestHedgingRequestRMFailoverProxyProvider.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
 2016-12-23 04:15:39 +09:00
Xuan
e0401e0781 YARN-4559. Make leader elector and zk store share the same curator 
client. Contributed by Jian He

(cherry picked from commit 890a2ebd1af51d24ccbbc5d1d65d17b24ad8ab9b)
(cherry picked from commit be3322792f8961eb3e1bec7a0119a4bf092543ee)
 2016-12-20 16:15:59 -08:00
Xuan
120f3a0ff9 YARN-4438. Implement RM leader election with curator. Contributed by Jian He 
(cherry picked from commit 89022f8d4bac0e9d0b848fd91e9c4d700fe1cdbe)
(cherry picked from commit 2cbbf76c3dd9570e08914077de8e610d93ee0c9e)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-12-13 13:30:15 -08:00
Wangda Tan
6a18ae849f YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda) 
(cherry-picked from commit 90dd3a8148468ac37a3f2173ad8d45e38bfcb0c9)
 2016-12-12 23:13:08 +00:00
Karthik Kambatla
a0fa1f8386 YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha) 
(cherry picked from commit aedd5c4c1b6695d46d7660597685a9a71aaffc64)
 2016-12-10 09:05:33 -08:00
Jian He
40367c8da3 YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 
(cherry picked from commit bb62e0592566b2fcae7136b30972aad2d3ac55b0)
 2016-12-09 20:33:54 +00:00
Naganarasimha
5b44b0c9ed YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 
(cherry picked from commit f3b8ff54ab08545d7093bf8861b44ec9912e8dc3)
 2016-12-08 07:37:12 +05:30
Junping Du
879caf688b YARN-5184. Fix up incompatible changes introduced on ContainerStatus and NodeReport. Contributed by Sangjin Lee. 2016-12-06 14:48:36 -08:00
Jian He
802a1fb2fc YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 
(cherry picked from commit 43ebff2e354142bddcb42755766a965ae8a503a6)
 2016-12-05 11:40:46 -08:00
Wangda Tan
a87abf9b69 YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda) 
(cherry picked from commit fdc46bfb37776d8c41b68f6c33a2379d0f329994)
 2016-12-02 22:53:19 +00:00
Wangda Tan
f9ef3e3719 YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda) 2016-12-02 21:17:18 +00:00
Jason Lowe
e6394e3a35 YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria 
(cherry picked from commit 6b17d449bf93e55c4f373ba96f12af6319a8411c)
 2016-12-02 17:00:05 +00:00
Jason Lowe
74503ed6b4 YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger 
(cherry picked from commit 009452bb6dbe5dffb0b304d67a2f360fe0eee1e2)
 2016-11-21 16:40:00 +00:00
Jian He
f1e2e7354c YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan 2016-11-19 01:57:11 +00:00
Wangda Tan
c34072a890 YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan) 
(cherry picked from commit 7e8c9beb4156dcaeb3a11e60aaa06d2370626913)
 2016-11-18 16:13:54 +00:00
Wei-Chiu Chuang
20c90e13b8 YARN-5271. ATS client doesn't work with Jersey 2 on the classpath. Contributed by Weiwei Yang. 
(cherry picked from commit 09520cb439f8b002e3f2f3d8f5080ffc34f4bd5c)
(cherry picked from commit c5a3ec493b8ad2fdec8a23e078c5914317d7b7c5)
 2016-11-17 22:28:37 -06:00
Jason Lowe
cb0fccad19 YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang 
(cherry picked from commit 59bfcbf3579e45ddf96db3aafccf669c8e03648f)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
 2016-11-16 22:43:56 +00:00
Xiao Chen
fc4c0595be YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák. 
(cherry picked from commit d5c396ccb361e1a0cf4307ce1d1b37fae0eec250)
 2016-11-15 13:57:43 -08:00
Naganarasimha
5c13bb5008 YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung. 
(cherry picked from commit 7ffb9943b8838a3bb56684e0722db40d800743a2)
 2016-11-15 15:52:05 +05:30
Naganarasimha
59a3d9a576 Reverting it because of YARN-5287, blocker issue has been reported. 
Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang."

This reverts commit 16c8fd9dca75e8d4f36e63ef0940aed83f8ebecf.
 2016-11-13 17:43:01 +05:30
Eric Payne
1a6a5af44a YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 
(cherry picked from commit dd5b9dabf90dbd68afdc87421101a5f4bad357d9)
 2016-11-10 23:32:17 +00:00
Naganarasimha
108f09a763 YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 
(cherry picked from commit 8a2998c08c614f9c64f012f40d31404fe9217e63)
 2016-11-10 05:24:12 +05:30
Naganarasimha
5f386967a1 YARN-5720. Update document for "rmadmin -replaceLabelOnNode". Contributred by Tao Jie 
(cherry picked from commit 0e75496049001b00c00901fcbfe3d4229b2fd64a)
 2016-11-08 10:37:34 +05:30
Varun Vasudev
3603e525b1 YARN-4884. Fix missing documentation about rmadmin command regarding node labels. Contributed by Kai Sasaki. 
(cherry picked from commit f1b8f6b2c16403869f78a54268ae1165982a7050)
 2016-11-08 10:22:59 +05:30
Sunil
ee5dd2c388 YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributed by Bibin A Chundatt 2016-11-05 13:06:31 +05:30
Jason Lowe
4cb4e4c0a1 YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 
(cherry picked from commit 6bb741ff0ef208a8628bc64d6537999d4cd67955)
 2016-11-04 22:25:35 +00:00
Rohith Sharma K S
c89d462761 YARN-5608. TestAMRMClient.setup() fails with ArrayOutOfBoundsException. Contributed by Daniel Templeton. 
(cherry picked from commit f9557127b3660e138cab8d9535a589677809384a)
 2016-11-04 13:52:11 +00:00
Naganarasimha
605e9bf3bf YARN-5697. Use CliParser to parse options in RMAdminCLI. Contributed by Tao Jie. 2016-11-03 10:31:30 +05:30
Varun Saxena
8602e2cb0f YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:43:28 +05:30
Jason Lowe
4a70bd86da YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 
(cherry picked from commit 76893a41003d57d94eb1a5f486010815266046af)
 2016-11-01 20:24:44 +00:00
Naganarasimha
b8ae7a8ee4 YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt. 
(cherry picked from commit e4023f0d8177ac37a75685b55c11991ca4ea2c23)
 2016-10-31 19:09:53 +05:30
Varun Saxena
367cb029d4 YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-30 11:50:54 +05:30
Jason Lowe
93165c1ade YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 2016-10-28 17:44:56 +00:00
Jason Lowe
9cc7ab4e96 YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 
(cherry picked from commit 1b79c417dca17bcd2e031864bc6ca07254c61b47)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
 2016-10-28 16:11:59 +00:00
Jason Lowe
83bb428e6c YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 
(cherry picked from commit 7146359bfd436a76585fb1f3ea93716795308cec)
 2016-10-28 15:50:22 +00:00
Jason Lowe
558e53b10b HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da00a6cd71a2901c84a0298ce14a49e23)
 2016-10-28 15:06:57 +00:00
Carlo Curino
b32f14d806 YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 
(cherry picked from commit 892a8348fceb42069ea9877251c413fe33415e16)
(cherry picked from commit 58ac40b55296834a8e3f3375caddc03bee901e9a)
 2016-10-27 15:23:04 -07:00
Subru Krishnan
55ba220721 YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 
(cherry picked from commit 79ae78dcbec183ab53b26de408b4517e5a151878)
(cherry picked from commit 4274600b9527706259ec5df62a13a5a66adc3ff2)
 2016-10-27 15:19:01 -07:00
Jason Lowe
d21dc8aa6c YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li 
(cherry picked from commit 7e3c327d316b33d6a09bfd4e65e7e5384943bb1d)
 2016-10-27 20:45:18 +00:00
Naganarasimha
3b9df97158 YARN-3848. TestNodeLabelContainerAllocation is timing out. Contributed by Varun Saxena 
(cherry picked from commit 6c8830992c734313fc95ac9988539c4d813c3581)
 2016-10-27 17:16:34 +05:30