Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,437 Commits 326 Branches 371 Tags
Commit Graph

2035 Commits

Author SHA1 Message Date
Jason Lowe
3fac56e824 YARN-3975. WebAppProxyServlet should not redirect to RM page if AHS is enabled. Contributed by Mit Desai 2015-09-23 16:41:49 +00:00
Karthik Kambatla
f3fd5e30cd YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha) 
(cherry picked from commit 332b520a480994b7bd56c135f7941aad30b05e9c)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
 2015-09-17 23:35:47 -07:00
Jason Lowe
30ebb792f9 YARN-4158. Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers. Contributed by Zhihai Xu 
(cherry picked from commit 8c1cdb17a03fec91bfab172fe92b67bdeb9ebfba)
 2015-09-15 20:23:11 +00:00
Jian He
30569d930f YARN-4153. TestAsyncDispatcher failed at branch-2.7. Contributed by Zhihai Xu 2015-09-14 17:48:31 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
c05ba079e3 Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed111884d0375c177239073153139848)
(cherry picked from commit cbf90f93f9ab3463307292ddfe6fe98ab8b37842)
 2015-09-10 16:10:56 -07:00
Zhijie Shen
e48bffd055 YARN-3700. Made generic history service load a number of latest applications according to the parameter or the configuration. Contributed by Xuan Gong. 
(cherry picked from commit 54504133f41e36eaea6bb06c7b9ddb249468ecd7)
(cherry picked from commit 839f81a6326b2f8b3d5183178382c1551b0bc259)
 2015-09-10 11:56:27 -07:00
Hitesh Shah
f6b7dc24f0 YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe804743601a8b62cabc1e4dc2ec8e502f)
(cherry picked from commit 55b794e7fa205df655c19bbfe1de99091fa9dc64)
 2015-09-10 10:55:23 -07:00
Zhihai Xu
4b5767e456 YARN-4096. App local logs are leaked if log aggregation fails to initialize for the app. Contributed by Jason Lowe. 
(cherry picked from commit 16b9037dc1300b8bdbe54ba7cd47c53fe16e93d8)
 2015-09-08 12:43:55 -07:00
Xuan
b55fb0ac44 YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when 
state-store error occurs. Contributed by Jian He

(cherry picked from commit 9b78e6e33d8c117c1e909df414f20d9db56efe4b)
(cherry picked from commit a0b7ef15d0663076b65ae3f53271b54e42308bfb)
 2015-09-07 17:47:32 -07:00
Xuan
96b9455c6f YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He. 2015-09-07 17:23:56 -07:00
Jason Lowe
a7eed5128b YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li 2015-09-04 16:24:03 +00:00
Jason Lowe
1275ba2cac Revert "YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li" 
This reverts commit 71172a0bfcf549f13bf17c1f1cbb4996272dd470.
 2015-09-04 16:06:19 +00:00
Jason Lowe
71172a0bfc YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li 
(cherry picked from commit 6eaca2e3634a88dc55689e8960352d6248c424d9)
 2015-09-04 15:32:14 +00:00
Varun Vasudev
5002f885fc YARN-4103. RM WebServices missing scheme for appattempts logLinks. Contributed by Jonathan Eagles. 
(cherry picked from commit 1b2e2ca9104dfc05e9aa7f93954ae2e00b3e62c4)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppAttemptInfo.java
 2015-09-04 11:45:51 +05:30
Jian He
2d1ff2e1ca YARN-4101. RM should print alert messages if Zookeeper and Resourcemanager gets connection issue. Contributed by Xuan Gong 
(cherry picked from commit 09c64ba1ba8be7a2ac31f4e42efb8c99b682399f)
 2015-09-02 17:46:24 -07:00
Rohith Sharma K S
c2ed7e4a09 YARN-3893. Both RM in active state when Admin#transitionToActive failure from refeshAll() (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 7d6687fe76f6152a577ff2298c358dd30fce41fb)
 2015-09-02 15:29:22 +05:30
Vinod Kumar Vavilapalli
015696fb81 YARN-3222. Added the missing CHANGES.txt entry. 
(cherry picked from commit 4620767156ecc43424bc6c7c4d50519e2563cc69)
 2015-09-01 13:44:51 -07:00
Tsuyoshi Ozawa
1b5bf58001 YARN-2801. Add documentation for node labels feature. Contributed by Wangda Tan and Naganarasimha G R. 
(cherry picked from commit faa38e1aa49907254bf981662a8aeb5dc52e75e3)
 2015-09-01 17:56:08 +09:00
Jian He
f44ed4f4b0 YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. Contributed by Xuan Gong 
(cherry picked from commit a3fd2ccc869dfc1f04d1cf0a8678d4d90a43a80f)
(cherry picked from commit 48f5161cd5d4c2f4e385b253a5bea066b2e23b9e)
 2015-08-31 17:46:27 -07:00
Vinod Kumar Vavilapalli
dce4ef20f8 YARN-2945. Fixing the CHANGES.txt to have the right JIRA number. 
(cherry picked from commit cf831565e8344523e1bd0eaf686ed56a2b48b920)
 2015-08-30 20:03:28 -07:00
Vinod Kumar Vavilapalli
f333e367e1 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 
(cherry picked from commit 9da81a395584cb1dbfdee012222fd4de53522681)
 2015-08-20 17:39:43 -07:00
Vinod Kumar Vavilapalli
9faee040f9 Creating 2.6.2 entries in CHANGES.txt files. 
(cherry picked from commit 33474939a048ac7bbe07df772f0000c84979722f)
 2015-08-20 17:39:29 -07:00
Zhihai Xu
7409ed783e YARN-3857: Memory leak in ResourceManager with SIMPLE mode. Contributed by mujunchao. 2015-08-18 12:08:20 -07:00
Jian He
de1c4121a1 YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe 
(cherry picked from commit 7a445fcfabcf9c6aae219051f65d3f6cb8feb87c)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-08-13 16:15:44 -07:00
Xuan
617e294a63 YARN-3999. RM hangs on draing events. Contributed by Jian He 2015-08-11 18:27:23 -07:00
Vinod Kumar Vavilapalli
67ef0989bc Adding release 2.6.1 to CHANGES.txt 2015-08-11 13:25:36 -07:00
Jonathan Eagles
dcffe43419 YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 
(cherry picked from commit 3cd02b95224e9d43fd63a4ef9ac5c44f113f710d)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
 2015-08-03 10:53:43 -05:00
Jason Lowe
e1ae722a7d YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 
(cherry picked from commit 32e490b6c035487e99df30ce80366446fe09bd6c)
 2015-07-31 17:39:32 +00:00
Jason Lowe
e8410c0175 YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 
(cherry picked from commit ff9c13e0a739bb13115167dc661b6a16b2ed2c04)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
 2015-07-24 22:27:52 +00:00
carlo curino
b465c5b907 YARN-3969. Updating CHANGES.txt to reflect the correct set of branches where this is committed 
(cherry picked from commit fc42fa8ae3bc9d6d055090a7bb5e6f0c5972fcff)
(cherry picked from commit bddc48ed862bd783ce0907aa9c7c9585a5c48481)
 2015-07-24 13:42:13 -07:00
Xuan
c65b6b760f YARN-3967. Fetch the application report from the AHS if the RM does not 
know about it. Contributed by Mit Desai

(cherry picked from commit fbd6063269221ec25834684477f434e19f0b66af)
(cherry picked from commit 98ed4bca3be0ae635e0dfa1a9945c0f663c48145)
 2015-07-24 10:17:51 -07:00
carlo curino
76339bfb9f YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino) 
(cherry picked from commit 0fcb4a8cf2add3f112907ff4e833e2f04947b53e)
(cherry picked from commit 6a71a0a230b337c8d62cd116a9742c54f00728e9)
 2015-07-24 08:54:59 -07:00
Jian He
65af0dffa5 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. Contributed by Varun Saxena 
(cherry picked from commit 393fe71771e3ac6bc0efe59d9aaf19d3576411b3)
 2015-07-21 15:06:32 -07:00
Arun Suresh
63e4ada517 YARN-3535. Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh) 
(cherry picked from commit 9b272ccae78918e7d756d84920a9322187d61eed)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/RMContainerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
 2015-07-18 13:59:19 -07:00
Jonathan Eagles
009f5fbad6 YARN-3905. Application History Server UI NPEs when accessing apps run after RM restart (Eric Payne via jeagles) 
(cherry picked from commit 7faae0e6fe027a3886d9f4e290b6a488a2c55b3a)
 2015-07-17 11:16:37 -05:00
Tsuyoshi Ozawa
d9427a16b1 YARN-3170. YARN architecture document needs updating. Contirubted by Brahma Reddy Battula. 
(cherry picked from commit edcaae44c10b7e88e68fa97afd32e4da4a9d8df7)
 2015-07-15 15:43:49 +09:00
Jian He
9aa4411325 Revert "YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha)" 
This reverts commit aa067c6aa47b4c79577096817acc00ad6421180c.
(cherry picked from commit 2466460d4cd13ad5837c044476b26e63082c1d37)
 2015-07-13 14:32:18 -07:00
Karthik Kambatla
8bb8006b71 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha) 
(cherry picked from commit aa067c6aa47b4c79577096817acc00ad6421180c)
(cherry picked from commit ccf18705f7ef88893ac8c269f172772c6c103fa0)
 2015-07-09 09:50:14 -07:00
Akira Ajisaka
148b4db7e0 YARN-3690. [JDK8] 'mvn site' fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d6325745e26d6acb0bbdb0972ce036297d66d5b0)
(cherry picked from commit d260478d3a8bc6bc7dce32733920bb4812c883bf)
 2015-07-08 15:43:21 +09:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
dcafd3ccd2 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8ca27a34074b415f85599dd48b8fc50)
 2015-07-06 16:41:38 -07:00
Wangda Tan
a6f6ba95ef YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) 2015-07-02 11:08:21 -07:00
Jason Lowe
055d9292a7 YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena 
(cherry picked from commit b5cdf78e8e6cd6c5c1fb7286207dac72be32c0d6)
 2015-07-01 21:15:30 +00:00
Vinod Kumar Vavilapalli
a36f090c55 Preparing for 2.7.2 development: mvn versions:set -DnewVersion=2.7.2 2015-06-28 16:41:12 -07:00
Vinod Kumar Vavilapalli
2e99675c72 Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5dba5858d6e9845f18c4baec16c91911d)
 2015-06-28 16:33:49 -07:00
Jason Lowe
baf9e22284 YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena 
(cherry picked from commit 40b256949ad6f6e0dbdd248f2d257b05899f4332)
 2015-06-26 15:48:34 +00:00
Jason Lowe
41d9677740 YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula 
(cherry picked from commit 8d58512d6e6d9fe93784a9de2af0056bcc316d96)
 2015-06-24 16:38:40 +00:00
Jason Lowe
37b89deccf YARN-3809. Failed to launch new attempts because ApplicationMasterLauncher's threads all hang. Contributed by Jun Gong 
(cherry picked from commit 2a20dd9b61ba3833460cbda0e8c3e8b6366fc3ab)
 2015-06-24 16:25:22 +00:00
Karthik Kambatla
6b1a156e27 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 
(cherry picked from commit 5ebf2817e58e1be8214dc1916a694a912075aa0a)
(cherry picked from commit 9656ee4ee763dc48435ba8bb3ac9b10f6b306fe3)
 2015-06-22 17:49:55 -07:00
Xuan
c0a419b134 YARN-3804. Both RM are on standBy state when kerberos user not in yarn.admin.acl. Contributed by Varun Saxena 2015-06-17 16:27:01 -07:00
Tsuyoshi Ozawa
6c527f5514 YARN-3711. Documentation of ResourceManager HA should explain configurations about listen addresses. Contributed by Masatake Iwasaki. 
(cherry picked from commit e8c514373f2d258663497a33ffb3b231d0743b57)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-06-16 10:16:15 +09:00