Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,948 Commits 326 Branches 371 Tags
Commit Graph

1630 Commits

Author SHA1 Message Date
Jason Lowe
8c8b848b95 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1c6d9fc78cd5013c860c1086d86176f)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-12 16:00:06 +00:00
Jason Lowe
9668c965f5 YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti 
(cherry picked from commit 62e583c7dcbb30d95d8b32a4978fbdb3b98d67cc)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-12 15:28:31 +00:00
Jason Lowe
d43a651374 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 
(cherry picked from commit 5ebf2817e58e1be8214dc1916a694a912075aa0a)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-12 15:16:08 +00:00
Rohith Sharma K S
58a6142c14 YARN-3849. Too much of preemption activity causing continuos killing of containers across queues. (Sunil G via wangda) 2016-01-11 12:02:38 +05:30
Anubhav Dhoot
6b2abb7515 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 
(cherry picked from commit 9735afe967a660f356e953348cb6c34417f41055)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java

(cherry picked from commit 22f2501476d987afb7bc19080a7a0db94ea72be6)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java

(cherry picked from commit 7c9a368b45b0e38173521a94ab32dee8a2984bf8)
 2016-01-07 15:08:05 -08:00
Tsuyoshi Ozawa
4be18edf2f YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83af6a48a8d8d0e37cdf1f813124f366)
(cherry picked from commit d76b523b020fcd76e9abdd661967ca4a931d9863)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-06 14:34:15 -08:00
Junping Du
cfbb035420 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b6115e2cfbd97a94d6c1a3ddf0e94bb)
(cherry picked from commit 1d107d805841e9b39db01aa236068a0ff1ca46d1)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
 2016-01-06 14:15:55 -08:00
Junping Du
9da7b1fdd2 YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena. 2016-01-06 13:50:35 -08:00
Junping Du
74027c24c8 Addendum patch to fix build after porting YARN-4546. 2016-01-06 06:11:00 -08:00
Junping Du
c0ffe25a65 YARN-4546. ResourceManager crash due to scheduling opportunity overflow. Contributed by Jason Lowe. 
(cherry picked from commit c1462a67ff7bb632df50e1c52de971cced56c6a3)
(cherry picked from commit 1cc001db4c3767072b5d065d161bc5c6d1c480d4)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
 2016-01-06 05:56:16 -08:00
Sangjin Lee
51a2e6304a Preparing for 2.6.4 development 2016-01-05 15:40:31 -08:00
Karthik Kambatla
62e032a4da YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha) 
(cherry picked from commit 332b520a480994b7bd56c135f7941aad30b05e9c)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
 2016-01-05 00:30:37 -08:00
Akira Ajisaka
0381556a42 Set release date for 2.6.3 
(cherry picked from commit db99e30f670cb5d73d5ec79671026eeeaf337d3d)
(cherry picked from commit 247ab92464beb55bad58b6e63eeac1a41767f432)
 2016-01-04 16:18:38 +09:00
Rohith Sharma K S
7af87093ad YARN-3893. Both RM in active state when Admin#transitionToActive failure from refeshAll() (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 7d6687fe76f6152a577ff2298c358dd30fce41fb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2016-01-04 11:44:28 +05:30
Junping Du
0f9dd48842 YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 
(cherry picked from commit 50bd067e1d63d4c80dc1e7bf4024bfaf42cf4416)
(cherry picked from commit 1a2ef845b54166194b133e524ad9533cc259aed2)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java
 2015-12-17 05:53:56 -08:00
Zhihai Xu
9d9e0eaae8 Update CHANGES.txt to move MAPREDUCE-6436 from YARN to MAPREDUCE 
(cherry picked from commit 7092d47fc0b3b792dd31f967c01d460dc089f60b)
 2015-12-15 21:53:22 -08:00
Zhihai Xu
2d125f2bee MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d06921893200163a3d29c8901c3c0107cb)
 2015-12-15 01:05:49 -08:00
Arun Suresh
c53d45a687 YARN-3535. Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh) 
(cherry picked from commit 9b272ccae78918e7d756d84920a9322187d61eed)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/RMContainerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/event/SchedulerEventType.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
 2015-12-14 23:50:55 -08:00
Zhihai Xu
70289432f7 YARN-3857: Memory leak in ResourceManager with SIMPLE mode. Contributed by mujunchao. 
(cherry picked from commit 3a76a010b85176f2bcb85ed6f74c25dcb8acfe4d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-14 22:17:23 -08:00
Karthik Kambatla
843dac5353 YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha) 
(cherry picked from commit 2abec14ec6e7d6d8b7e59239ed2596a15adc8475)
 2015-12-09 10:53:26 -08:00
Akira Ajisaka
80037436c6 YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang. 
(cherry picked from commit 16dc65811d4e0e7ab61c61295149848fe53ddd89)
(cherry picked from commit 925bb94d87db7e916ed380d4512225e796f9e03d)
 2015-12-09 16:30:39 +09:00
Wangda Tan
5b063d6b7f YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 
(cherry picked from commit 7e4715186d31ac889fba26d453feedcebb11fc70)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 7013f9d6cda88e72a839b1c55757615b55101beb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-08 14:34:53 -08:00
Tsuyoshi Ozawa
b345ffd7df YARN-4348. ZKRMStateStore.syncInternal shouldn't wait for sync completion for avoiding blocking ZK's event thread. (ozawa) 
(cherry picked from commit 0460b8a8a3de232f236f49ef6769d38cda62cc28)
 2015-12-08 13:41:17 +09:00
Junping Du
e710a11a7f Creating 2.6.4 entries in CHANGES.txt files. 2015-12-01 08:28:37 -08:00
Jason Lowe
f61e3320cb YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla 
(cherry picked from commit f5acf94ecafb301a0cc8e8f91f19c8bcbc8da701)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestFileSystemNodeLabelsStore.java
 2015-11-24 23:59:40 +00:00
Sangjin Lee
a9001a210a YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha) 2015-11-23 14:14:50 -08:00
Jason Lowe
271875a426 YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 
(cherry picked from commit ff9c13e0a739bb13115167dc661b6a16b2ed2c04)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
(cherry picked from commit e8410c0175518a7ca4f44921e950050854895563)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-11-23 20:57:57 +00:00
Jason Lowe
5f05e5e5ba YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 2015-11-23 20:27:00 +00:00
Xuan
11c2326acb YARN-2859-addendum: fix the remaining issue from the previous patch. 2015-11-19 10:29:58 -08:00
Akira Ajisaka
6c178cc321 YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas. 
(cherry picked from commit 23d0db551cc63def9acbab2473e58fb1c52f85e0)
(cherry picked from commit f6f128c8676981a33ccc55c92321244391f9a3ac)
 2015-11-11 16:18:18 +09:00
Wangda Tan
fb7be09f20 YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda) 
(cherry picked from commit 0783184f4b3f669f7211e42b395b62d63144100d)
 2015-11-06 09:10:12 -08:00
Tsuyoshi Ozawa
6b27de0f36 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739512804da38cf87e0ce1059e3128da3)
 2015-11-06 00:19:41 -08:00
Tsuyoshi Ozawa
5a00b23106 YARN-4312. TestSubmitApplicationWithRMHA fails on branch-2.7 and branch-2.6 as some of the test cases time out. Contributed by Varun Saxena. 
(cherry picked from commit 66364419118c64c9d1c623f808f027ac45688759)
 2015-11-05 10:40:23 -08:00
Sangjin Lee
8f8f3b1027 Updated the 2.6.2 final release date. 
(cherry picked from commit a4a6b5b4b470b1e7a3c5e2d38433429c455bc709)
 2015-10-30 18:51:50 -07:00
Sangjin Lee
ccc0c54501 YARN-2859 addendum. Re-added the entry to hadoop-yarn/CHANGES.txt under 2.6.3. 2015-10-30 14:46:04 -07:00
Jason Lowe
0af4f222a1 Creating 2.6.3 entries in CHANGES.txt files. 
(cherry picked from commit eadf7b3096cb010eb7f0afd9afd4ae0d67b2645f)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-10-30 14:58:42 +00:00
Sangjin Lee
1d3bed1f94 Set the release date for 2.6.2 2015-10-28 20:57:16 -07:00
Xuan
9a97ff54e5 YARN-2859. ApplicationHistoryServer binds to default port 8188 in 
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli

(cherry picked from commit 27414dac66f278b61fc23762204b01a1c508178a)
(cherry picked from commit 9ce5069d16edefb0b2a24d6b7988347039624942)
(cherry picked from commit 336be63dad59c58d7a72d2ea2c4a9f811a075eae)
 2015-10-28 10:55:31 -07:00
Sangjin Lee
6466ead9e0 Preparing for 2.6.3 development 2015-10-21 10:58:06 -07:00
Tsuyoshi Ozawa
b898f8014f YARN-3798. ZKRMStateStore shouldn't create new session without occurrance of SESSIONEXPIED. (ozawa and Varun Saxena) 2015-10-21 23:08:02 +09:00
Jason Lowe
ac865de725 YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks) 
(cherry picked from commit feaf0349949e831ce3f25814c1bbff52f17bfe8f)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-10-08 16:39:46 +00:00
Jason Lowe
528b809d2d YARN-3194. RM should handle NMContainerStatuses sent by NM while registering if NM is Reconnected node. Contributed by Rohith 
(cherry picked from commit a64dd3d24bfcb9af21eb63869924f6482b147fd3)
 2015-10-08 16:33:34 +00:00
Jason Lowe
1484ebb602 YARN-3802. Two RMNodes for the same NodeId are used in RM sometimes 
after NM is reconnected. Contributed by zhihai xu
(cherry picked from commit 5b5bb8dcdc888ba1ebc7e4eba0fa0e7e79edda9a)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-10-08 16:01:20 +00:00
Jason Lowe
4770f190b8 YARN-3780. Should use equals when compare Resource in 
RMNodeImpl#ReconnectNodeTransition. Contributed by zhihai xu.
(cherry picked from commit c7ee6c151c5771043a6de3b8a951cea13f59dd7b)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-10-08 15:37:08 +00:00
Jason Lowe
2ecd173426 YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong 
(cherry picked from commit 38aed1a94ed7b6da62e2445b5610bc02b1cddeeb)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-10-08 15:16:11 +00:00
Jason Lowe
49335d9b2b YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu 
(cherry picked from commit 854d25b0c30fd40f640c052e79a8747741492042)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
(cherry picked from commit 493f07200837969d6e4d6af2cccc732e60f119ba)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-09-30 16:12:09 +00:00
Jason Lowe
c4cbbcbc26 YARN-3554. Default value for maximum nodemanager connect wait time is too high. Contributed by Naganarasimha G R 
(cherry picked from commit 9757864fd662b69445e0c600aedbe307a264982e)
(cherry picked from commit 1b95bf9e1be29f51c7272cfb9f31dda71f2dfae7)
 2015-09-28 10:48:40 -07:00
Xuan
1828ba00be YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when state-store error occurs. Contributed by Jian He 
(cherry picked from commit 9f7fcb54e798cf4fda1ea7972dd96491976e1857)
 2015-09-25 16:43:06 -07:00
Xuan
d27f09c936 YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He. 
(cherry picked from commit db57d91ac91e895bcb9a23fa50af0b2fbcb1db5a)
 2015-09-25 16:30:49 -07:00
Jian He
c09bb46579 YARN-4101. RM should print alert messages if Zookeeper and Resourcemanager gets connection issue. Contributed by Xuan Gong 
(cherry picked from commit 214fd1408c21f596d1d15217c11b58b34561aab7)
 2015-09-25 16:26:18 -07:00