Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,939 Commits 326 Branches 371 Tags
Commit Graph

498 Commits

Author SHA1 Message Date
Jason Lowe
3e3733437f YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 
(cherry picked from commit fdf02d1f26cea372bf69e071f57b8bfc09c092c4)
 2015-10-02 20:21:32 +00:00
Jason Lowe
d645ee1d62 YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu 
(cherry picked from commit 854d25b0c30fd40f640c052e79a8747741492042)
 2015-09-30 15:01:01 +00:00
Jian He
d592aa115b YARN-3868. Recovery support for container resizing. Contributed by Meng Ding 
(cherry picked from commit c57eac5dfe277845ab4522a1188023a73ee41539)
 2015-09-24 10:24:13 -07:00
Jian He
b6c594c6ea YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding 
(cherry picked from commit c3dc1af072574f5890a8d43e4d60526951b4b8bc)
 2015-09-24 10:24:04 -07:00
Jian He
b8955d81c5 YARN-1643. Make ContainersMonitor support changing monitoring size of an allocated container. Contributed by Meng Ding and Wangda Tan 
(cherry picked from commit c59ae4eeb17e52e8fc659b9962d20628719fc621)
 2015-09-24 10:23:55 -07:00
Jian He
fa0a554ae0 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit 5f5a968d65c44a831176764439e00db9203999ed)
 2015-09-24 10:23:46 -07:00
Jian He
afe4afd0c9 YARN-1645. ContainerManager implementation to support container resizing. Contributed by Meng Ding & Wangda Tan 
(cherry picked from commit ffd820c27a4f8cf4676ad8758696ed89fde80218)
 2015-09-24 10:23:35 -07:00
Jian He
c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10ee937a04e833a66e0a4642e776e510)
 2015-09-24 10:23:25 -07:00
Rohith Sharma K S
ace4d26936 YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for absent container. (Bibin A Chundatt via rohithsharmaks) 2015-09-24 11:31:47 +05:30
Jason Lowe
c829be5133 YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator between ShuffleHandler and LocalDirsHandlerService. Contributed by Zhihai Xu 
(cherry picked from commit c890c51a916894a985439497b8a44e8eee82d762)
 2015-09-23 15:44:13 +00:00
Xuan
f3c0a210b6 YARN-4149. yarn logs -am should provide an option to fetch all the log 
files. Contributed by Varun Vasudev

(cherry picked from commit 34ef1a092bcab369bb845481efffb8c47adef53a)
 2015-09-15 14:37:59 -07:00
Jason Lowe
181bda0421 YARN-4158. Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers. Contributed by Zhihai Xu 
(cherry picked from commit 8c1cdb17a03fec91bfab172fe92b67bdeb9ebfba)
 2015-09-15 20:22:23 +00:00
Varun Vasudev
a4288f1262 YARN-4136. LinuxContainerExecutor loses info when forwarding ResourceHandlerException. Contributed by Bibin A Chundatt. 
(cherry picked from commit 486d5cb803efec7b4db445ee65a3df83392940a3)
 2015-09-11 14:39:01 +05:30
Wangda Tan
5decd8f9f7 YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) 
(cherry picked from commit 77666105b4557d5706e5844a4ca286917d966c5f)
 2015-09-10 09:30:34 -07:00
Zhihai Xu
214338bd68 YARN-4096. App local logs are leaked if log aggregation fails to initialize for the app. Contributed by Jason Lowe. 
(cherry picked from commit 16b9037dc1300b8bdbe54ba7cd47c53fe16e93d8)
 2015-09-08 12:33:29 -07:00
Jian He
f23be93dd1 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 
(cherry picked from commit 6f72f1e6003ab11679bebeb96f27f1f62b3b3e02)
 2015-09-08 11:24:07 +08:00
Varun Vasudev
70575286b7 YARN-3591. Resource localization on a bad disk causes subsequent containers failure. Contributed by Lavkesh Lahngir. 
(cherry picked from commit 1dbd8e34a7d97c4d8586da79c980d8f2e0aad61d)
 2015-09-07 11:34:14 +05:30
Rohith Sharma K S
7ecdf51441 YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 095ab9ab5fa5f415fba41fee81b6008cf0455af8)
 2015-09-02 14:28:50 +05:30
Xuan
85363ea4bd YARN-221. Addendum patch to compilation issue which is caused by missing 
AllContainerLogAggregationPolicy. Contributed by Xuan Gong

(cherry picked from commit b71c6006f579ac6f0755975a9b908b0062618b46)
 2015-08-23 16:50:02 -07:00
Xuan
205b98c059 YARN-221. NM should provide a way for AM to tell it not to aggregate 
logs. Contributed by Ming Ma

(cherry picked from commit 37e1c3d82a96d781e1c9982988b7de4aa5242d0c)
 2015-08-22 16:26:39 -07:00
Wangda Tan
6452b31760 YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) 
(cherry picked from commit fc07464d1a48b0413da5e921614430e41263fdb7)
 2015-08-20 16:52:57 -07:00
Karthik Kambatla
b567aa2b4f YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha) 
(cherry picked from commit 13604bd5f119fc81b9942190dfa366afad61bc92)
 2015-08-20 16:52:33 -07:00
Zhihai Xu
051a622b57 YARN-4057. If ContainersMonitor is not enabled, only print related log info one time. Contributed by Jun Gong. 
(cherry picked from commit 14215c8ef83d58b8443c52a3cb93e6d44fc87065)
 2015-08-18 11:47:41 -07:00
Karthik Kambatla
d2ff763533 YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha) 
(cherry picked from commit def12933b38efd5e47c5144b729c1a1496f09229)
 2015-08-16 06:26:21 -07:00
Jian He
4a5fdb7c23 YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong 
(cherry picked from commit 38aed1a94ed7b6da62e2445b5610bc02b1cddeeb)
 2015-08-13 14:46:34 -07:00
Junping Du
7af531d2e3 YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. Contributed by Robert Kanter. 
(cherry picked from commit cfee02b3bdd1117370200c9d8ce216676cff8888)
 2015-08-06 06:50:40 -07:00
Xuan
89ccfd9fcc YARN-4004. container-executor should print output of docker logs if the 
docker container exits with non-0 exit status. Contributed by Varun
Vasudev

(cherry picked from commit c3364ca8e75acfb911ab92e19f357b132f128123)
 2015-08-03 18:11:16 -07:00
Jason Lowe
b264217bc3 YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo 
(cherry picked from commit 469cfcd695da979e56c83d9303f9bc1f898c08ce)
 2015-08-03 15:55:04 +00:00
Xuan
2ec008eb38 YARN-3982. container-executor parsing of container-executor.cfg broken 
in trunk and branch-2. Contributed by Varun Vasudev

(cherry picked from commit f17093421521efcbdc813f6f2b8411e45ecc7863)
 2015-07-27 23:47:31 -07:00
Varun Vasudev
9da487e0fd YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana. 
(cherry picked from commit 3e6fce91a471b4a5099de109582e7c6417e8a822)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2015-07-27 12:03:36 -07:00
Varun Vasudev
ec0f801f52 YARN-3852. Add docker container support to container-executor. Contributed by Abin Shahab. 
(cherry picked from commit f36835ff9b878fa20fe58a30f9d1e8c47702d6d2)
 2015-07-27 10:18:06 -07:00
Jason Lowe
2cf714123a YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 
(cherry picked from commit ff9c13e0a739bb13115167dc661b6a16b2ed2c04)
 2015-07-24 22:15:44 +00:00
Wangda Tan
3800e25d6e YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda) 
(cherry picked from commit 06e5dd2c84c49460884757b56980b1b9c58af996)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-07-22 12:03:14 -07:00
Colin Patrick Mccabe
aff955a14a YARN-3844. Make hadoop-yarn-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe) 
(cherry picked from commit 419c51d233bd124eadb38ff013693576ec02c4f1)
 2015-07-17 12:05:24 -07:00
Akira Ajisaka
0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d90e26616accee73b1f7743aab5df692)
 2015-07-13 17:53:27 +09:00
Zhijie Shen
37a93c2d78 YARN-3116. RM notifies NM whether a container is an AM container or normal task container. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 1ea36299a47af302379ae0750b571ec021eb54ad)
 2015-07-10 19:02:13 -07:00
Karthik Kambatla
d15f69457b YARN-1012. Report NM aggregated container resource utilization in heartbeat. (Inigo Goiri via kasha) 
(cherry picked from commit 527c40e4d664c721b8f32d7cd8df21b2666fea8a)
 2015-07-09 09:35:39 -07:00
Varun Vasudev
d17a7bbf6e YARN-2194. Addendum patch to fix failing unit test in TestPrivilegedOperationExecutor. Contributed by Sidharta Seethana. 
(cherry picked from ad5b904175a1f89951c7c32f744a4e20fafb51b5)
 2015-07-09 11:52:57 +05:30
Varun Vasudev
16ca0fcf99 YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan. 
(cherry-picked from commit 7e3ebc0fc59dbd1d7768b5a6c05af18b8c8fea4d)
 2015-07-07 17:05:06 +05:30
Jason Lowe
a721b9d812 YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena 
(cherry picked from commit b5cdf78e8e6cd6c5c1fb7286207dac72be32c0d6)
 2015-07-01 21:14:35 +00:00
Colin Patrick Mccabe
b1da58d23d YARN-3827. Migrate YARN native build to new CMake framework (Alan Burlison via Colin P. McCabe) 
(cherry picked from commit d0cc0380b57db5fdeb41775bb9ca42dac65928b8)
 2015-06-30 16:24:44 -07:00
Jian He
0100995c5e YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti 
(cherry picked from commit 62e583c7dcbb30d95d8b32a4978fbdb3b98d67cc)
 2015-06-29 13:38:04 -07:00
Jason Lowe
0221d19f4e YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena 
(cherry picked from commit 40b256949ad6f6e0dbdd248f2d257b05899f4332)
 2015-06-26 15:48:05 +00:00
Jason Lowe
15b1800b12 YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula 
(cherry picked from commit 8d58512d6e6d9fe93784a9de2af0056bcc316d96)
 2015-06-24 16:38:22 +00:00
Karthik Kambatla
9656ee4ee7 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 
(cherry picked from commit 5ebf2817e58e1be8214dc1916a694a912075aa0a)
 2015-06-22 17:48:07 -07:00
Xuan
c3254a63a9 YARN-3834. Scrub debug logging of tokens during resource localization. Contributed by Chris Nauroth 
(cherry picked from commit 6c7a9d502a633b5aca75c9798f19ce4a5729014e)
 2015-06-21 17:14:59 -07:00
Devaraj K
8c643e3bf9 YARN-3747. TestLocalDirsHandlerService should delete the created test 
directory logDir2. Contributed by David Moore.

(cherry picked from commit 126321eded7dc38c1eef2cfde9365404c924a5cb)
 2015-06-08 15:33:38 +05:30
Junping Du
d8c7ee1b53 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K. 
(cherry picked from commit d7e7f6aa03c67b6a6ccf664adcb06d90bc963e58)
 2015-06-04 05:00:32 -07:00
Jason Lowe
752caa95a4 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit e13b671aa510f553f4a6a232b4694b6a4cce88ae)
 2015-06-03 19:45:18 +00:00
Robert Kanter
bde4031a02 YARN-3713. Remove duplicate function call storeContainerDiagnostics in ContainerDiagnosticsUpdateTransition (zxu via rkanter) 
(cherry picked from commit 6aec13cb338b0fe62ca915f78aa729c9b0b86fba)
 2015-05-29 15:35:08 -07:00