Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,527 Commits 326 Branches 371 Tags
Commit Graph

738 Commits

Author SHA1 Message Date
Junping Du
b906bddd95 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 
(cherry picked from commit ad45d19998c1b0da25754d0016854046731fa623)
 2017-08-29 15:17:21 -07:00
Yufei Gu
221c1bdf7e YARN-7099. ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. (Contributed by Miklos Szegedi via Yufei Gu) 
(cherry picked from commit d5bc9582793de6ab2d2e489cde293bf076229a1c)
 2017-08-28 10:11:01 -07:00
Li Lu
b7f319c53c YARN-6999. Add log about how to solve Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster. (Linlin Zhou via gtcarrera9) 
(cherry picked from commit 3bee7046c52bff681a5aecc956ba9be6c3543397)
 2017-08-25 22:45:20 -07:00
Eric Payne
1617744801 YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 
(cherry picked from commit e864f81471407a384395fefe1ceb3b66fc7f87f2)
 2017-08-25 15:52:29 -05:00
Junping Du
e12358c5ce YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 
(cherry picked from commit c2cb7ea1ef6532020b69031dbd18b0f9b8369f0f)
 2017-08-24 15:27:40 -07:00
Akira Ajisaka
bf903396a0
YARN-7047. Moving logging APIs over to slf4j in hadoop-yarn-server-nodemanager. Contributed by Yeliang Cang. 2017-08-25 03:28:02 +09:00
bibinchundatt
cc50ca0722 YARN-7074. Fix NM state store update comment. Contributed by Botong Huang 
(cherry picked from commit de0cba700bcf4276726c0aa9df8d738787debc17)
 2017-08-24 16:40:18 +05:30
Ray Chiang
2299c8dffa YARN-6798. Fix NM startup failure with old state store due to version mismatch. (Botong Huang via rchiang) 
(cherry picked from commit f5f14a2ad67f91064a73685b44369c6314f0e1cd)
 2017-08-22 15:24:02 -07:00
Jason Lowe
b15a393440 YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger 2017-08-22 16:28:08 -05:00
Arun Suresh
e0ba8a9ed8 YARN-6979. [Addendum patch] Fixed classname and added javadocs. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 7a82d7bcea8124e1b65c275fac15bf2047d17471)
 2017-08-20 10:28:42 -07:00
Arun Suresh
f8f193c8e8 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 8410d862d3a72740f461ef91dddb5325955e1ca5)
 2017-08-20 08:31:17 -07:00
Sunil G
5511cea86b YARN-3254. HealthReport should include disk full information. Contributed by Suma Shivaprasad. 
(cherry picked from commit f9a0e2338150f1bd3ba2c29f76979183fd3ed80c)
 2017-08-17 15:10:07 +05:30
Arun Suresh
7b22df3da6 YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 4d7be1d8575e9254c59d41460960708e3718503a)
 2017-08-14 19:57:34 -07:00
Karthik Kambatla
f6c74ac5ea YARN-5927. BaseContainerManagerTest::waitForNMContainerState timeout accounting is not accurate. (Kai Sasaki via kasha) 
(cherry picked from commit 8c4b6d16a526610a03ccc85665744ad071e37400)
 2017-08-11 12:18:29 -07:00
Jian He
8b7590057e YARN-6804. Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi 2017-08-09 11:12:38 -07:00
Daniel Templeton
675d368950 YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path 
(Contributed by Miklos Szegedi via Daniel Templeton)

(cherry picked from commit 47b145b9b4e81d781891abce8a6638f0b436acc4)
 2017-08-08 11:12:34 -07:00
Arun Suresh
3f50800d41 YARN-6920. Fix resource leak that happens during container re-initialization. (asuresh) 
(cherry picked from commit 8d3fd81980275fa81e7a5539b1751f38a63b6911)
 2017-08-07 19:01:13 -07:00
Sunil G
d15bac141e YARN-6951. Fix debug log when Resource Handler chain is enabled. Contributed by Yang Wang. 
(cherry picked from commit 46b7054fa7eae9129c21c9f3dc70acff46bfdc41)
 2017-08-07 13:17:04 +05:30
Arun Suresh
88d6adf419 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 35dc7829236f92054d5ce6ea78d3a44ca6c8f3d3)
 2017-08-03 21:18:43 -07:00
Yufei Gu
ce609d06c0 YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 13:52:58 -07:00
Eric Payne
686bdc48b8 YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe. 
(cherry picked from commit 48899134d2a77935a821072b5388ab1b1b7b399c)
 2017-08-02 11:29:56 -05:00
Akira Ajisaka
72bde1dad5
HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor. 
(cherry picked from commit b38a1eea8e2917989d83d169a7b5773163e6832e)
 2017-08-01 15:16:21 +09:00
Arun Suresh
7e643130f2 YARN-6870. Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker. (Brook Zhou via asuresh) 
(cherry picked from commit 890e14c02a612c772cecd5dff2411060efd418a3)
 2017-07-28 16:33:42 -07:00
Jason Lowe
e76e7b9f0b YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)
 2017-07-20 11:06:39 -05:00
Arun Suresh
0101973dbf YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 2017-07-14 13:57:22 -07:00
Jason Lowe
a77fb561ef YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe 
(cherry picked from commit ebc048cc055d0f7d1b85bc0b6f56cd15673e837d)
 2017-07-13 17:49:55 -05:00
Jason Lowe
53c2441c10 YARN-6708. Nodemanager container crash after ext3 folder limit. Contributed by Bibin A Chundatt 
(cherry picked from commit 7576a688ea84aed7206321b1f03594e43a5f216e)
 2017-07-06 09:44:02 -05:00
Arun Suresh
66f780e6ab YARN-6127. Add support for work preserving NM restart when AMRMProxy is enabled. (Botong Huang via asuresh). 2017-06-22 15:07:38 -07:00
Daniel Templeton
c5b28c03a1 YARN-6679. Reduce Resource instance overhead via non-PBImpl 
(Contributed by Daryn Sharp via Daniel Templeton)
 2017-06-08 10:41:32 -07:00
Daniel Templeton
0bcbf2a09b YARN-6208. Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application 
(Contributed by Akira Ajisaka via Daniel Templeton)

(cherry picked from commit 73ecb19312879d54e1cbe80199fe950d81c81104)
 2017-06-02 08:52:27 -07:00
Varun Vasudev
1eecde3355 YARN-6366. Refactor the NodeManager DeletionService to support additional DeletionTask types. Contributed by Shane Kumpf. 
(cherry picked from commit 547f18cb96aeda55cc19b38be2be4d631b3a5f4f)
 2017-05-31 16:16:33 +05:30
Jason Lowe
3c0eb9c415 YARN-6641. Non-public resource localization on a bad disk causes subsequent containers failure. Contributed by Kuhu Shukla 
(cherry picked from commit aea42930bbb9566ea6988f684dbd72a72a2bdadf)
 2017-05-26 09:39:22 -05:00
Varun Vasudev
1a6c532301 Addendum patch to fix Docker sanitization. 
(cherry picked from commit 2ff2a1f50e8c7c0f33676b010b256d6c8daf912d)
 2017-05-25 14:54:08 +05:30
Akira Ajisaka
ca1c0cbc62
YARN-6141. ppc64le on Linux doesn't trigger __linux get_executable codepath. Contributed by Sonia Garudi and Ayappan. 
(cherry picked from commit bc28da65fb1c67904aa3cefd7273cb7423521014)
 2017-05-25 17:07:50 +09:00
Eric Payne
46c54bf200 YARN-6618. TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once. Contributed by Jason Lowe. 
(cherry picked from commit 19482e0d0990feb719678d71971aa849095982fd)
 2017-05-19 11:29:02 -05:00
Varun Vasudev
cd1a04e3c6 Sanitize arguments before launching Docker containers. 2017-05-18 11:46:03 +05:30
Varun Vasudev
603a0a316c Validate docker image name before launching container. 2017-05-18 11:31:12 +05:30
Chris Douglas
3d2afb209c YARN-6577. Remove unused ContainerLocalization classes. Contributed by ZhangBing Lin 
(cherry picked from commit b23fcc86c670aa896151a2bd8878154d7bb45d13)
 2017-05-17 20:40:20 -07:00
Jian He
82751ff0cb YARN-6306. NMClient API change for container upgrade. Contributed by Arun Suresh 2017-05-16 10:49:12 -07:00
Daniel Templeton
486a160735 YARN-5301. NM mount cpu cgroups failed on some systems 
(Contributed by Miklos Szegedi via Daniel Templeton)

(cherry picked from commit a2f680493f040704e2b85108e286731ee3860a52)
 2017-05-09 12:56:28 -07:00
Daniel Templeton
920a3078f6 YARN-6302. Fail the node if Linux Container Executor is not configured properly 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-05-09 10:08:07 -07:00
Subru Krishnan
d8f0d409b9 YARN-6281. Cleanup when AMRMProxy fails to initialize a new interceptor chain. (Botong Huang via Subru) 
(cherry picked from commit 57a9afbd45b7ef8e6021cc58f96bd0074bf1389d)
(cherry picked from commit 749e5c09b9990590c282ea944e24735b795351fc)
 2017-05-08 16:58:42 -07:00
Subru Krishnan
7511cfb842 YARN-6234. Support multiple attempts on the node when AMRMProxy is enabled. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit cd9ff27ffc9369820d0c39200a11bf00e6a767c8)
 2017-05-08 16:42:15 -07:00
Jason Lowe
6ca0c134e4 YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R 
(cherry picked from commit 424887ecb7d11a72837f2757ed3ff9e0fe8c5b5d)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManagerRecovery.java
 2017-05-08 17:25:11 -05:00
Subru Krishnan
36d97e7aa8 YARN-6565. Fix memory leak and finish app trigger in AMRMProxy. (Botong Huang via Subru). 
(cherry picked from commit d6eed5acca222a281401c1d524824f0460ea7826)
 2017-05-05 16:28:51 -07:00
Sidharta S
6186d2e742 YARN-6374. Improve test coverage and add utility classes for common Docker operations. Contributed by Shane Kumpf 2017-05-04 14:41:04 -07:00
Haibo Chen
49a2f286f7 YARN-6500. Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler. (Miklos Szegedi via Haibo Chen) 
(cherry picked from commit 8ac50e1322cb3f84bd998635924d85846aa47c94)
 2017-04-24 11:39:55 -07:00
Karthik Kambatla
53da65ef8b YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha) 
(cherry picked from commit 8a1d7480f73906d8e0342690ec6c6b008d6de21b)
 2017-04-14 15:07:54 -07:00
Eric Payne
607295d21f YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 
(cherry picked from commit a16ab2be91e29f0aa4ad3488d35977cb5d1e1f5c)
 2017-04-12 10:26:31 -05:00
Daniel Templeton
c1ae753de1 YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit e486569197e10bf48731d9e7ef5c031987c26acc)
 2017-04-10 14:57:19 -07:00