Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,218 Commits 326 Branches 371 Tags
Commit Graph

817 Commits

Author SHA1 Message Date
Akira Ajisaka
72bde1dad5
HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor. 
(cherry picked from commit b38a1eea8e2917989d83d169a7b5773163e6832e)
 2017-08-01 15:16:21 +09:00
Arun Suresh
7e643130f2 YARN-6870. Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker. (Brook Zhou via asuresh) 
(cherry picked from commit 890e14c02a612c772cecd5dff2411060efd418a3)
 2017-07-28 16:33:42 -07:00
Jason Lowe
e76e7b9f0b YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)
 2017-07-20 11:06:39 -05:00
Arun Suresh
0101973dbf YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 2017-07-14 13:57:22 -07:00
Jason Lowe
a77fb561ef YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe 
(cherry picked from commit ebc048cc055d0f7d1b85bc0b6f56cd15673e837d)
 2017-07-13 17:49:55 -05:00
Jason Lowe
53c2441c10 YARN-6708. Nodemanager container crash after ext3 folder limit. Contributed by Bibin A Chundatt 
(cherry picked from commit 7576a688ea84aed7206321b1f03594e43a5f216e)
 2017-07-06 09:44:02 -05:00
Arun Suresh
66f780e6ab YARN-6127. Add support for work preserving NM restart when AMRMProxy is enabled. (Botong Huang via asuresh). 2017-06-22 15:07:38 -07:00
Daniel Templeton
c5b28c03a1 YARN-6679. Reduce Resource instance overhead via non-PBImpl 
(Contributed by Daryn Sharp via Daniel Templeton)
 2017-06-08 10:41:32 -07:00
Daniel Templeton
0bcbf2a09b YARN-6208. Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application 
(Contributed by Akira Ajisaka via Daniel Templeton)

(cherry picked from commit 73ecb19312879d54e1cbe80199fe950d81c81104)
 2017-06-02 08:52:27 -07:00
Varun Vasudev
1eecde3355 YARN-6366. Refactor the NodeManager DeletionService to support additional DeletionTask types. Contributed by Shane Kumpf. 
(cherry picked from commit 547f18cb96aeda55cc19b38be2be4d631b3a5f4f)
 2017-05-31 16:16:33 +05:30
Jason Lowe
3c0eb9c415 YARN-6641. Non-public resource localization on a bad disk causes subsequent containers failure. Contributed by Kuhu Shukla 
(cherry picked from commit aea42930bbb9566ea6988f684dbd72a72a2bdadf)
 2017-05-26 09:39:22 -05:00
Varun Vasudev
1a6c532301 Addendum patch to fix Docker sanitization. 
(cherry picked from commit 2ff2a1f50e8c7c0f33676b010b256d6c8daf912d)
 2017-05-25 14:54:08 +05:30
Akira Ajisaka
ca1c0cbc62
YARN-6141. ppc64le on Linux doesn't trigger __linux get_executable codepath. Contributed by Sonia Garudi and Ayappan. 
(cherry picked from commit bc28da65fb1c67904aa3cefd7273cb7423521014)
 2017-05-25 17:07:50 +09:00
Eric Payne
46c54bf200 YARN-6618. TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once. Contributed by Jason Lowe. 
(cherry picked from commit 19482e0d0990feb719678d71971aa849095982fd)
 2017-05-19 11:29:02 -05:00
Varun Vasudev
cd1a04e3c6 Sanitize arguments before launching Docker containers. 2017-05-18 11:46:03 +05:30
Varun Vasudev
603a0a316c Validate docker image name before launching container. 2017-05-18 11:31:12 +05:30
Chris Douglas
3d2afb209c YARN-6577. Remove unused ContainerLocalization classes. Contributed by ZhangBing Lin 
(cherry picked from commit b23fcc86c670aa896151a2bd8878154d7bb45d13)
 2017-05-17 20:40:20 -07:00
Jian He
82751ff0cb YARN-6306. NMClient API change for container upgrade. Contributed by Arun Suresh 2017-05-16 10:49:12 -07:00
Daniel Templeton
486a160735 YARN-5301. NM mount cpu cgroups failed on some systems 
(Contributed by Miklos Szegedi via Daniel Templeton)

(cherry picked from commit a2f680493f040704e2b85108e286731ee3860a52)
 2017-05-09 12:56:28 -07:00
Daniel Templeton
920a3078f6 YARN-6302. Fail the node if Linux Container Executor is not configured properly 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-05-09 10:08:07 -07:00
Subru Krishnan
d8f0d409b9 YARN-6281. Cleanup when AMRMProxy fails to initialize a new interceptor chain. (Botong Huang via Subru) 
(cherry picked from commit 57a9afbd45b7ef8e6021cc58f96bd0074bf1389d)
(cherry picked from commit 749e5c09b9990590c282ea944e24735b795351fc)
 2017-05-08 16:58:42 -07:00
Subru Krishnan
7511cfb842 YARN-6234. Support multiple attempts on the node when AMRMProxy is enabled. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit cd9ff27ffc9369820d0c39200a11bf00e6a767c8)
 2017-05-08 16:42:15 -07:00
Jason Lowe
6ca0c134e4 YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R 
(cherry picked from commit 424887ecb7d11a72837f2757ed3ff9e0fe8c5b5d)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManagerRecovery.java
 2017-05-08 17:25:11 -05:00
Subru Krishnan
36d97e7aa8 YARN-6565. Fix memory leak and finish app trigger in AMRMProxy. (Botong Huang via Subru). 
(cherry picked from commit d6eed5acca222a281401c1d524824f0460ea7826)
 2017-05-05 16:28:51 -07:00
Sidharta S
6186d2e742 YARN-6374. Improve test coverage and add utility classes for common Docker operations. Contributed by Shane Kumpf 2017-05-04 14:41:04 -07:00
Haibo Chen
49a2f286f7 YARN-6500. Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler. (Miklos Szegedi via Haibo Chen) 
(cherry picked from commit 8ac50e1322cb3f84bd998635924d85846aa47c94)
 2017-04-24 11:39:55 -07:00
Karthik Kambatla
53da65ef8b YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha) 
(cherry picked from commit 8a1d7480f73906d8e0342690ec6c6b008d6de21b)
 2017-04-14 15:07:54 -07:00
Eric Payne
607295d21f YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 
(cherry picked from commit a16ab2be91e29f0aa4ad3488d35977cb5d1e1f5c)
 2017-04-12 10:26:31 -05:00
Daniel Templeton
c1ae753de1 YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit e486569197e10bf48731d9e7ef5c031987c26acc)
 2017-04-10 14:57:19 -07:00
Robert Kanter
cbb1c76195 YARN-6368. Decommissioning an NM results in a -1 exit code (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 63f7322522e0ab223ceb91440636eb62ca0a3e41)
 2017-04-07 14:28:39 -07:00
Ming Ma
db5b4c292b YARN-5797. Add metrics to the node manager for cleaning the PUBLIC and PRIVATE caches. (Chris Trezzo via mingma) 2017-04-06 17:08:59 -07:00
Jason Lowe
84b7f2e956 YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 
(cherry picked from commit 1b081ca27e05e97d8b7d284ca24200d43763e481)
 2017-04-06 16:25:41 -05:00
Jason Lowe
903278559a YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang 
(cherry picked from commit e8071aa249c7b21b1de084ee5a9ca2a44efd3bf0)
 2017-04-05 12:31:48 -05:00
Ming Ma
7507ccd38a YARN-6004. Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines. (Chris Trezzo via mingma) 2017-04-04 18:05:09 -07:00
Akira Ajisaka
554bc8342b
YARN-6329. Remove unnecessary TODO comment from AppLogAggregatorImpl.java. Contributed by victor bertschinger. 
(cherry picked from commit 82fb9ce8df7a534a4cbcec624d7b6c0b33e79123)
 2017-03-29 10:37:52 +09:00
Jason Lowe
75e6b7428e YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi 
(cherry picked from commit 4a8e3045027036afebbcb80f23b7a2886e56c255)
 2017-03-17 14:07:48 -05:00
Jason Lowe
a16ba4296e YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee 2017-03-16 09:31:20 -05:00
Junping Du
f254002f1d YARN-6314. Potential infinite redirection on YARN log redirection web service. Contributed by Xuan Gong. 
(cherry picked from commit 5a9dda796f0e73060ada794ad5752cc6a237ab2e)
 2017-03-14 02:58:44 -07:00
Sidharta S
3b6e5ef917 YARN-5669. Add support for docker pull command (Contribtued by luhuichun) 2017-03-09 16:23:21 -08:00
Robert Kanter
11d4cdc3a6 YARN-6297. TestAppLogAggregatorImp.verifyFilesUploaded() should check # of filed uploaded with that of files expected (haibochen via rkanter) 
(cherry picked from commit 287ba4ffa66212c02e1b1edc8fca53f6368a9efc)
 2017-03-08 10:45:59 -08:00
Junping Du
2aa6f317f6 YARN-6174. Log files pattern should be same for both running and finished container. Contributed by Xuan Gong. 
(cherry picked from commit ce2d5bfa5f84e7e563980796549b56ef1e4bbf1e)
 2017-02-15 09:08:54 -08:00
Junping Du
205fc1f10c YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong. 
(cherry picked from commit 464ff479ceec76609dca3539081de6b503b17325)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestNMWebServices.java
 2017-02-13 06:16:04 -08:00
Junping Du
ab41da7e64 YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong. 
(cherry picked from commit 9dbfab1284015af6983b4a55b79c9e65bc0194e5)
 2017-02-07 02:14:09 -08:00
Junping Du
c699ce70ea YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong. 2017-02-02 00:45:27 -08:00
Jason Lowe
4703f5d20b YARN-5641. Localizer leaves behind tarballs after container is complete. Contributed by Eric Badger 2017-01-27 15:25:57 +00:00
Jason Lowe
b806201f0a Revert "YARN-5641. Localizer leaves behind tarballs after container is complete. Contributed by Eric Badger" 
This reverts commit 45407acaed05284ad897d04f4575ab0a4ee7a88b.
 2017-01-26 14:42:22 +00:00
Jason Lowe
45407acaed YARN-5641. Localizer leaves behind tarballs after container is complete. Contributed by Eric Badger 
(cherry picked from commit 9e19f758c1950cbcfcd1969461a8a910efca0767)
 2017-01-25 21:54:17 +00:00
Junping Du
8027c3e8b9 YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong. 2017-01-24 15:28:52 -08:00
Jason Lowe
1672a06135 YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S 2017-01-24 16:19:02 +00:00
Chris Nauroth
5251de00fa YARN-5456. container-executor support for FreeBSD, NetBSD, and others if conf path is absolute. Contributed by Allen Wittenauer. 
(cherry picked from commit b913677365ad77ca7daa5741c04c14df1a0313cd)
 2017-01-19 20:48:27 +05:30