Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,819 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1137 Commits

Author SHA1 Message Date
Szilard Nemeth 6980f1740f YARN-9217. Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing. Contributed by Peter Bacsko 2019-08-21 16:49:34 +02:00
Szilard Nemeth a83718f130 YARN-9100. Add tests for GpuResourceAllocator and do minor code cleanup. Contributed by Peter Bacsko 2019-08-16 15:24:44 +02:00
Szilard Nemeth 8fee3808c5 YARN-9749. TestAppLogAggregatorImpl#testDFSQuotaExceeded fails on trunk. Contributed by Adam Antal 
(cherry picked from commit 2a05e0ff3b)
 2019-08-16 08:52:34 +02:00
Adam Antal d5446b3a23 YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl… (#1261) 
* YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl and connected classes

* Using {} placeholder, and increasing loglevel if log aggregation failed.

(cherry picked from commit c89bdfacc8)
 2019-08-14 17:36:41 +02:00
Szilard Nemeth 4bb238c480 YARN-9133. Make tests more easy to comprehend in TestGpuResourceHandler. Contributed by Peter Bacsko 2019-08-14 17:16:54 +02:00
Szilard Nemeth 4dc477b606 YARN-9140. Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager. Contributed by Peter Bacsko 2019-08-14 17:01:41 +02:00
Eric Badger cec71691be YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan. 
(cherry picked from commit 2ac029b949)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2019-08-13 16:34:29 +00:00
Szilard Nemeth b20fd9e212 YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko 2019-08-12 14:02:17 +02:00
Szilard Nemeth 2e6beb1550 Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph 
(cherry picked from commit 54ac80176e)
 2019-08-09 13:20:10 +02:00
Szilard Nemeth 02d0e54596 YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak 
(cherry picked from commit e0c21c6da9)
 2019-08-09 10:23:10 +02:00
Szilard Nemeth f0dfb8b832 YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak 
(cherry picked from commit 742e30b473)
 2019-08-09 10:02:35 +02:00
Szilard Nemeth 3bcf44f070 YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak 
(cherry picked from commit 72d7e570a7)
 2019-08-09 09:50:32 +02:00
Haibo Chen 8d357343c4 YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 
(cherry picked from commit f51702d539)
 2019-08-06 14:59:49 -07:00
Jonathan Hung 15344006bc YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-22 10:46:45 -07:00
Szilard Nemeth 28d6a453a9 YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko 
(cherry picked from commit 18ee1092b4)
 2019-07-15 12:02:39 +02:00
Szilard Nemeth 2fcbdf4131 YARN-9337. Addendum to fix compilation error due to mockito spy call 
(cherry picked from commit bb37c6cb7f)
 2019-07-13 00:45:38 +02:00
Szilard Nemeth 0ede873090 YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal 
(cherry picked from commit 61b0c2bb7c)
 2019-07-12 17:29:47 +02:00
Szilard Nemeth c61c969668 YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal 
(cherry picked from commit c416284bb7)
 2019-07-12 16:53:26 +02:00
bibinchundatt 5f8395f393 YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T. 2019-07-10 10:34:39 +05:30
Szilard Nemeth 4638fa00fc YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal. 
(cherry picked from commit a2a8be18cb)
 2019-07-04 10:26:29 +02:00
Weiwei Yang c9bccaf148 YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi. 
(cherry picked from commit 570eee30e5)
 2019-07-02 10:05:22 +08:00
Zhankun Tang 1e7201f9aa YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji. 
(cherry picked from commit 67414a1a80)
 2019-06-18 13:18:27 +08:00
bibinchundatt a37011bd5e YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie. 
(cherry picked from commit 6d80b9bc3f)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/scheduler/ContainerScheduler.java
 2019-06-11 23:01:03 +05:30
Weiwei Yang e027c87da2 YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T. 
(cherry picked from commit 4530f4500d)
 2019-06-03 14:15:20 +08:00
Eric Yang b2a39e8883 YARN-9542. Fix LogsCLI guessAppOwner ignores custome file format suffix. 
Contributed by Prabhu Joseph
 2019-05-29 18:04:13 -04:00
Takanobu Asanuma a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka 855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Haibo Chen c6573562cb YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung) 
(cherry picked from commit 597fa47ad1)
 2019-05-06 11:58:31 -07:00
Eric Yang ac85aa80d9 YARN-8587. Added retries for fetching docker exit code. 
Contributed by Charo Zhang

(cherry picked from commit c16c49b8c3)
 2019-04-19 15:40:23 -04:00
Eric Yang 4a64dab0dd YARN-8622. Fixed container-executor compilation on MacOSX. 
Contributed by Siyao Meng

(cherry picked from commit ef97a20831)
 2019-04-18 19:01:11 -04:00
Siyao Meng 742a3ad24b YARN-9487. NodeManager native build shouldn't link against librt on macOS. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 6e4399ea61)
 2019-04-17 22:57:33 -07:00
Eric Yang 10642a6205 YARN-9391. Fixed node manager environment leaks into Docker containers. 
Contributed by Jim Brennan

(cherry picked from commit 3c45762a0b)
 2019-03-25 15:54:52 -04:00
Sunil G d721634fea YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth. 
(cherry picked from commit 46045c5cb3)
 2019-03-06 16:01:56 +05:30
Sunil G d045f02a8d YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth. 2019-03-01 19:27:03 +05:30
Sunil G 809e3f2453 YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for easy access control. Contributed by Szilard Nemeth. 
(cherry picked from commit 5e91ebd91a)
 2019-02-27 12:03:58 +05:30
Sunil G a95a0cbf2f YARN-9087. Improve logging for initialization of Resource plugins. Contributed by Szilard Nemeth. 2019-02-27 11:54:43 +05:30
bibinchundatt 3e1bd53a37 YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph 2019-02-23 07:59:51 +05:30
Sunil G d75aa33612 YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth. 
(cherry picked from commit 95fbbfed75)
 2019-02-22 20:23:24 +05:30
Adam Antal 830aaac023
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d7)
 2019-02-15 18:48:21 +09:00
Masatake Iwasaki 6229469574 YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 
(cherry picked from commit e0ab1bdece)
 2019-02-09 00:28:59 +09:00
Vinayakumar B e2b91b2ccb YARN-8498. Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04. Contributed by Ayush Saxena. 2019-02-07 13:03:42 +05:30
Weiwei Yang a0fafbc3ef YARN-9263. TestConfigurationNodeAttributesProvider fails after Mockito updated. Contributed by Weiwei Yang. 
(cherry picked from commit f20b043a02)
 2019-02-04 12:45:40 +05:30
Sunil G 0e7060a1d5 YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth. 
(cherry picked from commit 71c49fa60f)
 2019-01-31 09:26:07 +05:30
Weiwei Yang 6b8dd8d113 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 
(cherry picked from commit 4f63ffe444)
 2019-01-29 10:42:09 +08:00
Wangda Tan 1dc2b49bfd YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda) 
Change-Id: Ib8044307a4241f6b1b7b9b8266b9256f39b16384
 2019-01-07 12:21:33 -08:00
Weiwei Yang 7deef08eb8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 16:31:03 +08:00
Eric Yang 29c9c8a893 YARN-9126. Fix container clean up for reinitialization. 
Contributed by Chandni Singh

(cherry picked from commit e815fd9c49)
 2018-12-19 14:58:19 -05:00
Eric Yang 52aafb9789 YARN-9125. Fixed Carriage Return detection in Docker container launch command. 
Contributed by Billie Rinaldi

(cherry picked from commit b2d7204ed0)
 2018-12-14 17:55:10 -05:00
Eric Yang 8c70728f7f YARN-9071. Improved status update for reinitialized containers. 
Contributed by Chandni Singh

(cherry picked from commit 1b790f4dd1)
 2018-12-05 19:04:55 -05:00
Jason Lowe df0e7766e4 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 3ce99e32f7)
 2018-11-28 08:50:18 -06:00