Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,785 Commits 326 Branches 371 Tags
Commit Graph

5062 Commits

Author SHA1 Message Date
Szilard Nemeth
6980f1740f YARN-9217. Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing. Contributed by Peter Bacsko 2019-08-21 16:49:34 +02:00
Szilard Nemeth
a83718f130 YARN-9100. Add tests for GpuResourceAllocator and do minor code cleanup. Contributed by Peter Bacsko 2019-08-16 15:24:44 +02:00
Szilard Nemeth
df616370f0 YARN-8586. Extract log aggregation related fields and methods from RMAppImpl. Contributed by Peter Bacsko 2019-08-16 11:52:51 +02:00
Szilard Nemeth
8fee3808c5 YARN-9749. TestAppLogAggregatorImpl#testDFSQuotaExceeded fails on trunk. Contributed by Adam Antal 
(cherry picked from commit 2a05e0ff3b5ab3be8654e9e96c6556865ef26096)
 2019-08-16 08:52:34 +02:00
Szilard Nemeth
e616037d1f YARN-9488. Skip YARNFeatureNotEnabledException from ClientRMService. Contributed by Prabhu Joseph 
(cherry picked from commit 1845a83cec6563482523d8c34b38c4e36c0aa9df)
 2019-08-15 17:16:06 +02:00
Adam Antal
d5446b3a23 YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl… (#1261) 
* YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl and connected classes

* Using {} placeholder, and increasing loglevel if log aggregation failed.

(cherry picked from commit c89bdfacc8715fa6d72acd85437ab8cd257c8aad)
 2019-08-14 17:36:41 +02:00
Szilard Nemeth
4bb238c480 YARN-9133. Make tests more easy to comprehend in TestGpuResourceHandler. Contributed by Peter Bacsko 2019-08-14 17:16:54 +02:00
Szilard Nemeth
4dc477b606 YARN-9140. Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager. Contributed by Peter Bacsko 2019-08-14 17:01:41 +02:00
Szilard Nemeth
9a87e74e54 YARN-9134. No test coverage for redefining FPGA / GPU resource types in TestResourceUtils. Contributed by Peter Bacsko 2019-08-14 16:46:34 +02:00
Eric Badger
cec71691be YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan. 
(cherry picked from commit 2ac029b949f041da2ee04da441c5f9f85e1f2c64)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2019-08-13 16:34:29 +00:00
Szilard Nemeth
c5aea8ca56 YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph 
(cherry picked from commit e4b538bbda6dc25d7f45bffd6a4ce49f3f84acdc)
 2019-08-12 15:16:18 +02:00
Szilard Nemeth
844259203f YARN-9451. AggregatedLogsBlock shows wrong NM http port. Contributed by Prabhu Joseph 
(cherry picked from commit b91099efd6e1fdcb31ec4ca7142439443c9ae536)
 2019-08-12 15:06:16 +02:00
Szilard Nemeth
b20fd9e212 YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko 2019-08-12 14:02:17 +02:00
Sunil G
02b4635ff0 YARN-9729. [UI2] Fix error message for logs when ATSv2 is offline. Contributed by Zoltan Siegl. 
(cherry picked from commit 1c5b28659fe1310030d14e0be40dcd77b25056d6)
 2019-08-11 11:49:25 +05:30
Szilard Nemeth
2e6beb1550 Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph 
(cherry picked from commit 54ac80176e8487b7a18cd9e16a11efa289d0b7df)
 2019-08-09 13:20:10 +02:00
Sunil G
9fb6c6e2a1 YARN-9715. [UI2] yarn-container-log URI need to be encoded to avoid potential misuses. Contributed by Akhil PB. 
(cherry picked from commit acffec7a92be540aa8531dbe06a3ea7bb813ab93)
 2019-08-09 16:07:04 +05:30
Szilard Nemeth
3e9071207a SUBMARINE-57. Add more elaborate message if submarine command is not recognized. Contributed by Adam Antal 
(cherry picked from commit e5f4cd0fdae7e689789dd74bfbcfa6c52895f037)
 2019-08-09 12:14:49 +02:00
Adam Antal
4c4f7d9c80 YARN-9124. Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently (#1121) 
(cherry picked from commit cbcada804d119b837ad99de71d7f44cb4629026e)
 2019-08-09 11:43:30 +02:00
Szilard Nemeth
02d0e54596 YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak 
(cherry picked from commit e0c21c6da91776caf661661a19c368939c81fcc4)
 2019-08-09 10:23:10 +02:00
Szilard Nemeth
f0dfb8b832 YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak 
(cherry picked from commit 742e30b47381ad63e2b2fe63738cd0fe6cbce106)
 2019-08-09 10:02:35 +02:00
Szilard Nemeth
3bcf44f070 YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak 
(cherry picked from commit 72d7e570a73989aa18b737c0e642d570a55c6781)
 2019-08-09 09:50:32 +02:00
Eric E Payne
e47c483d9f YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang. 
(cherry picked from commit 3b38f2019e4f8d056580f3ed67ecef591011d7a6)
 2019-08-08 12:54:31 +00:00
Haibo Chen
8d357343c4 YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 
(cherry picked from commit f51702d5398531835b24d812f6f95094a0e0493e)
 2019-08-06 14:59:49 -07:00
Eric E Payne
168dc3f258 YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan. 
(cherry picked from commit 42683aef1a694af883c14842bf41f30b91e039f3)
 2019-07-30 19:19:33 +00:00
Jonathan Hung
15344006bc YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-22 10:46:45 -07:00
Weiwei Yang
bf3d9f6282 YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam. 
(cherry picked from commit 85d9111a88f94a5e6833cd142272be2c5823e922)
 2019-07-17 10:47:25 +08:00
bibinchundatt
4866735cde YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T. 
(cherry picked from commit 7a93be0f6002ebb376c30f25a7d403e853c44280)
 2019-07-16 14:06:36 +05:30
Szilard Nemeth
7c9cfc0996 YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal 
(cherry picked from commit 5446308360f57cb98c54c416231788ba9ae332f8)
 2019-07-15 13:30:58 +02:00
Szilard Nemeth
28d6a453a9 YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko 
(cherry picked from commit 18ee1092b471c5337f05809f8f01dae415e51a3a)
 2019-07-15 12:02:39 +02:00
Szilard Nemeth
2fcbdf4131 YARN-9337. Addendum to fix compilation error due to mockito spy call 
(cherry picked from commit bb37c6cb7ff2b810efd139525ad0a37937baa93c)
 2019-07-13 00:45:38 +02:00
Szilard Nemeth
4fa0de9f04 YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl 
(cherry picked from commit 557056e18ea3d5b3fe3046f0ea4b4c7345ea21c5)
 2019-07-12 17:40:57 +02:00
Szilard Nemeth
0ede873090 YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal 
(cherry picked from commit 61b0c2bb7c0f18c4a666b96ca1603cbd4d27eb6d)
 2019-07-12 17:29:47 +02:00
Szilard Nemeth
c61c969668 YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal 
(cherry picked from commit c416284bb7581747beef36d7899d8680fe33abbd)
 2019-07-12 16:53:26 +02:00
Szilard Nemeth
3e3bbb7f5e YARN-9625. UI2 - No link to a queue on the Queues page for Fair Scheduler. Contributed by Zoltan Siegl 
(cherry picked from commit 9cec02318644c8430cbf65bcc3096ffe45992a8e)
 2019-07-11 20:01:52 +02:00
Szilard Nemeth
4216090f19 YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal. 2019-07-11 19:24:11 +02:00
bibinchundatt
5f8395f393 YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T. 2019-07-10 10:34:39 +05:30
Szilard Nemeth
4638fa00fc YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal. 
(cherry picked from commit a2a8be18cb5e912c8de0ea6beec1de4a99de656b)
 2019-07-04 10:26:29 +02:00
Sunil G
d18986e4e8 YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt. 2019-07-04 11:05:54 +05:30
Sunil G
bea79e7645 YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt. 
(cherry picked from commit 0c8813f135f8c17f88660bb92529c15bb3a157ca)
 2019-07-02 12:15:05 +05:30
Weiwei Yang
c9bccaf148 YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi. 
(cherry picked from commit 570eee30e5ab5cf37b1a758934987cbf61140f6a)
 2019-07-02 10:05:22 +08:00
Erik Krogen
49d7bb6a92 HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-06-28 14:20:01 -07:00
Eric Yang
860606fc67 YARN-9581. Add support for get multiple RM webapp URLs. 
Contributed by Prabhu Joseph

(cherry picked from commit f02b0e19940dc6fc1e19258a40db37d1eed89d21)
 2019-06-28 14:57:50 -04:00
bibinchundatt
a2f4e4698b YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T. 
(cherry picked from commit be80334cdf255616f589217726483194fb56dcc6)
 2019-06-27 10:04:40 +05:30
Weiwei Yang
1944a7d844 YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi. 
(cherry picked from commit 83dcb9d87ec75f2be0acb8972f5f0faefe6ffbcd)
 2019-06-21 17:52:22 +08:00
Wanqiang Ji
f148b29508 YARN-9630. [UI2] Add a link in docs's top page 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit eb6be4643f77b3284297950da4f7e6ca9db11793)
 2019-06-18 14:57:01 +09:00
Zhankun Tang
1e7201f9aa YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji. 
(cherry picked from commit 67414a1a80039e70e0afc1de171831a6e981f37a)
 2019-06-18 13:18:27 +08:00
Inigo Goiri
65f7ec2f39 YARN-8856. TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError. Contributed by Sushil Ks. 
(cherry picked from commit eeaf8edaa7d36715dfa00a29fe46e4c6de4b98cf)
 2019-06-13 14:22:16 -07:00
Sean Mackrory
e0b3cbd221 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:53:40 -06:00
Sunil G
253dcde517 YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B. 
(cherry picked from commit 52128e352a30b70b83483f9290d9e94e98929705)
 2019-06-12 19:25:02 +05:30
Sunil G
72203f7a12 YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 2019-06-12 19:23:40 +05:30