Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,393 Commits 326 Branches 371 Tags
Commit Graph

5457 Commits

Author SHA1 Message Date
Szilard Nemeth
02d0e54596 YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak 
(cherry picked from commit e0c21c6da91776caf661661a19c368939c81fcc4)
 2019-08-09 10:23:10 +02:00
Szilard Nemeth
f0dfb8b832 YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak 
(cherry picked from commit 742e30b47381ad63e2b2fe63738cd0fe6cbce106)
 2019-08-09 10:02:35 +02:00
Szilard Nemeth
3bcf44f070 YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak 
(cherry picked from commit 72d7e570a73989aa18b737c0e642d570a55c6781)
 2019-08-09 09:50:32 +02:00
Eric E Payne
e47c483d9f YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang. 
(cherry picked from commit 3b38f2019e4f8d056580f3ed67ecef591011d7a6)
 2019-08-08 12:54:31 +00:00
Haibo Chen
8d357343c4 YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 
(cherry picked from commit f51702d5398531835b24d812f6f95094a0e0493e)
 2019-08-06 14:59:49 -07:00
Eric E Payne
168dc3f258 YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan. 
(cherry picked from commit 42683aef1a694af883c14842bf41f30b91e039f3)
 2019-07-30 19:19:33 +00:00
Jonathan Hung
15344006bc YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-22 10:46:45 -07:00
Weiwei Yang
bf3d9f6282 YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam. 
(cherry picked from commit 85d9111a88f94a5e6833cd142272be2c5823e922)
 2019-07-17 10:47:25 +08:00
bibinchundatt
4866735cde YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T. 
(cherry picked from commit 7a93be0f6002ebb376c30f25a7d403e853c44280)
 2019-07-16 14:06:36 +05:30
Szilard Nemeth
7c9cfc0996 YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal 
(cherry picked from commit 5446308360f57cb98c54c416231788ba9ae332f8)
 2019-07-15 13:30:58 +02:00
Szilard Nemeth
28d6a453a9 YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko 
(cherry picked from commit 18ee1092b471c5337f05809f8f01dae415e51a3a)
 2019-07-15 12:02:39 +02:00
Szilard Nemeth
2fcbdf4131 YARN-9337. Addendum to fix compilation error due to mockito spy call 
(cherry picked from commit bb37c6cb7ff2b810efd139525ad0a37937baa93c)
 2019-07-13 00:45:38 +02:00
Szilard Nemeth
4fa0de9f04 YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl 
(cherry picked from commit 557056e18ea3d5b3fe3046f0ea4b4c7345ea21c5)
 2019-07-12 17:40:57 +02:00
Szilard Nemeth
0ede873090 YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal 
(cherry picked from commit 61b0c2bb7c0f18c4a666b96ca1603cbd4d27eb6d)
 2019-07-12 17:29:47 +02:00
Szilard Nemeth
c61c969668 YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal 
(cherry picked from commit c416284bb7581747beef36d7899d8680fe33abbd)
 2019-07-12 16:53:26 +02:00
Szilard Nemeth
3e3bbb7f5e YARN-9625. UI2 - No link to a queue on the Queues page for Fair Scheduler. Contributed by Zoltan Siegl 
(cherry picked from commit 9cec02318644c8430cbf65bcc3096ffe45992a8e)
 2019-07-11 20:01:52 +02:00
Szilard Nemeth
4216090f19 YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal. 2019-07-11 19:24:11 +02:00
bibinchundatt
5f8395f393 YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T. 2019-07-10 10:34:39 +05:30
Szilard Nemeth
4638fa00fc YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal. 
(cherry picked from commit a2a8be18cb5e912c8de0ea6beec1de4a99de656b)
 2019-07-04 10:26:29 +02:00
Sunil G
d18986e4e8 YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt. 2019-07-04 11:05:54 +05:30
Sunil G
bea79e7645 YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt. 
(cherry picked from commit 0c8813f135f8c17f88660bb92529c15bb3a157ca)
 2019-07-02 12:15:05 +05:30
Weiwei Yang
c9bccaf148 YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi. 
(cherry picked from commit 570eee30e5ab5cf37b1a758934987cbf61140f6a)
 2019-07-02 10:05:22 +08:00
Erik Krogen
49d7bb6a92 HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-06-28 14:20:01 -07:00
Eric Yang
860606fc67 YARN-9581. Add support for get multiple RM webapp URLs. 
Contributed by Prabhu Joseph

(cherry picked from commit f02b0e19940dc6fc1e19258a40db37d1eed89d21)
 2019-06-28 14:57:50 -04:00
bibinchundatt
a2f4e4698b YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T. 
(cherry picked from commit be80334cdf255616f589217726483194fb56dcc6)
 2019-06-27 10:04:40 +05:30
Weiwei Yang
1944a7d844 YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi. 
(cherry picked from commit 83dcb9d87ec75f2be0acb8972f5f0faefe6ffbcd)
 2019-06-21 17:52:22 +08:00
Wanqiang Ji
f148b29508 YARN-9630. [UI2] Add a link in docs's top page 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit eb6be4643f77b3284297950da4f7e6ca9db11793)
 2019-06-18 14:57:01 +09:00
Zhankun Tang
1e7201f9aa YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji. 
(cherry picked from commit 67414a1a80039e70e0afc1de171831a6e981f37a)
 2019-06-18 13:18:27 +08:00
Inigo Goiri
65f7ec2f39 YARN-8856. TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError. Contributed by Sushil Ks. 
(cherry picked from commit eeaf8edaa7d36715dfa00a29fe46e4c6de4b98cf)
 2019-06-13 14:22:16 -07:00
Sean Mackrory
e0b3cbd221 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:53:40 -06:00
Sunil G
253dcde517 YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B. 
(cherry picked from commit 52128e352a30b70b83483f9290d9e94e98929705)
 2019-06-12 19:25:02 +05:30
Sunil G
72203f7a12 YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 2019-06-12 19:23:40 +05:30
Sunil G
f1ead03672 Revert "YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl." 
This reverts commit f1d3a17d3e67ec2acad52227a3f4eb7cca83e468.
 2019-06-12 19:10:23 +05:30
bibinchundatt
3303723f55 YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T. 2019-06-11 23:42:29 +05:30
bibinchundatt
d9284d4a57 YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T. 
(cherry picked from commit 60c95e9b6a899e37ecdc8bce7bb6d9ed0dc7a6be)
 2019-06-11 23:13:18 +05:30
bibinchundatt
a37011bd5e YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie. 
(cherry picked from commit 6d80b9bc3ff3ba8073e3faf64551b9109d2aa2ad)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/scheduler/ContainerScheduler.java
 2019-06-11 23:01:03 +05:30
Eric Yang
68aec0a98d YARN-9581. Fixed yarn logs cli to access RM2. 
Contributed by Prabhu Joseph

(cherry picked from commit cb9bc6e64c590622ae04aea2c81962be59037f7a)
 2019-06-06 16:43:25 -04:00
Sunil G
f1d3a17d3e YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 2019-06-06 06:24:01 +05:30
Weiwei Yang
6e2b091515 YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang. 2019-06-04 15:24:37 +08:00
Sunil G
2f012044ff YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB. 
(cherry picked from commit 59719dc560cf67f485d8e5b4a6f0f38ef97d536b)
 2019-06-03 15:54:07 +05:30
Sunil G
58042dadc3 YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB. 
(cherry picked from commit 7f46dda513fb79c349acb73bdb90b689df9cc18d)
 2019-06-03 12:25:16 +05:30
Weiwei Yang
e027c87da2 YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T. 
(cherry picked from commit 4530f4500d308c9cefbcc5990769c04bd061ad87)
 2019-06-03 14:15:20 +08:00
Eric Yang
b2a39e8883 YARN-9542. Fix LogsCLI guessAppOwner ignores custome file format suffix. 
Contributed by Prabhu Joseph
 2019-05-29 18:04:13 -04:00
Eric E Payne
2e561cef47 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 
(cherry picked from commit 3c63551101dc17038efb9f5345833d2883fce86e)
 2019-05-29 18:43:13 +00:00
Ahmed Hussein
777f7345ef YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit abf76ac371e4611c8eb371736b433e3d89c9d2ae)
 2019-05-29 12:14:01 -05:00
Takanobu Asanuma
a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka
c917ba782e
YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang. 
(cherry picked from commit 4a692bc3be0800891ea58c13ba7d89efd358f6f6)
 2019-05-28 16:54:43 +09:00
Akira Ajisaka
855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
bibinchundatt
71f5bfb822 YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T. 
(cherry picked from commit 570fa2da20706490dc7823efd0ce0cef3ddc81f9)
 2019-05-15 13:31:07 +05:30
Sunil G
f4ee38df29 YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal. 
(cherry picked from commit 7d831eca645f93d064975ebae35a7cbea3bbad31)
 2019-05-14 10:49:09 -07:00