Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,848 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

5553 Commits

Author SHA1 Message Date
Akira Ajisaka 73775fe277
YARN-10720. YARN WebAppProxyServlet should support connection timeout to prevent proxy server from hanging. Contributed by Qi Zhu. (#4102) 
(cherry picked from commit a0deda1a77)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestWebAppProxyServlet.java

Co-authored-by: Peter Bacsko <pbacsko@cloudera.com>
 2022-03-24 17:36:44 +09:00
Masatake Iwasaki e13408d9d2
HADOOP-18088. Replace log4j 1.x with reload4j. (#4084) 
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-03-23 12:57:27 +09:00
Akira Ajisaka 523bb2efc3 YARN-9783. Remove low-level zookeeper test to be able to build Hadoop against zookeeper 3.5.5. Contributed by Mate Szalay-Beko. 
(cherry picked from commit dc0acceabb)
 2022-03-10 07:41:14 +00:00
srinivasst c64eda080e YARN-10538: Add RECOMMISSIONING nodes to the list of updated nodes returned to the AM (#2564) 
Contributed by Srinivas S T

(cherry picked from commit 1b1791075a)
(cherry picked from commit 98565b6c60)
 2022-03-10 16:35:40 +09:00
Szilard Nemeth 9e2e1c795f YARN-11014. YARN incorrectly validates maximum capacity resources on the validation API. Contributed by Benjamin Teke 2022-03-02 14:21:36 +01:00
Szilard Nemeth 4bd67d7516 YARN-11075. Explicitly declare serialVersionUID in LogMutation class. Contributed by Benjamin Teke 2022-03-01 18:06:39 +01:00
Gera Shegalov 76212079c9 YARN-11055. Add missing newline in cgroups-operations.c (#3851) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a94e9fcbde)
 2022-01-17 16:21:35 +09:00
Ashutosh Gupta e0e84431f0 YARN-8234. Improve RM system metrics publisher's performance by pushing events to timeline server in batch (#3793) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 00e2405fbd)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java
 2021-12-23 17:40:59 +09:00
Eric Payne 3924efb595 YARN-10178: Global Scheduler async thread crash caused by 'Comparison method violates its general contract. Contributed by Andras Gyori (gandras) and Qi Zhu (zhuqi). 
(cherry picked from commit e2d6fd075d)
 2021-12-21 19:34:29 +00:00
Szilard Nemeth 020acb8d39 YARN-10503. Support queue capacity in terms of absolute resources with custom resourceType. Contributed by Qi Zhu. 2021-12-20 17:59:06 +01:00
Szilard Nemeth 01de8fd9d6 YARN-6862. Nodemanager resource usage metrics sometimes are negative. Contributed by Benjamin Teke 2021-12-17 14:53:06 +01:00
Akira Ajisaka 94ca965e21 HADOOP-18040. Use maven.test.failure.ignore instead of ignoreTestFailure (#3774) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 9b9e2ef87f)

 Conflicts:
	hadoop-tools/hadoop-federation-balance/pom.xml

(cherry picked from commit 35c5c6bb83)

 Conflicts:
	hadoop-common-project/hadoop-registry/pom.xml
 2021-12-10 01:41:44 +09:00
Akira Ajisaka 3854b59bd1
HADOOP-18035. Ignore unit test failures to run all the unit tests from root (#3765) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-12-10 01:27:27 +09:00
Ashutosh Gupta c8cf7a9bdb YARN-9063. ATS 1.5 fails to start if RollingLevelDb files are corrupt or missing (#3728) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 5a950b8900)
 2021-12-06 20:15:11 +09:00
Prabhu Joseph 292527b325 YARN-10820. Make GetClusterNodesRequestPBImpl thread safe. Contributed by Swathi Chandrashekar. 
(cherry picked from commit 6b4987f09c)
 2021-12-06 14:46:34 +09:00
Eric Yang c35b3be700 YARN-7266. Fixed deadlock in Timeline Server thread initialization. 
Contributed by Prabhu Joseph

(cherry picked from commit 7b42e0e32a)
 2021-11-29 11:53:34 +09:00
GuoPhilipse 253883fd8d
YARN-11007. Correct words in YARN documents (#3680) 
Reviewed-by: cxorm <lianp964@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c9d64bad37)
 2021-11-25 17:51:58 +09:00
Ashutosh Gupta 850224255c
YARN-10991. Fix to ignore the grouping "[]" for resourcesStr in parseResourcesString method (#3592) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 08f3df3ea2)
 2021-11-24 18:50:31 +09:00
Shubham Gupta ccb711544b
YARN-10438. Handle null containerId in ClientRMService#getContainerReport() (#2313) 
Co-authored-by: Shubham Gupta <gshubham@microsoft.com>
(cherry picked from commit e3cd627069)
 2021-11-19 00:25:14 +09:00
Ahmed Hussein c36d8a1395 YARN-1115: Provide optional means for a scheduler to check real user ACLs. Contributed by Eric Payne (epayne) 2021-10-22 15:54:13 +00:00
Eric Badger ce3def0aa7 YARN-10935. AM Total Queue Limit goes below per-user AM Limit if parent is full. Contributed by Eric Payne. 2021-09-22 20:29:12 +00:00
Szilard Nemeth 4066cb0793 YARN-10870. Missing user filtering check -> yarn.webapp.filter-entity-list-by-user for RM Scheduler page. Contributed by Gergely Pollak 2021-09-14 18:09:35 +02:00
Tamas Domok 478d2f2897
YARN-10901. Permission checking error on an existing directory in LogAggregationFileController#verifyAndCreateRemoteLogDir (#3410) 
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
 2021-09-14 17:35:07 +02:00
Szilard Nemeth 7aea2e1b5c
YARN-10428. Zombie applications in the YARN queue using FAIR + sizebasedweight. Contributed by Guang Yang, Andras Gyori 
(cherry picked from commit 79a46599f7)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java
 2021-09-01 10:58:41 +09:00
Abhishek Modi 6a02596f42 YARN-9744. RollingLevelDBTimelineStore.getEntityByTime fails with NPE. Contributed by Prabhu Joseph. 
(cherry picked from commit b4097b96a3)
 2021-08-18 17:48:17 -05:00
Brahma Reddy Battula 3cf2479c04 Preparing for 3.2.4 development. 2021-08-08 18:35:16 +05:30
Akira Ajisaka c40abe10c0
YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3265) 
* YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3081)

Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit ca27744770)

* Use mockito 1.x

Co-authored-by: Viraj Jasani <vjasani@apache.org>
 2021-08-05 10:05:10 +09:00
Abhishek Modi c5e2ec465b
YARN-9990. Testcase fails with Insufficient configured threads: required=16 < max=10. Contributed by Prabhu Joseph. 
(cherry picked from commit a2dadac790)
 2021-08-04 17:05:35 +09:00
Jim Brennan 1f9a315625
YARN-10337. Amendment to fix import as in HADOOP-17100 
(cherry picked from commit 53a1c7653f)
 2021-08-04 16:38:52 +09:00
Prabhu Joseph b9d0412fd5
YARN-10337. Fix failing testcase TestRMHATimelineCollectors. 
Contributed by Bilwa S T.

(cherry picked from commit 2bbd00dff4)
 2021-08-04 16:37:47 +09:00
Akira Ajisaka d830e84217
YARN-9875. Improve fair scheduler configuration store on HDFS. (#3262) 
Contributed by Prabhu Joseph

(cherry picked from commit 155864da00)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/conf/TestFSSchedulerConfigurationStore.java

Co-authored-by: Eric Yang <eyang@apache.org>
 2021-08-04 16:12:54 +09:00
Akira Ajisaka e259d85a99
YARN-8992. Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue. (Contributed by Wilfred Spiegelenburg) (#3257) 
(cherry picked from commit a41b648e98)

Co-authored-by: Haibo Chen <haibochen@apache.org>
 2021-08-04 10:17:51 +09:00
Akira Ajisaka 786a43e729
YARN-8990. Fix fair scheduler race condition in app submit and queue cleanup. (Contributed by Wilfred Spiegelenburg) (#3254) 
(cherry picked from commit 524a7523c4)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java

Co-authored-by: Haibo Chen <haibochen@apache.org>
 2021-08-03 13:12:01 +09:00
Vrushali C 521c1085b4
YARN-9338 Timeline related testcases are failing. Contributed by Abhishek Modi. 
(cherry picked from commit 17a3e14d25)
 2021-08-03 11:42:28 +09:00
Szilard Nemeth e4df745c13 YARN-6221. Entities missing from ATS when summary log file info got returned to the ATS before the domain log. Contributed by Xiaomin Zhang 2021-07-31 14:54:51 +02:00
Szilard Nemeth 7a8b6265c6 YARN-10789. RM HA startup can fail due to race conditions in ZKConfigurationStore. Contributed by Tarun Parimi 2021-07-29 19:22:57 +02:00
9uapaw 1dc343726c YARN-9551. TestTimelineClientV2Impl.testSyncCall fails intermittent (#3212) 
Co-authored-by: Andras Gyori <gandras@cloudera.com>
(cherry picked from commit b4a524722a)
(cherry picked from commit f001595b47)
 2021-07-28 17:14:33 +02:00
Szilard Nemeth 6bf02a63e7 YARN-10813. Set default capacity of root for node labels. Contributed by Andras Gyori 2021-07-28 14:54:36 +02:00
zhuqi-lucas cfe36ba762 YARN-10860. Make max container per heartbeat configs refreshable. Contributed by Eric Badger. 2021-07-22 10:13:55 +08:00
Masatake Iwasaki f3dac4ac23
YARN-10858. [UI2] YARN-10826 breaks Queue view. (#3213) 
(cherry picked from commit 17bf2fcbc5)
 2021-07-20 14:16:35 +09:00
Jim Brennan 4a70248cb6 YARN-10456. RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry. Contributed by Eric Payne. 
(cherry picked from commit 632f64cadb)
 2021-07-15 14:49:22 +00:00
Artem Smotrakov 6c7f192f0f HADOOP-17793. Better token validation (#3189) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2021-07-10 14:25:08 +09:00
Jim Brennan c3c86b18cb YARN-10834. Intra-queue preemption: apps that don't use defined custom resource won't be preempted. Contributed by Eric Payne. 
(cherry picked from commit dc6f456e95)
 2021-06-28 15:28:38 +00:00
Jim Brennan 9c4ac62d68 YARN-10824. Title not set for JHS and NM webpages. Contributed by Bilwa S T. 
(cherry picked from commit 7c7d02edbd)
 2021-06-25 20:50:11 +00:00
Szilard Nemeth c8b090d9a4 YARN-10828. Backport YARN-9789 to branch-3.2. Contributed by Tarun Parimi 2021-06-24 20:20:23 +02:00
Masatake Iwasaki 7d2186ffcf YARN-10826. [UI2] Upgrade Node.js to v12.22.1. (#3128) 
(cherry picked from commit 11d144d228)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/yarn.lock
 2021-06-24 14:45:44 +09:00
dependabot[bot] b8f0be3299 Bump bower from 1.7.7 to 1.8.8 (#1683) 
Bumps [bower](https://github.com/bower/bower) from 1.7.7 to 1.8.8.
- [Release notes](https://github.com/bower/bower/releases)
- [Changelog](https://github.com/bower/bower/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bower/bower/compare/v1.7.7...v1.8.8)

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit e9c5bb8078)
 2021-06-24 14:33:33 +09:00
Akira Ajisaka 99d8133968
YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2. (#2592) 
(cherry picked from commit ae4945fb2c)

 Conflicts:
	dev-support/docker/Dockerfile
	dev-support/docker/Dockerfile_aarch64
 2021-06-24 13:17:47 +09:00
Akira Ajisaka 6827d49b10
YARN-10331. Upgrade node.js to 10.21.0. (#2106) 
(cherry picked from commit cd188ea9f0)

 Conflicts:
	dev-support/docker/Dockerfile_aarch64
 2021-06-24 13:14:37 +09:00
Masatake Iwasaki 2eb6bf6bb1
YARN-10037. Upgrade build tools for YARN Web UI v2. 
(cherry picked from commit 7868da894a)
 2021-06-24 13:12:44 +09:00