Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,441 Commits 326 Branches 371 Tags
Commit Graph

15441 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Abhishek Modi
b79cd11fcd YARN-9744. RollingLevelDBTimelineStore.getEntityByTime fails with NPE. Contributed by Prabhu Joseph. 
(cherry picked from commit b4097b96a39bad6214b01989e7f2fb37dad70793)
 2021-08-18 17:54:10 -05:00
Akira Ajisaka
bd7e20527e
HADOOP-17370. Upgrade commons-compress to 1.21 2021-08-08 11:37:11 +09:00
Akira Ajisaka
d07955ce50
Fix potential heap buffer overflow in hdfs.c. Contributed by Igor Chervatyuk. 
(cherry picked from commit 4972e7a246f4aab665fd04ce72d1848bc5da9d4e)
 2021-08-07 11:46:30 +09:00
Chris Douglas
f856092bdb MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec 
(cherry picked from commit 782ba3bf9da52699b27405a3f147464975d1df99)
 2021-08-03 21:00:44 +00:00
hchaverr
2187c6d366 HADOOP-17819. Add extensions to ProtobufRpcEngine RequestHeaderProto. Contributed by Hector Sandoval Chaverri. (#3242) 
(cherry picked from commit 3c8a48e681babe4038e49205624e7be183b3466e)
 2021-07-28 16:04:46 -07:00
Viraj Jasani
5e65126e38 HADOOP-17813. Checkstyle - Allow line length: 100 
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 98412ce2e1453308655d4f43e7f8c922960e6955)
 2021-07-22 15:33:09 -05:00
zhuqi-lucas
34acf9d4c8 YARN-10860. Make max container per heartbeat configs refreshable. Contributed by Eric Badger. 2021-07-21 15:35:45 +08:00
Abhishek Das
99b952fcb4 HADOOP-17028. ViewFS should initialize mounted target filesystems lazily. Contributed by Abhishek Das (#2260, #3218) 
(cherry picked from commit 1dd03cc4b573270dc960117c3b6c74bb78215caa)
 2021-07-20 18:26:24 -07:00
Jim Brennan
39c8297350 HDFS-16042. DatanodeAdminMonitor scan should be delay based. Contributed by Ahmed Hussein. 2021-07-15 21:13:25 +00:00
Jim Brennan
577ed175f9 YARN-10456. RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry. Contributed by Eric Payne. 2021-07-15 15:21:02 +00:00
Akira Ajisaka
992d2c5f4e
HADOOP-12665. Document hadoop.security.token.service.use_ip. (#3187) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit c81f82e21d137b6d45c20c481e7bbc6160d72f94)
 2021-07-12 10:19:06 +09:00
Artem Smotrakov
9241470dfd HADOOP-17793. Better token validation (#3189) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada573291266c4d6447862072fdf88af5)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2021-07-10 15:06:37 +09:00
Eric Payne
cef04b40b6 MAPREDUCE-7353: Mapreduce job fails when NM is stopped. Contributed by Bilwa S T (BilwaST) 
(cherry picked from commit 7581413156da396db218e36a966c5749589b31a7)
 2021-07-07 21:40:09 +00:00
Masatake Iwasaki
6df4bc1a8c
HADOOP-17775. Remove JavaScript package from Docker environment. (#3184) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-07-07 17:49:59 +09:00
Jim Brennan
f7bcc58e0f YARN-10834. Intra-queue preemption: apps that don't use defined custom resource won't be preempted. Contributed by Eric Payne. 2021-06-29 14:22:39 +00:00
Jim Brennan
51be95e234 YARN-10824. Title not set for JHS and NM webpages. Contributed by Bilwa S T. 
(cherry picked from commit 7c7d02edbd6c17ee8ae2c4bf75e87adace059b76)
 2021-06-25 20:58:50 +00:00
Jim Brennan
7d231f4b98 HDFS-15659. MiniDFSCluster dfs.namenode.redundancy.considerLoad default to false (#2443). Contributed by Ahmed Hussein. 2021-06-25 18:06:37 +00:00
Ahmed Hussein
b58be3a12d
HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery (#3131). Contributed by Ahmed Hussein. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Hui Fei <ferhui@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-06-23 20:05:31 +05:30
S O'Donnell
caa59d7a58 HDFS-15150. Introduce read write lock to Datanode. Contributed by Stephen O'Donnell and Ahmed Hussein 2021-06-22 16:24:19 +01:00
Kihwal Lee
ab6b5681e8 HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. 2021-06-16 11:38:30 -05:00
Kihwal Lee
e7c7fb2896 HDFS-15963. Unreleased volume references cause an infinite loop. (#2941) Contributed by Shuyan Zhang. 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 9f2db2c9fdcf1e61b39651a51f8223cfb8ce2e31)
 2021-06-15 19:57:21 -05:00
Takanobu Asanuma
a690e8ae59 HDFS-16068. WebHdfsFileSystem has a possible connection leak in connection with HttpFS (#3104) 
(cherry picked from commit 839fcf7682270bc6ba151875549d64d81917f425)
 2021-06-15 17:24:27 +09:00
Takanobu Asanuma
f22ff63a15 HADOOP-17756. Increase precommit job timeout from 20 hours to 24 hours. (#3090) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit ae9630f580d73e81042f3b8b2b337aba86130653)
 2021-06-14 10:23:00 +09:00
Hideyuki Furue
a07293f819
Fix container-executor 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9c7b8cf54ea88833d54fc71a9612c448dc0eb78d)
(cherry picked from commit f5568e15d0d6fdf9593ee869485aa6471b889d52)
 2021-06-06 09:14:18 +09:00
Konstantin V Shvachko
149325484f HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit 8ce30f51f999c0a80db53a2a96b5be5505d4d151)
 2021-05-27 20:19:44 -07:00
Konstantin V Shvachko
8715a18f95 HDFS-15915. Race condition with async edits logging due to updating txId outside of the namesystem log. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 1abd03d68f4f236674ce929164cc460037730abb)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java
 2021-05-26 16:00:18 -07:00
Akira Ajisaka
76eea30bdd
YARN-10770. container-executor permission is wrong in SecureContainer.md. Contributed by Siddharth Ahuja. 
(cherry picked from commit 1cbcde04f3b025e4c0c844e8bfe8d9007af2be3b)
 2021-05-25 18:30:21 +09:00
lujiefsi
13a2e751e0
YARN-10555. Missing access check before getAppAttempts (#2608) 
Co-authored-by: lujie <lujie@foxmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d92a25b790e5ad7d8e21fc3949cdd0f74d496b1b)
 2021-05-17 19:47:03 +09:00
hchaverri
7f6185b17c
HDFS-15912. Allow ProtobufRpcEngine to be extensible (#2999) 2021-05-17 00:37:07 -07:00
Hui Fei
77d9c6d0f7 HDFS-15652. Make block size from NNThroughputBenchmark configurable (#2416) 
(cherry picked from commit 8a6d5b9151cd4a922372835b18a9a031c9d3475e)
 2021-04-29 12:56:57 -07:00
Eric Badger
7b3a6e96d9 YARN-10479. Can't remove all node labels after add node label without 
nodemanager port, broken by YARN-10647. Contributed by D M Murali Krishna Reddy

(cherry picked from commit 6857a05d6ac566a60336c0a28951f09ecda39f24)
 2021-04-23 23:10:50 +00:00
Eric Badger
58b33d6570 MAPREDUCE-7302. Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail. Contributed by Peter Bacsko. Reviewed by Akira Ajisaka. 
Conflicts only in imports

(cherry picked from commit da93cd962c8d19874d09360726628cdde8595a7d)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java

(cherry picked from commit 7ad844ee9ed7eaab4fbb4d38576be10d497460cf)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestFetcher.java
 2021-04-23 20:16:11 +00:00
bibinchundatt
94c1cf2ff2 YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie. 
(cherry picked from commit 6d80b9bc3ff3ba8073e3faf64551b9109d2aa2ad)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/scheduler/ContainerScheduler.java

(cherry picked from commit 4a39165b41dc079dd2de140e610255f363982f59)
 2021-04-23 16:48:04 +00:00
Eric Badger
eb176bb2af YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko 2021-04-20 00:31:08 +00:00
Ahmed Hussein
55645d78e1
YARN-10733. TimelineService Hbase tests failing with timeouts (#2899) 2021-04-14 13:44:38 -05:00
Ahmed Hussein
1056ca3725
HADOOP-17601. Upgrade Jackson databind in branch-2.10 to 2.9.10.7. (#2835) 2021-04-12 17:42:40 -07:00
Ahmed Hussein
cb5e41ac07
HADOOP-17603. Upgrade tomcat-embed-core to 7.0.108 (#2834). Contributed by Ahmed Hussein 2021-04-12 10:11:36 -07:00
Viraj Jasani
3c1c1b40d8
HDFS-15931 : Fix non-static inner classes for better memory management (#2830). Contributed by Viraj Jasani 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-04-01 17:48:27 -07:00
Akira Ajisaka
d822bc3def
YARN-10716. Fix typo in ContainerRuntime. Contributed by xishuhai. 
(cherry picked from commit 8ad77a26c9f674e734e7f4c917ee2d51c77e527d)
 2021-03-30 17:49:01 +09:00
Eric Badger
616256b400 YARN-10501. Can't remove all node labels after add node label without 
nodemanager port. Contributed by caozhiqiang.
 2021-03-29 18:11:58 +00:00
Akira Ajisaka
f207d9e620
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin. Contributed by Ahmed Hussein. 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.
 2021-03-29 09:44:28 +09:00
Jim Brennan
7a3a749663 MAPREDUCE-7332. Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10. Contributed by Ahmed Hussein. 2021-03-26 15:39:59 +00:00
Akira Ajisaka
4829e1b7ed
HADOOP-17602. Upgrade JUnit to 4.13.1. Contributed by Ahmed Hussein. 2021-03-25 20:30:34 +09:00
Jim Brennan
33c4d4570d YARN-10697. Resources are displayed in bytes in UI for schedulers other than capacity. Contributed by Bilwa S T. 
(cherry picked from commit 34e507cb8c11d3b6ee561fd4aabde6dadadcee00)
 2021-03-23 19:05:57 +00:00
Jim Brennan
99c34a3576 MAPREDUCE-7325. Intermediate data encryption is broken in LocalJobRunner. Contributed by Ahmed Hussein 
(cherry picked from commit ede490d13182f0fd3831afe6db97e01917add5b4)
 2021-03-22 20:31:56 +00:00
Jim Brennan
b3532a7c22 MAPREDUCE-7322. revisiting TestMRIntermediateDataEncryption. Contributed by Ahmed Hussein. 2021-03-22 20:30:19 +00:00
kwangsun
e6b206b33d
HADOOP-17952. Fix the wrong CIDR range example in Proxy User documentation. (#2780) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c8d327a4f1a7f15d6be35051414199d1d3fdc5ef)
 2021-03-22 11:47:03 +09:00
touchida
7183e269ac
HDFS-13975. TestBalancer#testMaxIterationTime fails sporadically (#2726) 
Co-authored-by: toshihiko.uchida <toshihiko.uchida@linecorp.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 25390babc74ab8a6bbc53305b7a49df2437406f2)
 2021-03-17 16:34:52 +09:00
Viraj Jasani
3d0a0ed92b HDFS-15895 : Remove redundant String#format in DFSAdmin#printOpenFiles (#2772) 
(cherry picked from commit 9c43b60348bea7f89f7c0fc86f2813fdf2d1929f)
 2021-03-17 15:16:18 +09:00
Wangda Tan
c6033eeeef YARN-1151. Ability to configure auxiliary services from HDFS-based JAR files. (Xuan Gong via wangda) 
Change-Id: Ied37ff11e507fc86847753ba79486652c8fadfe9
(cherry picked from commit c962371430b8957500397fe71e944c953f46ed7d)
 2021-03-16 11:03:56 -07:00