Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,677 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

21677 Commits

Author SHA1 Message Date
Akira Ajisaka a9505cf5d1
HADOOP-12665. Document hadoop.security.token.service.use_ip. (#3187) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit c81f82e21d)
 2021-07-12 10:18:44 +09:00
Artem Smotrakov 6c7f192f0f HADOOP-17793. Better token validation (#3189) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2021-07-10 14:25:08 +09:00
Akira Ajisaka aae60521a3 HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS (#3190) 
(cherry picked from commit 6ac5d8a1ed)
 2021-07-09 22:52:20 +09:00
S O'Donnell e2eacbd089 HDFS-15796. ConcurrentModificationException error happens on NameNode occasionally. Contributed by Daniel Ma 
(cherry picked from commit b6c06c4b76)
 2021-07-09 10:34:14 +01:00
bshashikant 844b8865c0 HDFS-16121. Iterative snapshot diff report can generate duplicate records for creates, deletes and Renames. (#3188) 
(cherry picked from commit 6450c1ea9a)
 2021-07-09 09:35:36 +01:00
Eric Payne ab7d2f1f3a MAPREDUCE-7353: Mapreduce job fails when NM is stopped. Contributed by Bilwa S T (BilwaST) 
(cherry picked from commit 7581413156)
 2021-07-07 21:14:47 +00:00
Masatake Iwasaki 11929ec0e7
HADOOP-17775. Remove JavaScript package from Docker environment. (#3183) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-07-07 17:49:41 +09:00
liangxs b7108cfcee HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout (#3080) 
(cherry picked from commit a5db6831bc)
 2021-07-07 09:54:16 +08:00
Viraj Jasani 4912e4c903
HDFS-16108. Fix incorrect log placeholders used in JournalNodeSyncer (#3169) 
(cherry picked from commit da575a53cf)
 2021-07-05 13:43:10 +09:00
Viraj Jasani 707d003eb6
HDFS-16092. Avoid creating LayoutFlags redundant objects (#3150) 
(cherry picked from commit 93a1685073)
(cherry picked from commit 8538881710)
 2021-06-29 18:14:02 +08:00
Jim Brennan c3c86b18cb YARN-10834. Intra-queue preemption: apps that don't use defined custom resource won't be preempted. Contributed by Eric Payne. 
(cherry picked from commit dc6f456e95)
 2021-06-28 15:28:38 +00:00
Jim Brennan 9c4ac62d68 YARN-10824. Title not set for JHS and NM webpages. Contributed by Bilwa S T. 
(cherry picked from commit 7c7d02edbd)
 2021-06-25 20:50:11 +00:00
Jim Brennan 9731dfbf0d HDFS-15659. MiniDFSCluster dfs.namenode.redundancy.considerLoad default to false (#2443). Contributed by Ahmed Hussein. 2021-06-25 17:57:15 +00:00
Ahmed Hussein e0ec1849d7
HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery (#3130). Contributed by Ahmed Hussein. (#3139) 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 581f43dce1)
 2021-06-25 22:50:04 +05:30
Szilard Nemeth c8b090d9a4 YARN-10828. Backport YARN-9789 to branch-3.2. Contributed by Tarun Parimi 2021-06-24 20:20:23 +02:00
Masatake Iwasaki 7d2186ffcf YARN-10826. [UI2] Upgrade Node.js to v12.22.1. (#3128) 
(cherry picked from commit 11d144d228)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/yarn.lock
 2021-06-24 14:45:44 +09:00
dependabot[bot] b8f0be3299 Bump bower from 1.7.7 to 1.8.8 (#1683) 
Bumps [bower](https://github.com/bower/bower) from 1.7.7 to 1.8.8.
- [Release notes](https://github.com/bower/bower/releases)
- [Changelog](https://github.com/bower/bower/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bower/bower/compare/v1.7.7...v1.8.8)

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit e9c5bb8078)
 2021-06-24 14:33:33 +09:00
Akira Ajisaka 99d8133968
YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2. (#2592) 
(cherry picked from commit ae4945fb2c)

 Conflicts:
	dev-support/docker/Dockerfile
	dev-support/docker/Dockerfile_aarch64
 2021-06-24 13:17:47 +09:00
Akira Ajisaka 6827d49b10
YARN-10331. Upgrade node.js to 10.21.0. (#2106) 
(cherry picked from commit cd188ea9f0)

 Conflicts:
	dev-support/docker/Dockerfile_aarch64
 2021-06-24 13:14:37 +09:00
Masatake Iwasaki 2eb6bf6bb1
YARN-10037. Upgrade build tools for YARN Web UI v2. 
(cherry picked from commit 7868da894a)
 2021-06-24 13:12:44 +09:00
Masatake Iwasaki 3a224b7a93
YARN-10020. Fix build instruction of hadoop-yarn-ui. (#1750) 
(cherry picked from commit 92c8962f98)
 2021-06-24 13:09:44 +09:00
Ahmed Hussein dc46cec277 HDFS-16061. DFTestUtil.waitReplication can produce false positives (#3095). Contributed by Ahmed Hussein. 
Reviewed-by: Jim Brennan <jbrennan@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-06-20 10:37:27 +05:30
litao 00bd7b2aa4 HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeline() (#3119). Contributed by tomscut. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-06-20 10:37:27 +05:30
Hui Fei c14f3e4ed1
HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet (#3114) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DirectoryScanner.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReplicaMap.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
 2021-06-18 21:04:31 +08:00
Wei-Chiu Chuang 11034f2488
HDFS-16074. Remove an expensive debug string concatenation (#3107) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: litao <tomleescut@gmail.com>
(cherry picked from commit 1ad674e548)
 2021-06-17 11:38:58 +08:00
Takanobu Asanuma 2f29784293 HADOOP-17760. Delete hadoop.ssl.enabled and dfs.https.enable from docs and core-default.xml (#3099) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 9e7c7ad129)
 2021-06-17 10:01:28 +09:00
Ayush Saxena 77f61df352 HDFS-16073. Remove redundant RPC requests for getFileLinkInfo in ClientNamenodeProtocolTranslatorPB. Contributed by lei w. 2021-06-16 15:55:06 +05:30
Jim Brennan 2636b630b4 YARN-10767. Yarn Logs Command retrying on Standby RM for 30 times. Contributed by D M Murali Krishna Reddy. 
(cherry picked from commit 9a6a11c452)
 2021-06-15 19:51:18 +00:00
Takanobu Asanuma b8fc956a4d HDFS-16068. WebHdfsFileSystem has a possible connection leak in connection with HttpFS (#3104) 
(cherry picked from commit 839fcf7682)
 2021-06-15 17:23:34 +09:00
Takanobu Asanuma 8e6b09c0c2 HADOOP-17756. Increase precommit job timeout from 20 hours to 24 hours. (#3090) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit ae9630f580)
 2021-06-14 10:22:14 +09:00
Mingliang Liu d92af85045
Revert "HDFS-16033 Fix issue of the StatisticsDataReferenceCleaner cleanUp (#3042)" 
This reverts commit 20a4cb0c67.
 2021-06-11 00:38:36 -07:00
Suma Shivaprasad e62cdf915f
YARN-9246 NPE when executing a command yarn node -status or -states without additional arguments. Contributed by Masahiro Tanaka 
(cherry picked from commit 2044967e75)
 2021-06-10 21:47:40 +09:00
Hideyuki Furue f5568e15d0
Fix container-executor 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9c7b8cf54e)
 2021-06-06 08:01:39 +09:00
July 20a4cb0c67
HDFS-16033 Fix issue of the StatisticsDataReferenceCleaner cleanUp (#3042) 
Contributed by kaifeiYi (yikf).

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Steve Loughran <stevel@apache.org>
 2021-06-04 13:59:26 -07:00
Konstantin V Shvachko 549987ed87 HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit 8ce30f51f9)
 2021-05-27 18:55:35 -07:00
Takanobu Asanuma 1eef5c2861 Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055) 
This reverts commit 0774116756.

Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 53ff2dfed3)
 2021-05-27 13:20:08 +09:00
Konstantin V Shvachko 14fd4776ae HDFS-15915. Race condition with async edits logging due to updating txId outside of the namesystem log. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 1abd03d68f)
 2021-05-26 15:10:52 -07:00
Akira Ajisaka 996a065c3b
YARN-10770. container-executor permission is wrong in SecureContainer.md. Contributed by Siddharth Ahuja. 
(cherry picked from commit 1cbcde04f3)
 2021-05-25 18:29:39 +09:00
Szilard Nemeth 3951fb6dea YARN-10123. Error message around yarn app -stop/start can be improved. Contributed by Siddharth Ahuja 2021-05-22 14:35:47 +02:00
Viraj Jasani 3c1f3b87fa HADOOP-17700. ExitUtil#halt info log should log HaltException (#3015) 
Reviewed-by: Steve Loughran <stevel@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c70ee2d548)
 2021-05-22 19:12:23 +09:00
Wei-Chiu Chuang 2823d77f19
HADOOP-17703. checkcompatibility.py errors out when specifying annotations. (#3017) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 110cda3de6)
(cherry picked from commit 0df4743284)
 2021-05-18 11:36:56 +08:00
lujiefsi bc97dd0d26
YARN-10555. Missing access check before getAppAttempts (#2608) 
Co-authored-by: lujie <lujie@foxmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d92a25b790)
 2021-05-17 14:05:26 +09:00
litao 6df2dc8463
HDFS-15873. Add namenode address in logs for block report (#2743) 
(cherry picked from commit 6a922f9896)
(cherry picked from commit 2f56e2b54d)
 2021-05-17 12:29:38 +08:00
Wei-Chiu Chuang 2ba3890b15
HADOOP-16822. Provide source artifacts for hadoop-client-api. Contributed by Karel Kolman. 
(cherry picked from commit 2c4ab72a60)
(cherry picked from commit 1b8eb2654a)
 2021-05-13 20:31:20 +08:00
hchaverr f6af5bff9e HADOOP-17680. Allow ProtobufRpcEngine to be extensible (#2905) Contributed by Hector Chaverri. 
(cherry picked from commit f40e3eb059)
 2021-05-06 18:04:25 -07:00
Akira Ajisaka a5bb211a2b YARN-10756. Remove additional junit 4.11 dependency from javadoc. (#2960) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Bilwa S T <stbilwa@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 7cb7b8adc1)
 2021-05-07 00:25:43 +09:00
Hui Fei ab4e90cc28 HDFS-15652. Make block size from NNThroughputBenchmark configurable (#2416) 
(cherry picked from commit 8a6d5b9151)
 2021-04-29 12:48:06 -07:00
Ayush Saxena 5d1127c8f3 HDFS-15378. TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk. Contributed by hemanthboyina. 
(cherry picked from commit 8db38c98a6)
 2021-04-27 23:19:22 +09:00
Wei-Chiu Chuang 650bcaf694
HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2956) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: I7de255cb9a167e3a5e25ae339b050e82cb4b7c30
(cherry picked from commit 64264d726d)
 2021-04-27 11:11:46 +09:00
Stephen O'Donnell 7a81e50bd2 HDFS-15621. Datanode DirectoryScanner uses excessive memory (#2849). Contributed by Stephen O'Donnell 
(cherry picked from commit 605ed85c29)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java

(cherry picked from commit f6efb58b07)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
 2021-04-26 12:19:01 +01:00