Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
24,763 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

24763 Commits

Author SHA1 Message Date
Mukund Thakur 472a28f91d HADOOP-18065. ExecutorHelper.logThrowableFromAfterExecute() is too noisy. (#3860) 
Downgrading warn logs to debug in case of InterruptedException

Contributed By: Mukund Thakur
 2022-01-15 09:25:38 -08:00
monthonk d101e4d7fa HADOOP-14334. S3 SSEC tests to downgrade when running against a mandatory encryption object store (#3870) 
Contributed by Monthon Klongklaew

Change-Id: Ib275c9690bbc90170c6a442ded198fe006c20bc1
 2022-01-15 09:25:32 -08:00
Ayush Saxena 46b1411189 HADOOP-18056. DistCp: Filter duplicates in the source paths. (#3825). Contributed by Ayush Saxena. 
Reviewed-by: tomscut <litao@bigo.sg>
Reviewed-by: Steve Loughran <stevel@apache.org>
 2022-01-15 09:25:25 -08:00
Jackson Wang d7cacea07b HDFS-16420. Avoid deleting unique data blocks when deleting redundancy striped blocks. (#3880) 
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2022-01-14 09:07:32 -08:00
Brahma Reddy Battula 8af28b7cca Updated the index as per 3.3.2 release 2022-01-05 10:44:28 -08:00
Ashutosh Gupta 6ea9906a8b HDFS-16410. Insecure Xml parsing in OfflineEditsXmlLoader (#3854) 
Contributed by Ashutosh Gupta
 2022-01-05 10:01:38 -08:00
liever18 d167ef4850 HDFS-16408. Ensure LeaseRecheckIntervalMs is greater than zero (#3856) 
(cherry picked from commit e1d0aa9ee5)
 2022-01-05 10:01:20 -08:00
Wei-Chiu Chuang e42e483d00 Make upstream aware of 3.3.1 release 2022-01-04 14:51:49 -08:00
Ashutosh Gupta b9ed31f9ec HDFS-16409. Fix typo: testHasExeceptionsReturnsCorrectValue -> testHasExceptionsReturnsCorrectValue (#3835) 
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9eea0e28f2)
 2022-01-04 14:38:52 -08:00
jianghuazhu b8fff25645 HADOOP-18063. Remove unused import AbstractJavaKeyStoreProvider in Shell class. (#3846) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 7398a0f1b2)
 2022-01-04 14:38:41 -08:00
Igor Dvorzhak 9a24995d73 HADOOP-13464. Upgrade Gson dependency to version 2.8.9 (#2524) 
Change-Id: Ifd3fb9ec6ebfc8874bb799bc198219511fe55a2f

Update pom.xml

Update pom.xml

(cherry picked from commit 795054882a)
 2022-01-04 14:38:38 -08:00
Ashutosh Gupta 80a19c8a48 HDFS-14099. Unknown frame descriptor when decompressing multiple frames (#3836) 
Co-authored-by: xuzq <xuzengqiang@kuaishou.com>
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit caab29ec88)
 2022-01-04 14:38:35 -08:00
Akira Ajisaka 6db1165380 Revert "HDFS-14099. Unknown frame descriptor when decompressing multiple frames (#3836)" 
This reverts commit 05b43f2057.
 2022-01-04 14:38:31 -08:00
Ashutosh Gupta 8aac1c7dc9 HDFS-14099. Unknown frame descriptor when decompressing multiple frames (#3836) 
Co-authored-by: xuzq <xuzengqiang@kuaishou.com>
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit caab29ec88)
 2022-01-04 14:38:28 -08:00
Akira Ajisaka 84a767ea24 HADOOP-18045. Disable TestDynamometerInfra (#3829) 
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
(cherry picked from commit dba139cd0f)
 2022-01-04 14:38:24 -08:00
jianghuazhu 2ddc66fc0c HDFS-16395. Remove useless NNThroughputBenchmark#dummyActionNoSynch(). (#3823) 
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a5659fe0f7)
 2022-01-04 14:38:19 -08:00
Akira Ajisaka 5a7efbfbb5 HADOOP-18052. Support Apple Silicon in start-build-env.sh (#3817) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c8725de547)
 2022-01-04 14:38:07 -08:00
Ashutosh Gupta c63509127f YARN-8234. Improve RM system metrics publisher's performance by pushing events to timeline server in batch (#3793) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 00e2405fbd)
 2022-01-04 14:38:02 -08:00
Eric Payne 1be9d60f6c YARN-10178: Global Scheduler async thread crash caused by 'Comparison method violates its general contract. Contributed by Andras Gyori (gandras) and Qi Zhu (zhuqi). 
(cherry picked from commit e2d6fd075d)
 2022-01-04 14:37:59 -08:00
wangzhaohui bceab2e439 HDFS-16391. Avoid evaluation of LOG.debug statement in NameNodeHeartbeatService (#3820) 
(cherry picked from commit 8d251bd629)
 2022-01-04 14:37:55 -08:00
jianghuazhu 82cf0aa296 HDFS-16386. Reduce DataNode load when FsDatasetAsyncDiskService is working. (#3806) 
(cherry picked from commit 746b328554)
 2022-01-04 14:37:52 -08:00
Szilard Nemeth d3fded12dc YARN-6862. Nodemanager resource usage metrics sometimes are negative. Contributed by Benjamin Teke 2022-01-04 14:37:47 -08:00
Dhananjay Badaya 4201cdb44e HADOOP-13500. Synchronizing iteration of Configuration properties object (#3775) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4483607a4e)
 2022-01-04 14:37:43 -08:00
litao c0ae38c6b4 HDFS-16377. Should CheckNotNull before access FsDatasetSpi (#3784) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 22f5e1885d)
 2022-01-04 14:37:40 -08:00
litao 24917bae9f HDFS-16375. The FBR lease ID should be exposed to the log (#3769) 
(cherry picked from commit 10cdde59b4)
 2022-01-04 14:37:37 -08:00
Ayush Saxena 57c9a8509f HDFS-16373. Fix MiniDFSCluster restart in case of multiple namenodes. (#3756) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2022-01-04 14:37:26 -08:00
litao f16d9dfc31 HDFS-16327. Make DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_KEY reconfigurable (#3716) 
(cherry picked from commit c56a07f36b)
 2022-01-04 14:37:22 -08:00
Viraj Jasani a45df81822 HADOOP-18033. Upgrade fasterxml Jackson to 2.13.0 (#3764) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-01-04 14:37:19 -08:00
GuoPhilipse 45e6d22462 HDFS-16345. Fix test case fail in TestBlockStoragePolicy (#3696). Contributed by guophilipse. 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 25849ff7b7)
 2022-01-04 14:37:15 -08:00
Anoop Sam John de27fa0097 HADOOP-17643 WASB : Make metadata checks case insensitive (#3103) 2022-01-04 14:37:12 -08:00
Akira Ajisaka c257abca21 HADOOP-18040. Use maven.test.failure.ignore instead of ignoreTestFailure (#3774) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 9b9e2ef87f)

 Conflicts:
	hadoop-tools/hadoop-federation-balance/pom.xml
 2022-01-04 14:37:08 -08:00
qinyuren 04f0025ad2 HDFS-16333. fix balancer bug when transfer an EC block (#3777) 2022-01-04 14:37:04 -08:00
Szilard Nemeth 52c5bbc993 YARN-11020. [UI2] No container is found for an application attempt with a single AM container. Contributed by Andras Gyori 2022-01-04 14:37:01 -08:00
Takanobu Asanuma 1e3f94fa3c Revert "HDFS-16333. fix balancer bug when transfer an EC block (#3679)" 
This reverts commit 55c0e676fd.
 2022-01-04 14:36:57 -08:00
qinyuren 5214140123 HDFS-16333. fix balancer bug when transfer an EC block (#3679) 
(cherry picked from commit 35556ea043)
 2022-01-04 14:36:54 -08:00
lzx404243 47436346b4 HDFS-15862. Make TestViewfsWithNfs3.testNfsRenameSingleNN() idempotent (#2724) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 7ba91c1235)
 2022-01-04 14:36:50 -08:00
Ashutosh Gupta b410289745 YARN-9063. ATS 1.5 fails to start if RollingLevelDb files are corrupt or missing (#3728) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 5a950b8900)
 2022-01-04 14:36:46 -08:00
LeonGao d02bf2dcce HDFS-16268. Balancer stuck when moving striped blocks due to NPE (#3546) 
(cherry picked from commit 7279fe8661)
 2022-01-04 14:36:42 -08:00
Prabhu Joseph 77c1c482ef YARN-10820. Make GetClusterNodesRequestPBImpl thread safe. Contributed by Swathi Chandrashekar. 
(cherry picked from commit 6b4987f09c)
 2022-01-04 14:36:39 -08:00
Takanobu Asanuma 906eb831c6 HDFS-16293. Client sleeps and holds 'dataQueue' when DataNodes are congested. Contributed by Yuanxin Zhu. 
(cherry picked from commit e8e69de106)
 2022-01-04 14:36:36 -08:00
huhaiyang e50c9224fe HDFS-16314. Support to make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable (#3664) 
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c0f405a46b)
 2022-01-04 14:36:32 -08:00
huhaiyang 069426f5eb HDFS-16287. Support to make dfs.namenode.avoid.read.slow.datanode reconfigurable (#3596) 
(cherry picked from commit db89a9411e)
 2022-01-04 14:36:28 -08:00
Steve Loughran 19b99c1ecc HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags (#3633) 
Contributed by Steve Loughran

Change-Id: I596205d788f623114c12962941445432e2036c34
 2022-01-04 14:36:10 -08:00
litao ce0bc7b473 HDFS-16344. Improve DirectoryScanner.Stats#toString (#3695) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 829af89dc4)
 2022-01-04 14:35:59 -08:00
litao 30f0629d6e HDFS-16339. Show the threshold when mover threads quota is exceeded (#3689) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 86705eda82)
 2022-01-04 14:35:56 -08:00
GuoPhilipse e449daccf4 YARN-11007. Correct words in YARN documents (#3680) 
Reviewed-by: cxorm <lianp964@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c9d64bad37)
 2022-01-04 14:35:53 -08:00
Akira Ajisaka 5c18979782 Revert "HDFS-16336. Addendum: De-flake TestRollingUpgrade#testRollback (#3686)" 
This reverts commit 3b79e0f794.
 2022-01-04 14:35:49 -08:00
Viraj Jasani 544dffd179 HDFS-16171. De-flake testDecommissionStatus (#3280) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6342d5e523)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDecommissioningStatus.java
 2022-01-04 14:35:43 -08:00
Viraj Jasani c6914b1cb6 HDFS-16350. Datanode start time should be set after RPC server starts successfully (#3711) 
(cherry picked from commit cdc13e91b6)
 2022-01-04 14:35:40 -08:00
Ayush Saxena 328d3b84df HDFS-16336. Addendum: De-flake TestRollingUpgrade#testRollback (#3686) 2022-01-04 14:35:36 -08:00