Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,532 Commits 326 Branches 371 Tags
Commit Graph

21532 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Akira Ajisaka
710a24b6cd
YARN-10036. Install yarnpkg and upgrade nodejs in Dockerfile (#1772) 
(cherry picked from commit bc825b9628f2fca9d250f91728d4aebb155f1331)
 2021-02-04 14:28:25 +09:00
Hui Fei
b043e966e4 HDFS-15779. EC: fix NPE caused by StripedWriter.clearBuffers during reconstruct block. Contributed by Hongbing Wang 
(cherry picked from commit 15a1f7adfc02af81e89e9fb96691f6cdf19ceff5)
 2021-02-04 10:00:08 +08:00
Kihwal Lee
7c18c77d16 HDFS-15799. Make DisallowedDatanodeException terse. Contributed by Richard Ross. 
(cherry picked from commit 182623d2bc73cac4764149fcc9c7d94c1541f89c)
 2021-02-03 09:03:31 -06:00
crossfire
ba03f84812 HDFS-15795. EC: Wrong checksum when reconstruction was failed by exception. Contributed by Yushi Hayasaka (#2657) 
(cherry picked from commit 18978f2e204d105fb05807d33387a048e9ddb762)
 2021-02-02 10:06:08 +00:00
Akira Ajisaka
c62f90b4c5
HADOOP-17354. Move Jenkinsfile outside of the root directory. (#2647) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 84b154ebc054843234e9a3c4c27748f87582807f)
 2021-02-01 18:53:17 +09:00
Gautham B A
084b56c616 HADOOP-17506. Fix typo in BUILDING.txt (#2662) 
(cherry picked from commit 9801b78edf25289d02194d649bb5f679bd35600c)

 Conflicts:
	BUILDING.txt
 2021-01-31 14:26:44 +00:00
He Xiaoqiao
c7cdcbfe2b
HADOOP-17501. Fix logging typo in ShutdownHookManager. Contributed by Fengnan Li. 2021-01-31 20:10:08 +08:00
Jim Brennan
8b9c883ea4 [HDFS-15789] Lease renewal does not require namesystem lock. Contributed by Daryn Sharp and Jim Brennan. 
(cherry picked from commit f565476939332c2b17a0f470ac650ab55c6c0f64)
 2021-01-28 18:32:01 +00:00
Jim Brennan
6c564f0bc1 [HDFS-10498] Intermittent test failure TestSnapshotFileLength.testSnapshotFileLength. Contributed by Jim Brennan. 
(cherry picked from commit 6184ab07f670c181d1fda99e76729e920fff287e)
 2021-01-28 17:04:39 +00:00
Akira Ajisaka
52004affa8
HADOOP-17478. Improve the description of hadoop.http.authentication.signature.secret.file (#2628) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit d82009599a2e9f48050e0c41440b36c759ec068f)
 2021-01-24 22:57:48 +09:00
Akira Ajisaka
a5647b99da
HDFS-15731. Reduce threadCount for unit tests to reduce the memory usage (#2556) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 9cf408d6baf2fa9071729cac57516d366e928a9f)
 2021-01-24 22:53:20 +09:00
Anton Kutuzov
16e908e757 HDFS-15632. AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases. Contributed by Anton Kutuzov. 
(cherry picked from commit 91d4ba57c5b85379303ac8fb2a1a03ba10b07d4e)
 2021-01-22 18:12:46 -08:00
He Xiaoqiao
03151aa78b
HADOOP-16947. Stale record should be remove when MutableRollingAverages generating aggregate data. Contributed by Haibin Huang. 2021-01-22 00:44:15 +08:00
Akira Ajisaka
f1dff467e1
HDFS-15783. Speed up BlockPlacementPolicyRackFaultTolerant#verifyBlockPlacement (#2626) 
(cherry picked from commit 45b3a8470975e67cce0e3e4d8eb8c7b6e7b6992f)
 2021-01-21 10:09:51 +09:00
Masatake Iwasaki
371955fb50
HDFS-15672. TestBalancerWithMultipleNameNodes#testBalancingBlockpoolsWithBlockPoolPolicy fails on trunk. (#2475) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 630f8ddd2c72c27b02032ed3cf9dde15745ed605)
 2021-01-14 16:32:04 +09:00
Eric Badger
d79f705a30 YARN-10562. Follow up changes for YARN-9833. Contributed by Jim Brennan. 
(cherry picked from commit 768e2f42ba3f82802e2ebf39d44fe6c30037583c)
 2021-01-13 23:53:16 +00:00
Eric Payne
a093bd859d YARN-4589: Diagnostics for localization timeouts is lacking. Contributed by Chang Li (lichangleo) and Jim Brennan (Jim_Brennan) 2021-01-13 19:44:26 +00:00
touchida
e2681dba22
HDFS-15762. TestMultipleNNPortQOP#testMultipleNNPortOverwriteDownStream fails intermittently (#2598) 
Co-authored-by: Toshihiko Uchida <toshihiko.uchida@linecorp.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 8ec824f2ba462770c99f4fe3521f4448d5bc7b0a)
 2021-01-13 11:32:41 +09:00
lzx404243
53460b664d
MAPREDUCE-7310. Clear the fileMap in JHEventHandlerForSigtermTest (#2499) 
Co-authored-by: Zhengxi Li <zli89@illinois.edu>
Reviewed-by: Wanqiang Ji <jiwq@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ca7dd5fad331e3942547b88fdc4c0ad59865a71a)
 2021-01-12 15:55:44 +09:00
Ahmed Hussein
d8e6d85042
HADOOP-17438. Increase docker memory limit in Jenkins (#2560) 
Reviewed-by: Eric Badger <ebadger@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ae894ecfcc46e40d9e2f535ac6acec50f8136fae)
 2021-01-12 15:36:26 +09:00
He Xiaoqiao
a17793534c
Make upstream aware of 3.2.2 release. 2021-01-09 18:07:46 +08:00
Szilard Nemeth
59795ec3d6 YARN-10528. maxAMShare should only be accepted for leaf queues, not parent queues. Contributed by Siddharth Ahuja 2021-01-08 12:49:58 +01:00
Wei-Chiu Chuang
8af0fcf628 HDFS-15719. [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout (#2533) 
(cherry picked from commit 2b4febcf576e2da29ab86e2920302b82b47e435d)
(cherry picked from commit 94c126cc9e9aadaf9aa28ebd55a8712aca54e709)
 2021-01-04 20:56:32 -08:00
He Xiaoqiao
1c0ac5ed5c
HDFS-15751. Add documentation for msync() API to filesystem.md. Contributed by Konstantin V Shvachko. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
 2021-01-03 16:31:01 +08:00
He Xiaoqiao
6b0f0c33c3
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:48:09 +08:00
Eric Badger
264dd67018 YARN-10540. Node page is broken in YARN UI1 and UI2 including RMWebService api 
for nodes. Contributed by Jim Brennan.

(cherry picked from commit 4c5d88e230fa7adbb4a041c9a33bea2ef5a36fd3)
 2020-12-21 23:22:43 +00:00
Ahmed Hussein
0a3c6b218a HDFS-15704. Mitigate lease monitor's rapid infinite loop. (#2511). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit c2672bb234256e342fd9b5ef6fe4455471009693)
 2020-12-17 18:07:36 +00:00
He Xiaoqiao
8d7600ae6b HADOOP-17068. Client fails forever when namenode ipaddr changed. Contributed by Sean Chow. 
(cherry picked from commit fa14e4bc001e28d9912e8d985d09bab75aedb87c)
(cherry picked from commit 3a860e876e799ef44fca34e3cb19a89dbd4f6744)
 2020-12-15 14:02:01 -08:00
Ayush Saxena
079d3ea6ac HDFS-15200. Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage. Contributed by Ayush Saxena. 2020-12-15 10:53:26 +05:30
Ayush Saxena
40285be1d5 HDFS-15187. CORRUPT replica mismatch between namenodes after failover. Contributed by Ayush Saxena. 2020-12-15 10:34:22 +05:30
Ayush Saxena
ed78df0f75 HDFS-15170. EC: Block gets marked as CORRUPT in case of failover and pipeline recovery. Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 5e5c53070f1946b86e49c039e1f5ae42e5d022cb)
(cherry picked from commit faf26f70aaa628b750ea5ad2eb0d68e0fdbc55d5)
 2020-12-14 20:41:21 -08:00
S O'Donnell
52adda89c4 HDFS-15725. Lease Recovery never completes for a committed block which the DNs never finalize. Contributed by Stephen O'Donnell 2020-12-14 11:37:08 +00:00
Jim Brennan
f5a98b6000 HADOOP-13571. ServerSocketUtil.getPort() should use loopback address, not 0.0.0.0. Contributed by Eric Badger 
(cherry picked from commit 6de1a8eb678496393b625f430084a6c64e24b804)
 2020-12-11 20:42:59 +00:00
Jim Brennan
eec10a087e HDFS-15717. Improve fsck logging. (#2529) Contributed by Kihwal Lee and Ahmed Hussein 2020-12-11 18:10:24 +00:00
Neil
9123f9245c HDFS-15720 namenode audit async logger should add some log4j config (#2532) 
(cherry picked from commit 9bd3c9bc50672ebdd7f12b9a2235aec37b2a07cf)
(cherry picked from commit bc5458bbd4e6829bc02de57671c017fbdac06849)
 2020-12-10 13:50:42 -08:00
Jim Brennan
76316c4bc0 HDFS-15711. Add Metrics to HttpFS Server. (#2521) Contributed by Ahmed Hussein and Kihwal Lee 2020-12-10 21:30:03 +00:00
Ankit Kumar
1570bddb79
YARN-10491. Fix deprecation warnings in SLSWebApp.java (#2519) 
Signed-off-by: Akira Ajisaka <ajisaka@apache.org>
(cherry picked from commit aaf9e3d320a9d11d2fd8c86fb5fa4e2d1c45d9ef)
 2020-12-09 10:54:17 +09:00
Ahmed Hussein
bb23d4f14b HDFS-15716. WaitforReplication in TestUpgradeDomainBlockPlacementPolicy (#2528) 
(cherry picked from commit 01383a21722be706d7ab682323285e794f71cadf)
 2020-12-08 19:41:54 +00:00
crossfire
31a6b45117 HDFS-15709. Socket file descriptor leak in StripedBlockChecksumReconstructor. (#2518) 
(cherry picked from commit 40f7543a6d5765c98d41c78736124b7b7f078aa2)
(cherry picked from commit edd9b659caf77d14caf6df88bac3cedf025e1f25)
 2020-12-07 15:50:47 -08:00
Jim Brennan
d7c955b646 HDFS-15707. NNTop counts don't add up as expected. (#2516) Contributed by Ahmed Hussein and Daryn Sharp 
(cherry picked from commit 6a5864ee4ac387a390642f56aff8581fb5ed8ffd)
 2020-12-07 21:22:15 +00:00
Ahmed Hussein
d9bf607066 HADOOP-17389. KMS should log full UGI principal. (#2476) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Jim Brennan <jbrennan@apache.org>
(cherry picked from commit 8c234fc0d4472b65ff555a79acb9a12f2dae3b27)
 2020-12-07 17:29:05 +00:00
Chao Sun
3b2a131885
HDFS-15708. TestURLConnectionFactory fails by NoClassDefFoundError in branch-3.3 and branch-3.2 (#2517) 2020-12-04 10:42:46 -08:00
Takanobu Asanuma
3f14715215
HADOOP-15775. [JDK9] Add missing javax.activation-api dependency. Contributed by Akira Ajisaka. 2020-12-04 22:28:19 +08:00
Hui Fei
6e215953e6
HDFS-15240. Erasure Coding: dirty buffer causes reconstruction block error. Contributed by HuangTao. 2020-12-04 17:11:21 +08:00
Ayush Saxena
af437f1722
HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma. 
(cherry picked from commit b52fd05d42d9a76f6936a5d86c23fcd66244fe3d)
 2020-12-04 11:07:29 +09:00
Takanobu Asanuma
bdcca33e7c
HDFS-13404. RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails. 
(cherry picked from commit 559cb11551d7045015aa60afe7e7998fe0b2ef13)
 2020-12-04 11:07:19 +09:00
Akira Ajisaka
066d6f5d0d
MAPREDUCE-7284. TestCombineFileInputFormat#testMissingBlocks fails (#2136) 
(cherry picked from commit 3e7b65c3e5920ed78bd87b68de1075003d86e7a0)
 2020-12-04 10:08:22 +09:00
Chao Sun
d82b0cc439
HADOOP-16080. hadoop-aws does not work with hadoop-client-api (#2510). Contributed by Chao Sun 2020-12-03 16:06:33 -08:00
Ahmed Hussein
f547cd43d1 HDFS-15706. HttpFS: Log more information on request failures. (#2515) 
(cherry picked from commit 07655a7040806c4c9687288de89e42cfacf0365e)
(cherry picked from commit a7dbd3b6f978d28ef7e046add2e3270ae3d9d9f6)
 2020-12-03 15:15:31 -08:00
Jim Brennan
9fe779d0e2 HADOOP-17392. Remote exception messages should not include the exception class (#2486). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit 5bfb97bc7d5ff5e083aff90e5a380a850d9e4530)
 2020-12-03 18:08:37 +00:00