740e1ef357
HDFS-16706. ViewFS doc points to wrong mount table name ( #4803 )
2022-09-21 16:55:20 +05:30
40ab9c8ba7
HDFS-16341. Fix BlockPlacementPolicy details in hdfs defaults. ( #3691 ). Contributed by guophilipse.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-20 23:16:24 +05:30
e68006cd70
HDFS-16772. refreshHostsReader should use the latest configuration ( #4890 )
2022-09-19 13:27:07 -07:00
a73c4804d8
HADOOP-18446. [SBN read] Add a re-queue metric to RpcMetrics to quantify the number of re-queued RPCs ( #4871 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-09-16 10:09:01 -07:00
43c1ebae16
HDFS-16771. Follow-up for HDFS-16659: JN should tersely print logs about NewerTxnIdException ( #4882 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-09-15 12:44:36 -07:00
6422eaf301
HDFS-16767: RBF: Support observer node in Router-Based Federation.
...
Fixes #4127
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-09-14 16:00:14 -07:00
1923096adb
Allow block reconstruction pending timeout to be refreshable ( #4567 )
...
Reviewed-by: Hiroyuki Adachi <hadachi@yahoo-corp.jp>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-09-12 11:45:01 +09:00
c947c326e8
HDFS-16659. JournalNode should throw NewerTxnIdException when SinceTxId is bigger than HighestWrittenTxId ( #4560 )
...
Co-authored-by: Zander Xu <zengqiang.xu@shopee.com>
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2022-09-06 10:12:55 -07:00
be4c638e4c
HDFS-16748. RBF: DFSClient should uniquely identify writing files by namespace id and iNodeId via RBF ( #4813 ). Contributed by ZanderXu.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-05 17:46:15 +05:30
ac42519ade
HDFS-16593. Correct the BlocksRemoved metric on DataNode side ( #4353 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-09-05 19:35:48 +08:00
7b239a80fe
HDFS-16750. NameNode should use NameNode.getRemoteUser() to log audit event to avoid possible NPE ( #4821 )
2022-09-02 10:23:03 -07:00
2dd8b1342e
HDFS-16755. TestQJMWithFaults.testUnresolvableHostName() can fail due to unexpected host resolution ( #4833 )
...
Use ".invalid" domain from IETF RFC 2606 to ensure that the host doesn't resolve.
Contributed by Steve Vaughan Jr
2022-09-01 14:00:15 +01:00
71778a6cc5
HDFS-16735. Reduce the number of HeartbeatManager loops. ( #4780 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-29 11:30:21 +08:00
231a4468cd
HDFS-16732. [SBN READ] Avoid get location from observer when the block report is delayed ( #4756 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2022-08-25 10:37:25 -07:00
1691cccc89
HDFS-16738. Invalid CallerContext caused NullPointerException ( #4791 )
2022-08-25 17:12:27 +08:00
6fbc38db95
HDFS-16686. GetJournalEditServlet fails to authorize valid Kerberos request ( #4724 )
2022-08-23 08:03:57 -07:00
183f09b1da
HDFS-16717. Replace NPE with IOException in DataNode.class ( #4699 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-23 18:17:32 +08:00
a9e5fb3313
HDFS-16684. Exclude the current JournalNode ( #4723 )
...
Exclude bound local addresses, including the use of a wildcard address in the bound host configurations.
* Allow sync attempts with unresolved addresses
* Update the comments.
* Remove unused import
Signed-off-by: stack <stack@apache.org>
2022-08-22 09:52:45 -07:00
5cc8c574d1
HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously ( #4626 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-08-18 02:25:09 +08:00
86abeb401e
HDFS-16730. Update the doc that append to EC files is supported ( #4749 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-08-17 05:41:41 +08:00
b1d4af2492
HDFS-16704. Datanode return empty response instead of NPE for GetVolumeInfo during restarting ( #4661 ). Contributed by ZanderXu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-15 20:57:27 +08:00
86cc96c493
Revert "HDFS-16689. NameNode may crash when transitioning to Active with in-progress tailer if there are some abnormal JNs. ( #4628 )" ( #4743 )
2022-08-15 19:48:05 +08:00
59619ad247
HDFS-16689. NameNode may crash when transitioning to Active with in-progress tailer if there are some abnormal JNs. ( #4628 )
2022-08-12 12:19:28 +08:00
2005582a28
HDFS-16702. MiniDFSCluster should report cause of exception in assert error ( #4680 )
...
When the MiniDFSClsuter detects that an exception caused an exit, it should include that exception as the cause for the AssertionError that it throws. The current AssertError simply reports the message "Test resulted in an unexpected exit" and provides a stack trace to the location of the check for an exit exception. This patch adds the original exception as the cause of the AssertError.
2022-08-11 13:52:39 -07:00
895f7c51fd
HDFS-16709. Remove redundant cast in FSEditLogOp.class ( #4667 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-09 00:54:01 +05:30
25ccdc77af
HDFS-16648. Add isDebugEnabled check for debug blockLogs in some classes ( #4529 )
2022-08-06 21:34:01 +08:00
62dbefd8f2
HADOOP-18305. Release Hadoop 3.3.4: upstream changelog and jdiff files
...
Add the r3.3.4 changelog, release notes and jdiff xml files.
2022-08-05 14:06:22 +01:00
dbf73e16b1
HADOOP-18364. All method metrics related to the RPC protocol should be initialized. ( #4624 ). Contributed by Shuyan Zhang.
...
Reviewed-by: Erik Krogen <xkrogen@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-04 13:00:37 +08:00
8eebf40b1a
HDFS-16642. Moving the selecting inputstream from journalnode in EditLogTailer outof FSNLock ( #4497 )
2022-08-04 11:04:28 +08:00
0f36539d60
HDFS-16712. Fix incorrect placeholder in DataNode.java ( #4672 ). Contributed by ZanderXu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-03 13:01:41 +05:30
69f6fdb757
HADOOP-18301. Upgrade commons-io to 2.11.0 ( #4455 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-08-03 10:44:39 +09:00
a5adc27c99
HDFS-16658. Change logLevel from DEBUG to INFO if logEveryBlock is true ( #4559 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-07-28 15:30:22 +08:00
06ac327e88
HDFS-16619. Fix HttpHeaders.Values And HttpHeaders.Names Deprecated Import ( #4406 )
...
Co-authored-by: slfan1989 <louj1988@@>
2022-07-28 11:07:24 +08:00
24560f2eb5
HDFS-16660. Improve Code With Lambda in IPCLoggerChannel class ( #4561 )
2022-07-27 18:53:05 -07:00
01a2e0f6bd
HDFS-16533. COMPOSITE_CRC failed between replicated file and striped file due to invalid requested length. ( #4155 )
...
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-07-26 04:30:00 +08:00
2f49eec5dd
HDFS-16655. OIV: print out erasure coding policy name in oiv Delimited output ( #4541 ). Contributed by Max Xie.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-07-25 17:39:25 +08:00
2c96357051
HDFS-15079. RBF: Namenode needs to use the actual client Id and callId when going through RBF proxy. ( #4530 )
2022-07-23 22:19:37 +08:00
3cce41a1f6
Make upstream aware of 3.2.4 release.
...
(cherry picked from commit e1637a57df
2022-07-22 02:27:19 +00:00
08a940d5dd
HDFS-16640. RBF: Show datanode IP list when click DN histogram in Router ( #4488 )
2022-07-21 16:21:31 -07:00
9376b65989
HDFS-16566 Erasure Coding: Recovery may causes excess replicas when busy DN exsits ( #4252 )
2022-07-16 04:52:12 +08:00
2835174a4c
HDFS-16652. Upgrade jquery datatable version references to v1.10.19 ( #4562 )
2022-07-14 18:27:07 +05:30
6f9c4359ec
HDFS-16283. RBF: reducing the load of renewLease() RPC ( #4524 ). Contributed by ZanderXu.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-07-14 07:26:40 +05:30
ea46f49b04
HDFS-16638. Add isDebugEnabled check for debug blockLogs in BlockManager. ( #4480 ). Contributed by dzcxzl.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-07-04 02:36:29 +05:30
a333674785
HDFS-16647. Delete unused NameNode#FS_HDFS_IMPL_KEY. ( #4525 ). Contributed by JiangHua Zhu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-07-03 01:47:02 +05:30
073b8ea1d5
HADOOP-18284. Remove Unnecessary semicolon ';' ( #4422 ). Contributed by fanshilun.
2022-06-29 15:20:41 +05:30
321a4844ad
HADOOP-18314. Add some description for PowerShellFencer. ( #4505 )
2022-06-28 19:06:39 -07:00
b7edc6c60c
HDFS-16633. Fixing when Reserved Space For Replicas is not released on some cases ( #4452 )
...
* HDFS-16633.Reserved Space For Replicas is not released on some cases
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-06-24 18:35:00 +05:30
e8fd914c58
HDFS-16616. remove use of org.apache.hadoop.util.Sets ( #4400 )
...
Co-Authored by: Samrat Deb
2022-06-22 10:17:36 +05:30
7a1d811197
HDFS-16637. TestHDFSCLI#testAll consistently failing ( #4466 ). Contributed by Viraj Jasani.
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-06-21 13:42:43 +05:30
efc2761d32
HDFS-16635.Fixed javadoc error in Java 11 ( #4451 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-06-20 11:14:31 +05:30
Samrat
GuoPhilipse
ZanderXu
Simbarashe Dzinamarira
caozhiqiang
Steve Vaughan
zhangshuyan0
zhengchenyu
Ashutosh Gupta
Hui Fei
Steve Loughran
slfan1989
Neil
Masatake Iwasaki
wangzhaohui
RuinanGu
Murali Krishna
cxzl25
jianghuazhu
Viraj Jasani