6422eaf301
HDFS-16767: RBF: Support observer node in Router-Based Federation.
...
Fixes #4127
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-09-14 16:00:14 -07:00
8ce71c8882
[Documentation] RBF: Duplicate statement to be removed for better readabilty ( #4881 )
2022-09-13 09:24:05 -07:00
1923096adb
Allow block reconstruction pending timeout to be refreshable ( #4567 )
...
Reviewed-by: Hiroyuki Adachi <hadachi@yahoo-corp.jp>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-09-12 11:45:01 +09:00
e77d54d1ee
HDFS-13522: Add federated nameservices states to client protocol and propagate it between routers and clients.
...
Fixes #4311
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-09-09 15:43:28 -07:00
4a01fadb94
HDFS-16756. RBF proxies the client's user by the login user to enable CacheEntry ( #4853 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-09 12:47:36 +05:30
c947c326e8
HDFS-16659. JournalNode should throw NewerTxnIdException when SinceTxId is bigger than HighestWrittenTxId ( #4560 )
...
Co-authored-by: Zander Xu <zengqiang.xu@shopee.com>
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2022-09-06 10:12:55 -07:00
be4c638e4c
HDFS-16748. RBF: DFSClient should uniquely identify writing files by namespace id and iNodeId via RBF ( #4813 ). Contributed by ZanderXu.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-05 17:46:15 +05:30
ac42519ade
HDFS-16593. Correct the BlocksRemoved metric on DataNode side ( #4353 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-09-05 19:35:48 +08:00
7b239a80fe
HDFS-16750. NameNode should use NameNode.getRemoteUser() to log audit event to avoid possible NPE ( #4821 )
2022-09-02 10:23:03 -07:00
2dd8b1342e
HDFS-16755. TestQJMWithFaults.testUnresolvableHostName() can fail due to unexpected host resolution ( #4833 )
...
Use ".invalid" domain from IETF RFC 2606 to ensure that the host doesn't resolve.
Contributed by Steve Vaughan Jr
2022-09-01 14:00:15 +01:00
71778a6cc5
HDFS-16735. Reduce the number of HeartbeatManager loops. ( #4780 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-29 11:30:21 +08:00
5567154f71
HDFS-16734. RBF: fix some bugs when handling getContentSummary RPC ( #4763 )
2022-08-26 16:04:33 -07:00
5736b34b2a
HDFS-16736. Link to Boost library in libhdfspp ( #4782 )
2022-08-26 09:11:44 -07:00
231a4468cd
HDFS-16732. [SBN READ] Avoid get location from observer when the block report is delayed ( #4756 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2022-08-25 10:37:25 -07:00
1691cccc89
HDFS-16738. Invalid CallerContext caused NullPointerException ( #4791 )
2022-08-25 17:12:27 +08:00
8d4f51c432
HDFS-16728. RBF throw IndexOutOfBoundsException with disableNameServices ( #4734 ). Contributed by ZanderXu.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-24 20:27:15 +05:30
c37f01d95b
HDFS-16724. RBF should support get the information about ancestor mount points ( #4719 )
2022-08-23 13:25:42 -07:00
a3b1bafa34
HDFS-16669: Enhance client protocol to propagate last seen state IDs for multiple nameservices.
...
Fixes #4584
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-08-23 11:12:50 -07:00
6fbc38db95
HDFS-16686. GetJournalEditServlet fails to authorize valid Kerberos request ( #4724 )
2022-08-23 08:03:57 -07:00
183f09b1da
HDFS-16717. Replace NPE with IOException in DataNode.class ( #4699 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-23 18:17:32 +08:00
a9e5fb3313
HDFS-16684. Exclude the current JournalNode ( #4723 )
...
Exclude bound local addresses, including the use of a wildcard address in the bound host configurations.
* Allow sync attempts with unresolved addresses
* Update the comments.
* Remove unused import
Signed-off-by: stack <stack@apache.org>
2022-08-22 09:52:45 -07:00
7f176d080c
HDFS-16729. RBF: fix some unreasonably annotated docs. ( #4745 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-08-21 07:29:31 +09:00
5cc8c574d1
HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously ( #4626 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-08-18 02:25:09 +08:00
86abeb401e
HDFS-16730. Update the doc that append to EC files is supported ( #4749 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-08-17 05:41:41 +08:00
622ca0d51f
HDFS-16705. RBF supports healthMonitor timeout configurable and caching NN and client proxy in NamenodeHeartbeatService ( #4662 )
2022-08-15 13:55:16 -07:00
b1d4af2492
HDFS-16704. Datanode return empty response instead of NPE for GetVolumeInfo during restarting ( #4661 ). Contributed by ZanderXu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-15 20:57:27 +08:00
86cc96c493
Revert "HDFS-16689. NameNode may crash when transitioning to Active with in-progress tailer if there are some abnormal JNs. ( #4628 )" ( #4743 )
2022-08-15 19:48:05 +08:00
e0c8c6eed4
HDFS-16678. RBF should supports disable getNodeUsage() in RBFMetrics ( #4606 )
2022-08-12 12:01:58 -07:00
521e65acfe
HDFS-16723. Replace incorrect SafeModeException with StandbyException in RouterRpcServer.class ( #4716 ). Contributed by ZanderXu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-13 00:21:11 +05:30
59619ad247
HDFS-16689. NameNode may crash when transitioning to Active with in-progress tailer if there are some abnormal JNs. ( #4628 )
2022-08-12 12:19:28 +08:00
2005582a28
HDFS-16702. MiniDFSCluster should report cause of exception in assert error ( #4680 )
...
When the MiniDFSClsuter detects that an exception caused an exit, it should include that exception as the cause for the AssertionError that it throws. The current AssertError simply reports the message "Test resulted in an unexpected exit" and provides a stack trace to the location of the check for an exit exception. This patch adds the original exception as the cause of the AssertError.
2022-08-11 13:52:39 -07:00
09cabaad68
HDFS-13274. RBF: Extend RouterRpcClient to use multiple sockets ( #4531 )
2022-08-11 13:23:32 -07:00
895f7c51fd
HDFS-16709. Remove redundant cast in FSEditLogOp.class ( #4667 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-09 00:54:01 +05:30
25ccdc77af
HDFS-16648. Add isDebugEnabled check for debug blockLogs in some classes ( #4529 )
2022-08-06 21:34:01 +08:00
62dbefd8f2
HADOOP-18305. Release Hadoop 3.3.4: upstream changelog and jdiff files
...
Add the r3.3.4 changelog, release notes and jdiff xml files.
2022-08-05 14:06:22 +01:00
dbf73e16b1
HADOOP-18364. All method metrics related to the RPC protocol should be initialized. ( #4624 ). Contributed by Shuyan Zhang.
...
Reviewed-by: Erik Krogen <xkrogen@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-04 13:00:37 +08:00
8eebf40b1a
HDFS-16642. Moving the selecting inputstream from journalnode in EditLogTailer outof FSNLock ( #4497 )
2022-08-04 11:04:28 +08:00
0fc7dd8228
HDFS-16687. RouterFsckServlet replicates code from DfsServlet base class ( #4681 )
2022-08-03 09:14:11 -07:00
0f36539d60
HDFS-16712. Fix incorrect placeholder in DataNode.java ( #4672 ). Contributed by ZanderXu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-03 13:01:41 +05:30
69f6fdb757
HADOOP-18301. Upgrade commons-io to 2.11.0 ( #4455 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-08-03 10:44:39 +09:00
f80fab2b90
HDFS-16671. RBF: RouterRpcFairnessPolicyController supports configurable permit acquire timeout ( #4597 )
2022-07-28 15:42:53 +08:00
a5adc27c99
HDFS-16658. Change logLevel from DEBUG to INFO if logEveryBlock is true ( #4559 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-07-28 15:30:22 +08:00
06ac327e88
HDFS-16619. Fix HttpHeaders.Values And HttpHeaders.Names Deprecated Import ( #4406 )
...
Co-authored-by: slfan1989 <louj1988@@>
2022-07-28 11:07:24 +08:00
24560f2eb5
HDFS-16660. Improve Code With Lambda in IPCLoggerChannel class ( #4561 )
2022-07-27 18:53:05 -07:00
01a2e0f6bd
HDFS-16533. COMPOSITE_CRC failed between replicated file and striped file due to invalid requested length. ( #4155 )
...
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-07-26 04:30:00 +08:00
6ba2c53720
HDFS-16681. Do not pass GCC flags for MSVC in libhdfspp ( #4615 )
...
* This PR ensures that the GCC flag
-Wno-missing-field-initializers isn't passed
for MSVC.
2022-07-25 22:55:38 +05:30
2f49eec5dd
HDFS-16655. OIV: print out erasure coding policy name in oiv Delimited output ( #4541 ). Contributed by Max Xie.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-07-25 17:39:25 +08:00
8f83d9f56d
HDFS-16680. Skip libhdfspp Valgrind tests on Windows ( #4611 )
...
* The CMake test libhdfs_mini_stress_valgrind
requires Valgrind.
* This PR skips this test on Windows since
Valgrind isn't available.
2022-07-23 23:22:13 +05:30
7de9b5ee27
HDFS-16467. Ensure Protobuf generated headers are included first ( #4601 )
...
* This PR ensures that the Protobuf generated headers
are always included first, even when these headers
are included transitively.
* This problem is specific to Windows only.
2022-07-23 23:20:15 +05:30
2c96357051
HDFS-15079. RBF: Namenode needs to use the actual client Id and callId when going through RBF proxy. ( #4530 )
2022-07-23 22:19:37 +08:00
Simbarashe Dzinamarira
Renukaprasad C
caozhiqiang
ZanderXu
Steve Vaughan
zhangshuyan0
Gautham B A
zhengchenyu
jianghuazhu
Ashutosh Gupta
Hui Fei
Steve Loughran
slfan1989
Neil