Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
25,476 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

8642 Commits

Author SHA1 Message Date
huhaiyang c0f405a46b
HDFS-16314. Support to make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable (#3664) 
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-04 02:19:36 +09:00
bitterfox dd6b987c93
HDFS-16332. Handle invalid token exception in sasl handshake (#3677) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-03 23:30:13 +09:00
wangzhaohui 0cb6c28d19
HDFS-16364. Remove unnecessary brackets in NameNodeRpcServer#L453 (#3742) 2021-12-03 16:51:04 +05:30
litao 52ec65fd10
HDFS-16331. Make dfs.blockreport.intervalMsec reconfigurable (#3676) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-12-03 14:12:05 +09:00
litao 1ebdac2cd6
HDFS-16361. Fix log format for QueryCommand (#3732). Contributed by tomscut. 2021-12-01 12:48:21 +05:30
litao 829af89dc4
HDFS-16344. Improve DirectoryScanner.Stats#toString (#3695) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
 2021-11-29 17:48:10 +09:00
litao 86705eda82
HDFS-16339. Show the threshold when mover threads quota is exceeded (#3689) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
 2021-11-26 19:52:14 +09:00
Takanobu Asanuma 9c887e5b82
HADOOP-18014. CallerContext should not include some characters. (#3698) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
 2021-11-25 14:05:04 +09:00
Viraj Jasani cdc13e91b6
HDFS-16350. Datanode start time should be set after RPC server starts successfully (#3711) 2021-11-25 10:23:04 +08:00
huhaiyang 99b161dec7
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson (#3708) 2021-11-25 10:20:42 +08:00
Viraj Jasani e8566b3812
HDFS-16336. De-flake TestRollingUpgrade#testRollback (#3686) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-11-24 22:54:11 +09:00
Mukul Kumar Singh faa4eeacb1
HDFS-16343. Add some debug logs when the dfsUsed are not used during Datanode startup. (#3694) 2021-11-23 13:36:14 +00:00
litao 0ed817babb
HDFS-16337. Show start time of Datanode on Web (#3682) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
 2021-11-23 10:18:30 +08:00
Symious c88640c4ad
HDFS-16320. Datanode retrieve slownode information from NameNode (#3654) 2021-11-22 18:04:32 +08:00
litao 20b78c8f97
HDFS-16335. Fix HDFSCommands.md (#3681) 2021-11-22 17:07:52 +09:00
Hui Fei 8e9ad2d7af
Revert "HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson (#3630)" (#3697) 2021-11-22 10:05:21 +08:00
GuoPhilipse 3d1ca6f6dc
HDFS-16334. Correct NameNode ACL description (#3678) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-11-22 07:41:57 +09:00
litao 965e3874e9
HDFS-16326. Simplify the code for DiskBalancer (#3663) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
 2021-11-19 11:37:10 +09:00
GuoPhilipse 0c1eb0125b
HDFS-16328. Correct disk balancer desc (#3669) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-11-19 01:02:01 +09:00
Viraj Jasani 38614d05e1
HDFS-16330. Fix incorrect placeholder for Exception logs in DiskBalancer (#3672) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-11-19 00:55:39 +09:00
litao dc751df63b
HDFS-16329. Fix log format for BlockManager (#3670) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-11-19 00:42:01 +09:00
litao 5b05068fdc
HDFS-16310. RBF: Add client port to CallerContext for Router (#3635) 2021-11-18 13:46:35 +09:00
huhaiyang 91af256a5b
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson (#3630) 2021-11-17 22:41:06 +08:00
Viraj Jasani 646c470e5d
HDFS-16323. DatanodeHttpServer doesn't require handler state map while retrieving filter handlers (#3659) 2021-11-17 22:17:09 +09:00
Neil 5626734a36
HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. (#3320) 2021-11-16 18:21:50 +08:00
litao c9f95b01ef
HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode (#3643) 
Reviewed-by: Hui Fei <ferhui@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-11-16 11:19:14 +09:00
Viraj Jasani e14a2dcbba
HADOOP-18006. maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms (#3648) 
Reviewed-by: Ahmed Hussein <ahussein@apache.org>
 2021-11-15 22:57:24 +09:00
GuoPhilipse 6e6f2e4baa
HDFS-16321. Fix invalid config in TestAvailableSpaceRackFaultTolerantBPP (#3655). Contributed by guo. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-11-14 19:59:15 +05:30
huhaiyang db89a9411e
HDFS-16287. Support to make dfs.namenode.avoid.read.slow.datanode reconfigurable (#3596) 2021-11-12 10:07:34 +08:00
jianghuazhu 3d692c0936
HDFS-16305.Record the remote NameNode address when the rolling log is triggered. (#3629) 2021-11-12 09:52:02 +08:00
litao 1c1cf64616
HDFS-16311. Metric metadataOperationRate calculation error in DataNodeVolumeMetrics (#3636) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-11-12 10:23:25 +09:00
litao 2a1a11c039
HDFS-16312. Fix typo for DataNodeVolumeMetrics and ProfilingFileIoEvents (#3637) 
Reviewed-by: Hui Fei <ferhui@apache.org>
 2021-11-10 19:22:32 +09:00
jianghuazhu b307f18a58
HDFS-16301. Improve BenchmarkThroughput#SIZE naming standardization. (#3622) 2021-11-10 11:32:52 +09:00
litao cbc7fb6bca
HDFS-16299. Fix bug for TestDataNodeVolumeMetrics#verifyDataNodeVolumeMetrics (#3616) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-11-09 17:06:28 +09:00
GuoPhilipse 9b2fa7e47a
HDFS-16307. Improve HdfsBlockPlacementPolicies docs readability 2021-11-08 10:21:39 -08:00
jianghuazhu 72718a8dc1
HDFS-16294.Remove invalid DataNode#CONFIG_PROPERTY_SIMULATED. (#3605) 2021-11-04 16:43:25 +09:00
jianghuazhu 18b2cca74c
HDFS-16291.Make the comment of INode#ReclaimContext more standardized. (#3602) 2021-11-04 09:51:11 +08:00
litao 359b03c83e
HDFS-16266. Add remote port information to HDFS audit log (#3538) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-11-04 10:16:03 +09:00
daimin a21895a5b3
HDFS-16286. Add a debug tool to verify the correctness of erasure coding on file (#3593) 2021-11-03 20:19:56 +00:00
Stephen O'Donnell 2f35cc36cd
HDFS-16259. Catch and re-throw sub-classes of AccessControlException thrown by any permission provider plugins (eg Ranger) (#3598) 2021-11-02 11:14:48 +00:00
jianghuazhu 618fea27d2
HDFS-16269. [Fix] Improve NNThroughputBenchmark#blockReport operation. (#3544) 
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-11-02 00:55:50 +09:00
jianghuazhu 0a68413a4f
HDFS-16290.Make log more standardized when executing verifyAndSetNamespaceInfo(). (#3600) 
HDFS-16290.Make log more standardized when executing verifyAndSetNamespaceInfo(). (#3600). Contributed by JiangHua Zhu.
 2021-10-31 08:48:52 +05:30
Masatake Iwasaki 634f8a2209
HDFS-16091. WebHDFS should support getSnapshotDiffReportListing. (#3374) 2021-10-31 09:29:34 +09:00
litao 6aa664c624
HDFS-16279. Print detail datanode info when process first storage report (#3564) 2021-10-29 10:27:00 +09:00
litao c1a8285363
HDFS-16281. Fix flaky unit tests failed due to timeout (#3574) 2021-10-26 14:54:11 +08:00
Ahmed Hussein 62c86eaa0e
HADOOP-17970. unguava: remove Preconditions from hdfs-projects modules (#3566) 2021-10-23 16:13:07 -05:00
daimin 6e57449350
HDFS-16282. Remove duplicate generic usage information to hdfs debug command (#3576). Contributed by daimin. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-10-22 21:12:13 +05:30
Ayush Saxena f67e6c315b
Revert "Remove duplicate generic usage information to hdfs debug command (#3576). Contributed by daimin." 
Misses Jira ID in the commit Message: HDFS-16282
This reverts commit 7b913bc6bc.
 2021-10-22 21:10:38 +05:30
daimin 7b913bc6bc
Remove duplicate generic usage information to hdfs debug command (#3576). Contributed by daimin. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-10-22 21:09:37 +05:30
GuoPhilipse 1385cdfeb9
HDFS-16277. improve decision in AvailableSpaceBlockPlacementPolicy (#3559). Contributed by guo. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-10-22 07:59:27 +05:30