c2afb6a00b
HDFS-16370. Fix assert message for BlockInfo ( #3747 ). Contributed by tomscut.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-12-04 22:54:44 +05:30
2f55a6cced
HDFS-16357. Fix log format in DFSUtilClient ( #3729 ). Contributed by guophilipse.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-12-04 18:35:05 +05:30
cab7086fbc
HDFS-16369. RBF: Fix the retry logic of RouterRpcServer#invokeAtAvailableNs. ( #3745 ). Contributed by Ayush Saxena.
...
Reviewed-by: litao <tomleescut@gmail.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-12-04 10:54:29 +05:30
c0f405a46b
HDFS-16314. Support to make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable ( #3664 )
...
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-04 02:19:36 +09:00
47ea0d734f
HADOOP-18021. Provide a public wrapper of Configuration#substituteVars ( #3710 )
...
Contributed by Andras Gyori
2021-12-03 16:44:58 +00:00
dd6b987c93
HDFS-16332. Handle invalid token exception in sasl handshake ( #3677 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-03 23:30:13 +09:00
0cb6c28d19
HDFS-16364. Remove unnecessary brackets in NameNodeRpcServer#L453 ( #3742 )
2021-12-03 16:51:04 +05:30
52ec65fd10
HDFS-16331. Make dfs.blockreport.intervalMsec reconfigurable ( #3676 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-12-03 14:12:05 +09:00
0c62a514f7
HDFS-16358. HttpFS implementation for getSnapshotDiffReportListing ( #3730 )
2021-12-03 13:25:32 +09:00
196935a8d2
HDFS-16359. RBF: RouterRpcServer#invokeAtAvailableNs does not take effect when retrying ( #3731 ). Contributed by tomscut.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-12-02 19:10:35 +05:30
b34dcb5b3a
HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile ( #3722 )
2021-12-02 10:40:30 +08:00
df4197592f
HADOOP-18029: Update CompressionCodecFactory to handle uppercase file extensions ( #3739 )
...
Co-authored-by: Desmond Sisson <sissonde@amazon.com>
2021-12-01 15:36:54 -08:00
4dea4a7b67
HADOOP-18027. Include static imports in the maven plugin rules ( #3733 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-12-01 16:48:43 +09:00
1ebdac2cd6
HDFS-16361. Fix log format for QueryCommand ( #3732 ). Contributed by tomscut.
2021-12-01 12:48:21 +05:30
7b840f2693
YARN-10975 EntityGroupFSTimelineStore#ActiveLogParser parses already processed files ( #3735 )
...
Contributed by sravuri <sravuri@microsoft.com> and reviewed by Prabhu Joseph
2021-11-30 00:00:25 +05:30
c65c87f211
HADOOP-18026. Fix default value of Magic committer ( #3723 )
...
Contributed by guophilipse
2021-11-29 15:50:30 +00:00
932a78fe38
HADOOP-18023. Allow cp command to run with multi threads. ( #3721 )
2021-11-29 12:45:08 +00:00
829af89dc4
HDFS-16344. Improve DirectoryScanner.Stats#toString ( #3695 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-29 17:48:10 +09:00
215388beea
HADOOP-18022. Add restrict-imports-enforcer-rule for Guava Preconditions and remove remaining usages ( #3712 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-11-29 17:37:30 +09:00
86705eda82
HDFS-16339. Show the threshold when mover threads quota is exceeded ( #3689 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-26 19:52:14 +09:00
c9d64bad37
YARN-11007. Correct words in YARN documents ( #3680 )
...
Reviewed-by: cxorm <lianp964@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-25 17:48:54 +09:00
9c887e5b82
HADOOP-18014. CallerContext should not include some characters. ( #3698 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-25 14:05:04 +09:00
cdc13e91b6
HDFS-16350. Datanode start time should be set after RPC server starts successfully ( #3711 )
2021-11-25 10:23:04 +08:00
99b161dec7
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson ( #3708 )
2021-11-25 10:20:42 +08:00
98fe0d0fc3
HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags ( #3633 )
...
Contributed by Steve Loughran
2021-11-24 17:33:12 +00:00
e8566b3812
HDFS-16336. De-flake TestRollingUpgrade#testRollback ( #3686 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-24 22:54:11 +09:00
ae3ba45db5
HADOOP-17975. Fallback to simple auth does not work for a secondary DistributedFileSystem instance. ( #3579 )
2021-11-24 10:44:57 +00:00
08f3df3ea2
YARN-10991. Fix to ignore the grouping "[]" for resourcesStr in parseResourcesString method ( #3592 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-24 18:41:22 +09:00
a35f7dec25
HADOOP-18016. Make certain methods LimitedPrivate in S3AUtils.java ( #3685 )
...
Contributed By: Mehakmeet Singh
2021-11-24 13:32:59 +05:30
d3f0b7eab7
YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0 ( #3642 )
2021-11-23 13:21:51 -08:00
5b1b2c8ef6
YARN-11003. Make RMNode aware of all (OContainer inclusive) allocated resources ( #3646 )
2021-11-23 13:20:08 -08:00
faa4eeacb1
HDFS-16343. Add some debug logs when the dfsUsed are not used during Datanode startup. ( #3694 )
2021-11-23 13:36:14 +00:00
c0bdba8fac
HADOOP-18017. unguava: remove Preconditions from hadoop-yarn-project modules ( #3687 )
2021-11-23 13:36:22 +09:00
c7ec1897c4
HADOOP-18018. unguava: remove Preconditions from hadoop-tools modules ( #3688 )
2021-11-23 13:34:10 +09:00
0ed817babb
HDFS-16337. Show start time of Datanode on Web ( #3682 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-11-23 10:18:30 +08:00
63018dc73f
HADOOP-17998. Allow get command to run with multi threads. ( #3645 )
2021-11-22 11:37:05 +00:00
c88640c4ad
HDFS-16320. Datanode retrieve slownode information from NameNode ( #3654 )
2021-11-22 18:04:32 +08:00
20b78c8f97
HDFS-16335. Fix HDFSCommands.md ( #3681 )
2021-11-22 17:07:52 +09:00
8e9ad2d7af
Revert "HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson ( #3630 )" ( #3697 )
2021-11-22 10:05:21 +08:00
3d1ca6f6dc
HDFS-16334. Correct NameNode ACL description ( #3678 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-22 07:41:57 +09:00
965e3874e9
HDFS-16326. Simplify the code for DiskBalancer ( #3663 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
2021-11-19 11:37:10 +09:00
0c1eb0125b
HDFS-16328. Correct disk balancer desc ( #3669 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-19 01:02:01 +09:00
38614d05e1
HDFS-16330. Fix incorrect placeholder for Exception logs in DiskBalancer ( #3672 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-19 00:55:39 +09:00
dc751df63b
HDFS-16329. Fix log format for BlockManager ( #3670 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-19 00:42:01 +09:00
5b05068fdc
HDFS-16310. RBF: Add client port to CallerContext for Router ( #3635 )
2021-11-18 13:46:35 +09:00
91af256a5b
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson ( #3630 )
2021-11-17 22:41:06 +08:00
646c470e5d
HDFS-16323. DatanodeHttpServer doesn't require handler state map while retrieving filter handlers ( #3659 )
2021-11-17 22:17:09 +09:00
54a1d78e16
HADOOP-17999. No-op implementation of setWriteChecksum and setVerifyChecksum in ViewFileSystem. Contributed by Abhishek Das. ( #3639 )
2021-11-16 17:56:30 -08:00
3391b69692
HADOOP-18002. ABFS rename idempotency broken -remove recovery ( #3641 )
...
Cut modtime-based rename recovery as object modification time
is not updated during rename operation.
Applications will have to use etag API of HADOOP-17979
and implement it themselves.
Why not do the HEAD and etag recovery in ABFS client?
Cuts the IO capacity in half so kills job commit performance.
The manifest committer of MAPREDUCE-7341 will do this recovery
and act as the reference implementation of the algorithm.
Contributed by: Steve Loughran
2021-11-16 16:47:44 +05:30
5626734a36
HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. ( #3320 )
2021-11-16 18:21:50 +08:00
litao
GuoPhilipse
Ayush Saxena
huhaiyang
Andras Gyori
bitterfox
wangzhaohui
Viraj Jasani
Desmond Sisson
Sushmasree-28
smarthan
Takanobu Asanuma
Steve Loughran
Istvan Fajth
Ashutosh Gupta
Mehakmeet Singh
Andrew Chung
Mukul Kumar Singh
Symious
Hui Fei
Abhishek Das
Neil