52ec65fd10
HDFS-16331. Make dfs.blockreport.intervalMsec reconfigurable ( #3676 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-12-03 14:12:05 +09:00
0c62a514f7
HDFS-16358. HttpFS implementation for getSnapshotDiffReportListing ( #3730 )
2021-12-03 13:25:32 +09:00
196935a8d2
HDFS-16359. RBF: RouterRpcServer#invokeAtAvailableNs does not take effect when retrying ( #3731 ). Contributed by tomscut.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-12-02 19:10:35 +05:30
b34dcb5b3a
HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile ( #3722 )
2021-12-02 10:40:30 +08:00
df4197592f
HADOOP-18029: Update CompressionCodecFactory to handle uppercase file extensions ( #3739 )
...
Co-authored-by: Desmond Sisson <sissonde@amazon.com>
2021-12-01 15:36:54 -08:00
4dea4a7b67
HADOOP-18027. Include static imports in the maven plugin rules ( #3733 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-12-01 16:48:43 +09:00
1ebdac2cd6
HDFS-16361. Fix log format for QueryCommand ( #3732 ). Contributed by tomscut.
2021-12-01 12:48:21 +05:30
7b840f2693
YARN-10975 EntityGroupFSTimelineStore#ActiveLogParser parses already processed files ( #3735 )
...
Contributed by sravuri <sravuri@microsoft.com> and reviewed by Prabhu Joseph
2021-11-30 00:00:25 +05:30
c65c87f211
HADOOP-18026. Fix default value of Magic committer ( #3723 )
...
Contributed by guophilipse
2021-11-29 15:50:30 +00:00
932a78fe38
HADOOP-18023. Allow cp command to run with multi threads. ( #3721 )
2021-11-29 12:45:08 +00:00
829af89dc4
HDFS-16344. Improve DirectoryScanner.Stats#toString ( #3695 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-29 17:48:10 +09:00
215388beea
HADOOP-18022. Add restrict-imports-enforcer-rule for Guava Preconditions and remove remaining usages ( #3712 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-11-29 17:37:30 +09:00
86705eda82
HDFS-16339. Show the threshold when mover threads quota is exceeded ( #3689 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-26 19:52:14 +09:00
c9d64bad37
YARN-11007. Correct words in YARN documents ( #3680 )
...
Reviewed-by: cxorm <lianp964@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-25 17:48:54 +09:00
9c887e5b82
HADOOP-18014. CallerContext should not include some characters. ( #3698 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-25 14:05:04 +09:00
cdc13e91b6
HDFS-16350. Datanode start time should be set after RPC server starts successfully ( #3711 )
2021-11-25 10:23:04 +08:00
99b161dec7
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson ( #3708 )
2021-11-25 10:20:42 +08:00
98fe0d0fc3
HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags ( #3633 )
...
Contributed by Steve Loughran
2021-11-24 17:33:12 +00:00
e8566b3812
HDFS-16336. De-flake TestRollingUpgrade#testRollback ( #3686 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-24 22:54:11 +09:00
ae3ba45db5
HADOOP-17975. Fallback to simple auth does not work for a secondary DistributedFileSystem instance. ( #3579 )
2021-11-24 10:44:57 +00:00
08f3df3ea2
YARN-10991. Fix to ignore the grouping "[]" for resourcesStr in parseResourcesString method ( #3592 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-24 18:41:22 +09:00
a35f7dec25
HADOOP-18016. Make certain methods LimitedPrivate in S3AUtils.java ( #3685 )
...
Contributed By: Mehakmeet Singh
2021-11-24 13:32:59 +05:30
d3f0b7eab7
YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0 ( #3642 )
2021-11-23 13:21:51 -08:00
5b1b2c8ef6
YARN-11003. Make RMNode aware of all (OContainer inclusive) allocated resources ( #3646 )
2021-11-23 13:20:08 -08:00
faa4eeacb1
HDFS-16343. Add some debug logs when the dfsUsed are not used during Datanode startup. ( #3694 )
2021-11-23 13:36:14 +00:00
c0bdba8fac
HADOOP-18017. unguava: remove Preconditions from hadoop-yarn-project modules ( #3687 )
2021-11-23 13:36:22 +09:00
c7ec1897c4
HADOOP-18018. unguava: remove Preconditions from hadoop-tools modules ( #3688 )
2021-11-23 13:34:10 +09:00
0ed817babb
HDFS-16337. Show start time of Datanode on Web ( #3682 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-11-23 10:18:30 +08:00
63018dc73f
HADOOP-17998. Allow get command to run with multi threads. ( #3645 )
2021-11-22 11:37:05 +00:00
c88640c4ad
HDFS-16320. Datanode retrieve slownode information from NameNode ( #3654 )
2021-11-22 18:04:32 +08:00
20b78c8f97
HDFS-16335. Fix HDFSCommands.md ( #3681 )
2021-11-22 17:07:52 +09:00
8e9ad2d7af
Revert "HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson ( #3630 )" ( #3697 )
2021-11-22 10:05:21 +08:00
3d1ca6f6dc
HDFS-16334. Correct NameNode ACL description ( #3678 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-22 07:41:57 +09:00
965e3874e9
HDFS-16326. Simplify the code for DiskBalancer ( #3663 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
2021-11-19 11:37:10 +09:00
0c1eb0125b
HDFS-16328. Correct disk balancer desc ( #3669 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-19 01:02:01 +09:00
38614d05e1
HDFS-16330. Fix incorrect placeholder for Exception logs in DiskBalancer ( #3672 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-19 00:55:39 +09:00
dc751df63b
HDFS-16329. Fix log format for BlockManager ( #3670 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-19 00:42:01 +09:00
5b05068fdc
HDFS-16310. RBF: Add client port to CallerContext for Router ( #3635 )
2021-11-18 13:46:35 +09:00
91af256a5b
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson ( #3630 )
2021-11-17 22:41:06 +08:00
646c470e5d
HDFS-16323. DatanodeHttpServer doesn't require handler state map while retrieving filter handlers ( #3659 )
2021-11-17 22:17:09 +09:00
54a1d78e16
HADOOP-17999. No-op implementation of setWriteChecksum and setVerifyChecksum in ViewFileSystem. Contributed by Abhishek Das. ( #3639 )
2021-11-16 17:56:30 -08:00
3391b69692
HADOOP-18002. ABFS rename idempotency broken -remove recovery ( #3641 )
...
Cut modtime-based rename recovery as object modification time
is not updated during rename operation.
Applications will have to use etag API of HADOOP-17979
and implement it themselves.
Why not do the HEAD and etag recovery in ABFS client?
Cuts the IO capacity in half so kills job commit performance.
The manifest committer of MAPREDUCE-7341 will do this recovery
and act as the reference implementation of the algorithm.
Contributed by: Steve Loughran
2021-11-16 16:47:44 +05:30
5626734a36
HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. ( #3320 )
2021-11-16 18:21:50 +08:00
c9f95b01ef
HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode ( #3643 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2021-11-16 11:19:14 +09:00
89fcbd84f9
HDFS-16313. RBF: Add metrics for each sub cluster ( #3638 )
2021-11-16 09:47:25 +08:00
b933f5f54b
YARN-11001. Add docs on removing node label mapping from a node. Contributed by Manu Zhang
2021-11-15 16:02:18 +01:00
e14a2dcbba
HADOOP-18006. maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms ( #3648 )
...
Reviewed-by: Ahmed Hussein <ahussein@apache.org>
2021-11-15 22:57:24 +09:00
573b358fce
HADOOP-18003. Add a method appendIfAbsent for CallerContext ( #3644 )
2021-11-15 22:44:49 +09:00
60acf8434d
HDFS-16319. Add metrics doc for ReadLockLongHoldCount and WriteLockLongHoldCount ( #3653 ). Contributed by tomscut.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-11-14 20:10:24 +05:30
6e6f2e4baa
HDFS-16321. Fix invalid config in TestAvailableSpaceRackFaultTolerantBPP ( #3655 ). Contributed by guo.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-11-14 19:59:15 +05:30
litao
Viraj Jasani
Desmond Sisson
Sushmasree-28
GuoPhilipse
smarthan
Takanobu Asanuma
huhaiyang
Steve Loughran
Istvan Fajth
Ashutosh Gupta
Mehakmeet Singh
Andrew Chung
Mukul Kumar Singh
Symious
Hui Fei
Abhishek Das
Neil
Szilard Nemeth