d3c4c8be6a
HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena.
2020-01-11 10:34:01 +05:30
1ee90d6731
HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang.
...
(cherry picked from commit 26a969ec73
2020-01-10 17:17:11 -08:00
59260a0e0e
HDFS-15095. Fix TestDecommissioningStatus. Contributed by Ahmed Hussein.
...
(cherry picked from commit 5fb901ac40
2020-01-10 12:46:49 -06:00
6304f94bc0
HDFS-14957. INodeReference Space Consumed was not same in QuotaUsage and ContentSummary. Contributed by hemanthboyina.
2020-01-09 12:07:23 +05:30
d7f9a90e2c
HDFS-15080. Fix the issue in reading persistent memory cached data with an offset. Contributed by Feilong He.
...
(cherry picked from commit 7030722e5d
2020-01-08 14:29:05 +05:30
3e17b471b6
HDFS-15077. Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout. ( #1797 )
...
(cherry picked from commit aba3f6c3e1
2020-01-08 17:05:20 +09:00
3e32f5476a
HDFS-15072. HDFS MiniCluster fails to start when run in directory path with a %. ( #1775 )
...
(cherry picked from commit a43c177f1d
2020-01-08 11:29:23 +09:00
a27f9bd28e
HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He.
...
(cherry picked from commit d79cce20ab
2020-01-02 11:57:02 +05:30
5d9cbca31e
HDFS-15074. DataNode.DataTransfer thread should catch all the expception and log it. Contributed by hemanthboyina.
2019-12-29 11:21:33 +05:30
ad11b25df2
HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena.
...
(cherry picked from commit dc32f583af
2019-12-28 21:34:19 +09:00
0d94e0b1a0
HDFS-15073. Replace curator-shaded guava import with the standard one ( #1782 )
...
(cherry picked from commit d8cd7098b4
2019-12-25 11:12:03 +09:00
82957562ea
HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit b98ac2a3af
2019-12-24 12:50:57 -08:00
11c5396b4a
Revert "HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui."
...
This reverts commit 59cdf18d5a
2019-12-20 19:04:45 +00:00
59cdf18d5a
HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui.
...
(cherry picked from commit 52d7b745c6
2019-12-19 09:43:40 -08:00
afe00a1ca5
HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee.
...
(cherry picked from commit fdd96e46d1
2019-12-18 23:10:00 +05:30
d4a6901c42
HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang.
...
(cherry picked from commit 65c4660bcd
2019-12-12 15:57:15 -08:00
8ba4e13dcc
HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 99af49878cc47f5750b6c0edadc8a4c2fcd7a7c6)
2019-12-11 16:34:28 -08:00
3402c87353
HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen.
...
(cherry picked from c174d50b30
2019-12-11 09:40:36 -08:00
298cda22a3
HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore.
2019-11-28 23:58:46 +05:30
32fdb3004a
HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar.
2019-11-23 02:36:38 +05:30
39366b1370
HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina.
2019-11-22 00:33:42 +05:30
5fe9b81941
HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd
2019-11-19 12:22:30 -08:00
051dae1f89
HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun.
...
(cherry picked from commit 9e287054a8
2019-11-18 10:24:53 -08:00
60655bfe54
HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen.
...
(cherry picked from b2cc8b6b4a
2019-11-15 10:11:56 -08:00
dec765b329
HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen.
...
(cherry picked from 586defe711
2019-11-13 11:57:42 -08:00
0c12d633b9
HDFS-14959: [SBNN read] access time should be turned off ( #1706 ) Contributed by Chao Sun.
...
(cherry picked from commit 97ec34e117
2019-11-12 11:40:56 -08:00
e0cf1735e1
HADOOP-16676. Backport HADOOP-16152 to branch-3.2. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-11-12 11:38:42 -08:00
c4e22e0c80
HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina.
...
(cherry picked from commit 320008bb7c
2019-11-09 22:28:15 +05:30
2e843cb773
HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan.
2019-11-07 22:50:30 +05:30
c9ed0c3a37
HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen.
2019-11-06 16:12:51 -08:00
431edfcbe4
HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko.
2019-11-06 15:00:15 -08:00
5941a91f64
HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui.
2019-11-06 22:10:27 +05:30
aa1c795dc9
HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit c36014165c
2019-11-06 19:37:41 +05:30
240b5ee380
HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao.
...
(cherry picked from commit eb73ba6ed5
2019-11-04 09:40:57 -08:00
c9fc118991
HDFS-14925. Rename operation should check nest snapshot ( #1670 )
...
If the src directory or any of its descendant is snapshottable
and the dst directory or any of its ancestors is snapshottable,
we consider this as nested snapshot, which should be denied.
Reviewed-by: Shashikant Banerjee <shashikant@apache.org>
(cherry picked from commit de6b8b0c0b
2019-11-01 16:39:18 -07:00
b1821be1ec
HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong.
2019-11-01 22:51:50 +05:30
239505a0af
HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui.
2019-10-31 23:53:06 +05:30
5d2ffcc7aa
Make upstream aware of 2.10.0 release
...
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
2019-10-30 20:59:20 -07:00
a37a4bc62f
HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches
...
Signed-off-by: stack <stack@apache.org>
2019-10-30 20:26:54 -07:00
3d9127dc97
HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger
...
(cherry picked from commit 9ef6ed9c1c
2019-10-28 16:38:28 +00:00
a6cdcf676e
HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.
...
(cherry picked from commit 659c88801d
2019-10-28 10:41:40 +05:30
dc2fad4bd3
HDFS-14700. Clean up pmem cache before setting pmem cache capacity. Contributed by Feilong He.
...
(cherry picked from commit f6fa865d6f
2019-10-28 10:41:25 +05:30
c6d59e6a47
HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He.
...
(cherry picked from commit 30a8f840f1
2019-10-28 10:41:12 +05:30
75c66bf044
HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He.
...
(cherry picked from commit e98adb00b7
2019-10-28 10:40:59 +05:30
6eb5fb596f
HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He.
...
(cherry picked from commit d1aad44490
2019-10-28 10:40:46 +05:30
0fe720dc48
HDFS-14402. Use FileChannel.transferTo() method for transferring block to SCM cache. Contributed by Feilong He.
...
(cherry picked from commit 37900c5639
2019-10-28 10:40:32 +05:30
f3571c1f09
HDFS-14401. Refine the implementation for HDFS cache on SCM. Contributed by Feilong He.
2019-10-28 10:40:15 +05:30
6bc73a983d
HDFS-14355 : Implement HDFS cache on SCM by using pure java mapped byte buffer. Contributed by Feilong He.
2019-10-28 10:37:34 +05:30
0c6824120a
HDFS-14393. Refactor FsDatasetCache for SCM cache implementation. Contributed by Rakesh R
...
(cherry picked from commit f3f51284d5
2019-10-28 10:33:45 +05:30
3f6f095c06
HDFS-14354: Refactor MappableBlock to align with the implementation of SCM cache. Contributed by Feilong He.
...
(cherry picked from commit ba50a36a3e
2019-10-28 10:33:29 +05:30
Ayush Saxena
Chen Liang
Kihwal Lee
Surendra Singh Lilhore
Rakesh Radhakrishnan
Masatake Iwasaki
Takanobu Asanuma
Chandra Sanivarapu
Konstantin V Shvachko
Eric Badger
Inigo Goiri
Shashikant Banerjee
He Xiaoqiao
Erik Krogen
Chao Sun
Siyao Meng
Wei-Chiu Chuang
Zhao Junwang
Jonathan Hung
Duo Zhang
Sammi Chen
Uma Maheswara Rao G