He Xiaoqiao
5fe9b81941
HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd
)
2019-11-19 12:22:30 -08:00
Ayush Saxena
051dae1f89
HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun.
...
(cherry picked from commit 9e287054a8
)
2019-11-18 10:24:53 -08:00
Erik Krogen
60655bfe54
HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen.
...
(cherry picked from b2cc8b6b4a
)
2019-11-15 10:11:56 -08:00
Erik Krogen
dec765b329
HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen.
...
(cherry picked from 586defe711
)
2019-11-13 11:57:42 -08:00
Chao Sun
0c12d633b9
HDFS-14959: [SBNN read] access time should be turned off ( #1706 ) Contributed by Chao Sun.
...
(cherry picked from commit 97ec34e117
)
2019-11-12 11:40:56 -08:00
Siyao Meng
e0cf1735e1
HADOOP-16676. Backport HADOOP-16152 to branch-3.2. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-11-12 11:38:42 -08:00
Surendra Singh Lilhore
c4e22e0c80
HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina.
...
(cherry picked from commit 320008bb7c
)
2019-11-09 22:28:15 +05:30
Ayush Saxena
2e843cb773
HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan.
2019-11-07 22:50:30 +05:30
Chen Liang
c9ed0c3a37
HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen.
2019-11-06 16:12:51 -08:00
Chen Liang
431edfcbe4
HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko.
2019-11-06 15:00:15 -08:00
Ayush Saxena
5941a91f64
HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui.
2019-11-06 22:10:27 +05:30
Surendra Singh Lilhore
aa1c795dc9
HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit c36014165c
)
2019-11-06 19:37:41 +05:30
Wei-Chiu Chuang
240b5ee380
HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao.
...
(cherry picked from commit eb73ba6ed5
)
2019-11-04 09:40:57 -08:00
Zhao Junwang
c9fc118991
HDFS-14925. Rename operation should check nest snapshot ( #1670 )
...
If the src directory or any of its descendant is snapshottable
and the dst directory or any of its ancestors is snapshottable,
we consider this as nested snapshot, which should be denied.
Reviewed-by: Shashikant Banerjee <shashikant@apache.org>
(cherry picked from commit de6b8b0c0b
)
2019-11-01 16:39:18 -07:00
Ayush Saxena
b1821be1ec
HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong.
2019-11-01 22:51:50 +05:30
Ayush Saxena
239505a0af
HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui.
2019-10-31 23:53:06 +05:30
Jonathan Hung
5d2ffcc7aa
Make upstream aware of 2.10.0 release
...
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
2019-10-30 20:59:20 -07:00
Duo Zhang
a37a4bc62f
HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches
...
Signed-off-by: stack <stack@apache.org>
2019-10-30 20:26:54 -07:00
Eric Badger
3d9127dc97
HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger
...
(cherry picked from commit 9ef6ed9c1c
)
2019-10-28 16:38:28 +00:00
Rakesh Radhakrishnan
a6cdcf676e
HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.
...
(cherry picked from commit 659c88801d
)
2019-10-28 10:41:40 +05:30
Rakesh Radhakrishnan
dc2fad4bd3
HDFS-14700. Clean up pmem cache before setting pmem cache capacity. Contributed by Feilong He.
...
(cherry picked from commit f6fa865d6f
)
2019-10-28 10:41:25 +05:30
Rakesh Radhakrishnan
c6d59e6a47
HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He.
...
(cherry picked from commit 30a8f840f1
)
2019-10-28 10:41:12 +05:30
Rakesh Radhakrishnan
75c66bf044
HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He.
...
(cherry picked from commit e98adb00b7
)
2019-10-28 10:40:59 +05:30
Sammi Chen
6eb5fb596f
HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He.
...
(cherry picked from commit d1aad44490
)
2019-10-28 10:40:46 +05:30
Rakesh Radhakrishnan
0fe720dc48
HDFS-14402. Use FileChannel.transferTo() method for transferring block to SCM cache. Contributed by Feilong He.
...
(cherry picked from commit 37900c5639
)
2019-10-28 10:40:32 +05:30
Rakesh Radhakrishnan
f3571c1f09
HDFS-14401. Refine the implementation for HDFS cache on SCM. Contributed by Feilong He.
2019-10-28 10:40:15 +05:30
Uma Maheswara Rao G
6bc73a983d
HDFS-14355 : Implement HDFS cache on SCM by using pure java mapped byte buffer. Contributed by Feilong He.
2019-10-28 10:37:34 +05:30
Rakesh Radhakrishnan
0c6824120a
HDFS-14393. Refactor FsDatasetCache for SCM cache implementation. Contributed by Rakesh R
...
(cherry picked from commit f3f51284d5
)
2019-10-28 10:33:45 +05:30
Uma Maheswara Rao G
3f6f095c06
HDFS-14354: Refactor MappableBlock to align with the implementation of SCM cache. Contributed by Feilong He.
...
(cherry picked from commit ba50a36a3e
)
2019-10-28 10:33:29 +05:30
zhaoyim
9316ca149f
HDFS-14308. DFSStripedInputStream curStripeBuf is not freed by unbuffer() ( #1667 )
...
Reviewed-by: Aravindan Vijayan <avijayan@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 30db895b59
)
2019-10-25 13:10:56 -07:00
Wei-Chiu Chuang
8cf4bc4bda
HDFS-14910. Rename Snapshot with Pre Descendants Fail With IllegalArgumentException. Contributed by Wei-Chiu Chuang. ( #1672 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit a1b4eebcc9
)
2019-10-24 13:13:27 -07:00
Wei-Chiu Chuang
570ffa1cd6
HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. ( #1370 )
...
* HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang.
Change-Id: I9e5e450c07ad70aa1905973896c4f627042dbd37
* Fix checkstyle
Change-Id: I16d4bd4f03a971e1ed36cf57d89dc42357ef8fbf
(cherry picked from commit 6ef6594c7e
)
2019-10-24 13:13:19 -07:00
Vinayakumar B
e1e3d9c81d
HDFS-14921. Remove SuperUser Check in Setting Storage Policy in FileStatus During Listing. Contributed by Ayush Saxena.
2019-10-24 12:15:18 +05:30
Mukul Kumar Singh
c9d07a42dc
HDFS-14884. Add sanity check that zone key equals feinfo key while setting Xattrs. Contributed by Mukul Kumar Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a901405ad8
)
2019-10-22 13:31:34 -07:00
Wei-Chiu Chuang
31243f0d29
HDFS-13901. INode access time is ignored because of race between open and rename. Contributed by Jinglun.
2019-10-22 09:34:11 -07:00
Ayush Saxena
8c74717720
HDFS-14915. Move Superuser Check Before Taking Lock For Encryption API. Contributed by Ayush Saxena.
2019-10-22 18:45:19 +05:30
Kihwal Lee
2d1f3611cd
HDFS-12749. DN may not send block report to NN after NN restart. Contributed by Xiaoqiao He.
...
(cherry picked from commit c4e27ef773
)
2019-10-21 16:34:17 -05:00
Ayush Saxena
0f64ed89d2
HDFS-14913. Correct the value of available count in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Ayush Saxena.
2019-10-21 18:42:21 +05:30
Ayush Saxena
c2cd847a15
HDFS-14847. Erasure Coding: Blocks are over-replicated while EC decommissioning. Contributed by Fei Hui.
2019-10-20 06:29:16 +05:30
Surendra Singh Lilhore
5c50eb2243
HDFS-14909. DFSNetworkTopology#chooseRandomWithStorageType() should not decrease storage count for excluded node which is already part of excluded scope. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit 54dc6b7d72
)
2019-10-17 22:39:57 +05:30
Surendra Singh Lilhore
d1b30e8450
HDFS-14886. In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec. Contributed by hemanthboyina.
...
(cherry picked from commit 336abbd873
)
2019-10-14 23:47:42 +05:30
Surendra Singh Lilhore
2ca05ca02a
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced(addendum). Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit d76e2655ac
)
2019-10-09 23:32:21 +05:30
Chen Liang
a12ab911ee
HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko.
...
(cherry picked from commit 72ae371e7a
)
2019-10-08 12:03:04 -07:00
Ayush Saxena
57ab854492
HDFS-14859. Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero. Contributed by Srinivasu Majeti.
2019-10-08 14:22:16 +05:30
Surendra Singh Lilhore
a0cdebc409
HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit 382967be51
)
2019-10-08 00:26:27 +05:30
He Xiaoqiao
e29ae7db12
HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit dde9399b37
)
2019-10-04 12:07:30 -07:00
He Xiaoqiao
80392e94b6
HDFS-14497. Write lock held by metasave impact following RPC processing. Contributed by He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 33c62f8f4e
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
2019-10-04 12:06:55 -07:00
Erik Krogen
6630c9b75d
HDFS-14245. [SBN read] Enable ObserverReadProxyProvider to work with non-ClientProtocol proxy types. Contributed by Erik Krogen.
...
(cherry picked from 5847e00143
)
2019-10-04 11:15:49 -07:00
Erik Krogen
69b0c513a9
HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen.
...
(cherry picked from 64f28f9efa
)
2019-10-04 10:48:33 -07:00
Eric Yang
8bb2b00d38
HDFS-14890. Fixed namenode and journalnode startup on Windows.
...
Contributed by Siddharth Wagle
(cherry picked from commit aa24add8f0
)
2019-10-04 13:16:08 -04:00