Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,129 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

7808 Commits

Author SHA1 Message Date
Sammi Chen 6eb5fb596f HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 
(cherry picked from commit d1aad44490)
 2019-10-28 10:40:46 +05:30
Rakesh Radhakrishnan 0fe720dc48 HDFS-14402. Use FileChannel.transferTo() method for transferring block to SCM cache. Contributed by Feilong He. 
(cherry picked from commit 37900c5639)
 2019-10-28 10:40:32 +05:30
Rakesh Radhakrishnan f3571c1f09 HDFS-14401. Refine the implementation for HDFS cache on SCM. Contributed by Feilong He. 2019-10-28 10:40:15 +05:30
Uma Maheswara Rao G 6bc73a983d HDFS-14355 : Implement HDFS cache on SCM by using pure java mapped byte buffer. Contributed by Feilong He. 2019-10-28 10:37:34 +05:30
Rakesh Radhakrishnan 0c6824120a HDFS-14393. Refactor FsDatasetCache for SCM cache implementation. Contributed by Rakesh R 
(cherry picked from commit f3f51284d5)
 2019-10-28 10:33:45 +05:30
Uma Maheswara Rao G 3f6f095c06 HDFS-14354: Refactor MappableBlock to align with the implementation of SCM cache. Contributed by Feilong He. 
(cherry picked from commit ba50a36a3e)
 2019-10-28 10:33:29 +05:30
zhaoyim 9316ca149f HDFS-14308. DFSStripedInputStream curStripeBuf is not freed by unbuffer() (#1667) 
Reviewed-by: Aravindan Vijayan <avijayan@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 30db895b59)
 2019-10-25 13:10:56 -07:00
Wei-Chiu Chuang 8cf4bc4bda HDFS-14910. Rename Snapshot with Pre Descendants Fail With IllegalArgumentException. Contributed by Wei-Chiu Chuang. (#1672) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit a1b4eebcc9)
 2019-10-24 13:13:27 -07:00
Wei-Chiu Chuang 570ffa1cd6 HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. (#1370) 
* HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang.

Change-Id: I9e5e450c07ad70aa1905973896c4f627042dbd37

* Fix checkstyle

Change-Id: I16d4bd4f03a971e1ed36cf57d89dc42357ef8fbf
(cherry picked from commit 6ef6594c7e)
 2019-10-24 13:13:19 -07:00
Vinayakumar B e1e3d9c81d HDFS-14921. Remove SuperUser Check in Setting Storage Policy in FileStatus During Listing. Contributed by Ayush Saxena. 2019-10-24 12:15:18 +05:30
Mukul Kumar Singh c9d07a42dc HDFS-14884. Add sanity check that zone key equals feinfo key while setting Xattrs. Contributed by Mukul Kumar Singh. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a901405ad8)
 2019-10-22 13:31:34 -07:00
Wei-Chiu Chuang 31243f0d29 HDFS-13901. INode access time is ignored because of race between open and rename. Contributed by Jinglun. 2019-10-22 09:34:11 -07:00
Ayush Saxena 8c74717720 HDFS-14915. Move Superuser Check Before Taking Lock For Encryption API. Contributed by Ayush Saxena. 2019-10-22 18:45:19 +05:30
Kihwal Lee 2d1f3611cd HDFS-12749. DN may not send block report to NN after NN restart. Contributed by Xiaoqiao He. 
(cherry picked from commit c4e27ef773)
 2019-10-21 16:34:17 -05:00
Ayush Saxena 0f64ed89d2 HDFS-14913. Correct the value of available count in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Ayush Saxena. 2019-10-21 18:42:21 +05:30
Ayush Saxena c2cd847a15 HDFS-14847. Erasure Coding: Blocks are over-replicated while EC decommissioning. Contributed by Fei Hui. 2019-10-20 06:29:16 +05:30
Surendra Singh Lilhore 5c50eb2243 HDFS-14909. DFSNetworkTopology#chooseRandomWithStorageType() should not decrease storage count for excluded node which is already part of excluded scope. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 54dc6b7d72)
 2019-10-17 22:39:57 +05:30
Surendra Singh Lilhore d1b30e8450 HDFS-14886. In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec. Contributed by hemanthboyina. 
(cherry picked from commit 336abbd873)
 2019-10-14 23:47:42 +05:30
Surendra Singh Lilhore 2ca05ca02a HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced(addendum). Contributed by Surendra Singh Lilhore. 
(cherry picked from commit d76e2655ac)
 2019-10-09 23:32:21 +05:30
Chen Liang a12ab911ee HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko. 
(cherry picked from commit 72ae371e7a)
 2019-10-08 12:03:04 -07:00
Ayush Saxena 57ab854492 HDFS-14859. Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero. Contributed by Srinivasu Majeti. 2019-10-08 14:22:16 +05:30
Surendra Singh Lilhore a0cdebc409 HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 382967be51)
 2019-10-08 00:26:27 +05:30
He Xiaoqiao e29ae7db12 HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit dde9399b37)
 2019-10-04 12:07:30 -07:00
He Xiaoqiao 80392e94b6 HDFS-14497. Write lock held by metasave impact following RPC processing. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 33c62f8f4e)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2019-10-04 12:06:55 -07:00
Erik Krogen 6630c9b75d HDFS-14245. [SBN read] Enable ObserverReadProxyProvider to work with non-ClientProtocol proxy types. Contributed by Erik Krogen. 
(cherry picked from 5847e00143)
 2019-10-04 11:15:49 -07:00
Erik Krogen 69b0c513a9 HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen. 
(cherry picked from 64f28f9efa)
 2019-10-04 10:48:33 -07:00
Eric Yang 8bb2b00d38 HDFS-14890. Fixed namenode and journalnode startup on Windows. 
Contributed by Siddharth Wagle

(cherry picked from commit aa24add8f0)
 2019-10-04 13:16:08 -04:00
Ayush Saxena 702572434c HDFS-13693. Remove unnecessary search in INodeDirectory.addChild during image loading. Contributed by Lisheng Sun. 
(cherry picked from commit 377f95bbe8)
 2019-10-04 08:52:59 -07:00
Takanobu Asanuma 732a68cfb4 HDFS-14879. Header was wrong in Snapshot web UI. Contributed by hemanthboyina 
(cherry picked from commit b23bdaf085)
 2019-10-04 16:48:50 +09:00
Stephen O'Donnell 966193153f HDFS-14637. Namenode may not replicate blocks to meet the policy after enabling upgradeDomain. Contributed by Stephen O'Donnell. 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c99a12167f)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlocksWithNotEnoughRacks.java
	hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/BlockPlacementPolicyAlwaysSatisfied.java
 2019-10-03 22:18:29 -07:00
Inigo Goiri 21a89d544f HDFS-14624. When decommissioning a node, log remaining blocks to replicate periodically. Contributed by Stephen O'Donnell. 
(cherry picked from commit 5747f6cff5)
 2019-10-03 17:27:44 -07:00
Shashikant Banerjee f14fb9081f HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee. 
(cherry picked from commit f9fab9f22a)
 2019-10-03 17:27:06 -07:00
Vinayakumar B 4e223d9968 HDFS-14113. EC : Add Configuration to restrict UserDefined Policies. Contributed by Ayush Saxena. 
(cherry picked from commit c03024a530)
 2019-10-03 17:15:45 -07:00
Shweta Yakkali eaf09e51ba HDFS-14262. [SBN read] Make Log.WARN message in GlobalStateIdContext more informative. Contributed by Shweta Yakkali. 2019-10-03 16:13:44 -07:00
Ayush Saxena ebd7a0f155 HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun. 2019-10-03 16:00:38 -07:00
Shweta Yakkali 3186afa24c HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-10-03 15:58:00 -07:00
Ayush Saxena 9a9edfa87e HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao. 
(cherry picked from commit c4c8d5fd0e)
 2019-10-03 14:56:09 -07:00
Chen Liang 05a1eb788f HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang. 2019-10-02 21:12:54 -07:00
LeonGao91 885dd17595 HDFS-14678. Allow triggerBlockReport to a specific namenode. (#1252). Contributed by Leon Gao. 
(cherry picked from commit 9a1d8cfaf5)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestTriggerBlockReport.java
 2019-10-02 17:35:33 -07:00
Chen Liang 8a866f01c9 HDFS-14858. [SBN read] Allow configurably enable/disable AlignmentContext on NameNode. Contributed by Chen Liang. 2019-10-02 16:39:26 -07:00
paulward24 0559c1dba3 HDFS-14618. Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). Contributed by Paul Ward. 
(cherry picked from commit d203045c30)
 2019-10-02 15:07:40 -07:00
paulward24 76a9135947 HDFS-14610. HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. (#1015) 
(cherry picked from commit d8bac50e12)
 2019-10-02 14:31:13 -07:00
Inigo Goiri 6148acebc3 HDFS-14527. Stop all DataNodes may result in NN terminate. Contributed by He Xiaoqiao. 
(cherry picked from commit 944adc61b1)
 2019-10-02 13:55:47 -07:00
Ayush Saxena 2bb4ac18ec HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena. 
(cherry picked from commit 66400c1cbb)
 2019-10-02 13:49:12 -07:00
Surendra Singh Lilhore 7d7330aff0 HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina. 
(cherry picked from commit 4852a90e4b)
 2019-10-02 13:39:40 -07:00
Inigo Goiri d2a7a128d9 HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao. 
(cherry picked from commit 389e640f0c)
 2019-10-01 17:33:14 -07:00
Yiqun Lin c74027d9d3 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun. 
(cherry picked from commit a5bb1e8ee8)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2019-10-01 17:14:46 -07:00
Inigo Goiri 3510a9e5e5 HDFS-14192. Track missing DFS operations in Statistics and StorageStatistics. Contributed by Ayush Saxena. 
(cherry picked from commit f048512bb8)
 2019-10-01 16:57:20 -07:00
Inigo Goiri 917e53f066 HDFS-14460. DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured. Contributed by CR Hota. 
(cherry picked from commit 865c328930)
 2019-10-01 09:53:36 -07:00
Inigo Goiri fc8a7a9e5b HDFS-14202. dfs.disk.balancer.max.disk.throughputInMBperSec property is not working as per set value. Contributed by Ranith Sardar. 
(cherry picked from commit 0e79a86582)
 2019-10-01 09:35:04 -07:00
Inigo Goiri 06b2eceb76 HDFS-14418. Remove redundant super user priveledge checks from namenode. Contributed by Ayush Saxena. 
(cherry picked from commit be6c8014e6)
 2019-10-01 09:22:15 -07:00
Konstantin V Shvachko 665360ec6f HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko. 2019-09-30 16:57:56 -07:00
Konstantin V Shvachko 3912a6fc7d Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao." 
This reverts commit 0feba4396f.
 2019-09-30 16:57:26 -07:00
Ayush Saxena b7ee3938e9 HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar. 2019-09-28 08:29:58 +05:30
Ayush Saxena a0db762206 HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena. 2019-09-25 01:21:14 +05:30
Rohith Sharma K S f9f0338104 Make upstream aware of 3.2.1 release. 2019-09-23 12:03:53 +05:30
Chen Liang 4299bc0d3e HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang. 2019-09-18 16:48:58 -07:00
Kihwal Lee e29f91bad3 HDFS-13959. TestUpgradeDomainBlockPlacementPolicy is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 1851d06eb3)
 2019-09-18 15:53:08 -05:00
Wei-Chiu Chuang b906f53b0d HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 5dd859a8a0)
 2019-09-18 09:36:10 -07:00
Ayush Saxena 4126e67e31 HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 2019-09-14 10:12:17 +05:30
Surendra Singh Lilhore eb1ddcd04c HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming. 
(cherry picked from commit d1c303a497)
 2019-09-12 19:25:50 +05:30
Rohith Sharma K S 7d5bb2ebb7 Preparing for 3.2.2-SNAPSHOT development. 2019-09-07 08:52:08 +05:30
Xiaoyu Yao ebb3b8add0
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun. 
(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)
 2019-09-05 15:33:54 -07:00
Xiaoyu Yao 53d1bfe680
Revert "HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun." 
This reverts commit 259c6bb0c7.
 2019-09-05 15:31:14 -07:00
Xiaoyu Yao 259c6bb0c7
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun. 
(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)
 2019-09-03 09:12:19 -07:00
Stephen O'Donnell 61fd1a74a1 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 915cbc91c0)
 2019-09-02 10:06:39 -07:00
Wei-Chiu Chuang 84b2fe4b10 Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell." 
This reverts commit 9c0d6e1657.
 2019-09-02 10:06:39 -07:00
Ayush Saxena 3edffd90c9 HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B. 2019-08-30 08:58:12 +05:30
leosunli b516d2ea7b Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (#1314). Contributed by Lisheng Sun. 
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
(cherry picked from commit 915271245b)
 2019-08-29 18:46:15 -07:00
Stephen O'Donnell 9c0d6e1657 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7bebad61d9)
 2019-08-29 17:38:55 -07:00
Istvan Fajth 7c6fc964fd HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 03049290fe8ef1459a3c8bd05cf198320e6e4973)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNNStorageRetentionManager.java
 2019-08-29 16:55:18 -07:00
Akira Ajisaka 2d8799f4bc HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2019-08-27 19:08:39 +00:00
Jonathan Hung d255efa5f8 HDFS-14779. Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns 
(cherry picked from commit 8ab7020e64)
 2019-08-27 10:30:55 -07:00
Arpit Agarwal 8b12381717
HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle. 
(cherry picked from commit a64a43b77fb1032dcb66730a6b6257a24726c256)
 2019-08-26 15:57:41 -07:00
Chen Liang c824653955 HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui. 2019-08-23 12:28:13 -07:00
Erik Krogen 2408c2491f HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen. 
(cherry picked from d699022fce)
 2019-08-23 11:48:11 -07:00
Akira Ajisaka e8b93d04c4
HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui. 
(cherry picked from commit bd7baea5a5)
 2019-08-23 15:28:54 +09:00
Akira Ajisaka d3e5898712
HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui. 
(cherry picked from commit abc8fde4ca)
 2019-08-23 14:45:57 +09:00
Ayush Saxena b7a66471ca HDFS-14276. [SBN read] Reduce tailing overhead. Contributed by Wei-Chiu Chuang. 2019-08-22 22:35:48 +05:30
Wei-Chiu Chuang 933ea6aa4d Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow." 
This reverts commit 8b802d6b44.
 2019-08-21 09:00:23 -07:00
Surendra Singh Lilhore 03c62c7989 HDFS-14582. Failed to start DN with ArithmeticException when NULL checksum used. Contributed by Surendra Singh Lilhore. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit f95988113da3f06f6d975f99f1ee51d88a793537)
 2019-08-20 15:54:46 -07:00
Wei-Chiu Chuang 77fd690c05 HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai. 
(cherry picked from commit fbe87eddbc30fe5191c008b496fb83e51ef4ee4a)
 2019-08-20 10:37:05 -07:00
Wei-Chiu Chuang 6d93886231 HDFS-13709. Report bad block to NN when transfer block encounter EIO exception. Contributed by Chen Zhang. 
(cherry picked from commit 360a96f342)
 2019-08-19 13:11:11 -07:00
HUAN-PING SU abda503040 HDFS-14746. Trivial test code update after HDFS-14687. Contributed by kevin su. 
Reviewed-by: Surendra Singh Lilhore <surendralilhore@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit abae6ff2a2)
 2019-08-19 11:15:37 -07:00
Wei-Chiu Chuang 8b802d6b44 HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow. 2019-08-19 08:38:06 -07:00
Surendra Singh Lilhore 6b01effd01 HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore. 
Reviewed-by: Siyao Meng <smeng@cloudera.com>
Reviewed-by: Wei-Chiu CHuang <weichiu@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit b8db5b9a98)
 2019-08-18 18:17:41 -07:00
Shashikant Banerjee e89413da88 HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee. 
(cherry picked from commit 0a85af959c)
 2019-08-16 19:09:34 -07:00
Siyao Meng c6e4906ffc HDFS-14595. HDFS-11848 breaks API compatibility. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 3c0382f1b9)
 2019-08-16 19:08:05 -07:00
Chen Liang d8a6098a96 HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang. 2019-08-16 10:50:00 -07:00
Chen Liang f507bc059d HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-08-16 10:50:00 -07:00
Chen Liang ca15968d09 HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang 2019-08-16 10:50:00 -07:00
Chen Liang 22d81f7fd0 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2019-08-16 10:49:59 -07:00
Wei-Chiu Chuang f5661b630a HDFS-14491. More Clarity on Namenode UI Around Blocks and Replicas. Contributed by Siyao Meng. 
(cherry picked from commit 6a43d0fbd49b3ff1ce75a2334b51a98ae476e473)
 2019-08-13 17:15:19 -07:00
Masatake Iwasaki d7ca016d63 HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit da0006fe04)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
 2019-08-14 08:45:10 +09:00
Siyao Meng c5f433b11f HDFS-14148. HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory (#1274) Contributed by Siyao Meng. 
(cherry picked from commit c92b49876a)
 2019-08-12 17:30:18 -07:00
Yiqun Lin a7bc09c596 HDFS-13359. DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream. Contributed by Yiqun Lin. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 8a77a224c7)
 2019-08-09 18:41:03 -07:00
Wei-Chiu Chuang 2f2fa3dad6 HDFS-14693. NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. Contributed by Xudong Cao. 
(cherry picked from commit 6ad9a11494)
 2019-08-08 13:51:14 -07:00
Stephen O'Donnell a00a327594 HDFS-14459. ClosedChannelException silently ignored in FsVolumeList.addBlockPool(). Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit b0799148cf)
 2019-08-08 13:37:10 -07:00
Chao Sun d38b617baa HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from 3ae775d740)
 2019-08-08 10:15:48 -07:00
Erik Krogen 016aa13940 HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen. 
(cherry picked from 827dbb11e2)
 2019-08-07 09:32:32 -07:00