Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,739 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

805 Commits

Author SHA1 Message Date
Chao Sun d38b617baa HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from 3ae775d740)
 2019-08-08 10:15:48 -07:00
Erik Krogen 2634aaaf9a HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira. 
(cherry picked from e7a0b8aa83)
 2019-08-02 12:00:48 -07:00
Weiwei Yang fa8b8edbac HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang. 
(cherry picked from commit 729cb3aefe)
 2019-07-04 18:06:19 +08:00
Erik Krogen e53cee0c14 HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen. 2019-06-28 14:28:28 -07:00
Erik Krogen 3c6b74a6a4 HDFS-14211. [SBN Read]. Add a configurable flag to enable always-msync mode to ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-06-28 14:28:17 -07:00
Erik Krogen e1e4056353 HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen. 2019-06-28 14:28:10 -07:00
Erik Krogen 98434bed5c HDFS-14279. [SBN read] Fix race condition in ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-06-28 14:28:03 -07:00
Erik Krogen 73d56b9d2a HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun. 2019-06-28 14:27:54 -07:00
Konstantin V Shvachko c936562c72 HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2019-06-28 14:23:14 -07:00
Konstantin V Shvachko 2be93bd54b HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko. 2019-06-28 14:23:14 -07:00
Chao Sun eb7d7fa5e2 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2019-06-28 14:23:13 -07:00
Erik Krogen c1c061d767 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2019-06-28 14:23:12 -07:00
Chen Liang 589d2c5922 HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang. 2019-06-28 14:23:11 -07:00
Chen Liang f7f7cac2e4 HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang. 2019-06-28 14:23:11 -07:00
Konstantin V Shvachko f0b73502c5 HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena. 2019-06-28 14:23:10 -07:00
Chen Liang 03a2c60a4b HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang. 2019-06-28 14:23:10 -07:00
Chen Liang 3d3686a619 HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang. 2019-06-28 14:23:09 -07:00
Chao Sun fc54850509 HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang. 2019-06-28 14:23:09 -07:00
Chao Sun cc70e81931 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2019-06-28 14:23:09 -07:00
Erik Krogen 556b7a840b HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun. 2019-06-28 14:22:12 -07:00
Chen Liang 4c4ca86286 HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-06-28 14:22:11 -07:00
Erik Krogen afeaeadd4f HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-06-28 14:22:11 -07:00
Konstantin V Shvachko 0b56a5300b HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-06-28 14:22:11 -07:00
Konstantin V Shvachko 0c9d846c60 HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-06-28 14:22:10 -07:00
Chen Liang e2337bfc57 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-06-28 14:22:10 -07:00
Erik Krogen e3f3fb1c18 HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 2019-06-28 14:22:09 -07:00
Chao Sun a97f00e1c6 HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-06-28 14:22:08 -07:00
Konstantin V Shvachko 1f5852fb12 HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-06-28 14:22:08 -07:00
Plamen Jeliazkov a3521c53fe HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-06-28 14:22:03 -07:00
Erik Krogen dbf777e4a5 HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2019-06-28 14:20:02 -07:00
Erik Krogen c0ca2bb853 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-06-28 14:20:01 -07:00
Plamen Jeliazkov 1eeca2d9fb HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-06-28 14:20:00 -07:00
Inigo Goiri 0966407ad6 HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException. 2019-06-24 19:02:41 -07:00
Wei-Chiu Chuang 45543ffd6a HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu. 
(cherry picked from commit 62ad9885ea)
 2019-06-17 20:19:59 -07:00
huzheng 9b61ecfcbe HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read 
(cherry picked from commit ea3b0a1844)
 2019-06-17 07:20:34 -07:00
Takanobu Asanuma a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka 855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Srinivasu Majeti 85f3664bb8 HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3e5e5b028a)
 2019-05-17 19:21:42 +02:00
Takanobu Asanuma 162e9999c7 HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory. 
(cherry picked from commit 55fb3c32fb)
 2019-03-27 03:28:54 +09:00
Erik Krogen fec7c5f3eb HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 
(cherry picked from commit 66357574ae)
 2019-03-13 13:19:18 -07:00
Dinesh Chitlangia eb77e60163 HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 45caeee6cf)
 2019-01-25 07:37:43 -08:00
Vinayakumar B abe14d32d4 HDFS-14124. EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs. Contributed by Ayush Saxena. 
(cherry picked from commit 39dc7345b8)
 2018-12-11 18:10:51 +05:30
Brahma Reddy Battula e2fa9e8cdd HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena. 
(cherry picked from commit f534736867)
 2018-11-30 00:28:04 +05:30
Brahma Reddy Battula 7e10dd03b5 HDFS-14064. WEBHDFS: Support Enable/Disable EC Policy. Contributed by Ayush Saxena. 
(cherry picked from commit 15d8f592b51d08a658e88c6a7a7596e0edf5793b)
 2018-11-22 00:07:38 +05:30
Inigo Goiri e7fba62e6a HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 
(cherry picked from commit 887244de4a)
 2018-11-06 11:20:06 -08:00
Xiao Chen 8788489df4 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824)
 2018-10-29 19:13:43 -07:00
Sunil G bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Xiao Chen ced2596094 HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a025)
 2018-10-15 11:00:21 -07:00
Vinayakumar B 8a41edb089 Fix potential FSImage corruption. Contributed by Daryn Sharp. 
(cherry picked from commit b60ca37914)
 2018-10-15 15:49:56 +05:30
Xiao Chen 9cb0654fbb HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.

(cherry picked from commit 5ec86b445c)
 2018-10-12 12:08:08 -07:00