Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,666 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

790 Commits

Author SHA1 Message Date
Chen Liang 03a2c60a4b HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang. 2019-06-28 14:23:10 -07:00
Chen Liang 3d3686a619 HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang. 2019-06-28 14:23:09 -07:00
Chao Sun fc54850509 HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang. 2019-06-28 14:23:09 -07:00
Chao Sun cc70e81931 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2019-06-28 14:23:09 -07:00
Erik Krogen 556b7a840b HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun. 2019-06-28 14:22:12 -07:00
Chen Liang 4c4ca86286 HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-06-28 14:22:11 -07:00
Erik Krogen afeaeadd4f HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-06-28 14:22:11 -07:00
Konstantin V Shvachko 0b56a5300b HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-06-28 14:22:11 -07:00
Konstantin V Shvachko 0c9d846c60 HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-06-28 14:22:10 -07:00
Chen Liang e2337bfc57 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-06-28 14:22:10 -07:00
Erik Krogen e3f3fb1c18 HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 2019-06-28 14:22:09 -07:00
Chao Sun a97f00e1c6 HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-06-28 14:22:08 -07:00
Konstantin V Shvachko 1f5852fb12 HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-06-28 14:22:08 -07:00
Plamen Jeliazkov a3521c53fe HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-06-28 14:22:03 -07:00
Erik Krogen dbf777e4a5 HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2019-06-28 14:20:02 -07:00
Erik Krogen c0ca2bb853 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-06-28 14:20:01 -07:00
Plamen Jeliazkov 1eeca2d9fb HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-06-28 14:20:00 -07:00
Inigo Goiri 0966407ad6 HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException. 2019-06-24 19:02:41 -07:00
Wei-Chiu Chuang 45543ffd6a HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu. 
(cherry picked from commit 62ad9885ea)
 2019-06-17 20:19:59 -07:00
huzheng 9b61ecfcbe HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read 
(cherry picked from commit ea3b0a1844)
 2019-06-17 07:20:34 -07:00
Takanobu Asanuma a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka 855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Srinivasu Majeti 85f3664bb8 HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3e5e5b028a)
 2019-05-17 19:21:42 +02:00
Takanobu Asanuma 162e9999c7 HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory. 
(cherry picked from commit 55fb3c32fb)
 2019-03-27 03:28:54 +09:00
Erik Krogen fec7c5f3eb HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 
(cherry picked from commit 66357574ae)
 2019-03-13 13:19:18 -07:00
Dinesh Chitlangia eb77e60163 HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 45caeee6cf)
 2019-01-25 07:37:43 -08:00
Vinayakumar B abe14d32d4 HDFS-14124. EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs. Contributed by Ayush Saxena. 
(cherry picked from commit 39dc7345b8)
 2018-12-11 18:10:51 +05:30
Brahma Reddy Battula e2fa9e8cdd HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena. 
(cherry picked from commit f534736867)
 2018-11-30 00:28:04 +05:30
Brahma Reddy Battula 7e10dd03b5 HDFS-14064. WEBHDFS: Support Enable/Disable EC Policy. Contributed by Ayush Saxena. 
(cherry picked from commit 15d8f592b51d08a658e88c6a7a7596e0edf5793b)
 2018-11-22 00:07:38 +05:30
Inigo Goiri e7fba62e6a HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 
(cherry picked from commit 887244de4a)
 2018-11-06 11:20:06 -08:00
Xiao Chen 8788489df4 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824)
 2018-10-29 19:13:43 -07:00
Sunil G bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Xiao Chen ced2596094 HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a025)
 2018-10-15 11:00:21 -07:00
Vinayakumar B 8a41edb089 Fix potential FSImage corruption. Contributed by Daryn Sharp. 
(cherry picked from commit b60ca37914)
 2018-10-15 15:49:56 +05:30
Xiao Chen 9cb0654fbb HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.

(cherry picked from commit 5ec86b445c)
 2018-10-12 12:08:08 -07:00
Siyao Meng 15e0ba3fb4 HDFS-13877. HttpFS: Implement GETSNAPSHOTDIFF. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 396ce0d9f4)
 2018-10-11 15:03:55 -07:00
Hrishikesh Gadre 90a9837c9d HDFS-13956. iNotify should include information to identify a file as either replicated or erasure coded. Contributed by Hrishikesh Gadre. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit bf3d591f0c)
 2018-10-10 10:24:23 -07:00
Weiwei Yang 966d94ea2f HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3ead525c71)
 2018-10-10 10:14:57 -07:00
Hrishikesh Gadre a99658cd85 HDFS-13926. ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads. 
Contributed by Xiao Chen, Hrishikesh Gadre.

Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 08bb6c49a5)
 2018-10-08 20:39:48 -07:00
Takanobu Asanuma f6c5ef9903 HDFS-13943. [JDK10] Fix javadoc errors in hadoop-hdfs-client module. Contributed by Akira Ajisaka. 2018-10-02 09:49:48 +09:00
Wei-Chiu Chuang 28ceb34a72 HDFS-13868. WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is not. Contributed by Pranay Singh. 2018-09-19 03:12:20 -07:00
Wei-Chiu Chuang a1de8cbac5 HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng. 2018-09-14 05:22:56 +08:00
Vinayakumar B 30eceec342 HDFS-13806. EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory. Contributed by Ayush Saxena. 2018-09-10 09:10:51 +05:30
Giovanni Matteo Fumarola eca1a4bfe9 HDFS-13695. Move logging to slf4j in HDFS package. Contributed by Ian Pickering. 2018-09-06 14:48:00 -07:00
Kitti Nanasi 6e5ffb74dd HDFS-13885. Add debug logs in dfsclient around decrypting EDEK. Contributed by Kitti Nanasi. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-09-03 22:37:37 -07:00
Wei-Chiu Chuang 8aa6c4f079 Revert "HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng." 
This reverts commit 26c2a97c56.
 2018-08-30 11:44:20 -07:00
Wei-Chiu Chuang 26c2a97c56 HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng. 2018-08-27 16:02:35 -07:00
Konstantin V Shvachko a4121c71c2 HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 2018-08-24 18:27:30 -07:00
Xiao Chen c67b0650ea HDFS-13747. Statistic for list_located_status is incremented incorrectly by listStatusIterator. Contributed by Antal Mihalyi. 2018-08-16 23:14:21 -07:00
Uma Maheswara Rao G 39ed3a66db HDFS-13076: [SPS]: Cleanup work for HDFS-10285 merge. Contributed by Rakesh R. 2018-08-12 03:06:06 -07:00