Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,202 Commits 326 Branches 371 Tags
Commit Graph

602 Commits

Author SHA1 Message Date
Siyao Meng
224643a58c HDFS-14595. HDFS-11848 breaks API compatibility. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 3c0382f1b933b7acfe55081f5bad46f9fe05a14b)
(cherry picked from commit 136a97a74dbc12f05b88d0abda101690e7c727d9)
 2019-08-14 07:40:48 -07:00
Masatake Iwasaki
e66ad19395 HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit da0006fe0473e353ee2d489156248a01aa982dfd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java

(cherry picked from commit d7ca016d63d89e5c8377a035f93485a7c77c3430)
 2019-08-14 09:07:00 +09:00
Chao Sun
90447baeb0 HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from 3ae775d74029b6ae82263739f598ceb25c597dcd)
(cherry picked from d38b617baaf10cb35f3a8eba904b930a142a071e)
 2019-08-08 10:42:27 -07:00
Erik Krogen
da548cf263 HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira. 
(cherry picked from e7a0b8aa83c1fb933d409c514d2155e986e4e25b)
(cherry picked from ba4ca7d996cafa1b1786249cae2b207f0eaa1070)
 2019-08-02 11:51:22 -07:00
Erik Krogen
cc22373606 HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen. 2019-07-12 12:25:19 -07:00
Erik Krogen
769b5a802b HDFS-14211. [SBN Read]. Add a configurable flag to enable always-msync mode to ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-07-12 12:24:44 -07:00
Erik Krogen
35200b3671 HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen. 2019-07-12 12:24:44 -07:00
Erik Krogen
98684fb821 HDFS-14279. [SBN read] Fix race condition in ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-07-12 12:24:43 -07:00
Erik Krogen
ee0416b456 HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun. 2019-07-12 12:24:43 -07:00
Konstantin V Shvachko
321e26ced1 HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2019-07-12 12:24:43 -07:00
Konstantin V Shvachko
b8f9739024 HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko. 2019-07-12 12:24:42 -07:00
Chao Sun
8f36bdc3b9 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2019-07-12 12:24:41 -07:00
Erik Krogen
67c4768825 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2019-07-12 12:24:36 -07:00
Chen Liang
8e77349bee HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang. 2019-07-12 12:23:56 -07:00
Chen Liang
6e54d97458 HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang. 2019-07-12 12:23:17 -07:00
Konstantin V Shvachko
6cb1b5aa2f HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena. 2019-07-12 12:23:16 -07:00
Chen Liang
f5c923be3f HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang. 2019-07-12 12:23:11 -07:00
Chen Liang
53a88bfdb4 HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang. 2019-07-12 12:22:25 -07:00
Chao Sun
e0436b3438 HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang. 2019-07-12 12:11:19 -07:00
Chao Sun
8100dcbb34 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2019-07-12 12:11:18 -07:00
Erik Krogen
07558629db HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun. 2019-07-12 11:50:04 -07:00
Chen Liang
8ec17f6aea HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-07-12 11:50:03 -07:00
Erik Krogen
504cb70619 HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-07-12 11:50:03 -07:00
Konstantin V Shvachko
1be8e187b1 HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-12 11:50:02 -07:00
Konstantin V Shvachko
1ad1c800a4 HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-12 11:50:02 -07:00
Chen Liang
ee9977cc69 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-07-12 11:50:01 -07:00
Erik Krogen
849a6c6f0d HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 2019-07-12 11:49:58 -07:00
Chao Sun
9840dae72f HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-07-12 11:05:07 -07:00
Konstantin V Shvachko
405f6b6fbf HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-07-12 11:05:07 -07:00
Plamen Jeliazkov
7d0c3ba700 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-07-12 11:03:31 -07:00
Erik Krogen
18e545f0be HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2019-07-12 11:03:31 -07:00
Erik Krogen
b999e96c9a HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-07-12 11:03:30 -07:00
Plamen Jeliazkov
21a886a4ef HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-07-12 11:03:27 -07:00
Weiwei Yang
9c3806cf1b HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang. 2019-07-11 09:46:27 +08:00
Inigo Goiri
aaf74ea5d7 HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException. 
(cherry picked from commit 0966407ad6fddb7adf60dba846d1f7c8a13b8ec1)
 2019-06-24 19:03:26 -07:00
Wei-Chiu Chuang
fc8983a5b8 HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu. 
(cherry picked from commit 62ad9885ea8c75c134de43a3a925c76b253658e1)
(cherry picked from commit 45543ffd6af723bb83bd44a99f12165363c70abb)
 2019-06-17 20:20:08 -07:00
huzheng
b082628e5a HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read 
(cherry picked from commit ea3b0a184405c3feca024a560807ea215b6858b9)
(cherry picked from commit 9b61ecfcbeeb1d3c320f775b43c68d3e39a271a3)
 2019-06-17 07:21:07 -07:00
Takanobu Asanuma
8098ddaf40 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:39:49 +09:00
Akira Ajisaka
f8bd5deec1
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:21 +09:00
Srinivasu Majeti
e599e3b546 HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3e5e5b028ad7e199d08e524fe7cddeee5db51a6d)
(cherry picked from commit 85f3664bb84f393f8006d3ab781d8ce5ebb1ab62)
 2019-05-17 19:22:03 +02:00
Takanobu Asanuma
1fc74178eb HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory. 
(cherry picked from commit 55fb3c32fb48ca26a629d4d5f3f07e2858d09594)
 2019-03-27 03:29:42 +09:00
Erik Krogen
cb76fc5618 HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 
(cherry picked from commit 66357574ae1da09ced735da36bf7d80a40c3fa1b)
(cherry picked from commit fec7c5f3ebbaea7b290e0904570c3a485b541a22)
 2019-03-13 13:20:51 -07:00
Dinesh Chitlangia
b63941516f HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 45caeee6cfcf1ae3355cd880402159cf31e94a8a)
(cherry picked from commit eb77e60163e318ca42548e134fe79dba7289fdc6)
 2019-01-25 07:38:11 -08:00
Wangda Tan
31ea2f7806 Preparing for 3.1.3 development 
Change-Id: I3c3d3ee47dc4fef239127b4452ff14676fa26e3d
 2019-01-07 10:04:58 -08:00
Brahma Reddy Battula
0b83c95ff4 HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena. 
(cherry picked from commit f534736867eed962899615ca1b7eb68bcf591d17)
 2018-11-30 00:27:37 +05:30
Inigo Goiri
f65324aa47 HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 
(cherry picked from commit 887244de4adebe27693ed4ad3296a6f700cfa8c1)
 2018-11-06 11:20:33 -08:00
Xiao Chen
399645ebc1 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824a36b90ba1c8e9b2fba1162771700fe)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java
 2018-10-29 19:13:51 -07:00
Xiao Chen
96cedb87b9 Fix potential FSImage corruption. Contributed by Daryn Sharp and Vinayakumar B. 
(cherry picked from commit f1996ccbaee734d423caa9d47a571cfff98ef42c)
 2018-10-24 16:10:25 -07:00
Xiao Chen
d1749fbaae HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a0250ac3f2341ebe8a14d19b073e6224c8)
 2018-10-15 11:00:30 -07:00
Xiao Chen
6a1ce74fb1 HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.

(cherry picked from commit 5ec86b445cc492f52c33639efb6a09a0d2f27475)
(cherry picked from commit e93e401f492c7fc112da3d0c63b3b186c1b196ec)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
 2018-10-12 12:08:26 -07:00