Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,038 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

367 Commits

Author SHA1 Message Date
Chen Liang 1ff68d0e47 HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang 2019-09-03 17:18:41 -07:00
Chen Liang 163cb4d5ae HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-09-03 17:11:29 -07:00
Chen Liang e63619a6dd HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang 2019-09-03 17:11:29 -07:00
Chen Liang a06e35a197 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2019-09-03 17:11:28 -07:00
Masatake Iwasaki 670de354b3 HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit da0006fe04)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java

(cherry picked from commit d7ca016d63)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsUrl.java
 2019-08-14 12:35:14 +09:00
Chen Liang 097710131a [SBN read] Addendum: Misc fix of issues, javadoc, whitespace unused backported fields etc 2019-08-09 15:49:58 -07:00
Erik Krogen 2fe15053ab HDFS-14279. [SBN read] Fix race condition in ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-08-09 15:49:58 -07:00
Erik Krogen c377540704 HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen. 2019-08-09 15:49:58 -07:00
Erik Krogen 5190fd64be HDFS-14211. [SBN Read]. Add a configurable flag to enable always-msync mode to ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-08-09 15:49:58 -07:00
Erik Krogen c055e7812c HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen. 2019-08-09 15:49:58 -07:00
Konstantin V Shvachko 3804b8e7be HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2019-08-09 15:49:57 -07:00
Konstantin V Shvachko 3ec2ff756d HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko. 2019-08-09 15:49:57 -07:00
Chao Sun 41e85ccf44 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2019-08-09 15:49:57 -07:00
Erik Krogen a86d955aa7 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2019-08-09 15:49:57 -07:00
Chen Liang 68da1e48f9 HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang. 2019-08-09 15:49:57 -07:00
Chen Liang 3c73819a63 HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang. 2019-08-09 15:49:57 -07:00
Konstantin V Shvachko 3518767ce7 HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena. 2019-08-09 15:49:57 -07:00
Chen Liang c6267bb5ba HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang. 2019-08-09 15:49:57 -07:00
Chen Liang 17fac44040 HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang. 2019-08-09 15:49:57 -07:00
Chao Sun 697207c487 HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang. 2019-08-09 15:49:57 -07:00
Chao Sun 40d5cc15b2 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2019-08-09 15:49:57 -07:00
Erik Krogen a7d551efd5 HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun. 2019-08-09 15:49:56 -07:00
Chen Liang bf9d8ba66b HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-08-09 15:49:56 -07:00
Erik Krogen 7ccb2a48ff HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen. 
Also add a missing change from HDFS-13610
 2019-08-09 15:49:56 -07:00
Konstantin V Shvachko c09665d14a HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-08-09 15:49:56 -07:00
Konstantin V Shvachko fcafa72958 HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-08-09 15:49:56 -07:00
Chen Liang 9f90bcadf2 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-08-09 15:49:56 -07:00
Erik Krogen 33d8aae555 HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 
Also fix a issue in IPCLoggerChannel
 2019-08-09 15:49:56 -07:00
Chao Sun 04eefe245a HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-08-09 15:49:55 -07:00
Konstantin V Shvachko 272a198175 HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-08-09 15:49:55 -07:00
Plamen Jeliazkov 3ed8d45660 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Erik Krogen c450e69551 HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2019-08-09 15:49:55 -07:00
Erik Krogen d0c85f83da HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Plamen Jeliazkov dd70e6d364 HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Erik Krogen ce12c8fc54 HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. 2019-08-08 10:18:00 -07:00
Erik Krogen a503662d6c HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira. 
(cherry picked from e7a0b8aa83)
(cherry picked from ba4ca7d996cafa1b1786249cae2b207f0eaa1070)
(cherry picked from da548cf263)
(cherry picked from 055f824bcb)
 2019-08-02 11:53:34 -07:00
Wei-Chiu Chuang 3941b3c708 HDFS-14464. Remove unnecessary log message from DFSInputStream. Contributed by Chao Sun. 2019-07-30 11:36:56 -07:00
Weiwei Yang 3f112e7347 HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang. 
(cherry picked from commit 6c8cfd135e)
 2019-07-11 10:07:47 +08:00
stack a7c6d25dc3 HDFS-14483 Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9"" 
This is a revert of a revert, i.e. a reapplication of the HDFS-14483
patch. Original commit was missing the JIRA number.

Revert "Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9""

This reverts commit b26ad8a4c3.
 2019-07-09 08:58:16 -07:00
stack b26ad8a4c3 Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9" 
This reverts commit 4f09a6bcfd.
 2019-07-09 08:57:57 -07:00
sunlisheng 4f09a6bcfd Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9 
Signed-off-by: stack <stack@apache.org>
 2019-07-09 08:54:37 -07:00
stack bb7e143d34 HDFS-14585 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9 
Reapply but include JIRA # in the commit message this time.

Revert "Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9""

This reverts commit 1dd54163fc.
 2019-07-01 11:08:05 -07:00
stack 1dd54163fc Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9" 
This reverts commit bccb0fd846.
 2019-07-01 11:07:43 -07:00
sunlisheng bccb0fd846 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: stack <stack@apache.org>
 2019-07-01 10:59:58 -07:00
Inigo Goiri 1373889ac2 HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException. 
(cherry picked from commit 0966407ad6)
 2019-06-24 19:04:33 -07:00
Wei-Chiu Chuang ce49f8a5df HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu. 
(cherry picked from commit 62ad9885ea)
(cherry picked from commit 45543ffd6a)
(cherry picked from commit fc8983a5b8)
(cherry picked from commit 79f8e4d5bc)
 2019-06-17 20:20:34 -07:00
huzheng 8b90c69357 HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read 
(cherry picked from commit ea3b0a1844)
(cherry picked from commit 9b61ecfcbe)
(cherry picked from commit b082628e5a)
(cherry picked from commit 2b9c4fccf4)
 2019-06-17 07:22:01 -07:00
Siyao Meng cecba551aa HDFS-14514. Actual read size of open file in encryption zone still larger than listing size even after enabling HDFS-11402 in Hadoop 2. Contributed by Siyao Meng, Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Stephen O'Donnell <sodonnell@cloudera.com>
 2019-05-29 17:28:49 -07:00
Takanobu Asanuma e3d4f6bfed HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:41:21 +09:00
Akira Ajisaka cea1b0000c
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:54 +09:00