3796 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
ac1924896a HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina.
(cherry picked from 0f2dad6679b7fc35474a3d33dc40b0db89bb1d80)
(cherry picked from 9c9920897de182c3da3ad33ff53e506242bbd362)
(cherry picked from 6af1ecc8e4a4067dd4a40d0c09c8d700e6fcc539)
2019-09-18 10:31:33 -07:00
Masatake Iwasaki
3ded8ece6b HADOOP-16439. Upgrade bundled Tomcat in branch-2.
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-09-04 17:26:33 +09:00
Chen Liang
1ff68d0e47 HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang 2019-09-03 17:18:41 -07:00
Chen Liang
163cb4d5ae HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-09-03 17:11:29 -07:00
Chen Liang
e63619a6dd HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang 2019-09-03 17:11:29 -07:00
Chen Liang
a06e35a197 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2019-09-03 17:11:28 -07:00
Chen Liang
94d71ba3b2 HDFS-13547. Add ingress port based sasl resolver. Contributed by Chen Liang. 2019-09-03 17:11:28 -07:00
Masatake Iwasaki
670de354b3 HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS.
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit da0006fe0473e353ee2d489156248a01aa982dfd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java

(cherry picked from commit d7ca016d63d89e5c8377a035f93485a7c77c3430)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsUrl.java
2019-08-14 12:35:14 +09:00
Christopher Gregorian
99cd181a78 HADOOP-16459. Backport of HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian.
This commit also includes the follow-on commit 827a84778a4e3b8f165806dfd2966f0951a5e575.

(cherry-picked from f96a2df38d889f29314c57f4d94227b2e419a11f)
(cherry-picked from d4492bdd9edec60c236aff85de50b963097e5a7f)
(cherry-picked from 7b8f08f59e5c8906930ccc67b967b7cfcbd41768)
(cherry picked from ec00431eaaa25eff5bb5e0cafb52de685187a159)
2019-08-13 14:28:18 -07:00
Chen Liang
097710131a [SBN read] Addendum: Misc fix of issues, javadoc, whitespace unused backported fields etc 2019-08-09 15:49:58 -07:00
Konstantin V Shvachko
3804b8e7be HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2019-08-09 15:49:57 -07:00
Chao Sun
41e85ccf44 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2019-08-09 15:49:57 -07:00
Erik Krogen
7b1a3b5619 HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun. 2019-08-09 15:49:57 -07:00
Chen Liang
49bdc53b66 HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng. 2019-08-09 15:49:57 -07:00
Erik Krogen
a86d955aa7 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2019-08-09 15:49:57 -07:00
Chao Sun
52d066a9d3 HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun. 2019-08-09 15:49:57 -07:00
Chao Sun
40d5cc15b2 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2019-08-09 15:49:57 -07:00
Chen Liang
b4fc93d90b HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen. 2019-08-09 15:49:56 -07:00
Chen Liang
bf9d8ba66b HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-08-09 15:49:56 -07:00
Chen Liang
9f90bcadf2 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-08-09 15:49:56 -07:00
Plamen Jeliazkov
9803d8db4c HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Chao Sun
04eefe245a HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-08-09 15:49:55 -07:00
Erik Krogen
85f81fe458 HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen.
HDFS-10519. Add a configuration option to enable in-progress edit log tailing. Contributed by Jiayi Zhou.
2019-08-09 15:49:55 -07:00
Erik Krogen
88d65af8a1 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2019-08-09 15:49:55 -07:00
Plamen Jeliazkov
3ed8d45660 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Erik Krogen
d5da529acf HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-08-09 15:49:55 -07:00
Erik Krogen
d0c85f83da HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Plamen Jeliazkov
dd70e6d364 HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Yiqun Lin
63531be505 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun. 2019-08-07 23:01:51 +08:00
Steve Loughran
e8ccc299f6
HADOOP-15237. In KMS docs there should be one space between KMS_LOG and NOTE.
Contributed by Snigdhanjali Mishra.

Change-Id: I3abaa658c8786f8afa802ccbb629551313431b0e
2019-08-01 10:37:28 +01:00
Akira Ajisaka
46c00aca21
HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich.
(cherry picked from commit cbfa3f3e988b8a3f142abadfe4f18201b6438ac9)
2019-07-29 17:40:14 -07:00
Erik Krogen
edb46353c6 HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen.
(cherry-picked from 62efb6300619670d3e0554c3ba14c264fa0c705b)
(cherry-picked from 5ec34bf43adb498797afd24bb02989d4fec25409)
(cherry-picked from e2aab0efef9d504e4695a9c1df2835ed60f6c0c7)
(cherry-picked from f7e517ee2432f3c0a40c9172413753670bf07db4)
2019-07-26 11:44:12 -07:00
Steve Loughran
c0fd09c42c
HADOOP-9844. NPE when trying to create an error message response of SASL RPC
This closes #55

Change-Id: I10a20380565fa89762f4aa564b2f1c83b9aeecdc
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 07530314c2130ecd1525682c59bf51f15b82c024)
2019-07-26 17:55:45 +09:00
Masatake Iwasaki
62fdfaf093 HADOOP-16386. FindBugs warning in branch-2: GlobalStorageStatistics defines non-transient non-serializable instance field map.
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-07-17 07:50:24 +09:00
stack
a7c6d25dc3 HDFS-14483 Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9""
This is a revert of a revert, i.e. a reapplication of the HDFS-14483
patch. Original commit was missing the JIRA number.

Revert "Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9""

This reverts commit b26ad8a4c38d27894bd42d3fba7abff80bd895f4.
2019-07-09 08:58:16 -07:00
stack
b26ad8a4c3 Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9"
This reverts commit 4f09a6bcfd685bbc9e139497adfeeeff8170e344.
2019-07-09 08:57:57 -07:00
sunlisheng
4f09a6bcfd Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9
Signed-off-by: stack <stack@apache.org>
2019-07-09 08:54:37 -07:00
stack
bb7e143d34 HDFS-14585 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9
Reapply but include JIRA # in the commit message this time.

Revert "Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9""

This reverts commit 1dd54163fcb40b60efc7ca153d443129e424df7d.
2019-07-01 11:08:05 -07:00
stack
1dd54163fc Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9"
This reverts commit bccb0fd846b7cc69e0d83140193eaa7fc0ac081d.
2019-07-01 11:07:43 -07:00
sunlisheng
bccb0fd846 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-07-01 10:59:58 -07:00
Takanobu Asanuma
64102e865f HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles.
(cherry picked from commit 54f9f75a443d7d167a7aa7d04a87e3f5af049887)
2019-06-14 10:21:30 +09:00
Erik Krogen
e9eaf21874 HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen.
(cherry-picked from 76b94c274fe9775efcfd51c676d80c88a4f7fdb9)
(cherry-picked from f607efd1950211f85eb571564b774184f8fa1687)
(cherry-picked from cf3c1b9ffc8cfce2e2248a1c1600bd34ff412933)
(cherry-picked from d43219c21d830b198710f70c16ea880302f88d77)
2019-06-07 14:23:23 -07:00
Takanobu Asanuma
e3d4f6bfed HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka.
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:41:21 +09:00
Akira Ajisaka
cea1b0000c
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:54 +09:00
Alexis Daboville
b70fdc64d3
HADOOP-16248. MutableQuantiles leak memory under heavy load.
Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac703b10de59a8ca371fe957a4a338601)
2019-05-17 15:27:47 +01:00
Yuan Gao
18f66a05cf
HADOOP-16205 Backport ABFS driver from trunk to branch 2.0: Fix build and test failures.
Contributed by Yuan Gao.
2019-04-25 18:36:14 +01:00
Steve Loughran
94d0de2eab HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch.
Contributed by Steve Loughran.
2019-04-24 21:46:25 +00:00
Thomas Marquardt
3bd089ddca HADOOP-15663. ABFS: Simplify configuration.
Contributed by Da Zhou.
2019-04-24 21:46:25 +00:00
Thomas Marquardt
b537d8c442 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests.
Contributed by Steve Loughran and Da Zhou.
2019-04-24 21:46:25 +00:00
Steve Loughran
1b64b0f924 HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit.
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
2019-04-24 21:46:25 +00:00