Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,230 Commits 326 Branches 371 Tags
Commit Graph

3782 Commits

Author SHA1 Message Date
Dhiraj
23b15089ad
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-25 13:54:26 -07:00
Masatake Iwasaki
34862e4ff8 HADOOP-16647. Support OpenSSL 1.1.1 LTS. Contributed by Rakesh Radhakrishnan. 
(cherry picked from commit 8f8be6b92a32982c2fd5be4647ebe1610a533dc3)
 2020-04-20 13:22:45 +09:00
Yufei Gu
4f47d9e52b HADOOP-15062. TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9. Contributed by Miklos Szegedi. 
(cherry picked from commit 138c1ed5660f713d24bfebc44ea1846f76c00cb9)
 2020-04-20 13:16:32 +09:00
Ravi Prakash
2096113d06 HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash. 
(cherry picked from commit 94ca52ae9ec0ae04854d726bf2ac1bc457b96a9c)
 2020-04-20 12:53:09 +09:00
Kihwal Lee
3b86011c24 HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma. 
(cherry picked from commit b52fd05d42d9a76f6936a5d86c23fcd66244fe3d)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractAppendTest.java
 2020-02-25 12:28:56 -06:00
Wei-Chiu Chuang
f87efc1211 HADOOP-16868. ipc.Server readAndProcess threw NullPointerException. Contributed by Tsz-wo Sze. 
(cherry picked from commit cb3f3cca01d6ab9f4befb1bcd31f384c37c0231a)
(cherry picked from commit d411a849e10910d24cf1bbc8d64405470e00f510)
(cherry picked from commit b28347d282474ec1bd6b978b290873d05d1bd320)
 2020-02-18 21:54:12 -08:00
Kihwal Lee
9cc63d1f63 HADOOP-15787. TestIPC.testRTEDuringConnectionSetup fails with jdk8u242. Contributed by Zsolt Venczel. 
(cherry picked from commit a463cf75a0ab1f0dbb8cfa16c39a4e698bc1a625)
(cherry picked from commit 0b19c8e0860bf558d995951ab9ac4cd8a1674759)
 2020-02-05 13:33:30 -06:00
Chen Liang
b97fdb7b4c HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang. 2020-02-04 12:59:31 -08:00
Kihwal Lee
8d4671b611 HADOOP-16683. Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException. Contributed by Adam Antal 
(cherry picked from commit 3d249301f42130de80ab3ecf8c783bb51b0fa8a3)
 2020-01-24 11:08:24 -06:00
Kihwal Lee
2873a705f4 HADOOP-16580. Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException. Contributed by Adam Antal 
(cherry picked from commit c79a5f2d9930f58ad95864c59cd0a6164cd53280)
 2020-01-24 11:07:06 -06:00
sunlisheng
dc010b9844 HADOOP-16793. Redefine log level when ipc connection interrupted in Client#handleSaslConnectionFailure(). 
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
(cherry picked from commit d887e49dd4ed2b94bbb53b7608586f5da6cee037)
 2020-01-21 18:56:41 +09:00
Chen Liang
6deb6be252 HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang. 
(cherry picked from commit 26a969ec734dbdbf1d544f486dfa33f15c291789)
 2020-01-10 17:19:59 -08:00
Daryn Sharp
23a77050e0 HADOOP-16749. Configuration parsing of CDATA values are blank 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-01-10 10:48:45 -06:00
Chen Liang
82bc4778ef HADOOP-16789. In TestZKFailoverController, restore changes from HADOOP-11149 that were dropped by HDFS-6440. Contributed by Jim Brennan. 2020-01-03 16:07:56 -08:00
Erik Krogen
a22698fa52 HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen. 
(cherry picked from c174d50b30abc08a4642614fb35165e79792608b)
(cherry picked from 3402c87353c18730cb659ac1d8b43b4e21c634f6)
(cherry picked from b06965ae32c4be2e6923a2f057290fbdbda5f38a)
 2019-12-11 09:41:40 -08:00
Jonathan Hung
d74f64aa99 Setting version to 2.10.1-SNAPSHOT after branch-2 to branch-2.10 rename 2019-12-09 11:49:59 -08:00
Mingliang Liu
cdb0cafff5
HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN. Contributed by Mingliang Liu 
This closes #1733
 2019-12-05 17:56:02 -08:00
Erik Krogen
0ac6dc79b0 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq. 
(cherry picked from be77231452baee2c3dd68f7c0991411cae2eea1b)
(cherry picked from c1ed2474595c672d95237c42a6e2a51913f14d25)
(cherry picked from 23f25418e822f90aa6bd0daa0ba65ab548b05eb1)
 2019-12-03 14:19:27 -08:00
Yiqun Lin
fc2cdb6cca HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago. 
(cherry picked from commit 2b452b4e6063072b2bec491edd3f412eb7ac21f3)
 2019-11-28 10:45:25 +08:00
Chen Liang
238781ab98 HDFS-14991. Backport better time precision of configuration#getTimeDuration to branch-2 to support SBN read. Contributed by Chen Liang. 2019-11-18 20:26:27 -08:00
Akira Ajisaka
81060b3413
HADOOP-15097. AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path. Contributed by Xieming Li. 
(cherry picked from commit 92c28c100ee1aa414948cd510321ad13cb8639bc)
 2019-11-15 13:53:39 +09:00
Chen Liang
db887f66d8 HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko. 2019-11-06 15:47:57 -08:00
Duo Zhang
e649b30b32 HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches 
Reapply but this time with proper patch.

Signed-off-by: stack <stack@apache.org>
 2019-11-01 09:22:23 -07:00
stack
e880347d53 Revert "HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches" 
Wrong patch applied; missed jdk7 handling

This reverts commit a36dbe6f354b0318c944177833f4952a9136ef58.
 2019-11-01 09:21:33 -07:00
Jonathan Hung
fab62a145d Add 2.10.0 release notes for HDFS-12943 
(cherry picked from commit ef9d12df24c0db76fd37a95551db7920d27d740c)
(cherry picked from commit 9ccfc2d6d3f3819da58ea9f0ff056ed03970a6e1)
(cherry picked from commit f29481f0466350d7bba0f8ec421f51a619607dcd)
 2019-10-31 20:35:49 -07:00
Duo Zhang
a36dbe6f35 HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches 
Signed-off-by: stack <stack@apache.org>
 2019-10-30 22:33:51 -07:00
Jonathan Hung
915b795358 Make upstream aware of 2.10.0 release 
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
(cherry picked from commit 5d2ffcc7aac608746946dc2ed5711549a01a2055)
(cherry picked from commit 4d274f60bc4bf2800477800ec58b2b2c6f4f656e)
 2019-10-30 21:01:40 -07:00
Jonathan Hung
339ab2347d HDFS-14667. Backport [HDFS-14403] "Cost-based FairCallQueue" to branch-2. Contributed by Erik Krogen 2019-10-22 11:46:25 -07:00
Ayush Saxena
c78eb006f9 HADOOP-16662. Remove unnecessary InnerNode check in NetworkTopology#add(). Contributed by Lisheng Sun. 2019-10-20 11:58:30 +05:30
Jonathan Hung
108127238b Preparing for 2.11.0 development 2019-10-15 10:01:45 -07:00
Erik Krogen
8337d1a489 HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen. 2019-10-10 09:07:18 -07:00
Jonathan Hung
ab7181e9af YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:19:39 -07:00
Jonathan Hung
842b306b76 HADOOP-16588. Update commons-beanutils version to 1.9.4 in branch-2. Contributed by Wei-Chiu Chuang and Jonathan Hung. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-10-03 13:09:43 +09:00
Erik Krogen
0050f4363e HADOOP-16581. Addendum: Remove use of Java 8 functionality. Contributed by Masatake Iwasaki. 2019-09-23 08:06:27 -07:00
Yuval Degani
44193ede8d HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463) 
(cherry picked from dd0834696a694564af65a1355c9d13275f44df51)
(cherry picked from e31594f20ed43670a9ff1463a752d8b8e4b3b6f0)
(cherry picked from b091abc471e4dbeda697732809f0ee3db2622917)
 2019-09-20 09:59:14 -07:00
Kihwal Lee
ca93156cc5 HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee 
(cherry picked from commit d4205dce176287e863f567b333e0d408bf51ae6d)
 2019-09-19 08:27:23 -05:00
Wei-Chiu Chuang
ac1924896a HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina. 
(cherry picked from 0f2dad6679b7fc35474a3d33dc40b0db89bb1d80)
(cherry picked from 9c9920897de182c3da3ad33ff53e506242bbd362)
(cherry picked from 6af1ecc8e4a4067dd4a40d0c09c8d700e6fcc539)
 2019-09-18 10:31:33 -07:00
Masatake Iwasaki
3ded8ece6b HADOOP-16439. Upgrade bundled Tomcat in branch-2. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-09-04 17:26:33 +09:00
Chen Liang
1ff68d0e47 HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang 2019-09-03 17:18:41 -07:00
Chen Liang
163cb4d5ae HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-09-03 17:11:29 -07:00
Chen Liang
e63619a6dd HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang 2019-09-03 17:11:29 -07:00
Chen Liang
a06e35a197 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2019-09-03 17:11:28 -07:00
Chen Liang
94d71ba3b2 HDFS-13547. Add ingress port based sasl resolver. Contributed by Chen Liang. 2019-09-03 17:11:28 -07:00
Masatake Iwasaki
670de354b3 HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit da0006fe0473e353ee2d489156248a01aa982dfd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java

(cherry picked from commit d7ca016d63d89e5c8377a035f93485a7c77c3430)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsUrl.java
 2019-08-14 12:35:14 +09:00
Christopher Gregorian
99cd181a78 HADOOP-16459. Backport of HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian. 
This commit also includes the follow-on commit 827a84778a4e3b8f165806dfd2966f0951a5e575.

(cherry-picked from f96a2df38d889f29314c57f4d94227b2e419a11f)
(cherry-picked from d4492bdd9edec60c236aff85de50b963097e5a7f)
(cherry-picked from 7b8f08f59e5c8906930ccc67b967b7cfcbd41768)
(cherry picked from ec00431eaaa25eff5bb5e0cafb52de685187a159)
 2019-08-13 14:28:18 -07:00
Chen Liang
097710131a [SBN read] Addendum: Misc fix of issues, javadoc, whitespace unused backported fields etc 2019-08-09 15:49:58 -07:00
Konstantin V Shvachko
3804b8e7be HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2019-08-09 15:49:57 -07:00
Chao Sun
41e85ccf44 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2019-08-09 15:49:57 -07:00
Erik Krogen
7b1a3b5619 HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun. 2019-08-09 15:49:57 -07:00
Chen Liang
49bdc53b66 HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng. 2019-08-09 15:49:57 -07:00