Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,384 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3697 Commits

Author SHA1 Message Date
Jim Brennan f31c122ec9 MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722). Contributed by Ahmed Hussein 
(cherry picked from commit 4d323cd47c)
 2021-02-26 21:51:47 +00:00
Kengo Seki 5f969ced67
HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 
(cherry picked from commit fd7de2b82a)
 2021-02-20 19:16:30 +09:00
He Xiaoqiao 1ded537146
HADOOP-17501. Fix logging typo in ShutdownHookManager. Contributed by Fengnan Li. 2021-01-31 20:01:43 +08:00
He Xiaoqiao b30e9198bb
HADOOP-16947. Stale record should be remove when MutableRollingAverages generating aggregate data. Contributed by Felix N, Haibin Huang. 2021-01-30 12:11:44 +08:00
Akira Ajisaka 130d5af258
HADOOP-17478. Improve the description of hadoop.http.authentication.signature.secret.file (#2628) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit d82009599a)
 2021-01-24 22:59:54 +09:00
Takanobu Asanuma 3dcb06a1ba
HADOOP-16841. The description of hadoop.http.authentication.signature.secret.file contains outdated information. Contributed by Xieming Li. 
(cherry picked from commit 5cbc4c5461)
 2021-01-24 22:59:46 +09:00
Anton Kutuzov 0ed997abab HDFS-15632. AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases. Contributed by Anton Kutuzov. 
(cherry picked from commit 91d4ba57c5)
 2021-01-22 18:19:51 -08:00
Wei-Chiu Chuang e48b91c83c HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db)
 2021-01-21 17:48:14 +00:00
Yiqun Lin 1b046fb26f HADOOP-13965. Groups should be consistent in using default group mapping class. Contributed by Yiqun Lin. 
(cherry picked from commit 383aa9c9f5)
 2021-01-21 17:33:54 +00:00
He Xiaoqiao 1efadde067
HDFS-15751. Add documentation for msync() API to filesystem.md. Contributed by Konstantin V Shvachko. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
 2021-01-03 16:38:47 +08:00
David Mollitor 16fd00d62d HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.

(cherry picked from commit c90736350b)
 2020-12-22 10:19:44 -08:00
He Xiaoqiao 2c45e93682 HADOOP-17068. Client fails forever when namenode ipaddr changed. Contributed by Sean Chow. 
(cherry picked from commit fa14e4bc00)
(cherry picked from commit 3a860e876e)
(cherry picked from commit 8d7600ae6b)
(cherry picked from commit 1fe15113cf)
 2020-12-15 14:08:20 -08:00
Ahmed Hussein c46643549f HADOOP-17358. Improve excessive reloading of Configurations (#2436) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0f)
(cherry picked from commit 23fe3bdab3)
(cherry picked from commit ac3e10af63)
(cherry picked from commit 487eaa6dea)
 2020-11-12 10:55:35 -08:00
Steve Loughran 894614ee4e HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. (#2427). Contributed by Steve Loughran. 2020-11-07 17:06:17 +05:30
hchaverr 9160b0e8c8 HDFS-15623. Respect configured values of rpc.engine (#2403) Contributed by Hector Chaverri. 
(cherry picked from commit 6eacaffeea)
 2020-11-06 15:07:34 -08:00
Eric Badger 7a830d2633 HADOOP-17342. Creating a token identifier should not do kerberos name 
resolution. Contributed by Jim Brennan.

(cherry picked from commit af389d9897)
 2020-11-05 22:17:32 +00:00
Wei-Chiu Chuang fc4c98dc49 Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291)" 
This reverts commit 52f0119051.
 2020-11-04 16:19:13 -08:00
Chen Liang ac82334041 HADOOP-17336. Backport HADOOP-16005 NativeAzureFileSystem does not support setXAttr and HADOOP-16785. Improve wasb and abfs resilience on double close() calls. followup to abfs close() fix to branch-2.10. Contributed by Sally Zuo. 2020-11-03 12:00:57 -08:00
Akira Ajisaka 52f0119051 HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291) 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04c)
(cherry picked from commit dd1634ec3b)
(cherry picked from commit 1c7043b1ff)
(cherry picked from commit 166bee93af)
 2020-11-03 11:24:08 -08:00
Konstantin V Shvachko 104dd85ad8 HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b3786d6c3c)
 2020-10-12 18:02:05 -07:00
Masatake Iwasaki 0fd2ceb1b5 Publishing the bits for release 2.10.1 2020-09-22 11:48:18 +09:00
Owen O'Malley ca65409836 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
(cherry picked from commit fa6a8b78d4)

 Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpengo.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestAuthenticationWithProxyUserFilter.java
 2020-09-14 21:48:17 +09:00
Masatake Iwasaki f4e0c14fe9 Preparing for 2.10.2 development 2020-09-13 14:33:36 +09:00
Steve Loughran a75f2915f9 HADOOP-17199. S3A Directory Marker HADOOP-13230 backport 
This backports the listing-side changes of HADOOP-13230.

With this patch in, this branch of Hadoop is compatible with S3A clients
which do not delete directory markers when files are created underneath.

It does not allow this version to disable marker deletion; if the
fs.s3a.marker.retention option is changed to request this, a message
is printed at INFO and the policy remains at "delete"

The s3guard bucket-info command has been extended to support
probing for marker retention, as has the hasPathCapability method on
S3AFileSystem.

Read the documentation!

Change-Id: Ief23c39256b194abcc445231b5f7a018185ea9f6
 2020-09-03 18:29:30 +01:00
sguggilam e645204733
HADOOP-17159. Make UGI support forceful relogin from keytab ignoring the last login time (#2245) 
Contributed by Sandeep Guggilam.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
 2020-08-27 15:21:20 -07:00
sguggilam 4eb1f81876
HADOOP-17164 UGI loginUserFromKeytab doesn't set the last login time (#2194) 
Contributed by Sandeep Guggilam.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
 2020-08-05 09:04:01 -07:00
Chen Liang 3044a13080 HDFS-15404. ShellCommandFencer should expose info about source. Contributed by Chen Liang. 
(cherry picked from commit 3833c616e0)
(cherry picked from commit 370efc6d95)
(cherry picked from commit 1e33696bc8)
 2020-07-20 15:26:44 -07:00
Akira Ajisaka e53186954d HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li. 
(cherry picked from commit 1defe3a65a)
(cherry picked from commit 0d949d375e)
(cherry picked from commit 635067d03d)
 2020-07-20 13:39:00 -07:00
Erik Krogen f36d524a10 HADOOP-17127. Use RpcMetrics.TIMEUNIT to initialize rpc queueTime and processingTime. Contributed by Jim Brennan. 2020-07-15 08:33:42 -07:00
Chris Douglas d50f5b7df4 HADOOP-14354. SysInfoWindows is not thread safe. Contributed by Inigo Goiri 
(cherry picked from commit 689fb2fc66)
 2020-07-10 14:37:05 +09:00
Akira Ajisaka 3969327077 HADOOP-12940. Fix warnings from Spotbugs in hadoop-common. 
Conflicts:
	hadoop-common-project/hadoop-common/dev-support/findbugsExcludeFile.xml
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ECSchema.java
 2020-07-10 13:38:08 +09:00
Mingliang Liu 8e01bd317b
HADOOP-17059. ArrayIndexOfboundsException in ViewFileSystem#listStatus. Contributed by hemanthboyina 2020-06-10 09:03:57 -07:00
Kihwal Lee 14ff6171a5 HADOOP-16255. Add ChecksumFs.rename(path, path, boolean). Contributed by Jungtaek Lim. 2020-06-08 17:31:16 -05:00
Mingliang Liu dcc6d63828
HADOOP-17047. TODO comment exist in trunk while related issue HADOOP-6223 is already fixed. Contributed by Rungroj Maipradit 2020-06-08 11:34:01 -07:00
Dhiraj e49573631c
HADOOP-17052. NetUtils.connect() throws unchecked exception (UnresolvedAddressException) causing clients to abort (#2036) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-06-01 11:08:32 -07:00
Masatake Iwasaki e2c7c8d96f HADOOP-17045. `Configuration` javadoc describes supporting environment variables, but the feature is not available. 2020-05-18 11:11:46 +09:00
Virajith Jalaparti cf12c1349d Revert "HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov" 
This reverts commit e6b7112c88.
 2020-05-13 08:18:43 -07:00
Chris Douglas e6b7112c88 HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov 
(cherry picked from commit 6d3d01d7ad)
 2020-05-13 07:44:50 -07:00
Andrew Wang ae2bc55bfc HADOOP-8299. ViewFileSystem link slash mount point crashes with IndexOutOfBoundsException. Contributed by Manoj Govindassamy. 
(cherry picked from commit 22ff0eff4d)
 2020-05-13 06:56:15 -07:00
Virajith Jalaparti 284a1e58d7 HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun. 2020-05-12 20:41:19 -07:00
Dhiraj 23b15089ad
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-25 13:54:26 -07:00
Masatake Iwasaki 34862e4ff8 HADOOP-16647. Support OpenSSL 1.1.1 LTS. Contributed by Rakesh Radhakrishnan. 
(cherry picked from commit 8f8be6b92a)
 2020-04-20 13:22:45 +09:00
Yufei Gu 4f47d9e52b HADOOP-15062. TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9. Contributed by Miklos Szegedi. 
(cherry picked from commit 138c1ed566)
 2020-04-20 13:16:32 +09:00
Ravi Prakash 2096113d06 HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash. 
(cherry picked from commit 94ca52ae9e)
 2020-04-20 12:53:09 +09:00
Kihwal Lee 3b86011c24 HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma. 
(cherry picked from commit b52fd05d42)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractAppendTest.java
 2020-02-25 12:28:56 -06:00
Wei-Chiu Chuang f87efc1211 HADOOP-16868. ipc.Server readAndProcess threw NullPointerException. Contributed by Tsz-wo Sze. 
(cherry picked from commit cb3f3cca01)
(cherry picked from commit d411a849e1)
(cherry picked from commit b28347d282)
 2020-02-18 21:54:12 -08:00
Kihwal Lee 9cc63d1f63 HADOOP-15787. TestIPC.testRTEDuringConnectionSetup fails with jdk8u242. Contributed by Zsolt Venczel. 
(cherry picked from commit a463cf75a0)
(cherry picked from commit 0b19c8e086)
 2020-02-05 13:33:30 -06:00
Chen Liang b97fdb7b4c HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang. 2020-02-04 12:59:31 -08:00
Kihwal Lee 8d4671b611 HADOOP-16683. Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException. Contributed by Adam Antal 
(cherry picked from commit 3d249301f4)
 2020-01-24 11:08:24 -06:00
Kihwal Lee 2873a705f4 HADOOP-16580. Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException. Contributed by Adam Antal 
(cherry picked from commit c79a5f2d99)
 2020-01-24 11:07:06 -06:00