Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,510 Commits 326 Branches 371 Tags
Commit Graph

4810 Commits

Author SHA1 Message Date
Erik Krogen
0de8b55a09 HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed. 
(cherry-picked from 8c95cb9d6bef369fef6a8364f0c0764eba90e44a)
 2019-03-18 08:46:53 -07:00
Ben Roling
43e8ac6097
HADOOP-15625. S3A input stream to use etags/version number to detect changed source files. 
Author: Ben Roling <ben.roling@gmail.com>

Initial patch from Brahma Reddy Battula.
 2019-03-14 19:46:34 +00:00
Erik Krogen
fec7c5f3eb HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 
(cherry picked from commit 66357574ae1da09ced735da36bf7d80a40c3fa1b)
 2019-03-13 13:19:18 -07:00
Steve Loughran
b6f6c34223
HADOOP-16109. Parquet reading S3AFileSystem causes EOF 
Nobody gets seek right. No matter how many times they think they have.

Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran
 2019-03-11 11:15:25 +00:00
Praveen Krishna
451844fee5
HADOOP-16114. NetUtils#canonicalizeHost gives different value for same host. 
Author:    Praveen Krishna <praveenkrishna@tutanota.com>
(cherry picked from commit 2b94e51a8f17548e5ef6af8e1e934a94031bb1dc)
 2019-03-07 11:08:48 +00:00
Stephen O'Donnell
3fe31b36fa
HADOOP-16140. hadoop fs expunge to add -immediate option to purge trash immediately. 
Contributed by Stephen O'Donnell.

(cherry picked from commit 686c0141eff0886c285b8e52fddade43c1ce4570)
Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-03-05 14:11:49 +00:00
Daryn Sharp
ae8839e6e8 HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a87e458432609b7a35a2abd6410b02e8a2ffc974)
 2019-02-20 18:21:14 -08:00
Akira Ajisaka
3e3275a80b
YARN-9308. fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit dabfeab7854aab9b1eacf05bca954f2cf4e5ab89)
 2019-02-15 14:53:36 +09:00
Vinayakumar B
49af36b25c HADOOP-16108. Tail Follow Interval Should Allow To Specify The Sleep Interval To Save Unnecessary RPC's. Contributed by Ayush Saxena. 2019-02-13 16:45:34 +05:30
Wangda Tan
649da5af04 Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa221a336ef0dbb1a17358b1ad6f9356)
 2019-02-05 14:05:51 -08:00
Steve Loughran
bdd17be9ec
HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests. 
Contributed by Ewan Higgs and Steve Loughran.

(cherry picked from commit c1d24f848345f6d34a2ac2d570d49e9787a0df6a)
 2019-02-04 17:10:19 +00:00
Steve Loughran
3b47371fe2
HADOOP-16079. Token.toString faulting if any token listed can't load. 
Contributed by Steve Loughran.

(cherry picked from commit 7f46d13dac8cf85b094f41b3dd68e02c69e5afbc)
 2019-02-01 14:34:11 +00:00
Eric Yang
65d2b13dc2 HADOOP-15922. Fixed DelegationTokenAuthenticator URL decoding for doAs user. 
Contributed by He Xiaoqiao

(cherry picked from commit 0dd35e218fd4d6c660fd064e893be3112c546c9f)
 2019-01-22 19:39:05 -05:00
Chen Liang
54d4abdfe9 HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian. 2019-01-18 11:54:20 -08:00
luhuachao
18becec548
HADOOP-16019. ZKDelegationTokenSecretManager won't log exception message occured in function setJaasConfiguration. 
Contributed by luhuachao.

(cherry picked from commit ccc43621aba4f1fc9ca880e294a66862493805b0)
 2019-01-15 13:16:07 +00:00
Inigo Goiri
c1c15c1bd8 HADOOP-16045. Don't run TestDU on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 35fa3bd685605d8b3639e6c5cbe83cd9acd8cbe7)
 2019-01-11 18:08:14 -08:00
Akira Ajisaka
d345b1f441
HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper 
(cherry picked from commit d4ca907da636892e4ab98e232fe0f7f77f1f7aac)
 2019-01-11 13:04:40 +09:00
Jason Lowe
6236de9e23 Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh." 
This reverts commit 1f39eae7e6f59206b86f96063ffb2ebe15a9cbe1.
 2019-01-09 17:41:52 -06:00
Eric Yang
febafd0e4f HADOOP-15996. Improved Kerberos username mapping strategy in Hadoop. 
Contributed by Bolke de Bruin

(cherry picked from commit d43af8b3db4743b4b240751b6f29de6c20cfd6e5)
 2019-01-04 18:02:30 -05:00
Sunil G
7d752c1b9b HADOOP-15997. KMS client uses wrong UGI after HADOOP-14445. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 51427cbdfb39cb6f5774b7b70009d7ee4388edfc)
 2019-01-04 21:49:59 +05:30
Pranay Singh
1f39eae7e6 HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit ecdeaa7e6ad43555031aed032e6ba7a14a17d7bc)
 2019-01-03 09:57:08 -08:00
Surendra Singh Lilhore
5ab79f6377 HDFS-14166. Ls with -e option not giving the result in proper format. Contributed by Shubham Dewan. 
(cherry picked from commit 763e96ea2b18f584ba102b68271263be8c5d621c)
 2018-12-22 12:39:51 +05:30
Jason Lowe
9c89e2ea76 HADOOP-15973. Configuration: Included properties are not cached if resource is a stream. Contributed by Eric Payne 
(cherry picked from commit 3961690037fa4274a3640104b3c344e8626caa10)
 2018-12-20 09:33:22 -06:00
Takanobu Asanuma
1b941a680d HDFS-13661. Ls command with e option fails when the filesystem is not HDFS. 
(cherry picked from commit d9635759182b614a1dd5034c30978e7c4be8d0dd)
 2018-12-20 10:05:02 +09:00
Steve Loughran
bbe1ddd309
HADOOP-15808. Harden Token service loader use. 
Contributed by Steve Loughran.

(cherry picked from commit 202926ac3301298753abd0e6e1f324caf0202ec6)
 2018-12-11 17:34:16 +00:00
Steve Loughran
05e41dd4fe
HADOOP-15966. Hadoop Kerberos broken on macos as java.security.krb5.realm is reset. 
Contributed by Steve Loughran

(cherry picked from commit db2d8b01c65fdff3acf83b4c4f0a79fe48270487)
 2018-12-04 15:36:50 +00:00
Giovanni Matteo Fumarola
7cfcabc473 HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak. 2018-12-03 12:44:19 -08:00
Eric Yang
69be6e7352 HADOOP-15922. Revert patch 004. 
(cherry picked from commit 38ea3814bd0641d895e5d3b7415c6308e7f8491e)
 2018-11-30 12:51:58 -05:00
Akira Ajisaka
b722bfd6f5
MAPREDUCE-7165. mapred-site.xml is misformatted in single node setup document. Contributed by Zhaohui Xin. 
(cherry picked from commit c9bfca217f4b15a3a367db51147d0dc2075ca274)
 2018-11-30 13:30:32 +09:00
Steve Loughran
1a448565a8
HADOOP-15959. Revert "HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple" 
This reverts commit 829a2e4d271f05afb209ddc834cd4a0e85492eda.

(cherry picked from commit d0edd37269bb40290b409d583bcf3b70897c13e0)
 2018-11-29 17:53:23 +00:00
Eric Yang
f089796251 HADOOP-15922. Fixed doAsUser decoding for DelegationTokenAuthenticationFilter. 
Contributed by He Xiaoqiao

(cherry picked from commit a9d96948ede11d4d8a8fb58b31b150af84e118d9)
 2018-11-26 13:52:42 -05:00
Vinayakumar B
1b937d701d HADOOP-15948. Inconsistency in get and put syntax if filename/dirname contains space. Contributed by Ayush Saxena. 
(cherry picked from commit 23b441c2253bcb3be5229d3c5eb1e165369070c0)
 2018-11-26 18:13:09 +05:30
Akira Ajisaka
f6227367fb
Add 2.9.2 release notes and changes documents. 
(cherry picked from commit 1a00b4e325146988375c9ce5b11016c45f059a4e)
 2018-11-20 14:25:48 +09:00
Surendra Singh Lilhore
86deff0a06 HADOOP-15869. BlockDecompressorStream#decompress should not return -1 in case of IOException. Contributed by Surendra Singh Lilhore 
(cherry picked from commit 75291e6d53c13debf45493a870a898b63779914b)
 2018-11-13 21:26:22 +05:30
Steve Loughran
d4afeac285
HADOOP-15110. Gauges are getting logged in exceptions from AutoRenewalThreadForUserCreds. 
Contributed by LiXin Ge.

(cherry picked from commit 3c9d97b8f7d6eb75f08fc6d37cee37c22760bb86)
 2018-11-12 12:58:53 +00:00
Jason Lowe
9265934201 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:47:30 -06:00
Lars Francke
a15bcf97a4
HADOOP-15687. Credentials class should allow access to aliases. 
Author:    Lars Francke <lars.francke@gmail.com>

(cherry picked from commit cb8d679c95642842efacc5d38ccf2a61b043c689)
 2018-11-03 16:29:56 +00:00
Steve Loughran
d79ea3844c
HADOOP-15855. Review hadoop credential doc, including object store details. 
Contributed by Steve Loughran.

(cherry picked from commit 62d98ca92aee15d1790d169bfdf0043b05b748ce)
 2018-10-30 15:58:44 +00:00
Wei-Chiu Chuang
5f8dd40647 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." 
This reverts commit fd02c501c0cea3ec55956e11b390111519cedc00.
 2018-10-28 17:05:45 -07:00
Wei-Chiu Chuang
fd02c501c0 HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 
(cherry picked from commit fb2b72e6fce019130e10964a644b94cddbab1c06)
 2018-10-25 09:36:28 -07:00
Akira Ajisaka
63396beab4
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 
(cherry picked from commit 8f97d6f2cdfccefba5457ae3d561e9ce0109da3f)
 2018-10-24 13:50:30 +09:00
Steve Loughran
85bf008405
HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit cb8d11e7bf5055508949a6a4f074efacc8cb1305)
 2018-10-22 11:08:30 +01:00
Sunil G
bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Jason Lowe
30fc5966a2 HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe 
(cherry picked from commit 9abda83947a5babfe5a650b3409ad952f6782105)
 2018-10-17 14:40:57 -05:00
Steve Loughran
88ea877ac0
HADOOP-15861. Move DelegationTokenIssuer to the right path. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit 41b3603b5bcb74a7d78a314a4a5c177d941af27f)
 2018-10-17 11:02:47 +01:00
Márton Elek
7b06383163 HADOOP-15857. Addendum. Remove ozonefs class name definition from core-default.xml. Contributed by Elek, Marton. 2018-10-16 15:51:59 +02:00
Sunil G
c5bf07154f HADOOP-15857. Remove ozonefs class name definition from core-default.xml. Contributed by Elek, Marton. 2018-10-16 18:56:49 +05:30
Xiao Chen
ced2596094 HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a0250ac3f2341ebe8a14d19b073e6224c8)
 2018-10-15 11:00:21 -07:00
Xiao Chen
9cb0654fbb HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.

(cherry picked from commit 5ec86b445cc492f52c33639efb6a09a0d2f27475)
 2018-10-12 12:08:08 -07:00
Steve Loughran
1ee09906ce
HADOOP-15839. Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys. 
Contributed by Steve Loughran.

(cherry picked from commit cdc4350718055189fef8c70e31314607001d4009)
 2018-10-10 17:02:00 +01:00