Commit Graph

5323 Commits

Author SHA1 Message Date
Wanqiang Ji b417a4c854
MAPREDUCE-7214. Remove unused pieces related to `mapreduce.job.userlog.retain.hours`
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-06-11 18:40:35 +09:00
Shweta Yakkali d6d95d2686 HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 10:23:18 -07:00
Erik Krogen 76b94c274f HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen. 2019-06-07 14:20:44 -07:00
Steve Loughran 4e38dafde4
HADOOP-15563. S3Guard to support creating on-demand DDB tables.
Contributed by Steve Loughran

Change-Id: I2262b5b9f52e42ded8ed6f50fd39756f96e77087
2019-06-07 18:26:10 +01:00
Huan-Ping Su 14552d19e3
HADOOP-16344. Make DurationInfo public unstable.
Second attempt: imports fixed up.

Contributed by Huan-Ping Su and Kai Xie.

Change-Id: Ib2a85dd4b2d12b840692cc854cc53ddb3bdde7d5
2019-06-07 12:55:17 +01:00
Akira Ajisaka 3ea4f41d9f
MAPREDUCE-6794. Remove unused properties from TTConfig.java 2019-06-07 10:27:41 +09:00
Steve Loughran 1f244b4227
Revert "HADOOP-16344. Make DurationInfo public unstable."
This reverts commit 829848ba2e.

Change-Id: Ied91250e191b2ba701a8fc697c78b3756ce76be8
2019-06-06 17:49:31 +01:00
Huan-Ping Su 829848ba2e
HADOOP-16344. Make DurationInfo public unstable.
Contributed by Huan-Ping Su

Change-Id: I64a94cf382f9db78b4ef49a3912f25f0d906af7c
2019-06-06 12:52:49 +01:00
Eric Yang 294695dd57 HADOOP-16314. Make sure all web end points are covered by the same authentication filter.
Contributed by Prabhu Joseph
2019-06-05 18:55:13 -04:00
Sammi Chen d1aad44490 HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00
Steve Loughran 309501c6fa
Revert "HADOOP-16050: s3a SSL connections should use OpenSSL"
This reverts commit b067f8acaa.

Change-Id: I584b050a56c0e6f70b11fa3f7db00d5ac46e7dd8
2019-06-05 13:54:55 +01:00
Steve Loughran 7724d8031b Revert "HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors"
This reverts commit 5906268f0d.
2019-06-05 13:54:42 +01:00
Steve Loughran 827a84778a
HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer -follow-on patch.
This follow-on patch to HADOOP-16266 fixes up the problem where logs were being full of
stack traces because the timeout passed down to select was in nanos, whereas the API
expected millis.

Contributed by Erik Krogen.

Change-Id: I5c6e9ddf68127b1d7e0ca0e179d036eb9941e445
2019-06-04 15:56:47 +01:00
Shweta Yakkali 6f5a36c13c HADOOP-13656. fs -expunge to take a filesystem. Contributed by Shweta.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-30 13:21:58 -07:00
Akira Ajisaka afd844059c HADOOP-16331. Fix ASF License check in pom.xml
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
Akira Ajisaka 9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Shweta 93d7630062 HDFS-14451. Incorrect header or version mismatch log message. Contributed by Shweta.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-24 18:42:46 +02:00
Christopher Gregorian f96a2df38d HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian. 2019-05-23 10:28:37 -07:00
Eric Yang ea0b1d8fba HADOOP-16287. Implement ProxyUserAuthenticationFilter for web protocol impersonation.
Contributed by Prabhu Joseph
2019-05-23 11:36:32 -04:00
Akira Ajisaka a771e2a638
HADOOP-12948. Remove the defunct startKdc profile from hadoop-common. Contributed by Wei-Chiu Chuang. 2019-05-23 13:59:42 +09:00
Sahil Takiar 5906268f0d HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors 2019-05-21 11:30:45 -06:00
Ben Roling a36274d699
HADOOP-16085. S3Guard: use object version or etags to protect against inconsistent read after replace/overwrite.
Contributed by Ben Roling.

S3Guard will now track the etag of uploaded files and, if an S3
bucket is versioned, the object version.

You can then control how to react to a mismatch between the data
in the DynamoDB table and that in the store: warn, fail, or, when
using versions, return the original value.

This adds two new columns to the table: etag and version.
This is transparent to older S3A clients -but when such clients
add/update data to the S3Guard table, they will not add these values.
As a result, the etag/version checks will not work with files uploaded by older clients.

For a consistent experience, upgrade all clients to use the latest hadoop version.
2019-05-19 22:29:54 +01:00
Alexis Daboville 4cb3da6ac7
HADOOP-16248. MutableQuantiles leak memory under heavy load.
Contributed by Alexis Daboville,
2019-05-17 15:15:22 +01:00
Sahil Takiar b067f8acaa HADOOP-16050: s3a SSL connections should use OpenSSL
(cherry picked from commit aebf229c17)
2019-05-16 08:57:54 -06:00
David Mollitor 2713dcf6e9
HADOOP-16307. Intern User Name and Group Name in FileStatus.
Author:    David Mollitor
2019-05-16 16:02:07 +02:00
Bharat Viswanadham d4c8858586
HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy. 2019-05-15 17:41:36 -07:00
Inigo Goiri 389e640f0c HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao. 2019-05-13 11:46:16 -07:00
Akira Ajisaka f257497b0f HADOOP-16299. [JDK 11] Build fails without specifying -Djavac.version=11
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-09 14:49:46 +09:00
Prabhu Joseph 96dc5cedfe
HADOOP-16293. AuthenticationFilterInitializer doc has speudo instead of pseudo.
Author:    Prabhu Joseph
2019-05-08 10:18:20 +01:00
Peter Bacsko 713e8a27ae HADOOP-16238. Add the possbility to set SO_REUSEADDR in IPC Server Listener. Contributed by Peter Bacsko.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-07 17:48:27 -07:00
Siyao Meng 93f2283a69 HADOOP-16289. Allow extra jsvc startup option in hadoop_start_secure_daemon in hadoop-functions.sh. Contributed by Siyao Meng.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-06 15:47:33 -07:00
Vinayakumar B f1875b205e HADOOP-16059. Use SASL Factories Cache to Improve Performance. Contributed by Ayush Saxena. 2019-05-03 11:22:14 +05:30
Giovanni Matteo Fumarola 7a3188d054 HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena. 2019-05-02 12:58:42 -07:00
Sahil Takiar 4877f0aa51 HDFS-3246: pRead equivalent for direct read path (#597)
HDFS-3246: pRead equivalent for direct read path

Contributed by Sahil Takiar
2019-04-30 14:52:16 -07:00
Ben Roling 0af4011580
HADOOP-16221. S3Guard: add option to fail operation on metadata write failure. 2019-04-30 11:53:26 +01:00
Sean Mackrory a703dae25e HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota. 2019-04-24 10:39:00 -06:00
Anu Engineer f4ab9370f5 HADOOP-16026:Replace incorrect use of system property user.name.
Contributed by Dinesh Chitlangia.
2019-04-22 14:02:13 -07:00
Inigo Goiri fb1c549139 HDFS-14374. Expose total number of delegation tokens in AbstractDelegationTokenSecretManager. Contributed by CR Hota. 2019-04-22 13:32:08 -07:00
Erik Krogen 1ddb48872f HADOOP-16265. Fix bug causing Configuration#getTimeDuration to use incorrect units when the default value is used. Contributed by starphin. 2019-04-22 08:16:57 -07:00
Zsombor Gegesy 008766c119 HADOOP-15014. KMS should log the IP address of the clients. Contributed by Zsombor Gegesy.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-16 05:28:18 -07:00
Kenneth Yang b1120d27ab
HADOOP-16249. Make CallerContext LimitedPrivate scope to Public.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-16 11:18:21 +09:00
Gabor Bota 1943db5571
HADOOP-16237. Fix new findbugs issues after updating guava to 27.0-jre.
Author:    Gabor Bota <gabor.bota@cloudera.com>
2019-04-12 18:28:38 -07:00
Sahil Takiar 2382f63fc0
HADOOP-14747. S3AInputStream to implement CanUnbuffer.
Author:    Sahil Takiar <stakiar@cloudera.com>
2019-04-12 18:12:02 -07:00
Chen Liang 626fec652b HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-04-12 17:37:51 -07:00
Arpit Agarwal 87407553ef
HADOOP-16243. Change Log Level to trace in NetUtils.java. Contributed by chencan. 2019-04-10 13:21:04 -07:00
Giovanni Matteo Fumarola 813cee1a18 HDFS-14420. Fix typo in KeyShell console. Contributed by Hu Xiaodong. 2019-04-10 11:18:40 -07:00
Todd Lipcon 65deb1ac42 HADOOP-16179. hadoop-common pom should not depend on kerb-simplekdc
The hadoop-common pom currently has a dependency on kerb-simplekdc. In
fact, the only classes used from Kerby are in kerb-core and kerb-util
(which is a transitive dependency frmo kerb-core). Depending on
kerb-simplekdc pulls a bunch of other unnecessary classes into the
hadoop-common classpath.

This changes the hadoop-common pom to depend only on kerb-core.

hadoop-minikdc already had the appropriate dependency on kerb-simplekdc
so it continues to pull in what it needs.

Signed-off-by: Todd Lipcon <todd@apache.org>
2019-04-10 08:49:46 -07:00
Akira Ajisaka bb8dda2bf9
HADOOP-12890. Fix typo in AbstractService. Contributed by Gabor Liptak. 2019-04-08 15:26:12 +09:00
Akira Ajisaka ab645b3caa
HADOOP-14635. Javadoc correction for AccessControlList#buildACL. Contributed by Yeliang Cang. 2019-04-08 15:18:45 +09:00
Akira Ajisaka 72f4b9cd68
HADOOP-15242. Fix typos in hadoop-functions.sh. Contributed by Ray Chiang. 2019-04-08 13:20:21 +09:00
Akira Ajisaka 0d47d283a6
HADOOP-10848. Cleanup calling of sun.security.krb5.Config. 2019-04-08 10:02:34 +09:00
David Mollitor c90736350b
HADOOP-16208. Do Not Log InterruptedException in Client.
Contributed by David Mollitor.
2019-04-04 16:15:57 +01:00
Inigo Goiri 7b5b783f66 HDFS-14327. Using FQDN instead of IP to access servers with DNS resolving. Contributed by Fengnan Li. 2019-04-03 16:11:13 -07:00
Siyao Meng e62cbcbc83 HADOOP-16011. OsSecureRandom very slow compared to other SecureRandom implementations. Contributed by Siyao Meng.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-03 14:29:52 -07:00
Steve Loughran 366186d999
HADOOP-16233. S3AFileStatus to declare that isEncrypted() is always true (#685)
This is needed to fix up some confusion about caching of job.addCache() handling of S3A paths; all parent dirs -the files are downloaded by the NM without  using the DTs of the user submitting the job. This means that when you submit jobs to an EC2 cluster with lower IAM permissions than the user, cached resources don't get downloaded and the job doesn't start.

Production code changes:
* S3AFileStatus Adds "true" to the superclass's encrypted flag during construction.

Tests
* Base AbstractContractOpenTest can control whether zero byte files created in tests are encrypted. Not done via an XML attribute, just a subclass point. Thoughts?
* Verify that the filecache considers paths to not have the permissions which trigger reduce-privilege downloads
* And extend ITestDelegatedMRJob to test a completely different bucket (open street map), to verify that cached resources do get their tokens picked up

Docs:
* Advise FS developers to say all files are encrypted. It's otherwise harmless and it'll stop other people seeing impossible to debug error messages on app launch.

Contributed by Steve Loughran.

Change-Id: Ifaae4c9d735ccc5eafeebd2584b65daf2d4e5da3
2019-04-03 21:23:40 +01:00
Gabor Bota d7979079ea HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota. 2019-04-03 12:59:39 -06:00
Sahil Takiar 3b0c5016b2
HDFS-14394: Add -std=c99 / -std=gnu99 to libhdfs compile flags
Signed-off-by: Todd Lipcon <todd@apache.org>
2019-04-03 10:56:33 -07:00
Akira Ajisaka aaaf856f4b
HADOOP-16226. new Path(String str) does not remove all the trailing slashes of str 2019-04-03 13:16:59 +09:00
Lokesh Jain cf268114c9 HDFS-13960. hdfs dfs -checksum command should optionally show block size in output. Contributed by Lokesh Jain.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-02 12:24:55 -07:00
Steve Loughran 61d19110d4
HADOOP-16218. Findbugs warning of null param to non-nullable method in Configuration with Guava update. (#655)
Change-Id: I461e518ce9a4730b91a8138ad55b39e9a4b0a4b8
2019-04-02 09:15:11 +01:00
Akira Ajisaka ebd0d21538
HADOOP-16225. Fix links to the developer mailing lists in DownstreamDev.md. Contributed by Wanqiang Ji. 2019-04-02 10:51:47 +09:00
Xiaoyu Yao f41f938b2e
HADOOP-16199. KMSLoadBlanceClientProvider does not select token correctly. Contributed by Xiaoyu Yao.
This closes  #642.
2019-03-28 21:55:31 -07:00
Gabor Bota b5db238383
HADOOP-15999. S3Guard: Better support for out-of-band operations.
Author:    Gabor Bota
2019-03-28 15:59:25 +00:00
David Mollitor d18d0859eb
HADOOP-16181. HadoopExecutors shutdown Cleanup.
Author:    David Mollitor <david.mollitor@cloudera.com>
2019-03-22 10:29:27 +00:00
David Mollitor 246ab77f28
HADOOP-16196. Path Parameterize Comparable.
Author:    David Mollitor <david.mollitor@cloudera.com>
2019-03-22 10:26:24 +00:00
Eric Yang 5f6e225166 YARN-9363. Replaced debug logging with SLF4J parameterized log message.
Contributed by Prabhu Joseph
2019-03-18 13:57:18 -04:00
Eric Yang 5446e3cb8a HADOOP-16167. Fixed Hadoop shell script for Ubuntu 18.
Contributed by Daniel Templeton
2019-03-18 13:04:49 -04:00
Erik Krogen 8c95cb9d6b HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed. 2019-03-18 08:13:43 -07:00
Shweta Yakkali 2db38abffc HDFS-14328. [Clean-up] Remove NULL check before instanceof in TestGSet
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I5b9f0e66664714d7c5bbfa30492a09f770626711
2019-03-18 15:10:26 +01:00
Eric Yang 2064ca015d YARN-9349. Changed logging to use slf4j api.
Contributed by Prabhu Joseph
2019-03-15 19:20:59 -04:00
Ben Roling 6fa229891e
HADOOP-15625. S3A input stream to use etags/version number to detect changed source files.
Author: Ben Roling <ben.roling@gmail.com>

Initial patch from Brahma Reddy Battula.
2019-03-13 20:37:11 +00:00
Erik Krogen 66357574ae HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 2019-03-13 13:15:56 -07:00
Matt Foley f74159c8fc HADOOP-16166. TestRawLocalFileSystemContract fails with build Docker container running on Mac.
Also provided similar fix for Windows.
2019-03-13 09:33:24 -07:00
Konstantin V Shvachko 2a54feabb2 HDFS-14347. [SBN Read] Restore a comment line mistakenly removed in ProtobufRpcEngine. Contributed by Fengnan Li. 2019-03-11 18:59:15 -07:00
Konstantin V Shvachko 4ad295a4f1 HDFS-14270.[SBN Read] Add trace level logging for stateId in RPC Server. Contributed by Shweta Yakkali. 2019-03-11 13:48:06 -07:00
Steve Loughran 0cbe9ad8c2
HADOOP-16109. Parquet reading S3AFileSystem causes EOF
Nobody gets seek right. No matter how many times they think they have.

Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran
2019-03-09 16:00:34 +00:00
Praveen Krishna 2b94e51a8f
HADOOP-16114. NetUtils#canonicalizeHost gives different value for same host.
Author:    Praveen Krishna <praveenkrishna@tutanota.com>
2019-03-07 11:06:34 +00:00
Vivek Ratnavel Subramanian a55fc36299 HDDS-1093. Configuration tab in OM/SCM ui is not displaying the correct values. 2019-03-06 17:43:57 -08:00
Sahil Takiar 618e009ac0 HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 15:04:06 -08:00
Wei-Chiu Chuang 6192c1fe3b Revert "HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar."
This reverts commit f5a4b43a49.
2019-03-06 15:02:18 -08:00
Sahil Takiar f5a4b43a49 HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 14:58:45 -08:00
Eric Yang 3c5b7136e2 HADOOP-16150. Added concat method to ChecksumFS as unsupported operation.
Contributed by Steve Loughran

(cherry picked from commit 8b517e7ad6)
2019-03-05 13:32:00 -05:00
Stephen O'Donnell 686c0141ef
HADOOP-16140. hadoop fs expunge to add -immediate option to purge trash immediately.
Contributed by Stephen O'Donnell.

Signed-off-by: Steve Loughran <stevel@apache.org>
2019-03-05 14:09:00 +00:00
Prabhu Joseph e40e2d6ad5
YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-03-05 14:10:08 +09:00
coder-chenzhi fe7551f21b
HADOOP-16162. Remove unused Job Summary Appender configurations from log4j.properties
This closes #551

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-03-05 13:41:33 +09:00
David Mollitor 9fcd89ab93
HADOOP-16148. Cleanup LineReader Unit Test.
Contributed by David Mollitor.

Signed-off-by: Steve Loughran <stevel@apache.org>
2019-03-04 23:08:12 +00:00
Ajay Kumar 0d61facd37 HADOOP-15889. Add hadoop.token configuration parameter to load tokens. Contributed by Íñigo Goiri 2019-02-28 10:34:28 -08:00
Márton Elek 84c4966a5a
HADOOP-16067. Incorrect Format Debug Statement KMSACLs. Contributed by Charan Hebri. 2019-02-28 12:15:47 +01:00
Eric Yang feccd282fe HADOOP-16107. Update ChecksumFileSystem createFile/openFile API to generate checksum.
Contributed by Steve Loughran
2019-02-27 15:53:41 -05:00
Tsz Wo Nicholas Sze 9192f71e21 HADOOP-16127. In ipc.Client, put a new connection could happen after stop. 2019-02-26 15:14:21 -08:00
Abhishek Modi 52b2eab575
HADOOP-16093. Move DurationInfo from hadoop-aws to hadoop-common org.apache.hadoop.util.
Contributed by Abhishek Modi
2019-02-26 17:10:41 +00:00
Surendra Singh Lilhore 59ba3552d3 HDFS-14299. ViewFs: Correct error message for read only operations. Contributed by hu xiaodong. 2019-02-26 12:40:32 +05:30
Konstantin V Shvachko a6ab37192a HDFS-14130. [SBN read] Make ZKFC ObserverNode aware. Contributed by xiangheng and Konstantin Shvachko. 2019-02-25 14:35:02 -08:00
Inigo Goiri ba4e7bd192 HADOOP-16125. Support multiple bind users in LdapGroupsMapping. Contributed by Lukas Majercak. 2019-02-25 13:39:13 -08:00
Tsz Wo Nicholas Sze 0edb0c51dc HADOOP-16126. ipc.Client.stop() may sleep too long to wait for all connections. 2019-02-25 13:15:28 -08:00
Yongjun Zhang f7a27cdee4 HDFS-14118. Support using DNS to resolve nameservices to IP addresses. Contributed by Fengnan Li. 2019-02-23 09:35:36 -08:00
George Huang 9daf43c6fa HADOOP-16129. Misc. bug fixes for KMS Benchmark. Contributed by George Huang.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-22 17:52:09 -08:00
Daryn Sharp a87e458432 HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-20 18:17:04 -08:00
Shweta Yakkali 371a6db59a HDFS-14273. Fix checkstyle issues in BlockLocation's method javadoc
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I546aa4a0fe7f83b53735acd9925f366b2f1a00e2
2019-02-20 15:36:53 -08:00
George Huang 0525d85d57 HADOOP-15967. KMS Benchmark Tool. Contributed by George Huang.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-19 15:24:30 -08:00
Akira Ajisaka dabfeab785
YARN-9308. fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240. Contributed by Wilfred Spiegelenburg. 2019-02-15 14:49:49 +09:00
Erik Krogen 64f28f9efa HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen. 2019-02-14 11:22:04 -08:00
Siyao Meng fd026863b1 HDFS-14241. Provide feedback on successful renameSnapshot and deleteSnapshot. Contributed by Siyao Meng.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-13 19:48:02 -08:00
Chen Liang 024c87291c HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang 2019-02-13 12:40:31 -08:00
Vinayakumar B 00c5ffaee2 HADOOP-16108. Tail Follow Interval Should Allow To Specify The Sleep Interval To Save Unnecessary RPC's. Contributed by Ayush Saxena. 2019-02-13 16:44:32 +05:30
Yiqun Lin 7b11b404a3 HADOOP-16097. Provide proper documentation for FairCallQueue. Contributed by Erik Krogen. 2019-02-13 11:16:04 +08:00
Xiaoyu Yao ca4e46a05e HDDS-1075. Fix CertificateUtil#parseRSAPublicKey charsetName. Contributed by Siddharth Wagle. 2019-02-11 12:09:14 -08:00
Steve Loughran 668817a6ce
Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth."
(accidentally mixed in two patches)

This reverts commit fa8cd1bf28.
2019-02-07 21:57:22 +00:00
Wangda Tan 308f3168fa Make upstream aware of 3.1.2 release
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
2019-02-05 14:03:18 -08:00
Da Zhou fa8cd1bf28
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
Contributed by Da Zhou and Junhua Gu.
2019-02-05 19:23:15 +00:00
Steve Loughran f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile();
S3A to implement S3 Select through this API.

The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.

The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.

This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.

In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).

Contributed by Steve Loughran
2019-02-05 11:51:02 +00:00
Steve Loughran 7f46d13dac
HADOOP-16079. Token.toString faulting if any token listed can't load.
Contributed by Steve Loughran.
2019-02-01 14:31:47 +00:00
Inigo Goiri bcc3a79f58 HADOOP-16084. Fix the comment for getClass in Configuration. Contributed by Fengnan Li. 2019-01-31 10:06:05 -08:00
Akira Ajisaka 1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Hanisha Koneru b3bc94ebfd HDFS-14236. Lazy persist copy/ put fails with ViewFs. 2019-01-29 16:45:44 -08:00
Wei-Chiu Chuang d1714c20e9 Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
This reverts commit 1d523279da.
2019-01-29 15:43:09 -08:00
Pranay Singh 1d523279da HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-25 09:02:57 -08:00
Eric Yang 0dd35e218f HADOOP-15922. Fixed DelegationTokenAuthenticator URL decoding for doAs user.
Contributed by He Xiaoqiao
2019-01-22 18:59:36 -05:00
Akira Ajisaka a463cf75a0
HADOOP-15787. [JDK11] TestIPC.testRTEDuringConnectionSetup fails. Contributed by Zsolt Venczel. 2019-01-22 10:19:05 +09:00
Sunil G 2e2508b8e3 Make 3.2.0 aware to other branches 2019-01-21 21:24:51 +05:30
Xiaoyu Yao 751bc62df7 Merge branch 'HDDS-4' into trunk 2019-01-18 14:44:24 -08:00
Surendra Singh Lilhore dacc1a759e HDFS-14175. EC: Native XOR decoder should reset the output buffer before using it. Contributed by Ayush Saxena. 2019-01-18 13:55:12 +05:30
Xiaoyu Yao 570b503e3e Fix merge conflicts 2019-01-15 22:54:50 -07:00
Xiaoyu Yao ff61931f91 HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:16:58 -07:00
Akira Ajisaka 01405df87f
HADOOP-15941. Addendum patch. Contributed by Takanobu Asanuma. 2019-01-16 10:28:08 +09:00
luhuachao ccc43621ab
HADOOP-16019. ZKDelegationTokenSecretManager won't log exception message occured in function setJaasConfiguration.
Contributed by luhuachao.
2019-01-15 13:15:18 +00:00
Steve Loughran 6d0bffe17e
HADOOP-14556. S3A to support Delegation Tokens.
Contributed by Steve Loughran and Daryn Sharp.
2019-01-14 17:59:27 +00:00
Inigo Goiri 35fa3bd685 HADOOP-16045. Don't run TestDU on Windows. Contributed by Lukas Majercak. 2019-01-11 18:07:19 -08:00
Chen Liang bf08f4abae HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian. 2019-01-11 14:01:23 -08:00
Chen Liang 01cb958af4 HADOOP-16013. DecayRpcScheduler decay thread should run as a daemon. Contributed by Erik Krogen. 2019-01-11 12:51:07 -08:00
Giovanni Matteo Fumarola fb8932a727 HADOOP-16029. Consecutive StringBuilder.append can be reused. Contributed by Ayush Saxena. 2019-01-11 10:54:49 -08:00
Akira Ajisaka d4ca907da6
HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper 2019-01-11 11:24:05 +09:00
Kihwal Lee 33c009a45d HADOOP-15909. KeyProvider class should implement Closeable. Contributed
by Kuhu Shukla.
2019-01-10 13:21:37 -06:00
Jason Lowe c634589ab2 Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
This reverts commit ecdeaa7e6a.
2019-01-09 17:24:58 -06:00
Takanobu Asanuma 709ddb10e5 HADOOP-15941. [JDK 11] Compilation failure: package com.sun.jndi.ldap is not visible. 2019-01-09 13:55:25 +09:00
shwetayakkali@cloudera.com 4ab5260b7e HDFS-14132. Add BlockLocation.isStriped() to determine if block is replicated or Striped
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I0ed8996a0bae2ad2c7d3513143195533f7191af8
2019-01-08 17:04:57 -08:00
Akira Ajisaka 7f78397036
Revert "HADOOP-14556. S3A to support Delegation Tokens."
This reverts commit d7152332b3.
2019-01-08 14:51:30 +09:00
Eric Yang 802932ca0b HADOOP-16031. Fixed TestSecureLogins unit test. Contributed by Akira Ajisaka 2019-01-07 13:21:14 -05:00
Steve Loughran d7152332b3
HADOOP-14556. S3A to support Delegation Tokens.
Contributed by Steve Loughran.
2019-01-07 13:18:03 +00:00
Eric Yang d43af8b3db HADOOP-15996. Improved Kerberos username mapping strategy in Hadoop.
Contributed by Bolke de Bruin
2019-01-04 17:54:15 -05:00
Inigo Goiri f4e18242bd HADOOP-16028. Fix NetworkTopology chooseRandom function to support excluded nodes. Contributed by Sihai Ke. 2019-01-04 09:56:58 -08:00
Sunil G 51427cbdfb HADOOP-15997. KMS client uses wrong UGI after HADOOP-14445. Contributed by Wei-Chiu Chuang. 2019-01-04 21:49:06 +05:30
Pranay Singh ecdeaa7e6a HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-03 09:56:07 -08:00
Akira Ajisaka 57198bc511
HADOOP-15962. The buffer size is small when unpacking tar archives. Contributed by BELUGA BEHR. 2018-12-28 15:38:47 +09:00
Konstantin V Shvachko bd2a59e508 HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2018-12-24 09:39:20 -08:00
Chao Sun fa8550337d HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Erik Krogen ef3e1929dd HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Chen Liang cae279610b HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng. 2018-12-24 09:34:01 -08:00
Erik Krogen b73fb70f97 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2018-12-24 09:34:01 -08:00
Chao Sun cbc2f932eb HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Chao Sun 8c49135078 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Chen Liang a65bb97f5d HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen. 2018-12-24 09:34:00 -08:00
Chen Liang d2e85b0b6d HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2018-12-24 09:34:00 -08:00
Chen Liang 66ec3a67e8 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2018-12-24 09:34:00 -08:00
Plamen Jeliazkov e27708c2da HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2018-12-24 09:34:00 -08:00
Chao Sun 64b7cf59bd HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Erik Krogen 151c8ddbe4 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2018-12-24 09:33:59 -08:00
Plamen Jeliazkov e880660a20 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2018-12-24 09:33:59 -08:00
Erik Krogen 091ad974cd HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Erik Krogen 9bf0696c73 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2018-12-24 09:33:59 -08:00
Plamen Jeliazkov c9d73437e8 HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2018-12-24 09:33:59 -08:00
Yiqun Lin feb2664ac4 HDFS-13946. Log longest FSN write/read lock held stack trace. 2018-12-22 23:09:59 +08:00
Surendra Singh Lilhore 763e96ea2b HDFS-14166. Ls with -e option not giving the result in proper format. Contributed by Shubham Dewan. 2018-12-22 12:22:24 +05:30
Bharat Viswanadham 2499435d9d HADOOP-16014. Fix test, checkstyle and javadoc issues in TestKerberosAuthenticationHandler. Contributed by Dinesh Chitlangia. 2018-12-21 13:30:48 -08:00
Jason Lowe 3961690037 HADOOP-15973. Configuration: Included properties are not cached if resource is a stream. Contributed by Eric Payne 2018-12-20 09:29:48 -06:00
Takanobu Asanuma d963575918 HDFS-13661. Ls command with e option fails when the filesystem is not HDFS. 2018-12-20 10:03:33 +09:00
Surendra Singh Lilhore 788e7473a4 HDFS-14096. [SPS] : Add Support for Storage Policy Satisfier in ViewFs. Contributed by Ayush Saxena. 2018-12-17 11:24:57 +05:30
Gabor Bota 1ea29b7385
HADOOP-16000. Remove TLSv1 and SSLv2Hello from the default value of hadoop.ssl.enabled.protocols
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-12-15 10:53:52 +09:00
Giovanni Matteo Fumarola 76efeacd5f HADOOP-15995. Add ldap.bind.password.alias in LdapGroupsMapping to distinguish aliases when using multiple providers through CompositeGroupsMapping. Contributed by Lukas Majercak. 2018-12-12 11:10:24 -08:00
Steve Loughran 202926ac33
HADOOP-15808. Harden Token service loader use.
Contributed by Steve Loughran.
2018-12-11 17:33:49 +00:00
Giovanni Matteo Fumarola aa89492f29 Revert "HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr."
This reverts commit fb9deed41d.
2018-12-04 12:57:28 -08:00
Steve Loughran db2d8b01c6
HADOOP-15966. Hadoop Kerberos broken on macos as java.security.krb5.realm is reset.
Contributed by Steve Loughran
2018-12-04 15:35:43 +00:00
Giovanni Matteo Fumarola fb9deed41d HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr. 2018-12-03 13:25:02 -08:00
Giovanni Matteo Fumarola c9a3aa64dc HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak. 2018-12-03 12:10:05 -08:00
Eric Yang 38ea3814bd HADOOP-15922. Revert patch 004. 2018-11-30 12:49:46 -05:00
Akira Ajisaka c9bfca217f
MAPREDUCE-7165. mapred-site.xml is misformatted in single node setup document. Contributed by Zhaohui Xin. 2018-11-30 13:13:23 +09:00
Steve Loughran d0edd37269
HADOOP-15959. Revert "HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple"
This reverts commit 829a2e4d27.
2018-11-29 17:52:11 +00:00
Ewan Higgs c1d24f8483
HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests.
Contributed by Ewan Higgs and Steve Loughran.
2018-11-29 15:12:17 +00:00
Eric Yang a9d96948ed HADOOP-15922. Fixed doAsUser decoding for DelegationTokenAuthenticationFilter.
Contributed by He Xiaoqiao
2018-11-26 13:49:19 -05:00
Vinayakumar B 23b441c225 HADOOP-15948. Inconsistency in get and put syntax if filename/dirname contains space. Contributed by Ayush Saxena. 2018-11-26 18:11:01 +05:30
Márton Elek 6934a65402 HADOOP-15358. SFTPConnectionPool connections leakage. Contributed by Mikhail Pryakhin. 2018-11-23 09:48:26 +01:00
Akira Ajisaka 1a00b4e325
Add 2.9.2 release notes and changes documents. 2018-11-20 14:25:18 +09:00
Inigo Goiri 460a94a10f HDFS-14045. Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN. Contributed by Jiandan Yang. 2018-11-15 10:58:57 -08:00
Surendra Singh Lilhore 75291e6d53 HADOOP-15869. BlockDecompressorStream#decompress should not return -1 in case of IOException. Contributed by Surendra Singh Lilhore 2018-11-13 20:22:58 +05:30
Steve Loughran 3c9d97b8f7
HADOOP-15110. Gauges are getting logged in exceptions from AutoRenewalThreadForUserCreds.
Contributed by LiXin Ge.
2018-11-12 12:58:05 +00:00
Giovanni Matteo Fumarola 89b49167a5 HADOOP-15903. Allow HttpServer2 to discover resources in /static when symlinks are used. Contributed by Inigo Goiri. 2018-11-08 14:52:24 -08:00
Jason Lowe 0b6625a973 MAPREDUCE-7148. Fast fail jobs when exceeds dfs quota limitation. Contributed by Wang Yan 2018-11-07 08:20:49 -06:00
Jason Lowe ab6aa4c726 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 2018-11-06 08:40:59 -06:00
Lars Francke cb8d679c95
HADOOP-15687. Credentials class should allow access to aliases.
Author:    Lars Francke <lars.francke@gmail.com>
2018-11-03 16:21:29 +00:00
Giovanni Matteo Fumarola 44e37b4fd9 HADOOP-15885. Add base64 (urlString) support to DTUtil. Contributed by Inigo Goiri. 2018-11-02 10:54:12 -07:00
Akira Ajisaka f747f5b06c
HADOOP-15886. Fix findbugs warnings in RegistryDNS.java. 2018-10-31 10:01:31 +09:00
Steve Loughran 62d98ca92a
HADOOP-15855. Review hadoop credential doc, including object store details.
Contributed by Steve Loughran.
2018-10-30 15:58:04 +00:00
Wei-Chiu Chuang 63e7134df6 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao."
This reverts commit fb2b72e6fc.
2018-10-28 17:04:59 -07:00
Hrishikesh Gadre bfb9adc2b9 HADOOP-9567. Provide auto-renewal for keytab based logins. Contributed by Hrishikesh Gadre, Gary Helmling and Harsh J.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-27 08:59:47 -07:00
Wei-Chiu Chuang fb2b72e6fc HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 2018-10-25 09:33:31 -07:00
Akira Ajisaka 8f97d6f2cd
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 2018-10-24 13:49:28 +09:00
Haibo Chen 823bb5dda8 MAPREDUCE-4669. MRAM web UI does not work with HTTPS. (Contributed by Robert Kanter) 2018-10-23 15:28:13 -07:00
Chen Liang 635786a511 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2018-10-23 14:53:45 -07:00
Inigo Goiri dd268a64d3 Revert "HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR."
This reverts commit 00254d7b8c.
2018-10-23 09:23:03 -07:00
Steve Loughran 52cb766ad0 HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility.
Contributed by Wei-Chiu Chuang.
2018-10-22 11:21:12 +01:00
Eric Yang a043dfa216 HADOOP-15821. Addendum patch to fix Hadoop Registry parent project.
Contributed by Íñigo Goiri
2018-10-21 12:50:10 -04:00
Eric Yang e2a9fa8448 HADOOP-15821. Move YARN Registry to Hadoop Registry.
Contributed by Íñigo Goiri
2018-10-19 19:46:48 -04:00
Inigo Goiri 00254d7b8c HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR. 2018-10-19 13:57:43 -07:00
Wei-Chiu Chuang cd2158456d HADOOP-15418. Hadoop KMSAuthenticationFilter needs to use getPropsByPrefix instead of iterator to avoid ConcurrentModificationException. Contributed by lqjack and Suma Shivaprasad 2018-10-18 17:28:28 -07:00
Jitendra Pandey d93d515af5 HDDS-651. Rename o3 to o3fs for Filesystem. 2018-10-17 14:19:17 -07:00
Jason Lowe 9abda83947 HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe 2018-10-17 14:38:42 -05:00
Adam Antal 24dc068a36 HADOOP-11100. Support to configure ftpClient.setControlKeepAliveTimeout.
Contributed by Adam Antal.

Signed-off-by: Xiao Chen <xiao@apache.org>
2018-10-17 11:34:50 -07:00
Steve Loughran 41b3603b5b
HADOOP-15861. Move DelegationTokenIssuer to the right path.
Contributed by Wei-Chiu Chuang.
2018-10-17 11:01:53 +01:00
Steve Loughran b738cb148c
HADOOP-15854. AuthToken Use StringBuilder instead of StringBuffer.
Contributed by Beluga Behr.
2018-10-17 10:29:09 +01:00
Haibo Chen c2288ac45b YARN-8448. AM HTTPS Support for AM communication with RMWeb proxy. (Contributed by Robert Kanter) 2018-10-16 13:36:26 -07:00
Robert Kanter 7fe1a40a6b HADOOP-15853. TestConfigurationDeprecation leaves behind a temp file, resulting in a license issue (ayushtkn via rkanter) 2018-10-15 15:02:37 -07:00
Xiao Chen b6fc72a025 HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 2018-10-15 10:51:55 -07:00
Anu Engineer 3c1fe073d2 HDDS-445. Create a logger to print out all of the incoming requests.
Contributed by Bharat Viswanadham.
2018-10-12 16:27:54 -07:00
Xiao Chen 5ec86b445c HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances.
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.
2018-10-12 09:35:52 -07:00
Robert Kanter d91d47bc73 HADOOP-15717. TGT renewal thread does not log IOException (snemeth via rkanter) 2018-10-11 15:35:44 -07:00
Xiao Chen 64f2b32d57 HADOOP-15676. Cleanup TestSSLHttpServer. Contributed by Szilard Nemeth. 2018-10-11 15:12:36 -07:00
Takanobu Asanuma 7b57f2f71f HADOOP-15785. [JDK10] Javadoc build fails on JDK 10 in hadoop-common. Contributed by Dinesh Chitlangia. 2018-10-11 13:51:51 +09:00
Robert Kanter f261c31937 HADOOP-15708. Reading values from Configuration before adding deprecations make it impossible to read value with deprecated key (zsiegl via rkanter) 2018-10-10 18:51:37 -07:00
Steve Loughran cdc4350718
HADOOP-15839. Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys.
Contributed by Steve Loughran.
2018-10-10 17:00:37 +01:00
Akira Ajisaka 6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Eric Yang bf04f19456 YARN-8845. Removed unused hadoop.registry.rm reference.
Contributed by Íñigo Goiri
2018-10-09 15:54:45 -04:00
Takanobu Asanuma 9bbeb52486 HADOOP-15775. [JDK9] Add missing javax.activation-api dependency. Contributed by Akira Ajisaka. 2018-10-09 13:06:14 +09:00
Vidura Mudalige a30b1d1824
HADOOP-15818. Fix deprecated maven-surefire-plugin configuration in hadoop-kms module
This closes #425

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-10-09 12:55:34 +09:00
Jason Lowe f13e231025 HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe 2018-10-05 09:06:02 -05:00
Yiqun Lin 81f635f47f HADOOP-15817. Reuse Object Mapper in KMSJSONReader. Contributed by Jonathan Eagles. 2018-10-04 10:30:30 +08:00
Aaron Fabbri 046b8768af
HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota 2018-10-02 21:22:49 -07:00
Akira Ajisaka 31fa9bb799
HADOOP-15767. [JDK10] Building native package on JDK10 fails due to missing javah. Contributed by Takanobu Asanuma. 2018-10-03 11:29:18 +09:00
Sunil G 58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Ewan Higgs 6fab6886f6 HDFS-13936. Multipart upload to HDFS to support 0 byte upload. Contributed by Ewan Higgs. 2018-10-02 14:05:35 +02:00
Akira Ajisaka 5c8d90763c
HADOOP-15764. Addendum patch: Fix NPE in SecurityUtil. 2018-09-28 10:03:22 +09:00
Akira Ajisaka c62b440421
HADOOP-15693. Typo in Secure Mode documentation. Contributed by Thomas Levine. 2018-09-26 10:30:43 +09:00
Akira Ajisaka 93b0f540ed
HADOOP-15783. [JDK10] TestSFTPFileSystem.testGetModifyTime fails. 2018-09-25 16:03:28 +09:00
Sunil G d060cbea48 HDFS-13937. Multipart Uploader APIs to be marked as private/unstable in 3.2.0. Contributed by Steve Loughran. 2018-09-24 21:19:47 +05:30
Sean Mackrory 0def61482b Merge branch 'HADOOP-15407' into trunk 2018-09-22 21:19:12 -06:00
Ewan Higgs 429a07e08c HADOOP-15764. [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement. Contributed by Akira Ajisaka. 2018-09-20 15:13:55 +02:00
Ewan Higgs 3da94a36e2 HADOOP-15756. [JDK10] Migrate from sun.net.util.IPAddressUtil to the replacement. Contributed by Akira Ajisaka. 2018-09-20 14:53:21 +02:00
Steve Loughran 646874c326
HADOOP-15748. S3 listing inconsistency can raise NPE in globber.
Contributed by Steve Loughran.
2018-09-20 13:04:52 +01:00
Vinayakumar B 7ad27e97f0 HADOOP-15736. Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. Contributed by Ayush Saxena. 2018-09-20 09:31:35 +05:30
Chen Liang a30b4f9e71 HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-19 13:22:37 -07:00
Jason Lowe e71f61ecb8 HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia 2018-09-18 15:57:33 -05:00
Steve Loughran 1cf38a38da
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch.
Contributed by Steve Loughran.
2018-09-18 10:56:56 +01:00
Takanobu Asanuma f796cfde76 HADOOP-15304. [JDK10] Migrate from com.sun.tools.doclets to the replacement. Contributed by Akira Ajisaka. 2018-09-18 18:48:25 +09:00
Yiqun Lin ee051ef9fe HADOOP-15742. Log if ipc backoff is enabled in CallQueueManager. Contributed by Ryan Wu. 2018-09-18 11:10:33 +08:00
Sean Mackrory 9475fd902a HADOOP-15745. ABFS: Add ABFS configuration to ConfigRedactor. 2018-09-17 19:54:01 +00:00
Thomas Marquardt 81dc4a995c HADOOP-15663. ABFS: Simplify configuration.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt ce03a93f78 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests.
Contributed by Steve Loughran and Da Zhou.
2018-09-17 19:54:01 +00:00
Steve Loughran f044deedbb HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit.
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
2018-09-17 19:54:01 +00:00
Brahma Reddy Battula ef5c776a42 HADOOP-15733. Correct the log when Invalid emptier Interval configured. Contributed by Ayush Saxena 2018-09-14 07:32:27 +05:30
Hanisha Koneru 291dcf2247 HDDS-423. Introduce an ozone specific log4j.properties. Contributed by Elek, Marton. 2018-09-13 16:37:44 -07:00
Takanobu Asanuma 5d084d7eca HADOOP-12760. sun.misc.Cleaner has moved to a new location in OpenJDK 9. Contributed by Akira Ajisaka. 2018-09-13 20:42:09 +09:00