Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
23,182 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

247 Commits

Author SHA1 Message Date
Da Zhou 9a8edb0aed
HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI 2019-10-16 13:14:15 -07:00
bilaharith 1a77a15fe4
HADOOP-16587. Make ABFS AAD endpoints configurable. 
Contributed by Bilahari T H.

This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate
in China.

Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c
 2019-10-07 13:07:46 +01:00
Sneha Vijayarajan 770adc5d4a
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists 2019-10-01 17:38:11 -07:00
Sneha Vijayarajan c0edc848a8
HADOOP-16548 : Disable Flush() over config 2019-09-28 20:39:42 -07:00
Steve Loughran e346e3638c HADOOP-15691 Add PathCapabilities to FileSystem and FileContext. 
Contributed by Steve Loughran.

This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.

This is intended to allow applications to determine

* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.

As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
 fs. + schema + .

The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.

Note

* The remote store is not expected to be checked for the feature;
  It is more a check of client API and the client's configuration/knowledge
  of the state of the remote system.
* Permissions are not checked.

Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5
 2019-09-25 12:16:41 +01:00
Gabor Bota aa664d7259
HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container raises NPE (#1302). Contributed by Gabor Bota. 
Change-Id: I2f637865c871e400b95fe7ddaa24bf99fa192023
 2019-09-23 13:29:01 +02:00
Sahil Takiar 55ce454ce4
HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8. 
Contributed by Sahil Takiar.

This moves the SSLSocketFactoryEx class from hadoop-azure into hadoop-common
as the DelegatingSSLSocketFactory and binds the S3A connector to it so that
it can avoid using those HTTPS algorithms which are underperformant on Java 8.

Change-Id: Ie9e6ac24deac1aa05e136e08899620efa7d22abd
 2019-09-17 11:32:03 +01:00
Da Zhou 43a91f820a
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus 
Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290
 2019-08-09 12:38:13 +01:00
bilaharith 5840df86d7
HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC. 
Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0
 2019-08-08 19:08:48 +01:00
Arun Singh 0b45293abb
HADOOP-16404. ABFS default blocksize change(256MB from 512MB) 
Contributed by: Arun Singh
 2019-07-19 20:21:28 -07:00
Robert Levas 450c070a8f
HADOOP-16340. ABFS driver continues to retry on IOException responses from REST operations. 
Contributed by Robert Levas.

This makes the HttpException constructor protected rather than public, so it is possible
to implement custom subclasses of this exception -exceptions which will not be retried.

Change-Id: Ie8aaa23a707233c2db35948784908b6778ff3a8f
 2019-06-19 17:43:14 +01:00
Da Zhou 1da09bd9d5
HADOOP-16376. ABFS: Override access() to no-op. 
Contributed by Da Zhou.

Change-Id: Ia0024bba32250189a87eb6247808b2473c331ed0
 2019-06-16 19:20:46 +01:00
Steve Loughran 309501c6fa
Revert "HADOOP-16050: s3a SSL connections should use OpenSSL" 
This reverts commit b067f8acaa.

Change-Id: I584b050a56c0e6f70b11fa3f7db00d5ac46e7dd8
 2019-06-05 13:54:55 +01:00
Akira Ajisaka afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Akira Ajisaka 9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Sahil Takiar b067f8acaa HADOOP-16050: s3a SSL connections should use OpenSSL 
(cherry picked from commit aebf229c17)
 2019-05-16 08:57:54 -06:00
DadanielZ ff27e8eabd HADOOP-16251. ABFS: add FSMainOperationsBaseTest. Re-commit to fix git metadata. 
Author: Da Zhou
 2019-05-10 13:57:30 -07:00
Aaron Fabbri 5f1a34649f Revert "HADOOP-16251. ABFS: add FSMainOperationsBaseTest" 
This reverts commit 7c2d7c05a9.

Git Commit name and email were incorrect. Will re-commit.
 2019-05-10 13:55:56 -07:00
DadanielZ 7c2d7c05a9 HADOOP-16251. ABFS: add FSMainOperationsBaseTest 
Author: Da Zhou
 2019-05-10 12:46:03 -07:00
Da Zhou 3418bbbb59
HADOOP-16269. ABFS: add listFileStatus with StartFrom. 
Author:    Da Zhou
 2019-05-08 17:20:46 +01:00
Da Zhou 1cef194a28
HADOOP-16242. ABFS: add bufferpool to AbfsOutputStream. 
Contributed by Da Zhou.
 2019-04-29 13:27:28 +01:00
Akira Ajisaka 4b4fef2f0e
HADOOP-16227. Upgrade checkstyle to 8.19 2019-04-15 10:46:10 +09:00
Akira Ajisaka 8b6deebb1d
HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji. 2019-04-03 19:35:02 +09:00
Vishwajeet Dusane 1edf1914ac
HADOOP-16182. Update abfs storage back-end with "close" flag when application is done writing to a file. 
Contributed by Vishwajeet Dusane.
 2019-03-18 13:18:08 +00:00
Denes Gerencser ddede7ae6f
HADOOP-16174. Disable wildfly logs to the console. 
Follow-on to HADOOP-15851.

Author:    Denes Gerencser <dgerencser@cloudera.com>
 2019-03-09 18:45:17 +00:00
Shweta Yakkali bb8ad096e7 HADOOP-16157. [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore 
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I6269ae66378e46eed440a76f847ae1af1fa95450
 2019-03-09 09:55:56 -08:00
Da Zhou e0260417ad
HADOOP-16169. ABFS: Bug fix for getPathProperties. 
Author:    Da Zhou <da.zhou@microsoft.com>
 2019-03-08 13:53:08 +00:00
Da Zhou 05df151d09
HADOOP-16163. NPE in setup/teardown of ITestAbfsDelegationTokens. 
Contributed by Da Zhou.

Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-03-05 14:02:34 +00:00
Steve Loughran 65f60e56b0
HADOOP-16068. ABFS Authentication and Delegation Token plugins to optionally be bound to specific URI of the store. 
Contributed by Steve Loughran.
 2019-02-28 14:22:32 +00:00
Da Zhou 3988e75ca3
HADOOP-16136. ABFS: Should only transform username to short name 
Contributed by Da Zhou.
 2019-02-26 15:37:24 +00:00
Steve Loughran a868f59d52
HADOOP-16105. WASB in secure mode does not set connectingUsingSAS. 
Contributed by Steve Loughran.
 2019-02-21 14:08:46 +00:00
Masatake Iwasaki aa3ad36605 HADOOP-16104. Wasb tests to downgrade to skip when test a/c is namespace enabled. Contributed by Masatake Iwasaki. 2019-02-20 22:00:57 +09:00
Da Zhou 1f1655028e
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. 
Contributed by Da Zhou and Junhua Gu.
 2019-02-07 21:58:21 +00:00
Steve Loughran 668817a6ce
Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth." 
(accidentally mixed in two patches)

This reverts commit fa8cd1bf28.
 2019-02-07 21:57:22 +00:00
Da Zhou fa8cd1bf28
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. 
Contributed by Da Zhou and Junhua Gu.
 2019-02-05 19:23:15 +00:00
Da Zhou ba9efe06fa
HADOOP-16074. WASB: Update container not found error code. 
Contributed by Da Zhou.
 2019-02-05 14:39:45 +00:00
Akira Ajisaka 1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Shweta Yakkali 02eb91856e HADOOP-16041. Include Hadoop version in User-Agent string for ABFS. Contributed by Shweta Yakkali. 
Signed-off-by: Sean Mackrory <mackrorysd@apache.org>
 2019-01-29 16:50:14 -08:00
Da Zhou 00ad9e23e8
HADOOP-16048. ABFS: Fix Date format parser. 
Contributed by Da Zhou.
 2019-01-22 16:40:42 +00:00
Da Zhou 30863c5ae3
HADOOP-16044. ABFS: Better exception handling of DNS errors followup 
Contributed by Da Zhou.
 2019-01-14 19:32:56 +00:00
Da Zhou dddad985d7
HADOOP-15975. ABFS: remove timeout check for DELETE and RENAME. 
Contributed by Da Zhou.

(cherry picked from commit 8b5fbe7a12)
 2019-01-11 11:13:41 +00:00
Da Zhou 852701f793
HADOOP-16036. WASB: Disable jetty logging configuration announcement. 
Contributed by Da Zhou.
 2019-01-10 12:07:09 +00:00
Da Zhou 7211269142
HADOOP-15662. Better exception handling of DNS errors. 
Contributed by Da Zhou.
 2019-01-10 12:02:58 +00:00
Da Zhou e8d1900369
HADOOP-16040. ABFS: Bug fix for tolerateOobAppends configuration. 
Contributed by Da Zhou.
 2019-01-10 11:58:39 +00:00
Sean Mackrory 13f0ee21f2 HADOOP-15860. ABFS: Throw exception when directory / file name ends with a period (.). Contributed by Shweta Yakkali. 2019-01-02 09:33:09 -07:00
Da Zhou 346c0c8aff
HADOOP-16004. ABFS: Convert 404 error response in AbfsInputStream and AbfsOutPutStream to FileNotFoundException. 
Contributed by Da Zhou.
 2018-12-17 11:15:20 +00:00
Da Zhou 62df60737c
HADOOP-15972 ABFS: reduce list page size to to 500. 
Contributed by Da Zhou.

(cherry picked from commit d09dbcc8fb)
 2018-12-17 11:10:22 +00:00
Da Zhou b2523d8100
HADOOP-15969. ABFS: getNamespaceEnabled can fail blocking user access thru ACLs. 
Contributed by Da Zhou.
 2018-12-17 11:04:40 +00:00
Da Zhou a8bbd818d5
HADOOP-15968. ABFS: add try catch for UGI failure when initializing ABFS. 
Contributed by Da Zhou.
 2018-12-04 13:39:10 +00:00
Da Zhou 7ccb640a66
HADOOP-15957. WASB: Add asterisk wildcard support for PageBlobDirSet. 
Contributed by Da Zhou.
 2018-11-30 10:12:41 +00:00