Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,053 Commits 326 Branches 371 Tags
Commit Graph

239 Commits

Author SHA1 Message Date
bilaharith
ffeb6d8ece
HADOOP-16587. Make ABFS AAD endpoints configurable. 
Contributed by Bilahari T H.

This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate
in China.

Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c
 2019-12-08 21:31:39 -08:00
Sneha Vijayarajan
8b2c7e0c4d
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists 2019-12-08 21:31:24 -08:00
Sneha Vijayarajan
546db6428e
HADOOP-16548 : Disable Flush() over config 2019-12-08 21:31:08 -08:00
Andras Bokor
89e95370a4 HADOOP-16710. Testing_azure.md documentation is misleading. 
Contributed by Andras Bokor.

Change-Id: Icf07a53145936953629c7dace2e9648b7b21588d
 2019-11-17 17:06:10 +00:00
Da Zhou
fe96407451
HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI 
(cherry picked from commit 9a8edb0aeddd7787b2654f6e2a8465c325e048a2)
 2019-10-16 14:27:11 -07:00
Rohith Sharma K S
7d5bb2ebb7 Preparing for 3.2.2-SNAPSHOT development. 2019-09-07 08:52:08 +05:30
bilaharith
3b3c0c4b87 HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC. 
Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0
 2019-08-27 19:08:38 +00:00
Robert Levas
ce23e971b4 HADOOP-16340. ABFS driver continues to retry on IOException responses from REST operations. 
Contributed by Robert Levas.

This makes the HttpException constructor protected rather than public, so it is possible
to implement custom subclasses of this exception -exceptions which will not be retried.

Change-Id: Ie8aaa23a707233c2db35948784908b6778ff3a8f
 2019-08-27 19:08:29 +00:00
Da Zhou
a6d50a9054 HADOOP-16376. ABFS: Override access() to no-op. 
Contributed by Da Zhou.

Change-Id: Ia0024bba32250189a87eb6247808b2473c331ed0
 2019-08-27 19:04:16 +00:00
Da Zhou
dd636127e9 HADOOP-16269. ABFS: add listFileStatus with StartFrom. 
Author:    Da Zhou
 2019-08-27 19:01:21 +00:00
Da Zhou
006ae258b3 HADOOP-16163. NPE in setup/teardown of ITestAbfsDelegationTokens. 
Contributed by Da Zhou.

Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-08-27 19:01:21 +00:00
Da Zhou
330e450397
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus 
Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290
 2019-08-12 09:41:52 +08:00
Arun Singh
5f2d07af1b
HADOOP-16404. ABFS default blocksize change(256MB from 512MB) 
Contributed by: Arun Singh
 2019-07-19 20:34:28 -07:00
Masatake Iwasaki
b6718c754a HADOOP-16401. ABFS: port Azure doc to 3.2 branch. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-07-10 17:16:43 +09:00
DadanielZ
9c8e40fbdb
HADOOP-16251. ABFS: add FSMainOperationsBaseTest. Re-commit to fix git metadata. 
Author: Da Zhou
(cherry picked from commit ff27e8eabded6b1de9860da95155b304c07b4c6e)
 2019-06-07 18:09:38 +01:00
Da Zhou
bf0bb2470f
HADOOP-16242. ABFS: add bufferpool to AbfsOutputStream. 
Contributed by Da Zhou.

(cherry picked from commit 1cef194a28086991cd39fb62092d2b2105ece57b)
 2019-06-07 18:09:38 +01:00
Vishwajeet Dusane
907a016142
HADOOP-16182. Update abfs storage back-end with "close" flag when application is done writing to a file. 
Contributed by Vishwajeet Dusane.

(cherry picked from commit 1edf1914acb74e45f6717c703f519cb382aae173)
 2019-06-07 18:09:37 +01:00
Shweta Yakkali
6b115966bc
HADOOP-16157. [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore 
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I6269ae66378e46eed440a76f847ae1af1fa95450
(cherry picked from commit bb8ad096e785f7127a5c0de15167255d9b119578)
 2019-06-07 18:09:37 +01:00
Shweta Yakkali
57c6060c3a
HADOOP-15860. ABFS: Throw exception when directory / file name ends with a period (.). 
Contributed by Shweta Yakkali.

(cherry picked from commit 13f0ee21f2c17ebacaa35e14ee01f39624f38a8d)

Change-Id: Ibd010d2e6adc15f53a9c5357482e57313bf84d2e
 2019-06-07 18:09:37 +01:00
Da Zhou
3593b66693
HADOOP-15823. ABFS: Stop requiring client ID and tenant ID for MSI 
(Contributed by Da Zhou via Daniel Templeton)

Change-Id: I546ab3a1df1efec635c08c388148e718dc4a9843
(cherry picked from commit e374584479b687e41d5379bb6d827dcae620e123)
 2019-06-07 18:09:37 +01:00
Denes Gerencser
ede5cbd707
HADOOP-16174. Disable wildfly logs to the console. 
Follow-on to HADOOP-15851.

Author:    Denes Gerencser <dgerencser@cloudera.com>
(cherry picked from commit ddede7ae6fbbadbe08861bc85a664b73d66f77c7)
 2019-06-07 18:09:37 +01:00
Steve Loughran
96489069b0
HADOOP-15851. Disable wildfly logs to the console. 
Contributed by Vishwajeet Dusane.

(cherry picked from commit ef9dc6c44c686e836bb25e31ff355cff80572d23)
 2019-06-07 18:09:37 +01:00
Steve Loughran
baa8670256
HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth. 
Contributed by Da Zhou.

(cherry picked from commit bd50fa956b1ca25bb2136977b98a6aa6895eff8b)
 2019-06-07 18:09:37 +01:00
Takanobu Asanuma
a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka
855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Akira Ajisaka
df5d8f05d9
HADOOP-16227. Upgrade checkstyle to 8.19 
(cherry picked from commit 4b4fef2f0e0ed1e185ea1058db7a65d68d4970b9)
 2019-04-15 10:47:02 +09:00
Akira Ajisaka
80a8d3310e
HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji. 
(cherry picked from commit 8b6deebb1dda49e5e35180ed5c5fb5b5221c1516)
 2019-04-03 19:36:17 +09:00
Da Zhou
cfaf21a4ba
HADOOP-16169. ABFS: Bug fix for getPathProperties. 
Author:    Da Zhou <da.zhou@microsoft.com>
(cherry picked from commit e0260417ade5568ae37dcc3753aea0d1e0fd701b)
 2019-03-08 13:53:44 +00:00
Da Zhou
dc38fc598d
HADOOP-16136. ABFS: Should only transform username to short name 
Contributed by Da Zhou.

(cherry picked from commit 3988e75ca385aec31ca1fc49d6cffce1ea935825)
Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-03-05 10:47:58 +00:00
Da Zhou
075f6b061c
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. 
Contributed by Da Zhou and Junhua Gu.

(cherry picked from commit 1f1655028eede24197705a594b6ef19e6737db35)
Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-03-05 10:44:46 +00:00
Da Zhou
ae832ccffe
HADOOP-16041. Include Hadoop version in User-Agent string for ABFS. 
Contributed by Shweta Yakkali.

Signed-off-by: Sean Mackrory <mackrorysd@apache.org>
(cherry picked from commit 02eb91856e7e8477c62e0f8bf1bac6de3e00a8a4)
Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-03-05 10:39:37 +00:00
Steve Loughran
685a41f449
HADOOP-16105. WASB in secure mode does not set connectingUsingSAS. 
Contributed by Steve Loughran.

(cherry picked from commit 9cb2f470b759bbe7609a00e8f8f72779e2daae80)
 2019-02-21 13:39:37 +00:00
Masatake Iwasaki
dc9c3ce30b HADOOP-16104. Wasb tests to downgrade to skip when test a/c is namespace enabled. Contributed by Masatake Iwasaki. 
(cherry picked from commit aa3ad3660506382884324c4b8997973f5a68e29a)
 2019-02-20 22:17:18 +09:00
Da Zhou
84ce0f1bfa
HADOOP-16074. WASB: Update container not found error code. 
Contributed by Da Zhou.

(cherry picked from commit ba9efe06fadb9631763f3e623d8748bbe59ff748)
 2019-02-05 14:41:15 +00:00
Da Zhou
29de303e0a
HADOOP-16048. ABFS: Fix Date format parser. 
Contributed by Da Zhou.

(cherry picked from commit 00ad9e23e88d1e1b1f887b6c9473a7a924a95a97)
 2019-01-22 16:41:33 +00:00
Da Zhou
1d4390e16b
HADOOP-16044. ABFS: Better exception handling of DNS errors followup 
Contributed by Da Zhou.

(cherry picked from commit 30863c5ae3a7ad69b6b4853bad2e8f22c7c67639)
 2019-01-14 19:45:30 +00:00
Da Zhou
8b5fbe7a12
HADOOP-15975. ABFS: remove timeout check for DELETE and RENAME. 
Contributed by Da Zhou.
 2019-01-11 11:12:39 +00:00
Da Zhou
9cb6000c8a
HADOOP-16036. WASB: Disable jetty logging configuration announcement. 
Contributed by Da Zhou.

(cherry picked from commit 852701f7935eb94eda6f44b74a2ce060e7c159a7)
 2019-01-10 12:08:27 +00:00
Da Zhou
6c2500d7ca
HADOOP-15662. Better exception handling of DNS errors. 
Contributed by Da Zhou.

(cherry picked from commit 72112691421bedb66d887ba4e18d1b7d65a42f2c)
 2019-01-10 12:03:48 +00:00
Da Zhou
f7de630e85
HADOOP-16040. ABFS: Bug fix for tolerateOobAppends configuration. 
Contributed by Da Zhou.

(cherry picked from commit e8d19003695e3bc76bfa8e1187a238eec0220def)
 2019-01-10 11:59:29 +00:00
Da Zhou
f122ae7279
HADOOP-16004. ABFS: Convert 404 error response in AbfsInputStream and AbfsOutPutStream to FileNotFoundException. 
Contributed by Da Zhou.

(cherry picked from commit 346c0c8aff0b206d45f34dbce4fcc81364115d95)
 2018-12-17 11:18:12 +00:00
Da Zhou
d09dbcc8fb
HADOOP-15972 ABFS: reduce list page size to to 500. 
Contributed by Da Zhou.
 2018-12-17 11:08:17 +00:00
Da Zhou
87d9a54968
HADOOP-15969. ABFS: getNamespaceEnabled can fail blocking user access thru ACLs. 
Contributed by Da Zhou.

(cherry picked from commit b2523d8100844338e073531c47666d744a101caf)
 2018-12-17 11:05:39 +00:00
Da Zhou
2d2212a508
HADOOP-15968. ABFS: add try catch for UGI failure when initializing ABFS. 
Contributed by Da Zhou.

(cherry picked from commit a8bbd818d5bc4762324bcdb7cf1fdd5c2f93891b)
 2018-12-04 13:40:03 +00:00
Da Zhou
9bc1fd4721
HADOOP-15957. WASB: Add asterisk wildcard support for PageBlobDirSet. 
Contributed by Da Zhou.

(cherry picked from commit 7ccb640a66bb5bb9f657a2db85bcc8ae0ded4892)
 2018-11-30 10:13:57 +00:00
Da Zhou
1a3a4960d9
HADOOP-15940. ABFS: For HNS account, avoid unnecessary get call when doing Rename. 
Contributed by Da Zhou <da.zhou@microsoft.com>
 2018-11-27 18:11:30 +00:00
Da Zhou
f5d2806c81
HADOOP-15872. ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2. 
Contributed by Junhua Gu and Da Zhou.

(cherry picked from commit a8302e398c2ca4b0deff062c0e921053351b688e)
 2018-11-23 14:19:36 +00:00
Da Zhou
4039840510
HADOOP-15876. Use keySet().removeAll() to remove multiple keys from Map in AzureBlobFileSystemStore 
Contributed by Da Zhou.

(cherry picked from commit a13be203b7877ba56ef63aac4a2e65d4e1a4adbc)
 2018-11-13 21:48:05 +00:00
Da Zhou
7440bc5a9c
HADOOP-15812. ABFS: Improve AbfsRestOperationException format to ensure full msg can be displayed on console. 
Author:    Da Zhou <da.zhou@microsoft.com>
(cherry picked from commit 9dbb2b67c6fa69a663c7c23fbb876bac4b6c09ce)
 2018-11-09 11:07:51 +00:00
Junhua Gu
66715005f9
HADOOP-15846. ABFS: fix mask related bugs in setAcl, modifyAclEntries and removeAclEntries. 
Contributed by Junhua Gu.
 2018-11-08 17:20:52 +00:00