Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,821 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

179 Commits

Author SHA1 Message Date
Yuan Gao 18f66a05cf
HADOOP-16205 Backport ABFS driver from trunk to branch 2.0: Fix build and test failures. 
Contributed by Yuan Gao.
 2019-04-25 18:36:14 +01:00
Vishwajeet Dusane 662a75d65b HADOOP-16182. Update abfs storage back-end with "close" flag when application is done writing to a file. 
Contributed by Vishwajeet Dusane.
 2019-04-24 21:46:25 +00:00
Da Zhou 024a694684 HADOOP-16169. ABFS: Bug fix for getPathProperties. 
Author:    Da Zhou <da.zhou@microsoft.com>
 2019-04-24 21:46:25 +00:00
Da Zhou c2d7429691 HADOOP-16136. ABFS: Should only transform username to short name 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Da Zhou 80c8b77cec HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. 
Contributed by Da Zhou and Junhua Gu.
 2019-04-24 21:46:25 +00:00
Shweta Yakkali 0bbbb9c4b9 HADOOP-16041. Include Hadoop version in User-Agent string for ABFS. Contributed by Shweta Yakkali. 
Signed-off-by: Sean Mackrory <mackrorysd@apache.org>
 2019-04-24 21:46:25 +00:00
Da Zhou f799c4644d HADOOP-16048. ABFS: Fix Date format parser. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Da Zhou 00b990d0b6 HADOOP-16044. ABFS: Better exception handling of DNS errors followup 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Da Zhou 34dbac8347 HADOOP-15975. ABFS: remove timeout check for DELETE and RENAME. 
Contributed by Da Zhou.

(cherry picked from commit 8b5fbe7a12)
 2019-04-24 21:46:25 +00:00
Da Zhou a0b99f42e5 HADOOP-15662. Better exception handling of DNS errors. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Da Zhou 4a1761067d HADOOP-16040. ABFS: Bug fix for tolerateOobAppends configuration. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Sean Mackrory 175a69e863 HADOOP-15860. ABFS: Throw exception when directory / file name ends with a period (.). Contributed by Shweta Yakkali. 2019-04-24 21:46:25 +00:00
Da Zhou eaaf2839a5 HADOOP-16004. ABFS: Convert 404 error response in AbfsInputStream and AbfsOutPutStream to FileNotFoundException. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Da Zhou 38a36d4dec HADOOP-15972 ABFS: reduce list page size to to 500. 
Contributed by Da Zhou.

(cherry picked from commit d09dbcc8fb)
 2019-04-24 21:46:25 +00:00
Da Zhou 06e6a677f3 HADOOP-15969. ABFS: getNamespaceEnabled can fail blocking user access thru ACLs. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Da Zhou 6f200a61b9 HADOOP-15968. ABFS: add try catch for UGI failure when initializing ABFS. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Da Zhou 9cf1fb6e95 HADOOP-15940. ABFS: For HNS account, avoid unnecessary get call when doing Rename. 
Contributed by Da Zhou <da.zhou@microsoft.com>

(cherry picked from commit 1a3a4960d9)
 2019-04-24 21:46:25 +00:00
Da Zhou afa5705ff7 HADOOP-15872. ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2. 
Contributed by Junhua Gu and Da Zhou.
 2019-04-24 21:46:25 +00:00
Da Zhou 9e1d39bbae HADOOP-15876. Use keySet().removeAll() to remove multiple keys from Map in AzureBlobFileSystemStore 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Da Zhou 9f220c41c4 HADOOP-15812. ABFS: Improve AbfsRestOperationException format to ensure full msg can be displayed on console. 
Author:    Da Zhou <da.zhou@microsoft.com>
 2019-04-24 21:46:25 +00:00
Junhua Gu d37098196d HADOOP-15846. ABFS: fix mask related bugs in setAcl, modifyAclEntries and removeAclEntries. 
Contributed by Junhua Gu.

(cherry picked from commit 66715005f9)
 2019-04-24 21:46:25 +00:00
Daniel Templeton 8330f182dd HADOOP-15823. ABFS: Stop requiring client ID and tenant ID for MSI 
(Contributed by Da Zhou via Daniel Templeton)

Change-Id: I546ab3a1df1efec635c08c388148e718dc4a9843
 2019-04-24 21:46:25 +00:00
Steve Loughran 885c726d38 HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran 73bbada59a HADOOP-15809. ABFS: better exception handling when making getAccessToken call. 
Contributed by Da Zhou
 2019-04-24 21:46:25 +00:00
Steve Loughran 1b08e8d537 HADOOP-15795. Make HTTPS the default protocol for ABFS. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran e3033a5ba8 HADOOP-15792. typo in AzureBlobFileSystem.getIsNamespaceEnabeld. 
Contributed by Abhishek Modi.
 2019-04-24 21:46:25 +00:00
Steve Loughran d5e4c40760 HADOOP-15801. ABFS: Fixing skipUserGroupMetadata in AzureBlobFileSystemStore. 
Contributed by Da Zhou
 2019-04-24 21:46:25 +00:00
Steve Loughran 0c5074a228 HADOOP-15793. ABFS: Skip unsupported test cases when non namespace enabled in ITestAzureBlobFileSystemAuthorization 
Contributed by Yuan Gao.
 2019-04-24 21:46:25 +00:00
Steve Loughran bde00d56df HADOOP-15723. ABFS: Ranger Support. 
Contributed by Yuan Gao.
 2019-04-24 21:46:25 +00:00
Steve Loughran 4f3cc1d417 HADOOP-15778. ABFS: Fix client side throttling for read. 
Contributed by Sneha Varma.
 2019-04-24 21:46:25 +00:00
Steve Loughran 6c7c8b83f7 HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable. 
Contributed by Steve Loughran.
 2019-04-24 21:46:25 +00:00
Sean Mackrory 920b77c8d7 HADOOP-15773. Fixing checkstyle and other issues raised by Yetus. 2019-04-24 21:46:25 +00:00
Steve Loughran 10e149d882 HADOOP-15769. ABFS: distcp tests are always skipped. 
Contributed by Steve Loughran
 2019-04-24 21:46:25 +00:00
Steve Loughran 1efa1b73a9 HADOOP-15719. Fail-fast when using OAuth over http. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran 26ce4e9301 HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min. 
Contributed by Da Zhou
 2019-04-24 21:46:25 +00:00
Steve Loughran 94d0de2eab HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. 
Contributed by Steve Loughran.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 3599cd3dc9 HADOOP-15757. ABFS: remove dependency on common-codec Base64. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 263094f962 HADOOP-15753. ABFS: support path "abfs://mycluster/file/path" 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt e52f4a1ad2 HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts. 
Contributed by Sean Mackrory.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 36b2bab888 HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept. 
Contributed by Sneha Varma.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt a192d85a5f HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation 
for storage account with no namespace feature.

Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 8bb713e31a Fixing findbugs and license issues related to: 
HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 305e750e82 HADOOP-15703. ABFS - Implement client-side throttling. 
Contributed by Sneha Varma and Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 12c621eda4 HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 3bd089ddca HADOOP-15663. ABFS: Simplify configuration. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 92f3f9cf09 HADOOP-15692. ABFS: extensible support for custom oauth. 
Contributed by Junhua Gu and Rajeev Bansal.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt b59741f66b HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle. 
Contributed by Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 08d3d50275 HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice. 
Contributed by Sean Mackrory.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt e727c87500 HADOOP-15661. ABFS: Add support for ACL. 
Contributed by Junhua Gu and Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 9d47230c8f HADOOP-15660. ABFS: Add support for OAuth 
Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.
 2019-04-24 21:46:25 +00:00