Commit Graph

183 Commits

Author SHA1 Message Date
Steve Loughran a4abf02028
HADOOP-15739. ABFS: remove unused maven dependencies and add used undeclared dependencies.
Contributed by Da Zhou.
2018-09-25 20:58:32 +01:00
Steve Loughran d5da9928c9 HADOOP-15723. ABFS: Ranger Support.
Contributed by Yuan Gao.
2018-09-25 19:13:10 +01:00
Steve Loughran d0b4624c88
HADOOP-15778. ABFS: Fix client side throttling for read.
Contributed by Sneha Varma.
2018-09-21 11:06:24 +01:00
Steve Loughran a5692c2da5 HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable.
Contributed by Steve Loughran.
2018-09-20 17:36:18 +01:00
Sean Mackrory 8e831ba458 HADOOP-15773. Fixing checkstyle and other issues raised by Yetus. 2018-09-19 16:56:33 -06:00
Steve Loughran a55d26b23e
HADOOP-15769. ABFS: distcp tests are always skipped.
Contributed by Steve Loughran
2018-09-19 13:57:39 +01:00
Steve Loughran df2166a643
HADOOP-15719. Fail-fast when using OAuth over http.
Contributed by Da Zhou.
2018-09-18 12:20:52 +01:00
Steve Loughran 51d368982b
HADOOP-15714. Tune abfs/wasb parallel and sequential test execution.
Contributed by Da Zhou.
2018-09-18 12:09:25 +01:00
Steve Loughran 524776625d
HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min.
Contributed by Da Zhou
2018-09-18 11:48:46 +01:00
Steve Loughran 1cf38a38da
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch.
Contributed by Steve Loughran.
2018-09-18 10:56:56 +01:00
Thomas Marquardt b4c23043d3 HADOOP-15757. ABFS: remove dependency on common-codec Base64.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 26211019c8 HADOOP-15753. ABFS: support path "abfs://mycluster/file/path"
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt e5593cbd83 HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts.
Contributed by Sean Mackrory.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 13c70e9ba3 HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept.
Contributed by Sneha Varma.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 6801b30733 HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation
for storage account with no namespace feature.

Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 347a52a866 Fixing findbugs and license issues related to:
HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 97f06b3fc7 HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 4410eacba7 HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 81dc4a995c HADOOP-15663. ABFS: Simplify configuration.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt df57c6c3b1 HADOOP-15692. ABFS: extensible support for custom oauth.
Contributed by Junhua Gu and Rajeev Bansal.
2018-09-17 19:54:01 +00:00
Thomas Marquardt dd2b22fa31 HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle.
Contributed by Thomas Marquardt.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 6b6f8cc2be HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice.
Contributed by Sean Mackrory.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 9c1e4e8139 HADOOP-15661. ABFS: Add support for ACL.
Contributed by Junhua Gu and Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt 9149b9703e HADOOP-15660. ABFS: Add support for OAuth
Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.
2018-09-17 19:54:01 +00:00
Thomas Marquardt d6a4f39bd5 HADOOP-15669. ABFS: Improve HTTPS Performance.
Contributed by Vishwajeet Dusane.
2018-09-17 19:54:01 +00:00
Thomas Marquardt cc5cc60c41 Fixing issue due to commit 2b2399d6 after rebase onto trunk. 2018-09-17 19:54:01 +00:00
Thomas Marquardt b54b0c1b67 HADOOP-15659. Code changes for bug fix and new tests.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt ce03a93f78 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests.
Contributed by Steve Loughran and Da Zhou.
2018-09-17 19:54:01 +00:00
Steve Loughran a271fd0eca HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Steve Loughran f044deedbb HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit.
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
2018-09-17 19:54:01 +00:00
Steve Loughran 1eef0acfe5
HADOOP-15677. WASB: Add support for StreamCapabilities.
Contributed by Thomas Marquardt.
2018-09-10 14:24:41 +01:00
Steve Loughran e8d138ca7c
HADOOP-15680. ITestNativeAzureFileSystemConcurrencyLive times out.
Contributed by Andras Bokor.
2018-08-30 14:36:00 +01:00
Steve Loughran 45d9568aaa
HADOOP-15547/ WASB: improve listStatus performance.
Contributed by Thomas Marquardt.

(cherry picked from commit 749fff577e)
2018-07-19 12:31:19 -07:00
Akira Ajisaka 2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Todd Lipcon 7a3c6e9c3c HADOOP-15550. Avoid static initialization of ObjectMappers 2018-06-25 15:36:45 -07:00
Chris Douglas f34744603e HADOOP-15533. Make WASB listStatus messages consistent. Contributed by Esfandiar Manii 2018-06-17 23:12:18 -07:00
Steve Loughran d901be6795
HADOOP-15506. Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks.
Contributed by Esfandiar Manii.
2018-06-06 18:28:14 +01:00
Steve Loughran ba842847c9 HADOOP-15478. WASB: hflush() and hsync() regression.
Contributed by Thomas Marquardt.
2018-05-21 11:12:34 +01:00
Steve Loughran ba051b0686
HADOOP-15354. hadoop-aliyun & hadoop-azure modules to mark hadoop-common as provided
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-10 18:38:08 +09:00
Steve Loughran 5b11b9fd41 HADOOP-15446. WASB: PageBlobInputStream.skip breaks HBASE replication.
Contributed by Thomas Marquardt
2018-05-07 11:54:08 +01:00
Yiqun Lin 1312f9ae4c HADOOP-15391. Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules. 2018-04-18 16:04:00 +08:00
Chris Douglas 081c350188 HADOOP-15320. Remove customized getFileBlockLocations for hadoop-azure and hadoop-azure-datalake. Contributed by Shanyu Zhao 2018-03-28 11:58:59 -07:00
fang zhenyi 5e013d50d1
HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-15 16:12:07 +09:00
Wangda Tan 60f9e60b3b Preparing for 3.2.0 development
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
2018-02-11 11:17:38 +08:00
Steve Loughran 52babbb4a0 HADOOP-15086. NativeAzureFileSystem file rename is not atomic.
Contributed by Thomas Marquardt
2017-12-22 11:39:55 +00:00
Steve Loughran 0ea182d0fa HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0.
Contributed by Rajesh Balamohan
2017-11-28 11:52:59 +00:00
Akira Ajisaka 6903cf096e HADOOP-13514. Upgrade maven surefire plugin to 2.20.1
Signed-off-by: Allen Wittenauer <aw@apache.org>
2017-11-19 12:39:37 -08:00
John Zhuge bf6a660232 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-11-09 10:16:12 -08:00
Steve Loughran 9fcc3a1fc8 HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled.
Contributed by Santhosh G Nayak
2017-10-16 16:01:47 +01:00
Steve Loughran 3de574413c HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB.
Contributed by Varada Hemeswari.
2017-10-11 18:06:43 +01:00