Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,259 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1035 Commits

Author SHA1 Message Date
Sunil G 83ef50fded
Revert "HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu." 
This reverts commit e4fca6aae4.

Revert "HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu."

This reverts commit 2b635125fb.

(cherry picked from commit 1f425271a7)
(cherry picked from commit 71bee05339)
 2019-01-05 17:49:18 +09:00
Ted Yu a73860c89d HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e)
(cherry picked from commit a7dd244a49)
 2018-10-27 12:18:32 -07:00
Sunil G ece45559ca Preparing for 3.2.0 release 2018-10-19 14:32:23 +05:30
Giovanni Matteo Fumarola 38895fcef1 HDFS-14007. Incompatible layout when generating FSImage. Contributed by Inigo Goiri. 2018-10-18 17:32:48 -07:00
Steve Loughran c19864d8c8
HADOOP-15848. ITestS3AContractMultipartUploader#testMultipartUploadEmptyPart test error. 
Contributed by Ewan Higgs.
 2018-10-16 19:58:58 +01:00
bibinchundatt aff21d7271 YARN-8830. SLS tool fix node addition. Contributed by Bibin A Chundatt. 
(cherry picked from commit b4a38e7b3e)
 2018-10-15 18:11:01 +05:30
Steve Loughran 70d39c74d9
HADOOP-15837. DynamoDB table Update can fail S3A FS init. 
Contributed by Steve Loughran.

(cherry picked from commit ee816f1fd7)
 2018-10-11 14:58:37 +01:00
Steve Loughran 83b9b25c51
HADOOP-15809. ABFS: better exception handling when making getAccessToken call. 
Contributed by Da Zhou

(cherry picked from commit 273cc2d4e9)
 2018-10-05 11:29:43 +01:00
Steve Loughran c6942a315b
HADOOP-15792. typo in AzureBlobFileSystem.getIsNamespaceEnabeld. 
Contributed by Abhishek Modi.

(cherry picked from commit e8b8604314)
 2018-10-03 12:59:16 +01:00
Steve Loughran e5e9d7b595
HADOOP-15795. Make HTTPS the default protocol for ABFS. 
Contributed by Da Zhou.

(cherry picked from commit 7051bd78b1)
 2018-10-03 12:53:56 +01:00
Steve Loughran a383ac47ca
HADOOP-15801. ABFS: Fixing skipUserGroupMetadata in AzureBlobFileSystemStore. 
Contributed by Da Zhou
 2018-10-02 11:42:52 +01:00
Steve Loughran 43bc984891
HADOOP-15793. ABFS: Skip unsupported test cases when non namespace enabled in ITestAzureBlobFileSystemAuthorization 
Contributed by Yuan Gao.
 2018-10-02 11:37:28 +01:00
Steve Loughran a4abf02028
HADOOP-15739. ABFS: remove unused maven dependencies and add used undeclared dependencies. 
Contributed by Da Zhou.
 2018-09-25 20:58:32 +01:00
Steve Loughran d5da9928c9 HADOOP-15723. ABFS: Ranger Support. 
Contributed by Yuan Gao.
 2018-09-25 19:13:10 +01:00
Sammi Chen 2b635125fb HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu. 2018-09-25 19:48:30 +08:00
Mingliang Liu c07715e378 HADOOP-15781 S3A assumed role tests failing due to changed error text in AWS exceptions. Contributed by Steve Loughran 2018-09-24 12:53:21 -07:00
Sunil G d060cbea48 HDFS-13937. Multipart Uploader APIs to be marked as private/unstable in 3.2.0. Contributed by Steve Loughran. 2018-09-24 21:19:47 +05:30
Sean Mackrory 0def61482b Merge branch 'HADOOP-15407' into trunk 2018-09-22 21:19:12 -06:00
Steve Loughran d0b4624c88
HADOOP-15778. ABFS: Fix client side throttling for read. 
Contributed by Sneha Varma.
 2018-09-21 11:06:24 +01:00
Steve Loughran a5692c2da5 HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable. 
Contributed by Steve Loughran.
 2018-09-20 17:36:18 +01:00
Sean Mackrory 8e831ba458 HADOOP-15773. Fixing checkstyle and other issues raised by Yetus. 2018-09-19 16:56:33 -06:00
Steve Loughran a55d26b23e
HADOOP-15769. ABFS: distcp tests are always skipped. 
Contributed by Steve Loughran
 2018-09-19 13:57:39 +01:00
Steve Loughran df2166a643
HADOOP-15719. Fail-fast when using OAuth over http. 
Contributed by Da Zhou.
 2018-09-18 12:20:52 +01:00
Steve Loughran 51d368982b
HADOOP-15714. Tune abfs/wasb parallel and sequential test execution. 
Contributed by Da Zhou.
 2018-09-18 12:09:25 +01:00
Steve Loughran 524776625d
HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min. 
Contributed by Da Zhou
 2018-09-18 11:48:46 +01:00
Steve Loughran 1cf38a38da
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. 
Contributed by Steve Loughran.
 2018-09-18 10:56:56 +01:00
Steve Loughran 26d0c63a1e
HADOOP-15754. s3guard: testDynamoTableTagging should clear existing config. 
Contributed by Gabor Bota.
 2018-09-17 22:40:08 +01:00
Thomas Marquardt b4c23043d3 HADOOP-15757. ABFS: remove dependency on common-codec Base64. 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 26211019c8 HADOOP-15753. ABFS: support path "abfs://mycluster/file/path" 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt e5593cbd83 HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts. 
Contributed by Sean Mackrory.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 13c70e9ba3 HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept. 
Contributed by Sneha Varma.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 6801b30733 HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation 
for storage account with no namespace feature.

Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 347a52a866 Fixing findbugs and license issues related to: 
HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 97f06b3fc7 HADOOP-15703. ABFS - Implement client-side throttling. 
Contributed by Sneha Varma and Thomas Marquardt.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 4410eacba7 HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 81dc4a995c HADOOP-15663. ABFS: Simplify configuration. 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt df57c6c3b1 HADOOP-15692. ABFS: extensible support for custom oauth. 
Contributed by Junhua Gu and Rajeev Bansal.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt dd2b22fa31 HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle. 
Contributed by Thomas Marquardt.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 6b6f8cc2be HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice. 
Contributed by Sean Mackrory.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 9c1e4e8139 HADOOP-15661. ABFS: Add support for ACL. 
Contributed by Junhua Gu and Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt 9149b9703e HADOOP-15660. ABFS: Add support for OAuth 
Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt d6a4f39bd5 HADOOP-15669. ABFS: Improve HTTPS Performance. 
Contributed by Vishwajeet Dusane.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt cc5cc60c41 Fixing issue due to commit 2b2399d6 after rebase onto trunk. 2018-09-17 19:54:01 +00:00
Thomas Marquardt b54b0c1b67 HADOOP-15659. Code changes for bug fix and new tests. 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt ce03a93f78 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. 
Contributed by Steve Loughran and Da Zhou.
 2018-09-17 19:54:01 +00:00
Steve Loughran a271fd0eca HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies. 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Steve Loughran f044deedbb HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. 
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
 2018-09-17 19:54:01 +00:00
Steve Loughran d7c0a08a1c
HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran) 2018-09-12 21:04:49 -07:00
Aaron Fabbri d32a8d5d58
HADOOP-14734 add option to tag DDB table(s) created. (Contributed by Gabor Bota and Abe Fine) 2018-09-12 16:36:01 -07:00
Mingliang Liu 1f6c4545cf HADOOP-15750. Remove obsolete S3A test ITestS3ACredentialsInURL. Contributed by Steve Loughran 2018-09-12 10:58:39 -07:00