Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,946 Commits 326 Branches 371 Tags
Commit Graph

581 Commits

Author SHA1 Message Date
Mingliang Liu
8ee3e7e4b1 HADOOP-14273. Azure: NativeAzureFileSystem should respect config for kerberosSupportEnabled flag. Contributed by Santhosh G Nayak 
(cherry picked from commit 56ab02eed9b61e1c80605104dfc4c87fc6abac96)
 2017-04-04 11:16:48 -07:00
Mingliang Liu
ce8cdc109b HADOOP-14272. Azure: WasbRemoteCallHelper should use String equals for comparison. Contributed by Santhosh G Nayak 
(cherry picked from commit 27a44b60c1a55cabbf1873bca62db4e321c901ca)
 2017-04-04 11:04:42 -07:00
Mingliang Liu
8c21b2a259 HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka 
(cherry picked from commit 5faa949b782be48ef400d2eb1695f420455de764)
 2017-04-03 11:07:56 -07:00
Akira Ajisaka
bb187e145a
HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0d053eeb3002f348d34cf3aac5b43333af8ff10a)
 2017-03-30 18:13:01 +09:00
John Zhuge
ef43a62482 HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge. 
(cherry picked from commit 253e3e78abb9ebe0e8103d25538ff07875b33c84)
 2017-03-27 23:07:08 -07:00
John Zhuge
100b8ce05b HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge. 
(cherry picked from commit d1b7439b48caa18d64a94be1ad5e4927ce573ab8)
 2017-03-24 13:41:09 -07:00
Chris Douglas
62b8665726 HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge 
(cherry picked from commit c55195588f5853b16d34cb6389cc6f84acc3edea)
 2017-03-23 12:17:42 -07:00
Chris Douglas
05148e33ce HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria 
(cherry picked from commit 7d4884a57bcaf2ab1ad120c436f9c7e6739dbc2a)
 2017-03-23 12:13:33 -07:00
Mingliang Liu
b24b416db5 HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu 
(cherry picked from commit 2e30aa72e01de7b5774fcb312406a393221e0908)
 2017-03-23 11:37:14 -07:00
Steve Loughran
20878d052c
HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject(). 
Contributed by Yuanbo Liu
 2017-03-23 12:52:22 +00:00
John Zhuge
84e2c11f96 HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge. 2017-03-21 08:23:30 -07:00
Steve Loughran
b4481fef41
HADOOP-14204 S3A multipart commit failing, 
"UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".

Contributed by Steve Loughran

(cherry picked from commit 595501c9ddacd5fdb740ac1bf9bfeb0ffbbd6c0e)
 2017-03-21 13:14:43 +00:00
Mingliang Liu
22930b09a0 HADOOP-13945. Azure: Add Kerberos and Delegation token support to WASB client. Contributed by Santhosh G Nayak 
Change-Id: I39fb0a3b0491bf2160571366939a0502a0045429
(cherry picked from commit 8e15e240597f821968e14893eabfea39815de207)
 2017-03-20 16:37:57 -07:00
Arpit Agarwal
90e3283a0a HADOOP-14059. typo in s3a rename(self, subdir) error message. Contributed by Steve Loughran. 2017-03-20 11:29:30 -07:00
Mingliang Liu
ed0d426a88 HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu 2017-03-14 15:00:59 -07:00
John Zhuge
0da850a472 HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge. 
Change-Id: I5dc6f885816b8834f718874542dfa373458b0333
(cherry picked from commit 5a40bafdaeec693e613aa02e79dbaaccfdab6f60)
 2017-03-13 08:36:46 -07:00
Mingliang Liu
0d9030b732 HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu 
(cherry picked from commit 881ec4d97bd1db4582027aec3a4204156a4eda17)
 2017-03-10 13:22:24 -08:00
Steve Loughran
851c0df5d3 HADOOP-14113. Review ADL Docs. Contributed by Steve Loughran 
(cherry picked from commit e60c6543d57611039b0438d5dcb4cb19ee239bb6)
 2017-03-10 13:22:02 -08:00
John Zhuge
f298b992f4 HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge. 
Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e
(cherry picked from commit c5ee7fded46dcb1ac1ea4c1ada4949c50bc89afb)
 2017-03-09 19:53:23 -08:00
Mingliang Liu
acf20c8318 HADOOP-14017. User friendly name for ADLS user and group. Contributed by Vishwajeet Dusane 
(cherry picked from commit 924def78544a64449785f305cb6984c3559aea4d)
 2017-03-09 15:16:28 -08:00
Mingliang Liu
a146866802 HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane 
(cherry picked from commit f4329990250bed62efdebe3ce2bc740092cf9573)
 2017-03-09 15:16:28 -08:00
Lei Xu
ed4388f903 HADOOP-13929. ADLS connector should not check in contract-test-options.xml. (John Zhuge via lei) 
(cherry picked from commit 71c23c9fc94cfdf58de80effbc3f51c0925d0cfe)
 2017-03-09 15:16:28 -08:00
Lei Xu
d3a6124ffa HADOOP-13962. Update ADLS SDK to 2.1.4. (John Zhuge via lei) 
(cherry picked from commit ccf2d662443fc169835d66b715441529658b245c)
 2017-03-09 15:16:28 -08:00
Lei Xu
01624f0c1b HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei) 
(cherry picked from commit e015b563197a475e354bf84fd27e7bbcc67e00a4)
 2017-03-09 15:16:28 -08:00
Mingliang Liu
881b63499e HADOOP-13900. Remove snapshot version of SDK dependency from Azure Data Lake Store File System. Contributed by Vishwajeet Dusane 
(cherry picked from commit ef34bf2bb92a4e8def6617b185ae72db81450de8)
 2017-03-09 15:16:28 -08:00
Mingliang Liu
7fd0556b2b HADOOP-13257. Improve Azure Data Lake contract tests. Contributed by Vishwajeet Dusane 
(cherry picked from commit 4113ec5fa5ca049ebaba039b1faf3911c6a34f7b)
 2017-03-09 15:16:28 -08:00
Chris Douglas
edf149b979 HADOOP-13037. Refactor Azure Data Lake Store as an independent FileSystem. Contributed by Vishwajeet Dusane 2017-03-09 15:16:28 -08:00
John Zhuge
f1fd20be82 Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran" 
This reverts commit 53a12fa721bb431f7d481aac7d245c93efb56153.

(cherry picked from commit 98142d2f722e82d57b0e2bae6276f7c17fd99598)

Change-Id: Ia5fa901fa97391cd46d41f59cd05f7a19c7c7532
 2017-03-08 11:46:27 -08:00
Mingliang Liu
d0e26075b8 HADOOP-14150. Implement getHomeDirectory() method in NativeAzureFileSystem. Contributed by Santhosh G Nayak 
(cherry picked from commit 28daaf0eb206d723d2baf0f9d91e43d98bb2fd26)
 2017-03-07 14:57:41 -08:00
Akira Ajisaka
d2d0872f4d
HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri. 
(cherry picked from commit f597f4c43e0a6e2304b9bcaf727d6d8d15a365f9)
 2017-03-07 15:16:18 +09:00
Mingliang Liu
ff050866bf HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth 
(cherry picked from commit 686823529be09bea2a6cecb3503ef722017475bc)
 2017-03-06 17:35:06 -08:00
Mingliang Liu
8323e651a1 Revert "HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth" 
This reverts commit b26870c58ec6f88c6a3722c5fc856bbbd60c5550.
 2017-03-06 17:18:36 -08:00
Mingliang Liu
2d059b5a40 HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA 
(cherry picked from commit c571cda5c7d929477961dfff4176d7de4944d874)
 2017-03-06 17:01:01 -08:00
Mingliang Liu
b26870c58e HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth 2017-03-03 17:09:05 -08:00
Steve Loughran
51fefbe38b
HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. 
Contributed by Steve Loughran
 2017-03-02 21:41:49 +00:00
Yongjun Zhang
16ca70553c HADOOP-14127. Add log4j configuration to enable logging in hadoop-distcp's tests. (Xiao Chen via Yongjun Zhang) 
(cherry picked from commit d235dcdf0b6f36162280e32c7c0342b7970c9b7b)
 2017-02-27 20:46:17 -08:00
Steve Loughran
29fe5af017
HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures. 
Contributed by Steve Loughran.
 2017-02-25 15:31:26 +00:00
Steve Loughran
c6d7da2112
HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory. 
(cherry picked from commit ff87ca84418a710c6dc884fe8c70947fcc6489d5)
 2017-02-24 10:40:58 +00:00
Wangda Tan
650ff95e00 YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda) 
(cherry picked from commit 0a55bd841ec0f2eb89a0383f4c589526e8b138d4)
 2017-02-23 17:22:25 -08:00
Mingliang Liu
b0f8de184a HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari. 
(cherry picked from commit 82607fce39151fc6ba5bced738088e2bc176dc77)
 2017-02-23 14:11:08 -08:00
Mingliang Liu
db955f0844 HADOOP-14102. Relax error message assertion in S3A test ITestS3AEncryptionSSEC. Contributed by Mingliang Liu 
(cherry picked from commit 0013090fb4340eadf147054e65a73de20a62c1c1)
 2017-02-22 14:38:57 -08:00
Steve Loughran
171b18693a HADOOP-14099 Split S3 testing documentation out into its own file. Contributed by Steve Loughran. 2017-02-22 11:41:24 +00:00
Steve Loughran
6843659689 HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory. 2017-02-21 18:29:30 +00:00
Steve Loughran
274c02d2fa HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan 2017-02-20 16:21:00 +00:00
Steve Loughran
8a88e8ea57 HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge 
(cherry picked from commit b1c1f05b1dc997906390d653dfafb4f0d7e193c4)
 2017-02-18 18:17:54 +00:00
Mingliang Liu
b23e93d465 HADOOP-14040. Use shaded aws-sdk uber-JAR 1.11.86. Contributed by Steve Loughran and Sean Mackrory 
(cherry picked from commit 658702efffdf52cf5ddf8e92f959f1157c95a348)
 2017-02-16 16:59:31 -08:00
Mingliang Liu
29e1007172 HADOOP-14019. Fix some typos in the s3a docs. Contributed by Steve Loughran 
(cherry picked from commit bdad8b7b97d7f48119f016d68f32982d680c8796)
 2017-02-16 16:42:56 -08:00
Akira Ajisaka
25ac54a52e HADOOP-14058. Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks. Contributed by Yiqun Lin. 
(cherry picked from commit b9f8491252f5a23a91a1d695d748556a0fd803ae)
 2017-02-15 01:46:59 +09:00
Lei Xu
6d62d0ea87 HADOOP-13204. Add support for SSE-KMS and SSE-C in s3a filesystem. (Steve Moist via lei) 2017-02-11 15:02:19 -08:00
Arpit Agarwal
c88ec54588 HADOOP-14055. SwiftRestClient includes pass length in exception if auth fails. Contributed by Marcell Hegedus. 2017-02-10 10:09:39 -08:00