Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,334 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

552 Commits

Author SHA1 Message Date
John Zhuge de50bb6365 HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 
(cherry picked from commit 7769e96149)
 2017-08-13 00:32:25 -07:00
John Zhuge a1682876c9 HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge. 
(cherry picked from commit f735ad1b67)
 2017-07-28 09:55:14 -07:00
Naganarasimha 582bdfd423 YARN-6719. Fix findbugs warnings in SLSCapacityScheduler.java. Contributed by Akira Ajisaka. 2017-06-19 12:47:25 +05:30
Mingliang Liu 7eee4fed89 HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu 
(cherry picked from commit 325163f23f)
 2017-06-09 13:07:55 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 396de0fd8f Preparing for 2.8.3 development 2017-06-07 15:38:20 -07:00
Christopher Douglas d013f4134b YARN-1471. Preserve scheduler typeinfo in simulator to work with resource monitors. 
Contributed by Carlo Curino. Backport by Ye Zhou.

(cherry picked from commit e07f3c87cd)
 2017-06-05 16:19:55 -07:00
John Zhuge f8a3de64f8 HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge. 
(cherry picked from commit 6c6a7a5962)
 2017-05-28 22:29:49 -07:00
Xuan cef5ef2377 YARN-4612. Fix rumen and scheduler load simulator handle killed tasks 
properly. Contributed by Ming Ma.

(cherry picked from commit 4efdf3a979)
 2017-05-21 12:51:16 -07:00
Akira Ajisaka 8007d024b8
HADOOP-14374. License error in GridmixTestUtils.java. Contributed by lixinglong. 
(cherry picked from commit c60164fb60)
(cherry picked from commit 753954f7be)
 2017-05-09 05:39:26 -05:00
Mingliang Liu c3c9a91900 HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu 
(cherry picked from commit e4c32bccde)
(cherry picked from commit e7a08cc6dc1f729d04dea9239f0a8c90c8be5598)
 2017-04-25 20:55:42 -07:00
Steve Loughran 952113ebfd
HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs. 
Contributed by Steve Loughran

(cherry picked from commit 1ae6be46b4)
 2017-04-19 15:37:27 +01:00
Junping Du dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
John Zhuge fa8e411d47 HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge. 
Signed-off-by: John Zhuge <jzhuge@apache.org>
(cherry picked from commit 56e81f2a20)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2017-04-07 15:05:09 -07:00
Mingliang Liu e47c95ffbf HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka 2017-04-03 11:15:03 -07:00
Akira Ajisaka 6e005ad326
HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0d053eeb30)
(cherry picked from commit bb187e145a)
 2017-03-30 18:13:55 +09:00
John Zhuge c543c0549d HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge. 
(cherry picked from commit 253e3e78ab)
 2017-03-27 23:08:03 -07:00
John Zhuge d370d8ddb8 HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge. 
(cherry picked from commit d1b7439b48)
 2017-03-24 13:41:34 -07:00
Chris Douglas 5e794c74e5 HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge 
(cherry picked from commit c55195588f)
(cherry picked from commit 62b8665726)
 2017-03-23 12:17:52 -07:00
Chris Douglas 88252e85ad HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria 
(cherry picked from commit 7d4884a57bcaf2ab1ad120c436f9c7e6739dbc2a)
(cherry picked from commit 2cea8293ef519ead772564750bd5e7f8b9d33da5)
 2017-03-23 12:13:42 -07:00
Mingliang Liu 45956131b9 HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu 
(cherry picked from commit 2e30aa72e0)
 2017-03-23 11:42:23 -07:00
John Zhuge 3a8b120c78 HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge. 
(cherry picked from commit 84e2c11f96)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java
 2017-03-21 08:37:18 -07:00
Steve Loughran 595501c9dd
HADOOP-14204 S3A multipart commit failing, 
"UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".

Contributed by Steve Loughran
 2017-03-21 13:13:16 +00:00
John Zhuge 72fc7e0520 HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge. 
Change-Id: I5dc6f885816b8834f718874542dfa373458b0333
(cherry picked from commit 5a40bafdae)
 2017-03-13 09:15:09 -07:00
Mingliang Liu b4d768123b HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu 
(cherry picked from commit 881ec4d97b)
 2017-03-10 13:23:17 -08:00
Steve Loughran 1f92a9dbae HADOOP-14113. Review ADL Docs. Contributed by Steve Loughran 
(cherry picked from commit e60c6543d5)
 2017-03-10 13:23:11 -08:00
John Zhuge 843fa685a6 HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge. 
Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e
(cherry picked from commit c5ee7fded4)
 2017-03-09 19:54:19 -08:00
Mingliang Liu 518705b8e3 HADOOP-14017. User friendly name for ADLS user and group. Contributed by Vishwajeet Dusane 
(cherry picked from commit 924def7854)
(cherry picked from commit acf20c8318)
 2017-03-09 15:21:33 -08:00
Mingliang Liu f0df481c76 HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane 
(cherry picked from commit f432999025)
(cherry picked from commit a146866802)
 2017-03-09 15:21:32 -08:00
Lei Xu 674b9cdab1 HADOOP-13929. ADLS connector should not check in contract-test-options.xml. (John Zhuge via lei) 
(cherry picked from commit 71c23c9fc9)
(cherry picked from commit ed4388f903)
 2017-03-09 15:21:32 -08:00
Lei Xu 87abe01ab1 HADOOP-13962. Update ADLS SDK to 2.1.4. (John Zhuge via lei) 
(cherry picked from commit ccf2d66244)
(cherry picked from commit d3a6124ffa)
 2017-03-09 15:21:32 -08:00
Lei Xu 5eaec560d9 HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei) 
(cherry picked from commit e015b56319)
(cherry picked from commit 01624f0c1b)
 2017-03-09 15:21:32 -08:00
Mingliang Liu 8a51e4f89d HADOOP-13900. Remove snapshot version of SDK dependency from Azure Data Lake Store File System. Contributed by Vishwajeet Dusane 
(cherry picked from commit ef34bf2bb9)
(cherry picked from commit 881b63499e)
 2017-03-09 15:21:31 -08:00
Mingliang Liu f1671e9519 HADOOP-13257. Improve Azure Data Lake contract tests. Contributed by Vishwajeet Dusane 
(cherry picked from commit 4113ec5fa5)
(cherry picked from commit 7fd0556b2b)
 2017-03-09 15:21:31 -08:00
Chris Douglas b4b4ca9199 HADOOP-13037. Refactor Azure Data Lake Store as an independent FileSystem. Contributed by Vishwajeet Dusane 
(cherry picked from commit edf149b979)
 2017-03-09 15:21:21 -08:00
John Zhuge cdfa86cadd Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran" 
This reverts commit 53a12fa721.

(cherry picked from commit 98142d2f72)

Change-Id: I3a7613a7f4f0e9c80bb6d3c40a53742f807fc689
 2017-03-08 11:47:36 -08:00
Akira Ajisaka 20fa7664d1
HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri. 
(cherry picked from commit f597f4c43e)
(cherry picked from commit d2d0872f4d)
 2017-03-07 15:17:42 +09:00
Mingliang Liu ff706261be HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA 
(cherry picked from commit c571cda5c7)
 2017-03-06 17:01:26 -08:00
Steve Loughran 2acc4d913e
HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. 
Contributed by Steve Loughran

(cherry picked from commit 51fefbe38b)
 2017-03-02 21:43:05 +00:00
Steve Loughran 05ab09ca87
HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures. 
Contributed by Steve Loughran.
 2017-02-25 16:10:15 +00:00
Steve Loughran ff87ca8441
HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory. 2017-02-24 10:35:21 +00:00
Mingliang Liu b1ec790662 HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari. 
(cherry picked from commit 82607fce39)
 2017-02-23 14:11:34 -08:00
Steve Loughran bb4b2f641c HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory. 
(cherry picked from commit e3a74e0369e6e2217d1280179b390227fe1b1684)
 2017-02-21 18:29:46 +00:00
Steve Loughran 8a05ea4aba HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan 2017-02-20 16:21:12 +00:00
Steve Loughran b1c1f05b1d HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge 2017-02-18 18:16:19 +00:00
Mingliang Liu 3b4f840ccb HADOOP-14019. Fix some typos in the s3a docs. Contributed by Steve Loughran 
(cherry picked from commit bdad8b7b97)
 2017-02-16 16:43:22 -08:00
Akira Ajisaka 4718f65545 HADOOP-14058. Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks. Contributed by Yiqun Lin. 
(cherry picked from commit b9f8491252)
(cherry picked from commit 25ac54a52e)
 2017-02-15 01:47:21 +09:00
Steve Loughran 77401bd5fc HADOOP-13496. Include file lengths in Mismatch in length error for distcp. Contributed by Ted Yu 2017-01-19 11:24:58 +00:00
Steve Loughran e32cd52481 HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran 2017-01-11 17:15:01 +00:00
Junping Du 7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Chris Nauroth 15e89634a5 HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 4c8f9e1302)
(cherry picked from commit 9c61858ee7)
 2017-01-05 15:05:12 -08:00