Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,859 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

789 Commits

Author SHA1 Message Date
Thomas Marquardt 92f3f9cf09 HADOOP-15692. ABFS: extensible support for custom oauth. 
Contributed by Junhua Gu and Rajeev Bansal.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt b59741f66b HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle. 
Contributed by Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 08d3d50275 HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice. 
Contributed by Sean Mackrory.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt e727c87500 HADOOP-15661. ABFS: Add support for ACL. 
Contributed by Junhua Gu and Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 9d47230c8f HADOOP-15660. ABFS: Add support for OAuth 
Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 92e37a0b73 HADOOP-15669. ABFS: Improve HTTPS Performance. 
Contributed by Vishwajeet Dusane.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt e320b64367 Fixing issue due to commit 2b2399d6 after rebase onto trunk. 2019-04-24 21:46:25 +00:00
Thomas Marquardt 5da4bb8892 HADOOP-15659. Code changes for bug fix and new tests. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt b537d8c442 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. 
Contributed by Steve Loughran and Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran 37426c2da8 HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran 1b64b0f924 HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. 
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
 2019-04-24 21:46:25 +00:00
Wanqiang Ji 313608e459
HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji. 
(cherry picked from commit 8b6deebb1d)
(cherry picked from commit 875435dc7d)
(cherry picked from commit 6f0896e087)
 2019-04-24 11:11:51 +01:00
Masatake Iwasaki 50e30bd30a HADOOP-14544. DistCp documentation for command line options is misaligned. Contributed by Masatake Iwasaki. 
(cherry picked from commit bbdbc7a9a1)

Conflicts:
	hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm

(cherry picked from commit 7985d9b1ce)

Conflicts:
	hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
 2019-04-12 12:35:32 +09:00
Weiwei Yang 5f3abefab3 HADOOP-16191. AliyunOSS: improvements for copyFile/copyDirectory and logging. Contributed by wujinhu. 
(cherry picked from commit 568d3ab8b6)
 2019-03-19 10:15:50 +08:00
Steve Loughran 9b8044d00b
HADOOP-16109. Parquet reading S3AFileSystem causes EOF (#589) 
Nobody gets seek right. No matter how many times they think they have.

Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran

Change-Id: I39b87f3d5daa98f65de2c0a44e348821a4930573
 2019-03-13 15:33:58 +00:00
Kai Xie a49cb4465e
HADOOP-16018. DistCp won't reassemble chunks when blocks per chunk > 0. 
Contributed by Kai Xie.
 2019-02-27 20:09:27 +00:00
Kai Xie 6d3e7a8570
HADOOP-16049. DistCp result has data and checksum mismatch when blocks per chunk > 0. 
Contributed by Kai Xie.
 2019-01-27 16:56:57 +00:00
Weiwei Yang 2a416fe9f2 HADOOP-16030. AliyunOSS: bring fixes back from HADOOP-15671. Contributed by wujinhu. 
(cherry picked from commit f87b3b11c4)
 2019-01-07 17:30:38 +08:00
Sunil G 2c4e4937bd
Revert "HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu." 
This reverts commit e4fca6aae4.

Revert "HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu."

This reverts commit 2b635125fb.

(cherry picked from commit 1f425271a7)
 2019-01-05 17:51:02 +09:00
Weiwei Yang 37e36617c5 HADOOP-15323. AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore. Contributed wujinhu. 
(cherry picked from commit 040a202b20)
 2019-01-03 22:18:05 +08:00
Jonathan Hung 74e40981eb YARN-7794. SLSRunner is not loading timeline service jars, causing failure. 2018-11-30 17:03:01 -08:00
Weiwei Yang 408f60e6d8 HADOOP-15943. AliyunOSS: add missing owner & group attributes for oss FileStatus. Contributed by wujinhu. 
(cherry picked from commit 5ff0cf86a9)
 2018-11-23 14:46:48 +08:00
Weiwei Yang 311be8864f HADOOP-15919. AliyunOSS: Enable Yarn to use OSS. Contributed by wujinhu. 
(cherry picked from commit be0708c6eb)
 2018-11-19 15:03:54 +08:00
Sammi Chen 3aac324a07 HADOOP-15917. AliyunOSS: fix incorrect ReadOps and WriteOps in statistics. Contributed by Jinhu Wu. 
(cherry picked from commit 3fade865ce)
(cherry picked from commit 64cb97fb44)
(cherry picked from commit 5d532cfc6f)
(cherry picked from commit 37082a664a)
 2018-11-14 13:53:53 +08:00
Sammi Chen a0aa610143 HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu. 
(cherry picked from commit 7574d18538)
(cherry picked from commit 366541d834)
(cherry picked from commit c5a227062f)
(cherry picked from commit ca22bf175f)
 2018-10-26 15:35:30 +08:00
Wei-Chiu Chuang f2739f3f59 HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e)
(cherry picked from commit a7dd244a49)

	modified:   hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
(cherry picked from commit 84726679f0)

(cherry picked from commit 8360d15abc)
 2018-10-24 09:39:31 -07:00
Steve Loughran 437bfa90c2
HADOOP-15837. DynamoDB table Update can fail S3A FS init. 
Contributed by Steve Loughran.

(cherry picked from commit f1fbc56371)
 2018-10-12 09:49:28 +01:00
Sammi Chen c617dba497 HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu. 
(cherry picked from commit 2b635125fb)
(cherry picked from commit 5da3e83597)
(cherry picked from commit 85e00477b8)
 2018-09-25 19:55:13 +08:00
Steve Loughran 4e342f603f
HADOOP-15547/ WASB: improve listStatus performance. 
Contributed by Thomas Marquardt.
 2018-09-04 16:32:22 +01:00
Sammi Chen 418e957c64 HADOOP-15607. AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream. Contributed by Jinhu Wu. 2018-08-02 10:14:54 +08:00
Steve Loughran 11e7f2e584
HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled. 
Contributed by Anis Elleuch.

(cherry picked from commit 1dedc68f9d)
(cherry picked from commit cc0f14c13c)
 2018-06-25 17:27:49 +01:00
Chris Douglas 4789b8e9c9 HADOOP-15533. Make WASB listStatus messages consistent. Contributed by Esfandiar Manii 
(cherry picked from commit f34744603e)
(cherry picked from commit 7e655b642d)
(cherry picked from commit d1dcc39222)
 2018-06-17 23:14:12 -07:00
Inigo Goiri 85a7da0899 YARN-8422. TestAMSimulator failing with NPE. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit c35481594f)
 2018-06-12 11:01:45 -07:00
Steve Loughran 0b5d0c3740 HADOOP-15506. Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks. 
Contributed by Esfandiar Manii.

(cherry picked from commit d901be6795)
(cherry picked from commit baac7c2b28)
(cherry picked from commit df338f2e1a)
 2018-06-11 15:34:54 -07:00
Inigo Goiri 0fc8b43dc8 HADOOP-15498. TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 595b44e2dc)
 2018-05-28 16:48:19 -07:00
Steve Loughran e2d32bd9c5 HADOOP-15478. WASB: hflush() and hsync() regression. 
Contributed by Thomas Marquardt.

(cherry picked from commit ba842847c9)
 2018-05-21 11:13:57 +01:00
Steve Loughran 67468651b1 HADOOP-15446. WASB: PageBlobInputStream.skip breaks HBASE replication. 
Contributed by Thomas Marquardt

(cherry picked from commit 5b11b9fd41)
 2018-05-10 10:33:03 -07:00
Junping Du 2b48854cfd HADOOP-15385. Many tests are failing in hadoop-distcp project in branch-2. Contributed by Jason Lowe. 2018-04-25 10:11:41 +08:00
Sammi Chen 1d78825bf1 HADOOP-14999. AliyunOSS: provide one asynchronous multi-part based uploading mechanism. Contributed by Genmao Yu. 2018-04-10 14:24:31 +08:00
Steve Loughran e5351608ca HADOOP-14651. Update okhttp version to 2.7.5. 
Contributed by Ray Chiang and John Zhuge.
 2018-04-04 15:16:20 +01:00
Chris Douglas cc0a791794 HADOOP-15320. Remove customized getFileBlockLocations for hadoop-azure and hadoop-azure-datalake. Contributed by Shanyu Zhao 
(cherry picked from commit 081c350188)
(cherry picked from commit ac16e8f4d3)
 2018-03-28 12:04:20 -07:00
Sammi Chen 2285afb32e HADOOP-15262. AliyunOSS: move files under a directory in parallel when rename a directory. Contributed by Jinhu Wu. 
(cherry picked from commit d67a5e2dec)
 2018-03-19 15:09:23 +08:00
Chris Douglas 340cd5f1b1 HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 
(cherry picked from commit 45d1b0fdcc)
(cherry picked from commit 09940b1eb3)
(cherry picked from commit a03c8ea61f)
 2018-03-12 20:08:46 -07:00
Steve Loughran c245050538 HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran; backport to branch-2 by Masatake Iwasaki.
 2018-03-05 12:46:19 +00:00
Gergely Novák 278676508f
MAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e99a1e7767)
 2018-03-02 16:22:20 -08:00
Steve Loughran fb0f2cc2fb Revert "HADOOP-15090. Add ADL troubleshooting doc." 
This reverts commit 6f944452f7.
 2018-02-28 21:19:03 +00:00
Chris Douglas 762125b864 Backport HADOOP-13514 (surefire upgrade) to branch-2 2018-02-26 16:32:06 -08:00
Xuan Gong a0b55392d4 MAPREDUCE-7027: HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails. Contributed by Gergely Novák 
(cherry picked from commit 68ce193efc)
 2018-02-23 14:40:51 -08:00
Steve Loughran 07c7df4b26 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.

(cherry picked from commit 050f5287b7)
 2018-02-15 16:29:44 +00:00
Steve Loughran 6f944452f7 HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran.

(cherry picked from commit 58a2120e8a)
 2018-02-15 14:26:36 +00:00
Sammi Chen 2816bd1f43 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
(cherry picked from commit 91184299c5)
 2018-01-30 16:25:02 +08:00
Steve Loughran 312c571694 HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu
 2018-01-27 20:31:21 -08:00
Jason Lowe 4d7bbee0a1 YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 2018-01-25 17:56:36 -06:00
Jason Lowe fd09f72151 Revert "HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)" 
This reverts commit 896dc7c780.
 2018-01-17 09:47:43 -06:00
Sammi Chen 896dc7c780 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
(cherry picked from commit 55142849db)
(cherry picked from commit 082a707bae)
 2018-01-17 16:36:03 +08:00
Steve Loughran d2ceef0f20 HADOOP-15086. NativeAzureFileSystem file rename is not atomic. 
Contributed by Thomas Marquardt
(Backported to branch-2 via /HADOOP-15156)

(cherry picked from commit a35267b47a)
 2018-01-08 15:13:50 +00:00
Sammi Chen 1fee447bf2 HADOOP-15111. Aliyun OSS: backport HADOOP-14993 to branch-2. Contributed by Genmao Yu. 2017-12-15 14:28:39 +08:00
Sammi Chen 27807e4ccb HADOOP-15104. Aliyun OSS: change the default value of max error retry. Contributed by Jinhu Wu. 
(cherry picked from commit ce04340ec7)
 2017-12-15 13:53:48 +08:00
Surendra Singh Lilhore 8d69b7f2d4 HDFS-12833. Distcp : Update the usage of delete option for dependency with update and overwrite option. Contributed by usharani. 2017-12-13 10:29:19 +05:30
Sammi Chen 94390fcd52 HADOOP-15024. AliyunOSS: Support user agent configuration and include that & Hadoop version information to oss server. 
(cherry picked from commit 6e24fe2998)
 2017-12-08 22:02:00 +08:00
Xiao Chen c9d48352c2 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-12-07 21:20:33 -08:00
Steve Loughran 5c37a0b841 HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. 
Contributed by Rajesh Balamohan

(cherry picked from commit 0ea182d0fa)
 2017-11-28 11:57:51 +00:00
Kai Zheng 30ab9b6aef HADOOP-14964. AliyunOSS: backport Aliyun OSS module to branch-2. Contributed by Sammi Chen 
The consolidated commits in this backport are as follows:
    HADOOP-14787. AliyunOSS: Implement the `createNonRecursive` operator.
    HADOOP-14649. Update aliyun-sdk-oss version to 2.8.1. (Genmao Yu via rchiang)
    HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu
    HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang.
    HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu.
    HADOOP-14192. AliyunOSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu
    HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu
    HADOOP-13769. AliyunOSS: update oss sdk version. Contributed by Genmao Yu
    HADOOP-14069. AliyunOSS: listStatus returns wrong file info. Contributed by Fei Hui
    HADOOP-13768. AliyunOSS: handle the failure in the batch delete operation `deleteDirs`. Contributed by Genmao Yu
    HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. Contributed by Fei Hui
    HADOOP-14045. Aliyun OSS documentation missing from website. Contributed by Yiqun Lin.
    HADOOP-13723. AliyunOSSInputStream#read() should update read bytes stat correctly. Contributed by Mingliang Liu
    HADOOP-13624.  Rename TestAliyunOSSContractDispCp. Contributed by Genmao Yu
    HADOOP-13591. Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir. Contributed by Genmao Yu
    HADOOP-13481. User documents for Aliyun OSS FileSystem. Contributed by Genmao Yu.
    HADOOP-12756. Incorporate Aliyun OSS file system implementation. Contributed by Mingfei Shi and Lin Zhou
 2017-11-13 09:29:33 +08:00
Subru Krishnan 5991c218a0 HADOOP-15026. Rebase ResourceEstimator start/stop scripts for branch-2. (Rui Li via Subru). 2017-11-09 00:32:34 -08:00
Subru Krishnan 46a740a82e HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru). 
(cherry picked from commit f2df6b8983)
 2017-11-08 18:09:02 -08:00
Arun Suresh 0200fbac5b HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh) 
(cherry picked from commit 53c0fb7efe)
 2017-11-02 17:18:03 -07:00
Akira Ajisaka 82fc80a987
MAPREDUCE-6999. Fix typo onf in DynamicInputChunk.java. Contributed by fang zhenyi. 
(cherry picked from commit cc3f3eca40)
 2017-11-02 18:33:25 +09:00
Arun Suresh 71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Chris Douglas ee89241282 HADOOP-14991. Add missing figures to Resource Estimator tool. Contributed by Rui Li 
(cherry picked from commit 139cc752d2)
(cherry picked from commit 77ad73501c)
 2017-10-27 22:01:41 -07:00
Steve Loughran 3fbcf76324 HADOOP-14553. Add (parallelized) integration tests to hadoop-azure. 
Contributed by Steve Loughran.

(backport JIRA HADOOP-14870)
 2017-10-27 12:55:12 +01:00
Subru Krishnan 9897538a96 HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru). 
(cherry picked from commit 625039ef20)
 2017-10-26 19:14:24 -07:00
Akira Ajisaka 13f1a7e0ed
Revert "HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru)." 
This reverts commit da7f989d86.
 2017-10-26 15:33:53 +09:00
Subru Krishnan da7f989d86 HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru). 
(cherry picked from commit 625039ef20)
 2017-10-25 17:05:35 -07:00
Steve Loughran 062d9c9bab HADOOP-14942. DistCp#cleanup() should check whether jobFS is null. 
Contributed by Andras Bokor.

(cherry picked from commit f36cbc8475)
 2017-10-20 22:27:59 +01:00
Wangda Tan d894f910d0 YARN-6608. Backport all SLS improvements from trunk to branch-2. (Carlo Curino via wangda) 2017-10-17 19:45:09 -07:00
Steve Loughran 8902178d3e HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled. 
Contributed by Santhosh G Nayak

Cherry-picked from commit 9fcc3a1fc8
 2017-10-16 16:06:41 +01:00
Steve Loughran ccfd1a4a56 HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB. 
Contributed by Varada Hemeswari.

Cherry-picked from commit ref 3de574413c
 2017-10-11 18:09:13 +01:00
Steve Loughran 34040e8d73 HADOOP-14899. Restrict Access to setPermission operation when authorization is enabled in WASB 
Contributed by Kannapiran Srinivasan.
 2017-10-07 11:20:18 +01:00
Steve Loughran fa6b8feb9e
HADOOP-14845. Azure wasb: getFileStatus not making any auth check. 
Final fixup of TestNativeAzureFileSystemAuthorization in branch-2 to match trunk's changes
Contributed by Sivaguru Sankaridurg
 2017-10-05 16:03:13 +01:00
Steve Loughran 28e05dc1da
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. 
Contributed by Steve Loughran
 2017-10-03 20:06:45 +01:00
Steve Loughran 1eecf8a97c
HADOOP-14768. Honoring sticky bit during Deletion when authorization is enabled in WASB 
Contributed by Varada Hemeswari
 2017-10-02 14:28:59 +01:00
Steve Loughran a1afc6aa59
HADOOP-13345 S3Guard: Improved Consistency for S3A. 
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.

(cherry-picked from/based on commit 621b43e254)
 2017-09-29 11:29:22 +01:00
Steve Loughran a641bcec01
HADOOP-14845. Azure wasb: getFileStatus not making any auth check. 
Contributed by Sivaguru Sankaridurg
 2017-09-29 11:16:31 +01:00
Steve Loughran 27a41848ac
HADOOP-14890 Move up to AWS SDK 1.11.199. 
Contributed by Steve Loughran.
 2017-09-26 20:11:10 +01:00
Jason Lowe 2c10abaddd MAPREDUCE-6967. gridmix/SleepReducer should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari 
(cherry picked from commit b133dc5700)
 2017-09-22 15:30:53 -05:00
Steve Loughran 12a026badb
HADOOP-14583. wasb throws an exception if you try to create a file and there's no parent directory. 
Contributed by Esfandiar Manii.
 2017-09-19 11:02:10 +01:00
Xiaoyu Yao b669aca518 HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin. 2017-09-11 11:40:48 -07:00
Steve Loughran 87af3f4991
HADOOP-14520. WASB: Block compaction for Azure Block Blobs. 
Contributed by Georgi Chalakov
 2017-09-11 14:15:58 +01:00
John Zhuge 1421196d20 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1)
 2017-09-08 14:15:51 -07:00
Steve Loughran 6e20ae954c
HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter. 
Contributed by Yonger.

(cherry picked from commit 83449abb40)
 2017-09-07 19:58:40 +01:00
John Zhuge 4c34c03b27 HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge. 2017-09-06 01:03:18 -07:00
Steve Loughran f382f66255
HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS. 
Contributed by Sivaguru Sankaridurg
 2017-09-05 14:18:43 +01:00
John Zhuge 2442a8d716 HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria. 
(cherry picked from commit 7ecc6dbed6)
 2017-08-31 21:13:50 -07:00
Yufei Gu 626fa7c8e1 YARN-7077. TestAMSimulator and TestNMSimulator fail. (Contributed by Akira Ajisaka via Yufei Gu) 2017-08-30 23:08:02 -07:00
Steve Loughran 25e9c29608
HADOOP-14802. Add support for using container saskeys for all accesses. 
Contributed by Sivaguru Sankaridurg

(cherry picked from commit 021974f4cb)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2017-08-30 22:40:50 +01:00
Arun Suresh f8f193c8e8 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 8410d862d3)
 2017-08-20 08:31:17 -07:00
Steve Loughran c42e694ee5
HADOOP-14769. WASB: delete recursive should not fail if a file is deleted. 
Contributed by Thomas Marquardt

(cherry picked from commit c6b4e656b7)
 2017-08-18 14:38:04 +01:00
Steve Loughran d5680c08ed
HADOOP-14660. wasb: improve throughput by 34% when account limit exceeded. 
Contributed by Thomas Marquardt.
 2017-08-16 13:49:47 +01:00
John Zhuge 0efc590e6d HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 
(cherry picked from commit 7769e96149)
 2017-08-13 00:27:33 -07:00
Steve Loughran 96befde480
HADOOP-14183. Remove service loader config file for wasb fs. 
Contributed by Esfandiar Manii.

(cherry picked from commit 54356b1e83)
 2017-08-10 16:49:51 +01:00