Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,189 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

808 Commits

Author SHA1 Message Date
Steve Loughran ee0fa0d15c HADOOP-15739. ABFS: remove unused maven dependencies and add used undeclared dependencies. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran bde00d56df HADOOP-15723. ABFS: Ranger Support. 
Contributed by Yuan Gao.
 2019-04-24 21:46:25 +00:00
Steve Loughran 4f3cc1d417 HADOOP-15778. ABFS: Fix client side throttling for read. 
Contributed by Sneha Varma.
 2019-04-24 21:46:25 +00:00
Steve Loughran 6c7c8b83f7 HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable. 
Contributed by Steve Loughran.
 2019-04-24 21:46:25 +00:00
Sean Mackrory 920b77c8d7 HADOOP-15773. Fixing checkstyle and other issues raised by Yetus. 2019-04-24 21:46:25 +00:00
Steve Loughran 10e149d882 HADOOP-15769. ABFS: distcp tests are always skipped. 
Contributed by Steve Loughran
 2019-04-24 21:46:25 +00:00
Steve Loughran 1efa1b73a9 HADOOP-15719. Fail-fast when using OAuth over http. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran 8acab6c6aa HADOOP-15714. Tune abfs/wasb parallel and sequential test execution. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran 26ce4e9301 HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min. 
Contributed by Da Zhou
 2019-04-24 21:46:25 +00:00
Steve Loughran 94d0de2eab HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. 
Contributed by Steve Loughran.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 3599cd3dc9 HADOOP-15757. ABFS: remove dependency on common-codec Base64. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 263094f962 HADOOP-15753. ABFS: support path "abfs://mycluster/file/path" 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt e52f4a1ad2 HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts. 
Contributed by Sean Mackrory.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 36b2bab888 HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept. 
Contributed by Sneha Varma.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt a192d85a5f HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation 
for storage account with no namespace feature.

Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 8bb713e31a Fixing findbugs and license issues related to: 
HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 305e750e82 HADOOP-15703. ABFS - Implement client-side throttling. 
Contributed by Sneha Varma and Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 12c621eda4 HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 3bd089ddca HADOOP-15663. ABFS: Simplify configuration. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 92f3f9cf09 HADOOP-15692. ABFS: extensible support for custom oauth. 
Contributed by Junhua Gu and Rajeev Bansal.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt b59741f66b HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle. 
Contributed by Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 08d3d50275 HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice. 
Contributed by Sean Mackrory.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt e727c87500 HADOOP-15661. ABFS: Add support for ACL. 
Contributed by Junhua Gu and Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 9d47230c8f HADOOP-15660. ABFS: Add support for OAuth 
Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 92e37a0b73 HADOOP-15669. ABFS: Improve HTTPS Performance. 
Contributed by Vishwajeet Dusane.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt e320b64367 Fixing issue due to commit 2b2399d6 after rebase onto trunk. 2019-04-24 21:46:25 +00:00
Thomas Marquardt 5da4bb8892 HADOOP-15659. Code changes for bug fix and new tests. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt b537d8c442 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. 
Contributed by Steve Loughran and Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran 37426c2da8 HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran 1b64b0f924 HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. 
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
 2019-04-24 21:46:25 +00:00
Wanqiang Ji 313608e459
HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji. 
(cherry picked from commit 8b6deebb1d)
(cherry picked from commit 875435dc7d)
(cherry picked from commit 6f0896e087)
 2019-04-24 11:11:51 +01:00
Masatake Iwasaki 50e30bd30a HADOOP-14544. DistCp documentation for command line options is misaligned. Contributed by Masatake Iwasaki. 
(cherry picked from commit bbdbc7a9a1)

Conflicts:
	hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm

(cherry picked from commit 7985d9b1ce)

Conflicts:
	hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
 2019-04-12 12:35:32 +09:00
Weiwei Yang 5f3abefab3 HADOOP-16191. AliyunOSS: improvements for copyFile/copyDirectory and logging. Contributed by wujinhu. 
(cherry picked from commit 568d3ab8b6)
 2019-03-19 10:15:50 +08:00
Steve Loughran 9b8044d00b
HADOOP-16109. Parquet reading S3AFileSystem causes EOF (#589) 
Nobody gets seek right. No matter how many times they think they have.

Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran

Change-Id: I39b87f3d5daa98f65de2c0a44e348821a4930573
 2019-03-13 15:33:58 +00:00
Kai Xie a49cb4465e
HADOOP-16018. DistCp won't reassemble chunks when blocks per chunk > 0. 
Contributed by Kai Xie.
 2019-02-27 20:09:27 +00:00
Kai Xie 6d3e7a8570
HADOOP-16049. DistCp result has data and checksum mismatch when blocks per chunk > 0. 
Contributed by Kai Xie.
 2019-01-27 16:56:57 +00:00
Weiwei Yang 2a416fe9f2 HADOOP-16030. AliyunOSS: bring fixes back from HADOOP-15671. Contributed by wujinhu. 
(cherry picked from commit f87b3b11c4)
 2019-01-07 17:30:38 +08:00
Sunil G 2c4e4937bd
Revert "HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu." 
This reverts commit e4fca6aae4.

Revert "HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu."

This reverts commit 2b635125fb.

(cherry picked from commit 1f425271a7)
 2019-01-05 17:51:02 +09:00
Weiwei Yang 37e36617c5 HADOOP-15323. AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore. Contributed wujinhu. 
(cherry picked from commit 040a202b20)
 2019-01-03 22:18:05 +08:00
Jonathan Hung 74e40981eb YARN-7794. SLSRunner is not loading timeline service jars, causing failure. 2018-11-30 17:03:01 -08:00
Weiwei Yang 408f60e6d8 HADOOP-15943. AliyunOSS: add missing owner & group attributes for oss FileStatus. Contributed by wujinhu. 
(cherry picked from commit 5ff0cf86a9)
 2018-11-23 14:46:48 +08:00
Weiwei Yang 311be8864f HADOOP-15919. AliyunOSS: Enable Yarn to use OSS. Contributed by wujinhu. 
(cherry picked from commit be0708c6eb)
 2018-11-19 15:03:54 +08:00
Sammi Chen 3aac324a07 HADOOP-15917. AliyunOSS: fix incorrect ReadOps and WriteOps in statistics. Contributed by Jinhu Wu. 
(cherry picked from commit 3fade865ce)
(cherry picked from commit 64cb97fb44)
(cherry picked from commit 5d532cfc6f)
(cherry picked from commit 37082a664a)
 2018-11-14 13:53:53 +08:00
Sammi Chen a0aa610143 HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu. 
(cherry picked from commit 7574d18538)
(cherry picked from commit 366541d834)
(cherry picked from commit c5a227062f)
(cherry picked from commit ca22bf175f)
 2018-10-26 15:35:30 +08:00
Wei-Chiu Chuang f2739f3f59 HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e)
(cherry picked from commit a7dd244a49)

	modified:   hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
(cherry picked from commit 84726679f0)

(cherry picked from commit 8360d15abc)
 2018-10-24 09:39:31 -07:00
Steve Loughran 437bfa90c2
HADOOP-15837. DynamoDB table Update can fail S3A FS init. 
Contributed by Steve Loughran.

(cherry picked from commit f1fbc56371)
 2018-10-12 09:49:28 +01:00
Sammi Chen c617dba497 HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu. 
(cherry picked from commit 2b635125fb)
(cherry picked from commit 5da3e83597)
(cherry picked from commit 85e00477b8)
 2018-09-25 19:55:13 +08:00
Steve Loughran 4e342f603f
HADOOP-15547/ WASB: improve listStatus performance. 
Contributed by Thomas Marquardt.
 2018-09-04 16:32:22 +01:00
Sammi Chen 418e957c64 HADOOP-15607. AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream. Contributed by Jinhu Wu. 2018-08-02 10:14:54 +08:00
Steve Loughran 11e7f2e584
HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled. 
Contributed by Anis Elleuch.

(cherry picked from commit 1dedc68f9d)
(cherry picked from commit cc0f14c13c)
 2018-06-25 17:27:49 +01:00
Chris Douglas 4789b8e9c9 HADOOP-15533. Make WASB listStatus messages consistent. Contributed by Esfandiar Manii 
(cherry picked from commit f34744603e)
(cherry picked from commit 7e655b642d)
(cherry picked from commit d1dcc39222)
 2018-06-17 23:14:12 -07:00
Inigo Goiri 85a7da0899 YARN-8422. TestAMSimulator failing with NPE. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit c35481594f)
 2018-06-12 11:01:45 -07:00
Steve Loughran 0b5d0c3740 HADOOP-15506. Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks. 
Contributed by Esfandiar Manii.

(cherry picked from commit d901be6795)
(cherry picked from commit baac7c2b28)
(cherry picked from commit df338f2e1a)
 2018-06-11 15:34:54 -07:00
Inigo Goiri 0fc8b43dc8 HADOOP-15498. TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 595b44e2dc)
 2018-05-28 16:48:19 -07:00
Steve Loughran e2d32bd9c5 HADOOP-15478. WASB: hflush() and hsync() regression. 
Contributed by Thomas Marquardt.

(cherry picked from commit ba842847c9)
 2018-05-21 11:13:57 +01:00
Steve Loughran 67468651b1 HADOOP-15446. WASB: PageBlobInputStream.skip breaks HBASE replication. 
Contributed by Thomas Marquardt

(cherry picked from commit 5b11b9fd41)
 2018-05-10 10:33:03 -07:00
Junping Du 2b48854cfd HADOOP-15385. Many tests are failing in hadoop-distcp project in branch-2. Contributed by Jason Lowe. 2018-04-25 10:11:41 +08:00
Sammi Chen 1d78825bf1 HADOOP-14999. AliyunOSS: provide one asynchronous multi-part based uploading mechanism. Contributed by Genmao Yu. 2018-04-10 14:24:31 +08:00
Steve Loughran e5351608ca HADOOP-14651. Update okhttp version to 2.7.5. 
Contributed by Ray Chiang and John Zhuge.
 2018-04-04 15:16:20 +01:00
Chris Douglas cc0a791794 HADOOP-15320. Remove customized getFileBlockLocations for hadoop-azure and hadoop-azure-datalake. Contributed by Shanyu Zhao 
(cherry picked from commit 081c350188)
(cherry picked from commit ac16e8f4d3)
 2018-03-28 12:04:20 -07:00
Sammi Chen 2285afb32e HADOOP-15262. AliyunOSS: move files under a directory in parallel when rename a directory. Contributed by Jinhu Wu. 
(cherry picked from commit d67a5e2dec)
 2018-03-19 15:09:23 +08:00
Chris Douglas 340cd5f1b1 HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 
(cherry picked from commit 45d1b0fdcc)
(cherry picked from commit 09940b1eb3)
(cherry picked from commit a03c8ea61f)
 2018-03-12 20:08:46 -07:00
Steve Loughran c245050538 HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran; backport to branch-2 by Masatake Iwasaki.
 2018-03-05 12:46:19 +00:00
Gergely Novák 278676508f
MAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e99a1e7767)
 2018-03-02 16:22:20 -08:00
Steve Loughran fb0f2cc2fb Revert "HADOOP-15090. Add ADL troubleshooting doc." 
This reverts commit 6f944452f7.
 2018-02-28 21:19:03 +00:00
Chris Douglas 762125b864 Backport HADOOP-13514 (surefire upgrade) to branch-2 2018-02-26 16:32:06 -08:00
Xuan Gong a0b55392d4 MAPREDUCE-7027: HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails. Contributed by Gergely Novák 
(cherry picked from commit 68ce193efc)
 2018-02-23 14:40:51 -08:00
Steve Loughran 07c7df4b26 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.

(cherry picked from commit 050f5287b7)
 2018-02-15 16:29:44 +00:00
Steve Loughran 6f944452f7 HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran.

(cherry picked from commit 58a2120e8a)
 2018-02-15 14:26:36 +00:00
Sammi Chen 2816bd1f43 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
(cherry picked from commit 91184299c5)
 2018-01-30 16:25:02 +08:00
Steve Loughran 312c571694 HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu
 2018-01-27 20:31:21 -08:00
Jason Lowe 4d7bbee0a1 YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 2018-01-25 17:56:36 -06:00
Jason Lowe fd09f72151 Revert "HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)" 
This reverts commit 896dc7c780.
 2018-01-17 09:47:43 -06:00
Sammi Chen 896dc7c780 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
(cherry picked from commit 55142849db)
(cherry picked from commit 082a707bae)
 2018-01-17 16:36:03 +08:00
Steve Loughran d2ceef0f20 HADOOP-15086. NativeAzureFileSystem file rename is not atomic. 
Contributed by Thomas Marquardt
(Backported to branch-2 via /HADOOP-15156)

(cherry picked from commit a35267b47a)
 2018-01-08 15:13:50 +00:00
Sammi Chen 1fee447bf2 HADOOP-15111. Aliyun OSS: backport HADOOP-14993 to branch-2. Contributed by Genmao Yu. 2017-12-15 14:28:39 +08:00
Sammi Chen 27807e4ccb HADOOP-15104. Aliyun OSS: change the default value of max error retry. Contributed by Jinhu Wu. 
(cherry picked from commit ce04340ec7)
 2017-12-15 13:53:48 +08:00
Surendra Singh Lilhore 8d69b7f2d4 HDFS-12833. Distcp : Update the usage of delete option for dependency with update and overwrite option. Contributed by usharani. 2017-12-13 10:29:19 +05:30
Sammi Chen 94390fcd52 HADOOP-15024. AliyunOSS: Support user agent configuration and include that & Hadoop version information to oss server. 
(cherry picked from commit 6e24fe2998)
 2017-12-08 22:02:00 +08:00
Xiao Chen c9d48352c2 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-12-07 21:20:33 -08:00
Steve Loughran 5c37a0b841 HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. 
Contributed by Rajesh Balamohan

(cherry picked from commit 0ea182d0fa)
 2017-11-28 11:57:51 +00:00
Kai Zheng 30ab9b6aef HADOOP-14964. AliyunOSS: backport Aliyun OSS module to branch-2. Contributed by Sammi Chen 
The consolidated commits in this backport are as follows:
    HADOOP-14787. AliyunOSS: Implement the `createNonRecursive` operator.
    HADOOP-14649. Update aliyun-sdk-oss version to 2.8.1. (Genmao Yu via rchiang)
    HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu
    HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang.
    HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu.
    HADOOP-14192. AliyunOSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu
    HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu
    HADOOP-13769. AliyunOSS: update oss sdk version. Contributed by Genmao Yu
    HADOOP-14069. AliyunOSS: listStatus returns wrong file info. Contributed by Fei Hui
    HADOOP-13768. AliyunOSS: handle the failure in the batch delete operation `deleteDirs`. Contributed by Genmao Yu
    HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. Contributed by Fei Hui
    HADOOP-14045. Aliyun OSS documentation missing from website. Contributed by Yiqun Lin.
    HADOOP-13723. AliyunOSSInputStream#read() should update read bytes stat correctly. Contributed by Mingliang Liu
    HADOOP-13624.  Rename TestAliyunOSSContractDispCp. Contributed by Genmao Yu
    HADOOP-13591. Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir. Contributed by Genmao Yu
    HADOOP-13481. User documents for Aliyun OSS FileSystem. Contributed by Genmao Yu.
    HADOOP-12756. Incorporate Aliyun OSS file system implementation. Contributed by Mingfei Shi and Lin Zhou
 2017-11-13 09:29:33 +08:00
Subru Krishnan 5991c218a0 HADOOP-15026. Rebase ResourceEstimator start/stop scripts for branch-2. (Rui Li via Subru). 2017-11-09 00:32:34 -08:00
Subru Krishnan 46a740a82e HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru). 
(cherry picked from commit f2df6b8983)
 2017-11-08 18:09:02 -08:00
Arun Suresh 0200fbac5b HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh) 
(cherry picked from commit 53c0fb7efe)
 2017-11-02 17:18:03 -07:00
Akira Ajisaka 82fc80a987
MAPREDUCE-6999. Fix typo onf in DynamicInputChunk.java. Contributed by fang zhenyi. 
(cherry picked from commit cc3f3eca40)
 2017-11-02 18:33:25 +09:00
Arun Suresh 71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Chris Douglas ee89241282 HADOOP-14991. Add missing figures to Resource Estimator tool. Contributed by Rui Li 
(cherry picked from commit 139cc752d2)
(cherry picked from commit 77ad73501c)
 2017-10-27 22:01:41 -07:00
Steve Loughran 3fbcf76324 HADOOP-14553. Add (parallelized) integration tests to hadoop-azure. 
Contributed by Steve Loughran.

(backport JIRA HADOOP-14870)
 2017-10-27 12:55:12 +01:00
Subru Krishnan 9897538a96 HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru). 
(cherry picked from commit 625039ef20)
 2017-10-26 19:14:24 -07:00
Akira Ajisaka 13f1a7e0ed
Revert "HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru)." 
This reverts commit da7f989d86.
 2017-10-26 15:33:53 +09:00
Subru Krishnan da7f989d86 HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru). 
(cherry picked from commit 625039ef20)
 2017-10-25 17:05:35 -07:00
Steve Loughran 062d9c9bab HADOOP-14942. DistCp#cleanup() should check whether jobFS is null. 
Contributed by Andras Bokor.

(cherry picked from commit f36cbc8475)
 2017-10-20 22:27:59 +01:00
Wangda Tan d894f910d0 YARN-6608. Backport all SLS improvements from trunk to branch-2. (Carlo Curino via wangda) 2017-10-17 19:45:09 -07:00
Steve Loughran 8902178d3e HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled. 
Contributed by Santhosh G Nayak

Cherry-picked from commit 9fcc3a1fc8
 2017-10-16 16:06:41 +01:00
Steve Loughran ccfd1a4a56 HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB. 
Contributed by Varada Hemeswari.

Cherry-picked from commit ref 3de574413c
 2017-10-11 18:09:13 +01:00
Steve Loughran 34040e8d73 HADOOP-14899. Restrict Access to setPermission operation when authorization is enabled in WASB 
Contributed by Kannapiran Srinivasan.
 2017-10-07 11:20:18 +01:00
Steve Loughran fa6b8feb9e
HADOOP-14845. Azure wasb: getFileStatus not making any auth check. 
Final fixup of TestNativeAzureFileSystemAuthorization in branch-2 to match trunk's changes
Contributed by Sivaguru Sankaridurg
 2017-10-05 16:03:13 +01:00
Steve Loughran 28e05dc1da
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. 
Contributed by Steve Loughran
 2017-10-03 20:06:45 +01:00
Steve Loughran 1eecf8a97c
HADOOP-14768. Honoring sticky bit during Deletion when authorization is enabled in WASB 
Contributed by Varada Hemeswari
 2017-10-02 14:28:59 +01:00