Commit Graph

883 Commits

Author SHA1 Message Date
Wangda Tan 281d83604d YARN-7318. Fix shell check warnings of SLS. (Gergely Novák via wangda) 2017-10-20 15:54:15 -07:00
Steve Loughran f36cbc8475 HADOOP-14942. DistCp#cleanup() should check whether jobFS is null.
Contributed by Andras Bokor.
2017-10-20 22:27:04 +01:00
Haibo Chen 46eb1033a8 YARN-6546. SLS is slow while loading 10k queues. (Yufei Gu via Haibo Chen) 2017-10-17 16:04:19 -07:00
Steve Loughran 9fcc3a1fc8 HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled.
Contributed by Santhosh G Nayak
2017-10-16 16:01:47 +01:00
Steve Loughran 3de574413c HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB.
Contributed by Varada Hemeswari.
2017-10-11 18:06:43 +01:00
Steve Loughran 572cdb5463 HADOOP-14899. Restrict Access to setPermission operation when authorization is enabled in WASB
Contributed by Kannapiran Srinivasan.
2017-10-06 17:43:38 +01:00
Haibo Chen c071aad5da YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen) 2017-10-05 15:50:13 -07:00
Steve Loughran 9288206cb3
HADOOP-14845. Azure wasb: getFileStatus not making any auth check.
Contributed by Sivaguru Sankaridurg
2017-10-05 15:05:23 +01:00
Yufei Gu 79e37dc6db YARN-7211. AMSimulator in SLS does't work due to responseId refactoring.(Contributed by Botong Huang via Yufei Gu) 2017-10-03 15:10:50 -07:00
Akira Ajisaka c9b525bd94
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari. 2017-10-04 07:01:54 +09:00
John Zhuge 8aca46e53f HADOOP-14651. Update okhttp version to 2.7.5. Contributed by Ray Chiang and John Zhuge. 2017-09-29 12:15:06 -07:00
Steve Loughran a530e7ab3b
HADOOP-14768. Honoring sticky bit during Deletion when authorization is enabled in WASB
Contributed by Varada Hemeswari
2017-09-28 19:52:56 +01:00
Aaron Fabbri 47011d7dd3
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran 2017-09-25 15:59:38 -07:00
Jason Lowe b133dc5700 MAPREDUCE-6967. gridmix/SleepReducer should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari 2017-09-22 15:21:42 -05:00
Steve Loughran 2d2d97fa7d
HADOOP-14553. Add (parallelized) integration tests to hadoop-azure
Contributed by Steve Loughran
2017-09-15 17:03:01 +01:00
Aaron Fabbri 49467165a5
HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran. 2017-09-14 14:10:48 -07:00
ChenSammi e0b3c644e1 HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen 2017-09-14 09:15:29 +08:00
John Zhuge a4661850c1 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 2017-09-08 11:51:03 -07:00
Steve Loughran 5bbca80428
HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A.
Contributed by Aaron Fabbri
2017-09-08 12:07:02 +01:00
Steve Loughran 83449abb40
HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter.
Contributed by Yonger.
2017-09-07 19:55:56 +01:00
Steve Loughran 13eda50003
HADOOP-14520. WASB: Block compaction for Azure Block Blobs.
Contributed by Georgi Chalakov
2017-09-07 18:35:03 +01:00
Xiaoyu Yao 63720ef574 HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin. 2017-09-05 23:34:55 -07:00
John Zhuge 50506e90a8 HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge. 2017-09-05 23:26:57 -07:00
Steve Loughran 792eff9ea7
HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS.
Contributed by Sivaguru Sankaridurg
2017-09-05 14:16:57 +01:00
Andrew Wang 0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Allen Wittenauer a3fee475f7
HADOOP-14414. Calling maven-site-plugin directly for docs profile is unnecessary (Andras Bokor via aw) 2017-09-01 08:38:30 -07:00
Steve Loughran 621b43e254
HADOOP-13345 HS3Guard: Improved Consistency for S3A.
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
2017-09-01 14:13:41 +01:00
John Zhuge 7ecc6dbed6 HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria. 2017-08-31 21:13:22 -07:00
Allen Wittenauer 71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms
Signed-off-by: Chris Douglas <cdouglas@apache.org>
2017-08-30 21:47:24 -07:00
Haibo Chen d04f85f387 MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen) 2017-08-30 10:07:48 -07:00
Yufei Gu 26fafc3597 YARN-7077. TestAMSimulator and TestNMSimulator fail. (Contributed by Akira Ajisaka via Yufei Gu) 2017-08-29 16:33:51 -07:00
Steve Loughran 9374f38820
HADOOP-14583. wasb throws an exception if you try to create a file and there's no parent directory
Contributed by Esfandiar Manii and Thomas Marquardt.
2017-08-29 19:03:44 +01:00
Steve Loughran 021974f4cb
HADOOP-14802. Add support for using container saskeys for all accesses.
Contributed by Sivaguru Sankaridurg
2017-08-29 19:02:43 +01:00
Andrew Wang f29a0fc288 HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou. 2017-08-25 14:14:23 -07:00
Arpit Agarwal 8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Steve Loughran 27ab5f7385
HADOOP-14787. AliyunOSS: Implement the `createNonRecursive` operator.
Contributed by Genmao Yu
2017-08-22 11:55:48 +01:00
Kai Zheng 267e19a09f HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu 2017-08-21 13:36:28 +08:00
Arun Suresh 8410d862d3 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 2017-08-20 07:54:09 -07:00
Steve Loughran c6b4e656b7
HADOOP-14769. WASB: delete recursive should not fail if a file is deleted.
Contributed by Thomas Marquardt
2017-08-18 14:13:40 +01:00
Andrew Wang dd7916d3cd HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas. 2017-08-17 09:35:36 -07:00
Chris Douglas 645a8f2a4d HADOOP-14726. Mark FileStatus::isDir as final 2017-08-14 21:57:20 -07:00
John Zhuge 7769e96149 HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 2017-08-13 00:22:34 -07:00
Allen Wittenauer 4222c97108
HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw) 2017-08-11 09:25:56 -07:00
Steve Loughran 54356b1e83
HADOOP-14183. Remove service loader config file for wasb fs.
Contributed by Esfandiar Manii.
2017-08-10 16:46:33 +01:00
Steve Loughran f4e1aa0508
HADOOP-14715. TestWasbRemoteCallHelper failing.
Contributed by Esfandiar Manii.
2017-08-08 23:37:47 +01:00
Chris Douglas 55a181f845 HADOOP-14730. Support protobuf FileStatus in AdlFileSystem. 2017-08-07 21:31:28 -07:00
Steve Loughran d91b7a8451
HADOOP-14722. Azure: BlockBlobInputStream position incorrect after seek.
Contributed by Thomas Marquardt
2017-08-06 20:19:23 +01:00
Steve Loughran 7fc324aabd
HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws.
Contributed by Steve Loughran

(cherry picked from commit ced547d5f0)
2017-08-04 11:09:08 +01:00
Sean Mackrory 1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Steve Loughran 778d4edd9a
HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded.
Contributed by Thomas Marquardt.
2017-08-01 21:36:28 +01:00
John Zhuge f735ad1b67 HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge. 2017-07-28 09:46:54 -07:00
Jitendra Pandey a92bf39e23 HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt. 2017-07-25 16:26:48 -07:00
Jitendra Pandey f2921e51f0 HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov. 2017-07-24 23:01:01 -07:00
Ravi Prakash 94ca52ae9e HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash. 2017-07-24 16:01:45 -07:00
Wei-Chiu Chuang 44350fdf49 HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default). Contributed by Bharat Viswanadham. 2017-07-20 18:23:13 -07:00
Haibo Chen 0ba8cda135 YARN-6685. Add job count in to SLS JSON input format. (Yufei Gu via Haibo Chen) 2017-07-20 08:15:46 -07:00
Jitendra Pandey 2843c688bc HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg. 2017-07-19 00:13:06 -07:00
Jitendra Pandey b0e78ae085 HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak. 2017-07-17 02:27:55 -07:00
Steve Loughran 7d272ea124
HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb.
Contributed by Varada Hemeswari

(cherry picked from commit 1e69e52603)
2017-07-12 10:37:39 +01:00
Steve Loughran d670c3a4da
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs.
Contributed by Thomas Marquardt
2017-07-11 21:34:27 +01:00
Haibo Chen 147df300bf YARN-5067 Support specifying resources for AM containers in SLS. (Yufei Gu via Haibo Chen) 2017-06-30 17:03:44 -07:00
Mingliang Liu 38996fdcf0 HADOOP-14443. Azure: Support retry and client side failover for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak 2017-06-30 16:53:48 -07:00
Andrew Wang af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Mingliang Liu 72993b33b7 HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran
Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc
2017-06-29 17:07:52 -07:00
Mingliang Liu b08cc97396 HADOOP-14601. Azure: Reuse ObjectMapper. Contributed by Mingliang Liu 2017-06-29 09:22:00 -07:00
Allen Wittenauer 20ba86d66a MAPREDUCE-6536. hadoop-pipes doesn't use maven properties for openssl
Signed-off-by: Ravi Prakash <raviprak@apache.org>
2017-06-28 17:33:44 -07:00
Mingliang Liu 990aa34de2 HADOOP-14609. NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code. Contributed by Steve Loughran 2017-06-28 14:18:59 -07:00
Steve Loughran ee243e5289
HADOOP-14190. Add more on S3 regions to the s3a documentation.
Contributed by Steve Loughran
2017-06-28 10:22:13 +01:00
Mingliang Liu 7e031c2c18 HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas 2017-06-27 17:32:07 -07:00
Mingliang Liu 686a634f01 HADOOP-14573. regression: Azure tests which capture logs failing with move to SLF4J. Contributed by Steve Loughran 2017-06-27 16:48:47 -07:00
Mingliang Liu bc4dfe9c9c HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu 2017-06-27 13:19:14 -07:00
Steve Loughran 48f4a229a9
HADOOP-14461 Azure: handle failure gracefully in case of missing account access key.
Contributed by Mingliang Liu.
2017-06-26 13:39:47 +01:00
Akira Ajisaka a594f96fb1
HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he. 2017-06-26 16:41:00 +09:00
Steve Loughran c6bd73c6c5
HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations.
Contributed by Thomas.
2017-06-23 21:38:41 +01:00
Jason Lowe dd65eea74b HADOOP-8143. Change distcp to have -pb on by default. Contributed by Mithun Radhakrishnan 2017-06-20 09:53:47 -05:00
Akira Ajisaka 3369540653
HADOOP-14296. Move logging APIs over to slf4j in hadoop-tools. 2017-06-20 13:18:26 +09:00
Mingliang Liu f214a9961f HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran 2017-06-15 11:16:16 -07:00
Mingliang Liu 325163f23f HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu 2017-06-09 13:07:11 -07:00
Mingliang Liu df35e4cc3b HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan 2017-06-07 13:54:52 -07:00
Carlo Curino b65100c14b YARN-6547. Enhance SLS-based tests leveraging invariant checker. 2017-06-06 17:19:15 -07:00
Mingliang Liu 536f057158 HADOOP-14491. Azure has messed doc structure. Contributed by Mingliang Liu 2017-06-06 11:09:28 -07:00
Mingliang Liu 6b5285bbcb HADOOP-14472. Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently. Contributed by Mingliang Liu 2017-06-06 11:06:49 -07:00
Mingliang Liu 5fd9742c83 HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan 2017-06-05 15:56:43 -07:00
Mingliang Liu 6aeda55bb8 HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu 2017-06-05 11:26:56 -07:00
Mingliang Liu ece33208b8 HADOOP-14460. Azure: update doc for live and contract tests. Contributed by Mingliang Liu 2017-06-01 11:52:11 -07:00
Akira Ajisaka 6e6e96583f
HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang. 2017-06-01 13:08:01 +09:00
Akira Ajisaka 07e60f85d8
HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu. 2017-05-30 15:11:10 +09:00
John Zhuge 6c6a7a5962 HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge. 2017-05-28 22:25:00 -07:00
Mingliang Liu 89bb8bfe58 HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari 2017-05-26 17:54:00 -07:00
Andrew Wang 16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Akira Ajisaka 6a52b5e144
HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu. 2017-05-25 15:36:44 +09:00
Sunil G b6f66b0da1 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-22 14:10:06 +05:30
Yufei Gu fcbdecc54d YARN-6111. Rumen input does't work in SLS. Contributed by Yufei Gu. 2017-05-21 21:04:06 -07:00
Mingliang Liu 6672810eea HADOOP-14432. S3A copyFromLocalFile to be robust, tested. Contributed by Steve Loughran 2017-05-19 11:51:43 -07:00
Yongjun Zhang b4adc8392c HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang) 2017-05-18 15:35:22 -07:00
Sunil G 40e6a85d25 YARN-6560. SLS doesn't honor node total resource specified in sls-runner.xml. Contributed by Wangda Tan. 2017-05-18 22:49:42 +05:30
Steve Loughran ba70225cf6
HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries.
Contributed by Steve Loughran.

(cherry picked from commit 2ac5aab8d7)
2017-05-18 15:44:39 +01:00
Akira Ajisaka d4aa9e3c69
HADOOP-14415. Use java.lang.AssertionError instead of junit.framework.AssertionFailedError. Contributed by Chen Liang. 2017-05-17 15:38:28 -04:00
Robert Kanter 101852ca11 YARN-6535. Program needs to exit when SLS finishes. (yufeigu via rkanter) 2017-05-16 17:52:17 -07:00
Mingliang Liu b415c6fe74 HADOOP-14416. Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger. Contributed by Sivaguru Sankaridurg 2017-05-16 11:22:32 -07:00
Akira Ajisaka 0d5c8ed8e0
HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor. 2017-05-11 16:37:32 -05:00
Mingliang Liu eed731496f HADOOP-14361. Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently. Contributed by Santhosh G Nayak 2017-05-10 15:05:11 -07:00
Carlo Curino 5cb6e3e082 YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations. 2017-05-10 14:49:25 -07:00
Akira Ajisaka 2ba9903932
HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang. 2017-05-10 05:57:12 -05:00
Akira Ajisaka c60164fb60
HADOOP-14374. License error in GridmixTestUtils.java. Contributed by lixinglong. 2017-05-09 05:37:43 -05:00
Andrew Wang 54fd0e44b7 HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy. 2017-05-08 21:59:49 -07:00
Andrew Wang 8065129d87 HADOOP-14298. TestHadoopArchiveLogsRunner fails. Contribute dby Akira Ajisaka. 2017-05-07 13:59:15 -07:00
Steve Loughran 4e6bbd049d
HADOOP-14382 Remove usages of MoreObjects.toStringHelper.
Contributed by Andrew Wang
2017-05-05 13:03:09 +01:00
Robert Kanter 3082552b3b YARN-6522. Make SLS JSON input file format simple and scalable (yufeigu via rkanter) 2017-05-04 17:21:46 -07:00
Mingliang Liu 83dded556d HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran 2017-05-03 11:22:44 -07:00
Andrew Wang b062b323b7 HADOOP-14281. Fix TestKafkaMetrics#testPutMetrics. Contributed by Alison Yu. 2017-05-02 11:49:19 -07:00
Akira Ajisaka 62579b69a0
HADOOP-11614. Remove httpclient dependency from hadoop-openstack. Contributed by Akira Ajisaka, Brahma Reddy Battula, and Steve Loughran. 2017-04-27 16:57:08 +09:00
Mingliang Liu 8b5f2c372e HADOOP-14351. Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache. Contributed by Santhosh G Nayak 2017-04-26 13:47:18 -07:00
Mingliang Liu e4c32bccde HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu 2017-04-25 20:54:56 -07:00
Robert Kanter 475f933b41 YARN-6423. Queue metrics doesn't work for Fair Scheduler in SLS (yufeigu via rkanter) 2017-04-25 16:26:09 -07:00
Steve Loughran 5f934f8386
HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus.
Contributed by Steve Moist.
2017-04-24 20:33:19 +01:00
Wangda Tan de69d6e811 YARN-6363. Extending SLS: Synthetic Load Generator. (Carlo Curino via wangda) 2017-04-20 21:54:30 -07:00
Mingliang Liu 667966c13c HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran 2017-04-20 17:13:36 -07:00
Andrew Wang 7e075a50e3 YARN-6365. Get static SLS html resources from classpath. Contributed by Yufei Gu. 2017-04-19 11:08:59 -07:00
Steve Loughran 6b015d00c9
HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs.
Contributed by Steve Loughran
2017-04-19 10:21:44 +01:00
Mingliang Liu 0cab57223e HADOOP-14274. Azure: Simplify Ranger-WASB policy model. Contributed by Sivaguru Sankaridurg 2017-04-12 16:07:10 -07:00
Mingliang Liu b053fdc547 HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu 2017-04-12 14:30:34 -07:00
Chris Nauroth b8305e6d06 HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. Contributed by Mingliang Liu. 2017-04-12 10:02:13 -07:00
John Zhuge 56e81f2a20 HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge.
Signed-off-by: John Zhuge <jzhuge@apache.org>
2017-04-07 12:04:44 -07:00
Andrew Wang 7eea8d83fa HADOOP-14280. Fix compilation of TestKafkaMetrics. 2017-04-05 15:12:03 -07:00
Mingliang Liu 56ab02eed9 HADOOP-14273. Azure: NativeAzureFileSystem should respect config for kerberosSupportEnabled flag. Contributed by Santhosh G Nayak 2017-04-04 11:10:28 -07:00
Mingliang Liu 27a44b60c1 HADOOP-14272. Azure: WasbRemoteCallHelper should use String equals for comparison. Contributed by Santhosh G Nayak 2017-04-04 11:03:59 -07:00
Chris Douglas 6eba79232f HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang 2017-04-03 20:13:14 -07:00
Mingliang Liu 5faa949b78 HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka 2017-04-03 11:07:14 -07:00
Mingliang Liu 26172a94d6 HADOOP-14267. Make DistCpOptions immutable. Contributed by Mingliang Liu 2017-03-31 20:04:26 -07:00
Yongjun Zhang bf3fb585aa HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li. 2017-03-30 17:38:56 -07:00
Yongjun Zhang 144f1cf765 Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen."
This reverts commit 064c8b25ec.
2017-03-30 17:38:18 -07:00
Yongjun Zhang 064c8b25ec HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen. 2017-03-30 17:01:15 -07:00
Wei-Chiu Chuang 8c591b8d19 HDFS-10974. Document replication factor for EC files. Contributed by Yiqun Lin. 2017-03-30 11:16:05 -07:00
Akira Ajisaka 0d053eeb30
HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula. 2017-03-30 18:11:50 +09:00
Robert Kanter 6a5516c238 YARN-5654. Not be able to run SLS with FairScheduler (yufeigu via rkanter) 2017-03-29 16:18:13 -07:00
Andrew Wang 4966a6e26e HADOOP-14223. Extend FileStatus#toString() to include details like Erasure Coding and Encryption. Contributed by Manoj Govindassamy. 2017-03-29 14:37:21 -07:00
Andrew Wang 0e6f8e4bc6 HDFS-10971. Distcp should not copy replication factor if source file is erasure coded. Contributed by Manoj Govindassamy. 2017-03-28 22:14:03 -07:00
John Zhuge 253e3e78ab HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge. 2017-03-27 23:05:57 -07:00
John Zhuge d1b7439b48 HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge. 2017-03-24 13:37:04 -07:00
Andrew Wang 52b00600df HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy. 2017-03-24 11:44:46 -07:00
Chris Douglas c55195588f HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge 2017-03-23 12:17:27 -07:00
Chris Douglas 03955612b7 HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria 2017-03-23 12:13:23 -07:00
Mingliang Liu 2e30aa72e0 HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu 2017-03-23 11:33:29 -07:00
Steve Loughran a5a4867f3b
HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject().
Contributed by Yuanbo Liu

(cherry picked from commit 20878d052c)
2017-03-23 12:54:54 +00:00
Robert Kanter cc938e99ec YARN-6326. Shouldn't use AppAttemptIds to fetch applications while AM Simulator tracks app in SLS (yufeigu via rkanter) 2017-03-21 15:21:11 -07:00
Steve Loughran 2841666f1f
HADOOP-14204 S3A multipart commit failing,
"UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".

Contributed by Steve Loughran

(cherry picked from commit 595501c9dd)
2017-03-21 13:15:15 +00:00
Mingliang Liu 8e15e24059 HADOOP-13945. Azure: Add Kerberos and Delegation token support to WASB client. Contributed by Santhosh G Nayak
Change-Id: I39fb0a3b0491bf2160571366939a0502a0045429
2017-03-20 16:31:24 -07:00
Arpit Agarwal 6c399a88e9 HADOOP-14059. typo in s3a rename(self, subdir) error message. Contributed by Steve Loughran. 2017-03-20 11:29:23 -07:00
Mingliang Liu 86035c1644 HADOOP-14192. Aliyun OSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu 2017-03-17 11:01:00 -07:00