Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,611 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

653 Commits

Author SHA1 Message Date
Xiaoyu Yao b669aca518 HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin. 2017-09-11 11:40:48 -07:00
Steve Loughran 87af3f4991
HADOOP-14520. WASB: Block compaction for Azure Block Blobs. 
Contributed by Georgi Chalakov
 2017-09-11 14:15:58 +01:00
John Zhuge 1421196d20 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1)
 2017-09-08 14:15:51 -07:00
Steve Loughran 6e20ae954c
HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter. 
Contributed by Yonger.

(cherry picked from commit 83449abb40)
 2017-09-07 19:58:40 +01:00
John Zhuge 4c34c03b27 HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge. 2017-09-06 01:03:18 -07:00
Steve Loughran f382f66255
HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS. 
Contributed by Sivaguru Sankaridurg
 2017-09-05 14:18:43 +01:00
John Zhuge 2442a8d716 HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria. 
(cherry picked from commit 7ecc6dbed6)
 2017-08-31 21:13:50 -07:00
Yufei Gu 626fa7c8e1 YARN-7077. TestAMSimulator and TestNMSimulator fail. (Contributed by Akira Ajisaka via Yufei Gu) 2017-08-30 23:08:02 -07:00
Steve Loughran 25e9c29608
HADOOP-14802. Add support for using container saskeys for all accesses. 
Contributed by Sivaguru Sankaridurg

(cherry picked from commit 021974f4cb)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2017-08-30 22:40:50 +01:00
Arun Suresh f8f193c8e8 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 8410d862d3)
 2017-08-20 08:31:17 -07:00
Steve Loughran c42e694ee5
HADOOP-14769. WASB: delete recursive should not fail if a file is deleted. 
Contributed by Thomas Marquardt

(cherry picked from commit c6b4e656b7)
 2017-08-18 14:38:04 +01:00
Steve Loughran d5680c08ed
HADOOP-14660. wasb: improve throughput by 34% when account limit exceeded. 
Contributed by Thomas Marquardt.
 2017-08-16 13:49:47 +01:00
John Zhuge 0efc590e6d HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 
(cherry picked from commit 7769e96149)
 2017-08-13 00:27:33 -07:00
Steve Loughran 96befde480
HADOOP-14183. Remove service loader config file for wasb fs. 
Contributed by Esfandiar Manii.

(cherry picked from commit 54356b1e83)
 2017-08-10 16:49:51 +01:00
Steve Loughran bbf8c4bacb
HADOOP-14715. TestWasbRemoteCallHelper failing. 
Contributed by Esfandiar Manii.
 2017-08-10 16:49:19 +01:00
Steve Loughran 7f7ab0302f
HADOOP-14722. Azure: BlockBlobInputStream position incorrect after seek. 
Contributed by Thomas Marquardt

(cherry picked from commit d91b7a8451)
 2017-08-06 20:30:40 +01:00
Steve Loughran ced547d5f0
HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws. 
Contributed by Steve Loughran
 2017-08-04 11:07:34 +01:00
John Zhuge ba94c68e0b HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge. 
(cherry picked from commit f735ad1b67)
 2017-07-28 09:51:32 -07:00
Jitendra Pandey 554c3cd21d HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt. 2017-07-25 16:35:40 -07:00
Jitendra Pandey ab95086ed1 HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov. 2017-07-25 08:31:35 -07:00
Jitendra Pandey 23b920cd7a HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg. 2017-07-19 11:37:36 -07:00
Jitendra Pandey b9f2effaf6 HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak. 2017-07-17 16:25:05 -07:00
Steve Loughran df69b13b47
HADOOP-14658. branch-2 compilation is broken in hadoop-azure 
Contributed by Sunil G.

Signed-off-by: Steve Loughran <stevel@apache.org>
 2017-07-13 18:25:38 +01:00
Steve Loughran 3dfd107070
HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb. 
Contributed by Varada Hemeswari
(fix for java 7 compilation)
 2017-07-12 11:07:22 +01:00
Steve Loughran 1e69e52603
HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb. 
Contributed by Varada Hemeswari
 2017-07-12 10:35:59 +01:00
Steve Loughran 41e83b2ca2
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs. 
Contributed by Thomas Marquardt

(cherry picked from commit d670c3a4da)
 2017-07-11 21:45:24 +01:00
Jitendra Pandey d94b30cb03 HADOOP-14443. Azure: Support retry and client side failover for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak. 2017-07-10 17:30:31 -07:00
Mingliang Liu 2d69952925 HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran 
Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc
(cherry picked from commit 72993b33b7)
 2017-06-29 17:08:51 -07:00
Mingliang Liu ba76897392 HADOOP-14609. NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code. Contributed by Steve Loughran 
(cherry picked from commit 990aa34de2)
 2017-06-28 14:20:44 -07:00
Steve Loughran 44d43a8848
HADOOP-14190. Add more on S3 regions to the s3a documentation. 
Contributed by Steve Loughran

(cherry picked from commit ee243e5289)
 2017-06-28 10:26:05 +01:00
Mingliang Liu ce57458ec5 HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas 
(cherry picked from commit 7e031c2c18)
 2017-06-27 17:32:56 -07:00
Mingliang Liu 1b1065a163 HADOOP-14573. regression: Azure tests which capture logs failing with move to SLF4J. Contributed by Steve Loughran 
(cherry picked from commit 686a634f01)
 2017-06-27 16:49:48 -07:00
Mingliang Liu ea402c364f HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu 
(cherry picked from commit bc4dfe9c9c)
 2017-06-27 13:20:03 -07:00
Steve Loughran 3a87591df8 HADOOP-14461 Azure: handle failure gracefully in case of missing account access key. Contributed by Mingliang Liu. 
(cherry picked from commit 48f4a229a9)
 2017-06-26 15:22:44 -07:00
Akira Ajisaka 239b382153
HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he. 
(cherry picked from commit a594f96fb1)
 2017-06-26 16:41:44 +09:00
Steve Loughran 093eb0d361
HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations. 
Contributed by Thomas.

(cherry picked from commit c6bd73c6c5)
 2017-06-23 21:40:05 +01:00
Akira Ajisaka 8a77cf1ee4
HADOOP-14296. Move logging APIs over to slf4j in hadoop-tools. 2017-06-20 13:21:12 +09:00
Naganarasimha ac058c75a4 YARN-6719. Fix findbugs warnings in SLSCapacityScheduler.java. Contributed by Akira Ajisaka. 2017-06-19 12:41:13 +05:30
Mingliang Liu c953e0ce8b HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu 
(cherry picked from commit 325163f23f)
 2017-06-09 13:07:35 -07:00
Mingliang Liu de4e4ec774 HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan 
(cherry picked from commit df35e4cc3b)
 2017-06-07 13:55:11 -07:00
Mingliang Liu e3a72c2b5c HADOOP-14491. Azure has messed doc structure. Contributed by Mingliang Liu 
(cherry picked from commit 536f057158)
 2017-06-06 11:13:42 -07:00
Mingliang Liu bacfd1baae HADOOP-14472. Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently. Contributed by Mingliang Liu 
(cherry picked from commit 6b5285bbcb)
 2017-06-06 11:07:42 -07:00
Christopher Douglas 95c2978640 YARN-1471. Preserve scheduler typeinfo in simulator to work with resource monitors. 
Contributed by Carlo Curino. Backport by Ye Zhou.

(cherry picked from commit e07f3c87cd)
 2017-06-05 16:13:51 -07:00
Mingliang Liu 53e90a8064 HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan 
(cherry picked from commit 5fd9742c83)
 2017-06-05 16:01:27 -07:00
Mingliang Liu 867026d94e HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu 
(cherry picked from commit 6aeda55bb8)
 2017-06-05 11:31:13 -07:00
Mingliang Liu 0b78db35fd HADOOP-14460. Azure: update doc for live and contract tests. Contributed by Mingliang Liu 
(cherry picked from commit ece33208b8)
 2017-06-01 11:52:37 -07:00
John Zhuge f72f194e24 HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge. 
(cherry picked from commit 6c6a7a5962)
 2017-05-28 22:29:03 -07:00
Mingliang Liu 9d7e67ccf9 HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari 
(cherry picked from commit 89bb8bfe58)
 2017-05-26 17:59:14 -07:00
Yongjun Zhang dd552a97b7 HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang) 2017-05-24 19:05:40 -07:00
Yongjun Zhang 49dc547252 HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li. 2017-05-24 18:54:51 -07:00