Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,522 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

814 Commits

Author SHA1 Message Date
Sammi Chen ce7ebbe2cc HADOOP-14999. AliyunOSS: provide one asynchronous multi-part based uploading mechanism. Contributed by Genmao Yu. 
(cherry picked from commit 6542d17ea4)
(cherry picked from commit e96c7bf82d)
 2018-04-13 10:17:46 +08:00
Lei Xu 3717df89ee Preparing for 3.0.3 development 2018-04-12 13:57:46 -07:00
Steve Loughran a3945e4cf1 HADOOP-14651. Update okhttp version to 2.7.5. Contributed by Ray Chiang and John Zhuge. 
(cherry picked from commit 8aca46e53f)
 2018-04-02 16:38:19 +01:00
Sean Mackrory ca92d1810b HADOOP-15342. Updating ADLS connector to use the current SDK version (2.2.7). Contributed by Atul Sikaria. 
(cherry picked from commit 9d7a9031a5)
 2018-03-29 14:05:42 -06:00
Sammi Chen 7985c5fdc9 HADOOP-15262. AliyunOSS: move files under a directory in parallel when rename a directory. Contributed by Jinhu Wu. 
(cherry picked from commit d67a5e2dec)
(cherry picked from commit 2285afb32e)
(cherry picked from commit 322520eb76)
 2018-03-19 15:44:15 +08:00
Chris Douglas a03c8ea61f HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 
(cherry picked from commit 45d1b0fdcc)
(cherry picked from commit 09940b1eb3)
 2018-03-12 20:06:22 -07:00
Steve Loughran 1771af2320 HADOOP-15273.distcp can't handle remote stores with different checksum algorithms. 
Contributed by Steve Loughran.

(cherry picked from commit 7ef4d942dd)
 2018-03-09 10:49:11 +00:00
Steve Loughran cc0f14c13c HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled. 
Contributed by Anis Elleuch.

(cherry picked from commit 1dedc68f9d)
 2018-03-07 19:20:53 +00:00
Gergely Novák 8b9a7b2e48
MAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e99a1e7767)
 2018-03-02 16:22:06 -08:00
Steve Loughran 3c03672e87 HADOOP-15263. hadoop cloud-storage module to mark hadoop-common as provided; add azure-datalake. 
Contributed by Steve Loughran

(cherry picked from commit edc9f1451b)
(cherry picked from commit abfe1eaf29)
 2018-02-28 15:33:21 +00:00
Steve Loughran c2bbe22c5a HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-21 11:11:28 +00:00
Steve Loughran b302f728f8 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.

(cherry picked from commit 050f5287b7)
 2018-02-15 16:29:30 +00:00
Steve Loughran 1888b94806 HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran.

(cherry picked from commit 58a2120e8a)
 2018-02-15 14:26:18 +00:00
Anu Engineer 2d6ab20007 HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 
(cherry picked from commit 4304fcd5bd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java

(cherry picked from commit 9264f10bb3)
 2018-02-07 11:16:53 -08:00
Lei Xu 7b377f43d0 Preparing for 3.0.2 development 2018-02-06 10:50:37 -08:00
Sammi Chen 91184299c5 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
 2018-01-30 15:21:27 +08:00
Steve Loughran 95a96b13e2 HADOOP-15186. Allow Azure Data Lake SDK dependency version to be set on the command line. 
Contributed by Vishwajeet Dusane.
 2018-01-29 09:46:53 -08:00
Steve Loughran ec4f5f0f0d HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu
 2018-01-26 15:23:08 -08:00
Jason Lowe 217f408aaa YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit ff8378eb1b)
 2018-01-25 17:51:51 -06:00
Steve Loughran 1de0712578 HADOOP-15185. Update adls connector to use the current version of ADLS SDK. 
Contributed by Atul Sikaria.

(cherry picked from commit f63d13f10d)
 2018-01-23 11:37:33 -08:00
Jason Lowe b1cc14e9cb Revert "HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)" 
This reverts commit 55142849db.
 2018-01-17 09:53:24 -06:00
Sammi Chen 55142849db HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
 2018-01-17 16:12:23 +08:00
Kai Zheng 539fa11325 HADOOP-15104. AliyunOSS: change the default value of max error retry. Contributed by Jinhu Wu 
(cherry picked from commit ce04340ec7)
 2018-01-09 15:41:12 +08:00
Steve Loughran 72a49503fe HADOOP-15086. NativeAzureFileSystem file rename is not atomic. 
Contributed by Thomas Marquardt

(cherry picked from commit 52babbb4a0)
 2017-12-22 14:31:20 +00:00
Steve Loughran 11f577f3e6 HADOOP-14965. S3a input stream "normal" fadvise mode to be adaptive 
(cherry picked from commit 1ba491ff90)
 2017-12-22 11:41:45 +00:00
Sammi Chen abaabb5dec HADOOP-15024. AliyunOSS: Support user agent configuration and include that & Hadoop version information to oss server. 
(cherry picked from commit c326fc89b0)
 2017-12-08 21:28:19 +08:00
Sammi Chen fb809e05dc HADOOP-14993. AliyunOSS: Override listFiles and listLocatedStatus. Contributed Genmao Yu. 
(cherry picked from commit 18621af7ae)
 2017-12-08 20:40:14 +08:00
John Zhuge 9942952dd8 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 
(cherry picked from commit bf6a660232)
 2017-12-07 21:06:25 -08:00
Steve Loughran 8bf74f325a HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. 
Contributed by Rajesh Balamohan

(cherry picked from commit 0ea182d0fa)
 2017-11-28 11:57:12 +00:00
Andrew Wang 1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Daniel Templeton 38f5d4cbba YARN-7413. Support resource type in SLS (Contributed by Yufei Gu via Daniel Templeton) 
Change-Id: Ic0a897c123c5d2f57aae757ca6bcf1dad7b90d2b
(cherry picked from commit ba8136615a)
 2017-11-09 13:39:08 -08:00
Yufei Gu ddef7d05a4 YARN-6505. Define the strings used in SLS JSON input file format. (Contributed by Gergely Novak) 
(cherry picked from commit 99880d0a16)
 2017-11-09 13:38:57 -08:00
Subru Krishnan 7441d14f1c HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru). 
(cherry picked from commit f2df6b8983)
 2017-11-08 18:08:09 -08:00
Arun Suresh cbd81f305b HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh) 
(cherry picked from commit 53c0fb7efe)
 2017-11-02 17:17:27 -07:00
Akira Ajisaka 2ba71afcd5
MAPREDUCE-6999. Fix typo onf in DynamicInputChunk.java. Contributed by fang zhenyi. 
(cherry picked from commit cc3f3eca40)
 2017-11-02 18:32:57 +09:00
Chris Douglas 77ad73501c HADOOP-14991. Add missing figures to Resource Estimator tool. Contributed by Rui Li 
(cherry picked from commit 139cc752d2)
 2017-10-27 22:00:44 -07:00
Subru Krishnan 82c9b3bbb5 HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru). 
(cherry picked from commit 625039ef20)
 2017-10-25 16:01:13 -07:00
Wangda Tan 881263260c YARN-7318. Fix shell check warnings of SLS. (Gergely Novák via wangda) 
(cherry picked from commit 281d83604d)
 2017-10-20 16:02:04 -07:00
Steve Loughran 2bacb429a1 HADOOP-14942. DistCp#cleanup() should check whether jobFS is null. 
Contributed by Andras Bokor.

(cherry picked from commit f36cbc8475)
 2017-10-20 22:29:22 +01:00
Haibo Chen 175c4472d5 YARN-6546. SLS is slow while loading 10k queues. (Yufei Gu via Haibo Chen) 
(cherry picked from commit 46eb1033a8)
 2017-10-17 16:04:49 -07:00
Haibo Chen c021c8a19f YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen) 
(cherry picked from commit c071aad5da)
 2017-10-05 15:51:08 -07:00
Yufei Gu f982ffe336 YARN-7211. AMSimulator in SLS does't work due to responseId refactoring.(Contributed by Botong Huang via Yufei Gu) 
(cherry picked from commit 79e37dc6db)
 2017-10-03 15:12:09 -07:00
Akira Ajisaka b658ae002f
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari. 
(cherry picked from commit c9b525bd94)
 2017-10-04 07:02:31 +09:00
Andrew Wang 5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Aaron Fabbri 60b966cb45
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran 2017-09-25 16:01:15 -07:00
Jason Lowe 0eb0298027 MAPREDUCE-6967. gridmix/SleepReducer should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari 
(cherry picked from commit b133dc5700)
 2017-09-22 15:30:20 -05:00
Steve Loughran 9f6b08f840
HADOOP-14553. Add (parallelized) integration tests to hadoop-azure 
Contributed by Steve Loughran

(cherry picked from commit 2d2d97fa7d)
 2017-09-15 17:04:43 +01:00
Aaron Fabbri 62e8a5ceed
HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran. 2017-09-14 16:12:29 -07:00
Kai Zheng b873b1395e HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen 2017-09-14 09:22:00 +08:00
John Zhuge bb107f5d28 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1)
 2017-09-08 12:57:58 -07:00