3fbf4cd5da
HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran.
2016-10-20 12:33:58 -07:00
9ae270af02
HADOOP-13735 ITestS3AFileContextStatistics.testStatistics() failing. Contributed by Pieter Reuse
2016-10-20 14:50:59 +01:00
8650cc84f2
HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang.
2016-10-19 17:37:54 -07:00
efdf810cf9
HADOOP-7352. FileSystem#listStatus should throw IOE upon access error. Contributed by John Zhuge.
2016-10-18 18:18:43 -07:00
6c348c5691
HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran
2016-10-18 21:16:02 +01:00
0bc6d37f3c
Revert "HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang."
...
This reverts commit 412c4c9a34
2016-10-17 22:47:37 -07:00
987ee51141
HADOOP-13724. Fix a few typos in site markdown documents. Contributed by Ding Fei.
2016-10-17 13:25:58 -07:00
412c4c9a34
HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang.
2016-10-17 11:04:42 -07:00
d9f73f1b7c
HADOOP-13723. AliyunOSSInputStream#read() should update read bytes stat correctly. Contributed by Mingliang Liu
2016-10-13 22:33:55 -07:00
0a85d07983
HADOOP-13024. Distcp with -delete feature on raw data not implemented. Contributed by Mavin Martin.
2016-10-13 13:24:54 -07:00
c31b5e61b1
HADOOP-13701. AbstractContractRootDirectoryTest can fail when handling delete "/". Contributed by Genmao Yu
2016-10-08 17:35:59 +06:00
a57bba470b
Merge branch 'trunk' into HADOOP-12756
2016-10-09 10:29:40 +08:00
69620f9559
HADOOP-13692. hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts. Contributed by Chris Nauroth.
2016-10-07 11:41:19 -07:00
ebd4f39a39
HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran.
2016-10-07 12:51:40 +01:00
2cc841f16e
HADOOP-13678 Update jackson from 1.9.13 to 2.x in hadoop-tools. Contributed by Akira Ajisaka.
2016-10-06 16:31:00 +01:00
d6be1e75d8
HADOOP-12667 s3a to support createNonRecursive API. Contributed by Sean Mackrory
2016-10-05 15:02:11 +01:00
6cfe649375
HADOOP-13681. Reduce Kafka dependencies in hadoop-kafka module. Contributed by Grant Henke.
...
This closes #133
2016-10-05 20:44:39 +09:00
88b9444a81
HADOOP-13674. S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint. Contributed by Chris Nauroth.
2016-10-04 10:36:58 -07:00
ee0c722dc8
HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan.
2016-09-29 17:01:00 +01:00
47f80922dc
HADOOP-13599. s3a close() to be non-synchronized, so avoid risk of deadlock on shutdown. Contributed by Steve Loughran.
2016-09-28 16:19:32 -07:00
d1443988f8
Revert "HADOOP-13584. hdoop-aliyun: merge HADOOP-12756 branch back"
...
This reverts commit 5707f88d85
2016-09-29 09:18:27 +08:00
e17a4970be
HDFS-9885. Correct the distcp counters name while displaying counters. Contributed by Surendra Singh Lilhore
2016-09-27 10:45:12 +05:30
5707f88d85
HADOOP-13584. hdoop-aliyun: merge HADOOP-12756 branch back.
...
HADOOP-12756 branch: Incorporate Aliyun OSS file system implementation. Contributors:
Mingfei Shi (mingfei.shi@intel.com )
Genmao Yu (genmao.ygm@alibaba-inc.com )
2016-09-26 20:42:22 +08:00
e5ef51e717
HADOOP-13643. Math error in AbstractContractDistCpTest. Contributed by Aaron Fabbri.
2016-09-23 10:01:30 +01:00
26d5df390c
HADOOP-13634. Some configuration in doc has been outdated. Contributed by Genmao Yu
2016-09-23 08:44:28 +08:00
a49b3be38e
Merge branch 'trunk' into HADOOP-12756
2016-09-21 14:03:21 +08:00
22af6f8db3
HADOOP-13624. Rename TestAliyunOSSContractDispCp. Contributed by Genmao Yu
2016-09-21 14:02:44 +08:00
08b37603d9
HADOOP-13591. Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir. Contributed by Genmao Yu
2016-09-20 15:12:02 +08:00
9cd4760257
HADOOP-13609. Refine credential provider related codes for AliyunOss integration. Contributed by Genmao Yu
2016-09-20 11:39:02 +08:00
005f4528c7
Merge branch 'trunk' into HADOOP-12756
2016-09-20 08:09:29 +08:00
98bdb51397
HADOOP-13169. Randomize file list in SimpleCopyListing. Contributed by Rajesh Balamohan.
2016-09-19 15:16:47 -07:00
a1940464a4
HADOOP-13610. Clean up AliyunOss integration tests. Contributed by Genmao Yu
2016-09-18 19:10:48 +08:00
dbb28eb1f2
Merge branch 'trunk' into HADOOP-12756
2016-09-18 19:03:39 +08:00
96142efa2d
HADOOP-13621. s3:// should have been fully cut off from trunk. Contributed by Mingliang Liu.
2016-09-17 22:07:46 -07:00
53a12fa721
HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran
2016-09-14 15:45:04 +01:00
60f66a9306
Merge branch 'trunk' into HADOOP-12756
2016-09-14 16:12:40 +08:00
58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
9faccd1046
HADOOP-13587. distcp.map.bandwidth.mb is overwritten even when -bandwidth flag isn't set. Contributed by Zoran Dimitrijevic
2016-09-12 08:26:08 -07:00
4b6d795f28
HADOOP-13540 improve section on troubleshooting s3a auth problems. Contributed by Steve Loughran
2016-09-09 18:55:32 +01:00
7fdfcd8a6c
HADOOP-13541 explicitly declare the Joda time version S3A depends on. Contributed by Stevel Loughran
2016-09-07 12:25:23 +01:00
e671a0f52b
HADOOP-13481. User documents for Aliyun OSS FileSystem. Contributed by Genmao Yu.
2016-09-07 11:17:43 +08:00
d33e928fbe
HADOOP-13529. Do some code refactoring. Contributed by Genmao Yu.
2016-09-07 11:15:48 +08:00
cdb77110e7
HADOOP-13498. The number of multi-part upload part should not bigger than 10000. Contributed by Genmao Yu.
2016-09-07 11:15:48 +08:00
6bb741b9f8
HADOOP-13499. Support session credentials for authenticating with Aliyun. Contributed by Genmao Yu.
2016-09-07 11:15:48 +08:00
bd2d97adee
HADOOP-13483. File create should throw error rather than overwrite directories. Contributed by Genmao Yu.
2016-09-07 11:15:47 +08:00
4d84c814fc
HADOOP-13491. Fix several warnings from findbugs. Contributed by Genmao Yu.
2016-09-07 11:15:47 +08:00
a5d5342228
HADOOP-12756. Incorporate Aliyun OSS file system implementation. Contributed by Mingfei Shi and Lin Zhou
2016-09-07 11:15:47 +08:00
d152557cf7
HADOOP-13447. Refactor S3AFileSystem to support introduction of separate metadata repository and tests. Contributed by Chris Nauroth.
2016-09-06 09:36:21 -07:00
d6d9cff21b
YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh)
2016-08-30 15:52:29 -07:00
8b57be108f
HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth.
...
(cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41)
(cherry picked from commit a65e159c59
2016-08-29 15:30:49 -07:00
Anu Engineer
Steve Loughran
Yongjun Zhang
Xiao Chen
Andrew Wang
Mingliang Liu
Jing Zhao
Kai Zheng
Chris Nauroth
Akira Ajisaka
Brahma Reddy Battula
Allen Wittenauer
Ravi Prakash
Mingfei
Arun Suresh
Xiaoyu Yao