Jonathan Hung
|
108127238b
|
Preparing for 2.11.0 development
|
2019-10-15 10:01:45 -07:00 |
Takanobu Asanuma
|
e3d4f6bfed
|
HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka.
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
|
2019-05-29 17:41:21 +09:00 |
Akira Ajisaka
|
cea1b0000c
|
HADOOP-16323. https everywhere in Maven settings.
|
2019-05-27 15:28:54 +09:00 |
Masatake Iwasaki
|
50e30bd30a
|
HADOOP-14544. DistCp documentation for command line options is misaligned. Contributed by Masatake Iwasaki.
(cherry picked from commit bbdbc7a9a1 )
Conflicts:
hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
(cherry picked from commit 7985d9b1ce )
Conflicts:
hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
|
2019-04-12 12:35:32 +09:00 |
Kai Xie
|
a49cb4465e
|
HADOOP-16018. DistCp won't reassemble chunks when blocks per chunk > 0.
Contributed by Kai Xie.
|
2019-02-27 20:09:27 +00:00 |
Kai Xie
|
6d3e7a8570
|
HADOOP-16049. DistCp result has data and checksum mismatch when blocks per chunk > 0.
Contributed by Kai Xie.
|
2019-01-27 16:56:57 +00:00 |
Wei-Chiu Chuang
|
f2739f3f59
|
HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e )
(cherry picked from commit a7dd244a49 )
modified: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
(cherry picked from commit 84726679f0 )
(cherry picked from commit 8360d15abc )
|
2018-10-24 09:39:31 -07:00 |
Junping Du
|
2b48854cfd
|
HADOOP-15385. Many tests are failing in hadoop-distcp project in branch-2. Contributed by Jason Lowe.
|
2018-04-25 10:11:41 +08:00 |
Surendra Singh Lilhore
|
8d69b7f2d4
|
HDFS-12833. Distcp : Update the usage of delete option for dependency with update and overwrite option. Contributed by usharani.
|
2017-12-13 10:29:19 +05:30 |
Akira Ajisaka
|
82fc80a987
|
MAPREDUCE-6999. Fix typo onf in DynamicInputChunk.java. Contributed by fang zhenyi.
(cherry picked from commit cc3f3eca40 )
|
2017-11-02 18:33:25 +09:00 |
Arun Suresh
|
71a55bebfb
|
Preparing for 2.10.0 development
|
2017-10-31 12:51:14 -07:00 |
Steve Loughran
|
062d9c9bab
|
HADOOP-14942. DistCp#cleanup() should check whether jobFS is null.
Contributed by Andras Bokor.
(cherry picked from commit f36cbc8475 )
|
2017-10-20 22:27:59 +01:00 |
Xiaoyu Yao
|
b669aca518
|
HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin.
|
2017-09-11 11:40:48 -07:00 |
Yongjun Zhang
|
dd552a97b7
|
HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang)
|
2017-05-24 19:05:40 -07:00 |
Yongjun Zhang
|
49dc547252
|
HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li.
|
2017-05-24 18:54:51 -07:00 |
Yongjun Zhang
|
97317b1aaa
|
Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li."
This reverts commit c411adac32 .
|
2017-05-24 18:54:51 -07:00 |
Yongjun Zhang
|
39474ed50d
|
Revert "HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang)"
This reverts commit fe185e2c3a .
|
2017-05-24 18:54:50 -07:00 |
Sunil G
|
ec346d6377
|
YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.
|
2017-05-23 07:57:36 +05:30 |
Yongjun Zhang
|
fe185e2c3a
|
HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang)
|
2017-05-19 21:11:38 -07:00 |
Yongjun Zhang
|
c411adac32
|
HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li.
|
2017-04-14 10:14:02 -07:00 |
Chris Douglas
|
72aa0a7ab3
|
HADOOP-13626. Remove distcp dependency on FileStatus serialization
(cherry picked from commit a1a0281e12 )
|
2017-04-05 15:47:46 -07:00 |
Yongjun Zhang
|
16ca70553c
|
HADOOP-14127. Add log4j configuration to enable logging in hadoop-distcp's tests. (Xiao Chen via Yongjun Zhang)
(cherry picked from commit d235dcdf0b )
|
2017-02-27 20:46:17 -08:00 |
Steve Loughran
|
b2704702f9
|
HADOOP-13496. Include file lengths in Mismatch in length error for distcp. Contributed by Ted Yu
(cherry picked from commit 77401bd5fc )
|
2017-01-19 11:25:20 +00:00 |
Chris Nauroth
|
9c61858ee7
|
HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore.
(cherry picked from commit 4c8f9e1302 )
|
2017-01-05 15:05:01 -08:00 |
Mingliang Liu
|
9af7d1e298
|
HADOOP-13655. document object store use with fs shell and distcp. Contributed by Steve Loughran
|
2016-11-22 13:19:32 -08:00 |
Mingliang Liu
|
4e0fcff8ab
|
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
(cherry picked from commit 5af572b644 )
|
2016-11-15 11:36:08 -08:00 |
Masatake Iwasaki
|
7e918637de
|
HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran.
(cherry picked from commit 0bdd263d82 )
|
2016-10-27 15:54:03 +09:00 |
Yongjun Zhang
|
ce72314657
|
HDFS-11040. Add documentation for HDFS-9820 distcp improvement. Contributed by Yongjun Zhang.
(cherry picked from commit 0f0c15f7a5 )
|
2016-10-25 12:29:28 -07:00 |
Yongjun Zhang
|
7bc170ba26
|
HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang.
(cherry picked from commit 8650cc84f2 )
Conflicts
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpOptions.java
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
|
2016-10-25 11:32:26 -07:00 |
Allen Wittenauer
|
a5a56c3564
|
HADOOP-13034. Log message about input options in distcp lacks some items (Takashi Ohnishi via aw)
(cherry picked from commit 422c73a865 )
|
2016-10-17 16:35:32 -07:00 |
Jing Zhao
|
ad69baf6a9
|
HADOOP-13024. Distcp with -delete feature on raw data not implemented. Contributed by Mavin Martin.
(cherry picked from commit 0a85d07983 )
|
2016-10-13 13:26:02 -07:00 |
Brahma Reddy Battula
|
5737d04c5f
|
HDFS-9885. Correct the distcp counters name while displaying counters. Contributed by Surendra Singh Lilhore
(cherry picked from commit e17a4970be )
|
2016-09-27 10:47:07 +05:30 |
Steve Loughran
|
247cdbb86a
|
HADOOP-13643. Math error in AbstractContractDistCpTest. Contributed by Aaron Fabbri.
|
2016-09-23 10:00:32 +01:00 |
Chris Nauroth
|
e19f910245
|
HADOOP-13169. Randomize file list in SimpleCopyListing. Contributed by Rajesh Balamohan.
(cherry picked from commit 98bdb51397 )
|
2016-09-19 15:16:56 -07:00 |
Andrew Wang
|
38b1eafdbc
|
HDFS-10300. TestDistCpSystem should share MiniDFSCluster. Contributed by John Zhuge.
(cherry picked from commit f292624bd8 )
|
2016-07-11 18:06:35 -07:00 |
Yongjun Zhang
|
8a121dc3b8
|
HDFS-10396. Using -diff option with DistCp may get "Comparison method violates its general contract" exception. Contributed by Yongjun Zhang.
(cherry picked from commit 8113855b3a )
|
2016-06-28 23:29:48 -07:00 |
Jing Zhao
|
856bc4e28b
|
HDFS-10556. DistCpOptions should be validated automatically. Contributed by Mingliang Liu.
|
2016-06-22 15:15:47 -07:00 |
Yongjun Zhang
|
82a88c7e67
|
HADOOP-13199. Add doc for distcp -filters. (John Zhuge via Yongjun Zhang)
(cherry picked from commit cfb860dee7 )
|
2016-05-26 23:32:46 -07:00 |
Steve Loughran
|
e29d7fb585
|
HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth.
|
2016-05-20 12:21:47 +01:00 |
Jing Zhao
|
dd7f5a3b88
|
HDFS-10397. Distcp should ignore -delete option if -diff option is provided instead of exiting. Contributed by Mingliang Liu.
(cherry picked from commit 03788d3015 )
|
2016-05-17 15:53:39 -07:00 |
Steve Loughran
|
dfc062a092
|
HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel)
|
2016-05-17 13:01:50 +01:00 |
Chris Nauroth
|
bb41547fb4
|
HADOOP-13148. TestDistCpViewFs to include IOExceptions in test error reports. Contributed by Steve Loughran.
(cherry picked from commit b9685e85d5 )
|
2016-05-16 11:53:25 -07:00 |
Jing Zhao
|
d43d8a1bcd
|
HADOOP-12469. distcp should not ignore the ignoreFailures option. Contributed by Mingliang Liu.
(cherry picked from commit af942585a1 )
|
2016-05-04 10:24:06 -07:00 |
Yongjun Zhang
|
7955b6f66e
|
HDFS-10313. Distcp need to enforce the order of snapshot names passed to -diff. (Lin Yiqun via Yongjun Zhang)
(cherry picked from commit 959a28dd12 )
|
2016-04-26 16:27:59 -07:00 |
Akira Ajisaka
|
58642d2244
|
HDFS-10298. Document the usage of distcp -diff option. Contributed by Takashi Ohnishi.
(cherry picked from commit 02c51c27d9 )
|
2016-04-25 22:34:18 +09:00 |
Yongjun Zhang
|
f98f4151ac
|
HDFS-9670. DistCp throws NPE when source is root. (John Zhuge via Yongjun Zhang)
(cherry picked from commit a749ba0cea )
|
2016-04-21 12:23:42 -07:00 |
Jing Zhao
|
864baf23aa
|
HDFS-10216. Distcp -diff throws exception when handling relative path. Contributed by Takashi Ohnishi.
(cherry picked from commit 404f57f328 )
|
2016-04-14 10:35:43 -07:00 |
Masatake Iwasaki
|
8095c612a3
|
HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)
(cherry picked from commit 33a412e8a4 )
|
2016-03-03 18:57:41 +09:00 |
Yongjun Zhang
|
b144e0f15d
|
HDFS-9764. DistCp doesn't print value for several arguments including -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang)
(cherry picked from commit ba1c9d484a )
|
2016-02-19 10:29:27 -08:00 |
Yongjun Zhang
|
2fb711bde9
|
HDFS-9638. Improve DistCp Help and documentation. (Wei-Chiu Chuang via Yongjun Zhang)
(cherry picked from commit eddd823cd6 )
|
2016-01-29 12:13:03 -08:00 |