Commit Graph

143 Commits

Author SHA1 Message Date
Jing Zhao 3671a5e16f MAPREDUCE-5899. Support incremental data copy in DistCp. Contributed by Jing Zhao.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596931 13f79535-47bb-0310-9956-ffa450edef68
2014-05-22 18:17:11 +00:00
Chris Nauroth 11be7334c4 MAPREDUCE-5809. Enhance distcp to support preserving HDFS ACLs. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595283 13f79535-47bb-0310-9956-ffa450edef68
2014-05-16 18:25:23 +00:00
Tsz-wo Sze 03db13206f MAPREDUCE-5402. In DynamicInputFormat, change MAX_CHUNKS_TOLERABLE, MAX_CHUNKS_IDEAL, MIN_RECORDS_PER_CHUNK and SPLIT_RATIO to be configurable. Contributed by Tsuyoshi OZAWA
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592703 13f79535-47bb-0310-9956-ffa450edef68
2014-05-06 10:24:10 +00:00
Jonathan Turner Eagles a059eadbe9 MAPREDUCE-5639. Port DistCp2 document to trunk (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590058 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 15:23:42 +00:00
Chris Nauroth 06c15b6a3e MAPREDUCE-5852. Prepare MapReduce codebase for JUnit 4.11. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589006 13f79535-47bb-0310-9956-ffa450edef68
2014-04-21 23:44:41 +00:00
Aaron Myers 640a097533 HADOOP-10459. distcp V2 doesn't preserve root dir's attributes when -p is specified. Contributed by Yongjun Zhang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584227 13f79535-47bb-0310-9956-ffa450edef68
2014-04-03 00:32:25 +00:00
Haohui Mai fb1d7fb596 HDFS-5570. Deprecate hftp / hsftp and replace them with webhdfs / swebhdfs. Contributed by Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584100 13f79535-47bb-0310-9956-ffa450edef68
2014-04-02 17:28:08 +00:00
Jing Zhao 0b3ee72af8 HDFS-6090. Use MiniDFSCluster.Builder instead of deprecated constructors. Contributed by Akira AJISAKA.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578669 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 23:53:48 +00:00
Jing Zhao 067d52b98c HADOOP-10295. Allow distcp to automatically identify the checksum type of source files and use it for the target. Contributed by Jing Zhao and Laurent Goujon.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563019 13f79535-47bb-0310-9956-ffa450edef68
2014-01-30 23:53:23 +00:00
Konstantin Boudnik 4c197b5d56 HADOOP-10167. Mark hadoop-common source as UTF-8 in Maven pom files / refactoring. Contributed by Mikhail Antonov.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560831 13f79535-47bb-0310-9956-ffa450edef68
2014-01-23 22:29:23 +00:00
Daryn Sharp 9ea61e4415 HADOOP-10129. Distcp may succeed when it fails (daryn)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548175 13f79535-47bb-0310-9956-ffa450edef68
2013-12-05 15:47:55 +00:00
Steve Loughran 01ccd4d3ab MAPREDUCE-5624 Move grizzly-test and junit dependencies to test scope
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541702 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 20:46:54 +00:00
Jonathan Turner Eagles 74a2191aae MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532887 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 20:21:21 +00:00
Jonathan Turner Eagles 034039b917 MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7 (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532875 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 19:57:55 +00:00
Jason Darrell Lowe e1d398b2c9 MAPREDUCE-5440. TestCopyCommitter Fails on JDK7. Contributed by Robert Parker
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509900 13f79535-47bb-0310-9956-ffa450edef68
2013-08-02 22:51:34 +00:00
Daryn Sharp 723ef662a9 MAPREDUCE-5315. DistCp reports success even on failure. (mithun and jlowe via daryn)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491914 13f79535-47bb-0310-9956-ffa450edef68
2013-06-11 19:22:53 +00:00
Jason Darrell Lowe 85623a2d75 HADOOP-9287. Parallel-testing hadoop-common. Contributed by Andrey Klochkov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488040 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 22:43:16 +00:00
Kihwal Lee 2e789dd9a6 MAPREDUCE-5065. DistCp should skip checksum comparisons if block-sizes are different on source/target. Contributed by Mithun Radhakrishnan.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468629 13f79535-47bb-0310-9956-ffa450edef68
2013-04-16 22:01:18 +00:00
Amareshwari Sri Ramadasu 6f92ad9868 MAPREDUCE-5014. Extend Distcp to accept a custom CopyListing. Contributed by Srikanth Sundarrajan
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459690 13f79535-47bb-0310-9956-ffa450edef68
2013-03-22 10:09:06 +00:00
Tsz-wo Sze 718f0f92a9 MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does not close the wrapped InputStream. Contributed by Chris Nauroth
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458741 13f79535-47bb-0310-9956-ffa450edef68
2013-03-20 10:40:14 +00:00
Alejandro Abdelnur 6401d5d2ec HADOOP-9230. TestUniformSizeInputFormat fails intermittently. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451291 13f79535-47bb-0310-9956-ffa450edef68
2013-02-28 18:51:39 +00:00
Thomas Graves ab0e94d2a5 HADOOP-9097. Maven RAT plugin is not checking all source files (tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432934 13f79535-47bb-0310-9956-ffa450edef68
2013-01-14 14:37:12 +00:00
Robert Joseph Evans 966e20fa3d YARN-204. test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414239 13f79535-47bb-0310-9956-ffa450edef68
2012-11-27 16:05:16 +00:00
Jason Darrell Lowe 1ebe1d8303 HADOOP-9025. org.apache.hadoop.tools.TestCopyListing failing. Contributed by Jonathan Eagles
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408604 13f79535-47bb-0310-9956-ffa450edef68
2012-11-13 05:53:26 +00:00
Robert Joseph Evans 416c7ded54 HADOOP-9022. Hadoop distcp tool fails to copy file if -m 0 specified (Jonathan Eagles vai bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408488 13f79535-47bb-0310-9956-ffa450edef68
2012-11-12 22:43:07 +00:00
Thomas White aa934616dd MAPREDUCE-4654. TestDistCp is ignored. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396047 13f79535-47bb-0310-9956-ffa450edef68
2012-10-09 14:37:46 +00:00
Aaron Myers 6ba9ff9177 HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382119 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 18:40:30 +00:00
Eli Collins 404ab8ec66 HADOOP-8431. Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381840 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 01:25:30 +00:00
Todd Lipcon e57843e02a HDFS-3054. distcp -skipcrccheck has no effect. Contributed by Colin Patrick McCabe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381296 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 18:54:45 +00:00
Robert Joseph Evans aeb364d8b5 HADOOP-8225. DistCp fails when invoked by Oozie (daryn via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376618 13f79535-47bb-0310-9956-ffa450edef68
2012-08-23 18:06:56 +00:00
Robert Joseph Evans b05c70bd96 HADOOP-8703: Fix formatting issue.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373599 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 19:50:07 +00:00
Robert Joseph Evans 6167123bcb HADOOP-8703. distcpV2: turn CRC checking off for 0 byte size (Dave Thompson via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373581 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 19:16:13 +00:00
Harsh J 1fab229f30 HADOOP-8268. A few pom.xml across Hadoop project may fail XML validation. Contributed by Radim Kolar. (harsh)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1343272 13f79535-47bb-0310-9956-ffa450edef68
2012-05-28 14:51:05 +00:00
Robert Joseph Evans a9808de0d9 HADOOP-8341. Fix or filter findbugs issues in hadoop-tools (bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335505 13f79535-47bb-0310-9956-ffa450edef68
2012-05-08 13:20:56 +00:00
Alejandro Abdelnur 0fca4fbaf3 MAPREDUCE-4205. retrofit all JVM shutdown hooks to use ShutdownHookManager (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333748 13f79535-47bb-0310-9956-ffa450edef68
2012-05-04 03:20:16 +00:00
Robert Joseph Evans 500dc61523 HADOOP-8305. distcp over viewfs is broken (John George via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331440 13f79535-47bb-0310-9956-ffa450edef68
2012-04-27 14:39:35 +00:00
Arun Murthy 67a74198b1 Changed version in trunk to 3.0.0-SNAPSHOT.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1306200 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 07:37:06 +00:00
Robert Joseph Evans 03e4684fa5 HADOOP-8064. Remove unnecessary dependency on w3c.org in document processing (Khiwal Lee via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1297274 13f79535-47bb-0310-9956-ffa450edef68
2012-03-05 23:07:31 +00:00
Aaron Myers dcd2056e46 HADOOP-8116. RetriableCommand is using RetryPolicy incorrectly after HADOOP-7896. Contributed by Aaron T. Myers.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1294729 13f79535-47bb-0310-9956-ffa450edef68
2012-02-28 16:51:55 +00:00
Mahadev Konar faa6806df3 MAPREDUCE-3735. Add distcp jar to the distribution (tar). (mahadev)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236336 13f79535-47bb-0310-9956-ffa450edef68
2012-01-26 18:10:11 +00:00
Mahadev Konar 34a6a9f4bc MAPREDUCE-3733. Add Apache License Header to hadoop-distcp/pom.xml. (mahadev)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236066 13f79535-47bb-0310-9956-ffa450edef68
2012-01-26 08:30:17 +00:00
Mahadev Konar d5bcc58828 MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236048 13f79535-47bb-0310-9956-ffa450edef68
2012-01-26 06:46:25 +00:00
Mahadev Konar d06948002f MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236045 13f79535-47bb-0310-9956-ffa450edef68
2012-01-26 06:36:52 +00:00