Commit Graph

64 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli 6604613bc5 MAPREDUCE-6248. Exposed the internal MapReduce job's information as a public API in DistCp. Contributed by Jing Zhao.
(cherry picked from commit 5af693fde2)
2015-03-03 16:29:36 -08:00
Akira Ajisaka f7a724ca9e HADOOP-10976. Backport "moving the source code of hadoop-tools docs to the directory under hadoop-tools" to branch-2. Contributed by Masatake Iwasaki.
(cherry picked from commit 9112f093cd)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
2015-02-28 17:06:58 -08:00
Haohui Mai a1e4a12dc0 HADOOP-11381. Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack. Contributed by Li Lu. 2014-12-09 20:46:44 -08:00
Jing Zhao 582f96e41d HADOOP-11354. ThrottledInputStream doesn't perform effective throttling. Contributed by Ted Yu. 2014-12-08 11:09:34 -08:00
Arun C. Murthy adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Allen Wittenauer afde00a0f5 MAPREDUCE-6109. Fix minor typo in distcp -p usage text (Charles Lamb via aw) 2014-09-25 11:28:50 -07:00
Allen Wittenauer ea1fb1461a HADOOP-11009. Add Timestamp Preservation to DistCp (Gary Steelman via aw) 2014-09-24 15:39:55 -07:00
Alejandro Abdelnur c77bd85b62 HDFS-6134 and HADOOP-10150 subtasks. 2014-08-28 15:03:08 -07:00
Andrew Wang 59c91b22d7 MAPREDUCE-5971. Move the default options for distcp -p to DistCpOptionSwitch. Contributed by Charles Lamb.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1611218 13f79535-47bb-0310-9956-ffa450edef68
2014-07-16 23:43:00 +00:00
Karthik Kambatla 9133e3142c Preparing for 2.6.0 development. Updated versions.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607541 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 06:11:57 +00:00
Andrew Wang a18dce22b9 HADOOP-10666. Remove Copyright /d/d/d/d Apache Software Foundation from the source files license header. Contributed by Henry Saputra.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1603026 13f79535-47bb-0310-9956-ffa450edef68
2014-06-16 23:27:03 +00:00
Uma Maheswara Rao G 2f25a4f3a4 Merged from trunk to branch-2. MAPREDUCE-5898. distcp to support preserving HDFS extended attributes(XAttrs). Contributed by Yi Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1602116 13f79535-47bb-0310-9956-ffa450edef68
2014-06-12 10:59:28 +00:00
Jing Zhao 50acb4032f MAPREDUCE-5899. Merge change r1596931 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1596937 13f79535-47bb-0310-9956-ffa450edef68
2014-05-22 18:50:20 +00:00
Chris Nauroth d20f565b4c MAPREDUCE-5809. Merging change r1595283 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1595290 13f79535-47bb-0310-9956-ffa450edef68
2014-05-16 18:32:15 +00:00
Tsz-wo Sze 6b7d074a54 svn merge -c 1592703 from trunk for MAPREDUCE-5402. In DynamicInputFormat, change MAX_CHUNKS_TOLERABLE, MAX_CHUNKS_IDEAL, MIN_RECORDS_PER_CHUNK and SPLIT_RATIO to be configurable.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1592704 13f79535-47bb-0310-9956-ffa450edef68
2014-05-06 10:25:10 +00:00
Jonathan Turner Eagles e0cd544a53 MAPREDUCE-5639. Port DistCp2 document to trunk (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590131 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 20:32:09 +00:00
Chris Nauroth f0d3664d83 MAPREDUCE-5852. Merging change r1589006 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589007 13f79535-47bb-0310-9956-ffa450edef68
2014-04-21 23:49:28 +00:00
Aaron Myers c5b5947777 HADOOP-10459. distcp V2 doesn't preserve root dir's attributes when -p is specified. Contributed by Yongjun Zhang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1584233 13f79535-47bb-0310-9956-ffa450edef68
2014-04-03 00:56:12 +00:00
Aaron Myers 71a7f2b72b MAPREDUCE-5014. Extending DistCp through a custom CopyListing is not possible. (Contributed by Srikanth Sundarrajan)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1584232 13f79535-47bb-0310-9956-ffa450edef68
2014-04-03 00:49:52 +00:00
Jing Zhao d611f42c37 Merge r1578669 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578676 13f79535-47bb-0310-9956-ffa450edef68
2014-03-18 00:10:22 +00:00
Arun Murthy 1ccccb8534 Branching for hadoop-2.4 releases, also set version to 2.5.0-SNAPSHOT in branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569186 13f79535-47bb-0310-9956-ffa450edef68
2014-02-18 04:41:01 +00:00
Jing Zhao c21e39b79b HADOOP-10295. Merge change r1563019 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1563020 13f79535-47bb-0310-9956-ffa450edef68
2014-01-30 23:58:56 +00:00
Konstantin Boudnik f3136cd6b9 HADOOP-10167. Mark hadoop-common source as UTF-8 in Maven pom files / refactoring. Contributed by Mikhail Antonov.
Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1560832 13f79535-47bb-0310-9956-ffa450edef68
2014-01-23 22:31:27 +00:00
Daryn Sharp 92fc940ea8 svn merge -c 1548175 FIXES: HADOOP-10129. Distcp may succeed when it fails (daryn)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1548176 13f79535-47bb-0310-9956-ffa450edef68
2013-12-05 15:51:28 +00:00
Arun Murthy 439a690ad1 Setting version to 2.4.0-SNAPSHOT in branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1547165 13f79535-47bb-0310-9956-ffa450edef68
2013-12-02 19:40:42 +00:00
Steve Loughran a7bfe3efe3 MAPREDUCE-6524 Move grizzly-test and junit dependencies to test scope
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1541699 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 20:43:04 +00:00
Jonathan Turner Eagles f35f88b1cf MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1532890 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 20:28:13 +00:00
Jonathan Turner Eagles 85b5e96f10 MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7 (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1532878 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 20:06:53 +00:00
Jason Darrell Lowe 4cd5cb862c svn merge -c 1509900 FIXES: MAPREDUCE-5440. TestCopyCommitter Fails on JDK7. Contributed by Robert Parker
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1509901 13f79535-47bb-0310-9956-ffa450edef68
2013-08-02 22:53:52 +00:00
Arun Murthy 83c776c85d Set version in branch-2 to 2.3.0-SNAPSHOT to reflect that we will only be adding new content to hadoop-2.3.0 and not hadoop-2.2.0.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1504744 13f79535-47bb-0310-9956-ffa450edef68
2013-07-19 01:35:34 +00:00
Daryn Sharp 45ca24c1d8 svn merge -c 1491914 FIXES: MAPREDUCE-5315. DistCp reports success even on failure. (mithun and jlowe via daryn)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1491918 13f79535-47bb-0310-9956-ffa450edef68
2013-06-11 19:30:44 +00:00
Arun Murthy 27ed92a3fb Set version in branch-2 to 2.2.0-SNAPSHOT.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489479 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 15:34:52 +00:00
Jason Darrell Lowe eb7af141e4 HADOOP-9287. Parallel-testing hadoop-common. Contributed by Andrey Klochkov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489250 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 00:29:37 +00:00
Konstantin Boudnik 6e0b55c5b7 Setting artifact versions to 2.1.0-SNAPSHOT to refrecl recent release of 2,0.5-alpha
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488586 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 20:06:41 +00:00
Kihwal Lee 88086534a8 svn merge -c 1468629 Merging from trunk to branch-2 to fix MR-5065.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1468631 13f79535-47bb-0310-9956-ffa450edef68
2013-04-16 22:03:40 +00:00
Arun Murthy 91a676fc41 Setting version in branch-2 to be 2.0.5-SNAPSHOT.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1466320 13f79535-47bb-0310-9956-ffa450edef68
2013-04-10 03:16:30 +00:00
Tsz-wo Sze 84664ccde2 svn merge -c 1458741 from trunk for MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does not close the wrapped InputStream.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1458747 13f79535-47bb-0310-9956-ffa450edef68
2013-03-20 10:43:21 +00:00
Alejandro Abdelnur b0c23b2c27 HADOOP-9230. TestUniformSizeInputFormat fails intermittently. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1451292 13f79535-47bb-0310-9956-ffa450edef68
2013-02-28 18:52:24 +00:00
Arun Murthy 196341fc09 Setting version to 2.0.4-SNAPSHOT in branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1442956 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 13:23:45 +00:00
Thomas Graves 2fa1932e9d HADOOP-9097. Maven RAT plugin is not checking all source files (tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1432942 13f79535-47bb-0310-9956-ffa450edef68
2013-01-14 15:02:21 +00:00
Robert Joseph Evans de89166116 svn merge -c 1414239 FIXES: YARN-204. test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1414241 13f79535-47bb-0310-9956-ffa450edef68
2012-11-27 16:06:57 +00:00
Jason Darrell Lowe 57657df4dc svn merge -c 1408604 FIXES: HADOOP-9025. org.apache.hadoop.tools.TestCopyListing failing. Contributed by Jonathan Eagles
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1408605 13f79535-47bb-0310-9956-ffa450edef68
2012-11-13 05:55:51 +00:00
Robert Joseph Evans 877b8200df svn merge -c 1408488 FIXES: HADOOP-9022. Hadoop distcp tool fails to copy file if -m 0 specified (Jonathan Eagles vai bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1408491 13f79535-47bb-0310-9956-ffa450edef68
2012-11-12 22:47:05 +00:00
Thomas White 4a63906756 Merge -r 1396046:1396047 from trunk to branch-2. Fixes: MAPREDUCE-4654. TestDistCp is ignored. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1396051 13f79535-47bb-0310-9956-ffa450edef68
2012-10-09 14:49:51 +00:00
Arun Murthy f403be7569 Set version in branch-2 to 2.0.3-SNAPSHOT
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1382809 13f79535-47bb-0310-9956-ffa450edef68
2012-09-10 12:57:48 +00:00
Aaron Myers ee6d9ff79d HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1382121 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 18:41:28 +00:00
Eli Collins 6628e4f726 HADOOP-8431. Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1381841 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 01:27:15 +00:00
Todd Lipcon 0c5de0e4f6 HDFS-3054. distcp -skipcrccheck has no effect. Contributed by Colin Patrick McCabe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1381295 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 18:54:32 +00:00
Robert Joseph Evans bc5f02bac0 svn merge -c 1376618 FIXES: HADOOP-8225. DistCp fails when invoked by Oozie (daryn via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1376621 13f79535-47bb-0310-9956-ffa450edef68
2012-08-23 18:07:59 +00:00
Robert Joseph Evans e26c63b238 svn emrge -c 1373599. FIXES: HADOOP-8703
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1373600 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 19:51:16 +00:00