Commit Graph

5964 Commits

Author SHA1 Message Date
Andrew Wang 72cf96300b HDFS-6102. Lower the default maximum items per directory to fix PB fsimage loading. Contributed by Andrew Wang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577429 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 04:53:13 +00:00
Vinod Kumar Vavilapalli 3ac3e5f897 YARN-1658. Modified web-app framework to let standby RMs redirect web-service calls to the active RM. Contributed by Cindy Li.
svn merge --ignore-ancestry -c 1577408 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577409 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 02:40:28 +00:00
Haohui Mai bd2cb0117e HDFS-6084. Merge r1577401 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577402 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 01:00:35 +00:00
Christopher Douglas 8f915e55fd HADOOP-3679. Fixup assert ordering in unit tests to yield meaningful error
messages. Contributed by Jay Vyas


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577397 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 00:47:04 +00:00
Christopher Douglas 9695bc7af6 YARN-1771. Reduce the number of NameNode operations during localization of
public resources using a cache. Contributed by Sangjin Lee



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577392 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 00:31:08 +00:00
Colin McCabe 5d65afe2db HDFS-6097. Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577355 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 22:38:39 +00:00
Brandon Li 2cf495a464 HDFS-6080. Merging change r1577319 from trunk
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577320 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 21:07:14 +00:00
Jonathan Turner Eagles d2990f85b3 MAPREDUCE-5456. TestFetcher.testCopyFromHostExtraBytes is missing (Jason Lowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577318 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 20:58:27 +00:00
Arpit Agarwal fd4d1d1d05 MAPREDUCE-5794. Merging r1577281 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577283 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 18:57:01 +00:00
Suresh Srinivas a19b5531ca HDFS-5244. Merge 1577254 from trunk:
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577255 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 17:24:33 +00:00
Jonathan Turner Eagles eaf54d5a10 MAPREDUCE-5713. InputFormat and JobConf JavaDoc Fixes (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577240 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 16:50:13 +00:00
Jonathan Turner Eagles e6111498f8 HADOOP-10332. HttpServer's jetty audit log always logs 200 OK (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577221 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 16:11:45 +00:00
Jason Darrell Lowe 7f5a157995 svn merge -c 1577202 FIXES: MAPREDUCE-5789. Average Reduce time is incorrect on Job Overview page. Contributed by Rushabh S Shah
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577207 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 15:42:51 +00:00
Jonathan Turner Eagles 50c410546b MAPREDUCE-5765. Update hadoop-pipes examples README (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577203 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 15:35:46 +00:00
Zhijie Shen 7834c7f193 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal.
svn merge --ignore-ancestry -c 1577052 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577053 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 07:45:30 +00:00
Arpit Agarwal 502a0da830 HDFS-6096. Merging r1576999 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577000 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 23:25:28 +00:00
Arpit Agarwal 9127f3c1dc HDFS-5705. Merging r1555190 and r1576989 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576991 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 23:16:04 +00:00
Andrew Wang 49f08eebcc HDFS-6079. Timeout for getFileBlockStorageLocations does not work. Contributed by Andrew Wang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576980 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 22:56:35 +00:00
Jason Darrell Lowe 16669efeea svn merge -c 1576960 FIXES: YARN-1789. ApplicationSummary does not escape newlines in the app name. Contributed by Tsuyoshi OZAWA
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576962 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 22:27:01 +00:00
Arun Murthy 9e695db974 Merge -c 1576946 from trunk to branch-2 to fix MAPREDUCE-5553. Allow users to easily access completed/pending/successful/failed tasks on MR AM web-ui. Contributed by Paul Han.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576947 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 22:05:54 +00:00
Jason Darrell Lowe 804f86f42c svn merge -c 1576940 FIXES: MAPREDUCE-5778. JobSummary does not escape newlines in the job name. Contributed by Akira AJISAKA
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576943 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 21:59:50 +00:00
Vinod Kumar Vavilapalli eac9b53bd8 YARN-1816. Fixed ResourceManager to get RMApp correctly handle ATTEMPT_FINISHED event at ACCEPTED state that can happen after RM restarts. Contributed by Jian He.
svn merge --ignore-ancestry -c 1576911 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576913 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 21:19:22 +00:00
Vinod Kumar Vavilapalli 15744c614b YARN-1812. Fixed ResourceManager to synchrously renew tokens after recovery and thus recover app itself synchronously and avoid races with resyncing NodeManagers. Contributed by Jian He.
svn merge --ignore-ancestry -c 1576843 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576844 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 18:26:53 +00:00
Arun Murthy 8c4578e894 Merge -c 1576751 from trunk to branch-2 to fix YARN-1444. Fix CapacityScheduler to deal with cases where applications specify host/rack requests without off-switch request. Contributed by Wangda Tan.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576752 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 14:37:27 +00:00
Vinod Kumar Vavilapalli d5120ccc6b YARN-1800. Fixed NodeManager to gracefully handle RejectedExecutionException in the public-localizer thread-pool. Contributed by Varun Vasudev.
svn merge --ignore-ancestry -c 1576545 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576546 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 23:34:20 +00:00
Colin McCabe 567654e6ee HDFS-6086. Fix a case where zero-copy or no-checksum reads were not allowed even when the block was cached. (cmccabe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576534 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 22:42:30 +00:00
Karthik Kambatla 6f4d705a63 YARN-1821. NPE on registerNodeManager if the request has containers for UnmanagedAMs (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576527 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 22:19:30 +00:00
Haohui Mai 237a2054b4 Merge r1576513 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576515 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 21:48:26 +00:00
Chris Nauroth a6e3cbc7f3 HDFS-5638. Merging change r1576405 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576410 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 16:21:45 +00:00
Colin McCabe 1be546ff2b HDFS-6085. Improve CacheReplicationMonitor log messages a bit (cmccabe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576195 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 06:25:27 +00:00
Michael Stack 243db1c57e HADOOP-10337 ConcurrentModificationException from MetricsDynamicMBeanBase.createMBeanInfo() (Liang Xie via stack)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576189 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 04:38:06 +00:00
Vinod Kumar Vavilapalli ad5c0e4613 MAPREDUCE-5028. Fixed a bug in MapTask that was causing mappers to fail when a large value of io.sort.mb is set. Contributed by Karthik Kambatla.
svn merge --ignore-ancestry -c 1576170 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576171 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 01:30:20 +00:00
Vinod Kumar Vavilapalli cbe653f9a2 YARN-1764. Modified YarnClient to correctly handle failover of ResourceManager after the submitApplication call goes through. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1576160 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576162 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 00:52:33 +00:00
Andrew Wang 0f7d99268c HDFS-3405. Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages. Contributed by Vinayakumar B.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576157 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 00:44:56 +00:00
Andrew Wang 0697a50337 Revert bad HDFS-3405 merge from trunk to retry
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576155 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 00:40:46 +00:00
Andrew Wang 1356761f0f HDFS-3405. Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages. Contributed by Vinayakumar B.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576154 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 00:34:16 +00:00
Tsz-wo Sze 5d2252b6dc svn merge -c 1575452 from trunk for HDFS-6076. DataNode with SimulatedDataSet should not create DatanodeRegistration with namenode layout version and namenode node type.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576151 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 00:22:43 +00:00
Tsz-wo Sze a42d3356db svn merge -c 1575021 from trunk for HDFS-6060. NameNode should not check DataNode layout version
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576149 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 00:12:57 +00:00
Arpit Agarwal bb7e14902b HADOOP-10394. Merging r1576145 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576146 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 00:03:11 +00:00
Arpit Agarwal 4af360d629 HADOOP-10395. Merging r1575562 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576142 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 23:58:05 +00:00
Tsz-wo Sze 134a29f226 svn merge -c 1574259 from trunk for HDFS-5535.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576128 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 23:37:46 +00:00
Chris Nauroth 0663d54767 HADOOP-10399. Merging change r1576124 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576126 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 23:15:51 +00:00
Chris Nauroth 946db612ed HDFS-6055. Merging change r1576095 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576098 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 21:47:45 +00:00
Jing Zhao 9fcbd9413f HDFS-6077. Merge change r1576076 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576077 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 20:50:40 +00:00
Vinod Kumar Vavilapalli 8a3978b18a YARN-1788. Fixed a bug in ResourceManager to set the apps-completed and apps-killed metrics correctly for killed applications. Contributed by Varun Vasudev.
svn merge --ignore-ancestry -c 1576072 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576073 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 20:37:09 +00:00
Andrew Wang 3826799270 HDFS-6070. Cleanup use of ReadStatistics in DFSInputStream.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576048 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 19:05:10 +00:00
Haohui Mai c3c267af8f Merge r1576035 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576036 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 18:18:52 +00:00
Jian He 8d6fb7e920 Merge r1576023 from trunk. Fixed ClientRMService#forceKillApplication not killing unmanaged application. Contributed by Karthik Kambatla
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576026 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 18:03:47 +00:00
Colin McCabe bc927a10cf HDFS-6071. BlockReaderLocal does not return -1 on EOF when doing a zero-length read on a short file. (cmccabe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575799 13f79535-47bb-0310-9956-ffa450edef68
2014-03-10 02:48:11 +00:00
Arpit Agarwal 03f20f77e1 HDFS-6078. Merging r1575559 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575561 13f79535-47bb-0310-9956-ffa450edef68
2014-03-08 16:40:27 +00:00