Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,436 Commits 326 Branches 371 Tags
Commit Graph

11436 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Steve Loughran
15f997cca6 HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran) 2016-05-21 14:19:56 +01:00
Chris Nauroth
e7b56cf93c HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth. 
(cherry picked from commit 500e946729e1b85fdc0ce8475d8ce118ba886f46)
 2016-05-20 13:01:59 -07:00
Chris Nauroth
d38f2090fa HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth. 
(cherry picked from commit bcde1562d25c4f5595f4e3436dc3630315b1ceed)
 2016-05-20 12:47:52 -07:00
Jian He
6e0522ee1b YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi 
(cherry picked from commit d364ceac85622e99133b3eb3becef0c8188e6f89)
 2016-05-20 11:36:08 -07:00
Steve Loughran
eb160561e0 HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong. 2016-05-20 13:52:30 +01:00
Steve Loughran
e29d7fb585 HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth. 2016-05-20 12:21:47 +01:00
Varun Vasudev
e7f396fd47 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 
(cherry picked from commit 0287c491071c15e1c47dc640e6ea437be8cf32ae)
 2016-05-20 12:32:29 +05:30
Chris Nauroth
60e5397a9b HADOOP-13183. S3A proxy tests fail after httpclient/httpcore upgrade. Contributed by Steve Loughran. 
(cherry picked from commit 93258459faf56bc84121ba99d20eaef95273329e)
 2016-05-19 22:00:32 -07:00
Jian He
db3cf5b354 MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu 
(cherry picked from commit 21d2b90213e8e188bcac31f9360c5176ac89d083)
 2016-05-19 19:25:48 -07:00
Tsz-Wo Nicholas Sze
658072d62e HADOOP-13168. Support Future.get with timeout in ipc async calls. 2016-05-19 15:58:15 -07:00
Junping Du
2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Kihwal Lee
bae151801d HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu. 
(cherry picked from commit 204de430bf6a931e5688789c7a783c9c568a98c2)
 2016-05-19 15:53:41 -05:00
Jian He
2d846fdcbd YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca536e7deac50976b8a8774450fe089f)
 2016-05-19 13:02:05 -07:00
Allen Wittenauer
f7ffa1a534 HADOOP-13177. Native tests fail on OS X, because DYLD_LIBRARY_PATH is not defined to include libhadoop.dylib (Chris Nauroth via aw) 
(cherry picked from commit c7159f0240be0c555edf01d9a580987b0997125d)
 2016-05-19 09:58:48 -07:00
Junping Du
416274b53e YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong. 
(cherry picked from commit 141873ca7d68619e3458712b17aed57fec8fd7c8)
 2016-05-19 09:14:42 -07:00
Steve Loughran
151ecdf26d HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:44:44 +01:00
Arun Suresh
15808717f3 YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 1597630681c784a3d59f5605b87e96197b8139d7)
 2016-05-18 23:20:54 -07:00
Arun Suresh
a555a320e8 YARN-5090. Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster. (asuresh) 
(cherry picked from commit 8a9ecb7584b529be1de3a6e8850041e211c88ebc)
 2016-05-18 23:20:36 -07:00
Arun Suresh
f31602893e YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh) 
(cherry picked from commit ccc93e78127c14bfd84179395b055c4061ea436a)
 2016-05-18 23:20:08 -07:00
Arun Suresh
bbcea3abf3 YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh) 
(cherry picked from commit f45bc5a83ed2f2292ecf1f4c990b7153d8051fc9)
 2016-05-18 23:12:19 -07:00
Arun Suresh
69a9ce3494 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 
(cherry picked from commit f0ac18d001d97914a9ee810b1fab56c5cebff830)
 2016-05-18 23:09:11 -07:00
Arun Suresh
005dae521c YARN-5073. Refactor startContainerInternal() in ContainerManager to remove unused parameter. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d971bf252f36e902a0fe446cb11201835ad16bcb)
 2016-05-18 22:36:28 -07:00
Arun Suresh
1b4600abcf YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 
(cherry picked from commit 341888a0aa23f24458b4e6e34868794b9735c06a)
 2016-05-18 22:35:55 -07:00
Arun Suresh
307cda70db YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d464f4d1c4dec483852fc8c0496787cba0af8f57)
 2016-05-18 22:09:30 -07:00
Arun Suresh
d80d24aabf YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
(cherry picked from commit c282a08f3892e2e8ceb58e1e9a411062fbd1fb2b)
 2016-05-18 22:08:55 -07:00
Karthik Kambatla
b56fc51b70 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 
(cherry picked from commit c8172f5f143d2fefafa5a412899ab7cd081b406d)
 2016-05-18 22:08:20 -07:00
Arun Suresh
68964bbaa2 YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh) 
(cherry picked from commit 8ffabfdf4fec104cdaa68133385cfe9cf88b95a8)
(cherry picked from commit b2a654c5ee6524f81c971ea0b70e58ea0a455f1d)
 2016-05-18 22:02:43 -07:00
Karthik Kambatla
2c6c701aff YARN-2882. Add an OPPORTUNISTIC ExecutionType. (Konstantinos Karanasos and Inigo Goiri via kasha) 
(cherry picked from commit fb00794368e0aa7aafa9dfc8d453810f641b82b2)
 2016-05-18 22:02:28 -07:00
Andrew Wang
686691df60 HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder. 2016-05-18 14:55:31 -07:00
Andrew Wang
a44c87c2d0 HDFS-2173. saveNamespace should not throw IOE when only one storage directory fails to write VERSION file. Contributed by Andras Bokor. 
(cherry picked from commit 010e6ac328855bad59f138b6aeaec535272f448c)
 2016-05-18 14:13:47 -07:00
Robert Kanter
27aabcab45 MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter) 
(cherry picked from commit 992a49353f0bdbc71534bfeeb134112c5624b94b)
 2016-05-18 12:16:45 -07:00
Xiaoyu Yao
1470af8c95 HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu. 
(cherry picked from commit ccb1cade5b5f8421b907cafe147a4d610fb5fff4)
 2016-05-18 11:02:31 -07:00
Naganarasimha
a166286b9e YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt 
(cherry picked from commit f04c81c9ce93512bc714531a7731debbe6b794ce)
 2016-05-19 01:35:47 +08:00
Varun Vasudev
3c9392a31d YARN-4913. Yarn logs should take a -out option to write to a directory. Contributed by Xuan Gong. 
(cherry picked from commit ef1757790d89cc72f88f5330761b1c8901c59e94)
 2016-05-18 22:45:06 +05:30
Chris Nauroth
42f64b3bcc HADOOP-13138. Unable to append to a SequenceFile with Compression.NONE. Contributed by Vinayakumar B. 
(cherry picked from commit 0a527196dcb35a699e12121165bc3738766b569e)
 2016-05-18 09:32:15 -07:00
Arpit Agarwal
24bed7768d HDFS-9926. MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils. (Contributed by Josh Elser) 2016-05-18 09:21:41 -07:00
Colin Patrick Mccabe
feabfe9cda HADOOP-13140. FileSystem#initialize must not attempt to create StorageStatistics objects with null or empty schemes (Mingliang Liu via cmccabe) 
(cherry picked from commit f2c1d9181f80e53890b50309b15c9ea37cb24987)
 2016-05-18 09:19:45 -07:00
Akira Ajisaka
e9dd0d84e5 Revert "HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder." 
This reverts commit e205421555841dbf482d6aa1464cc15788b2b1fb.
 2016-05-18 17:57:46 +09:00
Andrew Wang
e205421555 HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder. 
(cherry picked from commit 7154ace71212e9fb9dd6209a92165fb075df7806)
 2016-05-17 17:44:34 -07:00
Lei Xu
2d1d2dde17 HDFS-10360. DataNode may format directory and lose blocks if current/VERSION is missing. (Wei-Chiu Chuang via lei) 
(cherry picked from commit bae11b95d492e0be7c5576097424879c2539b474)
 2016-05-17 16:37:36 -07:00
Yongjun Zhang
456fe08a6d HDFS-10381, DataStreamer DataNode exclusion log message should be warning. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 16c07cc68a3e0a06f57b7f4c7207cc8e5dce211f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java
 2016-05-17 15:59:00 -07:00
Jing Zhao
dd7f5a3b88 HDFS-10397. Distcp should ignore -delete option if -diff option is provided instead of exiting. Contributed by Mingliang Liu. 
(cherry picked from commit 03788d3015c962eac1a35fa5df39356e8b84731c)
 2016-05-17 15:53:39 -07:00
Xiaoyu Yao
937bc924cb HADOOP-13159. Fix potential NPE in Metrics2 source for DecayRpcScheduler. Contributed by Xiaoyu Yao. 
(cherry picked from commit 94784848456a92a6502f3a3c0074e44fba4b19c9)
(cherry picked from commit 744d5edc87686a0e70e6a78a67ee55723e1cf26a)
 2016-05-17 15:51:47 -07:00
Jian He
264c06a438 YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 2016-05-17 15:02:14 -07:00
Junping Du
7e36b9159e MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen. 
(cherry picked from commit f6ef876fe158a5334cad7075f1966573a1c4dec9)
 2016-05-17 14:42:11 -07:00
Kihwal Lee
1e7eb2d2ac Revert "HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu." 
This reverts commit 70ee477e5f5f220fba98cf96bdbab5c0a6295841.
 2016-05-17 15:20:59 -05:00
Kihwal Lee
70ee477e5f HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu. 
(cherry picked from commit 0942954e8ab6bef636c2648d0d5a0227a50f799c)
 2016-05-17 15:10:47 -05:00
Colin Patrick Mccabe
6ebb92c9c1 HDFS-10404. Fix formatting of CacheAdmin command usage help text (Yiqun Lin via cmccabe) 
(cherry picked from commit 7cd5ae62f639a857f24f5463f2aefc099c631a14)
 2016-05-17 11:59:10 -07:00
Jason Lowe
9c47862d87 MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen 
(cherry picked from commit 34fddd1e912dcd9f898e500b8df11295ee49c0d8)
 2016-05-17 15:04:02 +00:00
Jason Lowe
51e06c085f MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen 
(cherry picked from commit 12fa4ec141f8d0cb105b6f60c3739a4b26890fd5)
 2016-05-17 14:56:23 +00:00