Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,447 Commits 326 Branches 371 Tags
Commit Graph

11447 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Tsz-Wo Nicholas Sze
076a73f207 HDFS-9365. Balaner does not work with the HDFS-6376 HA setup. 2016-05-24 13:08:04 -07:00
Karthik Kambatla
108db38788 YARN-4878. Expose scheduling policy and max running apps over JMX for Yarn queues. (Yufei Gu via kasha) 
(cherry picked from commit f979d779e192ac05906b7eb880dfc8884d0078a0)
 2016-05-24 10:54:33 -07:00
Arun Suresh
ce886ff2b1 YARN-4991. Fix ContainerRequest Constructor to set nodelabelExpression correctly. (Bibin A Chundatt via asuresh) 
(cherry picked from commit a5fed8bbfeed9b58c59584afa5931eae635afa2f)
 2016-05-24 05:12:03 -07:00
Naganarasimha
155f25061e YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 
(cherry picked from commit b4078bd17b41fbfff5a5c0bca5bf903a327826a7)
 2016-05-24 08:14:53 +08:00
Karthik Kambatla
f33f0b697b YARN-4979. FSAppAttempt demand calculation considers demands at multiple locality levels different. (Zhihai Xu via kasha) 
(cherry picked from commit 6d043aa4cf14bc88b1a19991e3bb0752499312b8)
 2016-05-23 14:30:05 -07:00
Jason Lowe
7602070457 YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 
(cherry picked from commit ac954486c5102b8fbbc4229a0d3a512bcc7013c0)
 2016-05-23 15:56:00 +00:00
Jason Lowe
e8bd2a5be6 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 
(cherry picked from commit d1df0266cf4e9ff0ec70813c156556ca4e74f791)
 2016-05-23 15:16:52 +00:00
Junping Du
867cd2f5a2 YARN-5112. Excessive log warnings for directory permission issue on NM recovery. Contributed by Jian He. 2016-05-22 16:02:57 -07:00
Akira Ajisaka
a59f30272d MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki. 2016-05-23 07:25:52 +09:00
Akira Ajisaka
1fb74e569a YARN-5126. Remove CHANGES.txt from branch-2. (aajisaka) 2016-05-23 07:13:40 +09:00
Masatake Iwasaki
094bc727c5 HDFS-10439. Update setOwner doc in HdfsPermissionsGuide. Contributed by John Zhuge. 
(cherry picked from commit 12eb5a7b11c36df5a57a3894a3a44bf47f47b954)
 2016-05-22 19:15:28 +09:00
Steve Loughran
15f997cca6 HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran) 2016-05-21 14:19:56 +01:00
Chris Nauroth
e7b56cf93c HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth. 
(cherry picked from commit 500e946729e1b85fdc0ce8475d8ce118ba886f46)
 2016-05-20 13:01:59 -07:00
Chris Nauroth
d38f2090fa HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth. 
(cherry picked from commit bcde1562d25c4f5595f4e3436dc3630315b1ceed)
 2016-05-20 12:47:52 -07:00
Jian He
6e0522ee1b YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi 
(cherry picked from commit d364ceac85622e99133b3eb3becef0c8188e6f89)
 2016-05-20 11:36:08 -07:00
Steve Loughran
eb160561e0 HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong. 2016-05-20 13:52:30 +01:00
Steve Loughran
e29d7fb585 HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth. 2016-05-20 12:21:47 +01:00
Varun Vasudev
e7f396fd47 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 
(cherry picked from commit 0287c491071c15e1c47dc640e6ea437be8cf32ae)
 2016-05-20 12:32:29 +05:30
Chris Nauroth
60e5397a9b HADOOP-13183. S3A proxy tests fail after httpclient/httpcore upgrade. Contributed by Steve Loughran. 
(cherry picked from commit 93258459faf56bc84121ba99d20eaef95273329e)
 2016-05-19 22:00:32 -07:00
Jian He
db3cf5b354 MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu 
(cherry picked from commit 21d2b90213e8e188bcac31f9360c5176ac89d083)
 2016-05-19 19:25:48 -07:00
Tsz-Wo Nicholas Sze
658072d62e HADOOP-13168. Support Future.get with timeout in ipc async calls. 2016-05-19 15:58:15 -07:00
Junping Du
2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Kihwal Lee
bae151801d HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu. 
(cherry picked from commit 204de430bf6a931e5688789c7a783c9c568a98c2)
 2016-05-19 15:53:41 -05:00
Jian He
2d846fdcbd YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca536e7deac50976b8a8774450fe089f)
 2016-05-19 13:02:05 -07:00
Allen Wittenauer
f7ffa1a534 HADOOP-13177. Native tests fail on OS X, because DYLD_LIBRARY_PATH is not defined to include libhadoop.dylib (Chris Nauroth via aw) 
(cherry picked from commit c7159f0240be0c555edf01d9a580987b0997125d)
 2016-05-19 09:58:48 -07:00
Junping Du
416274b53e YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong. 
(cherry picked from commit 141873ca7d68619e3458712b17aed57fec8fd7c8)
 2016-05-19 09:14:42 -07:00
Steve Loughran
151ecdf26d HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:44:44 +01:00
Arun Suresh
15808717f3 YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 1597630681c784a3d59f5605b87e96197b8139d7)
 2016-05-18 23:20:54 -07:00
Arun Suresh
a555a320e8 YARN-5090. Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster. (asuresh) 
(cherry picked from commit 8a9ecb7584b529be1de3a6e8850041e211c88ebc)
 2016-05-18 23:20:36 -07:00
Arun Suresh
f31602893e YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh) 
(cherry picked from commit ccc93e78127c14bfd84179395b055c4061ea436a)
 2016-05-18 23:20:08 -07:00
Arun Suresh
bbcea3abf3 YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh) 
(cherry picked from commit f45bc5a83ed2f2292ecf1f4c990b7153d8051fc9)
 2016-05-18 23:12:19 -07:00
Arun Suresh
69a9ce3494 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 
(cherry picked from commit f0ac18d001d97914a9ee810b1fab56c5cebff830)
 2016-05-18 23:09:11 -07:00
Arun Suresh
005dae521c YARN-5073. Refactor startContainerInternal() in ContainerManager to remove unused parameter. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d971bf252f36e902a0fe446cb11201835ad16bcb)
 2016-05-18 22:36:28 -07:00
Arun Suresh
1b4600abcf YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 
(cherry picked from commit 341888a0aa23f24458b4e6e34868794b9735c06a)
 2016-05-18 22:35:55 -07:00
Arun Suresh
307cda70db YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d464f4d1c4dec483852fc8c0496787cba0af8f57)
 2016-05-18 22:09:30 -07:00
Arun Suresh
d80d24aabf YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
(cherry picked from commit c282a08f3892e2e8ceb58e1e9a411062fbd1fb2b)
 2016-05-18 22:08:55 -07:00
Karthik Kambatla
b56fc51b70 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 
(cherry picked from commit c8172f5f143d2fefafa5a412899ab7cd081b406d)
 2016-05-18 22:08:20 -07:00
Arun Suresh
68964bbaa2 YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh) 
(cherry picked from commit 8ffabfdf4fec104cdaa68133385cfe9cf88b95a8)
(cherry picked from commit b2a654c5ee6524f81c971ea0b70e58ea0a455f1d)
 2016-05-18 22:02:43 -07:00
Karthik Kambatla
2c6c701aff YARN-2882. Add an OPPORTUNISTIC ExecutionType. (Konstantinos Karanasos and Inigo Goiri via kasha) 
(cherry picked from commit fb00794368e0aa7aafa9dfc8d453810f641b82b2)
 2016-05-18 22:02:28 -07:00
Andrew Wang
686691df60 HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder. 2016-05-18 14:55:31 -07:00
Andrew Wang
a44c87c2d0 HDFS-2173. saveNamespace should not throw IOE when only one storage directory fails to write VERSION file. Contributed by Andras Bokor. 
(cherry picked from commit 010e6ac328855bad59f138b6aeaec535272f448c)
 2016-05-18 14:13:47 -07:00
Robert Kanter
27aabcab45 MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter) 
(cherry picked from commit 992a49353f0bdbc71534bfeeb134112c5624b94b)
 2016-05-18 12:16:45 -07:00
Xiaoyu Yao
1470af8c95 HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu. 
(cherry picked from commit ccb1cade5b5f8421b907cafe147a4d610fb5fff4)
 2016-05-18 11:02:31 -07:00
Naganarasimha
a166286b9e YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt 
(cherry picked from commit f04c81c9ce93512bc714531a7731debbe6b794ce)
 2016-05-19 01:35:47 +08:00
Varun Vasudev
3c9392a31d YARN-4913. Yarn logs should take a -out option to write to a directory. Contributed by Xuan Gong. 
(cherry picked from commit ef1757790d89cc72f88f5330761b1c8901c59e94)
 2016-05-18 22:45:06 +05:30
Chris Nauroth
42f64b3bcc HADOOP-13138. Unable to append to a SequenceFile with Compression.NONE. Contributed by Vinayakumar B. 
(cherry picked from commit 0a527196dcb35a699e12121165bc3738766b569e)
 2016-05-18 09:32:15 -07:00
Arpit Agarwal
24bed7768d HDFS-9926. MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils. (Contributed by Josh Elser) 2016-05-18 09:21:41 -07:00
Colin Patrick Mccabe
feabfe9cda HADOOP-13140. FileSystem#initialize must not attempt to create StorageStatistics objects with null or empty schemes (Mingliang Liu via cmccabe) 
(cherry picked from commit f2c1d9181f80e53890b50309b15c9ea37cb24987)
 2016-05-18 09:19:45 -07:00
Akira Ajisaka
e9dd0d84e5 Revert "HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder." 
This reverts commit e205421555841dbf482d6aa1464cc15788b2b1fb.
 2016-05-18 17:57:46 +09:00
Andrew Wang
e205421555 HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder. 
(cherry picked from commit 7154ace71212e9fb9dd6209a92165fb075df7806)
 2016-05-17 17:44:34 -07:00