Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,460 Commits 326 Branches 371 Tags
Commit Graph

11460 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Tsz-Wo Nicholas Sze
e5dab68066 HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma 2016-05-25 12:12:27 -07:00
Karthik Kambatla
0ac8c098e8 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha) 
(cherry picked from commit 77d5ce9069b9ad64673f04edac9b1b1beddfd403)
 2016-05-25 10:51:09 -07:00
Robert Kanter
71e5b36763 YARN-4766. NM should not aggregate logs older than the retention policy (haibochen via rkanter) 
(cherry picked from commit e07519b8dbb96d73c48e910a4de12563c5c2f8aa)
 2016-05-25 10:25:59 -07:00
Varun Vasudev
5361a91909 YARN-5115. Avoid setting CONTENT-DISPOSITION header in the container-logs web-service. Contributed by Xuan Gong. 
(cherry picked from commit c52c15dd2df5050c6c3ddbe5ee708404f1920721)
 2016-05-25 19:01:22 +05:30
Varun Vasudev
36f2ae0692 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit f440a9d8c4a177bc5062d21d4b4bc4d9b2944344)
 2016-05-25 19:01:22 +05:30
Junping Du
2e42bafdc8 YARN-5131. Distributed shell AM fails when extra container arrives during finishing. Contributed by Wangda Tan. 
(cherry picked from commit 48c931331cc43970e31866732f9ac82ee806ee03)
 2016-05-25 06:19:29 -07:00
Vinayakumar B
e28a51e988 HDFS-10217. show 'blockScheduled' tooltip in datanodes table. 
(cherry picked from commit 2dab64da2ff50bbb7937aad2d376eacd4ed320aa)
 2016-05-25 12:35:15 +05:30
Rohith Sharma K S
b9567b89b9 YARN-5024. TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers random failure. Contributed by Bibin A Chundatt 
(cherry picked from commit 28bd63e92b2ef6717cd3dcbe04ac9d9ca5420cf4)
 2016-05-25 10:24:59 +05:30
Jian He
396d222d7a MAPREDUCE-6703. Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers. Contributed by Arun Suresh 
(cherry picked from commit ae353ea96993ec664090c5d84f6675c29d9f0f5f)
 2016-05-24 19:48:11 -07:00
Andrew Wang
2ea17f4578 HADOOP-13198. Add support for OWASP's dependency-check. Contributed by Mike Yoder. 
(cherry picked from commit 09b866fd45664ff977702b58b6338ce209729a97)
 2016-05-24 18:29:48 -07:00
Karthik Kambatla
afe603d973 HDFS-9782. RollingFileSystemSink should have configurable roll interval. (Daniel Templeton via kasha) 
(cherry picked from commit 57c31a3fef625f1ec609d7e8873d4941f7ed5cbc)
 2016-05-24 16:34:45 -07:00
Naganarasimha
02323603e2 YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt 
(cherry picked from commit edd716e99cb57e7804eb7d699b9456372f65c6bc)
 2016-05-25 06:15:50 +08:00
Tsz-Wo Nicholas Sze
56745a0cf4 HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-05-24 14:52:50 -07:00
Tsz-Wo Nicholas Sze
076a73f207 HDFS-9365. Balaner does not work with the HDFS-6376 HA setup. 2016-05-24 13:08:04 -07:00
Karthik Kambatla
108db38788 YARN-4878. Expose scheduling policy and max running apps over JMX for Yarn queues. (Yufei Gu via kasha) 
(cherry picked from commit f979d779e192ac05906b7eb880dfc8884d0078a0)
 2016-05-24 10:54:33 -07:00
Arun Suresh
ce886ff2b1 YARN-4991. Fix ContainerRequest Constructor to set nodelabelExpression correctly. (Bibin A Chundatt via asuresh) 
(cherry picked from commit a5fed8bbfeed9b58c59584afa5931eae635afa2f)
 2016-05-24 05:12:03 -07:00
Naganarasimha
155f25061e YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 
(cherry picked from commit b4078bd17b41fbfff5a5c0bca5bf903a327826a7)
 2016-05-24 08:14:53 +08:00
Karthik Kambatla
f33f0b697b YARN-4979. FSAppAttempt demand calculation considers demands at multiple locality levels different. (Zhihai Xu via kasha) 
(cherry picked from commit 6d043aa4cf14bc88b1a19991e3bb0752499312b8)
 2016-05-23 14:30:05 -07:00
Jason Lowe
7602070457 YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 
(cherry picked from commit ac954486c5102b8fbbc4229a0d3a512bcc7013c0)
 2016-05-23 15:56:00 +00:00
Jason Lowe
e8bd2a5be6 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 
(cherry picked from commit d1df0266cf4e9ff0ec70813c156556ca4e74f791)
 2016-05-23 15:16:52 +00:00
Junping Du
867cd2f5a2 YARN-5112. Excessive log warnings for directory permission issue on NM recovery. Contributed by Jian He. 2016-05-22 16:02:57 -07:00
Akira Ajisaka
a59f30272d MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki. 2016-05-23 07:25:52 +09:00
Akira Ajisaka
1fb74e569a YARN-5126. Remove CHANGES.txt from branch-2. (aajisaka) 2016-05-23 07:13:40 +09:00
Masatake Iwasaki
094bc727c5 HDFS-10439. Update setOwner doc in HdfsPermissionsGuide. Contributed by John Zhuge. 
(cherry picked from commit 12eb5a7b11c36df5a57a3894a3a44bf47f47b954)
 2016-05-22 19:15:28 +09:00
Steve Loughran
15f997cca6 HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran) 2016-05-21 14:19:56 +01:00
Chris Nauroth
e7b56cf93c HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth. 
(cherry picked from commit 500e946729e1b85fdc0ce8475d8ce118ba886f46)
 2016-05-20 13:01:59 -07:00
Chris Nauroth
d38f2090fa HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth. 
(cherry picked from commit bcde1562d25c4f5595f4e3436dc3630315b1ceed)
 2016-05-20 12:47:52 -07:00
Jian He
6e0522ee1b YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi 
(cherry picked from commit d364ceac85622e99133b3eb3becef0c8188e6f89)
 2016-05-20 11:36:08 -07:00
Steve Loughran
eb160561e0 HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong. 2016-05-20 13:52:30 +01:00
Steve Loughran
e29d7fb585 HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth. 2016-05-20 12:21:47 +01:00
Varun Vasudev
e7f396fd47 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 
(cherry picked from commit 0287c491071c15e1c47dc640e6ea437be8cf32ae)
 2016-05-20 12:32:29 +05:30
Chris Nauroth
60e5397a9b HADOOP-13183. S3A proxy tests fail after httpclient/httpcore upgrade. Contributed by Steve Loughran. 
(cherry picked from commit 93258459faf56bc84121ba99d20eaef95273329e)
 2016-05-19 22:00:32 -07:00
Jian He
db3cf5b354 MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu 
(cherry picked from commit 21d2b90213e8e188bcac31f9360c5176ac89d083)
 2016-05-19 19:25:48 -07:00
Tsz-Wo Nicholas Sze
658072d62e HADOOP-13168. Support Future.get with timeout in ipc async calls. 2016-05-19 15:58:15 -07:00
Junping Du
2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Kihwal Lee
bae151801d HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu. 
(cherry picked from commit 204de430bf6a931e5688789c7a783c9c568a98c2)
 2016-05-19 15:53:41 -05:00
Jian He
2d846fdcbd YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca536e7deac50976b8a8774450fe089f)
 2016-05-19 13:02:05 -07:00
Allen Wittenauer
f7ffa1a534 HADOOP-13177. Native tests fail on OS X, because DYLD_LIBRARY_PATH is not defined to include libhadoop.dylib (Chris Nauroth via aw) 
(cherry picked from commit c7159f0240be0c555edf01d9a580987b0997125d)
 2016-05-19 09:58:48 -07:00
Junping Du
416274b53e YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong. 
(cherry picked from commit 141873ca7d68619e3458712b17aed57fec8fd7c8)
 2016-05-19 09:14:42 -07:00
Steve Loughran
151ecdf26d HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:44:44 +01:00
Arun Suresh
15808717f3 YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 1597630681c784a3d59f5605b87e96197b8139d7)
 2016-05-18 23:20:54 -07:00
Arun Suresh
a555a320e8 YARN-5090. Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster. (asuresh) 
(cherry picked from commit 8a9ecb7584b529be1de3a6e8850041e211c88ebc)
 2016-05-18 23:20:36 -07:00
Arun Suresh
f31602893e YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh) 
(cherry picked from commit ccc93e78127c14bfd84179395b055c4061ea436a)
 2016-05-18 23:20:08 -07:00
Arun Suresh
bbcea3abf3 YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh) 
(cherry picked from commit f45bc5a83ed2f2292ecf1f4c990b7153d8051fc9)
 2016-05-18 23:12:19 -07:00
Arun Suresh
69a9ce3494 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 
(cherry picked from commit f0ac18d001d97914a9ee810b1fab56c5cebff830)
 2016-05-18 23:09:11 -07:00
Arun Suresh
005dae521c YARN-5073. Refactor startContainerInternal() in ContainerManager to remove unused parameter. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d971bf252f36e902a0fe446cb11201835ad16bcb)
 2016-05-18 22:36:28 -07:00
Arun Suresh
1b4600abcf YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 
(cherry picked from commit 341888a0aa23f24458b4e6e34868794b9735c06a)
 2016-05-18 22:35:55 -07:00
Arun Suresh
307cda70db YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d464f4d1c4dec483852fc8c0496787cba0af8f57)
 2016-05-18 22:09:30 -07:00
Arun Suresh
d80d24aabf YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
(cherry picked from commit c282a08f3892e2e8ceb58e1e9a411062fbd1fb2b)
 2016-05-18 22:08:55 -07:00
Karthik Kambatla
b56fc51b70 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 
(cherry picked from commit c8172f5f143d2fefafa5a412899ab7cd081b406d)
 2016-05-18 22:08:20 -07:00