11678 Commits

Author SHA1 Message Date
Chris Nauroth
193f822b9a HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.
(cherry picked from commit 21890c4239b6a82fd6aab3454ce3922226efe7b5)
2016-05-27 14:20:30 -07:00
Chris Nauroth
03c6854c30 HADOOP-13193. Upgrade to Apache Yetus 0.3.0. Contributed by Kengo Seki.
(cherry picked from commit da074771977fe3de8acea441b096291c96cf59d9)
2016-05-27 14:10:06 -07:00
Arun Suresh
0b1472fbc0 YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh)
(cherry picked from commit aa975bc7811fc7c52b814ad9635bff8c2d34655b)
2016-05-27 14:07:15 -07:00
Yongjun Zhang
2dfaedeb43 HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang)
(cherry picked from commit 5ea6fd85c7aff6df28b87789f607bb57ee920639)
2016-05-27 10:14:30 -07:00
Akira Ajisaka
ec811909e4 YARN-5166. javadoc:javadoc goal fails on hadoop-yarn-client. Contributed by Andras Bokor.
(cherry picked from commit e4022debf717083ab9192164af9978500035d1be)
2016-05-27 23:15:48 +09:00
Junping Du
dea79a7dd6 YARN-4987. Read cache concurrency issue between read and evict in EntityGroupFS timeline store. Contributed by Li Lu.
(cherry picked from commit 705286ccaeea36941d97ec1c1700746b74264924)
2016-05-27 06:59:15 -07:00
Xuan
38ae5954df YARN-5089. Improve "yarn log" command-line "logFiles" option to support
regex. Contributed by Xuan Gong

(cherry picked from commit bde819abbbcea940cfa6426a9e4920e6c8dc9cf1)
2016-05-26 23:50:14 -07:00
Yongjun Zhang
82a88c7e67 HADOOP-13199. Add doc for distcp -filters. (John Zhuge via Yongjun Zhang)
(cherry picked from commit cfb860dee72a27382a26bf450bb8b16784aeebbb)
2016-05-26 23:32:46 -07:00
Yongjun Zhang
41c02757a5 HADOOP-12847. hadoop daemonlog should support https and SPNEGO for Kerberized cluster. (Wei-Chiu Chuang via Yongjun Zhang) 2016-05-26 22:54:07 -07:00
Rohith Sharma K S
ef3e0c6fd6 YARN-5005. TestRMWebServices#testDumpingSchedulerLogs fails randomly. Contributed by Bibin A Chundatt.
(cherry picked from commit 0a544f8a3e52f3467003a731d77380f56343d322)
2016-05-27 10:48:47 +05:30
Karthik Kambatla
59335b4d7a YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha)
(cherry picked from commit 04ded558b03ee0fbf68a611cf1f25508b4447e44)
2016-05-26 15:09:47 -07:00
Arun Suresh
44cbf5b7f5 YARN-5162. Fix Exceptions thrown during in registerAM call when Distributed Scheduling is Enabled (Hitesh Sharma via asuresh)
(cherry picked from commit 5b41b288d01b0124664ddf6a3d6b545058bcfe6f)
2016-05-26 14:58:15 -07:00
Karthik Kambatla
b4c8729cf9 HDFS-10463. TestRollingFileSystemSinkWithHdfs needs some cleanup. (Daniel Templeton via kasha) 2016-05-26 14:39:07 -07:00
Tsz-Wo Nicholas Sze
72773f8ea8 HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou 2016-05-26 12:16:40 -07:00
Kai Zheng
4c8ab5e162 HADOOP-12782. Faster LDAP group name resolution with ActiveDirectory. Contributed by Wei-Chiu Chuang 2016-05-27 17:16:32 +08:00
Karthik Kambatla
999cdd91b7 YARN-4866. FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy. (Yufei Gu via kasha)
(cherry picked from commit 4f513a4a8ed73beec89b7b28c0cd056ee5f4ed0d)
2016-05-25 22:13:42 -07:00
Carlo Curino
c656977961 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino)
(cherry picked from commit 013532a95e63d7c53e601be530021d6d5a15ab7f)
2016-05-25 16:58:19 -07:00
Jason Lowe
da77040793 YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong
(cherry picked from commit 1ba31fe9e906dbd093afd4b254216601967a4a7b)
2016-05-25 21:29:27 +00:00
Tsz-Wo Nicholas Sze
e5dab68066 HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma 2016-05-25 12:12:27 -07:00
Karthik Kambatla
0ac8c098e8 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha)
(cherry picked from commit 77d5ce9069b9ad64673f04edac9b1b1beddfd403)
2016-05-25 10:51:09 -07:00
Robert Kanter
71e5b36763 YARN-4766. NM should not aggregate logs older than the retention policy (haibochen via rkanter)
(cherry picked from commit e07519b8dbb96d73c48e910a4de12563c5c2f8aa)
2016-05-25 10:25:59 -07:00
Varun Vasudev
5361a91909 YARN-5115. Avoid setting CONTENT-DISPOSITION header in the container-logs web-service. Contributed by Xuan Gong.
(cherry picked from commit c52c15dd2df5050c6c3ddbe5ee708404f1920721)
2016-05-25 19:01:22 +05:30
Varun Vasudev
36f2ae0692 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli.
(cherry picked from commit f440a9d8c4a177bc5062d21d4b4bc4d9b2944344)
2016-05-25 19:01:22 +05:30
Junping Du
2e42bafdc8 YARN-5131. Distributed shell AM fails when extra container arrives during finishing. Contributed by Wangda Tan.
(cherry picked from commit 48c931331cc43970e31866732f9ac82ee806ee03)
2016-05-25 06:19:29 -07:00
Vinayakumar B
e28a51e988 HDFS-10217. show 'blockScheduled' tooltip in datanodes table.
(cherry picked from commit 2dab64da2ff50bbb7937aad2d376eacd4ed320aa)
2016-05-25 12:35:15 +05:30
Rohith Sharma K S
b9567b89b9 YARN-5024. TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers random failure. Contributed by Bibin A Chundatt
(cherry picked from commit 28bd63e92b2ef6717cd3dcbe04ac9d9ca5420cf4)
2016-05-25 10:24:59 +05:30
Jian He
396d222d7a MAPREDUCE-6703. Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers. Contributed by Arun Suresh
(cherry picked from commit ae353ea96993ec664090c5d84f6675c29d9f0f5f)
2016-05-24 19:48:11 -07:00
Andrew Wang
2ea17f4578 HADOOP-13198. Add support for OWASP's dependency-check. Contributed by Mike Yoder.
(cherry picked from commit 09b866fd45664ff977702b58b6338ce209729a97)
2016-05-24 18:29:48 -07:00
Karthik Kambatla
afe603d973 HDFS-9782. RollingFileSystemSink should have configurable roll interval. (Daniel Templeton via kasha)
(cherry picked from commit 57c31a3fef625f1ec609d7e8873d4941f7ed5cbc)
2016-05-24 16:34:45 -07:00
Naganarasimha
02323603e2 YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt
(cherry picked from commit edd716e99cb57e7804eb7d699b9456372f65c6bc)
2016-05-25 06:15:50 +08:00
Tsz-Wo Nicholas Sze
56745a0cf4 HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-05-24 14:52:50 -07:00
Tsz-Wo Nicholas Sze
076a73f207 HDFS-9365. Balaner does not work with the HDFS-6376 HA setup. 2016-05-24 13:08:04 -07:00
Karthik Kambatla
108db38788 YARN-4878. Expose scheduling policy and max running apps over JMX for Yarn queues. (Yufei Gu via kasha)
(cherry picked from commit f979d779e192ac05906b7eb880dfc8884d0078a0)
2016-05-24 10:54:33 -07:00
Arun Suresh
ce886ff2b1 YARN-4991. Fix ContainerRequest Constructor to set nodelabelExpression correctly. (Bibin A Chundatt via asuresh)
(cherry picked from commit a5fed8bbfeed9b58c59584afa5931eae635afa2f)
2016-05-24 05:12:03 -07:00
Naganarasimha
155f25061e YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt
(cherry picked from commit b4078bd17b41fbfff5a5c0bca5bf903a327826a7)
2016-05-24 08:14:53 +08:00
Karthik Kambatla
f33f0b697b YARN-4979. FSAppAttempt demand calculation considers demands at multiple locality levels different. (Zhihai Xu via kasha)
(cherry picked from commit 6d043aa4cf14bc88b1a19991e3bb0752499312b8)
2016-05-23 14:30:05 -07:00
Jason Lowe
7602070457 YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger
(cherry picked from commit ac954486c5102b8fbbc4229a0d3a512bcc7013c0)
2016-05-23 15:56:00 +00:00
Jason Lowe
e8bd2a5be6 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du
(cherry picked from commit d1df0266cf4e9ff0ec70813c156556ca4e74f791)
2016-05-23 15:16:52 +00:00
Junping Du
867cd2f5a2 YARN-5112. Excessive log warnings for directory permission issue on NM recovery. Contributed by Jian He. 2016-05-22 16:02:57 -07:00
Akira Ajisaka
a59f30272d MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki. 2016-05-23 07:25:52 +09:00
Akira Ajisaka
1fb74e569a YARN-5126. Remove CHANGES.txt from branch-2. (aajisaka) 2016-05-23 07:13:40 +09:00
Masatake Iwasaki
094bc727c5 HDFS-10439. Update setOwner doc in HdfsPermissionsGuide. Contributed by John Zhuge.
(cherry picked from commit 12eb5a7b11c36df5a57a3894a3a44bf47f47b954)
2016-05-22 19:15:28 +09:00
Steve Loughran
15f997cca6 HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran) 2016-05-21 14:19:56 +01:00
Chris Nauroth
e7b56cf93c HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth.
(cherry picked from commit 500e946729e1b85fdc0ce8475d8ce118ba886f46)
2016-05-20 13:01:59 -07:00
Chris Nauroth
d38f2090fa HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth.
(cherry picked from commit bcde1562d25c4f5595f4e3436dc3630315b1ceed)
2016-05-20 12:47:52 -07:00
Jian He
6e0522ee1b YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi
(cherry picked from commit d364ceac85622e99133b3eb3becef0c8188e6f89)
2016-05-20 11:36:08 -07:00
Steve Loughran
eb160561e0 HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong. 2016-05-20 13:52:30 +01:00
Steve Loughran
e29d7fb585 HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth. 2016-05-20 12:21:47 +01:00
Varun Vasudev
e7f396fd47 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong.
(cherry picked from commit 0287c491071c15e1c47dc640e6ea437be8cf32ae)
2016-05-20 12:32:29 +05:30
Chris Nauroth
60e5397a9b HADOOP-13183. S3A proxy tests fail after httpclient/httpcore upgrade. Contributed by Steve Loughran.
(cherry picked from commit 93258459faf56bc84121ba99d20eaef95273329e)
2016-05-19 22:00:32 -07:00