Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,213 Commits 326 Branches 371 Tags
Commit Graph

11213 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jian He
3c2bd19fa5 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29794f8e61bebe03b2a7ca4b6743ec92)
 2016-06-03 11:11:49 -07:00
Jian He
01a3f7899c YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan 
(cherry picked from commit 097baaaebae021c47bb7d69aa1ff1a2440df5166)
 2016-06-03 10:50:12 -07:00
Chris Nauroth
b8216c10d8 HADOOP-13171. Add StorageStatistics to S3A; instrument some more operations. Contributed by Steve Loughran. 2016-06-03 08:56:36 -07:00
Akira Ajisaka
d0dc5aaa2d HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (aajisaka) 
(cherry picked from commit b6d5546e2414009311ca7e92203b7b4a6b29e165)
 2016-06-03 16:04:40 +09:00
Akira Ajisaka
a8a2f4b500 HDFS-10471. DFSAdmin#SetQuotaCommand's help msg is not correct. Contributed by Yiqun Lin. 
(cherry picked from commit 1df6f5735c9d85e644d99d3ebfc4459490657004)
(cherry picked from commit cddf6b4fc7c2abe93355716c380c05495f4aeb4c)
 2016-06-03 04:11:13 +09:00
Masatake Iwasaki
5b3d9ca0cd HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit aadb77e412ab9d4ad05a0bd8b37d547ba5adad03)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java

(cherry picked from commit 37409744135186a9b90865436a731366b259ea81)
 2016-06-02 21:04:48 +09:00
Akira Ajisaka
f7e1a1eaa7 HADOOP-13232. Typo in exception in ValueQueue.java. Contributed by Jiayi Zhou. 
(cherry picked from commit 99675e00df65e84dcc02082b795f1f93c90b794e)
(cherry picked from commit 6843e552087d77bade52599a10c59ff55e1e3ed7)
 2016-06-02 19:15:57 +09:00
Akira Ajisaka
fe4ad4a39c HDFS-9476. TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail. Contributed by Masatake Iwasaki. 
(cherry picked from commit 69555fca066815053dd9168ebe15868a5c02cdcd)
(cherry picked from commit 3dd557e4b6b852ee99f6acc29cca1088dc3a39ff)
 2016-06-02 18:53:42 +09:00
Chris Nauroth
3ab0e8d500 HADOOP-13131. Add tests to verify that S3A supports SSE-S3 encryption. Contributed by Steve Loughran. 
(cherry picked from commit 16b1cc7af9bd63b65ef50e1056f275a7baf111a2)
(cherry picked from commit e9942c5b3439ec5d13fd6b3f42e028ebee21727e)
 2016-06-01 14:49:41 -07:00
Andrew Wang
4ad9ca85fb HADOOP-13228. Add delegation token to the connection in DelegationTokenAuthenticator. Contributed by Xiao Chen. 
(cherry picked from commit 35356de1ba1cad0fa469ff546263290109c61b77)
(cherry picked from commit 55eda958ccbf3941986d6da63ad7ee16646d7067)
 2016-06-01 13:13:38 -07:00
Steve Loughran
809e359d2d HADOOP-13162. Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs. (Rajesh Balamohan via stevel) 2016-06-01 14:17:18 +01:00
Wangda Tan
2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f75c796413ce860b1859a31cc5292ca.
 2016-05-31 22:16:53 -07:00
Colin Patrick Mccabe
6fe1c67fa1 HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe) 
(cherry picked from commit 8ceb06e2392763726210f96bb1c176e6a9fe7b53)
(cherry picked from commit a64e1425ed12edc7434b2f40d111111caf1db274)
 2016-05-31 18:26:37 -07:00
Colin Patrick Mccabe
0d6a65a40a HDFS-10415. TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called (Mingliang Liu via cmccabe) 
(cherry picked from commit 14907d9b8cf5ad7a8241aa66db79cc8e79767cbe)
 2016-05-31 17:27:14 -07:00
Tsz-Wo Nicholas Sze
a8941d7790 HADOOP-13226 Support async call retry and failover. 2016-05-31 17:16:44 -07:00
Jing Zhao
03c4491d89 HADOOP-13146. Refactor RetryInvocationHandler. Contributed by Tsz Wo Nicholas Sze. 2016-05-31 17:16:26 -07:00
Andrew Wang
63f73d78a1 HADOOP-13132. Handle ClassCastException on AuthenticationException in LoadBalancingKMSClientProvider. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit bca31fe276ccf7d02b13f25d43c81cca0b0b905b)
 2016-05-31 09:35:30 -07:00
Varun Vasudev
457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Karthik Kambatla
e5b1fb2a91 YARN-5132. Exclude generated protobuf sources from YARN Javadoc build. (Subru Krishnan via kasha) 
(cherry picked from commit f5ff05cc8a67c125717261392c996136ba66785b)
(cherry picked from commit b3f5337bb3f773777aa4cca822318b64fb2465f5)
 2016-05-28 08:12:56 -07:00
Xiaoyu Yao
1d2928ece7 HADOOP-13197. Add non-decayed call metrics for DecayRpcScheduler. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4ca8859583839761663fc1fc1de1b3ce2e3fc5b5)
(cherry picked from commit 001c8f5d7b54812bb992c7d73db0a08cab2267ec)
 2016-05-27 18:51:46 -07:00
Chris Nauroth
de69b1da34 HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou. 
(cherry picked from commit 21890c4239b6a82fd6aab3454ce3922226efe7b5)
(cherry picked from commit 193f822b9a6e95575e90e2703e30abde70c6ba6c)
 2016-05-27 14:20:50 -07:00
Chris Nauroth
8baec07689 HADOOP-13193. Upgrade to Apache Yetus 0.3.0. Contributed by Kengo Seki. 
(cherry picked from commit da074771977fe3de8acea441b096291c96cf59d9)
(cherry picked from commit 03c6854c3064b2035b3ba3b1c5f5fd0d76c495f8)
 2016-05-27 14:10:28 -07:00
Yongjun Zhang
3e4c7906c2 HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang) 
(cherry picked from commit 5ea6fd85c7aff6df28b87789f607bb57ee920639)
 2016-05-27 10:18:47 -07:00
Akira Ajisaka
01e7c2bbe4 YARN-5166. javadoc:javadoc goal fails on hadoop-yarn-client. Contributed by Andras Bokor. 
(cherry picked from commit e4022debf717083ab9192164af9978500035d1be)
(cherry picked from commit ec811909e41cdea84a0d9c9254ed7c756cbae37b)
 2016-05-27 23:16:00 +09:00
Junping Du
6b41fceb69 YARN-4987. Read cache concurrency issue between read and evict in EntityGroupFS timeline store. Contributed by Li Lu. 
(cherry picked from commit 705286ccaeea36941d97ec1c1700746b74264924)
(cherry picked from commit dea79a7dd66775788e73f962a1c68fe261684799)
 2016-05-27 07:03:18 -07:00
Yongjun Zhang
3fc4d4a369 HADOOP-13199. Add doc for distcp -filters. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit cfb860dee72a27382a26bf450bb8b16784aeebbb)
 2016-05-26 23:34:32 -07:00
Karthik Kambatla
427e3f995e YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha) 
(cherry picked from commit 04ded558b03ee0fbf68a611cf1f25508b4447e44)
(cherry picked from commit 59335b4d7a969378cb765b000ba1e13dabc44a3a)
 2016-05-26 16:54:38 -07:00
Tsz-Wo Nicholas Sze
2e755a7f0e HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou 2016-05-26 12:24:21 -07:00
Tsz-Wo Nicholas Sze
339b803a52 HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-05-26 12:23:05 -07:00
Tsz-Wo Nicholas Sze
fa7c7f2510 HADOOP-13168. Support Future.get with timeout in ipc async calls. 2016-05-26 12:22:57 -07:00
Tsz-Wo Nicholas Sze
ac04900450 HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-05-26 12:20:31 -07:00
Carlo Curino
7982933c09 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e63d7c53e601be530021d6d5a15ab7f)
(cherry picked from commit c656977961e2ba0f9dfd349ed59bf1d0d41c57f5)
 2016-05-25 17:02:22 -07:00
Jason Lowe
fe10caee8d YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong 
(cherry picked from commit 1ba31fe9e906dbd093afd4b254216601967a4a7b)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2016-05-25 21:37:31 +00:00
Tsz-Wo Nicholas Sze
f6dfb71762 HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma 2016-05-25 12:26:25 -07:00
Karthik Kambatla
b491d24bb6 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha) 
(cherry picked from commit 77d5ce9069b9ad64673f04edac9b1b1beddfd403)
(cherry picked from commit 0ac8c098e877e56f3da8e66e456e989060790116)
 2016-05-25 10:53:01 -07:00
Varun Vasudev
35456bb7c9 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit f440a9d8c4a177bc5062d21d4b4bc4d9b2944344)
(cherry picked from commit 36f2ae0692d73a865a5c0c520d1346b6d4498c25)
 2016-05-25 19:03:33 +05:30
Vinayakumar B
00011e6a98 HDFS-10217. show 'blockScheduled' tooltip in datanodes table. 
(cherry picked from commit 2dab64da2ff50bbb7937aad2d376eacd4ed320aa)
(cherry picked from commit e28a51e988beb27741c801641be7d96faaf6b191)
 2016-05-25 12:35:42 +05:30
Andrew Wang
4b40d7d21f HADOOP-13198. Add support for OWASP's dependency-check. Contributed by Mike Yoder. 
(cherry picked from commit 09b866fd45664ff977702b58b6338ce209729a97)
(cherry picked from commit 2ea17f4578fcada8c1bcc5e97f085ca878ec5ec4)
 2016-05-24 18:29:51 -07:00
Naganarasimha
943cefc81e YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt 2016-05-25 06:27:44 +08:00
Tsz-Wo Nicholas Sze
2ab789b19e HDFS-9365. Balaner does not work with the HDFS-6376 HA setup. 2016-05-24 13:11:32 -07:00
Naganarasimha
abab31b8f2 YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 
(cherry picked from commit b4078bd17b41fbfff5a5c0bca5bf903a327826a7)
 2016-05-24 08:19:26 +08:00
Jason Lowe
0bfad9675e YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 
(cherry picked from commit ac954486c5102b8fbbc4229a0d3a512bcc7013c0)
 2016-05-23 15:56:33 +00:00
Jason Lowe
8e404b4321 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 
(cherry picked from commit d1df0266cf4e9ff0ec70813c156556ca4e74f791)
 2016-05-23 15:17:26 +00:00
Akira Ajisaka
4422beab72 MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki. 
(cherry picked from commit a59f30272dd2a8901a41c5c4300ce9d196e62606)
 2016-05-23 07:26:14 +09:00
Masatake Iwasaki
1ab79e255c HDFS-10439. Update setOwner doc in HdfsPermissionsGuide. Contributed by John Zhuge. 
(cherry picked from commit 12eb5a7b11c36df5a57a3894a3a44bf47f47b954)
 2016-05-22 19:43:47 +09:00
Steve Loughran
5930e813c5 HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth. 2016-05-21 19:10:16 +01:00
Steve Loughran
f2aef95bd3 HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran) 2016-05-21 14:20:19 +01:00
Chris Nauroth
5b907a17ed HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth. 
(cherry picked from commit 500e946729e1b85fdc0ce8475d8ce118ba886f46)
(cherry picked from commit e7b56cf93c7ac269e40b556875f16d0c5ad2f819)
 2016-05-20 13:02:06 -07:00
Chris Nauroth
877af3ec7e HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth. 
(cherry picked from commit bcde1562d25c4f5595f4e3436dc3630315b1ceed)
(cherry picked from commit d38f2090fa998b716a5d7a8d2314927d6eb30e41)
 2016-05-20 12:50:23 -07:00
Jian He
dbde956bf5 YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi 
(cherry picked from commit d364ceac85622e99133b3eb3becef0c8188e6f89)
(cherry picked from commit 6e0522ee1b46b262d031eb7eddd1e7e1aa3ca736)
 2016-05-20 11:36:40 -07:00