Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,288 Commits 326 Branches 371 Tags
Commit Graph

11288 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arun Suresh
934bd8989b YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh) 
(cherry picked from commit 7a9b7372a1a917c7b5e1beca7e13c0419e3dbfef)
(cherry picked from commit f0a869b52a4e4ad7e02143a7e703700a4f4b1f88)
 2016-06-06 21:08:52 -07:00
Zhe Zhang
0274636529 HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster. 
(cherry picked from commit 6de9213df111a9a4ed875db995d67af72d08a798)
(cherry picked from commit 06e38c835d0ad9619d4bea662f2dd7d0f62007a9)
 2016-06-06 15:54:36 -07:00
Ming Ma
ec4f9a14f9 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5)
(cherry picked from commit 74e2b5efa26f27027fed212b4b2108f0e95587fb)
 2016-06-06 14:49:43 -07:00
Steve Loughran
f9478c95bd HADOOP-12807 S3AFileSystem should read AWS credentials from environment variables. Contributed by Tobin Baker. 2016-06-06 23:40:49 +02:00
Tsz-Wo Nicholas Sze
49535c0683 Revert "Revert "HADOOP-13226 Support async call retry and failover."" 
This reverts commit 886e2396062ef076b3ea4aa93395cc90287efd4f.
 2016-06-06 16:54:02 +08:00
Tsz-Wo Nicholas Sze
359a316736 Revert "Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou."" 
This reverts commit f176311366eab0364458d055f248fb5796bdec23.
 2016-06-06 16:53:56 +08:00
Tsz-Wo Nicholas Sze
025f62afcc Revert "Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou"" 
This reverts commit 0af02cfd37a564b4a0701dbc6d562fe237565bd5.
 2016-06-06 16:53:51 +08:00
Tsz-Wo Nicholas Sze
79beac49ec Revert "Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou"" 
This reverts commit 1de712f22abbe313692cca6bece7f265da96aee5.
 2016-06-06 16:53:46 +08:00
Tsz-Wo Nicholas Sze
c949001eee Revert "Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls."" 
This reverts commit 308d28640d8c797f07aa41a8adc5dcd9420f1d0a.
 2016-06-06 16:53:41 +08:00
Tsz-Wo Nicholas Sze
5b054b1bfc Revert "Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou"" 
This reverts commit ffa85f28a2ec612a3d63c890806a17cfd3a2e7e2.
 2016-06-06 16:53:36 +08:00
Tsz-Wo Nicholas Sze
8ccc8aeb6c Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"" 
This reverts commit 8a07026e7792e43e692409c3ec998a306b7733ff.
 2016-06-06 16:53:31 +08:00
Tsz-Wo Nicholas Sze
2cf663bb59 Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"" 
This reverts commit d740a90260d9d26a67688b004e5fcda60cc30723.
 2016-06-06 16:53:24 +08:00
Andrew Wang
d740a90260 Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou" 
This reverts commit bdc45bef646cafdc04a59c19e23dcba3bb16b20c.
 2016-06-03 18:17:33 -07:00
Andrew Wang
8a07026e77 Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou" 
This reverts commit a1ba6eee38420ae9436f9e9ecc3db83702fbe762.
 2016-06-03 18:17:33 -07:00
Andrew Wang
ffa85f28a2 Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou" 
This reverts commit ac049004507b8df5d22694300fa845427d757866.
 2016-06-03 18:12:31 -07:00
Andrew Wang
308d28640d Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls." 
This reverts commit fa7c7f25105bc157c652a2f44ac49620fc61c0f4.
 2016-06-03 18:12:31 -07:00
Andrew Wang
1de712f22a Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou" 
This reverts commit 339b803a52ed5e867b8c405a7e7a92adcfe6c4df.
 2016-06-03 18:12:31 -07:00
Andrew Wang
0af02cfd37 Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou" 
This reverts commit 2e755a7f0ebb05d7f79ec30aba2d6b288936dacc.
 2016-06-03 18:12:31 -07:00
Andrew Wang
f176311366 Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou." 
This reverts commit de69b1da34ed419946b56f63efa82a6162788df9.
 2016-06-03 18:12:30 -07:00
Andrew Wang
886e239606 Revert "HADOOP-13226 Support async call retry and failover." 
This reverts commit a8941d7790b2209ac779c9372298b833ededd132.
 2016-06-03 18:12:30 -07:00
Andrew Wang
6f69113417 HDFS-10481. HTTPFS server should correctly impersonate as end user to open file. Contributed by Xiao Chen. 
(cherry picked from commit 47e0321ee91149331e6ae72e7caa41d1de078b6c)
(cherry picked from commit d5609e34995edf08624d8c3ce8ce6630a7713e79)
 2016-06-03 17:21:34 -07:00
Andrew Wang
d712b2ee3b HADOOP-13155. Implement TokenRenewer to renew and cancel delegation tokens in KMS. Contributed by Xiao Chen. 
(cherry picked from commit 713cb71820ad94a5436f35824d07aa12fcba5cc6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyProviderDelegationTokenExtension.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java

(cherry picked from commit 03c4724c884b133eeb1114e417f7d49d239f4783)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java
 2016-06-03 17:03:40 -07:00
Chris Nauroth
e54f073cfe HADOOP-13105. Support timeouts in LDAP queries in LdapGroupsMapping. Contributed by Mingliang Liu. 
(cherry picked from commit d82bc8501869be78780fc09752dbf7af918c14af)
(cherry picked from commit 5e6ee5aafb9b9f200d906444e4731cfc60ac6ebb)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMapping.java
 2016-06-03 16:40:48 -07:00
Arun Suresh
6963d50e2f YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh) 
(cherry picked from commit db54670e83a84c1d7deff2c225725687cf9e5f14)
(cherry picked from commit 8ebd89dcfb358cd973a1e71ea4429d0abb58b536)
 2016-06-03 14:52:45 -07:00
Vinod Kumar Vavilapalli
c87b9c1471 YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He. 
Made one minor edit for branch-2 patch.

(cherry picked from commit f10ebc67f57a4a2e3cc916c41154ab9b6a4635c9)
(cherry picked from commit 029888871fb13da75872ab345eb03d8f4e502ad4)
 2016-06-03 13:15:44 -07:00
Jian He
3c2bd19fa5 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29794f8e61bebe03b2a7ca4b6743ec92)
 2016-06-03 11:11:49 -07:00
Jian He
01a3f7899c YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan 
(cherry picked from commit 097baaaebae021c47bb7d69aa1ff1a2440df5166)
 2016-06-03 10:50:12 -07:00
Chris Nauroth
b8216c10d8 HADOOP-13171. Add StorageStatistics to S3A; instrument some more operations. Contributed by Steve Loughran. 2016-06-03 08:56:36 -07:00
Akira Ajisaka
d0dc5aaa2d HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (aajisaka) 
(cherry picked from commit b6d5546e2414009311ca7e92203b7b4a6b29e165)
 2016-06-03 16:04:40 +09:00
Akira Ajisaka
a8a2f4b500 HDFS-10471. DFSAdmin#SetQuotaCommand's help msg is not correct. Contributed by Yiqun Lin. 
(cherry picked from commit 1df6f5735c9d85e644d99d3ebfc4459490657004)
(cherry picked from commit cddf6b4fc7c2abe93355716c380c05495f4aeb4c)
 2016-06-03 04:11:13 +09:00
Masatake Iwasaki
5b3d9ca0cd HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit aadb77e412ab9d4ad05a0bd8b37d547ba5adad03)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java

(cherry picked from commit 37409744135186a9b90865436a731366b259ea81)
 2016-06-02 21:04:48 +09:00
Akira Ajisaka
f7e1a1eaa7 HADOOP-13232. Typo in exception in ValueQueue.java. Contributed by Jiayi Zhou. 
(cherry picked from commit 99675e00df65e84dcc02082b795f1f93c90b794e)
(cherry picked from commit 6843e552087d77bade52599a10c59ff55e1e3ed7)
 2016-06-02 19:15:57 +09:00
Akira Ajisaka
fe4ad4a39c HDFS-9476. TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail. Contributed by Masatake Iwasaki. 
(cherry picked from commit 69555fca066815053dd9168ebe15868a5c02cdcd)
(cherry picked from commit 3dd557e4b6b852ee99f6acc29cca1088dc3a39ff)
 2016-06-02 18:53:42 +09:00
Chris Nauroth
3ab0e8d500 HADOOP-13131. Add tests to verify that S3A supports SSE-S3 encryption. Contributed by Steve Loughran. 
(cherry picked from commit 16b1cc7af9bd63b65ef50e1056f275a7baf111a2)
(cherry picked from commit e9942c5b3439ec5d13fd6b3f42e028ebee21727e)
 2016-06-01 14:49:41 -07:00
Andrew Wang
4ad9ca85fb HADOOP-13228. Add delegation token to the connection in DelegationTokenAuthenticator. Contributed by Xiao Chen. 
(cherry picked from commit 35356de1ba1cad0fa469ff546263290109c61b77)
(cherry picked from commit 55eda958ccbf3941986d6da63ad7ee16646d7067)
 2016-06-01 13:13:38 -07:00
Steve Loughran
809e359d2d HADOOP-13162. Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs. (Rajesh Balamohan via stevel) 2016-06-01 14:17:18 +01:00
Wangda Tan
2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f75c796413ce860b1859a31cc5292ca.
 2016-05-31 22:16:53 -07:00
Colin Patrick Mccabe
6fe1c67fa1 HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe) 
(cherry picked from commit 8ceb06e2392763726210f96bb1c176e6a9fe7b53)
(cherry picked from commit a64e1425ed12edc7434b2f40d111111caf1db274)
 2016-05-31 18:26:37 -07:00
Colin Patrick Mccabe
0d6a65a40a HDFS-10415. TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called (Mingliang Liu via cmccabe) 
(cherry picked from commit 14907d9b8cf5ad7a8241aa66db79cc8e79767cbe)
 2016-05-31 17:27:14 -07:00
Tsz-Wo Nicholas Sze
a8941d7790 HADOOP-13226 Support async call retry and failover. 2016-05-31 17:16:44 -07:00
Jing Zhao
03c4491d89 HADOOP-13146. Refactor RetryInvocationHandler. Contributed by Tsz Wo Nicholas Sze. 2016-05-31 17:16:26 -07:00
Andrew Wang
63f73d78a1 HADOOP-13132. Handle ClassCastException on AuthenticationException in LoadBalancingKMSClientProvider. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit bca31fe276ccf7d02b13f25d43c81cca0b0b905b)
 2016-05-31 09:35:30 -07:00
Varun Vasudev
457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Karthik Kambatla
e5b1fb2a91 YARN-5132. Exclude generated protobuf sources from YARN Javadoc build. (Subru Krishnan via kasha) 
(cherry picked from commit f5ff05cc8a67c125717261392c996136ba66785b)
(cherry picked from commit b3f5337bb3f773777aa4cca822318b64fb2465f5)
 2016-05-28 08:12:56 -07:00
Xiaoyu Yao
1d2928ece7 HADOOP-13197. Add non-decayed call metrics for DecayRpcScheduler. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4ca8859583839761663fc1fc1de1b3ce2e3fc5b5)
(cherry picked from commit 001c8f5d7b54812bb992c7d73db0a08cab2267ec)
 2016-05-27 18:51:46 -07:00
Chris Nauroth
de69b1da34 HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou. 
(cherry picked from commit 21890c4239b6a82fd6aab3454ce3922226efe7b5)
(cherry picked from commit 193f822b9a6e95575e90e2703e30abde70c6ba6c)
 2016-05-27 14:20:50 -07:00
Chris Nauroth
8baec07689 HADOOP-13193. Upgrade to Apache Yetus 0.3.0. Contributed by Kengo Seki. 
(cherry picked from commit da074771977fe3de8acea441b096291c96cf59d9)
(cherry picked from commit 03c6854c3064b2035b3ba3b1c5f5fd0d76c495f8)
 2016-05-27 14:10:28 -07:00
Yongjun Zhang
3e4c7906c2 HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang) 
(cherry picked from commit 5ea6fd85c7aff6df28b87789f607bb57ee920639)
 2016-05-27 10:18:47 -07:00
Akira Ajisaka
01e7c2bbe4 YARN-5166. javadoc:javadoc goal fails on hadoop-yarn-client. Contributed by Andras Bokor. 
(cherry picked from commit e4022debf717083ab9192164af9978500035d1be)
(cherry picked from commit ec811909e41cdea84a0d9c9254ed7c756cbae37b)
 2016-05-27 23:16:00 +09:00
Junping Du
6b41fceb69 YARN-4987. Read cache concurrency issue between read and evict in EntityGroupFS timeline store. Contributed by Li Lu. 
(cherry picked from commit 705286ccaeea36941d97ec1c1700746b74264924)
(cherry picked from commit dea79a7dd66775788e73f962a1c68fe261684799)
 2016-05-27 07:03:18 -07:00