Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,080 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

12080 Commits

Author SHA1 Message Date
Jason Lowe 0a405c4f71 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMergeManager.java
 2016-10-28 14:40:28 +00:00
Naganarasimha 8a664eba7d MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af)
 2016-10-28 06:04:03 +05:30
Erik Krogen 2ec90a1a4b HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen. 
(cherry picked from commit f3ac1f41b8)
 2016-10-27 16:04:58 -07:00
Carlo Curino b32f14d806 YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 
(cherry picked from commit 892a8348fc)
(cherry picked from commit 58ac40b552)
 2016-10-27 15:23:04 -07:00
Subru Krishnan 55ba220721 YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 
(cherry picked from commit 79ae78dcbe)
(cherry picked from commit 4274600b95)
 2016-10-27 15:19:01 -07:00
Wei-Chiu Chuang bc2f429c0e HDFS-11055. Update default-log4j.properties for httpfs to imporve test logging. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 31ff42b510)
(cherry picked from commit 2cf3138d72)
 2016-10-27 13:47:26 -07:00
Jason Lowe d21dc8aa6c YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li 
(cherry picked from commit 7e3c327d31)
 2016-10-27 20:45:18 +00:00
Kihwal Lee 40dbf2a18d HDFS-11069. Tighten the authorization of datanode RPC. Contributed by Kihwal Lee 
(cherry picked from commit ae48c496dc)
 2016-10-27 14:18:48 -05:00
Kihwal Lee 5b0d32b4aa Revert "HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu" 
This reverts commit d6dfc76dfe.
 2016-10-27 14:10:52 -05:00
Brahma Reddy Battula d6dfc76dfe HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu 
(cherry picked from commit ac35ee9393)
 2016-10-27 20:23:07 +05:30
Brahma Reddy Battula 26cf988c3a HADOOP-13201. Print the directory paths when ViewFs denies the rename operation on internal dirs. Contributed by Tianyin Xiu 
(cherry picked from commit 0c837db8a8)
 2016-10-27 19:51:05 +05:30
Steve Loughran 5275d37495 Revert "HADOOP-13514. Upgrade maven surefire plugin to 2.19.1. Contributed by Ewan Higgs." 
This reverts commit 444312b54b.
 2016-10-27 15:21:29 +02:00
Steve Loughran ca6f0a1d77 Revert "Addendum patch for HADOOP-13514 Upgrade maven surefire plugin to 2.19.1. Contributed by Akira Ajisaka." 
This reverts commit 9a98feb769.
 2016-10-27 15:21:17 +02:00
Naganarasimha 27029393b7 MAPREDUCE-6541. Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock. Contributed by Varun Saxena 
(cherry picked from commit 060558c6f2)
 2016-10-27 18:06:34 +05:30
Naganarasimha 3b9df97158 YARN-3848. TestNodeLabelContainerAllocation is timing out. Contributed by Varun Saxena 
(cherry picked from commit 6c8830992c)
 2016-10-27 17:16:34 +05:30
Vinayakumar B 7279ce831d HDFS-8492. DN should notify NN when client requests a missing block (Contributed by Walter Su) 
(cherry picked from commit 1cf6ec4ad4)
(cherry picked from commit 144f7a9e0e)
 2016-10-27 16:56:25 +05:30
Masatake Iwasaki 6d9b6fac86 HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran. 
(cherry picked from commit 0bdd263d82)
 2016-10-27 15:55:35 +09:00
Mingliang Liu a7684f6517 HDFS-10921. TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode. Contributed by Eric Badger 
(cherry picked from commit 55e1fb8e32)
 2016-10-26 14:18:00 -07:00
Wei-Chiu Chuang 9a98feb769 Addendum patch for HADOOP-13514 Upgrade maven surefire plugin to 2.19.1. Contributed by Akira Ajisaka. 
(cherry picked from commit e48b592f8b)
(cherry picked from commit 0c96ceaca9)
 2016-10-26 14:10:26 -07:00
Chris Nauroth 4df1780991 HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth. 
(cherry picked from commit 1f8490a5ba)
(cherry picked from commit 082d69ee66)
 2016-10-26 08:57:46 -07:00
Chris Nauroth c940c68c79 HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran. 
(cherry picked from commit 9cad3e2350)
(cherry picked from commit 67e01f7218)
 2016-10-26 08:27:39 -07:00
Kihwal Lee e7bd2e8d63 HDFS-11053. Unnecessary superuser check in versionRequest(). Contributed by Kihwal Lee. 
(cherry picked from commit e90af4a89b)
 2016-10-26 09:18:01 -05:00
Kihwal Lee 49adab66b8 HDFS-11050. Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR. Contributed by Kuhu Shukla. 
(cherry picked from commit ce6bbfb23c)
 2016-10-26 08:11:51 -05:00
Arpit Agarwal df83d86da7 HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. Contributed by Chen Liang. 2016-10-25 19:51:53 -07:00
Xiaoyu Yao e0fc749a46 HDFS-11011. Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'. Contributed by Xiaobing Zhou. 
(cherry picked from commit 85f6fec0370b3cd94d6c2f19920c0b6d33f127c6)
(cherry picked from commit c2ba4b650168be4fad7ffdb546b8a0d985268393)
 2016-10-25 11:22:07 -07:00
Zhe Zhang 262518fa5b HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee. 
(cherry picked from commit f6367c5f44)
(cherry picked from commit ff806cbfc7)
 2016-10-25 10:59:03 -07:00
Xiao Chen a8aabf9be6 HADOOP-13669. Addendum patch 2 for KMS Server should log exceptions before throwing. 
(cherry picked from commit 09ef97dccb)
 2016-10-25 09:59:48 -07:00
Chris Nauroth 3d5f41544a HADOOP-13309. Document S3A known limitations in file ownership and permission model. Contributed by Chris Nauroth. 
(cherry picked from commit 309a43925c)
(cherry picked from commit 05d772e297)
 2016-10-25 09:03:19 -07:00
Akira Ajisaka 444312b54b HADOOP-13514. Upgrade maven surefire plugin to 2.19.1. Contributed by Ewan Higgs. 
(cherry picked from commit dbd205762e)
(cherry picked from commit 6bb23a14b6)
 2016-10-25 15:58:57 +09:00
Chris Nauroth 02f0472d1e HADOOP-12774. s3a should use UGI.getCurrentUser.getShortname() for username. Contributed by Steve Loughran. 
(cherry picked from commit 3372e94030)
(cherry picked from commit 5c2f67bdae)
 2016-10-24 21:54:25 -07:00
Chris Nauroth 78970e0db3 HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. Contributed by Chris Nauroth. 
(cherry picked from commit d8fa1cfa67)
(cherry picked from commit 5b7cbb5a3c)
 2016-10-24 21:23:00 -07:00
Kihwal Lee c82910203c HDFS-11042. Add missing cleanupSSLConfig() call for tests that use setupSSLConfig(). Contributed by Kuhu Shukla. 
(cherry picked from commit 086577c67a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/TestHttpFSFWithSWebhdfsFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java
 2016-10-24 09:00:28 -05:00
Jian He dab611e605 Revert YARN-4126. RM should not issue delegation tokens in unsecure mode. 
(cherry picked from commit bb8214cab8)
 2016-10-23 16:56:12 -07:00
Xiaoyu Yao 7d826631ea HADOOP-13749. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 
(cherry picked from commit d0a347984d)
 2016-10-23 10:59:10 -07:00
Xiaoyu Yao 751568e1d3 Revert "HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao." 
This reverts commit b82364759c.
 2016-10-23 08:30:20 -07:00
Benoy Antony 653ceab7d3 HADOOP-12082 Support multiple authentication schemes via AuthenticationFilter. Contributed by Hrishikesh Gadre. 2016-10-21 19:43:22 -07:00
Xiaoyu Yao b82364759c HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 
(cherry picked from commit be72372248)
 2016-10-21 15:05:57 -07:00
Xiao Chen 7d19394bd5 HADOOP-13381. KMS clients should use KMS Delegation Tokens from current UGI. Contributed by Xiao Chen. 
(cherry picked from commit 8ebf2e95d2)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java

(cherry picked from commit 8fe4b2429a)
 2016-10-21 14:59:56 -07:00
Steve Loughran 4b56954fea HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-21 19:12:03 +01:00
Brahma Reddy Battula 5754772bd6 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc03)
 2016-10-21 16:22:42 +05:30
Mingliang Liu cfac82a443 HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou 
(cherry picked from commit d7d87deece)
 2016-10-20 19:59:32 -07:00
Andrew Wang b0a77db3a2 HADOOP-12859. Disable hiding field style checks in class setters. Contributed by Kai Zheng. 
(cherry picked from commit 480302b4ba)
(cherry picked from commit 3fecfe1352)
 2016-10-20 18:00:07 -07:00
Steve Loughran da7e2f08f1 Revert "HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran." 
This reverts commit 671d219c9c.
 2016-10-20 22:49:11 +01:00
Anu Engineer 671d219c9c HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-20 12:50:49 -07:00
Wei-Chiu Chuang b4edd115b4 HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. Contributed by Yiqun Lin. 
(cherry picked from commit 6d2da38d16)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java

(cherry picked from commit 6f4192d77d)
 2016-10-20 12:41:00 -07:00
Steve Loughran 2b575a90f3 Revert "HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula" 
This reverts commit b63dd7b63b.
 2016-10-20 19:43:16 +01:00
Brahma Reddy Battula b63dd7b63b HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc03)
 2016-10-20 21:49:21 +05:30
Kihwal Lee f57d81955e HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah. 
(cherry picked from commit 5c0bffddc0)
 2016-10-20 11:02:57 -05:00
Brahma Reddy Battula d2fde9c8b0 HDFS-9480. Expose nonDfsUsed via StorageTypeStats. Contributed by Brahma Reddy Battula 
(cherry picked from commit 4c73be135c)
 2016-10-20 20:51:41 +05:30
Steve Loughran 769babf470 HADOOP-13735 ITestS3AFileContextStatistics.testStatistics() failing. Contributed by Pieter Reuse 2016-10-20 14:50:44 +01:00