Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,498 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

11498 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli (I am also known as @tshooter.) b2efb2040c HADOOP-13428. Fixed hadoop-common build files to generate jdiff by working around a jdiff-bug. Contributed by Wangda Tan. 
(cherry picked from commit 99603e9022)
 2016-08-19 19:11:04 -07:00
Wangda Tan 2e9d84d24c YARN-4685. Disable AM blacklisting by default to mitigate situations that application get hanged. (Rohith Sharma K S via wangda) 
(cherry picked from commit 2da32a6ef9)
(cherry picked from commit 3e511f39c2)
 2016-08-19 16:55:40 -07:00
Wangda Tan 9bf420b3bc YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-08-19 16:55:34 -07:00
Wangda Tan 3bf2e16f76 YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda) 
(cherry picked from commit 444b2ea7af)
(cherry picked from commit c7d782d2f6)
 2016-08-19 16:54:03 -07:00
Wangda Tan e2bd490236 MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee)
(cherry picked from commit 65822e55fd)
 2016-08-19 16:49:54 -07:00
Varun Saxena 6f1a9f1b6a MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:08:55 +05:30
Kihwal Lee 4f3df8e22a HDFS-7959. WebHdfs logging is missing on Datanode (Kihwal Lee via sjlee) 
(cherry picked from commit ae90d4dd90)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
 2016-08-19 15:32:11 -05:00
Chris Nauroth dd6c149c4a HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran. 
(cherry picked from commit 763f0497bb)
(cherry picked from commit 6dba38f08b)
 2016-08-19 12:55:57 -07:00
Jing Zhao 5d91f9fe35 HADOOP-13503. Improve SaslRpcClient failure logging. Contributed by Xiaobing Zhou. 
(cherry picked from commit 9fdd1ea724)
 2016-08-18 15:00:17 -07:00
Mingliang Liu ef38aa64c6 HADOOP-13512. ReloadingX509TrustManager should keep reloading in case of exception. (Contributed by Mingliang Liu) 
(cherry picked from commit 0f51eae0c0)
 2016-08-18 14:22:58 -07:00
Varun Vasudev 5b49bcaee3 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 
(cherry picked from commit 308d63f382)
(cherry picked from commit 99966b1912)
 2016-08-18 12:33:52 -07:00
Karthik Kambatla 42a88f9c58 YARN-3446. FairScheduler headroom calculation should exclude nodes in the blacklist. (Zhihai Xu via kasha) 
(cherry picked from commit 9d04f26d4c)
(cherry picked from commit f0923819c3)
 2016-08-18 12:30:39 -07:00
Chris Nauroth 1c4e039ba7 HADOOP-13513. Java 1.7 support for org.apache.hadoop.fs.azure testcases. Contributed by Tibor Kiss. 
(cherry picked from commit ae4db25443)
(cherry picked from commit 804a40720b)
 2016-08-18 10:45:20 -07:00
Steve Loughran c09797ce9f HADOOP-13405 doc for fs.s3a.acl.default indicates incorrect values. Contributed by Shen Yinjie 2016-08-18 14:35:26 +01:00
Arpit Agarwal 1fe08c919a HDFS-10682. Replace FsDatasetImpl object lock with a separate lock object. (Contributed by Chen Liang) 2016-08-17 16:22:07 -07:00
Xiao Chen b89d79ca1d HDFS-10549. Correctly revoke file leases when closing files. Contributed by Yiqun Lin. 
(cherry picked from commit 85aacaadb5a3f8c78b191867c0bde09b3c4b3c3c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java

(cherry picked from commit a20b943cf9)
 2016-08-17 16:00:33 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) d65024eddc Revert "HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla." 
Reverting this on branch-2.* as it's an incompatible change.

This reverts commit 83f7f62be3.

(cherry picked from commit 2486c4c63a)
 2016-08-17 13:34:35 -07:00
Mingliang Liu 00eb790622 HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-16 17:41:51 -07:00
Chris Nauroth ac779f2554 HADOOP-13324. s3a tests don't authenticate with S3 frankfurt (or other V4 auth only endpoints). Contributed by Steve Loughran. 
(cherry picked from commit 3808876c73)
(cherry picked from commit 555a57adf2)
 2016-08-16 17:11:12 -07:00
Mingliang Liu b6d92c61fe Revert "HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu)" 
This reverts commit 657064e593.
 2016-08-16 16:27:36 -07:00
Andrew Wang 8b6eeb2595 HADOOP-13494. ReconfigurableBase can log sensitive information. Contributed by Sean Mackrory. 
(cherry picked from commit 4b689e7a75)
(cherry picked from commit c91e6f2828)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
 2016-08-16 15:14:06 -07:00
Kihwal Lee b76f6e5de3 HDFS-10342. BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt. Contributed by Kuhu Shukla. 
(cherry picked from commit b10c936020)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 65a41696b2)
 2016-08-16 09:39:33 -05:00
Mingliang Liu 54c9743263 HDFS-10641. TestBlockManager#testBlockReportQueueing fails intermittently. (Contributed by Daryn Sharp) 2016-08-15 20:46:48 -07:00
Mingliang Liu 657064e593 HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-15 20:46:40 -07:00
Mingliang Liu e9eaad0e67 HDFS-10747. o.a.h.hdfs.tools.DebugAdmin usage message is misleading. (Contributed by Mingliang Liu) 2016-08-15 20:46:34 -07:00
Mingliang Liu 0b934c375e HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu) 2016-08-15 20:46:28 -07:00
Mingliang Liu 6471ec31bc HDFS-10724. Document caller context config keys. (Contributed by Mingliang Liu) 
(cherry picked from commit 4bcbef39f7)
 2016-08-15 20:46:03 -07:00
Kihwal Lee 1a8280edde HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee. 
(cherry picked from commit 864f878d59)
(cherry picked from commit e78db7d2a4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-08-15 17:37:39 -05:00
Varun Saxena 6fed7b9c45 HADOOP-13333. testConf.xml ls comparators in wrong order (Vrushali C via Varun Saxena) 2016-08-16 03:14:24 +05:30
Jason Lowe 74831973cb YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit e0b14f26f5)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-08-15 19:45:45 +00:00
Junping Du 1a38dd9cee YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 
(cherry picked from commit 0005816743)
 2016-08-15 19:18:36 +00:00
Kihwal Lee 579709fc22 HDFS-9696. Garbage snapshot records linger forever. Contributed by Kihwal Lee 
(cherry picked from commit 83e57e083f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java

(cherry picked from commit 4766a3153d)
 2016-08-15 13:13:49 -05:00
Varun Saxena ca139a3f87 YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena) 2016-08-13 00:27:17 +05:30
Wei-Chiu Chuang 53f097a474 HDFS-10731. FSDirectory#verifyMaxDirItems does not log path name. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 9019606b69)
(cherry picked from commit 20a8ff0a8d)
 2016-08-12 06:11:22 -07:00
Zhe Zhang 9c1c48c0d4 HDFS-9888. Allow reseting KerberosName in unit tests. Contributed by Xiao Chen. 
(cherry picked from commit 3e8099a45a)
(cherry picked from commit 00ff3d737c)
 2016-08-11 12:42:17 -07:00
Akira Ajisaka f29d0739d7 HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin. 
(cherry picked from commit 932aed64d7)
(cherry picked from commit 0529da7239)
 2016-08-11 12:42:17 -07:00
Zhe Zhang b8b4ea67d8 HDFS-9804. Allow long-running Balancer to login with keytab. Contributed by Xiao Chen. 
(cherry picked from commit ccff6035f5)
(cherry picked from commit 7d402692b4)
 2016-08-11 12:42:17 -07:00
Wei-Chiu Chuang b0bc7681c3 HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit db719ef125)
(cherry picked from commit 6d53e096b2)
 2016-08-11 12:33:15 -07:00
Rohith Sharma K S f7cc0af611 YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C. 
(cherry picked from commit 5199db387d)
 2016-08-11 11:52:44 +05:30
Wei-Chiu Chuang 65e3657149 HDFS-8897. Balancer should handle fs.defaultFS trailing slash in HA. Contributed by John Zhuge. 
(cherry picked from commit f1d5a95cf3)
 2016-08-10 22:30:06 -07:00
Xiaoyu Yao 7a1a3f7754 HDFS-10643. Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit ec289bbece)
 2016-08-10 15:56:56 -07:00
Xiaoyu Yao 8808df09c6 Revert "Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao." 
This reverts commit 9f816d7ca7.

(cherry picked from commit fdafd0cb4f)
 2016-08-10 15:56:43 -07:00
Xiaoyu Yao c0e331935d Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9f816d7ca7)
 2016-08-10 15:19:10 -07:00
Xiaoyu Yao cc49133281 HADOOP-13285. DecayRpcScheduler MXBean should only report decayed CallVolumeSummary. Contributed by Xiaoyu Yao. 
(cherry picked from commit 0761379fe4)
 2016-08-10 15:19:04 -07:00
Jason Lowe 69217ba813 YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C 
(cherry picked from commit 5c95bb315b)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java
 2016-08-10 19:15:12 +00:00
Wei-Chiu Chuang 753edc4931 HDFS-8224. Schedule a block for scanning if its metadata file is corrupt. Contributed by Rushabh S Shah. 
(cherry picked from commit 8efd4959f3)
 2016-08-10 11:39:12 -07:00
Yuanbo Liu d46f5ca7e2 HDFS-10694. processReport() should print blockReportId in each log message. Contributed by Yuanbo Liu. 
(cherry picked from commit 10e84c6a6e)
 2016-08-10 11:12:21 -07:00
Jason Lowe e1845faeda YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee 
(cherry picked from commit 3d401206cf)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2016-08-10 18:09:11 +00:00
Rohith Sharma K S 3b4d6ece1a YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat. Contributed by Rohith Sharma K S 
(cherry picked from commit 26b5e6116f)
 2016-08-09 22:19:17 +00:00
Karthik Kambatla 5ea35b2af2 HADOOP-13299. JMXJsonServlet is vulnerable to TRACE. (Haibo Chen via kasha) 
(cherry picked from commit 85422bb7c5)
(cherry picked from commit 2df34ab6e2)
 2016-08-09 13:47:42 -07:00