Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,783 Commits 326 Branches 371 Tags
Commit Graph

11783 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiao Chen
b89d79ca1d HDFS-10549. Correctly revoke file leases when closing files. Contributed by Yiqun Lin. 
(cherry picked from commit 85aacaadb5a3f8c78b191867c0bde09b3c4b3c3c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java

(cherry picked from commit a20b943cf951cf38eb4950177c826bbcf424aade)
 2016-08-17 16:00:33 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
d65024eddc Revert "HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla." 
Reverting this on branch-2.* as it's an incompatible change.

This reverts commit 83f7f62be379045ad6933689b21b76c7086f919d.

(cherry picked from commit 2486c4c63a35fcef7338ea63f0d8aafa778cd05d)
 2016-08-17 13:34:35 -07:00
Mingliang Liu
00eb790622 HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-16 17:41:51 -07:00
Chris Nauroth
ac779f2554 HADOOP-13324. s3a tests don't authenticate with S3 frankfurt (or other V4 auth only endpoints). Contributed by Steve Loughran. 
(cherry picked from commit 3808876c7397ea68906bc5cc18fdf690c9c42131)
(cherry picked from commit 555a57adf2fa5f2d50c1141da907ae4ca032db2c)
 2016-08-16 17:11:12 -07:00
Mingliang Liu
b6d92c61fe Revert "HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu)" 
This reverts commit 657064e59397feb92cec95094ac34c85e3040c50.
 2016-08-16 16:27:36 -07:00
Andrew Wang
8b6eeb2595 HADOOP-13494. ReconfigurableBase can log sensitive information. Contributed by Sean Mackrory. 
(cherry picked from commit 4b689e7a758a55cec2ca8398727feefc8ac21bfd)
(cherry picked from commit c91e6f28283d1f8ad4504a2338a01e284ced6b24)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
 2016-08-16 15:14:06 -07:00
Kihwal Lee
b76f6e5de3 HDFS-10342. BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt. Contributed by Kuhu Shukla. 
(cherry picked from commit b10c936020e2616609dcb3b2126e8c34328c10ca)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 65a41696b2a22834f4592bcb49e40fffd2dbb96e)
 2016-08-16 09:39:33 -05:00
Mingliang Liu
54c9743263 HDFS-10641. TestBlockManager#testBlockReportQueueing fails intermittently. (Contributed by Daryn Sharp) 2016-08-15 20:46:48 -07:00
Mingliang Liu
657064e593 HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-15 20:46:40 -07:00
Mingliang Liu
e9eaad0e67 HDFS-10747. o.a.h.hdfs.tools.DebugAdmin usage message is misleading. (Contributed by Mingliang Liu) 2016-08-15 20:46:34 -07:00
Mingliang Liu
0b934c375e HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu) 2016-08-15 20:46:28 -07:00
Mingliang Liu
6471ec31bc HDFS-10724. Document caller context config keys. (Contributed by Mingliang Liu) 
(cherry picked from commit 4bcbef39f7ca07601092919a7f2bea531a2dfa07)
 2016-08-15 20:46:03 -07:00
Kihwal Lee
1a8280edde HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee. 
(cherry picked from commit 864f878d5912c82f3204f1582cfb7eb7c9f1a1da)
(cherry picked from commit e78db7d2a430983807750666fb72ebd5c97ce867)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-08-15 17:37:39 -05:00
Varun Saxena
6fed7b9c45 HADOOP-13333. testConf.xml ls comparators in wrong order (Vrushali C via Varun Saxena) 2016-08-16 03:14:24 +05:30
Jason Lowe
74831973cb YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit e0b14f26f5201a149218276469434df511697acc)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-08-15 19:45:45 +00:00
Junping Du
1a38dd9cee YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 
(cherry picked from commit 00058167431475c6e63c80207424f1d365569e3a)
 2016-08-15 19:18:36 +00:00
Kihwal Lee
579709fc22 HDFS-9696. Garbage snapshot records linger forever. Contributed by Kihwal Lee 
(cherry picked from commit 83e57e083f2cf6c0de8a46966c5492faeabd8f2a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java

(cherry picked from commit 4766a3153dd517ac832d4761c884ed88a83a6c09)
 2016-08-15 13:13:49 -05:00
Varun Saxena
ca139a3f87 YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena) 2016-08-13 00:27:17 +05:30
Wei-Chiu Chuang
53f097a474 HDFS-10731. FSDirectory#verifyMaxDirItems does not log path name. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 9019606b69bfb7019c8642b6cbcbb93645cc19e3)
(cherry picked from commit 20a8ff0a8d17c05305f712da887ad6665eb654b5)
 2016-08-12 06:11:22 -07:00
Zhe Zhang
9c1c48c0d4 HDFS-9888. Allow reseting KerberosName in unit tests. Contributed by Xiao Chen. 
(cherry picked from commit 3e8099a45a4cfd4c5c0e3dce4370514cb2c90da9)
(cherry picked from commit 00ff3d737c71e1a228ede64a6b9abd5227ef2117)
 2016-08-11 12:42:17 -07:00
Akira Ajisaka
f29d0739d7 HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin. 
(cherry picked from commit 932aed64d77edcc8483a95c1ce31a4c9ae679446)
(cherry picked from commit 0529da723966e03c7dfa8201a515320c904d32a7)
 2016-08-11 12:42:17 -07:00
Zhe Zhang
b8b4ea67d8 HDFS-9804. Allow long-running Balancer to login with keytab. Contributed by Xiao Chen. 
(cherry picked from commit ccff6035f50769eb69701128ae61efc69e82609d)
(cherry picked from commit 7d402692b4b06af0459ea81129e93a5182291a27)
 2016-08-11 12:42:17 -07:00
Wei-Chiu Chuang
b0bc7681c3 HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit db719ef125b11b01eab3353e2dc4b48992bf88d5)
(cherry picked from commit 6d53e096b2d96b9aace46fa9f34cd93e7f0f22a6)
 2016-08-11 12:33:15 -07:00
Rohith Sharma K S
f7cc0af611 YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C. 
(cherry picked from commit 5199db387d59f7233a0e52ac298df31e8ed8af20)
 2016-08-11 11:52:44 +05:30
Wei-Chiu Chuang
65e3657149 HDFS-8897. Balancer should handle fs.defaultFS trailing slash in HA. Contributed by John Zhuge. 
(cherry picked from commit f1d5a95cf31f593f362a4e9c0afa7587f6e14957)
 2016-08-10 22:30:06 -07:00
Xiaoyu Yao
7a1a3f7754 HDFS-10643. Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit ec289bbeceff064ad24e189db20a3e0a296822c1)
 2016-08-10 15:56:56 -07:00
Xiaoyu Yao
8808df09c6 Revert "Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao." 
This reverts commit 9f816d7ca74c6e6c43ecfc8770228c518620c9cb.

(cherry picked from commit fdafd0cb4f1626ee41539f4b80711c4e78e3e16c)
 2016-08-10 15:56:43 -07:00
Xiaoyu Yao
c0e331935d Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9f816d7ca74c6e6c43ecfc8770228c518620c9cb)
 2016-08-10 15:19:10 -07:00
Xiaoyu Yao
cc49133281 HADOOP-13285. DecayRpcScheduler MXBean should only report decayed CallVolumeSummary. Contributed by Xiaoyu Yao. 
(cherry picked from commit 0761379fe45898c44c8f161834c298ef932e4d8c)
 2016-08-10 15:19:04 -07:00
Jason Lowe
69217ba813 YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C 
(cherry picked from commit 5c95bb315ba605b3bed77966a99a63854234e2c5)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java
 2016-08-10 19:15:12 +00:00
Wei-Chiu Chuang
753edc4931 HDFS-8224. Schedule a block for scanning if its metadata file is corrupt. Contributed by Rushabh S Shah. 
(cherry picked from commit 8efd4959f3fd48fe281aa26a46668071461dee8b)
 2016-08-10 11:39:12 -07:00
Yuanbo Liu
d46f5ca7e2 HDFS-10694. processReport() should print blockReportId in each log message. Contributed by Yuanbo Liu. 
(cherry picked from commit 10e84c6a6e831fe2bea061fb21bd0dfe32bc9953)
 2016-08-10 11:12:21 -07:00
Jason Lowe
e1845faeda YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee 
(cherry picked from commit 3d401206cf1eca5360f7f53165732a73d644a1fb)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2016-08-10 18:09:11 +00:00
Rohith Sharma K S
3b4d6ece1a YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat. Contributed by Rohith Sharma K S 
(cherry picked from commit 26b5e6116f392b6be91dca57968259f87554ce33)
 2016-08-09 22:19:17 +00:00
Karthik Kambatla
5ea35b2af2 HADOOP-13299. JMXJsonServlet is vulnerable to TRACE. (Haibo Chen via kasha) 
(cherry picked from commit 85422bb7c5d3e70a49f620ba1c8800e0ba4b64f2)
(cherry picked from commit 2df34ab6e261613526bc7b8e4ef303617f89c758)
 2016-08-09 13:47:42 -07:00
Naganarasimha
16c8fd9dca YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang. 2016-08-09 17:34:48 +05:30
Masatake Iwasaki
a9d957d77d HADOOP-13439. Fix race between TestMetricsSystemImpl and TestGangliaMetrics. Contributed by Chen Liang. 
(cherry picked from commit 8f9b61852bf6600b65e49875fec172bac9e0a85d)
 2016-08-09 10:06:14 +09:00
Wei-Chiu Chuang
7b042c5c72 HADOOP-13395. Enhance TestKMSAudit. Contributed by Xiao Chen. 
(cherry picked from commit 070548943a16370a74277d1b1d10b713e2ca81d0)

Conflicts:
hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSAudit.java

(cherry picked from commit cc20316b55ee108ccd880e8fb0565eacae2b90bd)
 2016-08-08 15:18:19 -07:00
Arpit Agarwal
5006083806 HADOOP-13457. Remove hardcoded absolute path for shell executable. (Chen Liang) 2016-08-08 12:31:19 -07:00
Jitendra Pandey
f4bf3e22de HADOOP-10823. TestReloadingX509TrustManager is flaky. Contributed by Mingliang Liu. 
Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/TestReloadingX509TrustManager.java
 2016-08-08 11:06:05 -07:00
Konstantin V Shvachko
6b3114a2a5 HDFS-10693. metaSave should print blocks, not LightWeightHashSet. Contributed by Yuanbo Liu. 
(cherry picked from commit 4d3af47f2765f6f57936d316ef2a4150b787cc97)
 2016-08-07 14:36:24 -07:00
Naganarasimha
8055ff1c3c HADOOP-13353. LdapGroupsMapping getPassward shouldn't return null when IOException throws. Contributed by Wei-Chiu Chuang 2016-08-06 23:18:53 +05:30
Masatake Iwasaki
e1684f84e2 HADOOP-13418. Fix javadoc warnings by JDK8 in hadoop-nfs package. Contributed by Kai Sasaki. 
(cherry picked from commit 10ed06a0c96cbb8b70e03deeb5bb94df700f9518)
 2016-08-06 12:29:58 +09:00
Naganarasimha
e6a994badf YARN-4624. NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI. Contributed by Brahma Reddy Battula 
(cherry picked from commit d81b8163b4e5c0466a6af6e1068f512c5fd24a61)
(cherry picked from commit 991823482f6687be681be69931e8a4646bc68a69)
 2016-08-06 08:19:56 +05:30
Wei-Chiu Chuang
e7863da095 Revert "HADOOP-13353. LdapGroupsMapping getPassward shouldn't return null when IOException throws. Contributed by Zhaohao Liang and Wei-Chiu Chuang." 
This reverts commit 7dbfe2f09963a6104f2b3b8c95015d4017bdc5d0.
 2016-08-05 19:05:52 -07:00
Wei-Chiu Chuang
7dbfe2f099 HADOOP-13353. LdapGroupsMapping getPassward shouldn't return null when IOException throws. Contributed by Zhaohao Liang and Wei-Chiu Chuang. 
(cherry picked from commit 49ba09a9221ad1e25e89800f6c455bbaad41483b)
(cherry picked from commit d875dfef3dfa91c3af6a90830a6d9c28df490db3)
 2016-08-05 16:37:23 -07:00
Jason Lowe
470ea3da1f YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit c01bee010832ca31d8e60e5461181cdf05140602)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-08-05 20:10:04 +00:00
Kihwal Lee
5b3deac2e3 HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla. 
(cherry picked from commit f715f141856cb6a4c6574893f40f9865653b631e)
 2016-08-05 11:43:43 -05:00
Akira Ajisaka
5758026833 HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu. 
(cherry picked from commit ef432579a7763cc0e482fe049027c6e5325eb034)
(cherry picked from commit 68638661271e6e69c220b19fec7cbc7ae5e6208e)
 2016-08-05 17:33:07 +09:00
Kihwal Lee
e8cd1439d3 HDFS-10343. BlockManager#createLocatedBlocks may return blocks on failed storages. Contributed by Kuhu Shukla. 
(cherry picked from commit 57369a678c4c51627fe9a654e697a906a6bef123)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 690ec789f0448d9d2b908562f54a1731ed9e97aa)
 2016-08-04 15:27:51 -05:00