Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,962 Commits 326 Branches 371 Tags
Commit Graph

4148 Commits

Author SHA1 Message Date
Mingliang Liu
875c14004d HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu) 
(cherry picked from commit 6742fb6e68d349055f985eb640d845e689d75384)
 2016-08-29 10:19:33 -07:00
Yongjun Zhang
e813a3ea4a HDFS-10652. Add a unit test for HDFS-4660. Contributed by Vinayakumar B., Wei-Chiu Chuang, Yongjun Zhang. 
(cherry picked from commit c25817159af17753b398956cfe6ff14984801b01)
 2016-08-27 22:59:19 -07:00
Andrew Wang
1c47a356d8 HDFS-10793. Fix HdfsAuditLogger binary incompatibility introduced by HDFS-9184. Contributed by Manoj Govindassamy. 
(cherry picked from commit a445b82baaa58dcaaa3831e724a9915e9dde57c5)
 2016-08-26 15:40:05 -07:00
Zhe Zhang
f3a4cbcc60 HDFS-10798. Make the threshold of reporting FSNamesystem lock contention configurable. Contributed by Erik Krogen. 
(cherry picked from commit 407b519fb14f79f19ebc4fbdf08204336a7acf77)
 2016-08-26 14:20:48 -07:00
Kihwal Lee
442dfd1bbe HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp. 
(cherry picked from commit 8b7adf4ddf420a93c586c4b2eac27dd0f649682e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
 2016-08-26 15:49:56 -05:00
Kihwal Lee
2a6497cc7c HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki. 
(cherry picked from commit 13fb1b50e608558b2970184908ee5b9fcd7eb7b6)
 2016-08-26 10:00:36 -05:00
Xiaoyu Yao
fc67501393 HDFS-10748. TestFileTruncate#testTruncateWithDataNodesRestart runs sometimes timeout. Contributed by Yiqun Lin. 
(cherry picked from commit 4da5000dd33cf013e7212848ed2c44f1e60e860e)
 2016-08-25 09:22:02 -07:00
Kihwal Lee
ba3257baf5 HDFS-10772. Reduce byte/string conversions for get listing. Contributed by Daryn Sharp. 
(cherry picked from commit a1f3293762dddb0ca953d1145f5b53d9086b25b8)
 2016-08-24 15:25:33 -05:00
Wei-Chiu Chuang
d19e29bd68 HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-24 08:49:54 -07:00
Daryn Sharp
a30f6a68fa HDFS-10762. Pass IIP for file status related methods 
(cherry picked from commit ec252ce0fc0998ce13f31af3440c08a236328e5a)
 2016-08-24 08:51:05 -05:00
Zhe Zhang
dfcbc12026 HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen. Branch-2 patch contributed by Wei-Chiu Chuang. 2016-08-23 14:46:08 -07:00
Jason Lowe
8cd3996246 HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen 
(cherry picked from commit 126d165efd80e266a8309241f3cf059e358f5019)
 2016-08-23 14:48:35 +00:00
Akira Ajisaka
94a844b4f6 HDFS-10783. The option '-maxSize' and '-step' fail in OfflineImageViewer. Contributed by Yiqun Lin. 
(cherry picked from commit e90f3359de299ef5e3a54ca71070e3dfe1dbb98c)
 2016-08-23 19:58:13 +09:00
Eric Yang
9faacbec2c HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang) 2016-08-22 18:32:06 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
8a6a7c82f2 HDFS-10692. Update JDiff report's base version for HDFS from 2.6.0 to 2.7.2. Contributed by Wangda Tan. 
(cherry picked from commit dc7a1c54f4447fb01980ae61d5d67e90c4f52f00)
 2016-08-22 11:36:38 -07:00
Akira Ajisaka
62c7d09efa HDFS-10645. Make block report size as a metric and add this metric to datanode web ui. Contributed by Yuanbo Liu. 2016-08-22 14:22:37 +09:00
Jing Zhao
fff0418458 HDFS-10764. Fix INodeFile#getBlocks to not return null. Contributed by Arpit Agarwal. 
(cherry picked from commit 0faee62a0c8c1b8fd83227babfd00fbc2b26bddf)
 2016-08-19 22:23:55 -07:00
Kihwal Lee
43d1279df0 HDFS-10711. Optimize FSPermissionChecker group membership check. Contributed by Daryn Sharp. 
(cherry picked from commit 2550371f66c49fe0e40aadaa68744311270084ce)
 2016-08-19 09:46:30 -05:00
Kihwal Lee
922d045e11 HDFS-10745. Directly resolve paths into INodesInPath. Contributed by Daryn Sharp. 2016-08-18 12:37:57 -05:00
Xiao Chen
a20b943cf9 HDFS-10549. Correctly revoke file leases when closing files. Contributed by Yiqun Lin. 
(cherry picked from commit 85aacaadb5a3f8c78b191867c0bde09b3c4b3c3c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
 2016-08-17 15:50:13 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
2486c4c63a Revert "HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla." 
Reverting this on branch-2.* as it's an incompatible change.

This reverts commit 83f7f62be379045ad6933689b21b76c7086f919d.
 2016-08-17 13:33:09 -07:00
Kihwal Lee
65a41696b2 HDFS-10342. BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt. Contributed by Kuhu Shukla. 
(cherry picked from commit b10c936020e2616609dcb3b2126e8c34328c10ca)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2016-08-16 09:29:42 -05:00
Mingliang Liu
2f4660ce98 HDFS-10641. TestBlockManager#testBlockReportQueueing fails intermittently. (Contributed by Daryn Sharp) 2016-08-15 20:39:12 -07:00
Mingliang Liu
82623ea9c9 HDFS-10747. o.a.h.hdfs.tools.DebugAdmin usage message is misleading. (Contributed by Mingliang Liu) 2016-08-15 20:38:17 -07:00
Mingliang Liu
f9a7e59066 HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu) 2016-08-15 20:37:55 -07:00
Kihwal Lee
e78db7d2a4 HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee. 
(cherry picked from commit 864f878d5912c82f3204f1582cfb7eb7c9f1a1da)
 2016-08-15 17:33:16 -05:00
Kihwal Lee
1ef8d7a638 HDFS-10744. Internally optimize path component resolution. Contributed by Daryn Sharp. 
(cherry picked from commit 03dea65e0b17ca2f9460bb6110f6ab3a321b8bf2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
 2016-08-15 17:01:40 -05:00
Kihwal Lee
4766a3153d HDFS-9696. Garbage snapshot records linger forever. Contributed by Kihwal Lee 
(cherry picked from commit 83e57e083f2cf6c0de8a46966c5492faeabd8f2a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java
 2016-08-15 13:09:50 -05:00
Wei-Chiu Chuang
20a8ff0a8d HDFS-10731. FSDirectory#verifyMaxDirItems does not log path name. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 9019606b69bfb7019c8642b6cbcbb93645cc19e3)
 2016-08-12 06:02:47 -07:00
Zhe Zhang
00ff3d737c HDFS-9888. Allow reseting KerberosName in unit tests. Contributed by Xiao Chen. 
(cherry picked from commit 3e8099a45a4cfd4c5c0e3dce4370514cb2c90da9)
 2016-08-11 11:32:54 -07:00
Akira Ajisaka
0529da7239 HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin. 
(cherry picked from commit 932aed64d77edcc8483a95c1ce31a4c9ae679446)
 2016-08-11 10:55:53 -07:00
Zhe Zhang
7d402692b4 HDFS-9804. Allow long-running Balancer to login with keytab. Contributed by Xiao Chen. 
(cherry picked from commit ccff6035f50769eb69701128ae61efc69e82609d)
 2016-08-11 10:53:16 -07:00
Wei-Chiu Chuang
f1d5a95cf3 HDFS-8897. Balancer should handle fs.defaultFS trailing slash in HA. Contributed by John Zhuge. 2016-08-10 22:24:21 -07:00
Xiaoyu Yao
de6eafc696 HDFS-10643. Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit ec289bbeceff064ad24e189db20a3e0a296822c1)
 2016-08-10 15:55:12 -07:00
Xiaoyu Yao
c341280129 Revert "Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao." 
This reverts commit 9f816d7ca74c6e6c43ecfc8770228c518620c9cb.

(cherry picked from commit fdafd0cb4f1626ee41539f4b80711c4e78e3e16c)
 2016-08-10 15:54:59 -07:00
Xiaoyu Yao
d583a357f9 Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9f816d7ca74c6e6c43ecfc8770228c518620c9cb)
 2016-08-10 15:15:51 -07:00
Wei-Chiu Chuang
8efd4959f3 HDFS-8224. Schedule a block for scanning if its metadata file is corrupt. Contributed by Rushabh S Shah. 2016-08-10 11:34:59 -07:00
Yuanbo Liu
44162e555a HDFS-10694. processReport() should print blockReportId in each log message. Contributed by Yuanbo Liu. 
(cherry picked from commit 10e84c6a6e831fe2bea061fb21bd0dfe32bc9953)
 2016-08-10 11:10:00 -07:00
Kihwal Lee
7f1879abe6 HDFS-10738. Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure. Contributed by Rakesh R. 
(cherry picked from commit 0f701f433dd3be233bf53e856864c82349e8274e)
 2016-08-09 14:55:46 -05:00
Lei Xu
e7c701586d HDFS-4176. EditLogTailer should call rollEdits with a timeout. (lei) 2016-08-08 16:32:01 -07:00
Lei Xu
9cb6d291ea HDFS-10457. DataNode should not auto-format block pool directory if VERSION is missing. (Wei-Chiu Chuang via lei) 
(cherry picked from commit bb3bcb9397593fc8a2fa63a48eba126609f72c42)
 2016-08-08 16:19:58 -07:00
Konstantin V Shvachko
ab130f97ca HDFS-10693. metaSave should print blocks, not LightWeightHashSet. Contributed by Yuanbo Liu. 
(cherry picked from commit 4d3af47f2765f6f57936d316ef2a4150b787cc97)
 2016-08-07 14:35:35 -07:00
Akira Ajisaka
6863866127 HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu. 
(cherry picked from commit ef432579a7763cc0e482fe049027c6e5325eb034)
 2016-08-05 17:32:23 +09:00
Kihwal Lee
09b06a6e9e HDFS-10673. Optimize FSPermissionChecker's internal path usage. Contributed by Daryn Sharp. 
(cherry picked from commit 438a9f047eb6af2a4b916a4f6ef6f68adeab8068)
 2016-08-04 16:17:41 -05:00
Kihwal Lee
690ec789f0 HDFS-10343. BlockManager#createLocatedBlocks may return blocks on failed storages. Contributed by Kuhu Shukla. 
(cherry picked from commit 57369a678c4c51627fe9a654e697a906a6bef123)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2016-08-04 15:12:21 -05:00
Kihwal Lee
6ead6f3bb8 HDFS-10717. Fix findbugs warnings of hadoop-hdfs-client in branch-2. Contributed by Akira Ajisaka. 2016-08-04 12:34:54 -05:00
Kihwal Lee
3b2554f3b0 HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp. 
(cherry picked from commit 4a8e6dc02fb1d0dc89c3981b444575494511444a)
 2016-08-04 11:55:52 -05:00
Tsz-Wo Nicholas Sze
5004b0571e In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin 2016-08-04 09:53:15 -07:00
Kihwal Lee
77b61d1f4e HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp. 
(cherry picked from commit 6ae39199dac6ac7be6802b31452552c76da16e24)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
 2016-08-04 09:43:56 -05:00
Kihwal Lee
c0166b0743 HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 2016-08-04 09:28:30 -05:00