Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,138 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4302 Commits

Author SHA1 Message Date
Mingliang Liu fcedb6812b HDFS-11129. TestAppendSnapshotTruncate fails with bind exception. Contributed by Brahma Reddy Battula 
(cherry picked from commit 2ee18fc15e)
 2016-11-11 22:19:39 -08:00
Xiaoyu Yao c443ecb267 HDFS-10941. Improve BlockManager#processMisReplicatesAsync log. Contributed by Chen Liang. 
(cherry picked from commit 4484b48498)
 2016-11-11 14:32:11 -08:00
Mingliang Liu ae7a20f524 HDFS-11122. TestDFSAdmin#testReportCommand fails due to timed out. Contributed by Yiqun Lin 
(cherry picked from commit aa6010ccca)
 2016-11-11 12:35:21 -08:00
Brahma Reddy Battula 121b56de14 HDFS-11128. CreateEditsLog throws NullPointerException. Contributed by Hanisha Koneru. 
(cherry picked from commit 1ae57f0f75)
 2016-11-11 20:11:13 +05:30
Xiao Chen d81706cd99 HDFS-11120. TestEncryptionZones should waitActive. Contributed by John Zhuge. 
(cherry picked from commit 907b1431c0ead934e26cf7dbc51bfab5d7cc5ddc)
 2016-11-09 17:16:36 -08:00
Wei-Chiu Chuang 1d75da8e26 HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 16430c64b7)
 2016-11-09 13:32:35 -08:00
Mingliang Liu ad992e05a6 HDFS-11083. Add unit test for DFSAdmin -report command. Contributed by Xiaobing Zhou 
(cherry picked from commit 62d8c17dfd)
 2016-11-08 17:10:59 -08:00
Mingliang Liu 00625004b7 HDFS-11085. Add unit test for NameNode failing to start when name dir is unwritable. Contributed by Xiaobing Zhou 
(cherry picked from commit 0c0ab102ab)
 2016-11-04 14:45:43 -07:00
Mingliang Liu b082d4b746 HDFS-11076. Add unit test for extended Acls. Contributed by Chen Liang 
(cherry picked from commit 7534aee09a)
 2016-11-03 13:55:43 -07:00
Arpit Agarwal 3f2bfdba3e HDFS-11097. Fix warnings for deprecated StorageReceivedDeletedBlocks constructor. Contributed by Yiqun Lin. 2016-11-03 10:33:28 -07:00
Andrew Wang e393dd788e HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory. 2016-11-02 19:11:25 -07:00
Brahma Reddy Battula 77142827e8 HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu. 
(cherry picked from commit 35e6081020)
 2016-11-02 10:40:22 +05:30
Mingliang Liu 10eb21e52e HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou 
(cherry picked from commit fa1512a740)
 2016-11-01 14:50:16 -07:00
Kihwal Lee db6dfeca1a HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp. 
(cherry picked from commit 7c1a1834e4)
 2016-11-01 08:26:25 -05:00
Brahma Reddy Battula c895f08936 HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 
(cherry picked from commit 1c5708f6de)
 2016-11-01 10:53:37 +05:30
Kihwal Lee 91640a0170 Addendum of HDFS-10722 to include HDFS-7964 logic. 
(cherry picked from commit 4a8e6dc02f)
(cherry picked from commit 3b2554f3b0)
(cherry picked from commit c2bb9bf00c)
 2016-10-31 16:37:00 -07:00
Zhe Zhang 7900b1e803 Revert "HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp." 
This reverts commit c2bb9bf00c.
 2016-10-31 16:35:05 -07:00
Kihwal Lee c2bb9bf00c HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp. 
(cherry picked from commit 4a8e6dc02f)
(cherry picked from commit 3b2554f3b0)
 2016-10-31 16:30:39 -07:00
Akira Ajisaka a17fc1dca3 HDFS-10485. Fix findbugs warning in FSEditLog.java. (aajisaka) 
(cherry picked from commit e620530301)
(cherry picked from commit b8c1db6f63)
 2016-10-31 16:24:35 -07:00
Jing Zhao e327325a37 HDFS-7964. Add support for async edit logging. Contributed by Daryn Sharp. 
(cherry picked from commit 2151716832)
(cherry picked from commit 3cb7ae11a8)
 2016-10-31 16:24:35 -07:00
Mingliang Liu fece3c881b HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu 
(cherry picked from commit 6bb12ecc3f)
 2016-10-31 16:06:41 -07:00
Arpit Agarwal f0a6d17e3b HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru. 2016-10-31 12:40:03 -07:00
Mingliang Liu 616c1194fe HDFS-11047. Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode. Contributed by Xiaobing Zhou 
(cherry picked from commit 2848a27700)
 2016-10-28 11:18:26 -07:00
Mingliang Liu a8a99f3423 HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger 
(cherry picked from commit 8a9388e5f6)
 2016-10-28 11:01:09 -07:00
Erik Krogen 2ec90a1a4b HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen. 
(cherry picked from commit f3ac1f41b8)
 2016-10-27 16:04:58 -07:00
Wei-Chiu Chuang bc2f429c0e HDFS-11055. Update default-log4j.properties for httpfs to imporve test logging. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 31ff42b510)
(cherry picked from commit 2cf3138d72)
 2016-10-27 13:47:26 -07:00
Kihwal Lee 40dbf2a18d HDFS-11069. Tighten the authorization of datanode RPC. Contributed by Kihwal Lee 
(cherry picked from commit ae48c496dc)
 2016-10-27 14:18:48 -05:00
Kihwal Lee 5b0d32b4aa Revert "HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu" 
This reverts commit d6dfc76dfe.
 2016-10-27 14:10:52 -05:00
Brahma Reddy Battula d6dfc76dfe HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu 
(cherry picked from commit ac35ee9393)
 2016-10-27 20:23:07 +05:30
Vinayakumar B 7279ce831d HDFS-8492. DN should notify NN when client requests a missing block (Contributed by Walter Su) 
(cherry picked from commit 1cf6ec4ad4)
(cherry picked from commit 144f7a9e0e)
 2016-10-27 16:56:25 +05:30
Masatake Iwasaki 6d9b6fac86 HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran. 
(cherry picked from commit 0bdd263d82)
 2016-10-27 15:55:35 +09:00
Mingliang Liu a7684f6517 HDFS-10921. TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode. Contributed by Eric Badger 
(cherry picked from commit 55e1fb8e32)
 2016-10-26 14:18:00 -07:00
Kihwal Lee e7bd2e8d63 HDFS-11053. Unnecessary superuser check in versionRequest(). Contributed by Kihwal Lee. 
(cherry picked from commit e90af4a89b)
 2016-10-26 09:18:01 -05:00
Kihwal Lee 49adab66b8 HDFS-11050. Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR. Contributed by Kuhu Shukla. 
(cherry picked from commit ce6bbfb23c)
 2016-10-26 08:11:51 -05:00
Arpit Agarwal df83d86da7 HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. Contributed by Chen Liang. 2016-10-25 19:51:53 -07:00
Xiaoyu Yao e0fc749a46 HDFS-11011. Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'. Contributed by Xiaobing Zhou. 
(cherry picked from commit 85f6fec0370b3cd94d6c2f19920c0b6d33f127c6)
(cherry picked from commit c2ba4b650168be4fad7ffdb546b8a0d985268393)
 2016-10-25 11:22:07 -07:00
Zhe Zhang 262518fa5b HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee. 
(cherry picked from commit f6367c5f44)
(cherry picked from commit ff806cbfc7)
 2016-10-25 10:59:03 -07:00
Kihwal Lee c82910203c HDFS-11042. Add missing cleanupSSLConfig() call for tests that use setupSSLConfig(). Contributed by Kuhu Shukla. 
(cherry picked from commit 086577c67a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/TestHttpFSFWithSWebhdfsFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java
 2016-10-24 09:00:28 -05:00
Brahma Reddy Battula 5754772bd6 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc03)
 2016-10-21 16:22:42 +05:30
Mingliang Liu cfac82a443 HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou 
(cherry picked from commit d7d87deece)
 2016-10-20 19:59:32 -07:00
Wei-Chiu Chuang b4edd115b4 HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. Contributed by Yiqun Lin. 
(cherry picked from commit 6d2da38d16)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java

(cherry picked from commit 6f4192d77d)
 2016-10-20 12:41:00 -07:00
Steve Loughran 2b575a90f3 Revert "HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula" 
This reverts commit b63dd7b63b.
 2016-10-20 19:43:16 +01:00
Brahma Reddy Battula b63dd7b63b HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc03)
 2016-10-20 21:49:21 +05:30
Kihwal Lee f57d81955e HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah. 
(cherry picked from commit 5c0bffddc0)
 2016-10-20 11:02:57 -05:00
Brahma Reddy Battula d2fde9c8b0 HDFS-9480. Expose nonDfsUsed via StorageTypeStats. Contributed by Brahma Reddy Battula 
(cherry picked from commit 4c73be135c)
 2016-10-20 20:51:41 +05:30
Brahma Reddy Battula f5b080e152 HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin 
(cherry picked from commit 73504b1bdc)
 2016-10-20 10:47:16 +05:30
Arpit Agarwal e9592f14d5 HDFS-10752. Several log refactoring/improvement suggestion in HDFS. Contributed by Hanisha Koneru. 2016-10-19 17:22:23 -07:00
Xiao Chen d22a6a8b82 HDFS-11009. Add a tool to reconstruct block meta file from CLI. 
(cherry picked from commit bc4a32aea50e86819730312e89315c0244ce64bf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetUtil.java

(cherry picked from commit ad7d3c4db8)
 2016-10-18 22:42:09 -07:00
Xiaoyu Yao 190a53b898 HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru. 
(cherry picked from commit 0c79d5af74944363f3d04f7e1702411db564bbfd)
 2016-10-18 14:09:03 -07:00
Kai Zheng d26e11f30e HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R 2016-10-18 15:09:25 +06:00
Vinitha Reddy Gankidi ae972dd080 HDFS-10712. TestDataNodeVolumeFailure should pass not null BlockReportContext. Contributed by Vinitha Gankidi. 
(cherry picked from commit 278eeb1056)
 2016-10-17 19:02:29 -07:00
Xiaoyu Yao f8b2d77207 HDFS-10922. Adding additional unit tests for Trash (II). Contributed by Weiwei Yang. 
(cherry picked from commit f1802d0be05ecc0b3248690b6f9efedbc7784112)
 2016-10-17 14:04:58 -07:00
Brahma Reddy Battula 1c473898aa HDFS-11003. Expose XmitsInProgress through DataNodeMXBean. Contributed By Brahma Reddy Battula 
(cherry picked from commit 5f4ae85bd8)
 2016-10-15 22:40:57 +05:30
Akira Ajisaka 65e6a1181c HDFS-11012. Unnecessary INFO logging on DFSClients for InvalidToken. Contributed by Harsh J. 
(cherry picked from commit 5ad037df25)
(cherry picked from commit 35832901e8)
 2016-10-15 22:19:14 +09:00
Vinitha Reddy Gankidi 2304501bcf HDFS-10301. Remove FBR tracking state to fix false zombie storage detection for interleaving block reports. Contributed by Vinitha Gankidi. 
(cherry picked from commit 391ce535a7)
 2016-10-14 18:52:00 -07:00
Lei Xu b9fdbd7107 HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei) 
(cherry picked from commit 78cb79fa91)
 2016-10-14 15:01:37 -07:00
Mingliang Liu 15b1d522ee HDFS-11008. Change unit test for testing parsing "-source" parameter in Balancer CLI. Contributed by Mingliang Liu 
(cherry picked from commit 76cc84e6d4)
 2016-10-14 14:44:34 -07:00
Kihwal Lee 8d1fbf786b Revert "HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei)" 
This reverts commit f981dd1bca.
 2016-10-14 16:37:51 -05:00
Lei Xu f981dd1bca HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei) 
(cherry picked from commit 8c520a27cb)
 2016-10-14 13:44:18 -07:00
Andrew Wang f13b84b4eb HDFS-10883. 's behavior is not consistent in DFS after enabling EZ. Contributed by Yuanbo Liu. 
(cherry picked from commit 0007360c33)
 2016-10-14 11:41:42 -07:00
Benoy Antony ca20ffea46 HDFS-10735 Distcp using webhdfs on secure HA clusters fails with StandbyException 
(cherry picked from commit 701c27a776)
 2016-10-14 10:32:37 -07:00
Kihwal Lee 8c71fc1221 HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin. 
(cherry picked from commit 8a9f6635a3)
 2016-10-14 11:44:25 -05:00
Brahma Reddy Battula 0884c31aec HDFS-10986. DFSAdmin should log detailed error message if any. Contributed by MingLiang Liu 2016-10-14 10:28:13 +05:30
Andrew Wang 3892fd810b Revert "HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin." 
This reverts commit 7a5aaa789d.
 2016-10-13 13:23:01 -07:00
Kihwal Lee ded91992ad HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c5a1303704)
 2016-10-13 14:57:44 -05:00
Andrew Wang 7a5aaa789d HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin. 
(cherry picked from commit fdce515091)
(cherry picked from commit 2e153bc8ab)
 2016-10-13 11:42:23 -07:00
Kihwal Lee 78777d4a96 HDFS-11000. webhdfs PUT does not work if requests are routed to call queue. Contributed by Kihwal Lee. 
(cherry picked from commit 9454dc5e80)
 2016-10-13 09:02:59 -05:00
Akira Ajisaka c7780b066d HDFS-11002. Fix broken attr/getfattr/setfattr links in ExtendedAttributes.md. Contributed by Mingliang Liu. 
(cherry picked from commit 901eca004d)
(cherry picked from commit 43cf0b2973)
 2016-10-13 14:31:09 +09:00
Mingliang Liu 9b2a719031 HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang 
(cherry picked from commit b36aaa913c)
 2016-10-12 17:41:07 -07:00
Kihwal Lee 136c6f6f7d HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah. 
(cherry picked from commit 5305a392c3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
 2016-10-12 15:38:34 -05:00
Mingliang Liu 0f481e2022 HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou 2016-10-11 17:18:48 -07:00
Xiaoyu Yao 6f2de54eb1 HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle. 
(cherry picked from commit 61f0490a73)
 2016-10-11 16:00:17 -07:00
Andrew Wang 67f0319db6 HDFS-10991. Export hdfsTruncateFile symbol in libhdfs. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit dacd3ec66b)
 2016-10-11 15:07:28 -07:00
Wei-Chiu Chuang aeb75213d4 HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma. 
(cherry picked from commit a6b3dc69d7a0835fa9159a0536c24ff6525e6c77)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2016-10-11 13:18:32 -07:00
Kihwal Lee 7f514e0c05 HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp. 
(cherry picked from commit e57fa81d95)
 2016-10-11 09:41:47 -05:00
Mingliang Liu 5a39a7560a HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou 2016-10-10 17:46:31 -07:00
Mingliang Liu 1c7baa505b HDFS-10988. Refactor TestBalancerBandwidth. Contributed by Brahma Reddy Battula 
(cherry picked from commit b963818621)
 2016-10-10 13:26:48 -07:00
Xiao Chen 998e44a972 HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory. 
(cherry picked from commit 3ea027e970ec7c95a10e581435cdb6c3a6b64abb)
 2016-10-07 17:31:42 -07:00
Kihwal Lee 0cc64759ab HDFS-10979. Pass IIP for FSDirDeleteOp methods. Contributed by Daryn Sharp. 
(cherry picked from commit 3565c9af17)
 2016-10-07 14:38:11 -05:00
Kihwal Lee d55b7bf1c7 HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn Sharp. 
(cherry picked from commit 76c03acc6d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-10-07 13:25:56 -05:00
Kihwal Lee 29afba8a15 HDFS-10955. Pass IIP for FSDirAttr methods. Contributed by Daryn Sharp. 
(cherry picked from commit e3a9666d28)
 2016-10-06 16:37:53 -05:00
Chris Nauroth e36b665f90 HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth. 
(cherry picked from commit 1d330fbaf6)
(cherry picked from commit a28ffd0fde)
 2016-10-06 13:24:54 -07:00
Brahma Reddy Battula a4ee4f9e51 HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen 
(cherry picked from commit b90fc70d67)
 2016-10-06 17:49:48 +05:30
Andrew Wang a2cdd2215e HDFS-10962. TestRequestHedgingProxyProvider is flaky. 
(cherry picked from commit e68c7b96c7)
(cherry picked from commit 6b2a853dd7)
 2016-10-05 15:09:13 -07:00
Kihwal Lee d03240e0bf HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp. 
(cherry picked from commit 44f48ee96e)
 2016-10-04 15:29:39 -05:00
Kihwal Lee 33cafc9fcd HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah. 
(cherry picked from commit 382307cbdd)
 2016-10-04 11:13:39 -05:00
Brahma Reddy Battula 253e2442e4 HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore 
(cherry picked from commit ef7f06f7d1)
 2016-10-04 18:53:12 +05:30
Brahma Reddy Battula 1ae35b6fff HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N 
(cherry picked from commit 5ea0210c7f)
 2016-10-04 18:39:31 +05:30
Mingliang Liu 438402a710 HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger 
(cherry picked from commit f61e3d13e9)
 2016-10-03 22:44:22 -04:00
Mingliang Liu 11ea5a59c2 HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula 
(cherry picked from commit d963ecb918)
 2016-10-03 22:04:49 -04:00
Xiao Chen bf043e63f9 HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 
(cherry picked from commit 853d65a157)
 2016-10-03 16:02:11 -07:00
Xiaoyu Yao b0b29de8ab HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu. 
(cherry picked from commit 607705c488)
 2016-10-03 11:05:58 -07:00
Kihwal Lee 52c534cd0f HDFS-10940. Reduce performance penalty of block caching when not used. Contributed by Daryn Sharp. 
(cherry picked from commit 03b797a6ac)
 2016-10-03 11:42:06 -05:00
Kihwal Lee 84b20d1c18 HDFS-10945. Fix the Findbugwaring FSNamesystem#renameTo() in branch-2. Contributed by Brahma Reddy Battula. 2016-10-03 11:09:40 -05:00
Kihwal Lee 3a89a8814f HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp. 
(cherry picked from commit 90020624b0)
 2016-10-03 09:26:41 -05:00
Arpit Agarwal c31d23db35 Revert "HDFS-10923. Make InstrumentedLock require ReentrantLock." 
This reverts commit 9d4e0168d5.
 2016-09-30 23:12:16 -07:00
Arpit Agarwal 9d4e0168d5 HDFS-10923. Make InstrumentedLock require ReentrantLock. 2016-09-30 23:04:30 -07:00
Kihwal Lee bfc7f0630b HDFS-10851. FSDirStatAndListingOp: stop passing path as string. Contributed by Daryn Sharp. 
(cherry picked from commit 2551ff80b7)
 2016-09-30 16:02:26 -05:00
Zhe Zhang 345debf22e HDFS-10896. Move lock logging logic from FSNamesystem into FSNamesystemLock. Contributed by Erik Krogen. 
(cherry-picked from commit 434c5ea75d)
 2016-09-30 13:43:49 -07:00
Kihwal Lee f5b193b482 HDFS-10850. getEZForPath should NOT throw FNF. Contributed by Andrew Wang. 
(cherry picked from commit 0670149c88)
 2016-09-30 09:16:13 -05:00
Mingliang Liu 7467628d43 HDFS-10892. Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'. Contributed by Mingliang Liu 
(cherry picked from commit 84c6264079)
 2016-09-28 16:09:42 -07:00
Kihwal Lee 5294eec729 HDFS-10779. Rename does not need to re-solve destination. Contributed by Daryn Sharp. 
(cherry picked from commit 5f34402ada)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-09-28 18:03:58 -05:00
Andrew Wang c4f81f199f HDFS-10914. Move remnants of oah.hdfs.client to hadoop-hdfs-client. 
(cherry picked from commit 92e5e91598)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java

(cherry picked from commit 9ddaa29301ea8ae2e3cb0f1e031343d5b91f92ee)
 2016-09-28 16:03:02 -07:00
Arpit Agarwal 76f92277c3 HDFS-10824. MiniDFSCluster#storageCapacities has no effects on real capacity. Contributed by Xiaobing Zhou. 2016-09-28 15:56:39 -07:00
Brahma Reddy Battula 5f754e8638 HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki 
(cherry picked from commit e9a34ae29c)
 2016-09-28 11:16:40 +05:30
Mingliang Liu 6da9a3c603 HDFS-10915. Fix time measurement bug in TestDatanodeRestart. Contributed by Xiaobing Zhou 
(cherry picked from commit 6437ba18c5)
 2016-09-27 20:47:04 -07:00
Wangda Tan 69f91d8c48 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc)
(cherry picked from commit 269401dc83)
 2016-09-27 11:46:49 -07:00
Arpit Agarwal 3f5482bb69 HDFS-10828. Fix usage of FsDatasetImpl object lock in ReplicaMap. (Arpit Agarwal) 2016-09-27 10:02:27 -07:00
Masatake Iwasaki 756dbc505e HDFS-10426. TestPendingInvalidateBlock failed in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit f55eb981dd)
 2016-09-28 01:47:31 +09:00
Arpit Agarwal a349c54603 HDFS-10713. Throttle FsNameSystem lock warnings. Contributed by Hanisha Koneru. 2016-09-27 09:05:53 -07:00
Brahma Reddy Battula 5bc297c58f HDFS-10889. Remove outdated Fault Injection Framework documentaion. Contributed by Brahma Reddy Battula 
(cherry picked from commit df1d0f5d56)
 2016-09-27 19:35:57 +05:30
Wei-Chiu Chuang f0f3b6a66a HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 3ae652f821)
(cherry picked from commit bde787db23)
 2016-09-26 14:46:02 -07:00
Brahma Reddy Battula 17b66c894b HDFS-10869. Remove the unused method InodeId#checkId(). Contributed by Jagadesh Kiran N 
(cherry picked from commit 3e37e243ee)
 2016-09-25 11:07:21 +05:30
Wei-Chiu Chuang 367d5b2a7e HDFS-10876. Dispatcher#dispatch should log IOException stacktrace. Contributed by Manoj Govindassamy. 
(cherry picked from commit 74b3dd514c)
(cherry picked from commit 6c98455728)
 2016-09-23 13:28:59 -07:00
Konstantin V Shvachko 20540df80d HDFS-10843. Update space quota when a UC block is completed rather than committed. Contributed by Erik Krogen. 
(cherry picked from commit a5bb88c8e0)
 2016-09-23 10:49:26 -07:00
Brahma Reddy Battula aa967869a5 HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document. 
(cherry picked from commit bbdf350ff9)
 2016-09-23 19:35:34 +05:30
Masatake Iwasaki 1524e63206 HDFS-9333. Some tests using MiniDFSCluster errored complaining port in use. (iwasakims) 
(cherry picked from commit 964e546ab1)

 Conflicts:
  hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFSStriped.java

(cherry picked from commit 1b91ebb714)
 2016-09-21 10:52:00 +09:00
Xiao Chen 11ed4f5d40 HDFS-10879. TestEncryptionZonesWithKMS#testReadWrite fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit d31bef575478fe78c0600fb602e117e4b81c9887)
 2016-09-20 16:57:44 -07:00
Xiao Chen e7daeea711 HDFS-10875. Optimize du -x to cache intermediate result. Contributed by Xiao Chen. 
(cherry picked from commit e52d6e7a46)
 2016-09-19 21:45:09 -07:00
Xiao Chen 900dc70a01 HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen. 
(cherry picked from commit ea839bd48e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

(cherry picked from commit 20585ade1d9d7fbd8b9df8b1e3db5a74fd4441a6)
 2016-09-19 16:38:00 -07:00
Andrew Wang 8c24388af0 HDFS-10832. Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissions. 
(cherry picked from commit cba973f036)
(cherry picked from commit f448ce2a89)
 2016-09-19 16:38:00 -07:00
Akira Ajisaka ab57c869e2 HDFS-10862. Typos in 4 log messages. Contributed by Mehran Hassani. 
(cherry picked from commit b09a03cd7d)
(cherry picked from commit 214b6d16d5)
 2016-09-16 16:13:00 +09:00
Arpit Agarwal bdc71bd608 HDFS-10805. Reduce runtime for append test. Contributed by Gergely Novak. 2016-09-14 09:31:35 -07:00
Akira Ajisaka c06cf0f89c HDFS-10856. Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReport. Contributed by Yiqun Lin. 
(cherry picked from commit f0876b8b60)
(cherry picked from commit 764cf431ca)
 2016-09-13 14:51:29 +09:00
Yongjun Zhang b96e2cbd20 HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit d2466ec3e0)
 2016-09-12 22:13:17 -07:00
Xiao Chen 76a18915bf HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy. 
(cherry picked from commit bb6d866207)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2016-09-10 23:14:22 -07:00
Arpit Agarwal ed35f99045 HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy) 2016-09-10 18:34:28 -07:00
Arpit Agarwal 04f620c4d0 HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang. 2016-09-10 18:01:37 -07:00
Andrew Wang 912631a226 HDFS-10831. Add log when URLConnectionFactory.openConnection failed. Contributed by yunjiong zhao. 
(cherry picked from commit b07c266dca)
(cherry picked from commit 72ea641468)
 2016-09-08 18:30:36 -07:00
Arpit Agarwal 162ee0f0a4 HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula) 2016-09-06 13:52:05 -07:00
Xiao Chen e84b5c5e3e HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee. 
(cherry picked from commit f6c0b7543f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java

(cherry picked from commit 3025992bd3)
 2016-09-06 11:54:02 -07:00
Andrew Wang 9b7e079d4b HDFS-10833. Fix JSON errors in WebHDFS.md examples. 
(cherry picked from commit cbd909ce2a)
(cherry picked from commit 40eab77562)
 2016-09-02 14:51:12 -07:00
Kihwal Lee 16c166249c HDFS-10772. Reduce byte/string conversions for get listing. Contributed by Daryn Sharp. 
(cherry picked from commit a1f3293762)
 2016-09-02 12:32:34 -05:00
Kihwal Lee 1c946a2665 HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp. 
(cherry picked from commit 442dfd1bbe)
 2016-09-02 12:28:11 -05:00
Kihwal Lee bf350e4480 HDFS-10762. Pass IIP for file status related methods 
(cherry picked from commit a30f6a68fa)
 2016-09-02 12:23:21 -05:00
Kihwal Lee c2708bc5fb HDFS-9621. Consolidate FSDirStatAndListingOp#createFileStatus to let its INodesInPath parameter always include the target INode. Contributed by Jing Zhao. 
(cherry picked from commit 313f03bfda)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
 2016-09-02 12:16:08 -05:00
Kihwal Lee 69bdcd9b1e HDFS-10745. Directly resolve paths into INodesInPath. Contributed by Daryn Sharp. 
(cherry picked from commit 922d045e11)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2016-09-02 11:44:01 -05:00
Kihwal Lee 3dac73080a HDFS-10744. Internally optimize path component resolution. Contributed by Daryn Sharp. 
(cherry picked from commit 1ef8d7a638)
 2016-09-02 11:31:48 -05:00
Kihwal Lee 5a779cce8e HDFS-10711. Optimize FSPermissionChecker group membership check. Contributed by Daryn Sharp. 
(cherry picked from commit 43d1279df0)
 2016-09-02 11:24:20 -05:00
Kihwal Lee f09b04b3f5 HDFS-10673. Optimize FSPermissionChecker's internal path usage. Contributed by Daryn Sharp. 
(cherry picked from commit 438a9f047e)
(cherry picked from commit 09b06a6e9e)
 2016-09-01 17:03:48 -07:00
Zhe Zhang 08b6808489 Addendum fix for HDFS-10817 to fix failure of the added testFSReadLockLongHoldingReport in branch-2. 
(cherry picked from commit 6c600360ca)
(cherry picked from commit 776eccd873)
 2016-08-31 23:45:45 -07:00
Zhe Zhang 7adbd569a6 HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen. 
(cherry picked from commit 6f4b0d33ca)
(cherry picked from commit f87e2f5d05)
 2016-08-31 15:54:46 -07:00
Kihwal Lee e53f6fde46 HDFS-10674. Optimize creating a full path from an inode. Contributed by Daryn Sharp. 
(cherry picked from commit 22ef5286bc)
(cherry picked from commit a5d12d9c1f)
 2016-08-31 15:16:07 -07:00
Kihwal Lee 7b5e12229d HDFS-10656. Optimize conversion of byte arrays back to path string. Contributed by Daryn Sharp. 
(cherry picked from commit bebf10d245)
(cherry picked from commit ab7072838f)
 2016-08-31 15:11:27 -07:00
Kihwal Lee ce902fc80a HDFS-10729. Improve log message for edit loading failures caused by FS limit checks. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 01721dd88e)
 2016-08-31 14:12:02 -05:00
Wei-Chiu Chuang 80bb9ffb67 HDFS-10760. DataXceiver#run() should not log InvalidToken exception as an error. Contributed by Pan Yuxuan. 
(cherry picked from commit c4ee6915a1)
 2016-08-30 10:44:48 -07:00
Zhe Zhang ee28034a8a HDFS-10814. Add assertion for getNumEncryptionZones when no EZ is created. Contributed by Vinitha Reddy Gankidi. 
(cherry picked from commit 4bd45f54ee)
 2016-08-29 23:43:04 -07:00
Xiao Chen a291306510 HDFS-4210. Throw helpful exception when DNS entry for JournalNode cannot be resolved. Contributed by Charles Lamb and John Zhuge. 
(cherry picked from commit 046f6bf0f944311fc70de3a96fc6ab8211cbbbd8)
 2016-08-29 17:42:30 -07:00
Yongjun Zhang 18f7628664 HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin. 
(cherry picked from commit 5d1609ddf2)
 2016-08-29 14:15:23 -07:00
Mingliang Liu f84ae6f091 HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu) 
(cherry picked from commit 6742fb6e68)
 2016-08-29 10:19:53 -07:00