Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,741 Commits 326 Branches 371 Tags
Commit Graph

4111 Commits

Author SHA1 Message Date
Brahma Reddy Battula
5754772bd6 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-21 16:22:42 +05:30
Mingliang Liu
cfac82a443 HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou 
(cherry picked from commit d7d87deece66333c188e9b7c10b4b56ddb529ce9)
 2016-10-20 19:59:32 -07:00
Wei-Chiu Chuang
b4edd115b4 HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. Contributed by Yiqun Lin. 
(cherry picked from commit 6d2da38d16cebe9b82f1048f87127eecee33664c)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java

(cherry picked from commit 6f4192d77d0551b2ca847c8c1ee5b1c4a3380c36)
 2016-10-20 12:41:00 -07:00
Steve Loughran
2b575a90f3 Revert "HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula" 
This reverts commit b63dd7b63befc503a02289c1eabd0bd0da48e223.
 2016-10-20 19:43:16 +01:00
Brahma Reddy Battula
b63dd7b63b HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-20 21:49:21 +05:30
Kihwal Lee
f57d81955e HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah. 
(cherry picked from commit 5c0bffddc0cb824a8a2751bcd0dc3e15ce081727)
 2016-10-20 11:02:57 -05:00
Brahma Reddy Battula
d2fde9c8b0 HDFS-9480. Expose nonDfsUsed via StorageTypeStats. Contributed by Brahma Reddy Battula 
(cherry picked from commit 4c73be135ca6ee2ba0b075a507097900db206b09)
 2016-10-20 20:51:41 +05:30
Brahma Reddy Battula
f5b080e152 HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin 
(cherry picked from commit 73504b1bdc4b93c64741de5eb9d022817fdfa22f)
 2016-10-20 10:47:16 +05:30
Arpit Agarwal
e9592f14d5 HDFS-10752. Several log refactoring/improvement suggestion in HDFS. Contributed by Hanisha Koneru. 2016-10-19 17:22:23 -07:00
Xiao Chen
d22a6a8b82 HDFS-11009. Add a tool to reconstruct block meta file from CLI. 
(cherry picked from commit bc4a32aea50e86819730312e89315c0244ce64bf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetUtil.java

(cherry picked from commit ad7d3c4db8bfab007cc2ec1bad3c388fd7144369)
 2016-10-18 22:42:09 -07:00
Xiaoyu Yao
190a53b898 HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru. 
(cherry picked from commit 0c79d5af74944363f3d04f7e1702411db564bbfd)
 2016-10-18 14:09:03 -07:00
Kai Zheng
d26e11f30e HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R 2016-10-18 15:09:25 +06:00
Vinitha Reddy Gankidi
ae972dd080 HDFS-10712. TestDataNodeVolumeFailure should pass not null BlockReportContext. Contributed by Vinitha Gankidi. 
(cherry picked from commit 278eeb105611444bee0261ce7a0e6ea983a5c615)
 2016-10-17 19:02:29 -07:00
Xiaoyu Yao
f8b2d77207 HDFS-10922. Adding additional unit tests for Trash (II). Contributed by Weiwei Yang. 
(cherry picked from commit f1802d0be05ecc0b3248690b6f9efedbc7784112)
 2016-10-17 14:04:58 -07:00
Brahma Reddy Battula
1c473898aa HDFS-11003. Expose XmitsInProgress through DataNodeMXBean. Contributed By Brahma Reddy Battula 
(cherry picked from commit 5f4ae85bd8a20510948696467873498723b06477)
 2016-10-15 22:40:57 +05:30
Vinitha Reddy Gankidi
2304501bcf HDFS-10301. Remove FBR tracking state to fix false zombie storage detection for interleaving block reports. Contributed by Vinitha Gankidi. 
(cherry picked from commit 391ce535a739dc92cb90017d759217265a4fd969)
 2016-10-14 18:52:00 -07:00
Lei Xu
b9fdbd7107 HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei) 
(cherry picked from commit 78cb79fa915ad0ca26d649ad88fdb44b310a99d8)
 2016-10-14 15:01:37 -07:00
Mingliang Liu
15b1d522ee HDFS-11008. Change unit test for testing parsing "-source" parameter in Balancer CLI. Contributed by Mingliang Liu 
(cherry picked from commit 76cc84e6d41c2b02218c2c98d60481cd565e067c)
 2016-10-14 14:44:34 -07:00
Kihwal Lee
8d1fbf786b Revert "HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei)" 
This reverts commit f981dd1bca1006f34f55361ba0d72e5d0a621753.
 2016-10-14 16:37:51 -05:00
Lei Xu
f981dd1bca HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei) 
(cherry picked from commit 8c520a27cbd9daba05367d3a83017a2eab5258eb)
 2016-10-14 13:44:18 -07:00
Andrew Wang
f13b84b4eb HDFS-10883. 's behavior is not consistent in DFS after enabling EZ. Contributed by Yuanbo Liu. 
(cherry picked from commit 0007360c3344b3485fa17de0fd2015a628de947c)
 2016-10-14 11:41:42 -07:00
Kihwal Lee
8c71fc1221 HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin. 
(cherry picked from commit 8a9f6635a33e9648e9396e9ec5571fa34aa0c773)
 2016-10-14 11:44:25 -05:00
Brahma Reddy Battula
0884c31aec HDFS-10986. DFSAdmin should log detailed error message if any. Contributed by MingLiang Liu 2016-10-14 10:28:13 +05:30
Andrew Wang
3892fd810b Revert "HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin." 
This reverts commit 7a5aaa789d9160ac55f022aae64a0b437ddccf06.
 2016-10-13 13:23:01 -07:00
Kihwal Lee
ded91992ad HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c5a13037048eb1e3b5a500aeec0e2e953e7d509a)
 2016-10-13 14:57:44 -05:00
Andrew Wang
7a5aaa789d HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin. 
(cherry picked from commit fdce515091f0a61ffd6c9ae464a68447dedf1124)
(cherry picked from commit 2e153bc8abc3fe2f50660136af32711c50e7e7e7)
 2016-10-13 11:42:23 -07:00
Kihwal Lee
78777d4a96 HDFS-11000. webhdfs PUT does not work if requests are routed to call queue. Contributed by Kihwal Lee. 
(cherry picked from commit 9454dc5e8091354cd0a4b8c8aa5f4004529db5d5)
 2016-10-13 09:02:59 -05:00
Akira Ajisaka
c7780b066d HDFS-11002. Fix broken attr/getfattr/setfattr links in ExtendedAttributes.md. Contributed by Mingliang Liu. 
(cherry picked from commit 901eca004d0e7e413b109a93128892176c808d61)
(cherry picked from commit 43cf0b29732ebafb8b5b23b06d6015fe4c7777b4)
 2016-10-13 14:31:09 +09:00
Mingliang Liu
9b2a719031 HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang 
(cherry picked from commit b36aaa913c54829adbf5fa852af7f45be4db0f07)
 2016-10-12 17:41:07 -07:00
Kihwal Lee
136c6f6f7d HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah. 
(cherry picked from commit 5305a392c39d298ecf38ca2dfd2526adeee9cd38)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
 2016-10-12 15:38:34 -05:00
Mingliang Liu
0f481e2022 HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou 2016-10-11 17:18:48 -07:00
Xiaoyu Yao
6f2de54eb1 HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle. 
(cherry picked from commit 61f0490a73085bbaf6639d9234277e59dc1145db)
 2016-10-11 16:00:17 -07:00
Wei-Chiu Chuang
aeb75213d4 HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma. 
(cherry picked from commit a6b3dc69d7a0835fa9159a0536c24ff6525e6c77)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2016-10-11 13:18:32 -07:00
Kihwal Lee
7f514e0c05 HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp. 
(cherry picked from commit e57fa81d9559a93d77fd724f7792326c31a490be)
 2016-10-11 09:41:47 -05:00
Mingliang Liu
5a39a7560a HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou 2016-10-10 17:46:31 -07:00
Mingliang Liu
1c7baa505b HDFS-10988. Refactor TestBalancerBandwidth. Contributed by Brahma Reddy Battula 
(cherry picked from commit b963818621c200160bb37624f177bdcb059de4eb)
 2016-10-10 13:26:48 -07:00
Xiao Chen
998e44a972 HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory. 
(cherry picked from commit 3ea027e970ec7c95a10e581435cdb6c3a6b64abb)
 2016-10-07 17:31:42 -07:00
Kihwal Lee
0cc64759ab HDFS-10979. Pass IIP for FSDirDeleteOp methods. Contributed by Daryn Sharp. 
(cherry picked from commit 3565c9af17ab05bf9e7f68b71b6c6850df772bb9)
 2016-10-07 14:38:11 -05:00
Kihwal Lee
d55b7bf1c7 HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn Sharp. 
(cherry picked from commit 76c03acc6df8f379e9e1b09b82d733337b9db655)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-10-07 13:25:56 -05:00
Kihwal Lee
29afba8a15 HDFS-10955. Pass IIP for FSDirAttr methods. Contributed by Daryn Sharp. 
(cherry picked from commit e3a9666d285c62afb0d50abea74d9e2ffe2767a8)
 2016-10-06 16:37:53 -05:00
Chris Nauroth
e36b665f90 HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth. 
(cherry picked from commit 1d330fbaf6b50802750aa461640773fb788ef884)
(cherry picked from commit a28ffd0fdeff64a12612e60f4ebc5e6311b4b112)
 2016-10-06 13:24:54 -07:00
Brahma Reddy Battula
a4ee4f9e51 HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen 
(cherry picked from commit b90fc70d671481564e468550c770c925f25d7db0)
 2016-10-06 17:49:48 +05:30
Andrew Wang
a2cdd2215e HDFS-10962. TestRequestHedgingProxyProvider is flaky. 
(cherry picked from commit e68c7b96c76b243da53ae539e7c760bf2c08f26e)
(cherry picked from commit 6b2a853dd797c27cab988452f0f5d1f42e9b4bdd)
 2016-10-05 15:09:13 -07:00
Kihwal Lee
d03240e0bf HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp. 
(cherry picked from commit 44f48ee96ee6b2a3909911c37bfddb0c963d5ffc)
 2016-10-04 15:29:39 -05:00
Kihwal Lee
33cafc9fcd HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah. 
(cherry picked from commit 382307cbdd94107350fe6fad1acf87d63c9be9d6)
 2016-10-04 11:13:39 -05:00
Brahma Reddy Battula
253e2442e4 HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore 
(cherry picked from commit ef7f06f7d1561db13bd3b07a5f62815ec29e1cdf)
 2016-10-04 18:53:12 +05:30
Brahma Reddy Battula
1ae35b6fff HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N 
(cherry picked from commit 5ea0210c7fea1324973e7f69256bc53f31ee4cf7)
 2016-10-04 18:39:31 +05:30
Mingliang Liu
438402a710 HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger 
(cherry picked from commit f61e3d13e9610cbd09886359553f27d6480f6735)
 2016-10-03 22:44:22 -04:00
Mingliang Liu
11ea5a59c2 HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula 
(cherry picked from commit d963ecb9185b8fcfba86bec8420a8daf7140a071)
 2016-10-03 22:04:49 -04:00
Xiao Chen
bf043e63f9 HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 
(cherry picked from commit 853d65a157362661ccab10379c2d82e780382f83)
 2016-10-03 16:02:11 -07:00