Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,617 Commits 326 Branches 371 Tags
Commit Graph

6886 Commits

Author SHA1 Message Date
Arpit Agarwal
480b69c03c HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 11:45:23 -08:00
Yiqun Lin
52ead8b90a HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 
(cherry picked from commit 7721fff74494eb7fbbbba7f8bb4b4692d880d301)
 2018-01-25 15:53:09 +08:00
Jianfei Jiang
2a36b4b1ed
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit eb2dd0868864da994612f1dd1452ea38d31f5e85)
 2018-01-25 15:31:45 +09:00
Inigo Goiri
e4ad5f7f37 HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 
(cherry picked from commit 95743c672e6b42b227a22dfa7cc16edc7bdb58bb)
 2018-01-23 19:16:27 -08:00
Lei Xu
d33050b8cd HDFS-13039. StripedBlockReader.createBlockReader leaks socket on IOException. (Lei (Eddy) Xu) 
(cherry picked from commit 2ed9d61aadaffbf56ae0ed124520edab97646b49)
 2018-01-19 16:42:22 -08:00
Inigo Goiri
6c10e0e12b HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 
(cherry picked from commit 130f8bce00e8cab59c3aa4d668e8bddd84544713)
 2018-01-19 09:48:45 -08:00
Vinayakumar B
2443692434 HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 
(cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)
 2018-01-17 13:01:52 -06:00
Lei Xu
ce394be809 HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei) 
(cherry picked from commit 3bd9ea63df769345a9d02a404cfb61323a4cd7e3)
 2018-01-17 12:59:53 -06:00
Inigo Goiri
49e285ded5 HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-01-17 12:57:14 -06:00
Yiqun Lin
cb07172e21 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-17 12:50:06 -06:00
Manoj Govindassamy
710555e150 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-17 12:44:09 -06:00
Brahma Reddy Battula
d03115d028 HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 
(cherry picked from commit 880b9d24ff7b5f350ec99bac9b0862009460b486)
 2018-01-16 16:22:10 +05:30
Wei-Chiu Chuang
b830040766 HDFS-11915. Sync rbw dir on the first hsync() to avoid file lost on power failure. Contributed by Vinayakumar B. 
(cherry picked from commit d6c31a3e6b60c4b8af9ae4661f16614805654e59)
 2018-01-12 10:42:58 -08:00
Inigo Goiri
5ed1eff48b HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 
(cherry picked from commit d9006d8a4e34eae78dfa1cf3be50eeb81c2fc11f)
 2018-01-09 18:55:46 -08:00
Lei Xu
92f63d04a1 HDFS-12994. TestReconstructStripedFile.testNNSendsErasureCodingTasks fails due to socket timeout. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 47563d86fe6ba1a2de934c9ed740d0aafbf72d4e)
 2018-01-09 12:26:30 -08:00
Wei Yan
cbecf554c9 HDFS-12945. Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods. Contributed by Wei Yan. 
(cherry picked from commit 2ee0d64aceed876f57f09eb9efe1872b6de98d2e)
 2018-01-08 14:56:22 -08:00
Inigo Goiri
53a7bd186d HDFS-12988. RBF: Mount table entries not properly updated in the local cache. Contributed by Inigo Goiri. 
(cherry picked from commit 83b513ac6d5448f3771b0b95f91e7aa7961ae2cc)
 2018-01-05 09:12:43 -08:00
Lei Xu
47773d68b2 HDFS-12860. StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 85067eaa8e352afeed9c094da3815100873c84e4)
 2018-01-04 10:33:34 -08:00
Sean Mackrory
91b59b7c28 HDFS-12913. TestDNFencingWithReplication sometimes fails because of NameNode in standby state. Contributed by Zsolt Venczel. 
(cherry picked from commit 45a47198b43a96817679d5bf9e570d405555e3eb)
 2018-01-04 08:47:02 -07:00
Lei Xu
8f9603cb9d HDFS-12915. Fix findbugs warning in INodeFile$HeaderFormat.getBlockLayoutRedundancy. (Contributed by Chris Douglas) 
(cherry picked from commit 6e3e1b8cde737e4c03b0f5279cab0239e7069a72)
 2017-12-29 12:23:33 -08:00
Xiao Chen
b7701495f4 HDFS-9023. When NN is not able to identify DN for replication, reason behind it can be logged. 
(cherry picked from commit 5bf7e594d7d54e5295fe4240c3d60c08d4755ab7)
 2017-12-28 11:55:14 -08:00
Lei Xu
2fbfaac6c0 HDFS-12938. TestErasureCodigCLI testAll failing consistently. (Contributed by Ajay Kumar) 
(cherry picked from commit b318bed01affa150d70661f263efff9a5c9422f6)
 2017-12-21 10:30:40 -08:00
Arpit Agarwal
d85576aefd HDFS-12845. JournalNode Test failures. Contributed by Bharat Viswanadham. 2017-12-19 14:12:25 -08:00
Yiqun Lin
f7b3950f27 HDFS-12937. RBF: Add more unit tests for router admin commands. Contributed by Yiqun Lin. 
(cherry picked from commit e040c97b7743469f363eeae52c8abcf4fe7c65d5)
 2017-12-19 15:34:01 +08:00
Junping Du
53b9d81f46 Add 2.8.3 release jdiff files. 
(cherry picked from commit c89f99aade575ab1f6a9836df719cda272293d90)
 2017-12-18 22:40:04 -08:00
Yiqun Lin
a56f4c2ca7 HDFS-12930. Remove the extra space in HdfsImageViewer.md. Contributed by Rahul Pathak. 
(cherry picked from commit 25a36b74528678f56c63be643c76d819d6f07840)
 2017-12-19 11:25:01 +08:00
Erik Krogen
7c648d3e72 HDFS-12818. Support multiple storages in DataNodeCluster / SimulatedFSDataset. Contributed by Erik Krogen. 
(cherry picked from commit 94576b17fbc19c440efafb6c3322f53ec78a5b55)
 2017-12-18 11:41:06 -08:00
Lei Xu
0191a862a3 HDFS-12927. Update erasure coding doc to address unsupported APIs. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 949be14b0881186d76c3b60ee2f39ce67dc1654c)
 2017-12-15 10:29:08 -08:00
Lei Xu
2821edf6d0 HDFS-12819. Setting/Unsetting EC policy shows warning if the directory is not empty. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 1c15b1751c0698bd3063d5c25f556d4821b161d2)
 2017-12-15 10:06:00 -08:00
Yiqun Lin
61e16bff63 HDFS-12895. RBF: Add ACL support for mount table. Contributed by Yiqun Lin. 
(cherry picked from commit ee028bfdf1c88a27cd925bed93ebb599a164dd2e)
 2017-12-15 14:12:00 +08:00
Xiao Chen
cb2c5e1707 HDFS-12910. Secure Datanode Starter should log the port when it fails to bind. Contributed by Stephen O'Donnell and Nanda kumar. 
(cherry picked from commit e1cb278cd0287ce47f923941147d17540b199a99)
 2017-12-14 19:23:18 -08:00
Kihwal Lee
52c36d6d4d HDFS-12907. Allow read-only access to reserved raw for non-superusers. Contributed by Rushabh S Shah. 
(cherry picked from commit f5a72424c0009c454aab6759c30f74b397a7e935)
 2017-12-14 15:00:27 -06:00
Jason Lowe
c661f30a78 HDFS-12881. Output streams closed with IOUtils suppressing write errors. Contributed by Ajay Kumar 
(cherry picked from commit 80db744ee57c52a1dc306c576c663ccc76cced4c)
 2017-12-14 09:53:56 -06:00
Yiqun Lin
925f3e62db HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 
(cherry picked from commit 91c96bdf8eb9a06193b719186b527563091d7666)
 2017-12-14 16:02:37 +08:00
Andrew Wang
58225e4030 Update CHANGES, RELEASENOTES, jdiff for 3.0.0 release. 
(cherry picked from commit d447152d4925a5f84d28a8ebd561286b39134d75)
 2017-12-13 13:05:45 -08:00
Wei-Chiu Chuang
8062a7428d HDFS-12891. Do not invalidate blocks if toInvalidate is empty. Contributed by Zsolt Venczel. 
(cherry picked from commit 55fc2d6485702a99c6d4bb261a720d1f0498af2b)
 2017-12-11 16:43:42 -08:00
Inigo Goiri
6a43fbe724 HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 
(cherry picked from commit 5cd1056ad77a2ebb0466e7bc5933337b57f6fe30)
 2017-12-11 15:16:19 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
ba874d6787 HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 
(cherry picked from commit f19638333b11da6dcab9a964e73a49947b8390fd)
 2017-12-08 08:03:23 -08:00
Xiao Chen
1d1401ad38 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 
(cherry picked from commit 19e089420999dd9d97d981dcd0abd64b6166152d)
 2017-12-07 21:06:25 -08:00
John Zhuge
9942952dd8 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 
(cherry picked from commit bf6a660232b01642b07697a289c773ea5b97217c)
 2017-12-07 21:06:25 -08:00
Lei Xu
f3143d225a HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 67662e2ac9e68f32b725c8118cf2be79a662fca5)
 2017-12-07 11:20:19 -08:00
Akira Ajisaka
a8be4b559a
HDFS-12889. Addendum patch to add missing file. 
(cherry picked from commit 6cca5b3bcb440095f12d3eda88101fa250ac000a)
 2017-12-07 13:23:58 +09:00
Xiao Chen
bf79246c5d HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 
(cherry picked from commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f)
 2017-12-05 20:48:46 -08:00
Akira Ajisaka
c81057d936
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham. 
(cherry picked from commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b)
 2017-12-06 11:41:40 +09:00
Xiao Chen
55fa509eb6 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 
(cherry picked from commit 404eab4dc0582e0384b93664ea6ee77ccd5eeebc)
 2017-12-04 10:49:58 -08:00
Chris Douglas
2399c96ee7 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 
(cherry picked from commit 42307e3c3abbfe0b83d9a2581deba327435b910f)
 2017-12-02 23:32:44 -08:00
Wei-Chiu Chuang
83b62a8eed HDFS-12836. startTxId could be greater than endTxId when tailing in-progress edit log. Contributed by Chao Sun. 
(cherry picked from commit 0faf50624580b86b64a828cdbbb630ae8994e2cd)
 2017-12-01 12:03:04 -08:00
Chris Douglas
6266c52680 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 482fd5a880994f37fc3ad9e0cc2d127737b70aef.
 2017-12-01 11:19:17 -08:00
Chris Douglas
482fd5a880 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 
(cherry picked from commit 5304698dc8c5667c33e6ed9c4a827ef57172a723)
 2017-12-01 10:30:08 -08:00
Konstantin V Shvachko
def87db9cb HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 
(cherry picked from commit 60fd0d7fd73198fd610e59d1a4cd007c5fcc7205)
 2017-11-30 18:27:16 -08:00