Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,892 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

7189 Commits

Author SHA1 Message Date
Inigo Goiri 90060d52f6 HDFS-13648. Fix TestGetConf#testGetJournalNodes on Windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 8f83b9abf9)
 2018-06-02 20:20:26 -07:00
Inigo Goiri 8a9c2d7b97 HDFS-13651. TestReencryptionHandler fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit e308ac88d3)
 2018-06-02 20:13:21 -07:00
Inigo Goiri fdacc8a088 HDFS-13637. RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN_VALUE. Contributed by CR Hota. 
(cherry picked from commit e11d674049)
 2018-06-01 16:59:35 -07:00
Sean Mackrory 58b5b3aa75 HDFS-13582. Improve backward compatibility for HDFS-13176 (WebHdfs file path gets truncated when having semicolon (;) inside). Contributed by Zsolt Venczel. 
(cherry picked from commit 1361030e59)
 2018-05-31 22:23:19 -06:00
Chao Sun 2bbdf362cf HDFS-13602. Add checkOperation(WRITE) checks in FSNamesystem. Contributed by Chao Sun. 
(cherry picked from commit ff013d2c95)
 2018-05-31 17:39:46 -07:00
Konstantin V Shvachko 442dd87dcd HDFS-12978. Fine-grained locking while consuming journal stream. Contributed by Konstantin Shvachko. 2018-05-31 14:58:06 -07:00
Inigo Goiri ad11e40557 HDFS-13631. TestDFSAdmin#testCheckNumOfBlocksInReportCommand should use a separate MiniDFSCluster path. Contributed by Anbang Hu. 
(cherry picked from commit 1f324e9661)
 2018-05-31 10:13:14 -07:00
Inigo Goiri 5c09dadd73 HDFS-13629. Some tests in TestDiskBalancerCommand fail on Windows due to MiniDFSCluster path conflict and improper path usage. Contributed by Anbang Hu. 
(cherry picked from commit 47c31ff16b)
 2018-05-30 10:23:00 -07:00
Inigo Goiri 6dc40bab4e HDFS-13632. Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA. Contributed by Anbang Hu. 
(cherry picked from commit e1102865e6e16babaff9346212c824cb1f47c31c)
 2018-05-30 10:02:50 -07:00
Inigo Goiri bc80290f28 HDFS-13591. TestDFSShell#testSetrepLow fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 9dbf4f0166)
 2018-05-28 16:34:31 -07:00
Inigo Goiri cc696f80d3 HDFS-13627. TestErasureCodingExerciseAPIs fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 91d7c74e6a)
 2018-05-28 10:27:23 -07:00
Takanobu Asanuma 499a90f152
HDFS-13628. Update Archival Storage doc for Provided Storage 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 04757e5864)
 2018-05-28 19:06:58 +09:00
Inigo Goiri c7d77a3bfe HDFS-13620. Randomize the test directory path for TestHDFSFileSystemContract. Contributed by Anbang Hu. 
(cherry picked from commit 8605a38514)
 2018-05-25 19:45:48 -07:00
Inigo Goiri 71ab85adce HDFS-13619. TestAuditLoggerWithCommands fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 13d2528907)
 2018-05-25 13:33:05 -07:00
Inigo Goiri 2d3d7a67f2 HDFS-13618. Fix TestDataNodeFaultInjector test failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit a8e9ee3fdf3aa1b889dfdd3bade91ba754ed9ee1)
 2018-05-25 09:11:07 -07:00
Andrew Wang 31d597f0c7 HDFS-13611. Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient. 
(cherry picked from commit c9b63deb53)
 2018-05-24 09:56:37 -07:00
Inigo Goiri f57e91a348 HDFS-13493. Reduce the HttpServer2 thread count on DataNodes. Contributed by Erik Krogen. 
(cherry picked from commit cddbbe5f69)
 2018-05-23 12:12:40 -07:00
Inigo Goiri a138496379 HDFS-13587. TestQuorumJournalManager fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit c13dea87d9)
 2018-05-23 11:36:34 -07:00
Bharat Viswanadham d0d3d5b2ad HDFS-13589: Add dfsAdmin command to query if upgrade is finalized. Contributed by Hanisha Koneru 
(cherry picked from commit bc6d9d4c79)
 2018-05-23 10:23:45 -07:00
Inigo Goiri 0c268a0755 HDFS-13588. Fix TestFsDatasetImpl test failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit c0c9b7a8ef)
 2018-05-23 09:47:10 -07:00
Sammi Chen 7d71b3a1cc HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen. 
(cherry picked from commit 34e8b9f9a8)
 2018-05-23 19:12:04 +08:00
Andrew Wang d47c0fc39a HDFS-13601. Optimize ByteString conversions in PBHelper. 
(cherry picked from commit 1d2640b613)
 2018-05-22 23:55:35 -07:00
Inigo Goiri 8a86269c51 HDFS-13593. TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 9775ecb235)
 2018-05-18 09:46:29 -07:00
Inigo Goiri da5eff8325 HDFS-13592. TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly. Contributed by Anbang Hu. 
(cherry picked from commit 57b893de3d)
 2018-05-18 09:38:23 -07:00
Inigo Goiri df54cdbaea HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 53b807a6a8)
 2018-05-17 17:03:56 -07:00
Inigo Goiri f67865ed7e HDFS-13556. TestNestedEncryptionZones does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit a97a2042f2)
 2018-05-17 16:53:53 -07:00
Inigo Goiri 8c463a2a9c HDFS-13554. TestDatanodeRegistration#testForcedRegistration does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit 65476458fa)
 2018-05-17 14:48:40 -07:00
Inigo Goiri e5ff9f3922 HDFS-13558. TestDatanodeHttpXFrame does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit 26f1e22fc9)
 2018-05-17 13:35:40 -07:00
Inigo Goiri b0c89690ed HDFS-13570. TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows. Contributed by Anbang Hu. 
(cherry picked from commit c0ec061e28)
 2018-05-17 09:22:29 -07:00
Kihwal Lee 4f1ec6a515 HDFS-13581. DN UI logs link is broken when https is enabled. Contributed by Shashikant Banerjee. 
(cherry picked from commit 0ce6290de6)
 2018-05-17 09:26:13 -05:00
Inigo Goiri 1d1806f862 HDFS-13559. TestBlockScanner does not close TestContext properly. Contributed by Anbang Hu. 
(cherry picked from commit 454de3b543)
 2018-05-16 19:23:33 -07:00
Arpit Agarwal 45dd200a2c HDFS-13512. WebHdfs getFileStatus doesn't return ecPolicy. Contributed by Ajay Kumar. 2018-05-16 11:28:51 -07:00
Inigo Goiri e12ddd9394 HDFS-13550.TestDebugAdmin#testComputeMetaCommand fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 55d5549087)
 2018-05-16 10:09:18 -07:00
Inigo Goiri 5310542fb4 HDFS-13557. TestDFSAdmin#testListOpenFiles fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 6b67161ca3)
 2018-05-16 09:32:15 -07:00
Inigo Goiri 899a7ec2fb HDFS-13567. TestNameNodeMetrics#testGenerateEDEKTime,TestNameNodeMetrics#testResourceCheck should use a different cluster basedir. Contributed by Anbang Hu. 
(cherry picked from commit 63480976a0)
 2018-05-15 15:36:03 -07:00
Inigo Goiri 34fbc1c2eb HDFS-13548. TestResolveHdfsSymlink#testFcResolveAfs fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 2d6195c0af)
 2018-05-15 11:21:44 -07:00
Inigo Goiri c168329cc4 HDFS-11700. TestHDFSServerPorts#testBackupNodePorts doesn't pass on Windows. Contributed by Anbang Hu. 
(cherry picked from commit eaca7980ad)
 2018-05-15 10:28:01 -07:00
Inigo Goiri 583b9b418a HDFS-13551. TestMiniDFSCluster#testClusterSetStorageCapacity does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit 92ebd466c7)
 2018-05-15 10:22:19 -07:00
Hanisha Koneru 4f55941390 HDFS-13544. Improve logging for JournalNode in federated cluster. 
(cherry picked from commit 6beb25ab7e)
 2018-05-14 10:14:38 -07:00
Xiao Chen 6e731eb20d HDFS-13539. DFSStripedInputStream NPE when reportCheckSumFailure. 
(cherry picked from commit 960940e0e0)
 2018-05-14 09:29:14 -07:00
Sammi Chen e5281d9f8b Add 2.9.1 release notes and changes documents 2018-05-14 15:26:00 +08:00
Inigo Goiri e332e011ae HDFS-13542. TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows. Contributed by Anbang Hu. 
(cherry picked from commit d50c4d71dc)
 2018-05-11 09:48:45 -07:00
Yiqun Lin 3665217d50 HDFS-13346. RBF: Fix synchronization of router quota and nameservice quota. 
(cherry picked from commit a922b9c82c)
 2018-05-11 15:09:54 +08:00
Inigo Goiri e3ff9e36df HDFS-13537. TestHdfsHelper does not generate jceks path properly for relative path in Windows. Contributed by Xiao Liang. 
(cherry picked from commit 914b98a713)
 2018-05-09 16:52:45 -07:00
Inigo Goiri c649c76566 HDFS-13388. RequestHedgingProxyProvider calls multiple configured NNs all the time. Contributed by Jinglun. 
(cherry picked from commit 63803e7051)
 2018-05-09 13:56:06 -07:00
Xiao Chen 6f3bbf364c HDFS-12981. renameSnapshot a Non-Existent snapshot to itself should throw error. Contributed by Kitti Nanasi. 
(cherry picked from commit 696a4be0da)
 2018-05-07 15:39:24 -07:00
Xiao Chen d762932d79 Revert "HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445." 
This reverts commit 128b09ee43.
 2018-05-07 13:47:52 -07:00
Inigo Goiri 524dde4006 HDFS-13525. RBF: Add unit test TestStateStoreDisabledNameservice. Contributed by Yiqun Lin. 
(cherry picked from commit a3b416f69d)
 2018-05-03 11:25:30 -07:00
Yiqun Lin 5fef28d0d4 HDFS-13488. RBF: Reject requests when a Router is overloaded. Contributed by Inigo Goiri. 
(cherry picked from commit 37269261d1)
 2018-05-02 14:56:15 +08:00
Inigo Goiri 4aadfb3c92 HDFS-13503. Fix TestFsck test failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 9e2cfb2d3f)
 2018-05-01 08:13:30 -07:00
Inigo Goiri 4c13e7e3a0 HDFS-13283. Percentage based Reserved Space Calculation for DataNode. Contributed by Lukas Majercak. 
(cherry picked from commit fc074a359c)
 2018-04-30 13:31:03 -07:00
Inigo Goiri ce62991648 HDFS-13509. Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit eb7fe1d588)
 2018-04-28 09:05:59 -07:00
Inigo Goiri 9923fa2490 HDFS-13508. RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries. Contributed by Ekanth S. 
(cherry picked from commit 484440602c)
 2018-04-27 16:29:41 -07:00
Inigo Goiri f26f764857 HDFS-13434. RBF: Fix dead links in RBF document. Contributed by Chetna Chaudhari. 
(cherry picked from commit f469628bba)
 2018-04-27 15:14:18 -07:00
Yiqun Lin e097998c22 HDFS-13499. RBF: Show disabled name services in the UI. Contributed by Inigo Goiri. 
(cherry picked from commit 0a293de00d)
 2018-04-27 10:39:21 +08:00
Inigo Goiri 921a3e1678 HDFS-13326. RBF: Improve the interfaces to modify and view mount tables. Contributed by Gang Li. 
(cherry picked from commit c394051a3d4d9d531f418503cb519606ae2b2e69)
 2018-04-26 13:01:16 -07:00
Yiqun Lin 999ea44b9f HDFS-13484. RBF: Disable Nameservices from the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 30fef0bf1e5c8c0ca073df99ad9b33cb0e4431a5)
 2018-04-25 15:23:54 +08:00
Yiqun Lin ea2863f2b5 HDFS-13490. RBF: Fix setSafeMode in the Router. Contributed by Inigo Goiri. 
(cherry picked from commit b06601acce)
 2018-04-24 11:27:14 +08:00
Inigo Goiri 5b2427cd73 HDFS-13336. Test cases of TestWriteToReplica failed in windows. Contributed by Xiao Liang. 
(cherry picked from commit df92a17e02)
 2018-04-23 19:12:53 -07:00
Bharat Viswanadham caf4d5000e HDFS-13356. Balancer:Set default value of minBlockSize to 10mb. Contributed by Bharat Viswanadham 
(cherry picked from commit 9b5375e0c1)
 2018-04-23 16:10:28 -07:00
Chris Douglas cf272c5179 HDFS-13408. MiniDFSCluster to support being built on randomized base directory. Contributed by Xiao Liang 
(cherry picked from commit f411de6a79)
 2018-04-23 11:13:52 -07:00
Arpit Agarwal 1f486a0649 HDFS-13433. webhdfs requests can be routed incorrectly in federated cluster. Contributed by Arpit Agarwal. 2018-04-23 10:09:02 -07:00
Yiqun Lin 3f72d077ff HDFS-13478. RBF: Disabled Nameservice store API. Contributed by Inigo Goiri. 
(cherry picked from commit b6dae26f44)
 2018-04-21 13:35:04 +08:00
Sangjin Lee 9f648f6e59 HDFS-10183. Prevent race condition during class initialization. Contributed by Pavel Avgustinov. 
(cherry picked from commit f40969a141)
 2018-04-20 20:36:31 -07:00
Inigo Goiri 18b6a2a2ee HDFS-13453. RBF: getMountPointDates should fetch latest subdir time/date when parent dir is not present but /parent/child dirs are present in mount table. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 1134af9ad1)
 2018-04-19 14:57:09 -07:00
Xiao Chen 128b09ee43 HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445. 
(cherry picked from commit 6503593711)
 2018-04-18 22:29:59 -05:00
Shashikant Banerjee a13b3edfd8
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e4c39f3247)
 2018-04-19 08:36:32 +09:00
Bharat Viswanadham 63c1b9d50c HDFS-13464. Fix javadoc in FsVolumeList#handleVolumeFailures. Contributed by Shashikant Banerjee 
(cherry picked from commit bf7694d020)
 2018-04-18 08:24:52 -07:00
Mukul Kumar Singh a07ba66ee9 HDFS-13426. Fix javadoc in FsDatasetAsyncDiskService#removeVolume. Contributed by Shashikant Banerjee. 2018-04-18 16:30:40 +05:30
Mukul Kumar Singh d99c8aa898 HDFS-13463. Fix javadoc in FsDatasetImpl#checkAndUpdate. Contributed by Shashikant Banerjee. 2018-04-18 16:30:24 +05:30
Erik Krogen fb350e8112
HDFS-12828. OIV ReverseXML Processor fails with escaped characters 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 2d9e791a90)
 2018-04-18 14:40:01 +09:00
Wei Yan 5d50f716e7 HDFS-13466. RBF: Add more router-related information to the UI. 
(cherry picked from commit e4313e7e47)
 2018-04-17 15:03:12 -07:00
Inigo Goiri 4be0fa28da HDFS-13462. Add BIND_HOST configuration for JournalNode's HTTP and RPC Servers. Contributed by Lukas Majercak. 
(cherry picked from commit c9b33514b8)
 2018-04-17 14:20:23 -07:00
Bharat Viswanadham fc11555caf HDFS-13129. Add a test for DfsAdmin refreshSuperUserGroupsConfiguration. Contributed by Mukul Kumar Singh 
(cherry picked from commit a68b04370f)
 2018-04-17 12:41:54 -07:00
Yiqun Lin 1d776f22f9 HDFS-13435. RBF: Improve the error loggings for printing the stack trace. 
(cherry picked from commit c4d3636c21)
 2018-04-17 11:25:14 +08:00
Arpit Agarwal 43a2af5569 HDFS-13439. Add test case for read block operation when it is moved. Contributed by Ajay Kumar. 2018-04-16 14:18:41 -07:00
Xiao Chen 805e33b62c HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-16 11:58:10 -07:00
Wei-Chiu Chuang ca8bb322be HDFS-13330. ShortCircuitCache#fetchOrCreate never retries. Contributed by Gabor Bota. 
(cherry picked from commit e66e287efe)
 2018-04-13 12:18:49 -07:00
Wei-Chiu Chuang 696d786063 HDFS-11900. Hedged reads thread pool creation not synchronized. Contributed by John Zhuge. 2018-04-13 10:16:23 -07:00
Xiaoyu Yao 867135af28 HDFS-11043. TestWebHdfsTimeouts fails. Contributed by Xiaoyu Yao and Chao Sun. 2018-04-13 10:14:47 -07:00
Yongjun Zhang 56acbe476b HDFS-13315. Add a test for the issue reported in HDFS-11481 which is fixed by HDFS-10997. Contributed by Yongjun Zhang. 2018-04-13 10:14:24 -07:00
Chris Douglas ffc39ec3ab HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-04-13 10:11:35 -07:00
Bharat Viswanadham 8df8cad019 HDFS-13438. Fix javadoc in FsVolumeList#removeVolume. Contributed by Shashikant Banerjee. 
(cherry picked from commit fa8b88ab2b)
 2018-04-13 09:05:24 -07:00
Yiqun Lin 16763d8229 HDFS-13418. NetworkTopology should be configurable when enable DFSNetworkTopology. Contributed by Tao Jie. 
(cherry picked from commit 0725953efe)
 2018-04-13 17:57:15 +08:00
Akira Ajisaka 5531c31f16
HDFS-13436. Fix javadoc of package-info.java 
(cherry picked from commit 9b0b9f2af2)
 2018-04-13 13:24:42 +09:00
Inigo Goiri 671e407ced Revert "HDFS-13388. RequestHedgingProxyProvider calls multiple configured NNs all the time. Contributed by Jinglun." 
This reverts commit 8c9dab978e.
 2018-04-12 08:43:12 -07:00
Akira Ajisaka 5f4762d950
HDFS-7101. Potential null dereference in DFSck#doWork(). Contributed by skrho and Ted Yu. 
(cherry picked from commit 113af12cfb)
 2018-04-12 17:48:14 +09:00
Akira Ajisaka 2d82a21b84
HDFS-13427. Fix the section titles of transparent encryption document. 
(cherry picked from commit c7cd362afd)
 2018-04-12 10:46:14 +09:00
Inigo Goiri 3c1cd08f04 HDFS-13386. RBF: Wrong date information in list file(-ls) result. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 18de6f2042)
 2018-04-11 12:03:40 -07:00
Inigo Goiri d032112e6b HDFS-13428. RBF: Remove LinkedList From StateStoreFileImpl.java. Contributed by BELUGA BEHR. 
(cherry picked from commit 6a881cd842fb4eab596a386b39670ea03536891a)
 2018-04-11 10:41:13 -07:00
Wei Yan 32efc77799 HDFS-13045. RBF: Improve error message returned from subcluster. Contributed by Inigo Goiri. 
(cherry picked from commit 0c93d43f3d)
 2018-04-11 08:38:52 -07:00
Rakesh Radhakrishnan 3414bf6dbb HDFS-13328. Abstract ReencryptionHandler recursive logic in separate class. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit f89594f0b8)
 2018-04-10 23:40:26 +05:30
Weiwei Yang b104de3f64 HDFS-13420. License header is displayed in ArchivalStorage/MemoryStorage html pages. Contributed by Akira Ajisaka. 2018-04-10 22:13:08 +08:00
Yiqun Lin 5eb32396d7 HDFS-13384. RBF: Improve timeout RPC call mechanism. Contributed by Inigo Goiri. 
(cherry picked from commit e87be8a2a4)
 2018-04-10 15:36:48 +08:00
Yiqun Lin 9a86cb0494 HDFS-13410. RBF: Support federation with no subclusters. Contributed by Inigo Goiri. 
(cherry picked from commit a92200f4a6)
 2018-04-10 14:31:59 +08:00
Wangda Tan 25bf1d3ac6 Updated branch-3.1 to track 3.1.1 
Change-Id: Idd55c79921b8acacc8a8a3258450e476f3abb706
 2018-04-09 10:58:50 -07:00
Inigo Goiri 1c9038bae5 HDFS-13380. RBF: mv/rm fail after the directory exceeded the quota limit. Contributed by Yiqun Lin. 
(cherry picked from commit e9b9f48dad)
 2018-04-09 10:09:55 -07:00
Inigo Goiri 8c9dab978e HDFS-13388. RequestHedgingProxyProvider calls multiple configured NNs all the time. Contributed by Jinglun. 
(cherry picked from commit ac32b3576d)
 2018-04-09 09:17:35 -07:00
Yiqun Lin ac5703207d HDFS-13402. RBF: Fix java doc for StateStoreFileSystemImpl. Contributed by Yiran Wu. 
(cherry picked from commit 5700556cd6)
 2018-04-08 12:03:35 +08:00
Wangda Tan a336561e2b Added CHANGES/RELEASES/Jdiff for 3.1.0 release 
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
 2018-04-05 15:52:02 -07:00
Wei Yan 89efe30f65 HDFS-13353. RBF: TestRouterWebHDFSContractCreate failed. Contributed by Takanobu Asanuma. 
(cherry picked from commit 3121e8c293)
 2018-04-05 13:13:16 -07:00
Lei Xu 2543a9dd77 HDFS-13350. Negative legacy block ID will confuse Erasure Coding to be considered as striped block. (Contributed by Lei (Eddy) Xu). 
(cherry picked from commit 1cbf23df145af01692b8aaa438642b64e330cd05)
 2018-04-05 10:01:05 -07:00
Yiqun Lin b89dc00302 HDFS-13364. RBF: Support NamenodeProtocol in the Router. Contributed by Inigo Goiri. 
(cherry picked from commit 2be64eb201)
 2018-04-03 15:23:39 +08:00
Xiao Chen dcb2176c5d HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 
(cherry picked from commit 57374c4737)
 2018-04-02 22:48:54 -07:00
Inigo Goiri 3c9983547c HDFS-13289. RBF: TestConnectionManager#testCleanup() test case need correction. Contributed by Dibyendu Karmakar. 
(cherry picked from commit dc8e343201)
 2018-03-31 09:46:44 -07:00
Konstantin V Shvachko 4ce45a8ec7 HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 
(cherry picked from commit acfd764fcc)
 2018-03-30 18:57:18 -07:00
Xiao Chen 2960592c6f HDFS-13087. Snapshotted encryption zone information should be immutable. Contributed by LiXin Ge. 
(cherry picked from commit 2c6cfad5a3)
 2018-03-29 16:07:01 -07:00
Arpit Agarwal d54e19a913 HDFS-13314. NameNode should optionally exit if it detects FsImage corruption. Contributed by Arpit Agarwal. 2018-03-28 12:49:14 -07:00
Yiqun Lin 99766992e6 HDFS-13347. RBF: Cache datanode reports. Contributed by Inigo Goiri. 
(cherry picked from commit a71656c1c1)
 2018-03-28 11:04:36 +08:00
Inigo Goiri c1fc87dcce HDFS-13352. RBF: Add xsl stylesheet for hdfs-rbf-default.xml. Contributed by Takanobu Asanuma. 
(cherry picked from commit 4cef01147f)
 2018-03-27 10:32:07 -07:00
Inigo Goiri 0dc16559ee HDFS-13204. RBF: Optimize name service safe mode icon. Contributed by liuhongtong. 
(cherry picked from commit 947b282984)
 2018-03-26 09:43:28 -07:00
Yiqun Lin 1e01ddfe4f HDFS-13291. RBF: Implement available space based OrderResolver. Contributed by Yiqun Lin. 
(cherry picked from commit cfc3a1c8f0)
 2018-03-26 18:36:22 +08:00
Lei Xu 4859cd7cc9 Update 3.0.1 jdiff file and jdiff stable api version 2018-03-23 11:53:30 -07:00
weiy d17aa8d052 HDFS-12512. RBF: Add WebHDFS. 
(cherry picked from commit 6e31a09084)
 2018-03-23 08:40:04 -07:00
Inigo Goiri 59ab86311c HDFS-12792. RBF: Test Router-based federation using HDFSContract. Contributed by Inigo Goiri. 
(cherry picked from commit e196d158a2)
 2018-03-22 13:33:24 -07:00
Kihwal Lee c245ecedda HDFS-13195. DataNode conf page cannot display the current value after reconfig. Contributed by maobaolong. 
(cherry picked from commit 66461ed98b)
 2018-03-22 12:28:34 -05:00
Inigo Goiri 485348a337 HDFS-13318. RBF: Fix FindBugs in hadoop-hdfs-rbf. Contributed by Ekanth S. 
(cherry picked from commit 3d0ee8b804)
 2018-03-22 09:22:16 -07:00
Konstantin V Shvachko 21db4e9efe HDFS-12884. BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo. Contributed by chencan. 
(cherry picked from commit 8d898ab25f)
 2018-03-21 16:51:33 -07:00
Yiqun Lin 0db4b587c6 HDFS-13250. RBF: Router to manage requests across multiple subclusters. Contributed by Inigo Goiri. 
(cherry picked from commit 2caba999bb)
 2018-03-21 11:40:02 +08:00
Yiqun Lin 131546002d HDFS-13307. RBF: Improve the use of setQuota command. Contributed by liuhongtong. 
(cherry picked from commit 69fe4407eb)
 2018-03-21 10:53:24 +08:00
weiy c804b28462 HDFS-13215. RBF: Move Router to its own module. 2018-03-19 23:05:33 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 0bd99621c5 Preparing for 3.1.0 development 2018-03-19 15:16:54 -07:00
Inigo Goiri 08ff1586d5 HDFS-12886. Ignore minReplication for block recovery. Contributed by Lukas Majercak. 2018-03-16 10:29:19 -07:00
Xiaoyu Yao 78b05fde6c HDFS-13280. WebHDFS: Fix NPE in get snasphottable directory list call. Contributed by Lokesh Jain. 2018-03-15 11:02:27 -07:00
Inigo Goiri e71bc00a47 HDFS-13224. RBF: Resolvers to support mount points across multiple subclusters. Contributed by Inigo Goiri. 2018-03-15 10:32:30 -07:00
Xiaoyu Yao da777a5498 HDFS-13251. Avoid using hard coded datanode data dirs in unit tests(addendum). Contributed by Ajay Kumar. 2018-03-15 10:18:44 -07:00
Inigo Goiri 6de135169e HDFS-12723. TestReadStripedFileWithMissingBlocks#testReadFileWithMissingBlocks failing consistently. Contributed by Ajay Kumar. 2018-03-15 10:14:35 -07:00
fang zhenyi 5e013d50d1
HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-15 16:12:07 +09:00
Xiao Chen 5ff22d4c3a HDFS-13246. FileInputStream redundant closes in readReplicasFromCache. Contributed by liaoyuxiangqin. 2018-03-14 21:59:51 -07:00
Yiqun Lin 3a0f4bc0d5 HDFS-13261. Fix incorrect null value check. Contributed by Jianfei Jiang. 2018-03-15 11:06:04 +08:00
Inigo Goiri 41fc7f80be HDFS-13198. RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router. Contributed by Wei Yan. 2018-03-14 13:14:36 -07:00
Xiaoyu Yao f83716b7f2 HDFS-13251. Avoid using hard coded datanode data dirs in unit tests. Contributed by Ajay Kumar. 2018-03-14 11:26:58 -07:00
Xiaoyu Yao 0be0f1ce1d HDFS-13275. Adding log for BlockPoolManager#refreshNamenodes failures. Contributed by Ajay Kumar. 2018-03-14 10:38:00 -07:00
Arpit Agarwal 4c57fb0cd9 HDFS-13257. Code cleanup: INode never throws QuotaExceededException. Contributed by Tsz Wo Nicholas Sze. 2018-03-14 09:42:48 -07:00
Xiaoyu Yao a34d83d2fd HDFS-13249. Document webhdfs support for getting snapshottable directory list. Contributed by Lokesh Jain. 2018-03-14 09:14:26 -07:00
Sammi Chen ad1b988a82 HDFS-11600. Refactor TestDFSStripedOutputStreamWithFailure test classes. Contributed by Sammi Chen. 2018-03-14 14:39:53 +08:00
Yiqun Lin 76be6cbf6c HDFS-12773. RBF: Improve State Store FS implementation. Contributed by Inigo Goiri. 2018-03-14 11:20:59 +08:00
Arpit Agarwal 9714fc1dd4 HDFS-336. dfsadmin -report should report number of blocks from datanode. Contributed by Bharat Viswanadham. 2018-03-13 16:39:17 -07:00
Arpit Agarwal 39537b7c84 HDFS-13235. DiskBalancer: Update Documentation to add newly added options. Contributed by Bharat Viswanadham. 2018-03-13 16:35:51 -07:00
Xiao Chen 8211a3d469 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 2018-03-13 10:48:45 -07:00
Xiao Chen f82d38dcb3 Revert "HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma." 
This reverts commit 84c1095586.
 2018-03-13 10:36:16 -07:00
Xiao Chen 3b8dbc2cb7 HDFS-12587. Use Parameterized tests in TestBlockInfoStriped and TestLowRedundancyBlockQueues to test all EC policies. Contributed by Takanobu Asanuma. 2018-03-13 10:14:05 -07:00
Xiao Chen d6931c30c5 HDFS-13239. Fix non-empty dir warning message when setting default EC policy. Contributed by Bharat Viswanadham. 2018-03-13 10:07:26 -07:00
Xiao Chen 84c1095586 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 2018-03-13 09:58:03 -07:00
Chris Douglas b2b9ce5859 HDFS-13271. WebHDFS: Add constructor in SnapshottableDirectoryStatus with HdfsFileStatus as argument. Contributed by Lokesh Jain 2018-03-13 09:43:22 -07:00
Xiaoyu Yao 0355ec20eb HDFS-13141. WebHDFS: Add support for getting snasphottable directory list. Contributed by Lokesh Jain. 2018-03-12 20:41:37 -07:00
Chris Douglas 45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Yiqun Lin 19292bc264 HDFS-13226. RBF: Throw the exception if mount table entry validated failed. Contributed by maobaolong. 2018-03-13 11:03:31 +08:00
Yiqun Lin 7fab787de7 HDFS-13253. RBF: Quota management incorrect parent-child relationship judgement. Contributed by Yiqun Lin. 2018-03-13 10:30:20 +08:00
Akira Ajisaka 319defafc1
HDFS-12156. TestFSImage fails without -Pnative 2018-03-13 11:26:56 +09:00
Yiqun Lin 4afd50b106 HDFS-10803. TestBalancerWithMultipleNameNodes#testBalancing2OutOf3Blockpools fails intermittently due to no free space available. Contributed by Yiqun Lin. 2018-03-13 10:15:51 +08:00
Inigo Goiri 91c82c90f0 HDFS-13241. RBF: TestRouterSafemode failed if the port 8888 is in use. Contributed by maobaolong. 2018-03-12 17:28:15 -07:00
Chris Douglas 39a5fbae47 HDFS-12677. Extend TestReconstructStripedFile with a random EC policy. Contributed by Takanobu Asanuma 2018-03-12 14:29:44 -07:00
Chris Douglas ddb67ca707 HADOOP-14742. Document multi-URI replication Inode for ViewFS. Contributed by Gera Shegalov 2018-03-12 13:43:27 -07:00
Anu Engineer cceb68ffea HDFS-10618. TestPendingReconstruction#testPendingAndInvalidate is flaky due to race condition. Contributed by Eric Badger. 2018-03-12 12:07:22 -07:00
Inigo Goiri ac627f561f HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin. 2018-03-12 09:58:56 -07:00
Yiqun Lin 4743d4a2c7 HDFS-13240. RBF: Update some inaccurate document descriptions. Contributed by Yiqun Lin. 2018-03-10 11:28:55 +08:00
Inigo Goiri 8133cd5305 HDFS-13232. RBF: ConnectionPool should return first usable connection. Contributed by Ekanth S. 2018-03-09 18:25:05 -08:00
Inigo Goiri afe1a3ccd5 HDFS-13212. RBF: Fix router location cache issue. Contributed by Weiwei Wu. 2018-03-09 17:18:51 -08:00
Tsz-Wo Nicholas Sze ba0da2785d HDFS-13252. Code refactoring: Remove Diff.ListType. 2018-03-09 15:50:26 -08:00
Hanisha Koneru 4eeff62f69 HDFS-13244. Add stack, conf, metrics links to utilities dropdown in NN webUI. Contributed by Bharat Viswanadham. 2018-03-09 15:27:17 -08:00
Lokesh Jain 7b0dc31020
HDFS-13190. Document WebHDFS support for snapshot diff 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
 2018-03-09 15:06:15 -08:00
Hanisha Koneru 9a082fbe6e HDFS-11394. Support for getting erasure coding policy through WebHDFS#FileStatus. 2018-03-09 13:20:32 -08:00
Yiqun Lin 122805b43a HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan. 2018-03-09 15:42:57 +08:00
Inigo Goiri 0c2b969e01 HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 2018-03-08 09:32:05 -08:00
Hanisha Koneru 19ae4429f9 HDFS-13225. StripeReader#checkMissingBlocks() 's IOException info is incomplete. Contributed by lufei. 2018-03-07 14:17:10 -08:00
Sean Mackrory 46d29e3d7e HDFS-13176. WebHdfs file path gets truncated when having semicolon (;) inside. Contributed by Zsolt Venczel. 2018-03-07 13:33:41 -07:00
Tsz-Wo Nicholas Sze 88fba00caa HDFS-13222. Update getBlocks method to take minBlockSize in RPC calls. Contributed by Bharat Viswanadham 2018-03-07 11:27:53 -08:00
Yiqun Lin 58ea2d7a65 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 2018-03-07 15:26:38 +08:00
Xiaoyu Yao edf9445708 HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru. 2018-03-06 16:44:20 -08:00
Tsz-Wo Nicholas Sze 346caa2095 HDFS-13227. Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList. Contributed by Shashikant Banerjee 2018-03-06 13:35:03 -08:00
Tsz-Wo Nicholas Sze 871d0d39fa HDFS-13223. Reduce DiffListBySkipList memory usage. Contributed by Shashikant Banerjee 2018-03-06 12:23:03 -08:00
Xiao Chen 9276ef0665 HDFS-13170. Port webhdfs unmaskedpermission parameter to HTTPFS. Contributed by Stephen O'Donnell. 2018-03-06 09:57:35 -08:00
Inigo Goiri 7060725662 HDFS-13188. Disk Balancer: Support multiple block pools during block move. Contributed by Bharat Viswanadham. 2018-03-06 09:09:32 -08:00
Robert Kanter 21176a8ba7
HADOOP-15282. HADOOP-15235 broke TestHttpFSServerWebServer 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-04 07:33:47 -08:00
Arpit Agarwal a3c304c5dd HDFS-13178. Disk Balancer: Add a force option to DiskBalancer Execute command. Contributed by Bharat Viswanadham. 2018-03-03 15:42:16 -08:00
Tsz-Wo Nicholas Sze ba82e5c488 HDFS-13173. Replace ArrayList with DirectoryDiffList(SnapshotSkipList) to store DirectoryDiffs. Contributed by Shashikant Banerjee 2018-03-02 17:47:48 -08:00
Xiao Chen c75105f07b HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 2018-03-02 16:46:04 -08:00
Tsz-Wo Nicholas Sze 60080fbfcf HDFS-13171. Handle Deletion of nodes in SnasphotSkipList. Contributed by Shashikant Banerjee 2018-03-02 10:51:05 -08:00
Tsz-Wo Nicholas Sze 850b2f2567 HDFS-1686. Federation: Add more Balancer tests with federation setting. Contributed by Bharat Viswanadham 2018-03-02 10:42:39 -08:00
Yiqun Lin 55669515f6 HDFS-13210. Fix the typo in MiniDFSCluster class. Contributed by fang zhenyi. 2018-03-02 11:49:12 +08:00
Tsz-Wo Nicholas Sze 96e8f260ab HDFS-13211. Fix a bug in DirectoryDiffList.getMinListForRange. Contributed by Shashikant Banerjee 2018-03-01 14:12:15 -08:00
Yiqun Lin ececeea8f2 HDFS-13202. Fix the outdated javadocs in HAUtil. Contributed by Chao Sun. 2018-03-01 17:51:43 +08:00
Tsz-Wo Nicholas Sze 81d9446a92 HDFS-13102. Implement SnapshotSkipList class to store Multi level DirectoryDiffs. Contributed by Shashikant Banerjee 2018-02-28 13:42:58 -08:00
Xiaoyu Yao f20e10b2dd HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 2018-02-28 09:44:39 -08:00
Xiaoyu Yao 2574375bf5 HDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path. Contributed by Hanisha Koneru. 2018-02-28 08:37:01 -08:00
Yiqun Lin a9c14b1119 HDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang. 2018-02-28 10:17:43 +08:00
Inigo Goiri d86f301d46 HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 2018-02-27 17:39:02 -08:00
Tsz-Wo Nicholas Sze 55c77bf722 HDFS-13143. SnapshotDiff - snapshotDiffReport might be inconsistent if the snapshotDiff calculation happens between a snapshot and the current tree. Contributed by Shashikant Banerjee 2018-02-27 15:28:41 -08:00
Arpit Agarwal 1cc9a58dda HDFS-13181. DiskBalancer: Add an configuration for valid plan hours . Contributed by Bharat Viswanadham. 2018-02-27 10:18:07 -08:00
Inigo Goiri cd4a97bf80 HDFS-13192. Change the code order in getFileEncryptionInfo to avoid unnecessary call of assignment. Contributed by LiXin Ge. 2018-02-27 08:53:00 -08:00
Inigo Goiri 225d877092 HDFS-13193. Various Improvements for BlockTokenSecretManager. Contributed by BELUGA BEHR. 2018-02-27 08:48:03 -08:00
Yiqun Lin 1e85a995d1 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 2018-02-27 10:48:52 +08:00
Chao Sun ae290a4bb4 HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun. 2018-02-26 16:15:00 -08:00
Wei Yan 6ce9f79cc9 HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 2018-02-26 15:13:41 -08:00
Arpit Agarwal 46c93453e9 HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula. 2018-02-26 13:56:34 -08:00
Kihwal Lee 451265a83d HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 2018-02-26 10:28:04 -06:00
Arpit Agarwal 2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Arpit Agarwal c30a26abc5 HDFS-12865. RequestHedgingProxyProvider should handle case when none of the proxies are available. Contributed by Mukul Kumar Singh. 2018-02-24 14:25:56 -08:00
Xiaoyu Yao 1e84e46f16 HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain. 2018-02-23 19:35:12 -08:00