Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,471 Commits 326 Branches 371 Tags
Commit Graph

6870 Commits

Author SHA1 Message Date
Yiqun Lin
6b5b58a495 HDFS-13692. StorageInfoDefragmenter floods log when compacting StorageInfo TreeSet. Contributed by Bharat Viswanadham. 
(cherry picked from commit 30728aced4a6b05394b3fc8c613f39fade9cf3c2)
 2018-06-22 10:53:33 +08:00
Xiao Chen
e8655b8c72 HDFS-13682. Cannot create encryption zone after KMS auth token expires. 
(cherry picked from commit 32f867a6a907c05a312657139d295a92756d98ef)
 2018-06-20 15:59:23 -07:00
Inigo Goiri
88e8a3e5bc HDFS-13681. Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 8762e9cf10fa100dd5f7fd695f5e52b75a94c5d4)
 2018-06-15 16:49:51 -07:00
Inigo Goiri
7c06be3912 HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 
(cherry picked from commit d31a3ce767d3bb68bdbb4f36d45600eab9f4f8b7)
 2018-06-15 15:59:49 -07:00
Wei-Chiu Chuang
975d4b3d60 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837af1c1a1c4a441f491b0d76d5c9e5d78)
 2018-06-15 13:37:19 -07:00
Inigo Goiri
271ea7a3d9 HDFS-13676. TestEditLogRace fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit eebeb6033fd09791fcbff626f128a98e393f0a88)
 2018-06-15 13:08:24 -07:00
Inigo Goiri
4add4c1be0 HDFS-13673. TestNameNodeMetrics fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit 43d994e4a6dfd1c24eafb909d6f8a0663b20769a)
 2018-06-15 10:34:08 -07:00
Inigo Goiri
fd7cd596b7 HDFS-13675. Speed up TestDFSAdminWithHA. Contributed by Lukas Majercak. 
(cherry picked from commit 5d7449d2b8bcd0963d172fc30df784279671176f)
 2018-06-14 13:43:51 -07:00
Inigo Goiri
e45541be51 HDFS-13563. TestDFSAdminWithHA times out on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 8d4926f38bf53b32453cd2bc7322c8818f752f85)
 2018-06-14 09:59:28 -07:00
Yiqun Lin
5c1248b2da HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 
(cherry picked from commit 8e7548d33be9c4874daab18b2e774bdc2ed216d3)
 2018-06-13 20:10:46 +08:00
Yongjun Zhang
425fe4e217 Update CHANGES, RELEASENOTES, and jdiff for 3.0.3 release. 
(cherry picked from commit 5a82f10e3254bc0745c2dda6bcef888d3ff0d1c3)
 2018-06-11 14:31:46 -07:00
Inigo Goiri
f0150f024f HDFS-13653. Make dfs.client.failover.random.order a per nameservice configuration. Contributed by Ekanth Sethuramalingam. 
(cherry picked from commit 784dbbd2d1fc0c361fee851d3f9b3aa50be5481d)
 2018-06-11 11:17:37 -07:00
Bharat Viswanadham
c0d46a84a6 HDFS-13667:Typo: Marking all datandoes as stale. Contributed by Nanda Kumar 
(cherry picked from commit ccfb816d39878abf4172933327d788c59b9eb082)
 2018-06-09 16:43:58 -07:00
Arpit Agarwal
33b6a68c81 HDFS-12670. can't renew HDFS tokens with only the hdfs client jar. Contributed by Arpit Agarwal. 2018-06-08 17:57:30 -07:00
Wei-Chiu Chuang
b2e919dc1a HDFS-13659. Add more test coverage for contentSummary for snapshottable path. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit e39b113db0f2e4bcf93f873801be770e472601da)
 2018-06-07 08:31:30 -07:00
Brahma Reddy Battula
630f43faf7 HDFS-12950. [oiv] ls will fail in secure cluster. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 9654dd1f472052c4bb4a48a7412149c2e4859a10)
 2018-06-07 10:21:00 +05:30
Akira Ajisaka
f358a77610
HDFS-13545. "guarded" is misspelled as "gaurded" in FSPermissionChecker.java. Contributed by Jianchao Jia. 
(cherry picked from commit 0e3c31579d1c733decfb9b00721bdc7b86be60f2)
 2018-06-05 17:55:01 +09:00
Inigo Goiri
ec5ef0bb20 HDFS-13650. Randomize baseDir for MiniDFSCluster in TestDFSStripedInputStream and TestDFSStripedInputStreamWithRandomECPolicy. Contributed by Anbang Hu. 
(cherry picked from commit 8d31ddcfeb5c2e97692dd386e5bd62f433b44f8e)
 2018-06-04 18:28:40 -07:00
Inigo Goiri
19b37e7080 HDFS-13649. Randomize baseDir for MiniDFSCluster in TestReconstructStripedFile and TestReconstructStripedFileWithRandomECPolicy. Contributed by Anbang Hu. 
(cherry picked from commit 16316b60112842cf8a328b79c89297f6a8b30e7b)
 2018-06-04 18:22:09 -07:00
Inigo Goiri
76546d23e2 HDFS-13652. Randomize baseDir for MiniDFSCluster in TestBlockScanner. Contributed by Anbang Hu. 
(cherry picked from commit a28848104f2a067669f9b1706cd0da05ba71bcb7)
 2018-06-04 18:13:14 -07:00
Rushabh Shah
6646b16450 HDFS-13281 Namenode#createFile should be /.reserved/raw/ aware.. Contributed by Rushabh S Shah 
(cherry picked from commit e2289c8d1496a5eff88e6bcb8776a11d45371ffc)
 2018-06-04 09:21:14 -05:00
Wei-Chiu Chuang
302c36ffdc HDFS-13155. BlockPlacementPolicyDefault.chooseTargetInOrder Not Checking Return Value for NULL. Contributed by Zsolt Venczel. 
(cherry picked from commit bccdfeee0aaef9cb98d09ee39909b63fdcbeeafc)
 2018-06-04 07:03:35 -07:00
Xiao Chen
fde45994a3 HDFS-13339. Volume reference can't be released and may lead to deadlock when DataXceiver does a check volume. Contributed by liaoyuxiangqin and Zsolt Venczel. 
(cherry picked from commit 9efb4b7db00d79aded52997ec89a1be94ecdd268)
 2018-06-03 22:37:27 -07:00
Inigo Goiri
90060d52f6 HDFS-13648. Fix TestGetConf#testGetJournalNodes on Windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 8f83b9abf9a7462c50a1db18d4d99e80c535b09c)
 2018-06-02 20:20:26 -07:00
Inigo Goiri
8a9c2d7b97 HDFS-13651. TestReencryptionHandler fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit e308ac88d399e9bed3581ff4b1450aa2070dcedf)
 2018-06-02 20:13:21 -07:00
Sean Mackrory
58b5b3aa75 HDFS-13582. Improve backward compatibility for HDFS-13176 (WebHdfs file path gets truncated when having semicolon (;) inside). Contributed by Zsolt Venczel. 
(cherry picked from commit 1361030e59d7557a2bffac0ea8df116ce2eaae4a)
 2018-05-31 22:23:19 -06:00
Chao Sun
2bbdf362cf HDFS-13602. Add checkOperation(WRITE) checks in FSNamesystem. Contributed by Chao Sun. 
(cherry picked from commit ff013d2c952272f3176dcf624251b05d610503b5)
 2018-05-31 17:39:46 -07:00
Konstantin V Shvachko
442dd87dcd HDFS-12978. Fine-grained locking while consuming journal stream. Contributed by Konstantin Shvachko. 2018-05-31 14:58:06 -07:00
Inigo Goiri
ad11e40557 HDFS-13631. TestDFSAdmin#testCheckNumOfBlocksInReportCommand should use a separate MiniDFSCluster path. Contributed by Anbang Hu. 
(cherry picked from commit 1f324e966136f69c3254f01b43b56cf077abba7f)
 2018-05-31 10:13:14 -07:00
Inigo Goiri
5c09dadd73 HDFS-13629. Some tests in TestDiskBalancerCommand fail on Windows due to MiniDFSCluster path conflict and improper path usage. Contributed by Anbang Hu. 
(cherry picked from commit 47c31ff16b452d47afc6ffc1cf936ac2de9b788d)
 2018-05-30 10:23:00 -07:00
Inigo Goiri
6dc40bab4e HDFS-13632. Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA. Contributed by Anbang Hu. 
(cherry picked from commit e1102865e6e16babaff9346212c824cb1f47c31c)
 2018-05-30 10:02:50 -07:00
Inigo Goiri
bc80290f28 HDFS-13591. TestDFSShell#testSetrepLow fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 9dbf4f01665d5480a70395a24519cbab5d4db0c5)
 2018-05-28 16:34:31 -07:00
Inigo Goiri
cc696f80d3 HDFS-13627. TestErasureCodingExerciseAPIs fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 91d7c74e6aa4850922f68bab490b585443e4fccb)
 2018-05-28 10:27:23 -07:00
Takanobu Asanuma
499a90f152
HDFS-13628. Update Archival Storage doc for Provided Storage 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 04757e5864bd4904fd5a59d143fff480814700e4)
 2018-05-28 19:06:58 +09:00
Inigo Goiri
c7d77a3bfe HDFS-13620. Randomize the test directory path for TestHDFSFileSystemContract. Contributed by Anbang Hu. 
(cherry picked from commit 8605a38514b4f7a2a549c7ecf8e1421e61bb4d67)
 2018-05-25 19:45:48 -07:00
Inigo Goiri
71ab85adce HDFS-13619. TestAuditLoggerWithCommands fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 13d25289076b39daf481fb1ee15939dbfe4a6b23)
 2018-05-25 13:33:05 -07:00
Inigo Goiri
2d3d7a67f2 HDFS-13618. Fix TestDataNodeFaultInjector test failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit a8e9ee3fdf3aa1b889dfdd3bade91ba754ed9ee1)
 2018-05-25 09:11:07 -07:00
Inigo Goiri
f57e91a348 HDFS-13493. Reduce the HttpServer2 thread count on DataNodes. Contributed by Erik Krogen. 
(cherry picked from commit cddbbe5f690e4617413f6e986adc6fa900629f03)
 2018-05-23 12:12:40 -07:00
Inigo Goiri
a138496379 HDFS-13587. TestQuorumJournalManager fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit c13dea87d9de7a9872fc8b0c939b41b1666a61e5)
 2018-05-23 11:36:34 -07:00
Bharat Viswanadham
d0d3d5b2ad HDFS-13589: Add dfsAdmin command to query if upgrade is finalized. Contributed by Hanisha Koneru 
(cherry picked from commit bc6d9d4c796d3c9d27dbbe3266031bf2adecde4f)
 2018-05-23 10:23:45 -07:00
Inigo Goiri
0c268a0755 HDFS-13588. Fix TestFsDatasetImpl test failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit c0c9b7a8ef2618b7641a0452d9277abd26815de2)
 2018-05-23 09:47:10 -07:00
Sammi Chen
7d71b3a1cc HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen. 
(cherry picked from commit 34e8b9f9a86fb03156861482643fba11bdee1dd4)
 2018-05-23 19:12:04 +08:00
Andrew Wang
d47c0fc39a HDFS-13601. Optimize ByteString conversions in PBHelper. 
(cherry picked from commit 1d2640b6132e8308c07476badd2d1482be68a298)
 2018-05-22 23:55:35 -07:00
Inigo Goiri
8a86269c51 HDFS-13593. TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 9775ecb2355d7bed3514fcd54bf69e8351c4ab99)
 2018-05-18 09:46:29 -07:00
Inigo Goiri
da5eff8325 HDFS-13592. TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly. Contributed by Anbang Hu. 
(cherry picked from commit 57b893de3d36d20f65ee81b5cc3cfef12594b75b)
 2018-05-18 09:38:23 -07:00
Inigo Goiri
df54cdbaea HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 53b807a6a8486cefe0b036f7893de9f619bd44a1)
 2018-05-17 17:03:56 -07:00
Inigo Goiri
f67865ed7e HDFS-13556. TestNestedEncryptionZones does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit a97a2042f210e9db97646baad6f56064d672f447)
 2018-05-17 16:53:53 -07:00
Inigo Goiri
8c463a2a9c HDFS-13554. TestDatanodeRegistration#testForcedRegistration does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit 65476458fa05656010809be632356e4015b59a17)
 2018-05-17 14:48:40 -07:00
Inigo Goiri
e5ff9f3922 HDFS-13558. TestDatanodeHttpXFrame does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit 26f1e22fc9ee326e9c76503d347552faeb6c2d3b)
 2018-05-17 13:35:40 -07:00
Inigo Goiri
b0c89690ed HDFS-13570. TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows. Contributed by Anbang Hu. 
(cherry picked from commit c0ec061e280c0d3e5299c0735c8b8697e0f31216)
 2018-05-17 09:22:29 -07:00