Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,055 Commits 326 Branches 371 Tags
Commit Graph

4696 Commits

Author SHA1 Message Date
Zhe Zhang
736fb3b66c Revert "HDFS-12502. nntop should support a category based on FilesInGetListingOps." 
This reverts commit 86c2adc4528ffd27b744ee7e88109b86aaa3ec02.
 2017-10-24 15:05:26 -07:00
Arpit Agarwal
a373094bf9 HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham. 2017-10-23 10:47:35 -07:00
Inigo Goiri
12c81c67d7 HDFS-12620. Backporting HDFS-10467 to branch-2. Contributed by Inigo Goiri. 2017-10-19 17:40:42 -07:00
Inigo Goiri
629b88b4dd HDFS-12577. Rename Router tooling. Contributed by Inigo Goiri. 
(cherry picked from commit 53e8d0d030525e4c7f3875e23807c6dbe778890f)
(cherry picked from commit 5d63a388d1c3ec8a658cb2fd9b34c240bddf15a0)
 2017-10-19 17:39:27 -07:00
Inigo Goiri
27295eecdd HDFS-12273. Federation UI. Contributed by Inigo Goiri. 
(cherry picked from commit adbb2e00c7b85524fd43bd68895d49814c16680a)
(cherry picked from commit 81601dac8ec7650bec14700b174910390a92fe1f)
 2017-10-19 17:39:19 -07:00
Inigo Goiri
7d795989e1 HDFS-12580. Rebasing HDFS-10467 after HDFS-12447. Contributed by Inigo Goiri. 
(cherry picked from commit b12440d3479f19138bc66ea59baf41eb89061906)
(cherry picked from commit 6c69e23dcdf1cdbddd47bacdf2dace5c9f06e3ad)
 2017-10-19 17:39:18 -07:00
Inigo Goiri
f8a71e2f10 HDFS-12381. [Documentation] Adding configuration keys for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit c2d6aa79055ef72406fa598e1c743b0c994b5da8)
(cherry picked from commit ad41c8155940f4da0e51439c97c6cc9c808c28df)
 2017-10-19 17:39:17 -07:00
Inigo Goiri
9ad1f90b9d HDFS-12450. Fixing TestNamenodeHeartbeat and support non-HA. Contributed by Inigo Goiri. 
(cherry picked from commit 928d1e87f9dbe64f89b858ccc1780723f3af58e7)
(cherry picked from commit 67785fe0063cfbfecab0e8deaae4a45d9c7c0073)
 2017-10-19 17:39:16 -07:00
Inigo Goiri
ec84369538 HDFS-12430. Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218. Contributed by Inigo Goiri. 
(cherry picked from commit 3302e792d469b7e8f3bfa10151e4e1c546589734)
(cherry picked from commit 1f06b81ecb14044964176dd16fafaa0ee96bfe3d)
 2017-10-19 17:39:15 -07:00
Inigo Goiri
550ce086a1 HDFS-12335. Federation Metrics. Contributed by Inigo Goiri. 
(cherry picked from commit 3b19e77752afce87936f5c0d1e6d272fba798d7b)
(cherry picked from commit bc9e588a19c0aaf518de8dab719362be4a8d6a54)
 2017-10-19 17:39:14 -07:00
Inigo Goiri
23753ff3c5 HDFS-12384. Fixing compilation issue with BanDuplicateClasses. Contributed by Inigo Goiri. 
(cherry picked from commit fabe02c8fafa807198054da0c02b2ebaafda76aa)
(cherry picked from commit cc58e7a983d8f1351089462f531993f7b4f0a9c5)
 2017-10-19 17:39:12 -07:00
Inigo Goiri
a100f63a16 HDFS-10631. Federation State Store ZooKeeper implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 23c4ddee11ab1300325a6361124ee8ad6f68d7a4)
(cherry picked from commit 7cb6bdf09ed361e067ebf234230babd1391a7d4b)
 2017-10-19 17:39:11 -07:00
Inigo Goiri
36c3515fbb HDFS-12312. Rebasing HDFS-10467 (2). Contributed by Inigo Goiri. 
(cherry picked from commit 90ba6843fb3ac5dc7576535e66a75a5e3433247b)
(cherry picked from commit 346c9fce43ebf6a90fc56e0dc7c403f97cc5391f)
 2017-10-19 17:39:08 -07:00
Inigo Goiri
0b599cdc87 HDFS-11554. [Documentation] Router-based federation documentation. Contributed by Inigo Goiri. 
(cherry picked from commit ee3260211d94aed223dd6f2386a166eb2c7d67af)
(cherry picked from commit 67d10087aff9d4ab2748aefc1b97522495c148f8)
 2017-10-19 17:38:57 -07:00
Inigo Goiri
78abcb8f52 HDFS-10646. Federation admin tool. Contributed by Inigo Goiri. 
(cherry picked from commit ae27e31fbcf546481db0b0345772db2e9132372e)
(cherry picked from commit b3e6bd22e3c02b3e4f50396538f56a1bcb007638)
 2017-10-19 17:38:54 -07:00
Inigo Goiri
cc7c12ff18 HDFS-10880. Federation Mount Table State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 58b97df661441150d35abd44b3a8606206b46441)
(cherry picked from commit 6f0de2731806628b5b01bd1350225692147590da)
 2017-10-19 17:38:33 -07:00
Inigo Goiri
e700db60a6 HDFS-11826. Federation Namenode Heartbeat. Contributed by Inigo Goiri. 
(cherry picked from commit 928f8dab52191e733984d37f47b69719ccf11313)
(cherry picked from commit d8c81073320320a019fb3868be4f06f46aebea43)
 2017-10-19 17:38:32 -07:00
Inigo Goiri
0228ead4d1 HDFS-10687. Federation Membership State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 95cae08849d23cf12c5d280c29f90908e6a90d40)
(cherry picked from commit 55da7fd7ebe2f3fa1c1c828dda727fddc75a1b81)
 2017-10-19 17:38:30 -07:00
Inigo Goiri
a09d7219c9 HDFS-12223. Rebasing HDFS-10467. Contributed by Inigo Goiri. 
(cherry picked from commit 47db6e9d8e2c264671c89fdd6cb11a7c762d2cce)
(cherry picked from commit 0ec82b8cdfaaa5f23d1a0f7f7fb8c9187c5e309b)
 2017-10-19 17:38:29 -07:00
Inigo Goiri
2f95eac8f6 HDFS-11546. Federation Router RPC server. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 8a9cdebebf26841a0f1e99fb08135f4597f2eba2)
(cherry picked from commit ca4f209b49e3aad6a80306f7342c9b6b560a79a7)
 2017-10-19 17:38:28 -07:00
Inigo Goiri
7bd91fa2ed HDFS-10630. Federation State Store FS Implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit c6e0bd640cdaf83a660fa050809cad6f1d4c6f4d)
(cherry picked from commit 4bf877b03f0e01c4bcedc689c66689701e62b560)
 2017-10-19 17:38:27 -07:00
Inigo
3f6d18ddf6 HDFS-10882. Federation State Store Interface API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6d94c90ece1c1d23d4c97e72c54e9991f5dbc481)
(cherry picked from commit 2c740a684a23663962119726bf0e7ecef173f6f1)
 2017-10-19 17:38:25 -07:00
Inigo
c3b8b5e49f HDFS-10881. Federation State Store Driver API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 0f88e049156dce173afc0dbda864e29190dd2210)
(cherry picked from commit 533b986633e0a9076cf3918fba3e3b591c6f65f2)
 2017-10-19 17:38:24 -07:00
Inigo
096e8c4f35 HDFS-10629. Federation Roter. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6821e801724ac38e9737538b2164c9ae88792282)
(cherry picked from commit 2761bbc91a7b0a36c42b1b6569c5ecd4f236281b)
 2017-10-19 17:38:21 -07:00
Wei-Chiu Chuang
ce93a2b84b HDFS-12619. Do not catch and throw unchecked exceptions if IBRs fail to process. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4ab0c8f96a41c573cc1f1e71c18871d243f952b9)
 2017-10-19 06:18:40 -07:00
Zhe Zhang
87f4407396 HDFS-12502. nntop should support a category based on FilesInGetListingOps. 
(cherry picked from commit 60bfee270ed3a653c44c0bc92396167b5022df6e)
(cherry picked from commit aecf6c50d351e7aed2008a682677a1edb104585d)
 2017-10-19 00:05:34 -07:00
Arpit Agarwal
0aa1b62b39 HDFS-12420. Add an option to disallow 'namenode format -force'. Contributed by Ajay Kumar. 
(cherry picked from commit b6942cbe9b8c9469e8c2b64c3268d671f5a43e75)
(cherry picked from commit 5897095d539be086ed37df011f024e37eb37b0cd)
 2017-10-17 23:44:13 -07:00
Wei-Chiu Chuang
a596c6f5eb HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 8dbc8909c92d502d10a7f94d1de3171878a43b04)
 2017-10-16 12:59:41 -07:00
Xiao Chen
b2f27641dd HDFS-12642. Log block and datanode details in BlockRecoveryWorker. 
(cherry picked from commit 971b3ca1acc15d9a1f8e774291813dafd16ef704)
 2017-10-16 10:34:24 -07:00
Xiao Chen
d82edb49c7 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 
(cherry picked from commit d0311dfc5fe92a0e0f3fb8ae92e412edce609740)
 2017-10-16 09:51:27 -07:00
Xiao Chen
4b734fe386 HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar. 
(cherry picked from commit 11a7729c1474aa379a62c0440c8c1a11be261a75)
 2017-10-15 19:28:38 -07:00
Andrew Wang
1ddc49527a Revert "HDFS-12603. Enable async edit logging by default." 
This reverts commit 42d3229e6f604d13a0419ac32979500dae306066.
 2017-10-09 13:29:52 -07:00
Andrew Wang
42d3229e6f HDFS-12603. Enable async edit logging by default. 2017-10-09 11:21:43 -07:00
Xiao Chen
dd0113d777 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit 212342d6db14673f8f5652a5209e1c4fcc0f0ede)
 2017-10-05 19:47:04 -07:00
Yiqun Lin
2e2f8e42cf HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit e9b790db9de7c3fbe451bcdc3fd928a27a0d8b4e)
 2017-09-27 10:45:44 +08:00
Daryn Sharp
7f3ed5f61e [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:21:37 -05:00
Anu Engineer
de5744c903 HDFS-12531. Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2. Contributed by Bharat Viswanadham. 2017-09-23 11:16:50 -07:00
Arpit Agarwal
09290659e7 HDFS-12530. Processor argument in Offline Image Viewer should be case insensitive. Contributed by Hanisha Koneru. 2017-09-22 09:48:10 -07:00
Arpit Agarwal
18e5f2068b HDFS-12496. Make QuorumJournalManager timeout properties configurable. Contributed by Ajay Kumar. 2017-09-21 08:53:27 -07:00
Kihwal Lee
9640bfb808 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 
(cherry picked from commit 6bf921a5c3152a307b5c0903056d73ce07775a08)
 2017-09-21 08:48:04 -05:00
Akira Ajisaka
3527327f34
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 
(cherry picked from commit 50849ec9eb9bad90586a95d2b2380ee6e8724d6b)
 2017-09-21 19:28:49 +09:00
Ming Ma
3bb23f4be9 HDFS-11035. Better documentation for maintenace mode and upgrade domain. 
(cherry picked from commit ce943eb17a4218d8ac1f5293c6726122371d8442)
 2017-09-20 09:42:22 -07:00
Ming Ma
7dd662eafd HDFS-12473. Change hosts JSON file format. 2017-09-20 09:08:41 -07:00
Brahma Reddy Battula
eddc581860 HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55101d97ac62e1ee4e3ddf9a915d5363)
 2017-09-19 11:31:22 +05:30
Arpit Agarwal
f42705b957 HDFS-12472. Add JUNIT timeout to TestBlockStatsMXBean. Contributed by Bharat Viswanadham. 2017-09-16 10:14:39 -07:00
Erik Krogen
791e20c244 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262df0243e795b675f3ac9f7b322ccd11)
 2017-09-15 14:03:10 -07:00
Nathan Roberts
bd14e8d8b0 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc1562c4d1c39d24738c7cf8a8869dcec9c)
 2017-09-15 14:47:07 -05:00
Kihwal Lee
07dee7fd63 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 
(cherry picked from commit 78bdf10ae40300a5f2f3b95491d280802edc1df2)
 2017-09-15 09:55:59 -05:00
John Zhuge
eaef8446f6 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065268511ec10a8b8852d0d6327419b9c)
 2017-09-14 11:29:34 -07:00
Arpit Agarwal
ecf352d645 HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:54:21 -07:00