Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,214 Commits 326 Branches 371 Tags
Commit Graph

6672 Commits

Author SHA1 Message Date
Inigo Goiri
e6e7143545 HDFS-12335. Federation Metrics. Contributed by Inigo Goiri. 
(cherry picked from commit 3b19e77752afce87936f5c0d1e6d272fba798d7b)
 2017-10-06 19:04:44 -07:00
Inigo Goiri
e8b511ca33 HDFS-12384. Fixing compilation issue with BanDuplicateClasses. Contributed by Inigo Goiri. 
(cherry picked from commit fabe02c8fafa807198054da0c02b2ebaafda76aa)
 2017-10-06 19:04:44 -07:00
Inigo Goiri
ad48e844fa HDFS-10631. Federation State Store ZooKeeper implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 23c4ddee11ab1300325a6361124ee8ad6f68d7a4)
 2017-10-06 19:04:44 -07:00
Inigo Goiri
503849477a HDFS-12312. Rebasing HDFS-10467 (2). Contributed by Inigo Goiri. 
(cherry picked from commit 90ba6843fb3ac5dc7576535e66a75a5e3433247b)
 2017-10-06 19:04:43 -07:00
Inigo Goiri
24d275842f HDFS-11554. [Documentation] Router-based federation documentation. Contributed by Inigo Goiri. 
(cherry picked from commit ee3260211d94aed223dd6f2386a166eb2c7d67af)
 2017-10-06 19:04:43 -07:00
Inigo Goiri
54a355dc8d HDFS-10646. Federation admin tool. Contributed by Inigo Goiri. 
(cherry picked from commit ae27e31fbcf546481db0b0345772db2e9132372e)
 2017-10-06 19:04:43 -07:00
Inigo Goiri
a0cce0fa0b HDFS-10880. Federation Mount Table State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 58b97df661441150d35abd44b3a8606206b46441)
 2017-10-06 19:04:42 -07:00
Inigo Goiri
c0fb7b283e HDFS-11826. Federation Namenode Heartbeat. Contributed by Inigo Goiri. 
(cherry picked from commit 928f8dab52191e733984d37f47b69719ccf11313)
 2017-10-06 19:04:41 -07:00
Inigo Goiri
a11e99efeb HDFS-10687. Federation Membership State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 95cae08849d23cf12c5d280c29f90908e6a90d40)
 2017-10-06 19:04:41 -07:00
Inigo Goiri
ad0fca5dea HDFS-12223. Rebasing HDFS-10467. Contributed by Inigo Goiri. 
(cherry picked from commit 47db6e9d8e2c264671c89fdd6cb11a7c762d2cce)
 2017-10-06 19:04:41 -07:00
Inigo Goiri
2b99a4d29f HDFS-11546. Federation Router RPC server. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 8a9cdebebf26841a0f1e99fb08135f4597f2eba2)
 2017-10-06 19:04:40 -07:00
Inigo Goiri
9b09f73006 HDFS-10630. Federation State Store FS Implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit c6e0bd640cdaf83a660fa050809cad6f1d4c6f4d)
 2017-10-06 19:04:40 -07:00
Inigo
05afc382d9 HDFS-10882. Federation State Store Interface API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6d94c90ece1c1d23d4c97e72c54e9991f5dbc481)
 2017-10-06 19:04:39 -07:00
Inigo
ff134679e0 HDFS-10881. Federation State Store Driver API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 0f88e049156dce173afc0dbda864e29190dd2210)
 2017-10-06 19:04:39 -07:00
Inigo
9e3491fafa HDFS-10629. Federation Roter. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6821e801724ac38e9737538b2164c9ae88792282)
 2017-10-06 19:04:16 -07:00
Xiao Chen
3106552b75 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit 212342d6db14673f8f5652a5209e1c4fcc0f0ede)
 2017-10-05 19:47:13 -07:00
Andrew Wang
e159272003 HDFS-12567. BlockPlacementPolicyRackFaultTolerant fails with racks with very few nodes. 
(cherry picked from commit 644c2f6924f341f51d809c91dccfff88fc82f6f0)
 2017-10-05 16:58:46 -07:00
Lei Xu
eb76d3dbec HDFS-12569. Unset EC policy logs empty payload in edit log. (Lei (Eddy) Xu) 
(cherry picked from commit 0eab2fcac615228447e7f8bdf4b3ee742ea796ae)
 2017-10-02 15:33:35 -07:00
Lei Xu
337506190b HDFS-12453. TestDataNodeHotSwapVolumes fails in trunk Jenkins runs. (Lei (Eddy) Xu) 2017-09-29 10:47:37 -07:00
Andrew Wang
5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Manoj Govindassamy
46031d84fb HDFS-12404. Rename hdfs config authorization.provider.bypass.users to attributes.provider.bypass.users. 
(cherry picked from commit 3b3be355b35d08a78d9dcd647650812a2d28207b)
 2017-09-28 10:22:25 -07:00
Lei Xu
d9dd856739 HDFS-12409. Add metrics of execution time of different stages in EC recovery task. (Lei (Eddy) Xu) 
(cherry picked from commit 73aed34dffa5e79f6f819137b69054c1dee2d4dd)
 2017-09-28 10:21:54 -07:00
Lei Xu
58eacdbb42 HDFS-12412. Change ErasureCodingWorker.stripedReadPool to cached thread pool. (Lei (Eddy) Xu) 
(cherry picked from commit 123342cd0759ff88801d4f5ab10987f6e3f344b0)
 2017-09-28 10:21:24 -07:00
Arpit Agarwal
7f371da52a HDFS-12470. DiskBalancer: Some tests create plan files under system directory. Contributed by Hanisha Koneru. 
(cherry picked from commit a2dcba18531c6fa4b76325f5132773f12ddfc6d5)
 2017-09-28 10:19:49 -07:00
Arpit Agarwal
14a05ee4c1 HDFS-12496. Make QuorumJournalManager timeout properties configurable. Contributed by Ajay Kumar. 
(cherry picked from commit b9e423fa8d30ea89244f6ec018a8064cc87d94a9)
 2017-09-28 10:19:11 -07:00
Arpit Agarwal
f8df655f31 HDFS-12530. Processor argument in Offline Image Viewer should be case insensitive. Contributed by Hanisha Koneru. 
(cherry picked from commit 08fca508e66e8eddc5d8fd1608ec0c9cd54fc990)
 2017-09-28 10:18:22 -07:00
Wei-Chiu Chuang
2ab2a9438f HDFS-12458. TestReencryptionWithKMS fails regularly. Contributed by Xiao Chen. 
(cherry picked from commit 7c34ceaf4fa28e2ecabd6626860bb1c7418e4b8d)
 2017-09-28 05:11:49 -07:00
Yiqun Lin
91e866ee6e HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit e9b790db9de7c3fbe451bcdc3fd928a27a0d8b4e)
 2017-09-27 10:53:33 +08:00
Daryn Sharp
61dc4bcd61 [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:27:32 -05:00
Akira Ajisaka
51b3a38064
HDFS-12304. Remove unused parameter from FsDatasetImpl#addVolume. Contributed by Chen Liang. 
(cherry picked from commit 02e2a9b1152b0e144fcf43bec2fce26d8a6c6dbc)
 2017-09-26 01:26:13 +09:00
Lei Xu
98612bb2d0 HDFS-12523. Thread pools in ErasureCodingWorker do not shutdown. (Huafeng Wang via Lei) 
(cherry picked from commit 8680131e4b38f0b876173cbee838fcdb91cb52b6)
 2017-09-21 16:12:10 -07:00
Kihwal Lee
9bd1f364a7 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 
(cherry picked from commit 6bf921a5c3152a307b5c0903056d73ce07775a08)
 2017-09-21 08:48:37 -05:00
Akira Ajisaka
b6551f7daf
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 
(cherry picked from commit 50849ec9eb9bad90586a95d2b2380ee6e8724d6b)
 2017-09-21 19:28:28 +09:00
Andrew Wang
1792093bae HDFS-12447. Rename AddECPolicyResponse to AddErasureCodingPolicyResponse. Contributed by SammiChen. 
(cherry picked from commit a12f09ba3c4a3aa4c4558090c5e1b7bcaebe3b94)
 2017-09-20 11:51:21 -07:00
Ming Ma
0006ee681a HDFS-11035. Better documentation for maintenace mode and upgrade domain. 
(cherry picked from commit ce943eb17a4218d8ac1f5293c6726122371d8442)
 2017-09-20 09:38:15 -07:00
Ming Ma
816933722a HDFS-12473. Change hosts JSON file format. 
(cherry picked from commit 230b85d5865b7e08fb7aaeab45295b5b966011ef)
 2017-09-20 09:05:56 -07:00
Andrew Wang
f4082bbb23 HDFS-12437. Fix test setup in TestLeaseRecoveryStriped. 
(cherry picked from commit 12d9d7bc509bca82b8f40301e3dc5ca764be45eb)
 2017-09-19 16:42:23 -07:00
Andrew Wang
5c8331c230 HDFS-12445. Correct spellings of choosen to chosen. Contributed by hu xiaodong. 
(cherry picked from commit 51edaacd09d86419f99ca96545a1393db1f43f73)
 2017-09-19 13:48:25 -07:00
Andrew Wang
fdfbeb0243 HDFS-12444. Reduce runtime of TestWriteReadStripedFile. Contributed by Huafeng Wang and Andrew Wang. 
(cherry picked from commit 59830ca772dfb5dcc8b3e5281ca482dea5a5fa3e)
 2017-09-19 13:44:53 -07:00
Lei Xu
7d9ca2e21b HDFS-12449. TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly cannot finish in 60s. (SammiChen via lei) 2017-09-19 11:51:44 -07:00
Brahma Reddy Battula
1ce365e91e HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55101d97ac62e1ee4e3ddf9a915d5363)
 2017-09-19 11:27:41 +05:30
Brahma Reddy Battula
6ee79d573a HDFS-12480. TestNameNodeMetrics#testTransactionAndCheckpointMetrics Fails in trunk. Contributed by Hanisha Koneru 
(cherry picked from commit 31b58406ac369716ef1665b7d60a3409117bdf9d)
 2017-09-19 10:46:01 +05:30
Kai Zheng
16a75b4ed3 HDFS-12479. Some misuses of lock in DFSStripedOutputStream. Contributed by Huafeng Wang 2017-09-19 17:46:39 +08:00
Erik Krogen
a57f76a3c3 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262df0243e795b675f3ac9f7b322ccd11)
 2017-09-18 13:35:28 -07:00
Kai Zheng
4f8240bd88 HDFS-12460. Make addErasureCodingPolicy an idempotent operation. Contributed by Sammi Chen 2017-09-18 18:06:18 +08:00
Arpit Agarwal
31cdd260f8 HDFS-12472. Add JUNIT timeout to TestBlockStatsMXBean. Contributed by Bharat Viswanadham. 2017-09-16 10:09:36 -07:00
Andrew Wang
7ea10e9f96 HDFS-10701. TestDFSStripedOutputStreamWithFailure#testBlockTokenExpired occasionally fails. Contributed by SammiChen. 
(cherry picked from commit ef8cd5dc565f901b4954befe784675e130e84c3c)
 2017-09-15 16:20:39 -07:00
Lei Xu
2787766b41 HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (Lei (Eddy) Xu) 
(cherry picked from commit fbe06b58805aac4861fb27dfa273914b69e8bdc6)
 2017-09-15 12:15:49 -07:00
Nathan Roberts
944b315dd1 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc1562c4d1c39d24738c7cf8a8869dcec9c)
 2017-09-15 13:29:14 -05:00
Kihwal Lee
ba26097ea7 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 
(cherry picked from commit 78bdf10ae40300a5f2f3b95491d280802edc1df2)
 2017-09-15 09:55:04 -05:00