17990 Commits

Author SHA1 Message Date
Erik Krogen
16a4043f01 HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen. 2019-07-25 10:07:05 -07:00
Erik Krogen
2598282293 HDFS-14279. [SBN read] Fix race condition in ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-07-25 10:07:05 -07:00
Erik Krogen
45cd309a40 HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko
380668559f HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2019-07-25 10:07:05 -07:00
Chao Sun
9c1c73a0d5 HDFS-14154. [SBN read] Document dfs.ha.tail-edits.period in user guide. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko
451b8b8f16 HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko. 2019-07-25 10:07:05 -07:00
Chao Sun
9aecfdcb52 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Chao Sun
6e19a945bc HDFS-14116. [SBN read] Fix class cast error in NNThroughputBenchmark with ObserverReadProxyProvider. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Erik Krogen
8b8ec65e65 HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Chen Liang
2e7610a029 HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng. 2019-07-25 10:07:05 -07:00
Erik Krogen
7b1e3c49d1 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2019-07-25 10:07:05 -07:00
Chen Liang
82f68a47c4 HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Erik Krogen
b8df864ab0 HDFS-14131. [SBN read] Create user guide for Consistent Reads from Observer feature. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Chen Liang
8769e6feda HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko
54a1c66e48 HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena. 2019-07-25 10:07:05 -07:00
Chao Sun
6916ee71fd HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Chen Liang
96cdd13de5 HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Chen Liang
683daedc1f HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Chao Sun
4ce7f9f2e6 HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Chao Sun
11897618b9 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko
ca565efe2f HDFS-13925. Unit Test for transitioning between different states. Contributed by Sherwood Zheng. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko
7b425c8703 HDFS-13523. Support observer nodes in MiniDFSCluster. Contributed by Konstantin Shvachko. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko
28b11b03d7 HDFS-13961. [SBN read] TestObserverNode refactoring. Contributed by Konstantin Shvachko. 2019-07-25 10:07:05 -07:00
Chen Liang
56af83c961 HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen. 2019-07-25 10:07:05 -07:00
Erik Krogen
894f095219 HDFS-13898. [SBN read] Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Erik Krogen
32737360a5 HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko
ea2d862320 HDFS-13778. [SBN read] TestStateAlignmentContextWithHA should use real ObserverReadProxyProvider instead of AlignmentContextProxyProvider. Contributed by Konstantin Shvachko and Plamen Jeliazkov. 2019-07-25 10:07:05 -07:00
Chen Liang
613c9e4f7b HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Erik Krogen
34b05a26ac HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Konstantin V Shvachko
be8beda8dc HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-25 10:07:04 -07:00
Konstantin V Shvachko
ad80383b34 HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-25 10:07:04 -07:00
Chen Liang
60f0e4df69 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-07-25 10:07:04 -07:00
Chao Sun
b41997ceac HDFS-13789. Reduce logging frequency of QuorumJournalManager#selectInputStreams. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Erik Krogen
0f811b0849 HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 2019-07-25 10:07:04 -07:00
Erik Krogen
9976b7bcbe HDFS-13610. [SBN read] Edit Tail Fast Path Part 4: Cleanup. Integration test, documentation, remove unnecessary dummy sync, minors fixups. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Plamen Jeliazkov
ac0cdb0a8d HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2019-07-25 10:07:04 -07:00
Chao Sun
13e86926a5 HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Konstantin V Shvachko
c02f4ccba1 HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-07-25 10:07:04 -07:00
Erik Krogen
f847983f2c HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Erik Krogen
db9cec6ad2 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Erik Krogen
eacf43252a HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Plamen Jeliazkov
d502e48a7c HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-07-25 10:07:04 -07:00
Erik Krogen
d7e0011b63 HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Erik Krogen
849e1f018e HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Erik Krogen
c2585f7e28 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-07-25 10:07:04 -07:00
Plamen Jeliazkov
ccad9ce59b HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-07-25 10:07:04 -07:00
Chao Sun
44d8b43697 HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Eric E Payne
6aa76eaee3 YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung)
Addendum to fix 3.0 build
2019-07-23 21:53:27 +00:00
Jonathan Hung
3b346f50de YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-22 10:54:38 -07:00
Weiwei Yang
6c8cfd135e HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang. 2019-07-11 09:53:28 +08:00