Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,003 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

18003 Commits

Author SHA1 Message Date
Weiwei Yang 373e32480b YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23)
(cherry picked from commit 70efe253f3)
 2019-08-05 10:02:33 -07:00
Weiwei Yang fd977a138a YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 78ea897b67)
(cherry picked from commit 7dd8eafe33)
 2019-08-05 10:02:33 -07:00
Wei Yan ea345de738 YARN-6851. Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat. 
(cherry picked from commit 333ef303ff)
 2019-08-05 10:02:33 -07:00
Wangda Tan d18ec19ed8 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07)
(cherry picked from commit 46baafedf1)
 2019-08-05 10:02:33 -07:00
Eric Badger 01f76b4f45 YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit 
(cherry picked from commit 144a55f0e3)
 2019-08-02 21:01:15 +00:00
Erik Krogen 055f824bcb HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira. 
(cherry picked from e7a0b8aa83)
(cherry picked from ba4ca7d996cafa1b1786249cae2b207f0eaa1070)
(cherry picked from da548cf263)
 2019-08-02 11:52:26 -07:00
Christopher Gregorian ec00431eaa HADOOP-16459. Backport of HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian. 
This commit also includes the follow-on commit 827a84778a.

(cherry-picked from f96a2df38d)
(cherry-picked from d4492bdd9e)
(cherry-picked from 7b8f08f59e)
 2019-07-30 14:46:11 -07:00
Eric E Payne c04ccf296d YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan. 2019-07-30 20:00:32 +00:00
Inigo Goiri 998593e7f2 HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao. 
(cherry picked from commit 3d396786cf6eaab49c1c9b8b2a4652c2e440b9e3)
(cherry picked from commit 950aa74d5f)
(cherry picked from commit fe40fbbd4d)
 2019-07-29 14:39:20 -07:00
Erik Krogen f7e517ee24 HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen. 
(cherry-picked from 62efb63006)
(cherry-picked from 5ec34bf43a)
(cherry-picked from e2aab0efef9d504e4695a9c1df2835ed60f6c0c7)
 2019-07-26 11:43:34 -07:00
Wei-Chiu Chuang b5303012a0 HDFS-14537. Journaled Edits Cache is not cleared when formatting the JN. Contributed by Ranith Sardar. 2019-07-25 10:07:06 -07:00
Erik Krogen 32d80d2201 HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen. 2019-07-25 10:07:06 -07:00
Erik Krogen 11fee2d4e1 HDFS-14211. [SBN Read]. Add a configurable flag to enable always-msync mode to ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-07-25 10:07:06 -07:00
Erik Krogen 16a4043f01 HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen. 2019-07-25 10:07:05 -07:00
Erik Krogen 2598282293 HDFS-14279. [SBN read] Fix race condition in ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-07-25 10:07:05 -07:00
Erik Krogen 45cd309a40 HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko 380668559f HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2019-07-25 10:07:05 -07:00
Chao Sun 9c1c73a0d5 HDFS-14154. [SBN read] Document dfs.ha.tail-edits.period in user guide. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko 451b8b8f16 HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko. 2019-07-25 10:07:05 -07:00
Chao Sun 9aecfdcb52 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Chao Sun 6e19a945bc HDFS-14116. [SBN read] Fix class cast error in NNThroughputBenchmark with ObserverReadProxyProvider. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Erik Krogen 8b8ec65e65 HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Chen Liang 2e7610a029 HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng. 2019-07-25 10:07:05 -07:00
Erik Krogen 7b1e3c49d1 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2019-07-25 10:07:05 -07:00
Chen Liang 82f68a47c4 HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Erik Krogen b8df864ab0 HDFS-14131. [SBN read] Create user guide for Consistent Reads from Observer feature. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Chen Liang 8769e6feda HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko 54a1c66e48 HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena. 2019-07-25 10:07:05 -07:00
Chao Sun 6916ee71fd HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Chen Liang 96cdd13de5 HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Chen Liang 683daedc1f HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Chao Sun 4ce7f9f2e6 HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Chao Sun 11897618b9 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko ca565efe2f HDFS-13925. Unit Test for transitioning between different states. Contributed by Sherwood Zheng. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko 7b425c8703 HDFS-13523. Support observer nodes in MiniDFSCluster. Contributed by Konstantin Shvachko. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko 28b11b03d7 HDFS-13961. [SBN read] TestObserverNode refactoring. Contributed by Konstantin Shvachko. 2019-07-25 10:07:05 -07:00
Chen Liang 56af83c961 HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen. 2019-07-25 10:07:05 -07:00
Erik Krogen 894f095219 HDFS-13898. [SBN read] Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Erik Krogen 32737360a5 HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun. 2019-07-25 10:07:05 -07:00
Konstantin V Shvachko ea2d862320 HDFS-13778. [SBN read] TestStateAlignmentContextWithHA should use real ObserverReadProxyProvider instead of AlignmentContextProxyProvider. Contributed by Konstantin Shvachko and Plamen Jeliazkov. 2019-07-25 10:07:05 -07:00
Chen Liang 613c9e4f7b HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-07-25 10:07:05 -07:00
Erik Krogen 34b05a26ac HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Konstantin V Shvachko be8beda8dc HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-25 10:07:04 -07:00
Konstantin V Shvachko ad80383b34 HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-25 10:07:04 -07:00
Chen Liang 60f0e4df69 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-07-25 10:07:04 -07:00
Chao Sun b41997ceac HDFS-13789. Reduce logging frequency of QuorumJournalManager#selectInputStreams. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Erik Krogen 0f811b0849 HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 2019-07-25 10:07:04 -07:00
Erik Krogen 9976b7bcbe HDFS-13610. [SBN read] Edit Tail Fast Path Part 4: Cleanup. Integration test, documentation, remove unnecessary dummy sync, minors fixups. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Plamen Jeliazkov ac0cdb0a8d HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2019-07-25 10:07:04 -07:00
Chao Sun 13e86926a5 HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00