Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,174 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

19174 Commits

Author SHA1 Message Date
Konstantin V Shvachko e860882d60 HDFS-13523. Support observer nodes in MiniDFSCluster. Contributed by Konstantin Shvachko. 2019-07-12 12:11:17 -07:00
Konstantin V Shvachko a054c65185 HDFS-13961. [SBN read] TestObserverNode refactoring. Contributed by Konstantin Shvachko. 2019-07-12 12:11:14 -07:00
Chen Liang 91803c8216 HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen. 2019-07-12 11:50:51 -07:00
Erik Krogen d4adf921a3 HDFS-13898. [SBN read] Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode. Contributed by Chao Sun. 2019-07-12 11:50:05 -07:00
Erik Krogen 07558629db HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun. 2019-07-12 11:50:04 -07:00
Konstantin V Shvachko e639297ba0 HDFS-13778. [SBN read] TestStateAlignmentContextWithHA should use real ObserverReadProxyProvider instead of AlignmentContextProxyProvider. Contributed by Konstantin Shvachko and Plamen Jeliazkov. 2019-07-12 11:50:04 -07:00
Chen Liang 8ec17f6aea HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-07-12 11:50:03 -07:00
Erik Krogen 504cb70619 HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen. 2019-07-12 11:50:03 -07:00
Konstantin V Shvachko 1be8e187b1 HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-12 11:50:02 -07:00
Konstantin V Shvachko 1ad1c800a4 HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-12 11:50:02 -07:00
Chen Liang ee9977cc69 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-07-12 11:50:01 -07:00
Chao Sun d96ddb3b87 HDFS-13789. Reduce logging frequency of QuorumJournalManager#selectInputStreams. Contributed by Erik Krogen. 2019-07-12 11:50:01 -07:00
Erik Krogen 849a6c6f0d HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 2019-07-12 11:49:58 -07:00
Erik Krogen ac2e79330c HDFS-13610. [SBN read] Edit Tail Fast Path Part 4: Cleanup. Integration test, documentation, remove unnecessary dummy sync, minors fixups. Contributed by Erik Krogen. 2019-07-12 11:05:50 -07:00
Plamen Jeliazkov cfb4399b36 HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2019-07-12 11:05:08 -07:00
Chao Sun 9840dae72f HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-07-12 11:05:07 -07:00
Konstantin V Shvachko 405f6b6fbf HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-07-12 11:05:07 -07:00
Erik Krogen 8e551708d8 HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen. 2019-07-12 11:05:04 -07:00
Erik Krogen abb8614dcf HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2019-07-12 11:04:24 -07:00
Erik Krogen c40ddf9b5b HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen. 2019-07-12 11:03:32 -07:00
Plamen Jeliazkov 7d0c3ba700 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-07-12 11:03:31 -07:00
Erik Krogen 18e545f0be HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2019-07-12 11:03:31 -07:00
Erik Krogen 07a6510e6a HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-07-12 11:03:31 -07:00
Erik Krogen b999e96c9a HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-07-12 11:03:30 -07:00
Plamen Jeliazkov 21a886a4ef HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-07-12 11:03:27 -07:00
Chao Sun 4cf63905d0 HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun. 2019-07-12 10:58:43 -07:00
Szilard Nemeth 773591ee42 YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl 
(cherry picked from commit 557056e18e)
 2019-07-12 17:41:23 +02:00
Szilard Nemeth 531e0c0bc1 YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal 
(cherry picked from commit 61b0c2bb7c)
 2019-07-12 17:30:50 +02:00
Szilard Nemeth 43c89d1e2b YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal 
(cherry picked from commit c416284bb7)
 2019-07-12 17:07:25 +02:00
Masatake Iwasaki c6ac7f511c
HDFS-14466. Add a regression test for HDFS-14323. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 00dd843a1a)
 2019-07-12 10:14:49 +09:00
Szilard Nemeth 872a039bac YARN-9625. UI2 - No link to a queue on the Queues page for Fair Scheduler. Contributed by Zoltan Siegl 
(cherry picked from commit 9cec023186)
 2019-07-11 20:02:19 +02:00
Szilard Nemeth d590745046 YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal. 2019-07-11 19:54:31 +02:00
Weiwei Yang 9c3806cf1b HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang. 2019-07-11 09:46:27 +08:00
bibinchundatt 5effeae1f3 YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T. 
(cherry picked from commit 5f8395f393)
 2019-07-10 14:47:29 +05:30
Erik Krogen a59553b5e6 HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun. 
(cherry-picked from 4632708148)
(cherry-picked from 491dc7cc9d)
 2019-07-08 15:20:29 -07:00
leosunli 7c7907f756 HDFS-13694. Making md5 computing being in parallel with image loading. 
(cherry picked from commit 9c90729486)
 2019-07-07 20:02:51 -07:00
Masatake Iwasaki 02ae1d6071 HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina. 
(cherry picked from commit 1c254a8ec7)
 2019-07-04 23:54:18 +09:00
Sunil G 9eb96b0fbf YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt. 
(cherry picked from commit d18986e4e8)
 2019-07-04 11:06:41 +05:30
Szilard Nemeth 46177ade8b YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal. 
(cherry picked from commit a2a8be18cb)
 2019-07-03 14:24:53 +02:00
Inigo Goiri 3ab475377a HADOOP-16385. Namenode crashes with 'RedundancyMonitor thread received Runtime exception'. Contributed by Ayush Saxena. 
(cherry picked from commit aa9f0850e8)
 2019-07-02 13:26:17 -07:00
Sunil G d2a5749482 YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt. 
(cherry picked from commit 0c8813f135)
 2019-07-02 12:15:40 +05:30
Weiwei Yang 46b81a982b YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi. 
(cherry picked from commit 570eee30e5)
 2019-07-02 10:17:56 +08:00
bibinchundatt 4f622ecad8 YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T. 
(cherry picked from commit be80334cdf)
 2019-06-27 10:11:30 +05:30
Anu Engineer 1bae5c7024 HDFS-14598. Findbugs warning caused by HDFS-12487. 
Contributed by He Xiaoqiao.

(cherry picked from commit 041e7a7dee)
(cherry picked from commit e7fce2104f)
 2019-06-24 21:20:00 -07:00
Inigo Goiri 90b88db35d HDFS-14247. Repeat adding node description into network topology. Contributed by HuangTao. 
(cherry picked from commit 80b77deb42)
(cherry picked from commit 9637124535)
 2019-06-24 21:17:04 -07:00
Inigo Goiri aaf74ea5d7 HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException. 
(cherry picked from commit 0966407ad6)
 2019-06-24 19:03:26 -07:00
Santosh Marella 026cc2647d HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2019-06-23 20:27:01 -07:00
Wei-Chiu Chuang 529d0955f5 HDFS-14074. DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. Contributed by guangyi lu. 
(cherry picked from commit 645d67bc4f)
(cherry picked from commit d5ecca1d4c)
 2019-06-21 18:27:50 -07:00
Wei-Chiu Chuang 4507c2f3a5 HDFS-12487. FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers. Contributed by liumi. 
(cherry picked from commit 1524e2e6c5)
(cherry picked from commit 845d7611ba)
 2019-06-21 18:18:36 -07:00
Wei-Chiu Chuang f33bcbb687 HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 
(cherry picked from commit a5d76a4965)
 2019-06-20 20:54:43 -07:00