Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,109 Commits 326 Branches 371 Tags
Commit Graph

19109 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Chao Sun
9840dae72f HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-07-12 11:05:07 -07:00
Konstantin V Shvachko
405f6b6fbf HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-07-12 11:05:07 -07:00
Erik Krogen
8e551708d8 HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen. 2019-07-12 11:05:04 -07:00
Erik Krogen
abb8614dcf HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2019-07-12 11:04:24 -07:00
Erik Krogen
c40ddf9b5b HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen. 2019-07-12 11:03:32 -07:00
Plamen Jeliazkov
7d0c3ba700 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-07-12 11:03:31 -07:00
Erik Krogen
18e545f0be HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2019-07-12 11:03:31 -07:00
Erik Krogen
07a6510e6a HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-07-12 11:03:31 -07:00
Erik Krogen
b999e96c9a HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-07-12 11:03:30 -07:00
Plamen Jeliazkov
21a886a4ef HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-07-12 11:03:27 -07:00
Chao Sun
4cf63905d0 HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun. 2019-07-12 10:58:43 -07:00
Szilard Nemeth
773591ee42 YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl 
(cherry picked from commit 557056e18ea3d5b3fe3046f0ea4b4c7345ea21c5)
 2019-07-12 17:41:23 +02:00
Szilard Nemeth
531e0c0bc1 YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal 
(cherry picked from commit 61b0c2bb7c0f18c4a666b96ca1603cbd4d27eb6d)
 2019-07-12 17:30:50 +02:00
Szilard Nemeth
43c89d1e2b YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal 
(cherry picked from commit c416284bb7581747beef36d7899d8680fe33abbd)
 2019-07-12 17:07:25 +02:00
Masatake Iwasaki
c6ac7f511c
HDFS-14466. Add a regression test for HDFS-14323. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 00dd843a1a6c9d8b616631cdcf24c00e82498dab)
 2019-07-12 10:14:49 +09:00
Szilard Nemeth
872a039bac YARN-9625. UI2 - No link to a queue on the Queues page for Fair Scheduler. Contributed by Zoltan Siegl 
(cherry picked from commit 9cec02318644c8430cbf65bcc3096ffe45992a8e)
 2019-07-11 20:02:19 +02:00
Szilard Nemeth
d590745046 YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal. 2019-07-11 19:54:31 +02:00
Weiwei Yang
9c3806cf1b HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang. 2019-07-11 09:46:27 +08:00
bibinchundatt
5effeae1f3 YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T. 
(cherry picked from commit 5f8395f393e0759215c56927dae1297dfdb0b955)
 2019-07-10 14:47:29 +05:30
Erik Krogen
a59553b5e6 HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun. 
(cherry-picked from 4632708148ed7dec75810feddb22ad98e4670483)
(cherry-picked from 491dc7cc9d6836f34d2647b12dcefa86cc97a927)
 2019-07-08 15:20:29 -07:00
leosunli
7c7907f756 HDFS-13694. Making md5 computing being in parallel with image loading. 
(cherry picked from commit 9c907294860a48f0d1676a31bda15795b7a6771a)
 2019-07-07 20:02:51 -07:00
Masatake Iwasaki
02ae1d6071 HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina. 
(cherry picked from commit 1c254a8ec78c8dab9111f3ba2709cef20794b73a)
 2019-07-04 23:54:18 +09:00
Sunil G
9eb96b0fbf YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt. 
(cherry picked from commit d18986e4e89bd4bb3600e95b4690fd32f54a41e5)
 2019-07-04 11:06:41 +05:30
Szilard Nemeth
46177ade8b YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal. 
(cherry picked from commit a2a8be18cb5e912c8de0ea6beec1de4a99de656b)
 2019-07-03 14:24:53 +02:00
Inigo Goiri
3ab475377a HADOOP-16385. Namenode crashes with 'RedundancyMonitor thread received Runtime exception'. Contributed by Ayush Saxena. 
(cherry picked from commit aa9f0850e85203b2ce4f4a8dc8968e9186cdc67a)
 2019-07-02 13:26:17 -07:00
Sunil G
d2a5749482 YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt. 
(cherry picked from commit 0c8813f135f8c17f88660bb92529c15bb3a157ca)
 2019-07-02 12:15:40 +05:30
Weiwei Yang
46b81a982b YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi. 
(cherry picked from commit 570eee30e5ab5cf37b1a758934987cbf61140f6a)
 2019-07-02 10:17:56 +08:00
bibinchundatt
4f622ecad8 YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T. 
(cherry picked from commit be80334cdf255616f589217726483194fb56dcc6)
 2019-06-27 10:11:30 +05:30
Anu Engineer
1bae5c7024 HDFS-14598. Findbugs warning caused by HDFS-12487. 
Contributed by He Xiaoqiao.

(cherry picked from commit 041e7a7dee4a17714f31952dc6364c77a65b1b73)
(cherry picked from commit e7fce2104f28c48c817822bed9349d44b0694699)
 2019-06-24 21:20:00 -07:00
Inigo Goiri
90b88db35d HDFS-14247. Repeat adding node description into network topology. Contributed by HuangTao. 
(cherry picked from commit 80b77deb42a3ef94d6bef160bc58d807f2faa104)
(cherry picked from commit 96371245357bda63b3ede10f37a37f5333a85d69)
 2019-06-24 21:17:04 -07:00
Inigo Goiri
aaf74ea5d7 HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException. 
(cherry picked from commit 0966407ad6fddb7adf60dba846d1f7c8a13b8ec1)
 2019-06-24 19:03:26 -07:00
Santosh Marella
026cc2647d HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2019-06-23 20:27:01 -07:00
Wei-Chiu Chuang
529d0955f5 HDFS-14074. DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. Contributed by guangyi lu. 
(cherry picked from commit 645d67bc4f4e29d10ef810386c89e6a7c8c61862)
(cherry picked from commit d5ecca1d4ca2b567167402bf8745fb2417bbf624)
 2019-06-21 18:27:50 -07:00
Wei-Chiu Chuang
4507c2f3a5 HDFS-12487. FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers. Contributed by liumi. 
(cherry picked from commit 1524e2e6c52aba966cbbf1d8025ba165688ab9bb)
(cherry picked from commit 845d7611babac68a2b0caa6e4b8963f0c8332540)
 2019-06-21 18:18:36 -07:00
Wei-Chiu Chuang
f33bcbb687 HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 
(cherry picked from commit a5d76a4965b686ec5532add511c64d5f87c88865)
 2019-06-20 20:54:43 -07:00
Takanobu Asanuma
2f6f42bc7b HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 98d20656433cdec76c2108d24ff3b935657c1e80)
 2019-06-20 20:18:00 -07:00
Wei-Chiu Chuang
e4a6976c30 HADOOP-14385. HttpExceptionUtils#validateResponse swallows exceptions. Contributed by Wei-Chiu Chuang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 840d02ca5ba013bddd28d7c1abd242b5bb8cc39f)
 2019-06-20 19:58:28 -07:00
Inigo Goiri
3dd4000884 HADOOP-16028. Fix NetworkTopology chooseRandom function to support excluded nodes. Contributed by Sihai Ke. 
(cherry picked from commit f4e18242bd8117a5c506ec6d3f25c85011fa82d0)
 2019-06-20 13:38:48 -07:00
Wei-Chiu Chuang
58c1920472 Revert "HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti." 
This reverts commit 49587e4df3e955466670384de867f690b557fe6e.
 2019-06-19 06:55:25 -07:00
Virajith Jalaparti
49587e4df3 HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 333633d9935acd5787ee8688bc9d2d7b7c058def)
(cherry picked from commit 36dc0810bf8032e4741e27a8f2b00e64dbdaef2a)
 2019-06-18 22:17:08 -07:00
Zsolt Venczel
4ea3b04bf3 HDFS-14101. Random failure of testListCorruptFilesCorruptedBlock. Contributed by Zsolt Venczel, Nikhil Navadia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Nikhil Navadia <nikhil.navadiya@cloudera.com>
(cherry picked from commit 7c00756aff6bf8d5e1d05748ba32b679e397b53f)
 2019-06-18 14:38:05 -07:00
Elek, Marton
293f8bbd84 HDFS-14078. Admin helper fails to prettify NullPointerExceptions. Contributed by Elek, Marton. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit cc229be9ebb1b3737c9f5fcb29788b51fa584e47)
(cherry picked from commit 1a99f9794519faee5a196ab3d9cb91d55bfd0002)
 2019-06-18 12:00:48 -07:00
Wei-Chiu Chuang
29c4aded34 HDFS-14010. Pass correct DF usage to ReservedSpaceCalculator builder. Contributed by Virajith Jalaparti. 
(cherry picked from commit 1cc1530b4e874391a17ec81e24cb55dfbbabd36a)
(cherry picked from commit 223a582eb839bb6e4298a2fe0243bc27d73fc977)
 2019-06-18 00:34:55 -07:00
Shweta Yakkali
0b3a2eb8a7 HADOOP-16156. [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl. Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit dd4a7633ece11c528a58146f92522b55e7be4dc6)
(cherry picked from commit 99abd35a75fc29d89a03aaf9f1919e9d48f4abec)
 2019-06-17 23:48:40 -07:00
Adam Antal
15689ce0e8 HADOOP-15914. hadoop jar command has no help argument. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 9f47a2e4a9845b84cc963708c7924b92df2964f6)
(cherry picked from commit 26671772486b71f27ba37c836006b4ab05fd2b96)
 2019-06-17 22:56:18 -07:00
Wei-Chiu Chuang
4362fcb629 HDFS-14340. Lower the log level when can't get postOpAttr. Contributed by Anuhan Torgonshar. 
(cherry picked from commit 098c325a78dce2697ec94a22a4a3981fa5a3849e)
(cherry picked from commit f5428d4062c0a5e98461ea3b8420dc68555681f9)
 2019-06-17 22:48:07 -07:00
Zhankun Tang
829202740a YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji. 
(cherry picked from commit 67414a1a80039e70e0afc1de171831a6e981f37a)
 2019-06-18 13:20:07 +08:00
Wei-Chiu Chuang
8f4204dc3b HADOOP-9157. Better option for curl in hadoop-auth-examples. Contributed by Andras Bokor. 
(cherry picked from commit f1c239c6a4c26e9057373b9b9400e54083290f65)
(cherry picked from commit e8db39b0e9ad66af3dada1ec7ad06b7d06f8e9f8)
 2019-06-17 21:56:04 -07:00
Weiwei Yang
3bdb661f35 HDFS-12770. Add doc about how to disable client socket cache. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 6ec57fe6a45df0e9216bbbba85b5bf61da1bf2ca)
(cherry picked from commit f36cdc4d7f45d1a3f8edd2d804c38b0b39990410)
 2019-06-17 21:47:07 -07:00
Wei-Chiu Chuang
fc8983a5b8 HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu. 
(cherry picked from commit 62ad9885ea8c75c134de43a3a925c76b253658e1)
(cherry picked from commit 45543ffd6af723bb83bd44a99f12165363c70abb)
 2019-06-17 20:20:08 -07:00