Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,150 Commits 326 Branches 371 Tags
Commit Graph

21150 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Chen Liang
e2337bfc57 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-06-28 14:22:10 -07:00
Chao Sun
6fe755f0c7 HDFS-13789. Reduce logging frequency of QuorumJournalManager#selectInputStreams. Contributed by Erik Krogen. 2019-06-28 14:22:09 -07:00
Erik Krogen
e3f3fb1c18 HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 2019-06-28 14:22:09 -07:00
Erik Krogen
ea402854aa HDFS-13610. [SBN read] Edit Tail Fast Path Part 4: Cleanup. Integration test, documentation, remove unnecessary dummy sync, minors fixups. Contributed by Erik Krogen. 2019-06-28 14:22:09 -07:00
Plamen Jeliazkov
7e0a71dd7f HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2019-06-28 14:22:08 -07:00
Chao Sun
a97f00e1c6 HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-06-28 14:22:08 -07:00
Konstantin V Shvachko
1f5852fb12 HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-06-28 14:22:08 -07:00
Erik Krogen
ac3eeeeb3c HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen. 2019-06-28 14:22:07 -07:00
Erik Krogen
b13110a6f0 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2019-06-28 14:22:07 -07:00
Erik Krogen
174c41c68f HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen. 2019-06-28 14:22:07 -07:00
Plamen Jeliazkov
a3521c53fe HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-06-28 14:22:03 -07:00
Erik Krogen
dbf777e4a5 HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2019-06-28 14:20:02 -07:00
Erik Krogen
49d7bb6a92 HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-06-28 14:20:01 -07:00
Erik Krogen
c0ca2bb853 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-06-28 14:20:01 -07:00
Plamen Jeliazkov
1eeca2d9fb HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-06-28 14:20:00 -07:00
Chao Sun
3932ac4ef7 HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun. 2019-06-28 14:20:00 -07:00
Eric Yang
860606fc67 YARN-9581. Add support for get multiple RM webapp URLs. 
Contributed by Prabhu Joseph

(cherry picked from commit f02b0e19940dc6fc1e19258a40db37d1eed89d21)
 2019-06-28 14:57:50 -04:00
Daniel Templeton
252c3a158d HDFS-14047. [libhdfs++] Fix hdfsGetLastExceptionRootCause bug in test_libhdfs_threaded.c 
(Contributed by Antoli Shein via Daniel Templeton)

Change-Id: Ia5546fd5f8270f285c373b9023a9bd9835059845
 2019-06-27 15:02:59 -07:00
bibinchundatt
a2f4e4698b YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T. 
(cherry picked from commit be80334cdf255616f589217726483194fb56dcc6)
 2019-06-27 10:04:40 +05:30
Anu Engineer
e7fce2104f HDFS-14598. Findbugs warning caused by HDFS-12487. 
Contributed by He Xiaoqiao.

(cherry picked from commit 041e7a7dee4a17714f31952dc6364c77a65b1b73)
 2019-06-24 21:19:50 -07:00
Inigo Goiri
9637124535 HDFS-14247. Repeat adding node description into network topology. Contributed by HuangTao. 
(cherry picked from commit 80b77deb42a3ef94d6bef160bc58d807f2faa104)
 2019-06-24 21:16:46 -07:00
Inigo Goiri
0966407ad6 HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException. 2019-06-24 19:02:41 -07:00
Wei-Chiu Chuang
d5ecca1d4c HDFS-14074. DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. Contributed by guangyi lu. 
(cherry picked from commit 645d67bc4f4e29d10ef810386c89e6a7c8c61862)
 2019-06-21 18:27:38 -07:00
Wei-Chiu Chuang
845d7611ba HDFS-12487. FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers. Contributed by liumi. 
(cherry picked from commit 1524e2e6c52aba966cbbf1d8025ba165688ab9bb)
 2019-06-21 18:18:08 -07:00
Weiwei Yang
1944a7d844 YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi. 
(cherry picked from commit 83dcb9d87ec75f2be0acb8972f5f0faefe6ffbcd)
 2019-06-21 17:52:22 +08:00
Wei-Chiu Chuang
a5d76a4965 HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 2019-06-20 20:54:16 -07:00
Lokesh Jain
461c624cc0 HDFS-13893. DiskBalancer: no validations for Disk balancer commands. Contributed by Lokesh Jain. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 272b96d243383d9f50241d48cb070f638243bc9c)
 2019-06-20 20:23:31 -07:00
Takanobu Asanuma
6dffad028e HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 98d20656433cdec76c2108d24ff3b935657c1e80)
 2019-06-20 20:17:45 -07:00
Wei-Chiu Chuang
77e391fd8b HADOOP-14385. HttpExceptionUtils#validateResponse swallows exceptions. Contributed by Wei-Chiu Chuang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 840d02ca5ba013bddd28d7c1abd242b5bb8cc39f)
 2019-06-20 19:58:17 -07:00
Inigo Goiri
f046b881f6 HADOOP-16028. Fix NetworkTopology chooseRandom function to support excluded nodes. Contributed by Sihai Ke. 
(cherry picked from commit f4e18242bd8117a5c506ec6d3f25c85011fa82d0)
 2019-06-20 13:38:00 -07:00
Wei-Chiu Chuang
7e9b9e2a2e Revert "HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti." 
This reverts commit 36dc0810bf8032e4741e27a8f2b00e64dbdaef2a.
 2019-06-19 06:55:08 -07:00
Virajith Jalaparti
36dc0810bf HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 333633d9935acd5787ee8688bc9d2d7b7c058def)
 2019-06-18 22:16:56 -07:00
Zsolt Venczel
7c00756aff HDFS-14101. Random failure of testListCorruptFilesCorruptedBlock. Contributed by Zsolt Venczel, Nikhil Navadia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Nikhil Navadia <nikhil.navadiya@cloudera.com>
 2019-06-18 14:37:16 -07:00
Elek, Marton
1a99f97945 HDFS-14078. Admin helper fails to prettify NullPointerExceptions. Contributed by Elek, Marton. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit cc229be9ebb1b3737c9f5fcb29788b51fa584e47)
 2019-06-18 12:00:36 -07:00
Wei-Chiu Chuang
223a582eb8 HDFS-14010. Pass correct DF usage to ReservedSpaceCalculator builder. Contributed by Virajith Jalaparti. 
(cherry picked from commit 1cc1530b4e874391a17ec81e24cb55dfbbabd36a)
 2019-06-18 00:34:38 -07:00
Shweta Yakkali
99abd35a75 HADOOP-16156. [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl. Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit dd4a7633ece11c528a58146f92522b55e7be4dc6)
 2019-06-17 23:48:29 -07:00
Wanqiang Ji
f148b29508 YARN-9630. [UI2] Add a link in docs's top page 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit eb6be4643f77b3284297950da4f7e6ca9db11793)
 2019-06-18 14:57:01 +09:00
Adam Antal
2667177248 HADOOP-15914. hadoop jar command has no help argument. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 9f47a2e4a9845b84cc963708c7924b92df2964f6)
 2019-06-17 22:56:04 -07:00
Wei-Chiu Chuang
0b5bd8e451 HDFS-14340. Lower the log level when can't get postOpAttr. Contributed by Anuhan Torgonshar. 
(cherry picked from commit 098c325a78dce2697ec94a22a4a3981fa5a3849e)
 2019-06-17 22:47:55 -07:00
Zhankun Tang
1e7201f9aa YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji. 
(cherry picked from commit 67414a1a80039e70e0afc1de171831a6e981f37a)
 2019-06-18 13:18:27 +08:00
Wei-Chiu Chuang
e8db39b0e9 HADOOP-9157. Better option for curl in hadoop-auth-examples. Contributed by Andras Bokor. 
(cherry picked from commit f1c239c6a4c26e9057373b9b9400e54083290f65)
 2019-06-17 21:55:55 -07:00
Weiwei Yang
f36cdc4d7f HDFS-12770. Add doc about how to disable client socket cache. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 6ec57fe6a45df0e9216bbbba85b5bf61da1bf2ca)
 2019-06-17 21:46:58 -07:00
Wei-Chiu Chuang
45543ffd6a HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu. 
(cherry picked from commit 62ad9885ea8c75c134de43a3a925c76b253658e1)
 2019-06-17 20:19:59 -07:00
Akira Ajisaka
f68099aa13 HDFS-11950. Disable libhdfs zerocopy test on Mac. Contributed by Akira Ajisaka. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a95e87d827381a7f87c01f0befc2475f9478e63b)
 2019-06-17 17:03:51 -07:00
Santosh Marella
1ee18ebd26 HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2019-06-17 16:26:39 -07:00
Wei-Chiu Chuang
412fbf12cb Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao." 
This reverts commit fe46d1c79f46997711cded4b604243624665adfa.
 2019-06-17 16:25:49 -07:00
Wei-Chiu Chuang
0eb42e1e05 HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai. 
(cherry picked from commit 7db922c6474cb29e0e9bad320fcda4769366340a)
 2019-06-17 15:18:00 -07:00
huzheng
9b61ecfcbe HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read 
(cherry picked from commit ea3b0a184405c3feca024a560807ea215b6858b9)
 2019-06-17 07:20:34 -07:00
Santosh Marella
fe46d1c79f HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2019-06-14 15:15:58 -07:00
Inigo Goiri
335aebb9c4 HDFS-14560. Allow block replication parameters to be refreshable. Contributed by Stephen O'Donnell. 
(cherry picked from commit 4f455290b15902e7e44c4b1a762bf915414b2bb6)
 2019-06-13 18:27:26 -07:00