Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,011 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

18011 Commits

Author SHA1 Message Date
Konstantin V Shvachko be8beda8dc HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-25 10:07:04 -07:00
Konstantin V Shvachko ad80383b34 HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko. 2019-07-25 10:07:04 -07:00
Chen Liang 60f0e4df69 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-07-25 10:07:04 -07:00
Chao Sun b41997ceac HDFS-13789. Reduce logging frequency of QuorumJournalManager#selectInputStreams. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Erik Krogen 0f811b0849 HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang. 2019-07-25 10:07:04 -07:00
Erik Krogen 9976b7bcbe HDFS-13610. [SBN read] Edit Tail Fast Path Part 4: Cleanup. Integration test, documentation, remove unnecessary dummy sync, minors fixups. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Plamen Jeliazkov ac0cdb0a8d HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2019-07-25 10:07:04 -07:00
Chao Sun 13e86926a5 HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Konstantin V Shvachko c02f4ccba1 HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-07-25 10:07:04 -07:00
Erik Krogen f847983f2c HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Erik Krogen db9cec6ad2 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Erik Krogen eacf43252a HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen. 2019-07-25 10:07:04 -07:00
Plamen Jeliazkov d502e48a7c HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-07-25 10:07:04 -07:00
Erik Krogen d7e0011b63 HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Erik Krogen 849e1f018e HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Erik Krogen c2585f7e28 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-07-25 10:07:04 -07:00
Plamen Jeliazkov ccad9ce59b HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-07-25 10:07:04 -07:00
Chao Sun 44d8b43697 HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Eric E Payne 6aa76eaee3 YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 
Addendum to fix 3.0 build
 2019-07-23 21:53:27 +00:00
Jonathan Hung 3b346f50de YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-22 10:54:38 -07:00
Weiwei Yang 6c8cfd135e HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang. 2019-07-11 09:53:28 +08:00
Weiwei Yang ef35954cbe HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao. 2019-07-09 22:22:50 +08:00
Weiwei Yang 9ab56b2637 Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao." 
This reverts commit 974dd2b4b6.
 2019-07-09 12:23:39 +08:00
Erik Krogen 172e0ee0b9 HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun. 
(cherry-picked from 4632708148)
(cherry-picked from 491dc7cc9d)
(cherry-picked from a59553b5e6)
 2019-07-08 15:20:41 -07:00
leosunli 14d2c09ea6 HDFS-13694. Making md5 computing being in parallel with image loading. 
(cherry picked from commit 9c90729486)
 2019-07-07 20:03:29 -07:00
Weiwei Yang 78a6e40994 YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi. 2019-07-02 23:05:18 +08:00
Inigo Goiri 9daa45f646 HDFS-14247. Repeat adding node description into network topology. Contributed by HuangTao. 
(cherry picked from commit 80b77deb42)
(cherry picked from commit 9637124535)
(cherry picked from commit 90b88db35d)
 2019-06-24 21:17:22 -07:00
Inigo Goiri 9aae8b05a7 HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException. 
(cherry picked from commit 0966407ad6)
 2019-06-24 19:03:58 -07:00
Santosh Marella 974dd2b4b6 HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 026cc2647d)
 2019-06-23 20:28:14 -07:00
Wei-Chiu Chuang 337a7090b3 HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 
(cherry picked from commit a5d76a4965)
(cherry picked from commit f33bcbb687)
 2019-06-20 20:54:58 -07:00
Zsolt Venczel 76faa41f18 HDFS-14101. Random failure of testListCorruptFilesCorruptedBlock. Contributed by Zsolt Venczel, Nikhil Navadia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Nikhil Navadia <nikhil.navadiya@cloudera.com>
(cherry picked from commit 7c00756aff)
(cherry picked from commit 4ea3b04bf3)
 2019-06-18 14:38:28 -07:00
Zhankun Tang 6da0151081 YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji. 
(cherry picked from commit 67414a1a80)
 2019-06-18 13:19:28 +08:00
Wei-Chiu Chuang 79f8e4d5bc HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu. 
(cherry picked from commit 62ad9885ea)
(cherry picked from commit 45543ffd6a)
(cherry picked from commit fc8983a5b8)
 2019-06-17 20:20:21 -07:00
Akira Ajisaka 010a99104b HDFS-11950. Disable libhdfs zerocopy test on Mac. Contributed by Akira Ajisaka. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a95e87d827)
(cherry picked from commit f68099aa13)
(cherry picked from commit 3d1d8623a1)
 2019-06-17 17:04:19 -07:00
Wei-Chiu Chuang fae98959ad HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai. 
(cherry picked from commit 7db922c6474cb29e0e9bad320fcda4769366340a)
(cherry picked from commit 0eb42e1e05)
(cherry picked from commit b3e0430c3b)
 2019-06-17 15:18:54 -07:00
huzheng 2b9c4fccf4 HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read 
(cherry picked from commit ea3b0a1844)
(cherry picked from commit 9b61ecfcbe)
(cherry picked from commit b082628e5a)
 2019-06-17 07:21:46 -07:00
Takanobu Asanuma 1be7c42cb7 HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles. 
(cherry picked from commit 54f9f75a44)
 2019-06-14 10:20:33 +09:00
Erik Krogen d43219c21d HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen. 
(cherry-picked from 76b94c274f)
(cherry-picked from f607efd195)
(cherry-picked from cf3c1b9ffc)
 2019-06-07 14:22:48 -07:00
Sean Mackrory c352b0011e HADOOP-16212. Update guava to 27.0-jre in hadoop-project branch-3.0. Contributed by Gabor Bota. 2019-06-03 07:45:56 -06:00
Kihwal Lee fd8b8c7494 HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee 
(cherry picked from commit 9122b9b649)
 2019-05-30 17:36:19 -05:00
Ayush Saxena 942c5b04f2 HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c1caab40f2)
(cherry picked from commit 2cd7577a5a)
(cherry picked from commit 90cc5ddd49)
 2019-05-29 20:58:54 -07:00
Eric E Payne 7da78f06a3 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 
(cherry picked from commit 3c63551101)
 2019-05-29 19:23:23 +00:00
Ahmed Hussein 9d63fc7ed6 YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2019-05-29 14:19:20 -05:00
Erik Krogen 2df08761e9 HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen. 
(cherry-picked from 55e0c134f0)
(cherry-picked from c26cf22d6b)
(cherry-picked from 9295e95174)
 2019-05-24 13:20:01 -07:00
Konstantin V Shvachko e4ed873594 HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko. 2019-05-24 12:38:07 -07:00
Alexis Daboville 3b09902dd2
HADOOP-16248. MutableQuantiles leak memory under heavy load. 
Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac7)
 2019-05-17 15:18:19 +01:00
Weiwei Yang afe2b98daf HADOOP-16306. AliyunOSS: Remove temporary files when upload small files to OSS. Contributed by wujinhu. 
(cherry picked from commit 2d8282bb82)
 2019-05-14 14:08:36 -07:00
Haibo Chen aa7ff8552e YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung) 
(cherry picked from commit 597fa47ad1)
(cherry picked from commit c6573562cb)
(cherry picked from commit ea1f0f282b)
 2019-05-06 11:59:49 -07:00
Eric E Payne 01fa72a825 YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein. 2019-05-03 15:56:20 +00:00
Erik Krogen c20d61c34d HADOOP-16265. Fix bug causing Configuration#getTimeDuration to use incorrect units when the default value is used. Contributed by starphin. 
(cherry-picked from 1ddb48872f)
(cherry-picked from 28ff96f367)
(cherry-picked from 9da70a1b4b)
 2019-04-22 08:20:17 -07:00