Chen Liang
bf9d8ba66b
HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang.
2019-08-09 15:49:56 -07:00
Erik Krogen
7ccb2a48ff
HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen.
...
Also add a missing change from HDFS-13610
2019-08-09 15:49:56 -07:00
Konstantin V Shvachko
c09665d14a
HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko.
2019-08-09 15:49:56 -07:00
Konstantin V Shvachko
fcafa72958
HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko.
2019-08-09 15:49:56 -07:00
Chen Liang
9f90bcadf2
HDFS-13767. Add msync server implementation. Contributed by Chen Liang.
2019-08-09 15:49:56 -07:00
Chao Sun
f531301e7e
HDFS-13789. Reduce logging frequency of QuorumJournalManager#selectInputStreams. Contributed by Erik Krogen.
2019-08-09 15:49:56 -07:00
Erik Krogen
33d8aae555
HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang.
...
Also fix a issue in IPCLoggerChannel
2019-08-09 15:49:56 -07:00
Erik Krogen
ea6d36763b
HDFS-13610. [SBN read] Edit Tail Fast Path Part 4: Cleanup. Integration test, documentation, remove unnecessary dummy sync, minors fixups. Contributed by Erik Krogen.
2019-08-09 15:49:55 -07:00
Plamen Jeliazkov
9803d8db4c
HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov.
2019-08-09 15:49:55 -07:00
Chao Sun
04eefe245a
HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun.
2019-08-09 15:49:55 -07:00
Konstantin V Shvachko
272a198175
HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko.
2019-08-09 15:49:55 -07:00
Erik Krogen
85f81fe458
HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen.
...
HDFS-10519. Add a configuration option to enable in-progress edit log tailing. Contributed by Jiayi Zhou.
2019-08-09 15:49:55 -07:00
Erik Krogen
88d65af8a1
HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen.
2019-08-09 15:49:55 -07:00
Erik Krogen
befe41074c
HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen.
2019-08-09 15:49:55 -07:00
Plamen Jeliazkov
3ed8d45660
HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov.
2019-08-09 15:49:55 -07:00
Erik Krogen
c450e69551
HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun.
2019-08-09 15:49:55 -07:00
Erik Krogen
d5da529acf
HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
2019-08-09 15:49:55 -07:00
Erik Krogen
d0c85f83da
HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov.
2019-08-09 15:49:55 -07:00
Plamen Jeliazkov
dd70e6d364
HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov.
2019-08-09 15:49:55 -07:00
Chao Sun
61b510815c
HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun.
2019-08-09 15:49:55 -07:00
Siyao Meng
15062b6d28
HDFS-14696. Backport HDFS-11273 to branch-2 (Move TransferFsImage#doGetUrl function to a Util class) ( #1251 ) Contributed by Siyao Meng.
2019-08-08 14:55:41 -07:00
Erik Krogen
ce12c8fc54
HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun.
2019-08-08 10:18:00 -07:00
Erik Krogen
97600cd916
HDFS-14631. The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun.
2019-08-07 13:46:56 -07:00
Yiqun Lin
63531be505
HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.
2019-08-07 23:01:51 +08:00
Haibo Chen
4a9fc45f6f
YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung)
...
(cherry picked from commit f51702d539
)
(cherry picked from commit 8d357343c4
)
(cherry picked from commit f943bff254
)
(cherry picked from commit 6109ecf16d
)
2019-08-06 15:14:45 -07:00
Weiwei Yang
567e1178d8
HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
2019-08-06 14:30:17 -07:00
Erik Krogen
ee1547a807
HDFS-14697. Backport of HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao.
2019-08-06 09:15:24 -07:00
Weiwei Yang
19af228957
YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg.
...
(cherry picked from commit 1d90a0dd23
)
(cherry picked from commit 70efe253f3
)
(cherry picked from commit cd8e9fa80b2903935ca9c337631e60161a6053b3)
2019-08-05 10:04:09 -07:00
Weiwei Yang
4e5ba315c6
YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang.
...
(cherry picked from commit 78ea897b67
)
(cherry picked from commit 7dd8eafe33
)
(cherry picked from commit d095325234fe8fa8d2513c2f03c03347ef5f3606)
2019-08-05 10:04:00 -07:00
Wangda Tan
75fc058290
YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda)
...
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07
)
(cherry picked from commit 46baafedf1
)
(cherry picked from commit bf4da6aacee62be906e71eca6e33821f5ee79a2d)
2019-08-05 10:03:56 -07:00
Eric Badger
9aae72af84
YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit
...
(cherry picked from commit 144a55f0e3
)
2019-08-02 21:06:54 +00:00
Erik Krogen
a503662d6c
HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.
...
(cherry picked from e7a0b8aa83
)
(cherry picked from ba4ca7d996cafa1b1786249cae2b207f0eaa1070)
(cherry picked from da548cf263
)
(cherry picked from 055f824bcb
)
2019-08-02 11:53:34 -07:00
Steve Loughran
e8ccc299f6
HADOOP-15237. In KMS docs there should be one space between KMS_LOG and NOTE.
...
Contributed by Snigdhanjali Mishra.
Change-Id: I3abaa658c8786f8afa802ccbb629551313431b0e
2019-08-01 10:37:28 +01:00
Eric E Payne
77d1aa9376
YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.
...
(cherry picked from commit c04ccf296d
)
2019-07-30 20:31:02 +00:00
Wei-Chiu Chuang
3941b3c708
HDFS-14464. Remove unnecessary log message from DFSInputStream. Contributed by Chao Sun.
2019-07-30 11:36:56 -07:00
Akira Ajisaka
46c00aca21
HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich.
...
(cherry picked from commit cbfa3f3e98
)
2019-07-29 17:40:14 -07:00
Erik Krogen
61581a86b2
HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao.
2019-07-29 14:36:57 -07:00
Wei-Chiu Chuang
27ead06a13
HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai.
2019-07-29 14:34:31 -07:00
Erik Krogen
edb46353c6
HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen.
...
(cherry-picked from 62efb63006
)
(cherry-picked from 5ec34bf43a
)
(cherry-picked from e2aab0efef9d504e4695a9c1df2835ed60f6c0c7)
(cherry-picked from f7e517ee24
)
2019-07-26 11:44:12 -07:00
Erik Krogen
653951e1f7
Revert "HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena."
...
This reverts commit 3bfae20e94
.
2019-07-26 08:27:36 -07:00
Steve Loughran
c0fd09c42c
HADOOP-9844. NPE when trying to create an error message response of SASL RPC
...
This closes #55
Change-Id: I10a20380565fa89762f4aa564b2f1c83b9aeecdc
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 07530314c2
)
2019-07-26 17:55:45 +09:00
Masatake Iwasaki
3bfae20e94
HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 6b8107ad97
)
2019-07-26 09:54:27 +09:00
Ahmed Hussein
dc4d7f69ce
YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 9d63fc7ed6
)
2019-07-24 11:21:29 -05:00
pingsutw
2cbd7eb5b6
MAPREDUCE-7076. TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build
...
This closes #1089
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ee87e9a42e
)
2019-07-23 10:36:43 +09:00
Haibo Chen
68c24b322d
YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung)
2019-07-19 11:00:38 -07:00
Masatake Iwasaki
1f0bf69e16
MAPREDUCE-6521. MiniMRYarnCluster should not create /tmp/hadoop-yarn/staging on local filesystem in unit test.
...
Contributed by Masatake Iwasaki.
Change-Id: Id74b90eb7cfb9d676188fc5aa47249ee8904a3d5
(cherry picked from commit 79f6118dcc
)
2019-07-19 09:49:44 +09:00
Masatake Iwasaki
62fdfaf093
HADOOP-16386. FindBugs warning in branch-2: GlobalStorageStatistics defines non-transient non-serializable instance field map.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-07-17 07:50:24 +09:00
Erik Krogen
cf7b9a5bd5
HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun.
2019-07-16 07:22:45 -07:00
Weiwei Yang
3f112e7347
HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.
...
(cherry picked from commit 6c8cfd135e
)
2019-07-11 10:07:47 +08:00
Yiqun Lin
78aebee5c5
HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao.
...
(cherry picked from commit 993dc8726b
)
2019-07-10 10:58:59 +08:00