Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,901 Commits 326 Branches 371 Tags
Commit Graph

3729 Commits

Author SHA1 Message Date
Chen Liang
b4fc93d90b HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen. 2019-08-09 15:49:56 -07:00
Chen Liang
bf9d8ba66b HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-08-09 15:49:56 -07:00
Chen Liang
9f90bcadf2 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-08-09 15:49:56 -07:00
Plamen Jeliazkov
9803d8db4c HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Chao Sun
04eefe245a HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-08-09 15:49:55 -07:00
Erik Krogen
85f81fe458 HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen. 
HDFS-10519. Add a configuration option to enable in-progress edit log tailing. Contributed by Jiayi Zhou.
 2019-08-09 15:49:55 -07:00
Erik Krogen
88d65af8a1 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2019-08-09 15:49:55 -07:00
Plamen Jeliazkov
3ed8d45660 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Erik Krogen
d5da529acf HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-08-09 15:49:55 -07:00
Erik Krogen
d0c85f83da HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Plamen Jeliazkov
dd70e6d364 HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Yiqun Lin
63531be505 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun. 2019-08-07 23:01:51 +08:00
Steve Loughran
e8ccc299f6
HADOOP-15237. In KMS docs there should be one space between KMS_LOG and NOTE. 
Contributed by Snigdhanjali Mishra.

Change-Id: I3abaa658c8786f8afa802ccbb629551313431b0e
 2019-08-01 10:37:28 +01:00
Akira Ajisaka
46c00aca21
HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich. 
(cherry picked from commit cbfa3f3e988b8a3f142abadfe4f18201b6438ac9)
 2019-07-29 17:40:14 -07:00
Erik Krogen
edb46353c6 HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen. 
(cherry-picked from 62efb6300619670d3e0554c3ba14c264fa0c705b)
(cherry-picked from 5ec34bf43adb498797afd24bb02989d4fec25409)
(cherry-picked from e2aab0efef9d504e4695a9c1df2835ed60f6c0c7)
(cherry-picked from f7e517ee2432f3c0a40c9172413753670bf07db4)
 2019-07-26 11:44:12 -07:00
Steve Loughran
c0fd09c42c
HADOOP-9844. NPE when trying to create an error message response of SASL RPC 
This closes #55

Change-Id: I10a20380565fa89762f4aa564b2f1c83b9aeecdc
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 07530314c2130ecd1525682c59bf51f15b82c024)
 2019-07-26 17:55:45 +09:00
Masatake Iwasaki
62fdfaf093 HADOOP-16386. FindBugs warning in branch-2: GlobalStorageStatistics defines non-transient non-serializable instance field map. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-07-17 07:50:24 +09:00
stack
a7c6d25dc3 HDFS-14483 Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9"" 
This is a revert of a revert, i.e. a reapplication of the HDFS-14483
patch. Original commit was missing the JIRA number.

Revert "Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9""

This reverts commit b26ad8a4c38d27894bd42d3fba7abff80bd895f4.
 2019-07-09 08:58:16 -07:00
stack
b26ad8a4c3 Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9" 
This reverts commit 4f09a6bcfd685bbc9e139497adfeeeff8170e344.
 2019-07-09 08:57:57 -07:00
sunlisheng
4f09a6bcfd Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9 
Signed-off-by: stack <stack@apache.org>
 2019-07-09 08:54:37 -07:00
stack
bb7e143d34 HDFS-14585 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9 
Reapply but include JIRA # in the commit message this time.

Revert "Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9""

This reverts commit 1dd54163fcb40b60efc7ca153d443129e424df7d.
 2019-07-01 11:08:05 -07:00
stack
1dd54163fc Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9" 
This reverts commit bccb0fd846b7cc69e0d83140193eaa7fc0ac081d.
 2019-07-01 11:07:43 -07:00
sunlisheng
bccb0fd846 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: stack <stack@apache.org>
 2019-07-01 10:59:58 -07:00
Takanobu Asanuma
64102e865f HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles. 
(cherry picked from commit 54f9f75a443d7d167a7aa7d04a87e3f5af049887)
 2019-06-14 10:21:30 +09:00
Erik Krogen
e9eaf21874 HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen. 
(cherry-picked from 76b94c274fe9775efcfd51c676d80c88a4f7fdb9)
(cherry-picked from f607efd1950211f85eb571564b774184f8fa1687)
(cherry-picked from cf3c1b9ffc8cfce2e2248a1c1600bd34ff412933)
(cherry-picked from d43219c21d830b198710f70c16ea880302f88d77)
 2019-06-07 14:23:23 -07:00
Takanobu Asanuma
e3d4f6bfed HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:41:21 +09:00
Akira Ajisaka
cea1b0000c
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:54 +09:00
Alexis Daboville
b70fdc64d3
HADOOP-16248. MutableQuantiles leak memory under heavy load. 
Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac703b10de59a8ca371fe957a4a338601)
 2019-05-17 15:27:47 +01:00
Yuan Gao
18f66a05cf
HADOOP-16205 Backport ABFS driver from trunk to branch 2.0: Fix build and test failures. 
Contributed by Yuan Gao.
 2019-04-25 18:36:14 +01:00
Steve Loughran
94d0de2eab HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. 
Contributed by Steve Loughran.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
3bd089ddca HADOOP-15663. ABFS: Simplify configuration. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
b537d8c442 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. 
Contributed by Steve Loughran and Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran
1b64b0f924 HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. 
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
 2019-04-24 21:46:25 +00:00
Chen Liang
7935485eec HDFS-14205. Backport HDFS-6440 to branch-2. Contributed by Chao Sun. 2019-03-26 12:12:14 -07:00
Erik Krogen
c3973c9ae6 HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed. 
(cherry-picked from 8c95cb9d6bef369fef6a8364f0c0764eba90e44a)
(cherry-picked from 0de8b55a095ada2b98c0a41899651bd8e524f42e)
(cherry-picked from d4fbbc83ad8c4d818deccf62b4c54cead1d17a8f)
(cherry-picked from e172fc62ce3fa10a36582bc6667edec931fea9fa)
 2019-03-18 09:00:40 -07:00
Steve Loughran
9b8044d00b
HADOOP-16109. Parquet reading S3AFileSystem causes EOF (#589) 
Nobody gets seek right. No matter how many times they think they have.

Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran

Change-Id: I39b87f3d5daa98f65de2c0a44e348821a4930573
 2019-03-13 15:33:58 +00:00
Tsz Wo Nicholas Sze
5bed24ad5f HADOOP-16126. ipc.Client.stop() may sleep too long to wait for all connections. 
(cherry picked from commit 0edb0c51dc2c4ae2f353e260f01912e28033d70f)
 2019-03-01 17:52:53 +09:00
Eric Yang
bae607f734 Revert "HADOOP-13707. Skip authorization for anonymous user to access Hadoop" 
This reverts commit 439422fff923ae6aea1f7547fe24d0e23fbd8f7f.
 2019-02-21 17:36:59 -05:00
Eric Yang
4a45a9b57e Revert "HADOOP-13707. Skip authorization for anonymous user to access Hadoop" 
This reverts commit a4356bb7a5bf2fdf41bbced45968416e8ed945e6.
 2019-02-21 17:36:02 -05:00
Eric Yang
da854234a2 Revert "HADOOP-13707. Skip authorization for anonymous user to access Hadoop" 
This reverts commit 7993fb5b8127a219e6c936206b5c5c0790e82946.
 2019-02-21 17:35:19 -05:00
Daryn Sharp
665cad03f3 HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a87e458432609b7a35a2abd6410b02e8a2ffc974)
(cherry picked from commit ae8839e6e8cc3e8f8d5e50525d3302038ada484b)
(cherry picked from commit 704330a616c17256b3e39370f999928ba1c463e6)
(cherry picked from commit 4eccf2a3cc6b1468085f48ee267b2093b4f5be9d)
 2019-02-20 18:22:48 -08:00
Chen Liang
d3b06d179d HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian. 2019-01-17 13:30:42 -08:00
Chen Liang
1dbe6c9780 HADOOP-16013. DecayRpcScheduler decay thread should run as a daemon. Contributed by Erik Krogen. 2019-01-11 12:57:30 -08:00
Akira Ajisaka
1684301485
HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper 
(cherry picked from commit d4ca907da636892e4ab98e232fe0f7f77f1f7aac)
 2019-01-11 13:05:58 +09:00
Jason Lowe
d62bfaf1a4 HADOOP-15973. Configuration: Included properties are not cached if resource is a stream. Contributed by Eric Payne 2018-12-20 10:02:18 -06:00
shwetayakkali@cloudera.com
500b2a0ca6
HADOOP-16008. Fix typo in CommandsManual.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-12-18 13:27:14 +09:00
Inigo Goiri
5cef413a8e HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak. 2018-12-03 13:23:10 -08:00
Inigo Goiri
a29eb49f85 Revert "HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak." 
This reverts commit b98ffbe3f228b706b8e547f82e134233ab691443.
 2018-12-03 13:22:14 -08:00
Giovanni Matteo Fumarola
b98ffbe3f2 HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak. 2018-12-03 12:46:30 -08:00
Steve Loughran
a3470c65d8
HADOOP-15959. Revert "HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple" 2018-11-30 11:33:19 +00:00