Chen Liang
3998bab3da
HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen.
2019-06-28 14:23:05 -07:00
Chen Liang
4c4ca86286
HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang.
2019-06-28 14:22:11 -07:00
Chen Liang
e2337bfc57
HDFS-13767. Add msync server implementation. Contributed by Chen Liang.
2019-06-28 14:22:10 -07:00
Plamen Jeliazkov
7e0a71dd7f
HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov.
2019-06-28 14:22:08 -07:00
Chao Sun
a97f00e1c6
HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun.
2019-06-28 14:22:08 -07:00
Erik Krogen
b13110a6f0
HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen.
2019-06-28 14:22:07 -07:00
Plamen Jeliazkov
a3521c53fe
HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov.
2019-06-28 14:22:03 -07:00
Erik Krogen
49d7bb6a92
HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
2019-06-28 14:20:01 -07:00
Erik Krogen
c0ca2bb853
HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov.
2019-06-28 14:20:01 -07:00
Plamen Jeliazkov
1eeca2d9fb
HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov.
2019-06-28 14:20:00 -07:00
Wei-Chiu Chuang
77e391fd8b
HADOOP-14385. HttpExceptionUtils#validateResponse swallows exceptions. Contributed by Wei-Chiu Chuang.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 840d02ca5b
)
2019-06-20 19:58:17 -07:00
Inigo Goiri
f046b881f6
HADOOP-16028. Fix NetworkTopology chooseRandom function to support excluded nodes. Contributed by Sihai Ke.
...
(cherry picked from commit f4e18242bd
)
2019-06-20 13:38:00 -07:00
Shweta Yakkali
99abd35a75
HADOOP-16156. [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl. Contributed by Shweta Yakkali.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit dd4a7633ec
)
2019-06-17 23:48:29 -07:00
Adam Antal
2667177248
HADOOP-15914. hadoop jar command has no help argument. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 9f47a2e4a9845b84cc963708c7924b92df2964f6)
2019-06-17 22:56:04 -07:00
Takanobu Asanuma
73829589b7
HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles.
...
(cherry picked from commit 54f9f75a44
)
2019-06-14 10:18:46 +09:00
Sean Mackrory
e0b3cbd221
HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota.
2019-06-13 07:53:40 -06:00
Erik Krogen
f607efd195
HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen.
...
(cherry-picked from 76b94c274f
)
2019-06-07 14:22:08 -07:00
Takanobu Asanuma
a9a3450560
HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka.
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:34:16 +09:00
Akira Ajisaka
855dc997d6
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:27:33 +09:00
Shweta
7a5d9ab9ee
HDFS-14451. Incorrect header or version mismatch log message. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 93d7630062
)
2019-05-24 18:43:30 +02:00
Alexis Daboville
3f05b699ef
HADOOP-16248. MutableQuantiles leak memory under heavy load.
...
Contributed by Alexis Daboville,
(cherry picked from commit 4cb3da6ac7
)
2019-05-17 15:16:59 +01:00
David Mollitor
6eb48542f1
HADOOP-16307. Intern User Name and Group Name in FileStatus.
...
Author: David Mollitor
(cherry picked from commit 2713dcf6e9
)
2019-05-16 16:03:06 +02:00
Bharat Viswanadham
2c72f48527
HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy.
...
(cherry picked from commit d4c8858586
)
2019-05-15 17:56:22 -07:00
Siyao Meng
095815220d
HADOOP-16289. Allow extra jsvc startup option in hadoop_start_secure_daemon in hadoop-functions.sh. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 93f2283a69
)
2019-05-06 15:47:58 -07:00
Erik Krogen
28ff96f367
HADOOP-16265. Fix bug causing Configuration#getTimeDuration to use incorrect units when the default value is used. Contributed by starphin.
...
(cherry-picked from 1ddb48872f
)
2019-04-22 08:18:17 -07:00
Akira Ajisaka
b28fdb98ee
HADOOP-14635. Javadoc correction for AccessControlList#buildACL. Contributed by Yeliang Cang.
...
(cherry picked from commit ab645b3caa
)
2019-04-08 15:19:17 +09:00
David Mollitor
50fc9577c0
HADOOP-16208. Do Not Log InterruptedException in Client.
...
Contributed by David Mollitor.
(cherry picked from commit c90736350b
)
2019-04-04 16:16:57 +01:00
Steve Loughran
b6ebe74526
HADOOP-16233. S3AFileStatus to declare that isEncrypted() is always true ( #685 )
...
This is needed to fix up some confusion about caching of job.addCache() handling of S3A paths; all parent dirs -the files are downloaded by the NM without using the DTs of the user submitting the job. This means that when you submit jobs to an EC2 cluster with lower IAM permissions than the user, cached resources don't get downloaded and the job doesn't start.
Production code changes:
* S3AFileStatus Adds "true" to the superclass's encrypted flag during construction.
Tests
* Base AbstractContractOpenTest can control whether zero byte files created in tests are encrypted. Not done via an XML attribute, just a subclass point. Thoughts?
* Verify that the filecache considers paths to not have the permissions which trigger reduce-privilege downloads
* And extend ITestDelegatedMRJob to test a completely different bucket (open street map), to verify that cached resources do get their tokens picked up
Docs:
* Advise FS developers to say all files are encrypted. It's otherwise harmless and it'll stop other people seeing impossible to debug error messages on app launch.
Contributed by Steve Loughran.
Change-Id: Ifaae4c9d735ccc5eafeebd2584b65daf2d4e5da3
(cherry picked from commit 366186d999
)
2019-04-03 21:35:19 +01:00
Akira Ajisaka
b6039e241d
HADOOP-16226. new Path(String str) does not remove all the trailing slashes of str
...
(cherry picked from commit aaaf856f4b
)
2019-04-03 13:18:36 +09:00
Akira Ajisaka
8037052c2b
HADOOP-16225. Fix links to the developer mailing lists in DownstreamDev.md. Contributed by Wanqiang Ji.
...
(cherry picked from commit ebd0d21538
)
2019-04-02 10:54:16 +09:00
David Mollitor
397b63ad0b
HADOOP-16181. HadoopExecutors shutdown Cleanup.
...
Author: David Mollitor <david.mollitor@cloudera.com>
(cherry picked from commit d18d0859eb
)
2019-03-22 10:30:21 +00:00
David Mollitor
9a449ac075
HADOOP-16196. Path Parameterize Comparable.
...
Author: David Mollitor <david.mollitor@cloudera.com>
(cherry picked from commit 246ab77f28
)
2019-03-22 10:27:17 +00:00
Erik Krogen
0de8b55a09
HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed.
...
(cherry-picked from 8c95cb9d6b
)
2019-03-18 08:46:53 -07:00
Ben Roling
43e8ac6097
HADOOP-15625. S3A input stream to use etags/version number to detect changed source files.
...
Author: Ben Roling <ben.roling@gmail.com>
Initial patch from Brahma Reddy Battula.
2019-03-14 19:46:34 +00:00
Erik Krogen
fec7c5f3eb
HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun.
...
(cherry picked from commit 66357574ae
)
2019-03-13 13:19:18 -07:00
Steve Loughran
b6f6c34223
HADOOP-16109. Parquet reading S3AFileSystem causes EOF
...
Nobody gets seek right. No matter how many times they think they have.
Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran
2019-03-11 11:15:25 +00:00
Praveen Krishna
451844fee5
HADOOP-16114. NetUtils#canonicalizeHost gives different value for same host.
...
Author: Praveen Krishna <praveenkrishna@tutanota.com>
(cherry picked from commit 2b94e51a8f
)
2019-03-07 11:08:48 +00:00
Stephen O'Donnell
3fe31b36fa
HADOOP-16140. hadoop fs expunge to add -immediate option to purge trash immediately.
...
Contributed by Stephen O'Donnell.
(cherry picked from commit 686c0141ef
)
Signed-off-by: Steve Loughran <stevel@apache.org>
2019-03-05 14:11:49 +00:00
Daryn Sharp
ae8839e6e8
HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a87e458432
)
2019-02-20 18:21:14 -08:00
Akira Ajisaka
3e3275a80b
YARN-9308. fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240. Contributed by Wilfred Spiegelenburg.
...
(cherry picked from commit dabfeab785
)
2019-02-15 14:53:36 +09:00
Vinayakumar B
49af36b25c
HADOOP-16108. Tail Follow Interval Should Allow To Specify The Sleep Interval To Save Unnecessary RPC's. Contributed by Ayush Saxena.
2019-02-13 16:45:34 +05:30
Wangda Tan
649da5af04
Make upstream aware of 3.1.2 release
...
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa
)
2019-02-05 14:05:51 -08:00
Steve Loughran
bdd17be9ec
HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests.
...
Contributed by Ewan Higgs and Steve Loughran.
(cherry picked from commit c1d24f8483
)
2019-02-04 17:10:19 +00:00
Steve Loughran
3b47371fe2
HADOOP-16079. Token.toString faulting if any token listed can't load.
...
Contributed by Steve Loughran.
(cherry picked from commit 7f46d13dac
)
2019-02-01 14:34:11 +00:00
Eric Yang
65d2b13dc2
HADOOP-15922. Fixed DelegationTokenAuthenticator URL decoding for doAs user.
...
Contributed by He Xiaoqiao
(cherry picked from commit 0dd35e218f
)
2019-01-22 19:39:05 -05:00
Chen Liang
54d4abdfe9
HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian.
2019-01-18 11:54:20 -08:00
luhuachao
18becec548
HADOOP-16019. ZKDelegationTokenSecretManager won't log exception message occured in function setJaasConfiguration.
...
Contributed by luhuachao.
(cherry picked from commit ccc43621ab
)
2019-01-15 13:16:07 +00:00
Inigo Goiri
c1c15c1bd8
HADOOP-16045. Don't run TestDU on Windows. Contributed by Lukas Majercak.
...
(cherry picked from commit 35fa3bd685
)
2019-01-11 18:08:14 -08:00
Akira Ajisaka
d345b1f441
HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper
...
(cherry picked from commit d4ca907da6
)
2019-01-11 13:04:40 +09:00
Jason Lowe
6236de9e23
Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
...
This reverts commit 1f39eae7e6
.
2019-01-09 17:41:52 -06:00