Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,756 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4858 Commits

Author SHA1 Message Date
Jianfei Jiang 0c02217d9b
HDFS-14691. Wrong usage hint for hadoop fs command "test". 
Contributed by Jianfei Jiang.

Change-Id: I9f5e89721ff210641375fbf42a70043f0d74458e
(cherry picked from commit 71aad60e51)
 2019-08-05 13:09:39 +01:00
He Xiaoqiao a405db8014 HDFS-13529. Fix default trash policy emptier trigger time correctly. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit f86de6f76a)
 2019-08-01 14:56:38 -07:00
Christopher Gregorian d4492bdd9e HADOOP-16459. Backport of HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian. 
This commit also includes the follow-on commit 827a84778a.

(cherry-picked from f96a2df38d)
 2019-07-30 14:41:36 -07:00
Akira Ajisaka f921c72fda
HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich. 
(cherry picked from commit cbfa3f3e98)
 2019-07-29 17:39:05 -07:00
Erik Krogen 5ec34bf43a HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen. 
(cherry-picked from 62efb63006)
 2019-07-26 11:39:14 -07:00
Steve Loughran 7418764a67
HADOOP-9844. NPE when trying to create an error message response of SASL RPC 
This closes #55

Change-Id: I10a20380565fa89762f4aa564b2f1c83b9aeecdc
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 07530314c2)
 2019-07-26 17:54:32 +09:00
S O'Donnell 23b470ee92
HADOOP-16443. Improve help text for setfacl --set option. 
Contributed by S O'Donnell.

Change-Id: I1da46c4c414a5d2b07ee15867508f0799440a413
 2019-07-23 10:25:59 +01:00
Gopal V 76d2cd2283
HADOOP-16341. ShutDownHookManager: Regressed performance on Hook removals after HADOOP-15679 
Contributed by Gopal V and Atilla Magyar.

Change-Id: I066d5eece332a1673594de0f9b484443f95530ec
 2019-07-17 13:52:58 +01:00
Inigo Goiri 1dbaf74985 HADOOP-16385. Namenode crashes with 'RedundancyMonitor thread received Runtime exception'. Contributed by Ayush Saxena. 
(cherry picked from commit aa9f0850e8)
 2019-07-02 13:23:55 -07:00
Konstantin V Shvachko c936562c72 HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2019-06-28 14:23:14 -07:00
Chao Sun eb7d7fa5e2 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2019-06-28 14:23:13 -07:00
Erik Krogen e917ac2854 HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun. 2019-06-28 14:23:12 -07:00
Chen Liang 790831824f HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng. 2019-06-28 14:23:12 -07:00
Erik Krogen c1c061d767 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2019-06-28 14:23:12 -07:00
Chao Sun 45d6834d4c HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun. 2019-06-28 14:23:10 -07:00
Chao Sun cc70e81931 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2019-06-28 14:23:09 -07:00
Chen Liang 3998bab3da HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen. 2019-06-28 14:23:05 -07:00
Chen Liang 4c4ca86286 HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2019-06-28 14:22:11 -07:00
Chen Liang e2337bfc57 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2019-06-28 14:22:10 -07:00
Plamen Jeliazkov 7e0a71dd7f HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2019-06-28 14:22:08 -07:00
Chao Sun a97f00e1c6 HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2019-06-28 14:22:08 -07:00
Erik Krogen b13110a6f0 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2019-06-28 14:22:07 -07:00
Plamen Jeliazkov a3521c53fe HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-06-28 14:22:03 -07:00
Erik Krogen 49d7bb6a92 HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-06-28 14:20:01 -07:00
Erik Krogen c0ca2bb853 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-06-28 14:20:01 -07:00
Plamen Jeliazkov 1eeca2d9fb HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-06-28 14:20:00 -07:00
Wei-Chiu Chuang 77e391fd8b HADOOP-14385. HttpExceptionUtils#validateResponse swallows exceptions. Contributed by Wei-Chiu Chuang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 840d02ca5b)
 2019-06-20 19:58:17 -07:00
Inigo Goiri f046b881f6 HADOOP-16028. Fix NetworkTopology chooseRandom function to support excluded nodes. Contributed by Sihai Ke. 
(cherry picked from commit f4e18242bd)
 2019-06-20 13:38:00 -07:00
Shweta Yakkali 99abd35a75 HADOOP-16156. [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl. Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit dd4a7633ec)
 2019-06-17 23:48:29 -07:00
Adam Antal 2667177248 HADOOP-15914. hadoop jar command has no help argument. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 9f47a2e4a9845b84cc963708c7924b92df2964f6)
 2019-06-17 22:56:04 -07:00
Takanobu Asanuma 73829589b7 HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles. 
(cherry picked from commit 54f9f75a44)
 2019-06-14 10:18:46 +09:00
Sean Mackrory e0b3cbd221 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:53:40 -06:00
Erik Krogen f607efd195 HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen. 
(cherry-picked from 76b94c274f)
 2019-06-07 14:22:08 -07:00
Takanobu Asanuma a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka 855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Shweta 7a5d9ab9ee HDFS-14451. Incorrect header or version mismatch log message. Contributed by Shweta. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 93d7630062)
 2019-05-24 18:43:30 +02:00
Alexis Daboville 3f05b699ef
HADOOP-16248. MutableQuantiles leak memory under heavy load. 
Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac7)
 2019-05-17 15:16:59 +01:00
David Mollitor 6eb48542f1
HADOOP-16307. Intern User Name and Group Name in FileStatus. 
Author:    David Mollitor

(cherry picked from commit 2713dcf6e9)
 2019-05-16 16:03:06 +02:00
Bharat Viswanadham 2c72f48527
HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy. 
(cherry picked from commit d4c8858586)
 2019-05-15 17:56:22 -07:00
Siyao Meng 095815220d HADOOP-16289. Allow extra jsvc startup option in hadoop_start_secure_daemon in hadoop-functions.sh. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 93f2283a69)
 2019-05-06 15:47:58 -07:00
Erik Krogen 28ff96f367 HADOOP-16265. Fix bug causing Configuration#getTimeDuration to use incorrect units when the default value is used. Contributed by starphin. 
(cherry-picked from 1ddb48872f)
 2019-04-22 08:18:17 -07:00
Akira Ajisaka b28fdb98ee
HADOOP-14635. Javadoc correction for AccessControlList#buildACL. Contributed by Yeliang Cang. 
(cherry picked from commit ab645b3caa)
 2019-04-08 15:19:17 +09:00
David Mollitor 50fc9577c0
HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.

(cherry picked from commit c90736350b)
 2019-04-04 16:16:57 +01:00
Steve Loughran b6ebe74526
HADOOP-16233. S3AFileStatus to declare that isEncrypted() is always true (#685) 
This is needed to fix up some confusion about caching of job.addCache() handling of S3A paths; all parent dirs -the files are downloaded by the NM without  using the DTs of the user submitting the job. This means that when you submit jobs to an EC2 cluster with lower IAM permissions than the user, cached resources don't get downloaded and the job doesn't start.

Production code changes:
* S3AFileStatus Adds "true" to the superclass's encrypted flag during construction.

Tests
* Base AbstractContractOpenTest can control whether zero byte files created in tests are encrypted. Not done via an XML attribute, just a subclass point. Thoughts?
* Verify that the filecache considers paths to not have the permissions which trigger reduce-privilege downloads
* And extend ITestDelegatedMRJob to test a completely different bucket (open street map), to verify that cached resources do get their tokens picked up

Docs:
* Advise FS developers to say all files are encrypted. It's otherwise harmless and it'll stop other people seeing impossible to debug error messages on app launch.

Contributed by Steve Loughran.

Change-Id: Ifaae4c9d735ccc5eafeebd2584b65daf2d4e5da3
(cherry picked from commit 366186d999)
 2019-04-03 21:35:19 +01:00
Akira Ajisaka b6039e241d
HADOOP-16226. new Path(String str) does not remove all the trailing slashes of str 
(cherry picked from commit aaaf856f4b)
 2019-04-03 13:18:36 +09:00
Akira Ajisaka 8037052c2b
HADOOP-16225. Fix links to the developer mailing lists in DownstreamDev.md. Contributed by Wanqiang Ji. 
(cherry picked from commit ebd0d21538)
 2019-04-02 10:54:16 +09:00
David Mollitor 397b63ad0b
HADOOP-16181. HadoopExecutors shutdown Cleanup. 
Author:    David Mollitor <david.mollitor@cloudera.com>
(cherry picked from commit d18d0859eb)
 2019-03-22 10:30:21 +00:00
David Mollitor 9a449ac075
HADOOP-16196. Path Parameterize Comparable. 
Author:    David Mollitor <david.mollitor@cloudera.com>

(cherry picked from commit 246ab77f28)
 2019-03-22 10:27:17 +00:00
Erik Krogen 0de8b55a09 HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed. 
(cherry-picked from 8c95cb9d6b)
 2019-03-18 08:46:53 -07:00
Ben Roling 43e8ac6097
HADOOP-15625. S3A input stream to use etags/version number to detect changed source files. 
Author: Ben Roling <ben.roling@gmail.com>

Initial patch from Brahma Reddy Battula.
 2019-03-14 19:46:34 +00:00