Commit Graph

21147 Commits

Author SHA1 Message Date
Yiqun Lin fb8c997a68 HDDS-1048. Remove SCMNodeStat from SCMNodeManager and use storage information from DatanodeInfo#StorageReportProto. Contributed by Nanda kumar. 2019-02-08 23:49:37 +08:00
Masatake Iwasaki e0ab1bdece YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 2019-02-09 00:26:50 +09:00
Yiqun Lin 177131793a HDFS-14172. Avoid NPE when SectionName#fromString returns null. Contributed by Xiang Li. 2019-02-08 20:51:30 +08:00
Nanda kumar 394a9f7d5c HDDS-1033. Add FSStatistics for OzoneFileSystem.
Contributed by Mukul Kumar Singh.
2019-02-08 17:33:37 +05:30
Márton Elek df7b7dadf9 HDDS-1073. Fix FindBugs issues on OzoneBucketStub#createMultipartKey. Contributed by Aravindan Vijayan. 2019-02-08 11:35:55 +01:00
Nanda kumar 0c1bc4dcee HDDS-981. Block allocation should involve pipeline selection and then container selection.
Contributed by Lokesh Jain.
2019-02-08 15:44:41 +05:30
Bharat Viswanadham a140a890c6 HDDS-1069. Temporarily disable the security acceptance tests by default in Ozone. Contributed by Marton Elek. 2019-02-07 18:05:05 -08:00
Surendra Singh Lilhore 4be87353e3 HDFS-14140. JournalNodeSyncer authentication is failing in secure cluster. Contributed by Surendra Singh Lilhore.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-07 16:51:59 -08:00
Da Zhou 1f1655028e
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
Contributed by Da Zhou and Junhua Gu.
2019-02-07 21:58:21 +00:00
Steve Loughran 668817a6ce
Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth."
(accidentally mixed in two patches)

This reverts commit fa8cd1bf28.
2019-02-07 21:57:22 +00:00
Ranith Sardar 546c5d70ef
HADOOP-16032. Distcp It should clear sub directory ACL before applying new ACL on. 2019-02-07 21:48:07 +00:00
Bharat Viswanadham 75e8441c61 HDDS-1071. Make Ozone s3 acceptance test suite centos compatible. Contributed by Elek Marton. 2019-02-07 08:53:11 -08:00
Eric E Payne d1ca9432dd YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 2019-02-07 16:38:11 +00:00
Márton Elek a65aca2fef HDDS-922. Create isolated classloder to use ozonefs with any older hadoop versions. Contributed by Elek, Marton. 2019-02-07 17:02:03 +01:00
Mukul Kumar Singh 214112b2d7 HDDS-1010. ContainerSet#getContainerMap should be renamed. Contributed by Supratim Deka. 2019-02-07 18:06:23 +05:30
Andrew Olson de804e53b9
HADOOP-15281. Distcp to add no-rename copy option.
Contributed by Andrew Olson.
2019-02-07 10:07:22 +00:00
Vinayakumar B 3c96a03dea YARN-8498. Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04. Contributed by Ayush Saxena. 2019-02-07 13:02:35 +05:30
Rohith Sharma K S 7fa62e150c YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi. 2019-02-07 09:58:53 +05:30
Rohith Sharma K S 1e5e08d83b YARN-7627. [ATSv2] When passing a non-number as metricslimit, the error message is wrong. Contributed by Charan Hebri. 2019-02-07 09:58:53 +05:30
Bharat Viswanadham 912d9f790d HDDS-1044. Client doesn't propogate correct error code to client on out of disk space. Contributed by Yiqun Lin. 2019-02-06 20:02:17 -08:00
Siyao Meng 6aa63452b3 HDFS-14242. OIV WebImageViewer: NPE when param op is not specified. Contributed by Siyao Meng.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-06 12:19:21 -08:00
Erik Krogen d3de8e162b HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun. 2019-02-06 09:54:47 -08:00
Weiwei Yang 711d22f166 YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph. 2019-02-06 22:36:51 +08:00
Mukul Kumar Singh 911790cc26 HDDS-1027. Add blockade Tests for datanode isolation and scm failures. Contributed by Nilotpal Nandi. 2019-02-06 11:32:38 +05:30
Kitti Nanasi 49ddd8a6ed HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-05 15:14:19 -08:00
Wangda Tan 308f3168fa Make upstream aware of 3.1.2 release
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
2019-02-05 14:03:18 -08:00
Da Zhou fa8cd1bf28
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
Contributed by Da Zhou and Junhua Gu.
2019-02-05 19:23:15 +00:00
Suma Shivaprasad 2044967e75 YARN-9246 NPE when executing a command yarn node -status or -states without additional arguments. Contributed by Masahiro Tanaka 2019-02-05 09:39:38 -08:00
Márton Elek 194f0b49fb HDDS-631. Ozone classpath shell command is not working. Contributed by Elek, Marton. 2019-02-05 16:20:30 +01:00
Márton Elek 9f2da01591 HDDS-776. Make OM initialization resilient to dns failures. Contributed by Doroszlai, Attila. 2019-02-05 16:11:19 +01:00
Da Zhou ba9efe06fa
HADOOP-16074. WASB: Update container not found error code.
Contributed by Da Zhou.
2019-02-05 14:39:45 +00:00
Márton Elek 9ace37b119 HDDS-987. MultipartUpload: S3API for list parts of a object. Contributed by Bharat Viswanadham. 2019-02-05 13:08:36 +01:00
Steve Loughran f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile();
S3A to implement S3 Select through this API.

The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.

The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.

This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.

In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).

Contributed by Steve Loughran
2019-02-05 11:51:02 +00:00
Rohith Sharma K S e3ec18b0c4 YARN-6735. Have a way to turn off container metrics from NMs. Contributed by Abhishek Modi. 2019-02-05 13:48:04 +05:30
Sunil G ba38db4f5b YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri. 2019-02-05 13:41:44 +05:30
Weiwei Yang aa7ce50e55 YARN-9275. Add link to NodeAttributes doc in PlacementConstraints document. Contributed by Masatake Iwasaki. 2019-02-05 16:07:41 +08:00
Rohith Sharma K S 571838920d YARN-9149. yarn container -status misses logUrl when integrated with ATSv2. Contributed by Abhishek Modi. 2019-02-05 12:39:10 +05:30
Yiqun Lin 3efa168e1f HDDS-1029. Allow option for force in DeleteContainerCommand. Contributed by Bharat Viswanadham. 2019-02-05 10:51:52 +08:00
Akira Ajisaka 529791cb18
HADOOP-15938. [JDK 11] Remove animal-sniffer-maven-plugin to fix build. Contributed by Dinesh Chitlangia. 2019-02-05 09:42:44 +09:00
Ajay Kumar 9a19d6d48b HDDS-1039. OzoneManager fails to connect with secure SCM. Contributed by Ajay Kumar 2019-02-04 16:30:07 -08:00
Kitti Nanasi 5f15a60e32 HDFS-14125. Use parameterized log format in ECTopologyVerifier. Contributed by Kitti Nanasi.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-04 14:52:07 -08:00
Inigo Goiri 0e79a86582 HDFS-14202. dfs.disk.balancer.max.disk.throughputInMBperSec property is not working as per set value. Contributed by Ranith Sardar. 2019-02-04 11:59:48 -08:00
Larry McCay 758e9ce781
HADOOP-16076. SPNEGO+SSL Client Connections with HttpClient Broken.
Contributed by Larry McCay
2019-02-04 17:38:40 +00:00
Sunil G 604b2489a9 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 2019-02-04 12:48:28 +05:30
Akira Ajisaka 0f9aa5b4d5
HADOOP-16089. AliyunOSS: update oss-sdk version to 3.4.1. Contributed by wujinhu. 2019-02-04 14:46:35 +09:00
Akira Ajisaka c991e2ca5a
MAPREDUCE-7177. Disable speculative execution in TestDFSIO. Contributed by Zhaohui Xin. 2019-02-04 14:03:34 +09:00
tiwalter 9aa3dc872c
HDFS-14158. Checkpointer ignores configured time period > 5 minutes
This closes #449

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-02-04 13:51:07 +09:00
Takanobu Asanuma ec77e95bbd HDFS-14232. libhdfs is not included in binary tarball. Contributed by Akira Ajisaka. 2019-02-04 10:12:09 +09:00
Mukul Kumar Singh b6f90d3957 HDDS-1021. Allow ozone datanode to participate in a 1 node as well as 3 node Ratis Pipeline. Contributed by Shashikant Banerjee. 2019-02-03 00:53:06 +05:30
Weiwei Yang 69bcff380f YARN-9231. TestDistributedShell fix timeout. Contributed by Prabhu Joseph. 2019-02-02 23:27:31 +08:00