Commit Graph

21218 Commits

Author SHA1 Message Date
Márton Elek 9f2da01591 HDDS-776. Make OM initialization resilient to dns failures. Contributed by Doroszlai, Attila. 2019-02-05 16:11:19 +01:00
Da Zhou ba9efe06fa
HADOOP-16074. WASB: Update container not found error code.
Contributed by Da Zhou.
2019-02-05 14:39:45 +00:00
Márton Elek 9ace37b119 HDDS-987. MultipartUpload: S3API for list parts of a object. Contributed by Bharat Viswanadham. 2019-02-05 13:08:36 +01:00
Steve Loughran f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile();
S3A to implement S3 Select through this API.

The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.

The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.

This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.

In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).

Contributed by Steve Loughran
2019-02-05 11:51:02 +00:00
Rohith Sharma K S e3ec18b0c4 YARN-6735. Have a way to turn off container metrics from NMs. Contributed by Abhishek Modi. 2019-02-05 13:48:04 +05:30
Sunil G ba38db4f5b YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri. 2019-02-05 13:41:44 +05:30
Weiwei Yang aa7ce50e55 YARN-9275. Add link to NodeAttributes doc in PlacementConstraints document. Contributed by Masatake Iwasaki. 2019-02-05 16:07:41 +08:00
Rohith Sharma K S 571838920d YARN-9149. yarn container -status misses logUrl when integrated with ATSv2. Contributed by Abhishek Modi. 2019-02-05 12:39:10 +05:30
Yiqun Lin 3efa168e1f HDDS-1029. Allow option for force in DeleteContainerCommand. Contributed by Bharat Viswanadham. 2019-02-05 10:51:52 +08:00
Akira Ajisaka 529791cb18
HADOOP-15938. [JDK 11] Remove animal-sniffer-maven-plugin to fix build. Contributed by Dinesh Chitlangia. 2019-02-05 09:42:44 +09:00
Ajay Kumar 9a19d6d48b HDDS-1039. OzoneManager fails to connect with secure SCM. Contributed by Ajay Kumar 2019-02-04 16:30:07 -08:00
Kitti Nanasi 5f15a60e32 HDFS-14125. Use parameterized log format in ECTopologyVerifier. Contributed by Kitti Nanasi.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-04 14:52:07 -08:00
Inigo Goiri 0e79a86582 HDFS-14202. dfs.disk.balancer.max.disk.throughputInMBperSec property is not working as per set value. Contributed by Ranith Sardar. 2019-02-04 11:59:48 -08:00
Larry McCay 758e9ce781
HADOOP-16076. SPNEGO+SSL Client Connections with HttpClient Broken.
Contributed by Larry McCay
2019-02-04 17:38:40 +00:00
Sunil G 604b2489a9 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 2019-02-04 12:48:28 +05:30
Akira Ajisaka 0f9aa5b4d5
HADOOP-16089. AliyunOSS: update oss-sdk version to 3.4.1. Contributed by wujinhu. 2019-02-04 14:46:35 +09:00
Akira Ajisaka c991e2ca5a
MAPREDUCE-7177. Disable speculative execution in TestDFSIO. Contributed by Zhaohui Xin. 2019-02-04 14:03:34 +09:00
tiwalter 9aa3dc872c
HDFS-14158. Checkpointer ignores configured time period > 5 minutes
This closes #449

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-02-04 13:51:07 +09:00
Takanobu Asanuma ec77e95bbd HDFS-14232. libhdfs is not included in binary tarball. Contributed by Akira Ajisaka. 2019-02-04 10:12:09 +09:00
Mukul Kumar Singh b6f90d3957 HDDS-1021. Allow ozone datanode to participate in a 1 node as well as 3 node Ratis Pipeline. Contributed by Shashikant Banerjee. 2019-02-03 00:53:06 +05:30
Weiwei Yang 69bcff380f YARN-9231. TestDistributedShell fix timeout. Contributed by Prabhu Joseph. 2019-02-02 23:27:31 +08:00
Weiwei Yang f20b043a02 YARN-9263. TestConfigurationNodeAttributesProvider fails after Mockito updated. Contributed by Weiwei Yang. 2019-02-02 23:04:34 +08:00
Weiwei Yang 28ad20a711 YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie. 2019-02-02 23:02:45 +08:00
Vrushali C 2c13513040 YARN-8549 Adding a NoOp timeline writer and reader plugin classes for ATSv2. Contributed by Prabha Manepalli. 2019-02-01 14:26:50 -08:00
Shashikant Banerjee 4123353151 HDDS-1037. Fix the block discard logic in Ozone client. Contributed by Shashikant Banerjee. 2019-02-01 20:44:48 +05:30
Steve Loughran 7f46d13dac
HADOOP-16079. Token.toString faulting if any token listed can't load.
Contributed by Steve Loughran.
2019-02-01 14:31:47 +00:00
Mukul Kumar Singh 13aa939224 HDDS-997. Add blockade Tests for scm isolation and mixed node isolation. Contributed by Nilotpal Nandi. 2019-02-01 13:10:17 +05:30
Yiqun Lin 16195eaee1 HDDS-1025. Handle replication of closed containers in DeadNodeHanlder. Contributed by Bharat Viswanadham. 2019-02-01 11:34:51 +08:00
Kitti Nanasi 0ab7fc9200 HDFS-14187. Make warning message more clear when there are not enough data nodes for EC write. Contributed by Kitti Nanasi.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-31 16:07:24 -08:00
Giovanni Matteo Fumarola f738b397ae YARN-9191. Add cli option in DS to support enforceExecutionType in resource requests. Contributed by Abhishek Modi. 2019-01-31 11:24:15 -08:00
Inigo Goiri bcc3a79f58 HADOOP-16084. Fix the comment for getClass in Configuration. Contributed by Fengnan Li. 2019-01-31 10:06:05 -08:00
Márton Elek 033d97abac HDDS-956. MultipartUpload: List Parts for a Multipart upload key. Contributed by Bharat Viswanadham. 2019-01-31 14:51:31 +01:00
Sunil G 71c49fa60f YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth. 2019-01-31 09:25:29 +05:30
Bharat Viswanadham 53729279c7 HDDS-1035. Intermittent TestRootList failure. Contributed by Doroszlai Attila. 2019-01-30 16:21:42 -08:00
Bharat Viswanadham 945a61c164 HDDS-549. Add support for key rename in Ozone Shell. Contributed by Doroszlai Attila. 2019-01-30 16:00:18 -08:00
Arpit Agarwal c35419579b HDDS-1016. Allow marking containers as unhealthy. Contributed by Arpit Agarwal. 2019-01-30 11:40:50 -08:00
Bharat Viswanadham 7456fc99ee HDDS-1031. Update ratis version to fix a DN restart Bug. Contributed by Bharat Viswanadham. 2019-01-30 11:14:02 -08:00
Márton Elek 0e95ae402c HDDS-1030. Move auditparser robot tests under ozone basic. Contributed by Dinesh Chitlangia. 2019-01-30 17:34:32 +01:00
Akira Ajisaka a3a9ae3cea
YARN-9251. Build failure for -Dhbase.profile=2.0. Contributed by Rohith Sharma K S. 2019-01-30 05:17:04 -08:00
Márton Elek 14441ccbc6 HDDS-1032. Package builds are failing with missing org.mockito:mockito-core dependency version. Contributed by Doroszlai, Attila. 2019-01-30 13:40:45 +01:00
Yiqun Lin d583cc45c6 HDDS-1024. Handle DeleteContainerCommand in the SCMDatanodeProtocolServer. Contributed by Bharat Viswanadham. 2019-01-30 13:56:28 +08:00
Akira Ajisaka 1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Shweta Yakkali 02eb91856e HADOOP-16041. Include Hadoop version in User-Agent string for ABFS. Contributed by Shweta Yakkali.
Signed-off-by: Sean Mackrory <mackrorysd@apache.org>
2019-01-29 16:50:14 -08:00
Hanisha Koneru b3bc94ebfd HDFS-14236. Lazy persist copy/ put fails with ViewFs. 2019-01-29 16:45:44 -08:00
Wei-Chiu Chuang d1714c20e9 Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
This reverts commit 1d523279da.
2019-01-29 15:43:09 -08:00
Eric E Payne 04105bbfdb YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 2019-01-29 17:22:24 +00:00
Shashikant Banerjee 5d578d0c4a HDDS-1004. SCMContainerManager#updateContainerStateInternal fails for QUASI_CLOSE and FORCE_CLOSE events. Contributed by Lokesh Jain. 2019-01-29 14:11:56 +05:30
Weiwei Yang 085f0e8ae7 YARN-9086. [CSI] Run csi-driver-adaptor as aux service. Contributed by Weiwei Yang. 2019-01-29 14:53:08 +08:00
Yiqun Lin 2d06112b74 HDDS-1022. Add cmd type in getCommandResponse in SCMDatanodeProtocolServer. Contributed by Bharat Viswanadham. 2019-01-29 11:26:41 +08:00
Weiwei Yang 4f63ffe444 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 2019-01-29 10:34:26 +08:00