Commit Graph

23205 Commits

Author SHA1 Message Date
Chao Sun 97ec34e117 HDFS-14959: [SBNN read] access time should be turned off (#1706) Contributed by Chao Sun. 2019-11-12 11:35:50 -08:00
Siyao Meng 3ab98eefa5 HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-11-12 11:34:24 -08:00
Inigo Goiri 40150da1e1 HDFS-14922. Prevent snapshot modification time got change on startup. Contributed by hemanthboyina. 2019-11-12 10:53:54 -08:00
Steve Loughran 990063d2af
HADOOP-16665. Filesystems to be closed if they failed during initialize().
Contributed by Steve Loughran.

This FileSystem instantiation so if an IOException or RuntimeException is
raised in the invocation of FileSystem.initialize() then a best-effort
attempt is made to close the FS instance; exceptions raised that there
are swallowed.

The S3AFileSystem is also modified to do its own cleanup if an
IOException is raised during its initialize() process, it being the
FS we know has the "potential" to leak threads, especially in
extension points (e.g AWS Authenticators) which spawn threads.

Change-Id: Ib84073a606c9d53bf53cbfca4629876a03894f04
2019-11-12 18:17:21 +00:00
Yufei Gu b83b9ab418 YARN-9537. Add configuration to disable AM preemption. Contributed by ZhouKang 2019-11-12 09:40:16 -08:00
Steve Loughran f6697aa82b
HADOOP-16477. S3A delegation token tests fail if fs.s3a.encryption.key set.
Contributed by Steve Loughran.

Change-Id: I843989f32472bbdefbd4fa504b26c7a614ab1cee
2019-11-12 15:31:53 +00:00
Abhishek Modi fb512f5087 YARN-9697. Efficient allocation of Opportunistic containers. Contributed by Abhishek Modi. 2019-11-12 16:34:04 +05:30
Masatake Iwasaki a98c4dfc47 HADOOP-16702. Move documentation of hadoop-cos to under src directory.
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-11-12 17:47:17 +09:00
Masatake Iwasaki b988487cae HADOOP-16701. Fix broken links in site index.
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-11-12 16:43:52 +09:00
Szilard Nemeth 30b93f914b YARN-9865. Capacity scheduler: add support for combined %user + %secondary_group mapping. Contributed by Manikandan R 2019-11-11 13:28:02 +01:00
Abhishek Modi 516377bfa6 YARN-9965. Fix NodeManager failing to start when Hdfs Auxillary Jar is set. Contributed by Prabhu Joseph. 2019-11-11 16:01:44 +05:30
Ayush Saxena 77934bc07b HDFS-14975. Add CR for SetECPolicyCommand usage. Contributed by Fei Hui. 2019-11-11 11:05:34 +05:30
Wang Yuxuan b25a37c322 HDFS-14962. RBF: ConnectionPool#newConnection() error log wrong protocol class (#1699). Contributed by Yuxuan Wang. 2019-11-11 10:48:38 +05:30
Takanobu Asanuma 6663d6a5c2 HDFS-14928. UI: unifying the WebUI across different components. Contributed by Xieming Li 2019-11-11 14:04:57 +09:00
Surendra Singh Lilhore 320008bb7c HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina. 2019-11-09 22:14:35 +05:30
Szilard Nemeth 3d249301f4 HADOOP-16683. Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException. Contributed by Adam Antal 2019-11-09 15:43:53 +01:00
Szilard Nemeth ceb9c6175e YARN-9890. [UI2] Add Application tag to the app table and app detail page. Contributed by Julia Kinga Marton 2019-11-09 14:12:04 +01:00
Szilard Nemeth 31f172fd96 YARN-9677. Make FpgaDevice and GpuDevice classes more similar to each other. Contributed by kevin su 2019-11-09 14:04:14 +01:00
Ayush Saxena 42fc8884ab HDFS-14815. RBF: Update the quota in MountTable when calling setQuota on a MountTable src. Contributed by Jinglun. 2019-11-08 08:48:52 +05:30
Ayush Saxena 247584eb63 HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan. 2019-11-07 22:49:01 +05:30
Chen Liang dd900259c4 HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko. 2019-11-06 09:56:19 -08:00
Chen Liang 9d0d580031 HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen. 2019-11-06 09:30:04 -08:00
Ayush Saxena 9e287054a8 HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun. 2019-11-06 22:02:03 +05:30
Ayush Saxena dcf55838ae HDFS-14880. Correct the sequence of statistics & exit message in balencer. Contributed by Renukaprasad C. 2019-11-06 21:45:03 +05:30
Surendra Singh Lilhore c36014165c HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore. 2019-11-06 19:28:55 +05:30
Wei-Chiu Chuang ee8addbec4 Revert "HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng."
This reverts commit f9b99d2f24.
2019-11-05 15:41:52 -08:00
Ahmed Hussein ed302f1fed MAPREDUCE-7208. Tuning TaskRuntimeEstimator. (Ahmed Hussein via jeagles)
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2019-11-05 14:55:20 -06:00
Inigo Goiri bfb8f28cc9 HDFS-14775. Add Timestamp for longest FSN write/read lock held log. Contributed by Chen Zhang. 2019-11-05 09:19:52 -08:00
Takanobu Asanuma d17ba85482 HADOOP-16681. mvn javadoc:javadoc fails in hadoop-aws. Contributed by Xieming Li 2019-11-05 15:24:59 +09:00
Takanobu Asanuma 7901062707 HDFS-14891. RBF: namenode links in NameFederation Health page (federationhealth.html) cannot use https scheme. Contributed by Xieming Li 2019-11-05 15:03:09 +09:00
Ayush Saxena b643a1cbe8 HDFS-14938. Add check if excludedNodes contain scope in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Lisheng Sun. 2019-11-05 09:23:20 +05:30
zhoukang 7d0adddf09 YARN-9689: Support proxy user for Router to support kerberos (#1654) 2019-11-04 17:13:29 -08:00
belugabehr 6f0190d8e4 HADOOP-16678: Review of ArrayWritable (#1692) 2019-11-04 14:24:44 -08:00
Ayush Saxena 2ffec347eb HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui. 2019-11-05 01:37:15 +05:30
Sunil G 51e7d1b37e YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph. 2019-11-04 23:27:49 +05:30
Wei-Chiu Chuang eb73ba6ed5 HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao. 2019-11-04 09:40:42 -08:00
Gabor Bota dca19fc3aa
HADOOP-16484. S3A to warn or fail if S3Guard is disabled (#1661). Contributed by Gabor Bota. 2019-11-04 12:55:36 +01:00
Sunil G d462308e04 YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. Contributed by Prabhu Joseph. 2019-11-03 08:47:27 +05:30
Zhao Junwang de6b8b0c0b HDFS-14925. Rename operation should check nest snapshot (#1670)
If the src directory or any of its descendant is snapshottable
and the dst directory or any of its ancestors is snapshottable,
we consider this as nested snapshot, which should be denied.

Reviewed-by: Shashikant Banerjee <shashikant@apache.org>
2019-11-01 16:37:53 -07:00
Ayush Saxena 7d7acb004a HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong. 2019-11-01 22:50:10 +05:30
Inigo Goiri f18bbdd9d8 HDFS-14927. RBF: Add metrics for async callers thread pool. Contributed by Leon Gao. 2019-11-01 10:14:31 -07:00
Surendra Singh Lilhore 02009c3bb7 HDFS-14768. EC : Busy DN replica should be consider in live replica check. Contributed by guojh. 2019-11-01 22:15:05 +05:30
Takanobu Asanuma 477505ccfc HDFS-14824. [Dynamometer] Dynamometer in org.apache.hadoop.tools does not output the benchmark results. (#1685) 2019-11-01 09:32:32 -07:00
Jonathan Hung ef9d12df24 Add 2.10.0 release notes for HDFS-12943 2019-10-31 20:33:59 -07:00
Siyao Meng f9b99d2f24 HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-31 12:59:46 -07:00
Ayush Saxena 9d25ae7669 HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui. 2019-10-31 23:49:37 +05:30
Ayush Saxena d9fbedc4ae HDFS-14936. Add getNumOfChildren() for interface InnerNode. Contributed by Lisheng Sun. 2019-10-31 23:39:27 +05:30
Szilard Nemeth 8c9173c87c HADOOP-16510. [hadoop-common] Fix order of actual and expected expression in assert statements. Contributed by Adam Antal 2019-10-31 14:35:20 +01:00
Jonathan Hung e6137d0e50 Make upstream aware of 2.10.0 release 2019-10-30 20:58:32 -07:00
Sunil G 9a2e43e29e YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph. 2019-10-31 00:34:27 +05:30