bcd9c67082
HADOOP-16721. Improve S3A rename resilience ( #2742 )
...
The S3A connector's rename() operation now raises FileNotFoundException if
the source doesn't exist; a FileAlreadyExistsException if the destination
exists and is unsuitable for the source file/directory.
When renaming to a path which does not exist, the connector no longer checks
for the destination parent directory existing -instead it simply verifies
that there is no file immediately above the destination path.
This is needed to avoid race conditions with delete() and rename()
calls working on adjacent subdirectories.
Contributed by Steve Loughran.
2021-03-11 12:47:39 +00:00
23b343aed1
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin ( #2753 )
...
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
2021-03-11 10:56:07 +09:00
ebfba0b6fa
[HADOOP-17567] typo in MagicCommitTracker ( #2749 )
...
Contributed by Pierrick Hymbert
2021-03-10 15:39:55 +00:00
7a851c7176
HADOOP-17573. Fix compilation error of OBSFileSystem in trunk. ( #2758 )
2021-03-10 22:23:54 +09:00
6a55baeee4
HDFS-15875. Check whether file is being truncated before truncate ( #2746 )
2021-03-10 14:11:29 +08:00
176bd88890
HADOOP-16080. hadoop-aws does not work with hadoop-client-api. ( #2522 )
...
Contributed by Chao Sun.
(Cherry-picked via PR #2575 )
2021-03-09 20:01:29 +00:00
b2a565629d
YARN-10671.Fix Typo in TestSchedulingRequestContainerAllocation. Contributed by D M Murali Krishna Reddy.
2021-03-09 20:26:07 +05:30
d5e035dbe1
YARN-9615. Add dispatcher metrics to RM. Contributed by Jonathan Hung and Qi Zhu.
2021-03-09 14:33:14 +01:00
3851994cd6
Revert "YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu."
...
This reverts commit 369f75b7a7
2021-03-09 14:32:02 +01:00
369f75b7a7
YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu.
2021-03-09 14:28:23 +01:00
c3aa413ee3
YARN-10679. Better logging of uncaught exceptions throughout SLS. Contributed by Szilard Nemeth.
2021-03-09 14:02:12 +01:00
099f58f8f4
YARN-10681. Fix assertion failure message in BaseSLSRunnerTest. Contributed by Szilard Nemeth.
2021-03-09 13:22:48 +01:00
7f522c92fa
YARN-10677. Logger of SLSFairScheduler is provided with the wrong class. Contributed by Szilard Nemeth.
2021-03-09 12:53:32 +01:00
ea90cd3556
YARN-10678. Try blocks without catch blocks in SLS scheduler classes can swallow other exceptions. Contributed by Szilard Nemeth.
2021-03-09 12:03:53 +01:00
20416bc62d
YARN-10675. Consolidate YARN-10672 and YARN-10447. Contributed by Szilard Nemeth.
2021-03-09 11:34:37 +01:00
e472ee2aa5
YARN-10676. Improve code quality in TestTimelineAuthenticationFilterForV1. Contributed by Szilard Nemeth.
2021-03-09 11:28:24 +01:00
63aa142b77
HADOOP-17557. skip-dir option is not processed by Yetus. Contributed by Ahmed Hussein.
2021-03-09 11:10:10 +09:00
6a922f9896
HDFS-15873. Add namenode address in logs for block report ( #2743 )
2021-03-08 11:50:51 -05:00
04cd3115ba
YARN-10658. CapacityScheduler QueueInfo add queue path field to avoid ambiguous QueueName. Contributed by Qi Zhu.
2021-03-08 12:29:44 +01:00
4a0b7f7ebe
HDFS-15384. Document getLocatedBlocks(String src, long start) of DFSClient only return partial blocks. Contributed by Yang Yun.
2021-03-08 12:18:46 +08:00
9cb51bf106
HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time. ( #2668 ) Contributed by tomscut.
2021-03-06 13:40:44 -08:00
ef7ab535c5
HADOOP-17552. Change ipc.client.rpc-timeout.ms from 0 to 120000 by default to avoid potential hang. ( #2727 )
2021-03-06 22:26:16 +09:00
c6b30a59dd
YARN-10672. All testcases in TestReservations are flaky. Contributed By Szilard Nemeth.
2021-03-05 23:13:35 +01:00
9d088639bf
YARN-10664. Allow parameter expansion in NM_ADMIN_USER_ENV. Contributed by Jim
...
Brennan.
2021-03-05 19:46:40 +00:00
e82e7c597a
YARN-10640. Adjust the queue Configured capacity to Configured weight number for weight mode in UI. Contributed by Qi Zhu.
2021-03-05 15:56:56 +01:00
0774116756
HADOOP-17563. Update Bouncy Castle to 1.68. ( #2740 )
2021-03-05 22:56:51 +09:00
e7cad3811f
YARN-10642. Race condition: AsyncDispatcher can get stuck by the changes introduced in YARN-8995. Contributed by zhengchenyu.
2021-03-05 13:50:45 +01:00
e19c00925f
YARN-10639. Queueinfo related capacity, should adjusted to weight mode. Contributed by Qi Zhu.
2021-03-05 13:18:06 +01:00
6699198b54
YARN-10532. Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used. Contributed by Qi Zhu.
2021-03-04 17:18:35 +01:00
a85aeee876
YARN-10623. Capacity scheduler should support refresh queue automatically by a thread policy. Contributed by Qi Zhu.
2021-03-04 12:23:11 +01:00
d615e2d3bd
YARN-10649. Fix RMNodeImpl.updateExistContainers leak ( #2719 ). Contributed by Max Xie
2021-03-04 14:52:58 +05:30
9fd2198daa
HADOOP-17546. Update Description of hadoop-http-auth-signature-secret in HttpAuthentication.md. Contributed by Ravuri Sushma sree.
2021-03-04 14:55:37 +09:00
7e8040e6ad
YARN-10655. Limit queue creation depth relative to its first static parent. Contributed by Andras Gyori.
2021-03-03 17:44:30 +01:00
cdba06e380
HDFS-15870. Remove unused configuration dfs.namenode.stripe.min ( #2739 )
...
Co-authored-by: tomscut <litao@bigo.sg>
2021-03-03 22:34:02 +09:00
8af56de1fa
HADOOP-17560. Fix some spelling errors ( #2730 )
...
Co-authored-by: jiaguodong5 <jiaguodong5@jd.com>
2021-03-03 11:41:05 +09:00
a17dc4e821
[MAPREDUCE-7234] ClientHSSecurityInfo class is in wrong META-INF file.
...
Contributed by Eric Badger.
2021-03-02 21:47:31 +00:00
1f1a1ef52d
HDFS-15856: Make write pipeline retry times configurable. ( #2721 ). Contributed by Qi Zhu
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-03-02 13:16:11 +08:00
9501c698f4
HDFS-15849. ExpiredHeartbeats metric should be of Type.COUNTER. Contributed by Qi Zhu.
2021-03-01 17:17:23 -08:00
32353eb38a
HDFS-15854. Make some parameters configurable for SlowDiskTracker and SlowPeerTracker ( #2718 )
...
Authored-by: tomscut <litao@bigo.sg>
2021-03-01 07:52:59 -08:00
c3b3b36dee
HDFS-14013. Skip any credentials stored in HDFS when starting ZKFC. Contributed by Stephen O'Donnell
2021-03-01 11:36:41 +00:00
3101c39b49
YARN-10656. Parsing error in CapacityScheduler.md ( #2725 )
...
Reviewed-by: Peter Bacsko <pbacsko@apache.org>
2021-02-27 12:46:36 +09:00
947b50489d
YARN-10627. Extend logging to give more information about weight mode. Contributed by Benjamin Teke.
2021-02-26 21:57:54 +01:00
e04bcb3a06
MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase ( #2722 ). Contributed by Ahmed Hussein
2021-02-26 13:42:33 -06:00
fcdb1e7fec
YARN-10653. Fixed the findbugs issues introduced by YARN-10647. Contributed by
...
Qi Zhu
2021-02-26 19:03:59 +00:00
ac37ca0b67
YARN-10651. CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource(). Contributed by Haibo Chen
2021-02-25 16:01:18 -08:00
c373da9f88
[YARN-10613] Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF. Contributed by Eric Payne
2021-02-25 16:46:19 +00:00
a89c624002
YARN-10609. Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue). Contributed by Qi Zhu.
2021-02-24 20:41:35 +01:00
9c583f36d5
HDFS-15422. Reported IBR is partially replaced with stored info when queuing. Contributed by Stephen O'Donnell and Kihwal Lee
2021-02-24 10:29:36 +00:00
924fa87010
HADOOP-17510. Hadoop prints sensitive Cookie information. ( #2673 )
2021-02-24 01:29:25 -08:00
940c780feb
HDFS-15781. Add metrics for how blocks are moved in replaceBlock. ( #2704 )
2021-02-23 22:29:48 -08:00
Steve Loughran
Akira Ajisaka
Pierrick Hymbert
Masatake Iwasaki
Hui Fei
Chao Sun
Brahma Reddy Battula
Peter Bacsko
litao
He Xiaoqiao
tomscut
Haoze Wu
Eric Badger
Takanobu Asanuma
Neil
litao
JJiaguodong
zhuqi
Konstantin V Shvachko
S O'Donnell
Ahmed Hussein
Jonathan Hung
Jim Brennan
Renukaprasad C
LeonGao