Commit Graph

22912 Commits

Author SHA1 Message Date
Vivek Ratnavel Subramanian 3b9584d12b HDDS-2087. Remove the hard coded config key in ChunkManager (#1409) 2019-09-08 20:44:07 -07:00
Jonathan Hung 43e389b980 YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar 2019-09-08 19:07:11 -07:00
Bharat Viswanadham ca32917e0f
HDDS-2100. Ozone TokenRenewer provider is incorrectly configured. Contributed by Jitendra Nath Pandey. 2019-09-07 12:55:35 -07:00
Abhishek Modi 162af6f554 YARN-9784. org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue is flaky. Contributed by Julia Kinga Marton. 2019-09-07 18:08:04 +05:30
Abhishek Modi e8ca4fac53 YARN-9812. mvn javadoc:javadoc fails in hadoop-sls. Contributed by Abhishek Modi. 2019-09-07 12:44:40 +05:30
Wangda Tan 34b82e6da0 YARN-9813. RM does not start on JDK11 when UIv2 is enabled. (Adam Antal/Eric Yang via wangda)
Change-Id: I18b8edc930b2efa0652f59c246931ad0d46827f3
2019-09-06 19:18:13 -07:00
Sammi Chen c46d43ab13 HDDS-1553. Add metrics in rack aware container placement policy. (#1361) 2019-09-06 17:11:54 -07:00
Tao Yang 73575701ab YARN-9795. ClusterMetrics to include AM allocation delay. Contributed by Fengnan Li. 2019-09-07 07:52:39 +08:00
Tao Yang 5d497abe21 YARN-9817. Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero. Contributed by Prabhu Joseph. 2019-09-07 07:39:02 +08:00
Jungtaek Lim (HeartSaVioR) bb0b922a71
HADOOP-16255. Add ChecksumFs.rename(path, path, boolean)
Contributed by Jungtaek Lim

Change-Id: If00a4d7d30456c08eb2b0f7e2b242197bc4ee05d
2019-09-06 21:53:00 +01:00
dchitlangia b15c116c1e HDDS-2015. Encrypt/decrypt key using symmetric key while writing/reading
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-06 11:35:00 -07:00
Jonathan Hung b71a7f195c YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar 2019-09-06 11:00:07 -07:00
Erik Krogen a23417533e HADOOP-16531. Log more timing information for slow RPCs. Contributed by Chen Zhang. 2019-09-06 10:28:21 -07:00
Erik Krogen 9637097ef9 HDFS-14817. [Dynamometer] Fix start script options parsing which incorrectly interpret options starting with h as a help argument. Contributed by Soya Miyoshi. 2019-09-06 10:24:51 -07:00
Erik Krogen c92a3e94d8 HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun. 2019-09-06 10:22:28 -07:00
Jonathan Hung e7d44e48f7 YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar 2019-09-06 10:17:07 -07:00
Erik Krogen ae42c8cb61 HDFS-4819. [Dynamometer] Fix parsing of audit logs which contain = in path names. Contributed by Soya Miyoshi. 2019-09-06 10:13:26 -07:00
Surendra Singh Lilhore d98c54816d HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green. 2019-09-06 18:12:42 +05:30
Surendra Singh Lilhore fa7f03fc56 HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C. 2019-09-06 17:55:30 +05:30
Lokesh Jain 6e4cdf89ef HDDS-1561: Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove (#1401) 2019-09-06 13:15:49 +05:30
Inigo Goiri 494d75eb2b HDFS-14784. Add more methods to WebHdfsTestUtil to support tests outside of package. Contributed by Chen Zhang. 2019-09-05 21:15:17 -07:00
Anu Engineer acbea8d976 HDDS-1708. Add container scrubber metrics.
Contributed by Hrishikesh Gadre.
2019-09-05 14:33:06 -07:00
Jonathan Hung 0ccf4b0fe1 YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta 2019-09-05 14:01:42 -07:00
Billie Rinaldi 2e2e5401f2 YARN-9718. Fixed yarn.service.am.java.opts shell injection. Contributed by Eric Yang 2019-09-05 12:49:58 -07:00
Inigo Goiri 72d8b92ba5 HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun. 2019-09-05 11:44:02 -07:00
Steve Loughran 511df1e837 HADOOP-16430. S3AFilesystem.delete to incrementally update s3guard with deletions
Contributed by Steve Loughran.

This overlaps the scanning for directory entries with batched calls to S3 DELETE and updates of the S3Guard tables.
It also uses S3Guard to list the files to delete, so find newly created files even when S3 listings are not use consistent.

For path which the client considers S3Guard to be authoritative, we also do a recursive LIST of the store and delete files; this is to find unindexed files and do guarantee that the delete(path, true) call really does delete everything underneath.

Change-Id: Ice2f6e940c506e0b3a78fa534a99721b1698708e
2019-09-05 14:25:15 +01:00
Nanda kumar 2b16d5377c
HDDS-1898. GrpcReplicationService#download cannot replicate the container. (#1326) 2019-09-05 18:12:36 +05:30
Tao Yang 172bcd8e01 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. Contributed by zhuqi. 2019-09-05 16:20:05 +08:00
Hrishikesh Gadre f347c348d8 Add support for checksum verification in data scrubber
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-04 20:31:49 -07:00
Rohith Sharma K S 3db7184082 YARN-9804. Update ATSv2 document for latest feature supports. 2019-09-05 08:42:17 +05:30
Ayush Saxena 2f70b52a5b HDFS-14812. RBF: MountTableRefresherService should load cache when refresh. Contributed by xuzq. 2019-09-05 08:29:56 +05:30
Xiaoyu Yao ae287474c0
HDDS-2079. Fix TestSecureOzoneManager. Contributed by Xiaoyu Yao. (#1400) 2019-09-04 14:03:38 -07:00
Erik Krogen 337e9b794d HADOOP-16268. Allow StandbyException to be thrown as CallQueueOverflowException when RPC call queue is filled. Contributed by CR Hota. 2019-09-04 08:22:02 -07:00
Nanda kumar 1ae775975b
HDDS-2077. Add maven-gpg-plugin.version to pom.ozone.xml. (#1396) 2019-09-04 15:28:59 +05:30
Surendra Singh Lilhore 05704754a0 HDFS-14777. RBF: Set ReadOnly is failing for mount Table but actually readonly succeed to set. Contributed by Ranith Sardar. 2019-09-04 15:09:09 +05:30
Bharat Viswanadham cfa41a49af
HDDS-2018. Handle Set DtService of token for OM HA. (#1371) 2019-09-03 14:06:14 -07:00
Bharat Viswanadham f25fe92743
HDDS-1909. Use new HA code for Non-HA in OM. (#1225) 2019-09-03 13:24:32 -07:00
Xiaoyu Yao 3c117163a3
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun. 2019-09-03 09:29:58 -07:00
Ayush Saxena 91b01a1db7 HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena. 2019-09-03 18:08:42 +05:30
Sammi Chen 0f549ec02a HDDS-2069. Default values of properties hdds.datanode.storage.utilization.{critical | warning}.threshold are not reasonable. (#1393) 2019-09-03 17:58:48 +05:30
Nanda kumar ac5a0ae6d0
HADOOP-16534. Exclude submarine from hadoop source build. (#1356) 2019-09-03 17:40:38 +05:30
Nanda kumar 0b9704f610
HDDS-1810. SCM command to Activate and Deactivate pipelines. (#1224) 2019-09-03 16:50:57 +05:30
avijayanhwx b53d19a343 HDDS-1783 : Latency metric for applyTransaction in ContainerStateMachine (#1363). 2019-09-03 15:18:50 +05:30
Surendra Singh Lilhore 5ff76cb8bc HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina. 2019-09-03 12:37:09 +05:30
Zhankun Tang bb26514ba9 YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T. 2019-09-03 14:55:15 +08:00
bibinchundatt 03489124ea YARN-9797. LeafQueue#activateApplications should use resourceCalculator#fitsIn. Contributed by Bilwa S T. 2019-09-03 11:53:34 +05:30
Stephen O'Donnell 915cbc91c0 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-02 09:47:04 -07:00
Wei-Chiu Chuang d207aba026 Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell."
This reverts commit 7bebad61d9.
2019-09-02 09:43:44 -07:00
Ayush Saxena 040f6e93bb HDFS-14654. RBF: TestRouterRpc#testNamenodeMetrics is flaky. Contributed by Chen Zhang. 2019-09-02 13:01:52 +05:30
Abhishek Modi f4d6e82075 YARN-9400. Remove unnecessary if at EntityGroupFSTimelineStore#parseApplicationId. Contributed by Prabhu Joseph. 2019-09-02 10:58:23 +05:30