Commit Graph

23633 Commits

Author SHA1 Message Date
Ayush Saxena f8644fbe9f HDFS-15091. Cache Admin and Quota Commands Should Check SuperUser Before Taking Lock. Contributed by Ayush Saxena. 2020-01-04 19:02:59 +05:30
Masatake Iwasaki 037ec8cfb1 HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago.
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2020-01-04 01:55:36 +09:00
Rajesh Balamohan b19d87c2b7
HADOOP-16751. DurationInfo text parsing/formatting should be moved out of hotpath.
Contributed by Rajesh Balamohan

Change-Id: Icc3dcfa81aa69164f2c088f9b533d231138cbb8b
2020-01-02 17:03:07 +00:00
Ayush Saxena 1b04bcc0d9 HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena. 2020-01-02 21:42:36 +05:30
Steve Loughran 958764479d
HADOOP-16777. Add Tez to LimitedPrivate of ClusterStorageCapacityExceededException
Contributed by Wang Yan.

Change-Id: I92dfe7079ba8ebe89d70255bb845309be0603a8e
2020-01-02 15:49:42 +00:00
Steve Loughran b6dc00f481
HADOOP-16775. DistCp reuses the same temp file within the task for different files.
Contributed by Amir Shenavandeh.

This avoids overwrite consistency issues with S3 and other stores -though
given S3's copy operation is O(data), you are still best of using -direct
when distcp-ing to it.

Change-Id: I8dc9f048ad0cc57ff01543b849da1ce4eaadf8c3
2020-01-02 15:36:33 +00:00
Prabhu Joseph eca7e14c2f YARN-10053. Use Shared Group Mapping Service in Placement Rules.
Contributed by Wilfred Spiegelenburg.
2020-01-02 14:13:57 +05:30
Prabhu Joseph 21ada4d1b0 Revert "YARN-10053. Use Shared Group Mapping Service in Placement Rules."
This reverts commit 217b56ffdd.
2020-01-02 14:12:43 +05:30
Prabhu Joseph 217b56ffdd YARN-10053. Use Shared Group Mapping Service in Placement Rules.
Contributed by Wilfred Spiegelenburg.
2020-01-02 14:07:49 +05:30
Rakesh Radhakrishnan d79cce20ab HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He. 2020-01-02 11:44:00 +05:30
Takanobu Asanuma 074050ca59 HDFS-15063. HttpFS: getFileStatus doesn't return ecPolicy. Contributed by hemanthboyina. 2020-01-01 11:26:38 +09:00
Ayush Saxena 62423910a4 HDFS-14937. [SBN read] ObserverReadProxyProvider should throw InterruptException. Contributed by xuzq. 2019-12-29 13:07:22 +05:30
Surendra Singh Lilhore ee51eadda0 HDFS-15074. DataNode.DataTransfer thread should catch all the expception and log it. Contributed by hemanthboyina. 2019-12-29 11:15:54 +05:30
Takanobu Asanuma dc32f583af HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena. 2019-12-28 21:32:15 +09:00
Ayush Saxena 926d0b48f0 HDFS-15081. Typo in RetryCache#waitForCompletion annotation. Contributed by Fei Hui. 2019-12-27 18:32:15 +05:30
Liu sheng 0fed874adf YARN-10041. Should not use AbstractPath to create unix domain socket (#1771) 2019-12-27 16:50:15 +05:30
Chandra Sanivarapu d80d86264d HADOOP-16042. Update the link to HadoopJavaVersion (#1784) 2019-12-27 17:18:10 +09:00
Ayush Saxena 8730a7bf60 HDFS-15003. RBF: Make Router support storage type quota. Contributed by Jinglun. 2019-12-27 09:22:58 +05:30
Ayush Saxena 80f91d14ab HDFS-14997. Addendum: BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He. 2019-12-27 09:12:44 +05:30
Ayush Saxena 300505c562 HDFS-15054. Delete Snapshot not updating new modification time. Contributed by hemanthboyina. 2019-12-25 12:42:50 +05:30
Ayush Saxena df622cf4a3 HDFS-12999. When reach the end of the block group, it may not need to flush all the data packets(flushAllInternals) twice. Contributed by lufei and Fei Hui. 2019-12-25 11:07:25 +05:30
Akira Ajisaka 40887c9b12
YARN-10057. Upgrade the dependencies managed by yarnpkg. (#1780) 2019-12-25 14:10:44 +09:00
Chandra Sanivarapu d8cd7098b4 HDFS-15073. Replace curator-shaded guava import with the standard one (#1782) 2019-12-25 10:50:01 +09:00
Konstantin V Shvachko b98ac2a3af HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko. 2019-12-24 12:08:34 -08:00
Akira Ajisaka 34ff7dbaf5
YARN-10055. bower install fails. (#1778) 2019-12-23 19:45:47 +09:00
Akira Ajisaka c44943d1fc
YARN-10054. Upgrade yarn to 1.21.1 in Dockerfile. (#1777) 2019-12-23 14:08:14 +09:00
Akira Ajisaka bc825b9628
YARN-10036. Install yarnpkg and upgrade nodejs in Dockerfile (#1772) 2019-12-23 10:30:26 +09:00
Vinayakumar B e2a5448d2b
HADOOP-16774. TestDiskChecker and TestReadWriteDiskValidator fails when run with -Pparallel-tests (#1776). Contributed by Vinayakumar B. 2019-12-21 00:08:15 +05:30
Eric Badger 412035b47a YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne. 2019-12-20 18:07:00 +00:00
Akira Ajisaka f777cd398f
HADOOP-16771. Update checkstyle to 8.26 and maven-checkstyle-plugin to 3.1.0. Contributed by Andras Bokor. 2019-12-20 13:10:26 +09:00
Zhankun Tang 12722ab0c7 YARN-10042. Upgrade grpc-xxx depdencies to 1.26.0. Contributed by Sheng Liu. 2019-12-20 11:10:27 +08:00
Giovanni Matteo Fumarola ef59ffd362 YARN-10038. [UI] Finish Time is not correctly parsed in the RM Apps page. Contributed by Inigo Goiri. 2019-12-19 11:37:17 -08:00
Inigo Goiri 52d7b745c6 HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui. 2019-12-19 09:42:17 -08:00
Inigo Goiri b86895485d HDFS-14997. BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He. 2019-12-19 09:34:43 -08:00
Masatake Iwasaki 7868da894a YARN-10037. Upgrade build tools for YARN Web UI v2. 2019-12-19 19:27:31 +09:00
Jonathan Hung 7b93575b92 YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne 2019-12-18 13:18:11 -08:00
Jonathan Hung fddc3d55c3 YARN-10039. Allow disabling app submission from REST endpoints 2019-12-18 10:48:05 -08:00
Shashikant Banerjee fdd96e46d1 HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee. 2019-12-18 22:50:46 +05:30
Masatake Iwasaki 92c8962f98
YARN-10020. Fix build instruction of hadoop-yarn-ui. (#1750) 2019-12-18 13:34:12 +09:00
Eric Badger f47dcf2d4c YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne. 2019-12-17 17:11:23 +00:00
Inigo Goiri 24080666e5 HDFS-14908. LeaseManager should check parent-child relationship when filter open files. Contributed by Jinglun. 2019-12-16 18:41:45 -08:00
Mate Szalay-Beko 578bd101a6 HADOOP-16765. Fix curator dependencies for gradle projects using hadoop-minicluster. (#1763)
Gradle behaves differently than Maven when processing exclude rules through transitive
dependencies. This causes gradle projects (e.g. in Kudu) that are using Hadoop MiniCluster
to fail because different Guava version come to the classpath through Curator. In this
fix we make sure that ZooKeeper, SLF4J and Gradle gets always excluded from Curator Client
even if they would come through other Curator dependencies transitively.
2019-12-16 14:43:59 -08:00
Steve Loughran 382151670b HADOOP-16450. ITestS3ACommitterFactory to not use useInconsistentClient. (#1145)
Contributed by Steve Loughran.

Change-Id: Ifb9771a73a07f744e4ed5f5e6be72473179db439
2019-12-16 14:29:30 +01:00
Masatake Iwasaki dc6cf17b34 HDFS-15048. Fix findbug in DirectoryScanner. 2019-12-16 19:00:57 +09:00
Ayush Saxena ab32762f43 HADOOP-16755. Fix broken link in single node cluster setup documentation. Contributed by Denes Gerencser. 2019-12-16 08:44:28 +05:30
Kengo Seki fd7de2b82a HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 2019-12-16 11:04:20 +09:00
Szilard Nemeth 631dbbc6f2 YARN-9923. Introduce HealthReporter interface to support multiple health checker files. Contributed by Adam Antal 2019-12-15 17:28:04 +01:00
Ayush Saxena 7a87007545 HDFS-15038. TestFsck testFsckListCorruptSnapshotFiles is failing in trunk. Contributed by hemanthboyina. 2019-12-15 00:33:32 +05:30
Ayush Saxena 72aee114f8 HDFS-15053. RBF: Add permission check for safemode operation. Contributed by Xiaoqiao He. 2019-12-14 17:57:23 +05:30
Inigo Goiri 7fe924b1c0 HDFS-15016. RBF: getDatanodeReport() should return the latest update. Contributed by Inigo Goiri. 2019-12-13 10:51:14 -08:00