Commit Graph

23329 Commits

Author SHA1 Message Date
Abhishek Modi a2dadac790 YARN-9990. Testcase fails with Insufficient configured threads: required=16 < max=10. Contributed by Prabhu Joseph. 2019-11-29 11:54:33 +05:30
Surendra Singh Lilhore 44f7b9159d HDFS-15013. Reduce NameNode overview tab response time. Contributed by HuangTao. 2019-11-29 00:42:31 +05:30
Surendra Singh Lilhore 0384687811 HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore. 2019-11-28 23:49:35 +05:30
Ayush Saxena 46166bd8d1 HDFS-14961. [SBN read] Prevent ZKFC changing Observer Namenode state. Contributed by Ayush Saxena. 2019-11-28 18:28:20 +05:30
Yiqun Lin c3659f8f94 HDFS-15019. Refactor the unit test of TestDeadNodeDetection. Contributed by Lisheng Sun. 2019-11-28 14:41:49 +08:00
prabhujoseph 013c6e51b6 MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure.
Contributed by Wilfred Spiegelenburg.
2019-11-28 10:57:15 +05:30
Yiqun Lin 2b452b4e60 HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago. 2019-11-28 10:43:35 +08:00
Sneha Vijayarajan 82ad9b549f
HADOOP-16660. ABFS: Make RetryCount in ExponentialRetryPolicy Configurable.
Contributed by Sneha Vijayarajan.
2019-11-27 15:10:21 -08:00
bilaharith 9e69628f55 HADOOP-16455. ABFS: Implement FileSystem.access() method.
Contributed by Bilahari T H.
2019-11-27 15:56:38 +00:00
prabhujoseph 7f2ea2ac46 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER.
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.
2019-11-27 14:27:27 +05:30
Yiqun Lin c8bef4d6a6 HDFS-14649. Add suspect probe for DeadNodeDetector. Contributed by Lisheng Sun. 2019-11-27 10:57:20 +08:00
Szilard Nemeth ef950b0863 YARN-9290. Invalid SchedulingRequest not rejected in Scheduler PlacementConstraintsHandler. Contributed by Prabhu Joseph 2019-11-26 22:04:07 +01:00
Szilard Nemeth 828ab400ee YARN-9362. Code cleanup in TestNMLeveldbStateStoreService. Contributed by Denes Gerencser 2019-11-26 21:45:12 +01:00
prabhujoseph aa7ab2719f YARN-9991. Fix Application Tag prefix to userid. Contributed by Szilard Nemeth. 2019-11-27 01:59:12 +05:30
Szilard Nemeth 8c9018d5c7 YARN-9899. Migration tool that help to generate CS config based on FS config [Phase 2]. Contributed by Peter Bacsko 2019-11-26 21:22:35 +01:00
Sahil Takiar 3161813482 HADOOP-16685: FileSystem#listStatusIterator does not check if given path exists (#1695) 2019-11-26 11:11:26 -08:00
Szilard Nemeth 52e9ee39a1 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 2019-11-26 16:42:59 +01:00
Gabor Bota ea25f4de23
HADOOP-16709. S3Guard: Make authoritative mode exclusive for metadata - don't check for expiry for authoritative paths (#1721). Contributed by Gabor Bota. 2019-11-26 16:36:19 +01:00
Szilard Nemeth 448ffb12ec YARN-9937. addendum: Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph 2019-11-26 13:41:41 +01:00
Eric Badger 6c20512b68 Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger"
This reverts commit 289bbca870.
2019-11-25 17:46:42 +00:00
HUAN-PING SU f8e36e03b4 YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 2019-11-25 15:28:53 +09:00
Akira Ajisaka da9435182a
HADOOP-16717. Remove GenericsUtil isLog4jLogger dependency on Log4jLoggerAdapter. Contributed by Xieming Li. 2019-11-25 14:36:27 +09:00
Ayush Saxena 049940e77b HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar. 2019-11-23 02:30:02 +05:30
Szilard Nemeth 4c1a1287bc YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 12:58:10 +01:00
Tao Yang 8afabe41e4 YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:06:14 +08:00
Ayush Saxena c422e36397 HDFS-13842. RBF: Exceptions are conflicting when creating the same mount entry twice. Contributed by Ranith Sardar. 2019-11-22 11:42:19 +05:30
Takanobu Asanuma b25e94ce29 HDFS-14924. RenameSnapshot not updating new modification time. Contributed by hemanthboyina 2019-11-22 13:48:02 +09:00
Yiqun Lin 9b6906fe91 HDFS-14651. DeadNodeDetector checks dead node periodically. Contributed by Lisheng Sun. 2019-11-22 10:53:55 +08:00
Ayush Saxena b89fd4dfe9 HDFS-15002. RBF: Fix annotation in RouterAdmin. Contributed by Jinglun. 2019-11-22 01:08:11 +05:30
Surendra Singh Lilhore 26270196a2 HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina. 2019-11-22 00:31:25 +05:30
Ayush Saxena 98d249dcda HDFS-14996. RBF: GetFileStatus fails for directory with EC policy set in case of multiple destinations. Contributed by Ayush Saxena. 2019-11-21 12:08:42 +05:30
Sneha Vijayarajan de38045021 HADOOP-16687. ABFS: Fix testcase added for HADOOP-16138 for namespace enabled account. (#1701) 2019-11-21 11:24:12 +09:00
Siyao Meng 1a0c0e42e7 HADOOP-16276. Fix jsvc startup command in hadoop-functions.sh due to jsvc >= 1.0.11 changed default current working directory (#1272) 2019-11-20 16:44:44 -08:00
David Mollitor 6f899e9030 HADOOP-15852. Refactor QuotaUsage. Contributed by David Mollitor.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-11-20 14:40:10 -08:00
Wei-Chiu Chuang 9664b9c7a6 HADOOP-15686. Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr. Contributed by Wei-Chiu Chuang.
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
2019-11-20 13:07:43 -08:00
Inigo Goiri 3037762b2c HDFS-14949. Add getServerDefaults() support to HttpFS. Contributed by hemanthboyina. 2019-11-20 10:51:58 -08:00
Surendra Singh Lilhore fd264b8265 HDFS-14995. Use log variable directly instead of passing as argument in InvalidateBlocks#printBlockDeletionTime(). Contributed by Lisheng Sun. 2019-11-21 00:00:42 +05:30
Erik Krogen be77231452 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq. 2019-11-20 07:56:33 -08:00
Steve Loughran 1415ad3800
HADOOP-16707. NPE in UGI.getCurrentUser in ITestAbfsIdentityTransformer setup.
Contributed by Steve Loughran.

Change-Id: I38fdba2fa70e534d78b15e61de19368912588b0c
2019-11-20 14:32:12 +00:00
Wanqiang Ji dfc61d8ea5 HADOOP-16719. Remove the disallowed element config within maven-checkstyle-plugin (#1724) 2019-11-20 16:18:56 +09:00
He Xiaoqiao 0b50aa29fd HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
2019-11-19 12:16:57 -08:00
Szilard Nemeth dfdc6d6dd9 YARN-9886. Queue mapping based on userid passed through application tag. Contributed by Julia Kinga Marton 2019-11-19 21:14:15 +01:00
Eric Yang ea8ffac121 YARN-9983. Fixed typo in YARN Service overview.
Contributed by Denes Gerencser
2019-11-19 14:16:51 -05:00
Surendra Singh Lilhore c870514740 HDFS-14992. TestOfflineEditsViewer is failing in Trunk. Contributed by hemanthboyina. 2019-11-20 00:07:43 +05:30
Jeetesh Mangwani b033c681e4
HADOOP-16612. Track Azure Blob File System client-perceived latency
Contributed by Jeetesh Mangwani.

This add the ability to track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency in the Hadoop ABFS driver.
The latency information is sent back to the ADLS Gen 2 REST API endpoints in the subsequent requests.
2019-11-19 09:00:24 -08:00
Steve Loughran 9fbfe6c8f9
HADOOP-16632 Speculating & Partitioned S3A magic committers can leave pending files under __magic (#1599)
Contributed by Steve Loughran.

This downgrade the checks for leftover __magic entries from fail to warn now the parallel
test runs make speculation more likely. 

Change-Id: Ia4df2e90f82a06dbae69f3fdaadcbb0e0d713b38
2019-11-19 13:54:33 +00:00
Prabhu Joseph 30b2096933 YARN-9986. Allow app owner to signal container via RMWebServices. Contributed by kyungwan nam. 2019-11-19 11:35:00 +00:00
Duo Zhang 3cecb2a469 HADOOP-16110 Upgrade to yetus 0.11.1 and use emoji vote on github pre commit (#1527). Contributed by Duo Zhang.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2019-11-19 14:21:49 +05:30
Sunil G ea68756c0c YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg. 2019-11-19 14:10:41 +05:30
Sunil G 215f2052fc YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg. 2019-11-19 14:03:02 +05:30