Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,435 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

21435 Commits

Author SHA1 Message Date
Chao Sun d82b0cc439
HADOOP-16080. hadoop-aws does not work with hadoop-client-api (#2510). Contributed by Chao Sun 2020-12-03 16:06:33 -08:00
Ahmed Hussein f547cd43d1 HDFS-15706. HttpFS: Log more information on request failures. (#2515) 
(cherry picked from commit 07655a7040)
(cherry picked from commit a7dbd3b6f9)
 2020-12-03 15:15:31 -08:00
Jim Brennan 9fe779d0e2 HADOOP-17392. Remote exception messages should not include the exception class (#2486). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit 5bfb97bc7d)
 2020-12-03 18:08:37 +00:00
Steve Loughran 3efd993a8d
HADOOP-17343. Upgrade AWS SDK to 1.11.901 (#2468) 
Contributed by Steve Loughran.

Change-Id: I8ea1e248f9b145471118ea729a6a67e40acea6ff
 2020-12-03 17:10:42 +00:00
Andrea Scarpino 66e8f1ae09
YARN-10511. Update yarn.nodemanager.env-whitelist value in docs (#2512) 
Reviewed-by: Adam Antal <adamantal@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9170eb566b)
 2020-12-04 00:17:48 +09:00
Ahmed Hussein 91310102d7 HDFS-15703. Don't generate edits for set operations that are no-op (#2508). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit 60201cbf69)
 2020-12-02 21:52:52 +00:00
Ahmed Hussein 6cc0824072 HDFS-15695. NN should not let the balancer run in safemode (#2489). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit 2b5b556dd7)
 2020-12-02 20:31:04 +00:00
Eric Payne 1184284baf YARN-10278: CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework. Contributed by Szilard Nemeth (snemeth) 2020-12-02 17:22:49 +00:00
Ahmed Hussein 625f85f40f HDFS-15694. Avoid calling UpdateHeartBeatState inside DataNodeDescriptor. (#2487) Contributed by Kuhu Shukla and Ahmed Hussein 
(cherry picked from commit 918ba9e82f)
 2020-12-01 21:31:18 +00:00
kevinzhao1661 7e89bd5e42
YARN-10498. Fix typo in CapacityScheduler Markdown document (#2484) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4d2ae5b398)
 2020-11-30 11:18:41 +09:00
Steve Loughran 091d538c19
HADOOP-17325. WASB Test Failures 
Contributed by Ayush Saxena and Steve Loughran

Change-Id: I4bb76815bc1d11d1804dc67bafde68b6a995b974
 2020-11-28 21:49:12 +08:00
Peter Bacsko d727ab4906 MAPREDUCE-7309. Improve performance of reading resource request for mapper/reducers from config. Contributed by Peter Bacsko & Wangda Tan. 2020-11-25 11:44:48 +01:00
Akira Ajisaka 752c890f9d
YARN-10470. When building new web ui with root user, the bower install should support it. Contributed by zhuqi. 
(cherry picked from commit c4ba0ab7df)
 2020-11-24 15:23:13 +09:00
Eric Payne 56ecf469f8 HADOOP-17346. Fair call queue is defeated by abusive service principals. Contributed by Ahmed Hussein (ahussein). 2020-11-23 20:48:07 +00:00
Peter Bacsko 82db198d24 MAPREDUCE-7304. Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class. Contributed by Zoltan Erdmann 2020-11-20 13:15:47 +01:00
Jim Brennan 6b3d65b46e HADOOP-17367. Add InetAddress api to ProxyUsers.authorize (#2449). Contributed by Daryn Sharp and Ahmed Hussein 2020-11-19 21:55:28 +00:00
Ahmed Hussein d2ae8bcc66 HADOOP-17360. Log the remote address for authentication success (#2441) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 1ea3f74246)
 2020-11-16 22:00:19 +00:00
Ahmed Hussein c107e75424 YARN-10485. TimelineConnector swallows InterruptedException (#2450). Contributed by Ahmed Hussein 
(cherry picked from commit 0b2510ee1f)
 2020-11-16 21:21:58 +00:00
Peter Bacsko c5ae78b793 YARN-10396. Max applications calculation per queue disregards queue level settings in absolute mode. Contributed by Benjamin Teke. 2020-11-16 11:48:50 +01:00
Jim Brennan 4bdba35b9f HADOOP-17362. reduce RPC calls doing ls on HAR file (#2444). Contributed by Daryn Sharp and Ahmed Hussein 2020-11-13 21:49:35 +00:00
Takanobu Asanuma 3ee0a174dd HDFS-15538. Fix the documentation for dfs.namenode.replication.max-streams in hdfs-default.xml. Contributed by Xieming Li. 
(cherry picked from commit f56cd88d7d)
 2020-11-13 11:08:57 +09:00
Ahmed Hussein ac3e10af63 HADOOP-17358. Improve excessive reloading of Configurations (#2436) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0f)
(cherry picked from commit 23fe3bdab3)
 2020-11-12 10:35:47 -08:00
Stephen Jung 75c9aefe16 HADOOP-17096. Fix ZStandardCompressor input buffer offset (#2104). Contributed by Stephen Jung (Stripe). 
(cherry picked from commit 45434c93e8)
 2020-11-10 11:42:17 -08:00
Yizhong Zhang 52c2ce53c2 HADOOP-17142. Fix outdated properties of JournalNode when performing rollback. Contributed by Deegue. 
(cherry picked from commit 82de07c794)
(cherry picked from commit 12bfd9194a)
 2020-11-10 09:14:25 -08:00
Giovanni Matteo Fumarola 4af5f55623
HDFS-14395. Remove WARN Logging From Interrupts. Contributed by David Mollitor. 2020-11-10 14:44:34 +08:00
Akira Ajisaka 9cffa8c647
HADOOP-17352. Update PATCH_NAMING_RULE in the personality file. (#2433) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 34e2abd811)
 2020-11-10 14:01:03 +09:00
Steve Loughran b807fa2b3b HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. (#2427). Contributed by Steve Loughran. 2020-11-07 17:05:40 +05:30
hchaverr 92db25d64b HDFS-15623. Respect configured values of rpc.engine (#2403) Contributed by Hector Chaverri. 
(cherry picked from commit 6eacaffeea)
 2020-11-06 14:43:54 -08:00
Eric E Payne d6a55caa9a YARN-10479. RMProxy should retry on SocketTimeout Exceptions. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 55339c2bdd)
 2020-11-05 22:23:24 +00:00
Eric Badger 0a90c2ee89 HADOOP-17342. Creating a token identifier should not do kerberos name 
resolution. Contributed by Jim Brennan.

(cherry picked from commit af389d9897)
 2020-11-05 22:04:16 +00:00
Jim Brennan 431046d766 Revert "HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09 (#2387)" 
This reverts commit 8c6478b405.
 2020-11-05 17:22:22 +00:00
Wei-Chiu Chuang d6fd82ca56 Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291)" 
This reverts commit 1c7043b1ff.
 2020-11-04 16:18:41 -08:00
Ayush Saxena ce883fe1bb HDFS-15643. EC: Fix checksum computation in case of native encoders. (#2424). Contributed by Ayush Saxena. 2020-11-04 09:53:28 +05:30
Konstantin V Shvachko 23dcd8edb6 HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko. 
(cherry picked from commit d07dc7afb4)
 2020-11-03 12:04:38 -08:00
Akira Ajisaka 1c7043b1ff HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291) 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04c)
(cherry picked from commit dd1634ec3b)
 2020-11-03 11:23:02 -08:00
Eric E Payne 31154fdde5 YARN-10475: Scale RM-NM heartbeat interval based on node utilization. Contributed by Jim Brennan (Jim_Brennan). 2020-11-02 17:33:57 +00:00
Siyao Meng beb6cf6cc2
YARN-10472. Backport YARN-10314. YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars (#2412) 2020-10-29 16:17:07 -07:00
Jim Brennan 63888afdd0 YARN-10471. Prevent logs for any container from becoming larger than a configurable size. Contributed by Eric Payne 2020-10-29 20:17:51 +00:00
Sunil G 14a4606a9a HADOOP-17329. mvn site commands fails due to MetricsSystemImpl changes. Contributed by Xiaoqiao He. 
(cherry picked from commit f17e067d52)
 2020-10-29 07:21:31 +05:30
Chen Liang 576f99ba40 HADOOP-17330. Backport HADOOP-16005 NativeAzureFileSystem does not support setXAttr to branch-3.2. Contributed by Sally Zuo. 2020-10-28 15:39:18 -07:00
Jonathan Hung d0104e72c5 YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen 
(cherry picked from commit bab5bf9743)
(cherry picked from commit f95c0824b0)
 2020-10-28 10:38:58 -07:00
Ahmed Hussein dedf0d42bc
HDFS-9776. testMultipleAppendsDuringCatchupTailing is flaky (#2410) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit ae74407ac4)
 2020-10-27 20:52:23 +09:00
Ahmed Hussein 244fd5eed2
HDFS-15461. TestDFSClientRetries testGetFileChecksum fails (#2404) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit afaab3d332)
 2020-10-27 17:09:20 +09:00
Hui Fei b32926f110 HDFS-15641. DataNode could meet deadlock if invoke refreshNameNode. Contributed by Hongbing Wang. 
(cherry picked from commit 12c908c827)
 2020-10-26 22:34:06 +08:00
Ayush Saxena d2ecaf217f HADOOP-17119. Jetty upgrade to 9.4.x causes MR app fail with IOException. Contributed by Bilwa S T. 
(cherry picked from commit 8a58a12626)
 2020-10-23 17:53:29 -07:00
Wei-Chiu Chuang 87666e2438 HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein. 
(cherry picked from commit 74634eb002)
(cherry picked from commit c8eb86ee3d)
 2020-10-23 12:06:36 -07:00
Inigo Goiri 7119af2c6d HDFS-15459. TestBlockTokenWithDFSStriped fails intermittently. Contributed by Ahmed Hussein. 
(cherry picked from commit e7aa4da0a2)
 2020-10-23 09:49:46 -07:00
Vinayakumar B 8c6478b405
HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09 (#2387) 2020-10-23 11:39:43 +05:30
Kihwal Lee f363c3b315 HDFS-15622. Deleted blocks linger in the replications queue. Contributed by Ahmed Hussein. 
(cherry picked from commit da1b6e3cc2)
 2020-10-22 19:48:42 -05:00
Kihwal Lee f3b2d85690 HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. 
(cherry picked from commit cf932a7e2d)
 2020-10-22 14:59:09 -05:00