Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,787 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

19787 Commits

Author SHA1 Message Date
Ahmed Hussein 194e0c3cc2
HADOOP-17438. Increase docker memory limit in Jenkins (#2560) 
Reviewed-by: Eric Badger <ebadger@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ae894ecfcc)
 2021-01-12 15:36:49 +09:00
Szilard Nemeth 770a56315e YARN-10528. maxAMShare should only be accepted for leaf queues, not parent queues. Contributed by Siddharth Ahuja 2021-01-08 12:51:06 +01:00
Wei-Chiu Chuang 13768fe6f0 HDFS-15719. [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout (#2533) 
(cherry picked from commit 2b4febcf57)
(cherry picked from commit 94c126cc9e)
(cherry picked from commit 8af0fcf628)
 2021-01-04 21:00:29 -08:00
He Xiaoqiao 5d0eb4c6f0
HDFS-15751. Add documentation for msync() API to filesystem.md. Contributed by Konstantin V Shvachko. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
 2021-01-03 16:35:36 +08:00
He Xiaoqiao 6e2493f48e
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:49:26 +08:00
Eric Badger 52791d2005 YARN-10540. Node page is broken in YARN UI1 and UI2 including RMWebService api 
for nodes. Contributed by Jim Brennan.

(cherry picked from commit 4c5d88e230)
 2020-12-21 23:32:18 +00:00
Ahmed Hussein a23faaec18 HDFS-15704. Mitigate lease monitor's rapid infinite loop. (#2511). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit c2672bb234)
 2020-12-17 18:29:03 +00:00
He Xiaoqiao 1fe15113cf HADOOP-17068. Client fails forever when namenode ipaddr changed. Contributed by Sean Chow. 
(cherry picked from commit fa14e4bc00)
(cherry picked from commit 3a860e876e)
(cherry picked from commit 8d7600ae6b)
 2020-12-15 14:02:15 -08:00
Ayush Saxena 9e50a1082b HDFS-15200. Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage. Contributed by Ayush Saxena. 2020-12-15 11:05:17 +05:30
Ayush Saxena a68ca4a292 HDFS-15187. CORRUPT replica mismatch between namenodes after failover. Contributed by Ayush Saxena. 2020-12-15 10:35:01 +05:30
Ayush Saxena a09d636e2c HDFS-15170. EC: Block gets marked as CORRUPT in case of failover and pipeline recovery. Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 5e5c53070f1946b86e49c039e1f5ae42e5d022cb)
(cherry picked from commit faf26f70aa)
(cherry picked from commit ed78df0f75)
 2020-12-14 20:41:36 -08:00
S O'Donnell e7df47c45c HDFS-15725. Lease Recovery never completes for a committed block which the DNs never finalize. Contributed by Stephen O'Donnell 
(cherry picked from commit 52adda89c4)
 2020-12-14 11:51:44 +00:00
Jim Brennan 392f69471d HADOOP-13571. ServerSocketUtil.getPort() should use loopback address, not 0.0.0.0. Contributed by Eric Badger 
(cherry picked from commit 6de1a8eb67)
 2020-12-11 21:07:03 +00:00
Jim Brennan 777f426d6a HDFS-15711. Add Metrics to HttpFS Server. (#2521) Contributed by Ahmed Hussein and Kihwal Lee 2020-12-10 22:16:52 +00:00
Neil 6258405f10 HDFS-15720 namenode audit async logger should add some log4j config (#2532) 
(cherry picked from commit 9bd3c9bc50)
(cherry picked from commit bc5458bbd4)
(cherry picked from commit 9123f9245c)
 2020-12-10 13:50:55 -08:00
Ankit Kumar b89aff6db4
YARN-10491. Fix deprecation warnings in SLSWebApp.java (#2519) 
Signed-off-by: Akira Ajisaka <ajisaka@apache.org>
(cherry picked from commit aaf9e3d320)
 2020-12-09 10:54:39 +09:00
Ahmed Hussein 8b8a685b65 HDFS-15716. WaitforReplication in TestUpgradeDomainBlockPlacementPolicy (#2528) 
(cherry picked from commit 01383a2172)
 2020-12-08 19:54:12 +00:00
Hui Fei 2fde1f8bbf HDFS-15240. Erasure Coding: dirty buffer causes reconstruction block error. Contributed by HuangTao. 2020-12-08 11:03:02 +08:00
crossfire 173d298c63 HDFS-15709. Socket file descriptor leak in StripedBlockChecksumReconstructor. (#2518) 
(cherry picked from commit 40f7543a6d)
(cherry picked from commit edd9b659ca)
(cherry picked from commit 31a6b45117)
 2020-12-07 15:50:58 -08:00
Jim Brennan 8df2902a53 HDFS-15707. NNTop counts don't add up as expected. (#2516) Contributed by Ahmed Hussein and Daryn Sharp 
(cherry picked from commit 6a5864ee4a)
 2020-12-07 21:38:39 +00:00
Jim Brennan 486da59e1f HADOOP-17389. KMS should log full UGI principal. (#2476) Contributed by Ahmed Hussein 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
    Reviewed-by: Jim Brennan <jbrennan@apache.org>
 2020-12-07 17:57:28 +00:00
Ayush Saxena cfb96240dc
HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma. 
(cherry picked from commit b52fd05d42)
 2020-12-04 11:19:21 +09:00
Takanobu Asanuma 43f7ab2cf0
HDFS-13404. RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails. 
(cherry picked from commit 559cb11551)
 2020-12-04 11:19:14 +09:00
Akira Ajisaka e17b7efd3b
MAPREDUCE-7284. TestCombineFileInputFormat#testMissingBlocks fails (#2136) 
(cherry picked from commit 3e7b65c3e5)
 2020-12-04 10:43:32 +09:00
Ahmed Hussein 5f5d7ecf59 HDFS-15706. HttpFS: Log more information on request failures. (#2515) 
(cherry picked from commit 07655a7040)
(cherry picked from commit a7dbd3b6f9)
(cherry picked from commit f547cd43d1)
 2020-12-03 15:15:47 -08:00
Jim Brennan c43eca6cdc HADOOP-17392. Remote exception messages should not include the exception class (#2486). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit 5bfb97bc7d)
 2020-12-03 19:47:46 +00:00
Andrea Scarpino c8178a9caa
YARN-10511. Update yarn.nodemanager.env-whitelist value in docs (#2512) 
Reviewed-by: Adam Antal <adamantal@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9170eb566b)
 2020-12-04 00:18:12 +09:00
Jim Brennan 521d5b381a HDFS-15703. Don't generate edits for set operations that are no-op (#2508). Contributed by Daryn Sharp and Ahmed Hussein 2020-12-02 22:25:12 +00:00
Ahmed Hussein a63a30ff99 HDFS-15695. NN should not let the balancer run in safemode (#2489). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit 2b5b556dd7)
 2020-12-02 20:54:32 +00:00
Eric Payne 73cf0bdba1 YARN-10278: CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework. Contributed by Szilard Nemeth (snemeth) 
(cherry picked from commit 1184284baf)
 2020-12-02 17:27:32 +00:00
Ahmed Hussein 2331f86dfb HDFS-15694. Avoid calling UpdateHeartBeatState inside DataNodeDescriptor. (#2487) Contributed by Kuhu Shukla and Ahmed Hussein 
(cherry picked from commit 918ba9e82f)
 2020-12-01 21:43:37 +00:00
kevinzhao1661 a60aacc0c8
YARN-10498. Fix typo in CapacityScheduler Markdown document (#2484) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4d2ae5b398)
 2020-11-30 11:19:03 +09:00
Peter Bacsko 7d60217754 MAPREDUCE-7309. Improve performance of reading resource request for mapper/reducers from config. Contributed by Peter Bacsko & Wangda Tan. 2020-11-25 11:46:00 +01:00
Akira Ajisaka 4638ed94db
YARN-10470. When building new web ui with root user, the bower install should support it. Contributed by zhuqi. 
(cherry picked from commit c4ba0ab7df)
 2020-11-24 15:23:34 +09:00
Eric Payne c7845dc574 HADOOP-17346. Fair call queue is defeated by abusive service principals. Contributed by Ahmed Hussein (ahussein). 2020-11-23 21:34:52 +00:00
Peter Bacsko 6062978768 MAPREDUCE-7304. Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class. Contributed by Zoltan Erdmann 2020-11-20 13:16:48 +01:00
Jim Brennan da87653b66 HADOOP-17367. Add InetAddress api to ProxyUsers.authorize (#2449). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit 6b3d65b46e)
 2020-11-19 21:59:57 +00:00
Ahmed Hussein 56e327b7ca HADOOP-17360. Log the remote address for authentication success (#2441) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 1ea3f74246)
 2020-11-16 22:08:14 +00:00
Ahmed Hussein de9f778750 YARN-10485. TimelineConnector swallows InterruptedException (#2450). Contributed by Ahmed Hussein 
(cherry picked from commit 0b2510ee1f)
 2020-11-16 21:30:50 +00:00
Peter Bacsko a86c91bc06 YARN-10396. Max applications calculation per queue disregards queue level settings in absolute mode. Contributed by Benjamin Teke. 2020-11-16 11:54:19 +01:00
Jim Brennan 118d8f5aa1 HADOOP-17362. reduce RPC calls doing ls on HAR file (#2444). Contributed by Daryn Sharp and Ahmed Hussein 
(cherry picked from commit 4bdba35b9f)
 2020-11-13 21:52:10 +00:00
Takanobu Asanuma cbc352ea2d HDFS-15538. Fix the documentation for dfs.namenode.replication.max-streams in hdfs-default.xml. Contributed by Xieming Li. 
(cherry picked from commit f56cd88d7d)
 2020-11-13 11:10:32 +09:00
Ahmed Hussein 487eaa6dea HADOOP-17358. Improve excessive reloading of Configurations (#2436) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0f)
(cherry picked from commit 23fe3bdab3)
(cherry picked from commit ac3e10af63)
 2020-11-12 10:35:57 -08:00
Stephen Jung 9f7553bab4 HADOOP-17096. Fix ZStandardCompressor input buffer offset (#2104). Contributed by Stephen Jung (Stripe). 
(cherry picked from commit 45434c93e8)
(cherry picked from commit 0712505b59)
 2020-11-10 11:41:34 -08:00
Yizhong Zhang e2d7e72c1b HADOOP-17142. Fix outdated properties of JournalNode when performing rollback. Contributed by Deegue. 
(cherry picked from commit 82de07c794)
(cherry picked from commit 12bfd9194a)
(cherry picked from commit 52c2ce53c2)
 2020-11-10 09:14:37 -08:00
Akira Ajisaka 5085934c90
HADOOP-17352. Update PATCH_NAMING_RULE in the personality file. (#2433) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 34e2abd811)
 2020-11-10 14:01:28 +09:00
Steve Loughran d903278607 HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. (#2427). Contributed by Steve Loughran. 2020-11-07 17:05:59 +05:30
hchaverr 3443d75946 HDFS-15623. Respect configured values of rpc.engine (#2403) Contributed by Hector Chaverri. 
(cherry picked from commit 6eacaffeea)
 2020-11-06 15:02:47 -08:00
Eric E Payne c7b1bdba63 YARN-10479. RMProxy should retry on SocketTimeout Exceptions. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 55339c2bdd)
 2020-11-05 22:33:58 +00:00
Eric Badger 4cec4bebd9 HADOOP-17342. Creating a token identifier should not do kerberos name 
resolution. Contributed by Jim Brennan.

(cherry picked from commit af389d9897)
 2020-11-05 22:11:52 +00:00