Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,367 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

15367 Commits

Author SHA1 Message Date
yzhangal ceea43115b
HADOOP-16767 Handle non-IO exceptions in reopen() (#2685) 
Contributed by Sergei Poganshev.

Catches Exception instead of IOException in closeStream() 
and so handle exceptions such as SdkClientException by 
aborting the wrapped stream. This will increase resilience
to failures, as any which occuring during stream closure
will be caught. Furthermore, because the
underlying HTTP connection is aborted, rather than closed,
it will not be recycled to cause problems on subsequent
operations.
 2021-02-08 16:03:43 +00:00
He Xiaoqiao ef14e16d36
HDFS-15792. ClasscastException while loading FSImage. Contributed by Renukaprasad C. 2021-02-07 15:50:10 +08:00
Akira Ajisaka 961bd01b17
YARN-10036. Install yarnpkg and upgrade nodejs in Dockerfile (#1772) 
(cherry picked from commit bc825b9628)
 2021-02-04 14:47:02 +09:00
Symious 5dd6db1856
HDFS-15801. Backport HDFS-14582 to branch-2.10 (Failed to start DN with ArithmeticException when NULL checksum used) (#2659) 
Co-authored-by: Symious <yiyang0203@gmail.com>
 2021-02-01 15:43:44 -08:00
Akira Ajisaka dded428e9a
HADOOP-17354. Move Jenkinsfile outside of the root directory. (#2647) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 84b154ebc0)
 2021-02-01 18:54:06 +09:00
He Xiaoqiao 1ded537146
HADOOP-17501. Fix logging typo in ShutdownHookManager. Contributed by Fengnan Li. 2021-01-31 20:01:43 +08:00
He Xiaoqiao b30e9198bb
HADOOP-16947. Stale record should be remove when MutableRollingAverages generating aggregate data. Contributed by Felix N, Haibin Huang. 2021-01-30 12:11:44 +08:00
Jim Brennan c1a3e8158d [HDFS-10498] Intermittent test failure TestSnapshotFileLength.testSnapshotFileLength. Contributed by Jim Brennan. 
(cherry picked from commit 6184ab07f6)
 2021-01-28 17:46:06 +00:00
Akira Ajisaka 130d5af258
HADOOP-17478. Improve the description of hadoop.http.authentication.signature.secret.file (#2628) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit d82009599a)
 2021-01-24 22:59:54 +09:00
Takanobu Asanuma 3dcb06a1ba
HADOOP-16841. The description of hadoop.http.authentication.signature.secret.file contains outdated information. Contributed by Xieming Li. 
(cherry picked from commit 5cbc4c5461)
 2021-01-24 22:59:46 +09:00
Anton Kutuzov 0ed997abab HDFS-15632. AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases. Contributed by Anton Kutuzov. 
(cherry picked from commit 91d4ba57c5)
 2021-01-22 18:19:51 -08:00
Wei-Chiu Chuang e48b91c83c HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db)
 2021-01-21 17:48:14 +00:00
Yiqun Lin 1b046fb26f HADOOP-13965. Groups should be consistent in using default group mapping class. Contributed by Yiqun Lin. 
(cherry picked from commit 383aa9c9f5)
 2021-01-21 17:33:54 +00:00
Eric Badger ea3c3d9022 YARN-10562. Follow up changes for YARN-9833. Contributed by Jim Brennan. 
(cherry picked from commit 768e2f42ba)
 2021-01-15 19:53:02 +00:00
Eric Badger b80ff6ea57 YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko. 
(cherry picked from commit c474e24c0b)
 2021-01-15 19:18:58 +00:00
Akira Ajisaka db5b804238
HDFS-15717. Improve fsck logging. Contributed by Kihwal Lee and Ahmed Hussein (#2620) 
(cherry picked from commit be35fa186c)
 2021-01-15 10:28:11 +09:00
Ahmed Hussein 4ddd41bd47
HADOOP-17438. Increase docker memory limit in Jenkins (#2560) 
Reviewed-by: Eric Badger <ebadger@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ae894ecfcc)
 2021-01-12 15:37:21 +09:00
He Xiaoqiao 1efadde067
HDFS-15751. Add documentation for msync() API to filesystem.md. Contributed by Konstantin V Shvachko. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
 2021-01-03 16:38:47 +08:00
He Xiaoqiao 45774daee8
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:51:01 +08:00
David Mollitor 16fd00d62d HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.

(cherry picked from commit c90736350b)
 2020-12-22 10:19:44 -08:00
Eric Badger f239da08af YARN-10540. Node page is broken in YARN UI1 and UI2 including RMWebService api 
for nodes. Contributed by Jim Brennan.

(cherry picked from commit 4c5d88e230)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeResourceMonitorImpl.java
 2020-12-21 23:42:18 +00:00
He Xiaoqiao 2c45e93682 HADOOP-17068. Client fails forever when namenode ipaddr changed. Contributed by Sean Chow. 
(cherry picked from commit fa14e4bc00)
(cherry picked from commit 3a860e876e)
(cherry picked from commit 8d7600ae6b)
(cherry picked from commit 1fe15113cf)
 2020-12-15 14:08:20 -08:00
S O'Donnell 35c99be3da HDFS-15725. Lease Recovery never completes for a committed block which the DNs never finalize. Contributed by Stephen O'Donnell 2020-12-14 12:37:26 +00:00
Zhankun Tang 406957f5e8 YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T. 
(cherry picked from commit fff4fbc957)
 2020-12-10 18:05:30 +00:00
Jim Brennan 8af247df85 HDFS-15716. WaitforReplication in TestUpgradeDomainBlockPlacementPolicy (#2528) Contributed by Ahmed Hussein 2020-12-08 20:32:31 +00:00
Yiqun Lin 8ddd85b0e6 HDFS-15660. StorageTypeProto is not compatiable between 3.x and 2.6. Contributed by Ryan Wu. 
(cherry picked from commit da1ea2530f)
 2020-12-07 18:56:15 +08:00
kevinzhao1661 2fe36b0038
YARN-10498. Fix typo in CapacityScheduler Markdown document (#2484) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4d2ae5b398)
 2020-11-30 11:19:24 +09:00
Jim Brennan 6a58e6293f YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 2020-11-17 17:36:22 +00:00
Ahmed Hussein c46643549f HADOOP-17358. Improve excessive reloading of Configurations (#2436) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0f)
(cherry picked from commit 23fe3bdab3)
(cherry picked from commit ac3e10af63)
(cherry picked from commit 487eaa6dea)
 2020-11-12 10:55:35 -08:00
Akira Ajisaka 266c9ad66a
HADOOP-17352. Update PATCH_NAMING_RULE in the personality file. (#2433) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 34e2abd811)
 2020-11-10 14:02:18 +09:00
Steve Loughran 894614ee4e HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. (#2427). Contributed by Steve Loughran. 2020-11-07 17:06:17 +05:30
hchaverr 9160b0e8c8 HDFS-15623. Respect configured values of rpc.engine (#2403) Contributed by Hector Chaverri. 
(cherry picked from commit 6eacaffeea)
 2020-11-06 15:07:34 -08:00
Eric Badger 7a830d2633 HADOOP-17342. Creating a token identifier should not do kerberos name 
resolution. Contributed by Jim Brennan.

(cherry picked from commit af389d9897)
 2020-11-05 22:17:32 +00:00
Wei-Chiu Chuang fc4c98dc49 Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291)" 
This reverts commit 52f0119051.
 2020-11-04 16:19:13 -08:00
Konstantin V Shvachko a9bf374380 HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko. 
(cherry picked from commit d07dc7afb4)
 2020-11-03 14:39:38 -08:00
Chen Liang ac82334041 HADOOP-17336. Backport HADOOP-16005 NativeAzureFileSystem does not support setXAttr and HADOOP-16785. Improve wasb and abfs resilience on double close() calls. followup to abfs close() fix to branch-2.10. Contributed by Sally Zuo. 2020-11-03 12:00:57 -08:00
Akira Ajisaka 52f0119051 HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291) 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04c)
(cherry picked from commit dd1634ec3b)
(cherry picked from commit 1c7043b1ff)
(cherry picked from commit 166bee93af)
 2020-11-03 11:24:08 -08:00
Jonathan Hung 6f436a6776 YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen 2020-10-28 10:45:34 -07:00
Wei-Chiu Chuang e7e165a927 HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein. 2020-10-28 09:22:47 -07:00
Wei-Chiu Chuang 90ebbaa393 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837a)
(cherry picked from commit 975d4b3d60)
 2020-10-24 13:38:58 -07:00
Akira Ajisaka 633e19cb49
HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs. (#2391) 
(cherry picked from commit 3a0922fb42)

 Conflicts:
	hadoop-project/pom.xml
 2020-10-19 11:04:28 +09:00
Eric Badger c4b42fa1ae YARN-10450. Add cpu and memory utilization per node and cluster-wide metrics. 
Contributed by Jim Brennan.
 2020-10-16 19:29:04 +00:00
Konstantin V Shvachko 104dd85ad8 HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b3786d6c3c)
 2020-10-12 18:02:05 -07:00
Swaroopa Kadam 893fceadee
MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-10-12 11:10:52 -07:00
Jim Brennan 4bb0124c69 YARN-9667. Container-executor.c duplicates messages to stdout. Contributed by Peter Bacsko 2020-10-12 15:41:33 +00:00
Eric Badger 3335fcfb05 YARN-10455. TestNMProxy.testNMProxyRPCRetry is not consistent. Contributed by 
Ahmed Hussein.
 2020-10-09 20:41:58 +00:00
Adam Antal 1c0fe2eb20 YARN-10393. MR job live lock caused by completed state container leak in heartbeat between node manager and RM. Contributed by zhenzhao wang and Jim Brennan 2020-10-07 16:46:22 +02:00
Jim Brennan 0bf270d2ed YARN-10451. RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. Contributed by Eric Payne 
(cherry picked from commit ecf91638a8)
 2020-10-06 18:46:08 +00:00
Wanqiang Ji e8699d0bd5
MAPREDUCE-7289. Fix wrong comment in LongLong.java (#2338) 
(cherry picked from commit 143bdd4188)
 2020-09-29 23:09:12 +09:00
zz be421490da
MAPREDUCE-7294. Only application master should upload resource to Yarn Shared Cache. (#2319) 
Contributed by Zhenzhao Wang <zhenzhaowang@gmail.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-09-22 11:57:36 -07:00