Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,380 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

15380 Commits

Author SHA1 Message Date
Jonathan Hung 1d76a8e73f YARN-10651. CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource(). Contributed by Haibo Chen 
(cherry picked from commit f348ab3f2f468751af329a1ffce4917cb000fcbf)
(cherry picked from commit be6e99963d)
(cherry picked from commit 6863a5bb8a)
(cherry picked from commit eb6c08e423)
 2021-02-25 15:47:36 -08:00
Jim Brennan 4ed7b80b19 [YARN-10613] Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF. Contributed by Eric Payne 2021-02-25 20:30:42 +00:00
S O'Donnell 628ccf81ef HDFS-15422. Reported IBR is partially replaced with stored info when queuing. Contributed by Stephen O'Donnell and Kihwal Lee 2021-02-24 14:47:36 +00:00
Renukaprasad C 7c83923192 HADOOP-17510. Hadoop prints sensitive Cookie information. (#2673) 
(cherry picked from commit 924fa87010)
(cherry picked from commit 9951ea9fa8)
 2021-02-24 17:34:45 +08:00
Xieming Li ee40e98d10
HADOOP-16052. Remove Subversion and Forrest from Dockerfile 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit bdbca0e53b)
 2021-02-22 23:37:19 +09:00
Akira Ajisaka f7ed17e636
MAPREDUCE-7323. Remove job_history_summary.py. (#2712) 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 8bca105655)
 2021-02-22 23:17:12 +09:00
Kengo Seki 5f969ced67
HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 
(cherry picked from commit fd7de2b82a)
 2021-02-20 19:16:30 +09:00
Jim Brennan 7363e6eee1 [YARN-10626] Log resource allocation in NM log at container start time. Contributed by Eric Badger 
(cherry picked from commit e6f5dbbe7f)
 2021-02-16 17:30:08 +00:00
Jim Brennan d0562d6cd0 YARN-10500. TestDelegationTokenRenewer fails intermittently. (#2619) Contributed by Masatake Iwasaki 2021-02-11 22:45:08 +00:00
Akira Ajisaka a5a6c973af
HADOOP-17516. Upgrade ant to 1.10.9. (#2683) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 933fbeffee)
 2021-02-11 08:45:42 +09:00
Eric Badger 585d285e1d MAPREDUCE-7319. Log list of mappers at trace level in ShuffleHandler audit log. 
Contributed by Jim Brennan.

(cherry picked from commit 462561654b)
 2021-02-09 19:29:22 +00:00
yzhangal bac234b0a8
HADOOP-17338 Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc (#2692)) 
Backported from commit ID 3d2193cd64

Contributed by Yongjun Zhang
 2021-02-09 12:04:14 +00:00
Eric Badger 7b4034cd88 YARN-6977. Node information is not provided for non am containers in RM logs. (Suma Shivaprasad via wangda) 
Change-Id: I0c44d09a560446dee2ba68c2b9ae69fce0ec1d3e
(cherry picked from commit 8a42e922fad613f3cf1cc6cb0f3fa72546a9cc56)
(cherry picked from commit 958e8c0e25)
 2021-02-08 20:04:56 +00:00
yzhangal ceea43115b
HADOOP-16767 Handle non-IO exceptions in reopen() (#2685) 
Contributed by Sergei Poganshev.

Catches Exception instead of IOException in closeStream() 
and so handle exceptions such as SdkClientException by 
aborting the wrapped stream. This will increase resilience
to failures, as any which occuring during stream closure
will be caught. Furthermore, because the
underlying HTTP connection is aborted, rather than closed,
it will not be recycled to cause problems on subsequent
operations.
 2021-02-08 16:03:43 +00:00
He Xiaoqiao ef14e16d36
HDFS-15792. ClasscastException while loading FSImage. Contributed by Renukaprasad C. 2021-02-07 15:50:10 +08:00
Akira Ajisaka 961bd01b17
YARN-10036. Install yarnpkg and upgrade nodejs in Dockerfile (#1772) 
(cherry picked from commit bc825b9628)
 2021-02-04 14:47:02 +09:00
Symious 5dd6db1856
HDFS-15801. Backport HDFS-14582 to branch-2.10 (Failed to start DN with ArithmeticException when NULL checksum used) (#2659) 
Co-authored-by: Symious <yiyang0203@gmail.com>
 2021-02-01 15:43:44 -08:00
Akira Ajisaka dded428e9a
HADOOP-17354. Move Jenkinsfile outside of the root directory. (#2647) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 84b154ebc0)
 2021-02-01 18:54:06 +09:00
He Xiaoqiao 1ded537146
HADOOP-17501. Fix logging typo in ShutdownHookManager. Contributed by Fengnan Li. 2021-01-31 20:01:43 +08:00
He Xiaoqiao b30e9198bb
HADOOP-16947. Stale record should be remove when MutableRollingAverages generating aggregate data. Contributed by Felix N, Haibin Huang. 2021-01-30 12:11:44 +08:00
Jim Brennan c1a3e8158d [HDFS-10498] Intermittent test failure TestSnapshotFileLength.testSnapshotFileLength. Contributed by Jim Brennan. 
(cherry picked from commit 6184ab07f6)
 2021-01-28 17:46:06 +00:00
Akira Ajisaka 130d5af258
HADOOP-17478. Improve the description of hadoop.http.authentication.signature.secret.file (#2628) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit d82009599a)
 2021-01-24 22:59:54 +09:00
Takanobu Asanuma 3dcb06a1ba
HADOOP-16841. The description of hadoop.http.authentication.signature.secret.file contains outdated information. Contributed by Xieming Li. 
(cherry picked from commit 5cbc4c5461)
 2021-01-24 22:59:46 +09:00
Anton Kutuzov 0ed997abab HDFS-15632. AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases. Contributed by Anton Kutuzov. 
(cherry picked from commit 91d4ba57c5)
 2021-01-22 18:19:51 -08:00
Wei-Chiu Chuang e48b91c83c HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db)
 2021-01-21 17:48:14 +00:00
Yiqun Lin 1b046fb26f HADOOP-13965. Groups should be consistent in using default group mapping class. Contributed by Yiqun Lin. 
(cherry picked from commit 383aa9c9f5)
 2021-01-21 17:33:54 +00:00
Eric Badger ea3c3d9022 YARN-10562. Follow up changes for YARN-9833. Contributed by Jim Brennan. 
(cherry picked from commit 768e2f42ba)
 2021-01-15 19:53:02 +00:00
Eric Badger b80ff6ea57 YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko. 
(cherry picked from commit c474e24c0b)
 2021-01-15 19:18:58 +00:00
Akira Ajisaka db5b804238
HDFS-15717. Improve fsck logging. Contributed by Kihwal Lee and Ahmed Hussein (#2620) 
(cherry picked from commit be35fa186c)
 2021-01-15 10:28:11 +09:00
Ahmed Hussein 4ddd41bd47
HADOOP-17438. Increase docker memory limit in Jenkins (#2560) 
Reviewed-by: Eric Badger <ebadger@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ae894ecfcc)
 2021-01-12 15:37:21 +09:00
He Xiaoqiao 1efadde067
HDFS-15751. Add documentation for msync() API to filesystem.md. Contributed by Konstantin V Shvachko. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
 2021-01-03 16:38:47 +08:00
He Xiaoqiao 45774daee8
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:51:01 +08:00
David Mollitor 16fd00d62d HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.

(cherry picked from commit c90736350b)
 2020-12-22 10:19:44 -08:00
Eric Badger f239da08af YARN-10540. Node page is broken in YARN UI1 and UI2 including RMWebService api 
for nodes. Contributed by Jim Brennan.

(cherry picked from commit 4c5d88e230)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeResourceMonitorImpl.java
 2020-12-21 23:42:18 +00:00
He Xiaoqiao 2c45e93682 HADOOP-17068. Client fails forever when namenode ipaddr changed. Contributed by Sean Chow. 
(cherry picked from commit fa14e4bc00)
(cherry picked from commit 3a860e876e)
(cherry picked from commit 8d7600ae6b)
(cherry picked from commit 1fe15113cf)
 2020-12-15 14:08:20 -08:00
S O'Donnell 35c99be3da HDFS-15725. Lease Recovery never completes for a committed block which the DNs never finalize. Contributed by Stephen O'Donnell 2020-12-14 12:37:26 +00:00
Zhankun Tang 406957f5e8 YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T. 
(cherry picked from commit fff4fbc957)
 2020-12-10 18:05:30 +00:00
Jim Brennan 8af247df85 HDFS-15716. WaitforReplication in TestUpgradeDomainBlockPlacementPolicy (#2528) Contributed by Ahmed Hussein 2020-12-08 20:32:31 +00:00
Yiqun Lin 8ddd85b0e6 HDFS-15660. StorageTypeProto is not compatiable between 3.x and 2.6. Contributed by Ryan Wu. 
(cherry picked from commit da1ea2530f)
 2020-12-07 18:56:15 +08:00
kevinzhao1661 2fe36b0038
YARN-10498. Fix typo in CapacityScheduler Markdown document (#2484) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4d2ae5b398)
 2020-11-30 11:19:24 +09:00
Jim Brennan 6a58e6293f YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 2020-11-17 17:36:22 +00:00
Ahmed Hussein c46643549f HADOOP-17358. Improve excessive reloading of Configurations (#2436) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0f)
(cherry picked from commit 23fe3bdab3)
(cherry picked from commit ac3e10af63)
(cherry picked from commit 487eaa6dea)
 2020-11-12 10:55:35 -08:00
Akira Ajisaka 266c9ad66a
HADOOP-17352. Update PATCH_NAMING_RULE in the personality file. (#2433) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 34e2abd811)
 2020-11-10 14:02:18 +09:00
Steve Loughran 894614ee4e HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. (#2427). Contributed by Steve Loughran. 2020-11-07 17:06:17 +05:30
hchaverr 9160b0e8c8 HDFS-15623. Respect configured values of rpc.engine (#2403) Contributed by Hector Chaverri. 
(cherry picked from commit 6eacaffeea)
 2020-11-06 15:07:34 -08:00
Eric Badger 7a830d2633 HADOOP-17342. Creating a token identifier should not do kerberos name 
resolution. Contributed by Jim Brennan.

(cherry picked from commit af389d9897)
 2020-11-05 22:17:32 +00:00
Wei-Chiu Chuang fc4c98dc49 Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291)" 
This reverts commit 52f0119051.
 2020-11-04 16:19:13 -08:00
Konstantin V Shvachko a9bf374380 HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko. 
(cherry picked from commit d07dc7afb4)
 2020-11-03 14:39:38 -08:00
Chen Liang ac82334041 HADOOP-17336. Backport HADOOP-16005 NativeAzureFileSystem does not support setXAttr and HADOOP-16785. Improve wasb and abfs resilience on double close() calls. followup to abfs close() fix to branch-2.10. Contributed by Sally Zuo. 2020-11-03 12:00:57 -08:00
Akira Ajisaka 52f0119051 HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291) 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04c)
(cherry picked from commit dd1634ec3b)
(cherry picked from commit 1c7043b1ff)
(cherry picked from commit 166bee93af)
 2020-11-03 11:24:08 -08:00