Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,903 Commits 326 Branches 371 Tags
Commit Graph

17903 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wei-Chiu Chuang
cbac102566 HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang. 
(cherry picked from commit be488b6070a124234c77f16193ee925d32ca9a20)
(cherry picked from commit c8703dda0727e17d759d7ad27f0caee88103a530)
(cherry picked from commit 2a94603ae66d9000c0bb07df0d592279339af103)
 2019-04-03 11:02:14 -07:00
Akira Ajisaka
6f0896e087
HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji. 
(cherry picked from commit 8b6deebb1dda49e5e35180ed5c5fb5b5221c1516)
(cherry picked from commit 875435dc7d6a7cb7925a25ca3347a7ebaabec595)
 2019-04-03 19:40:52 +09:00
Akira Ajisaka
524dc4e662
HADOOP-16225. Fix links to the developer mailing lists in DownstreamDev.md. Contributed by Wanqiang Ji. 
(cherry picked from commit ebd0d2153884bf78a159534ea711c0b70acc1c05)
 2019-04-02 10:55:27 +09:00
Gabor Bota
461df1b511
HADOOP-16220. Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project 
This closes #665

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 53a86e2b8ecb83b666d4ed223fc270e1a46642c1)
 2019-04-01 13:53:11 +09:00
Takanobu Asanuma
abb06c9d7c HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory. 
(cherry picked from commit 55fb3c32fb48ca26a629d4d5f3f07e2858d09594)
 2019-03-27 03:30:12 +09:00
Weiwei Yang
9b1f13fbd2 HADOOP-16191. AliyunOSS: improvements for copyFile/copyDirectory and logging. Contributed by wujinhu. 
(cherry picked from commit 568d3ab8b65d1348dec9c971feffe200e6cba2ef)
 2019-03-19 10:13:57 +08:00
Erik Krogen
e172fc62ce HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed. 
(cherry-picked from 8c95cb9d6bef369fef6a8364f0c0764eba90e44a)
(cherry-picked from 0de8b55a095ada2b98c0a41899651bd8e524f42e)
(cherry-picked from d4fbbc83ad8c4d818deccf62b4c54cead1d17a8f)
 2019-03-18 08:57:41 -07:00
Inigo Goiri
5a8445789b HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun. 
(cherry picked from commit ff06ef0631cb8a0f67bbc39b5b5a1b0a81ca3b3c)
 2019-03-15 13:59:51 -07:00
Erik Krogen
9a5daa5ec6 HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 
(cherry picked from commit 66357574ae1da09ced735da36bf7d80a40c3fa1b)
(cherry picked from commit fec7c5f3ebbaea7b290e0904570c3a485b541a22)
(cherry picked from commit cb76fc5618c582887e4e71b98f3065c5a916334a)
 2019-03-13 13:21:40 -07:00
Steve Loughran
7f70a78ce3
HADOOP-16109. Parquet reading S3AFileSystem causes EOF (#589) 
Nobody gets seek right. No matter how many times they think they have.

Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran

Change-Id: I39b87f3d5daa98f65de2c0a44e348821a4930573
 2019-03-12 11:36:32 +00:00
Weiwei Yang
301ffee1bc MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang. 
(cherry picked from commit f0605146b34ddbd19fc9e52d4761f9a347d5173e)
 2019-03-11 16:57:08 +08:00
Erik Krogen
c924011f94 HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam. 
(cherry-picked from commit 1bc282e0b3f74968c92751f0972746b012e72810)
(cherry-picked from commit 6d076dd5e8be8fee5062a04d4cd2aa0c5da0ea56)
(cherry-picked from commit b0b80002402cf5fbe819367d96e42098c5c359ea)
 2019-03-07 08:45:20 -08:00
Jason Lowe
32f96b6be4 YARN-5714. ContainerExecutor does not order environment map. Contributed by Remi Catherinot and Jim Brennan 
(cherry picked from commit 8e728f39c961f034369b43e087d68d01aa4a0e7d)
 2019-03-07 15:10:52 +00:00
Wei-Chiu Chuang
e23a448e0e HDFS-14314. fullBlockReportLeaseId should be reset after registering to NN. Contributed by star. 
(cherry picked from commit 387dbe587aa66ac99ec5f5b50827ec3e0a327613)
(cherry picked from commit e58ccca3ce131c955ceb115cd0b75e452eea828b)
(cherry picked from commit d951497f57cf6556b0916cad08576481dfe2ae06)
 2019-03-04 10:46:16 -08:00
Erik Krogen
b170626c74 HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao. 2019-03-01 08:12:58 -08:00
Sunil G
c139fe7a37 YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang. 
(cherry picked from commit 6cec90653dc2ae60a10262745997ea91dd872a8b)
 2019-02-25 15:30:17 +05:30
Daryn Sharp
4eccf2a3cc HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a87e458432609b7a35a2abd6410b02e8a2ffc974)
(cherry picked from commit ae8839e6e8cc3e8f8d5e50525d3302038ada484b)
(cherry picked from commit 704330a616c17256b3e39370f999928ba1c463e6)
 2019-02-20 18:22:20 -08:00
Adam Antal
bebbdd290e
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d75109a2e6565faa10527cc56637bf5f)
 2019-02-15 18:59:02 +09:00
Masatake Iwasaki
ba035dda76 YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 
(cherry picked from commit e0ab1bdecec3d7ba7ddc0849781d7f71714f8687)
 2019-02-09 00:34:22 +09:00
Eric E Payne
f3ff6d3aa7 YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 
(cherry picked from commit d1ca9432dd0b3e9b46b4903e8c9d33f5c28fcc1b)
 2019-02-07 21:03:43 +00:00
Sunil G
ec0bed1008 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 
(cherry picked from commit 6ffe6ea8999c1216b6e510e350535754431bac2d)
 2019-02-07 19:09:19 +05:30
Kitti Nanasi
ad131757f2 HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 49ddd8a6ed5b40d12defb0771b4c8b53d4ffde3f)
(cherry picked from commit 38541b23753c5d0e2cb9baf1d2da536f846dc38c)
(cherry picked from commit c25b641cec8aca9c16759d5e392ffbc27a158a67)
 2019-02-05 15:15:53 -08:00
Todd Lipcon
02cfd5fa35
HADOOP-15549. Upgrade to commons-configuration 2.1 regresses task CPU consumption. 
Contributed by Todd Lipcon.

(cherry picked from commit 1f21be2d024de724990a0b5737c0c65c6e59d7f3)
 2019-02-05 19:57:08 +00:00
Akira Ajisaka
875b8c390b
HADOOP-16089. AliyunOSS: update oss-sdk version to 3.4.1. Contributed by wujinhu. 
(cherry picked from commit 0f9aa5b4d58e34653f42e98a244aaf1c806b37cf)
 2019-02-04 14:48:25 +09:00
Eric E Payne
50ef25464f YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb041a41062c856632641140de84fba8)
 2019-01-29 18:12:36 +00:00
Weiwei Yang
d182c81938 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 
(cherry picked from commit 4f63ffe444286dac91ed36fd647d8ce69e75b0f0)
 2019-01-29 11:07:51 +08:00
Masatake Iwasaki
00eee37a93 HDFS-12729. Document special paths in HDFS. Contributed by Masatake Iwasaki. 
(cherry picked from commit dc5af4c6b7bdf437713a8c6ca3c535c4b6d6992e)
 2019-01-29 02:31:59 +09:00
Jonathan Hung
bce8f28bec YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e212d3ee0aec988926a5a17c9cc58e645)
(cherry picked from commit bf760e7e81f8e02ad413c470fccf78aaa9cb9f86)
(cherry picked from commit 80a08c29adc9d07974bbe8087abcec750d5f7916)
 2019-01-25 13:51:21 -08:00
Dinesh Chitlangia
36ff7a4d02 HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 45caeee6cfcf1ae3355cd880402159cf31e94a8a)
(cherry picked from commit eb77e60163e318ca42548e134fe79dba7289fdc6)
(cherry picked from commit b63941516f78e0d6e22ecc629fc82b16fd17ad87)
 2019-01-25 07:38:49 -08:00
Haibo Chen
3368f141b2 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef96baa234966b60e464d1773fbf3f22)
(cherry picked from commit 3a764250262e745a6762dd6551b0d0a6af39e891)
 2019-01-24 15:58:27 -08:00
Sunil G
d11fdbb7ac YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB 
(cherry picked from commit c7264459901e59021e180228cea3a6ae334a339f)
 2019-01-24 15:03:16 +05:30
Takanobu Asanuma
5b8a99bf49 HDFS-14218. EC: Ls -e throw NPE when directory ec policy is disabled. Contributed by Ayush Saxena. 
(cherry picked from commit 7d6792e5d2001e1a83ae75a4777324a3b01c557a)
 2019-01-23 10:06:30 +09:00
Weiwei Yang
0e3d5f3064 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751bcadab77d42b31e3e1dd5748b471b5)
 2019-01-22 11:15:03 +08:00
Chen Liang
80b9fecaaf HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian. 2019-01-18 11:45:22 -08:00
Akira Ajisaka
98345dcca4
HDFS-14213. Remove Jansson from BUILDING.txt. Contributed by Dinesh Chitlangia. 
(cherry picked from commit 96a84b61fb693e9657f1f1ae3d6ed3a04a7e8efa)
(cherry picked from commit dea75af9a720164762c529f2f44f03c076c7b074)
 2019-01-18 15:35:05 +09:00
Akira Ajisaka
1a38a61716
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 
(cherry picked from commit 104ef5df36fd45f1c2d07a2f3d441263aa85e22e)
 2019-01-16 14:38:55 +09:00
Inigo Goiri
174fa73f99 HADOOP-16045. Don't run TestDU on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 35fa3bd685605d8b3639e6c5cbe83cd9acd8cbe7)
 2019-01-11 18:16:44 -08:00
Akira Ajisaka
9fd5a6b66b
HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper 
(cherry picked from commit d4ca907da636892e4ab98e232fe0f7f77f1f7aac)
 2019-01-11 13:05:31 +09:00
Jason Lowe
642213a74a Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh." 
This reverts commit a4f84213241c647c973acfa460ac4f4b0895aa2d.
 2019-01-09 17:56:32 -06:00
Kai Xie
a959aa61b4
HADOOP-16018. DistCp won't reassemble chunks when blocks per chunk > 0. 
Contributed by Kai Xie.

(cherry picked from commit 188bebbe7eaec71433548dc47f5a9a995cbea15a)
 2019-01-08 13:35:41 +00:00
Weiwei Yang
009f5b55ad YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed Wilfred Spiegelenburg. 
(cherry picked from commit 91e9c9f96eca01bbef7d22b4cff1ced3be1ac5e8)
 2019-01-08 14:09:49 +08:00
Weiwei Yang
ad5dd32843 HADOOP-16030. AliyunOSS: bring fixes back from HADOOP-15671. Contributed by wujinhu. 
(cherry picked from commit f87b3b11c46704dcdb63089dd971e2a5ba1deaac)
 2019-01-07 17:03:22 +08:00
Sunil G
a450d58b49
Revert "HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu." 
This reverts commit e4fca6aae46a3c04fc56897986a4ab4e5aa98503.

Revert "HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu."

This reverts commit 2b635125fb059fc204ed35bc0e264c42dd3a9fe9.

(cherry picked from commit 1f425271a73fff1fdbe3fbfdb71e906fd1ac0574)
 2019-01-05 17:50:37 +09:00
Pranay Singh
a4f8421324 HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit ecdeaa7e6ad43555031aed032e6ba7a14a17d7bc)
(cherry picked from commit 1f39eae7e6f59206b86f96063ffb2ebe15a9cbe1)
(cherry picked from commit e8e55839a0c2b5479d7a25256ed1db751e09c122)
 2019-01-03 13:44:57 -08:00
Jonathan Hung
a0291a015c YARN-9175. Null resources check in ResourceInfo for branch-3.0 2019-01-03 15:58:10 -05:00
Weiwei Yang
c4e39e3a59 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 
(cherry picked from commit cfe89e6f963ba25b5fff1ce48cad36d74b3c789c)
 2019-01-04 02:00:59 +08:00
Weiwei Yang
f17ef6f451 HADOOP-15323. AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore. Contributed wujinhu. 
(cherry picked from commit 040a202b202a37f3b922cd321eb0a8ded457d88b)
 2019-01-03 22:04:47 +08:00
Ayush Saxena
eb1f44ee04
HADOOP-16025. Update the year to 2019. 
Contributed by Ayush Saxena.

(cherry picked from commit cb26f154289ed065a967886b8eac04794907d643)
 2019-01-02 22:25:57 +00:00
Weiwei Yang
8e0aa7860a MAPREDUCE-7174. Mapreduce example wordmedian should handle generic options. Contributed by Fei Hui. 
(cherry picked from commit 9e7196749ee025db7668c648743d1deb72d3ca28)
 2019-01-02 22:27:41 +08:00
Akira Ajisaka
2556b70c6b
HADOOP-16009. Replace the url of the repository in Apache Hadoop source code. 
(cherry picked from commit eee29ed7066e9401c7530b8b6f3b221bacef9a8a)
 2018-12-31 06:32:05 +09:00