Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,001 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

18001 Commits

Author SHA1 Message Date
Akira Ajisaka 524dc4e662
HADOOP-16225. Fix links to the developer mailing lists in DownstreamDev.md. Contributed by Wanqiang Ji. 
(cherry picked from commit ebd0d21538)
 2019-04-02 10:55:27 +09:00
Gabor Bota 461df1b511
HADOOP-16220. Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project 
This closes #665

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 53a86e2b8e)
 2019-04-01 13:53:11 +09:00
Takanobu Asanuma abb06c9d7c HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory. 
(cherry picked from commit 55fb3c32fb)
 2019-03-27 03:30:12 +09:00
Weiwei Yang 9b1f13fbd2 HADOOP-16191. AliyunOSS: improvements for copyFile/copyDirectory and logging. Contributed by wujinhu. 
(cherry picked from commit 568d3ab8b6)
 2019-03-19 10:13:57 +08:00
Erik Krogen e172fc62ce HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed. 
(cherry-picked from 8c95cb9d6b)
(cherry-picked from 0de8b55a09)
(cherry-picked from d4fbbc83ad)
 2019-03-18 08:57:41 -07:00
Inigo Goiri 5a8445789b HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun. 
(cherry picked from commit ff06ef0631)
 2019-03-15 13:59:51 -07:00
Erik Krogen 9a5daa5ec6 HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 
(cherry picked from commit 66357574ae)
(cherry picked from commit fec7c5f3eb)
(cherry picked from commit cb76fc5618)
 2019-03-13 13:21:40 -07:00
Steve Loughran 7f70a78ce3
HADOOP-16109. Parquet reading S3AFileSystem causes EOF (#589) 
Nobody gets seek right. No matter how many times they think they have.

Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran

Change-Id: I39b87f3d5daa98f65de2c0a44e348821a4930573
 2019-03-12 11:36:32 +00:00
Weiwei Yang 301ffee1bc MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang. 
(cherry picked from commit f0605146b3)
 2019-03-11 16:57:08 +08:00
Erik Krogen c924011f94 HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam. 
(cherry-picked from commit 1bc282e0b3)
(cherry-picked from commit 6d076dd5e8)
(cherry-picked from commit b0b8000240)
 2019-03-07 08:45:20 -08:00
Jason Lowe 32f96b6be4 YARN-5714. ContainerExecutor does not order environment map. Contributed by Remi Catherinot and Jim Brennan 
(cherry picked from commit 8e728f39c9)
 2019-03-07 15:10:52 +00:00
Wei-Chiu Chuang e23a448e0e HDFS-14314. fullBlockReportLeaseId should be reset after registering to NN. Contributed by star. 
(cherry picked from commit 387dbe587a)
(cherry picked from commit e58ccca3ce)
(cherry picked from commit d951497f57)
 2019-03-04 10:46:16 -08:00
Erik Krogen b170626c74 HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao. 2019-03-01 08:12:58 -08:00
Sunil G c139fe7a37 YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang. 
(cherry picked from commit 6cec90653d)
 2019-02-25 15:30:17 +05:30
Daryn Sharp 4eccf2a3cc HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a87e458432)
(cherry picked from commit ae8839e6e8)
(cherry picked from commit 704330a616)
 2019-02-20 18:22:20 -08:00
Adam Antal bebbdd290e
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d7)
 2019-02-15 18:59:02 +09:00
Masatake Iwasaki ba035dda76 YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 
(cherry picked from commit e0ab1bdece)
 2019-02-09 00:34:22 +09:00
Eric E Payne f3ff6d3aa7 YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 
(cherry picked from commit d1ca9432dd)
 2019-02-07 21:03:43 +00:00
Sunil G ec0bed1008 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 
(cherry picked from commit 6ffe6ea899)
 2019-02-07 19:09:19 +05:30
Kitti Nanasi ad131757f2 HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 49ddd8a6ed)
(cherry picked from commit 38541b2375)
(cherry picked from commit c25b641cec)
 2019-02-05 15:15:53 -08:00
Todd Lipcon 02cfd5fa35
HADOOP-15549. Upgrade to commons-configuration 2.1 regresses task CPU consumption. 
Contributed by Todd Lipcon.

(cherry picked from commit 1f21be2d02)
 2019-02-05 19:57:08 +00:00
Akira Ajisaka 875b8c390b
HADOOP-16089. AliyunOSS: update oss-sdk version to 3.4.1. Contributed by wujinhu. 
(cherry picked from commit 0f9aa5b4d5)
 2019-02-04 14:48:25 +09:00
Eric E Payne 50ef25464f YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb)
 2019-01-29 18:12:36 +00:00
Weiwei Yang d182c81938 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 
(cherry picked from commit 4f63ffe444)
 2019-01-29 11:07:51 +08:00
Masatake Iwasaki 00eee37a93 HDFS-12729. Document special paths in HDFS. Contributed by Masatake Iwasaki. 
(cherry picked from commit dc5af4c6b7)
 2019-01-29 02:31:59 +09:00
Jonathan Hung bce8f28bec YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e21)
(cherry picked from commit bf760e7e81)
(cherry picked from commit 80a08c29adc9d07974bbe8087abcec750d5f7916)
 2019-01-25 13:51:21 -08:00
Dinesh Chitlangia 36ff7a4d02 HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 45caeee6cf)
(cherry picked from commit eb77e60163)
(cherry picked from commit b63941516f)
 2019-01-25 07:38:49 -08:00
Haibo Chen 3368f141b2 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef)
(cherry picked from commit 3a764250262e745a6762dd6551b0d0a6af39e891)
 2019-01-24 15:58:27 -08:00
Sunil G d11fdbb7ac YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB 
(cherry picked from commit c726445990)
 2019-01-24 15:03:16 +05:30
Takanobu Asanuma 5b8a99bf49 HDFS-14218. EC: Ls -e throw NPE when directory ec policy is disabled. Contributed by Ayush Saxena. 
(cherry picked from commit 7d6792e5d2)
 2019-01-23 10:06:30 +09:00
Weiwei Yang 0e3d5f3064 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751)
 2019-01-22 11:15:03 +08:00
Chen Liang 80b9fecaaf HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian. 2019-01-18 11:45:22 -08:00
Akira Ajisaka 98345dcca4
HDFS-14213. Remove Jansson from BUILDING.txt. Contributed by Dinesh Chitlangia. 
(cherry picked from commit 96a84b61fb)
(cherry picked from commit dea75af9a7)
 2019-01-18 15:35:05 +09:00
Akira Ajisaka 1a38a61716
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 
(cherry picked from commit 104ef5df36)
 2019-01-16 14:38:55 +09:00
Inigo Goiri 174fa73f99 HADOOP-16045. Don't run TestDU on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 35fa3bd685)
 2019-01-11 18:16:44 -08:00
Akira Ajisaka 9fd5a6b66b
HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper 
(cherry picked from commit d4ca907da6)
 2019-01-11 13:05:31 +09:00
Jason Lowe 642213a74a Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh." 
This reverts commit a4f8421324.
 2019-01-09 17:56:32 -06:00
Kai Xie a959aa61b4
HADOOP-16018. DistCp won't reassemble chunks when blocks per chunk > 0. 
Contributed by Kai Xie.

(cherry picked from commit 188bebbe7e)
 2019-01-08 13:35:41 +00:00
Weiwei Yang 009f5b55ad YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed Wilfred Spiegelenburg. 
(cherry picked from commit 91e9c9f96e)
 2019-01-08 14:09:49 +08:00
Weiwei Yang ad5dd32843 HADOOP-16030. AliyunOSS: bring fixes back from HADOOP-15671. Contributed by wujinhu. 
(cherry picked from commit f87b3b11c4)
 2019-01-07 17:03:22 +08:00
Sunil G a450d58b49
Revert "HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu." 
This reverts commit e4fca6aae4.

Revert "HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu."

This reverts commit 2b635125fb.

(cherry picked from commit 1f425271a7)
 2019-01-05 17:50:37 +09:00
Pranay Singh a4f8421324 HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit ecdeaa7e6a)
(cherry picked from commit 1f39eae7e6)
(cherry picked from commit e8e55839a0)
 2019-01-03 13:44:57 -08:00
Jonathan Hung a0291a015c YARN-9175. Null resources check in ResourceInfo for branch-3.0 2019-01-03 15:58:10 -05:00
Weiwei Yang c4e39e3a59 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 
(cherry picked from commit cfe89e6f96)
 2019-01-04 02:00:59 +08:00
Weiwei Yang f17ef6f451 HADOOP-15323. AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore. Contributed wujinhu. 
(cherry picked from commit 040a202b20)
 2019-01-03 22:04:47 +08:00
Ayush Saxena eb1f44ee04
HADOOP-16025. Update the year to 2019. 
Contributed by Ayush Saxena.

(cherry picked from commit cb26f15428)
 2019-01-02 22:25:57 +00:00
Weiwei Yang 8e0aa7860a MAPREDUCE-7174. Mapreduce example wordmedian should handle generic options. Contributed by Fei Hui. 
(cherry picked from commit 9e7196749e)
 2019-01-02 22:27:41 +08:00
Akira Ajisaka 2556b70c6b
HADOOP-16009. Replace the url of the repository in Apache Hadoop source code. 
(cherry picked from commit eee29ed706)
 2018-12-31 06:32:05 +09:00
Wei-Chiu Chuang 054594c614 HDFS-14171. Performance improvement in Tailing EditLog. Contributed by Kenneth Yang. 
(cherry picked from commit e9a005dfcf)
(cherry picked from commit c48640ffbd)
(cherry picked from commit f24684ca9c)
 2018-12-28 10:39:47 -08:00
Jason Lowe c9fdf7503e HADOOP-15973. Configuration: Included properties are not cached if resource is a stream. Contributed by Eric Payne 2018-12-20 10:00:17 -06:00