Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,509 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

20509 Commits

Author SHA1 Message Date
Inigo Goiri 4eb0497091 HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun. 
(cherry picked from commit ff06ef0631)
 2019-03-15 13:58:03 -07:00
Ben Roling 43e8ac6097
HADOOP-15625. S3A input stream to use etags/version number to detect changed source files. 
Author: Ben Roling <ben.roling@gmail.com>

Initial patch from Brahma Reddy Battula.
 2019-03-14 19:46:34 +00:00
Erik Krogen fec7c5f3eb HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 
(cherry picked from commit 66357574ae)
 2019-03-13 13:19:18 -07:00
Weiwei Yang 3d86dd8931 MAPREDUCE-7192. JobHistoryServer attempts page support jump to containers log page in NM when logAggregation is disable. Contributed by Jiandan Yang. 
(cherry picked from commit 159a715eef)
 2019-03-13 17:41:31 +08:00
Shweta Yakkali 1ceefa726e HDFS-14081. hdfs dfsadmin -metasave metasave_test results NPE. Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 1bea785020)
 2019-03-12 16:05:55 -07:00
Stephen O'Donnell a21e2e4dbc HDFS-14333. Datanode fails to start if any disk has errors during Namenode registration. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 34b14061b3)
 2019-03-12 10:18:56 -07:00
Steve Loughran b6f6c34223
HADOOP-16109. Parquet reading S3AFileSystem causes EOF 
Nobody gets seek right. No matter how many times they think they have.

Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran
 2019-03-11 11:15:25 +00:00
Weiwei Yang 9a2b24642d MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang. 
(cherry picked from commit f0605146b3)
 2019-03-11 16:03:38 +08:00
Da Zhou cfaf21a4ba
HADOOP-16169. ABFS: Bug fix for getPathProperties. 
Author:    Da Zhou <da.zhou@microsoft.com>
(cherry picked from commit e0260417ad)
 2019-03-08 13:53:44 +00:00
Erik Krogen 6d076dd5e8 HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam. 
(cherry-picked from commit 1bc282e0b3)
 2019-03-07 08:42:41 -08:00
Praveen Krishna 451844fee5
HADOOP-16114. NetUtils#canonicalizeHost gives different value for same host. 
Author:    Praveen Krishna <praveenkrishna@tutanota.com>
(cherry picked from commit 2b94e51a8f)
 2019-03-07 11:08:48 +00:00
Sunil G aff5973401 YARN-8803. [UI2] Show flow runs in the order of recently created time in graph widgets. Contributed by Akhil PB. 
(cherry picked from commit c79f139519)
 2019-03-06 16:49:49 +05:30
Sunil G d721634fea YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth. 
(cherry picked from commit 46045c5cb3)
 2019-03-06 16:01:56 +05:30
Stephen O'Donnell 3fe31b36fa
HADOOP-16140. hadoop fs expunge to add -immediate option to purge trash immediately. 
Contributed by Stephen O'Donnell.

(cherry picked from commit 686c0141ef)
Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-03-05 14:11:49 +00:00
Da Zhou dc38fc598d
HADOOP-16136. ABFS: Should only transform username to short name 
Contributed by Da Zhou.

(cherry picked from commit 3988e75ca3)
Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-03-05 10:47:58 +00:00
Da Zhou 075f6b061c
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. 
Contributed by Da Zhou and Junhua Gu.

(cherry picked from commit 1f1655028e)
Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-03-05 10:44:46 +00:00
Da Zhou ae832ccffe
HADOOP-16041. Include Hadoop version in User-Agent string for ABFS. 
Contributed by Shweta Yakkali.

Signed-off-by: Sean Mackrory <mackrorysd@apache.org>
(cherry picked from commit 02eb91856e)
Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-03-05 10:39:37 +00:00
Wei-Chiu Chuang e58ccca3ce HDFS-14314. fullBlockReportLeaseId should be reset after registering to NN. Contributed by star. 
(cherry picked from commit 387dbe587a)
 2019-03-04 10:45:31 -08:00
bibinchundatt 63ed16e076 Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph" 
This reverts commit cf1944eb6e.
 2019-03-04 17:01:40 +05:30
Weiwei Yang 4ceb4e4f05 YARN-9332. RackResolver tool should accept multiple hosts. Contributed by Lantao Jin. 
(cherry picked from commit fe6b2b2f23e69f0643e870d9c500117088983209)
 2019-03-02 16:04:24 +00:00
Erik Krogen af16db86d4 HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao. 2019-03-01 08:12:44 -08:00
Sunil G d045f02a8d YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth. 2019-03-01 19:27:03 +05:30
Eric Yang 3f3548b66a YARN-9334. Allow YARN Service client to send SPNEGO challenge header when authentication type is not simple. 
Contributed by Billie Rinaldi

(cherry picked from commit 04b228e43b)
 2019-02-28 09:33:05 -08:00
Weiwei Yang 7575e3090d YARN-9324. TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11. Contributed by Prabhu Joseph. 2019-02-28 09:32:07 +08:00
Weiwei Yang 7fa5373ec4 YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie. 
(cherry picked from commit 8c30114b00)
 2019-02-27 17:35:09 +08:00
Sunil G 809e3f2453 YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for easy access control. Contributed by Szilard Nemeth. 
(cherry picked from commit 5e91ebd91a)
 2019-02-27 12:03:58 +05:30
Sunil G a95a0cbf2f YARN-9087. Improve logging for initialization of Resource plugins. Contributed by Szilard Nemeth. 2019-02-27 11:54:43 +05:30
Weiwei Yang bdde6a612e YARN-9329. updatePriority is blocked when using FairScheduler. Contributed by Jiandan Yang. 
(cherry picked from commit 3e1739d589)
 2019-02-26 00:18:24 +08:00
Sunil G 359e459df1 YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang. 
(cherry picked from commit 6cec90653d)
 2019-02-25 15:29:31 +05:30
Sunil G f282f9c362 YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth. 2019-02-25 11:37:42 +05:30
Weiwei Yang cdce1c17a0 YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph. 
(cherry picked from commit 9cd5c5447f)
 2019-02-24 22:48:55 +08:00
Weiwei Yang 604a915bab YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie. 
(cherry picked from commit 9c88695bcd)
 2019-02-24 22:21:53 +08:00
bibinchundatt 3e1bd53a37 YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph 2019-02-23 07:59:51 +05:30
bibinchundatt cf1944eb6e YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph 2019-02-22 20:51:47 +05:30
Sunil G d75aa33612 YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth. 
(cherry picked from commit 95fbbfed75)
 2019-02-22 20:23:24 +05:30
Weiwei Yang fb2735de16 YARN-9258. Support to specify allocation tags without constraint in distributed shell CLI. Contributed by Prabhu Joseph. 
(cherry picked from commit 6c4ab0312b)
 2019-02-22 00:23:21 +08:00
Surendra Singh Lilhore 2e939515df HDFS-14216. NullPointerException happens in NamenodeWebHdfs. Contributed by lujie. 
(cherry picked from commit 92b53c40f0)
 2019-02-21 20:47:50 +05:30
Steve Loughran 685a41f449
HADOOP-16105. WASB in secure mode does not set connectingUsingSAS. 
Contributed by Steve Loughran.

(cherry picked from commit 9cb2f470b759bbe7609a00e8f8f72779e2daae80)
 2019-02-21 13:39:37 +00:00
Weiwei Yang c2ef443359 YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph. 2019-02-21 18:06:26 +08:00
Daryn Sharp ae8839e6e8 HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a87e458432)
 2019-02-20 18:21:14 -08:00
Masatake Iwasaki dc9c3ce30b HADOOP-16104. Wasb tests to downgrade to skip when test a/c is namespace enabled. Contributed by Masatake Iwasaki. 
(cherry picked from commit aa3ad36605)
 2019-02-20 22:17:18 +09:00
Surendra Singh Lilhore b93b127956 HDFS-14235. Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon. Contributed by Ranith Sardar. 
(cherry picked from commit 41e18feda3)
 2019-02-20 17:12:58 +05:30
bibinchundatt e6f2b8730f YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T. 
(cherry picked from commit b8de78c570)
 2019-02-20 01:20:15 +05:30
bibinchundatt bdfdf12178 YARN-9309. Improve graph text in SLS to avoid overlapping. Contributed by Bilwa S T. 
(cherry picked from commit 779dae4de7)
 2019-02-20 00:37:47 +05:30
Sunil G a060e8cb51 YARN-7824. [UI2] Yarn Component Instance page should include link to container logs. Contributed by Akhil PB. 2019-02-17 20:19:46 +05:30
Adam Antal 830aaac023
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d7)
 2019-02-15 18:48:21 +09:00
Sunil G b4dc62a120 YARN-8295. [UI2] Improve Resource Usage tab error message when there are no data available. Contributed by Charan Hebri. 
(cherry picked from commit 5b55f3538c)
 2019-02-15 12:43:01 +05:30
Akira Ajisaka f0b476bb31
YARN-9284. Fix the unit of yarn.service.am-resource.memory in the document. Contributed by Masahiro Tanaka. 
(cherry picked from commit 3a39d9a2d2)
 2019-02-15 15:43:32 +09:00
Akira Ajisaka 3e3275a80b
YARN-9308. fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit dabfeab785)
 2019-02-15 14:53:36 +09:00
bibinchundatt a8563d23f5 YARN-9295. [UI2] Fix label typo in Cluster Overview page. Contributed by Charan Hebri. 
(cherry picked from commit b66d5ae9e2)
 2019-02-14 23:10:21 +05:30