Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,313 Commits 326 Branches 371 Tags
Commit Graph

14313 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiao Chen
c9d48352c2 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-12-07 21:20:33 -08:00
Sammi Chen
c225ac3f8f HADOOP-15080. Aliyun OSS: update oss sdk from 2.8.1 to 2.8.3 to remove its dependency on Cat-x json-lib. 
(cherry picked from commit 67b2661e3d73a68ba7ca73b112bf6baea128631e)
 2017-12-07 14:15:21 -07:00
Chris Douglas
afc606c3aa HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-07 00:43:21 -08:00
Akira Ajisaka
046424cf8a
HDFS-12889. Addendum patch to add missing file. 
(cherry picked from commit 6cca5b3bcb440095f12d3eda88101fa250ac000a)
 2017-12-07 13:24:45 +09:00
Akira Ajisaka
4e92b1e89e
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham. 
(cherry picked from commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b)
 2017-12-06 11:42:51 +09:00
Brahma Reddy Battula
ceb4043ae1 HDFS-11751. DFSZKFailoverController daemon exits with wrong status code. Contributed by Bharat Viswanadham 
(cherry picked from commit 6d16a99ae8821c13eec90132e2c63a96fce4b08a)
 2017-12-05 20:01:33 +05:30
Akira Ajisaka
05fda24b1b
HADOOP-14985. Remove subversion related code from VersionInfoMojo.java. Contributed by Ajay Kumar. 
(cherry picked from commit 9f1bdafedb60f83598819c6a682f659d6e168eb0)
 2017-12-05 14:31:44 +09:00
Robert Kanter
54b5da83ec YARN-5594. Handle old RMDelegationToken format when recovering RM (rkanter) 
(cherry picked from commit d8863fc16fa3cbcdda5b99f79386c43e4fae5917)
 2017-12-04 13:18:04 -08:00
Xiao Chen
274bbb193f HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:22 -08:00
Jason Lowe
3fb83cbeb2 YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim Brennan 
(cherry picked from commit 60f95fb719f00a718b484c36a823ec5aa8b099f4)
 2017-12-01 15:51:49 -06:00
Robert Kanter
edf9fd51fb YARN-4813. TestRMWebServicesDelegationTokenAuthentication.testDoAs fails intermittently (grepas via rkanter) 
(cherry picked from commit c83fe4491731c994a4867759d80db31d9c1cab60)
 2017-12-01 12:21:18 -08:00
Jason Lowe
d0fc1cd0c8 MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko 2017-12-01 14:15:25 -06:00
Konstantin V Shvachko
3c57defaa4 HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 
(cherry picked from commit 60fd0d7fd73198fd610e59d1a4cd007c5fcc7205)
 2017-11-30 19:26:18 -08:00
Junping Du
14f251fcd5 YARN-7558. yarn logs command fails to get logs for running containers if UI authentication is enabled. Contributed by Xuan Gong. 
(cherry picked from commit a409425986fc128bb54f656b05373201545f7213)
 2017-11-30 13:49:45 -08:00
Jason Lowe
ee9ce1d75a YARN-7495. Improve robustness of the AggregatedLogDeletionService. Contributed by Jonathan Eagles 
(cherry picked from commit 5cfaee2e6db8b2ac55708de0968ff5539ee3bd76)
 2017-11-30 12:44:00 -06:00
Wei Yan
5bb6329215 YARN-6851. Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat. 
(cherry picked from commit 333ef303ff0caf9adfd378652a8f966377901768)
 2017-11-29 14:46:47 -08:00
Kihwal Lee
6fd9d11173 HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 
(cherry picked from commit 53509f295b5274059541565d7216bf98aa35347d)
 2017-11-29 15:15:41 -06:00
Wei Yan
9452cac210 HDFS-12835. Fix the javadoc errors in Router-based federation. 
(cherry picked from commit 301641811d93ac22dc6fe1a05f18c1f266cc5e54)
 2017-11-29 10:12:15 -08:00
Konstantin V Shvachko
9d406e5dc8 HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 
(cherry picked from commit d331762f24b3f22f609366740c9c4f449edc61ac)
 2017-11-28 17:52:08 -08:00
Steve Loughran
5c37a0b841 HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. 
Contributed by Rajesh Balamohan

(cherry picked from commit 0ea182d0faa35c726dcb37249d48786bfc8ca04c)
 2017-11-28 11:57:51 +00:00
Yiqun Lin
3d73d2f097 HDFS-12858. RBF: Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 2017-11-28 13:42:41 +08:00
Yufei Gu
de144175a4 YARN-7363. ContainerLocalizer don't have a valid log4j config in case of Linux container executor. (Contributed by Yufei Gu) 
(cherry picked from commit d8923cdbf1567aee10a54f144fef734d1465ebed)
(cherry picked from commit c0ab3c8407b110dd9083d775e5ea823cbf73e7d4)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
 2017-11-27 14:54:27 -08:00
Kihwal Lee
45f59bde60 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 2017-11-27 16:52:18 -06:00
Wangda Tan
94f7b0f99f YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) 
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
(cherry picked from commit 834e91ee91d22d74866afbf6252107e969bf8370)
 2017-11-23 20:06:16 -08:00
Xiao Chen
a4674c3e40 HADOOP-15067. GC time percentage reported in JvmMetrics should be a gauge, not counter. Contributed by Misha Dmitriev. 
(cherry picked from commit d162252d7a7223631ff66ba0210953296407e55f)
 2017-11-23 09:02:23 -08:00
Haibo Chen
db2651cc9d YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. 2017-11-21 11:08:27 -08:00
Akira Ajisaka
4eb4f37bcf
HADOOP-15045. ISA-L bulid options are documented in branch-2. 2017-11-21 13:50:49 +09:00
Tsz-Wo Nicholas Sze
ed879d4a72 HDFS-12813. RequestHedgingProxyProvider can hide Exception thrown from the Namenode for proxy size of 1. Contributed by Mukul Kumar Singh 2017-11-20 17:11:18 -08:00
Subru Krishnan
ed31091361 YARN-6128. Add support for AMRMProxy HA. (Botong Huang via Subru). 2017-11-20 14:21:58 -08:00
bibinchundatt
ea8a121423 YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 2017-11-21 00:31:15 +05:30
bibinchundatt
5c2986e58d YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 2017-11-20 21:31:50 +05:30
Inigo Goiri
b57ea38fcf HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 
(cherry picked from commit e182e777947a85943504a207deb3cf3ffc047910)
 2017-11-17 10:05:45 -08:00
Varun Vasudev
9cf9627e20 YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
 2017-11-17 12:05:35 +05:30
Robert Kanter
8db49df4a2 HADOOP-14982. Clients using FailoverOnNetworkExceptionRetry can go into a loop if they're used without authenticating with kerberos in HA env (pbacsko via rkanter) 
(cherry picked from commit f2efaf013f7577948061abbb49c6d17c375e92cc)
 2017-11-16 11:12:01 -08:00
Sunil G
a081d91480 YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-16 22:42:52 +05:30
Xiao Chen
ba2eab9122 HADOOP-15023. ValueQueue should also validate (int) (lowWatermark * numValues) > 0 on construction. 
(cherry picked from commit b1941b200d6b4fa6a7891421c0a1e212cad3d6eb)
 2017-11-15 16:45:06 -08:00
Jason Lowe
ec4159c4df YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)
 2017-11-15 12:33:48 -06:00
Junping Du
a72dcb9cad YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 2017-11-13 18:22:30 -08:00
Arun Suresh
f894eefec1 HADOOP-15036. Update LICENSE.txt for HADOOP-14840. (asuresh) 
(cherry picked from commit f871b7541a5375eb117eafb9a091e4f59401231f)
 2017-11-13 14:39:07 -08:00
Arpit Agarwal
f671c22e3e HDFS-12705. WebHdfsFileSystem exceptions should retain the caused by exception. Contributed by Hanisha Koneru. 2017-11-13 13:56:13 -08:00
Arun Suresh
4e847d63a3 Addendum patch for Configuration fix. (Jason Lowe via asuresh) 2017-11-13 13:49:08 -08:00
Kai Zheng
30ab9b6aef HADOOP-14964. AliyunOSS: backport Aliyun OSS module to branch-2. Contributed by Sammi Chen 
The consolidated commits in this backport are as follows:
    HADOOP-14787. AliyunOSS: Implement the `createNonRecursive` operator.
    HADOOP-14649. Update aliyun-sdk-oss version to 2.8.1. (Genmao Yu via rchiang)
    HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu
    HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang.
    HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu.
    HADOOP-14192. AliyunOSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu
    HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu
    HADOOP-13769. AliyunOSS: update oss sdk version. Contributed by Genmao Yu
    HADOOP-14069. AliyunOSS: listStatus returns wrong file info. Contributed by Fei Hui
    HADOOP-13768. AliyunOSS: handle the failure in the batch delete operation `deleteDirs`. Contributed by Genmao Yu
    HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. Contributed by Fei Hui
    HADOOP-14045. Aliyun OSS documentation missing from website. Contributed by Yiqun Lin.
    HADOOP-13723. AliyunOSSInputStream#read() should update read bytes stat correctly. Contributed by Mingliang Liu
    HADOOP-13624.  Rename TestAliyunOSSContractDispCp. Contributed by Genmao Yu
    HADOOP-13591. Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir. Contributed by Genmao Yu
    HADOOP-13481. User documents for Aliyun OSS FileSystem. Contributed by Genmao Yu.
    HADOOP-12756. Incorporate Aliyun OSS file system implementation. Contributed by Mingfei Shi and Lin Zhou
 2017-11-13 09:29:33 +08:00
Arun Suresh
c153bed6e3 YARN-5049. Addendum patch for branch-2. (asuresh) 2017-11-12 10:16:00 -08:00
Eric Yang
90ac8cd5a1 YARN-7412. Fix unit test for docker mount check on ubuntu. (Contributed by Eric Badger) 
(cherry picked from commit 7a49ddfdde2e2a7b407f4a62a42d97bfe456075a)
 2017-11-12 09:59:39 -08:00
Subru Krishnan
0282efab14 YARN-7476. Fix miscellaneous issues in ATSv2 after merge to branch-2. (Varun Saxena via Subru). 2017-11-12 09:19:30 -08:00
Subru Krishnan
3c5b46c2ed YARN-7475. Fix Container log link in new YARN UI. (Sunil G via Subru). 2017-11-12 09:18:08 -08:00
Arun Suresh
5cd92ed555 YARN-6627. Addendum patch for branch-2. (Sunil G via asuresh) 2017-11-11 09:10:17 -08:00
Subru Krishnan
132c2e7a2a HADOOP-15030. [branch-2] Include hadoop-cloud-storage-project in the main hadoop pom modules. 2017-11-10 13:48:46 -08:00
Arun Suresh
0411c710d2 YARN-6102. Addendum patch for branch-2. (Rohith Sharma K S via asuresh) 2017-11-10 13:44:03 -08:00
Xiao Chen
39fb402da4 HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev. 
(cherry picked from commit 3c6adda291745c592741b87cd613214ae11887e4)
 2017-11-09 21:06:55 -08:00