Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,201 Commits 326 Branches 371 Tags
Commit Graph

14201 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Weiwei Yang
db74ada80f Revert "YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang." 
This reverts commit 218d622529a45ea215cbf262221a06bc5b693f85.
 2017-12-06 19:58:51 +08:00
Weiwei Yang
218d622529 YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang. 
(cherry picked from commit 05c347fe51c01494ed8110f8f116a01c90205f13)
 2017-12-06 12:57:22 +08:00
Akira Ajisaka
7fd6407886
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham. 
(cherry picked from commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b)
(cherry picked from commit 4e92b1e89e19547e2dbb508c86f9acea19ad7158)
 2017-12-06 11:43:08 +09:00
Akira Ajisaka
981f117a1b
HADOOP-14985. Remove subversion related code from VersionInfoMojo.java. Contributed by Ajay Kumar. 
(cherry picked from commit 9f1bdafedb60f83598819c6a682f659d6e168eb0)
 2017-12-05 14:32:34 +09:00
Xiao Chen
55df9df9fb HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 
(cherry picked from commit 274bbb193f34d6f9a31d3d2f55d1b66ada6b47d4)
 2017-12-04 10:45:30 -08:00
Arun Suresh
cc0c966a4e YARN-7587. Skip dispatching opportunistic containers to nodes whose queue is already full. (Weiwei Yang via asuresh) 
(cherry picked from commit 37ca4169508c3003dbe9044fefd37eb8cd8c0503)
 2017-12-03 22:25:06 -08:00
Jason Lowe
487c920bb0 YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim Brennan 
(cherry picked from commit 60f95fb719f00a718b484c36a823ec5aa8b099f4)
 2017-12-01 15:54:03 -06:00
Jason Lowe
74cfca02dd MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko 
(cherry picked from commit d0fc1cd0c8a92bc8eb37b3134104173e10b9b52f)
 2017-12-01 14:19:34 -06:00
Konstantin V Shvachko
0f5ec01ef7 HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 
(cherry picked from commit 60fd0d7fd73198fd610e59d1a4cd007c5fcc7205)
 2017-11-30 18:28:18 -08:00
Jason Lowe
5a8a1e66c6 YARN-7495. Improve robustness of the AggregatedLogDeletionService. Contributed by Jonathan Eagles 
(cherry picked from commit 5cfaee2e6db8b2ac55708de0968ff5539ee3bd76)
 2017-11-30 12:47:43 -06:00
Wangda Tan
a6d7fa91d9 YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) 
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
(cherry picked from commit 834e91ee91d22d74866afbf6252107e969bf8370)
(cherry picked from commit 94f7b0f99fb24efbacaa36ee8f9c68646ddef440)
 2017-11-29 15:48:51 -08:00
Wei Yan
17289ca677 YARN-6851. Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat. 
(cherry picked from commit 333ef303ff0caf9adfd378652a8f966377901768)
 2017-11-29 14:48:19 -08:00
Kihwal Lee
7d2b13f0be HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 
(cherry picked from commit 53509f295b5274059541565d7216bf98aa35347d)
 2017-11-29 15:16:17 -06:00
Wei Yan
5e31b109c6 HDFS-12835. Fix the javadoc errors in Router-based federation. 
(cherry picked from commit 301641811d93ac22dc6fe1a05f18c1f266cc5e54)
 2017-11-29 09:49:51 -08:00
Konstantin V Shvachko
3f9f0055eb HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 
(cherry picked from commit d331762f24b3f22f609366740c9c4f449edc61ac)
 2017-11-28 18:11:38 -08:00
Yiqun Lin
83c40758fa HDFS-12858. RBF: Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 
(cherry picked from commit 3d73d2f097aee130c93270d399cad3ff793b57f2)
 2017-11-28 13:46:02 +08:00
Kihwal Lee
3516ef45f3 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 
(cherry picked from commit 45f59bde60a21138fdb0fb846588db422d1c97a5)
 2017-11-27 17:01:34 -06:00
Sammi Chen
b756beb679 HADOOP-14964. AliyunOSS: backport Aliyun OSS module to branch-2. Contributed by Sammi Chen. 
The consolidated commits in this backport are as follows:
        HADOOP-14787. AliyunOSS: Implement the `createNonRecursive` operator.
        HADOOP-14649. Update aliyun-sdk-oss version to 2.8.1. (Genmao Yu via rchiang)
        HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu
        HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang.
        HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu.
        HADOOP-14192. AliyunOSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu
        HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu
        HADOOP-13769. AliyunOSS: update oss sdk version. Contributed by Genmao Yu
        HADOOP-14069. AliyunOSS: listStatus returns wrong file info. Contributed by Fei Hui
        HADOOP-13768. AliyunOSS: handle the failure in the batch delete operation `deleteDirs`. Contributed by Genmao Yu
        HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. Contributed by Fei Hui
        HADOOP-14045. Aliyun OSS documentation missing from website. Contributed by Yiqun Lin.
        HADOOP-13723. AliyunOSSInputStream#read() should update read bytes stat correctly. Contributed by Mingliang Liu
        HADOOP-13624.  Rename TestAliyunOSSContractDispCp. Contributed by Genmao Yu
        HADOOP-13591. Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir. Contributed by Genmao Yu
        HADOOP-13481. User documents for Aliyun OSS FileSystem. Contributed by Genmao Yu.
        HADOOP-12756. Incorporate Aliyun OSS file system implementation. Contributed by Mingfei Shi and Lin Zhou

 (cherry picked from commit 30ab9b6aef2e3d31f2a8fc9211b5324b3d42f18e)
 2017-11-27 11:35:17 +08:00
Sammi Chen
12901cdcee Revert "HADOOP-14964. AliyunOSS: backport Aliyun OSS module to branch-2 and 2.8+ branches. Contributed by Sammi Chen." 
This reverts commit 32a88442d0f9e9860b1f179da586894cea6a9e10.
 2017-11-27 10:46:15 +08:00
Sammi Chen
32a88442d0 HADOOP-14964. AliyunOSS: backport Aliyun OSS module to branch-2 and 2.8+ branches. Contributed by Sammi Chen. 2017-11-24 14:06:01 +08:00
Haibo Chen
c08a0bcad8 YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. 
(cherry picked from commit db2651cc9dd90320907e5c510272520c582581a4)
 2017-11-21 13:29:00 -08:00
Akira Ajisaka
6388273539
HADOOP-15045. ISA-L bulid options are documented in branch-2. 
(cherry picked from commit 4eb4f37bcf47f420f0483051c42495e862062eea)
 2017-11-21 13:52:24 +09:00
Subru Krishnan
a6db6c9855 YARN-6128. Add support for AMRMProxy HA. (Botong Huang via Subru). 
(cherry picked from commit ed3109136100a21d971484f242d80f2a7e7d337d)
 2017-11-20 14:22:59 -08:00
bibinchundatt
49ba091f71 YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 
(cherry picked from commit ea8a121423a3d6b098129af5bbbec88f98550653)
 2017-11-21 00:32:49 +05:30
bibinchundatt
dc4132395e YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 
(cherry picked from commit 5c2986e58dfc86fada8554a3f48cd417a4458abc)
 2017-11-20 23:29:14 +05:30
Inigo Goiri
ee267565b5 HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 
(cherry picked from commit e182e777947a85943504a207deb3cf3ffc047910)
 2017-11-17 10:06:46 -08:00
Sunil G
9eab9a25c6 YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-17 19:42:00 +05:30
Varun Vasudev
f23f49d4ee YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
(cherry picked from commit 9cf9627e20258147ee0a13c938c6165406c160dd)
 2017-11-17 12:05:55 +05:30
Jason Lowe
e4344d45cb YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)
 2017-11-15 12:34:22 -06:00
Junping Du
9229fb3523 YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 
(cherry picked from commit a72dcb9cad7df7f7236092f5b86446c2ef4ea874)
 2017-11-14 12:50:21 -08:00
Arun Suresh
12bd1aaac5 HADOOP-15036. Update LICENSE.txt for HADOOP-14840. (asuresh) 
(cherry picked from commit f871b7541a5375eb117eafb9a091e4f59401231f)
 2017-11-13 14:38:32 -08:00
Arun Suresh
ed0658a579 Addendum patch for Configuration fix. (Jason Lowe via asuresh) 
(cherry picked from commit 4e847d63a39268d8a34e0f22ddb5e40c5ef71e3a)
 2017-11-13 13:50:44 -08:00
Arun Suresh
238a70c6c3 YARN-5049. Addendum patch for branch-2. (asuresh) 
(cherry picked from commit c153bed6e3b38d60815c3f62d96c604f23116eb3)
 2017-11-12 10:17:01 -08:00
Eric Yang
45c5b7f71d YARN-7412. Fix unit test for docker mount check on ubuntu. (Contributed by Eric Badger) 
(cherry picked from commit 7a49ddfdde2e2a7b407f4a62a42d97bfe456075a)
(cherry picked from commit 90ac8cd5a1c056d535eee92b148132c1674c5523)
 2017-11-12 10:13:49 -08:00
Subru Krishnan
8bcd1d62c8 YARN-7476. Fix miscellaneous issues in ATSv2 after merge to branch-2. (Varun Saxena via Subru). 
(cherry picked from commit 0282efab147203e7323450fa18704af46bc524ad)
 2017-11-12 09:25:12 -08:00
Subru Krishnan
8251601003 YARN-7475. Fix Container log link in new YARN UI. (Sunil G via Subru). 
(cherry picked from commit 3c5b46c2edd69bb238d635ae61ff91656dec23df)
 2017-11-12 09:25:12 -08:00
Arun Suresh
45aabb3c8d YARN-6627. Addendum patch for branch-2. (Sunil G via asuresh) 
(cherry picked from commit 5cd92ed555cadf4897e4ab378fcc6691bfbd6bbe)
 2017-11-11 09:11:43 -08:00
Subru Krishnan
b35c5ba472 HADOOP-15030. [branch-2] Include hadoop-cloud-storage-project in the main hadoop pom modules. 
(cherry picked from commit 132c2e7a2a2dcb8050770a85aca8e66a7c33934b)
 2017-11-10 13:52:39 -08:00
Arun Suresh
dc65df2a1a YARN-6102. Addendum patch for branch-2. (Rohith Sharma K S via asuresh) 
(cherry picked from commit 0411c710d2ea553fe783075c5299a013f89902cc)
 2017-11-10 13:45:14 -08:00
Arun Suresh
d754b24738 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 
(cherry picked from commit 7af9b8ad1e993ef791aa38740b6aabc4c233a30f)
 2017-11-09 15:17:59 -08:00
Haibo Chen
288d9506f0 YARN-7388. TestAMRestart should be scheduler agnostic. 
(cherry picked from commit a1382a18dff8a70aa25240d6fbba6e22832a7679)
(cherry picked from commit 91a7f8d2463de0a94ded0884143e4661c29868a4)
(cherry picked from commit 3fbe67d3e1a93b998e3d056ebd9008e17a35f8b0)
 2017-11-09 11:02:05 -08:00
John Zhuge
120d868963 HDFS-12318. Fix IOException condition for openInfo in DFSInputStream. Contributed by legend. 
(cherry picked from commit 3123895db0518552eb357093fddf0e8ff1a6cadf)
 2017-11-09 09:18:53 -08:00
Subru Krishnan
26110ec335 HADOOP-15026. Rebase ResourceEstimator start/stop scripts for branch-2. (Rui Li via Subru). 
(cherry picked from commit 5991c218a0cd72fc6a2b10ef77729ee6181c9443)
 2017-11-09 00:33:23 -08:00
Subru Krishnan
d45672a144 HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru). 
(cherry picked from commit f2df6b8983aace73ad27934bd9f7f4d766e0b25f)
(cherry picked from commit 46a740a82ebb5143038296bbedc9d455f68bae63)
 2017-11-08 18:11:32 -08:00
Arun Suresh
1ddd0a7958 YARN-7453. Fix issue where RM fails to switch to active after first successful start. (Rohith Sharma K S via asuresh) 
(cherry picked from commit a9c70b0e84dab0c41e480a0dc0cb1a22efdc64ee)
(cherry picked from commit 5486542074a7025061a427df968755f53c692477)
 2017-11-08 08:08:26 -08:00
Brahma Reddy Battula
ce027c6f48 HDFS-12788. Reset the upload button when file upload fails. Contributed by Brahma Reddy Battula 
(cherry picked from commit 410d0319cf72b9b5f3807c522237f52121d98cd5)
 2017-11-08 14:46:27 +05:30
Brahma Reddy Battula
53891b7f8f HDFS-12783. [branch-2] dfsrouter should use hdfsScript. Contributed by Brahma Reddy Battula 
(cherry picked from commit 135abcac4ba3fda0e287b0766c415ce4e0c37931)
 2017-11-08 10:19:58 +05:30
Subru Krishnan
8746a65e78 YARN-6932. Fix TestFederationRMFailoverProxyProvider test case failure. 
(cherry picked from commit 0542e6f86e6734f8ceea00960bebe719827136db)
 2017-11-03 11:15:48 -07:00
Arun Suresh
bb6a1aa5fa Preparing for 2.9.1 development 2017-11-02 22:49:38 -07:00
Inigo Goiri
6f5a95ff98 YARN-7434. Router getApps REST invocation fails with multiple RMs. Contributed by Inigo Goiri. 
(cherry picked from commit 6e2259264ad9525eeec2a14055d53236711659d7)
 2017-11-02 21:35:45 -07:00