Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,218 Commits 326 Branches 371 Tags
Commit Graph

14218 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Weiwei Yang
108f8b8fa1 YARN-7642. Add test case to verify context update after container promotion or demotion with or without auto update. Contributed by Weiwei Yang. 
(cherry picked from commit 89b6c482c1720a7f7ac86ce022c403825a086fa0)
 2017-12-15 12:33:41 +08:00
Subru Krishnan
2b77729fdf YARN-7630. Fix AMRMToken rollover handling in AMRMProxy. Contributed by Botong Huang. 
(cherry picked from commit 17ba74be29193ac15474f73baaaf4e647a95078b)
(cherry picked from commit 8d3d7fa1c5fda4ec394b2f64baf1008e7b0d9524)
 2017-12-14 14:11:07 -08:00
Kihwal Lee
43b9fbc2b4 HDFS-12907. Allow read-only access to reserved raw for non-superusers. Contributed by Rushabh S Shah. 
(cherry picked from commit 08a50da95fd8509d87c6a3e5b46862b8eb6f7318)
 2017-12-14 15:06:04 -06:00
Yiqun Lin
053a6fc126 HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 
(cherry picked from commit 91c96bdf8eb9a06193b719186b527563091d7666)
 2017-12-14 16:04:31 +08:00
Surendra Singh Lilhore
4f0ae1b108 HDFS-12833. Distcp : Update the usage of delete option for dependency with update and overwrite option. Contributed by usharani. 
(cherry picked from commit 8d69b7f2d41e6a76c8c706ead21b1abe86592881)
 2017-12-13 10:50:01 +05:30
Weiwei Yang
dff710632e YARN-7647. NM print inappropriate error log when node-labels is enabled. Contributed by Yang Wang. 2017-12-13 13:13:12 +08:00
Inigo Goiri
118d653eea HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 
(cherry picked from commit 5cd1056ad77a2ebb0466e7bc5933337b57f6fe30)
 2017-12-11 15:17:49 -08:00
Weiwei Yang
98eb4b2c6b YARN-7608. Incorrect sTarget column causing DataTable warning on RM application and scheduler web page. Contributed by Gergely Novák. 2017-12-11 10:37:22 +08:00
Wangda Tan
524ecb19ef YARN-7591. NPE in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) 
Change-Id: I67e086d6d35c3166ec0f6305490ea6527bd6d799
(cherry picked from commit 41796ea60ab9dbd6a41dd8fd3c6ff50990bba2f5)
(cherry picked from commit c27a5ba65c115f72a83c3bef9599579e38926aac)
 2017-12-08 21:57:26 -08:00
Subru Krishnan
39d0fdf1b2 YARN-6704. Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService. (Botong Huang via Subru). 
(cherry picked from commit 670e8d4ec7e71fc3b054cd3b2826f869b649a788)
(cherry picked from commit 850bd0ed7cebdddf982c8a586ee4d3ad833cfc57)
 2017-12-08 15:46:13 -08:00
Sammi Chen
4a064dd644 HADOOP-15024. AliyunOSS: Support user agent configuration and include that & Hadoop version information to oss server. 
(cherry picked from commit 94390fcd52d44aa678bfda835225f6b2ccff61b5)
 2017-12-08 22:37:17 +08:00
Sammi Chen
0f95a1d26a HADOOP-14997. Add hadoop-aliyun as dependency of hadoop-cloud-storage. Contributed by Genmao Yu. 
(cherry picked from commit 96adff737041774542aec97c5f036003e389389d)
 2017-12-08 20:15:32 +08:00
Sammi Chen
d06281486d HADOOP-15080. Aliyun OSS: update oss sdk from 2.8.1 to 2.8.3 to remove its dependency on Cat-x json-lib. 
(cherry picked from commit 67b2661e3d73a68ba7ca73b112bf6baea128631e)
(cherry picked from commit c225ac3f8f8cfd9686f6acf9cbd91705e0f0d661)
 2017-12-07 15:58:56 -07:00
Weiwei Yang
1a8f6bad34 YARN-7607. Remove the trailing duplicated timestamp in container diagnostics message. Contributed by Weiwei Yang. 2017-12-07 17:30:35 +08:00
Chris Douglas
ae3dc443f4 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 
(cherry picked from commit afc606c3aa9fc886ff2c17ec9a6a83e30c1adad0)
 2017-12-07 00:47:02 -08:00
Akira Ajisaka
89314b78dd
HDFS-12889. Addendum patch to add missing file. 
(cherry picked from commit 6cca5b3bcb440095f12d3eda88101fa250ac000a)
(cherry picked from commit 046424cf8ab31043cf3df254ae9ea093d20ad5da)
 2017-12-07 13:25:02 +09:00
Weiwei Yang
42deb54049 YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang. 
Previous commit 218d622529a45ea215cbf262221a06bc5b693f85 introduces a compile error,
  so it was reverted by commit db74ada80f4237726e15bd6def873081428a7210.
  This commit fixed the issue.

(cherry picked from commit 05c347fe51c01494ed8110f8f116a01c90205f13)
 2017-12-06 20:22:52 +08:00
Weiwei Yang
db74ada80f Revert "YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang." 
This reverts commit 218d622529a45ea215cbf262221a06bc5b693f85.
 2017-12-06 19:58:51 +08:00
Weiwei Yang
218d622529 YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang. 
(cherry picked from commit 05c347fe51c01494ed8110f8f116a01c90205f13)
 2017-12-06 12:57:22 +08:00
Akira Ajisaka
7fd6407886
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham. 
(cherry picked from commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b)
(cherry picked from commit 4e92b1e89e19547e2dbb508c86f9acea19ad7158)
 2017-12-06 11:43:08 +09:00
Akira Ajisaka
981f117a1b
HADOOP-14985. Remove subversion related code from VersionInfoMojo.java. Contributed by Ajay Kumar. 
(cherry picked from commit 9f1bdafedb60f83598819c6a682f659d6e168eb0)
 2017-12-05 14:32:34 +09:00
Xiao Chen
55df9df9fb HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 
(cherry picked from commit 274bbb193f34d6f9a31d3d2f55d1b66ada6b47d4)
 2017-12-04 10:45:30 -08:00
Arun Suresh
cc0c966a4e YARN-7587. Skip dispatching opportunistic containers to nodes whose queue is already full. (Weiwei Yang via asuresh) 
(cherry picked from commit 37ca4169508c3003dbe9044fefd37eb8cd8c0503)
 2017-12-03 22:25:06 -08:00
Jason Lowe
487c920bb0 YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim Brennan 
(cherry picked from commit 60f95fb719f00a718b484c36a823ec5aa8b099f4)
 2017-12-01 15:54:03 -06:00
Jason Lowe
74cfca02dd MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko 
(cherry picked from commit d0fc1cd0c8a92bc8eb37b3134104173e10b9b52f)
 2017-12-01 14:19:34 -06:00
Konstantin V Shvachko
0f5ec01ef7 HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 
(cherry picked from commit 60fd0d7fd73198fd610e59d1a4cd007c5fcc7205)
 2017-11-30 18:28:18 -08:00
Jason Lowe
5a8a1e66c6 YARN-7495. Improve robustness of the AggregatedLogDeletionService. Contributed by Jonathan Eagles 
(cherry picked from commit 5cfaee2e6db8b2ac55708de0968ff5539ee3bd76)
 2017-11-30 12:47:43 -06:00
Wangda Tan
a6d7fa91d9 YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) 
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
(cherry picked from commit 834e91ee91d22d74866afbf6252107e969bf8370)
(cherry picked from commit 94f7b0f99fb24efbacaa36ee8f9c68646ddef440)
 2017-11-29 15:48:51 -08:00
Wei Yan
17289ca677 YARN-6851. Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat. 
(cherry picked from commit 333ef303ff0caf9adfd378652a8f966377901768)
 2017-11-29 14:48:19 -08:00
Kihwal Lee
7d2b13f0be HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 
(cherry picked from commit 53509f295b5274059541565d7216bf98aa35347d)
 2017-11-29 15:16:17 -06:00
Wei Yan
5e31b109c6 HDFS-12835. Fix the javadoc errors in Router-based federation. 
(cherry picked from commit 301641811d93ac22dc6fe1a05f18c1f266cc5e54)
 2017-11-29 09:49:51 -08:00
Konstantin V Shvachko
3f9f0055eb HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 
(cherry picked from commit d331762f24b3f22f609366740c9c4f449edc61ac)
 2017-11-28 18:11:38 -08:00
Yiqun Lin
83c40758fa HDFS-12858. RBF: Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 
(cherry picked from commit 3d73d2f097aee130c93270d399cad3ff793b57f2)
 2017-11-28 13:46:02 +08:00
Kihwal Lee
3516ef45f3 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 
(cherry picked from commit 45f59bde60a21138fdb0fb846588db422d1c97a5)
 2017-11-27 17:01:34 -06:00
Sammi Chen
b756beb679 HADOOP-14964. AliyunOSS: backport Aliyun OSS module to branch-2. Contributed by Sammi Chen. 
The consolidated commits in this backport are as follows:
        HADOOP-14787. AliyunOSS: Implement the `createNonRecursive` operator.
        HADOOP-14649. Update aliyun-sdk-oss version to 2.8.1. (Genmao Yu via rchiang)
        HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu
        HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang.
        HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu.
        HADOOP-14192. AliyunOSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu
        HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu
        HADOOP-13769. AliyunOSS: update oss sdk version. Contributed by Genmao Yu
        HADOOP-14069. AliyunOSS: listStatus returns wrong file info. Contributed by Fei Hui
        HADOOP-13768. AliyunOSS: handle the failure in the batch delete operation `deleteDirs`. Contributed by Genmao Yu
        HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. Contributed by Fei Hui
        HADOOP-14045. Aliyun OSS documentation missing from website. Contributed by Yiqun Lin.
        HADOOP-13723. AliyunOSSInputStream#read() should update read bytes stat correctly. Contributed by Mingliang Liu
        HADOOP-13624.  Rename TestAliyunOSSContractDispCp. Contributed by Genmao Yu
        HADOOP-13591. Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir. Contributed by Genmao Yu
        HADOOP-13481. User documents for Aliyun OSS FileSystem. Contributed by Genmao Yu.
        HADOOP-12756. Incorporate Aliyun OSS file system implementation. Contributed by Mingfei Shi and Lin Zhou

 (cherry picked from commit 30ab9b6aef2e3d31f2a8fc9211b5324b3d42f18e)
 2017-11-27 11:35:17 +08:00
Sammi Chen
12901cdcee Revert "HADOOP-14964. AliyunOSS: backport Aliyun OSS module to branch-2 and 2.8+ branches. Contributed by Sammi Chen." 
This reverts commit 32a88442d0f9e9860b1f179da586894cea6a9e10.
 2017-11-27 10:46:15 +08:00
Sammi Chen
32a88442d0 HADOOP-14964. AliyunOSS: backport Aliyun OSS module to branch-2 and 2.8+ branches. Contributed by Sammi Chen. 2017-11-24 14:06:01 +08:00
Haibo Chen
c08a0bcad8 YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. 
(cherry picked from commit db2651cc9dd90320907e5c510272520c582581a4)
 2017-11-21 13:29:00 -08:00
Akira Ajisaka
6388273539
HADOOP-15045. ISA-L bulid options are documented in branch-2. 
(cherry picked from commit 4eb4f37bcf47f420f0483051c42495e862062eea)
 2017-11-21 13:52:24 +09:00
Subru Krishnan
a6db6c9855 YARN-6128. Add support for AMRMProxy HA. (Botong Huang via Subru). 
(cherry picked from commit ed3109136100a21d971484f242d80f2a7e7d337d)
 2017-11-20 14:22:59 -08:00
bibinchundatt
49ba091f71 YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 
(cherry picked from commit ea8a121423a3d6b098129af5bbbec88f98550653)
 2017-11-21 00:32:49 +05:30
bibinchundatt
dc4132395e YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 
(cherry picked from commit 5c2986e58dfc86fada8554a3f48cd417a4458abc)
 2017-11-20 23:29:14 +05:30
Inigo Goiri
ee267565b5 HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 
(cherry picked from commit e182e777947a85943504a207deb3cf3ffc047910)
 2017-11-17 10:06:46 -08:00
Sunil G
9eab9a25c6 YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-17 19:42:00 +05:30
Varun Vasudev
f23f49d4ee YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
(cherry picked from commit 9cf9627e20258147ee0a13c938c6165406c160dd)
 2017-11-17 12:05:55 +05:30
Jason Lowe
e4344d45cb YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)
 2017-11-15 12:34:22 -06:00
Junping Du
9229fb3523 YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 
(cherry picked from commit a72dcb9cad7df7f7236092f5b86446c2ef4ea874)
 2017-11-14 12:50:21 -08:00
Arun Suresh
12bd1aaac5 HADOOP-15036. Update LICENSE.txt for HADOOP-14840. (asuresh) 
(cherry picked from commit f871b7541a5375eb117eafb9a091e4f59401231f)
 2017-11-13 14:38:32 -08:00
Arun Suresh
ed0658a579 Addendum patch for Configuration fix. (Jason Lowe via asuresh) 
(cherry picked from commit 4e847d63a39268d8a34e0f22ddb5e40c5ef71e3a)
 2017-11-13 13:50:44 -08:00
Arun Suresh
238a70c6c3 YARN-5049. Addendum patch for branch-2. (asuresh) 
(cherry picked from commit c153bed6e3b38d60815c3f62d96c604f23116eb3)
 2017-11-12 10:17:01 -08:00