Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,607 Commits 326 Branches 371 Tags
Commit Graph

14607 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Konstantin V Shvachko
64681a362d HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 
(cherry picked from commit a4121c71c29d13866a605d9c0d013e5de9c147c3)
 2018-08-24 19:00:21 -07:00
Jason Lowe
c4e3df2261 YARN-8649. NPE in localizer hearbeat processing if a container is killed while localizing. Contributed by lujie 
(cherry picked from commit 585ebd873a55bedd2a364d256837f08ada8ba032)
 2018-08-23 09:43:03 -05:00
Takanobu Asanuma
947b9a27e2 HADOOP-14314. The OpenSolaris taxonomy link is dead in InterfaceClassification.md. Contributed by Rui Gao. 
(cherry picked from commit bed8cb6979e0460141ed77e3b15d4f18db098a8e)
 2018-08-23 21:56:30 +09:00
Takanobu Asanuma
ac7eef4131 HADOOP-15689. Add "*.patch" into .gitignore file of branch-2. Contributed by Rui Gao. 2018-08-23 00:39:03 +09:00
Yiqun Lin
8e09ff918e HDFS-13821. RBF: Add dfs.federation.router.mount-table.cache.enable so that users can disable cache. Contributed by Fei Hui. 
(cherry picked from commit 81847392badcd58d934333e7c3b5bf14b4fa1f3f)
 2018-08-22 11:48:34 +08:00
Giovanni Matteo Fumarola
8e6807ef4a YARN-8673. [AMRMProxy] More robust responseId resync after an YarnRM master slave switch. Contributed by Botong Huang. 2018-08-21 13:09:33 -07:00
Giovanni Matteo Fumarola
89da0e9901 YARN-8581. [AMRMProxy] Add sub-cluster timeout in LocalityMulticastAMRMProxyPolicy. Contributed by Botong Huang. 2018-08-21 13:04:49 -07:00
Rohith Sharma K S
1a53aab4d6 YARN-8129. Improve error message for invalid value in fields attribute. Contributed by Abhishek Modi. 
(cherry picked from commit d3fef7a5c5b83d27e87b5e49928254a7d1b935e5)
 2018-08-21 12:11:31 +05:30
Yiqun Lin
18ebe18305 HDFS-13750. RBF: Router ID in RouterRpcClient is always null. Contributed by Takanobu Asanuma. 2018-08-20 23:08:26 +08:00
Rohith Sharma K S
c68d1d49ca YARN-8679. [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked. Contributed by Wangda Tan. 
(cherry picked from commit 4aacbfff605262aaf3dbd926258afcadc86c72c0)
 2018-08-18 11:04:20 +05:30
Xiao Chen
8056472879 HADOOP-15674. Test failure TestSSLHttpServer.testExcludedCiphers with TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 cipher suite. Contributed by Szilard Nemeth. 
(cherry picked from commit 8d7c93186e3090b19aa59006bb6b32ba929bd8e6)
 2018-08-17 10:20:29 -07:00
Rohith Sharma K S
e2210a5175 YARN-8612. Fix NM Collector Service Port issue in YarnConfiguration. Contributed by Prabha Manepalli. 
(cherry picked from commit 1697a0230696e1ed6d9c19471463b44a6d791dfa)
 2018-08-17 11:17:30 +05:30
Wei-Chiu Chuang
04c8b5dbb6 HDFS-13758. DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction. Contributed by chencan. 2018-08-14 11:57:30 -07:00
Jason Lowe
a44e53a314 YARN-8640. Restore previous state in container-executor after failure. Contributed by Jim Brennan 
(cherry picked from commit d1d129aa9deecebf42261947fcb0b2ca46dacad5)
 2018-08-14 10:33:27 -05:00
Wei-Chiu Chuang
ad5c3300ff HDFS-13813. Exit NameNode if dangling child inode is detected when saving FsImage. Contributed by Siyao Meng. 
(cherry picked from commit 23854443efa62aa70a1c30c32c3816750e5d7a5b)
(cherry picked from commit b38649c59a70b3112811443464b3b3180f4b9873)
(cherry picked from commit 4b606eb9fd5dde5629e791d3338886a70b5aa57c)
 2018-08-13 16:15:36 -07:00
Chao Sun
305b9a652a Remove super user privilege requirement for NameNode.getServiceStatus. Contributed by Chao Sun. 
(cherry picked from commit a2a8c486998b81d2c73804a07cc74f5269bfd904)
 2018-08-10 16:17:04 -07:00
Jonathan Hung
7abffe4529 YARN-8559. Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint. Contributed by Weiwei Yang. 2018-08-10 15:34:21 -07:00
Jason Lowe
2024260af6 YARN-8331. Race condition in NM container launched after done. Contributed by Pradeep Ambati 
(cherry picked from commit cd04e954d2db27f0a15b7d1c492b7cdb656a51db)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java
 2018-08-09 10:35:07 -05:00
Haibo Chen
1991a1d760 YARN-6966. NodeManager metrics may return wrong negative values when NM restart. (Szilard Nemeth via Haibo Chen) 2018-08-02 10:06:16 -07:00
Arun Suresh
e2b82b82e2 YARN-7542. Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED. (Sampada Dehankar via asuresh) 
(cherry picked from commit a55884c68eb175f1c9f61771386c086bf1ee65a9)
(cherry picked from commit bd4dcc7772f9a6786e8ef4ef8fa97dfdd34d64d1)
 2018-08-02 09:59:04 -07:00
Sammi Chen
418e957c64 HADOOP-15607. AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream. Contributed by Jinhu Wu. 2018-08-02 10:14:54 +08:00
Rohith Sharma K S
21e416ad27 YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 2018-08-01 22:25:53 +05:30
Robert Kanter
a7163886ce HADOOP-15644. Hadoop Docker Image Pip Install Fails on branch-2 (haibochen via rkanter) 2018-07-31 13:28:49 -07:00
Chen Liang
d2ec117435 HADOOP-15637. LocalFs#listLocatedStatus does not filter out hidden .crc files. Contributed by Erik Krogen. 2018-07-31 11:44:08 -07:00
Wei-Chiu Chuang
5ef8fd5119 HDFS-11060. make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable. Contributed by Lantao Jin. 
(cherry picked from commit e95c5e9f62452ee848875ec2f8642eab4992cd23)
(cherry picked from commit 964f3454d1237fa25aa828504612b4a11d8e76ff)
(cherry picked from commit 0af04ad89f2e3d867ed4011d519a53b764e3be6e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
 2018-07-25 11:26:19 -07:00
Eric E Payne
3aa17bd737 YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 
(cherry picked from commit 9485c9aee6e9bb935c3e6ae4da81d70b621781de)
 2018-07-25 17:06:47 +00:00
Yiqun Lin
e6fcaccf5a HDFS-13583. RBF: Router admin clrQuota is not synchronized with nameservice. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 17a87977f29ced49724f561a68565217c8cb4e94)
 2018-07-24 11:21:37 +08:00
Robert Kanter
edb9d8b554 YARN-8518. test-container-executor test_is_empty() is broken (Jim_Brennan via rkanter) 
(cherry picked from commit 1bc106a738a6ce4f7ed025d556bb44c1ede022e3)
(cherry picked from commit 6e0db6fe1a8ce50977175567f2ba1f957e7b9c91)
 2018-07-22 05:28:21 +00:00
Robert Kanter
92f02f97fd Only mount non-empty directories for cgroups (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 0838fe833738e04f5e6f6408e97866d77bebbf30)
(cherry picked from commit c1dc4ca2c6080377159157ce97bf5d72fa3285a1)
 2018-07-22 05:28:20 +00:00
Robert Kanter
f5fd5aa025 Disable mounting cgroups by default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 351cf87c92872d90f62c476f85ae4d02e485769c)
(cherry picked from commit d61d84279f7f22867c23dd95e8bfeb70ea7e0690)
 2018-07-22 05:28:20 +00:00
Yiqun Lin
8a68a1590f HDFS-13743. RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver. Contributed by Takanobu Asanuma. 
(cherry picked from commit 7b25fb949bf6f02df997beeca7df46c9e84c8d96)
 2018-07-20 17:33:31 +08:00
Kihwal Lee
cf255dc01f HADOOP-15614. TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails. Contributed by Weiwei Yang. 
(cherry picked from commit ccf2db7fc2688d262df3309007cb12a4dfedc179)
 2018-07-19 11:29:58 -05:00
Yiqun Lin
4c2ab22130 HDFS-13733. RBF: Add Web UI configurations and descriptions to RBF document. Contributed by Takanobu Asanuma. 
(cherry picked from commit 1af87df242c4286474961078d306a5692f85debc)
 2018-07-17 10:49:50 +08:00
Konstantin V Shvachko
fccb3c02f0 Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. 
(cherry picked from commit 0a1e922f3d8eca4e852be57124ec1bcafaadb289)
 2018-07-16 18:44:01 -07:00
Eric E Payne
8ee439d791 YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 
(cherry picked from commit 937ef39b3ff90f72392b7a319e4346344db34e03)
 2018-07-16 17:01:38 +00:00
Inigo Goiri
436fceebf9 HDFS-13475. RBF: Admin cannot enforce Router enter SafeMode. Contributed by Chao Sun. 
(cherry picked from commit 359ea4e18147af5677c6d88265e26de6b6c72999)
 2018-07-16 09:47:57 -07:00
Jason Lowe
0e6efe06ea YARN-8515. container-executor can crash with SIGPIPE after nodemanager restart. Contributed by Jim Brennan 
(cherry picked from commit 17118f446c2387aa796849da8b69a845d9d307d3)
 2018-07-13 10:11:57 -05:00
Akira Ajisaka
ff46db2ea6
HDFS-13729. Fix broken links to RBF documentation. Contributed by Gabor Bota. 2018-07-12 13:53:08 -04:00
Yiqun Lin
fbe7192815 HDFS-13726. RBF: Fix RBF configuration links. Contributed by Takanobu Asanuma. 
(cherry picked from commit 2ae13d41dcd4f49e6b4ebc099e5f8bb8280b9872)
 2018-07-11 22:17:43 +08:00
Sunil G
6cc5d49fa3 YARN-8473. Containers being launched as app tears down can leave containers in NEW state. Contributed by Jason Lowe. 
(cherry picked from commit 705e2c1f7cba51496b0d019ecedffbe5fb55c28b)
 2018-07-10 20:13:56 +05:30
Giovanni Matteo Fumarola
aab9bfc13c YARN-7899. [AMRMProxy] Stateful FederationInterceptor for pending requests. Contributed by Botong Huang. 2018-07-09 16:47:44 -07:00
Hanisha Koneru
4c54ddd1c2 HADOOP-15121. Encounter NullPointerException when using DecayRpcScheduler. Contributed by Tao Jie. 
(cherry picked from commit 3fde0f1db599227773c0cd537b33312d368ad4d9)
 2018-07-09 15:23:33 -05:00
Andrew Wang
913417bbea HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi. 2018-07-09 15:17:26 +02:00
Yiqun Lin
512d20e9c5 HDFS-13710. RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable. Contributed by yanghuafeng. 
(cherry picked from commit 43f7fe8aae0eca89cce4d67bfc4965fe8ce63e38)
 2018-07-09 15:08:39 +08:00
Yiqun Lin
1a96293c63 HDFS-13528. RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 3b637155a47d2aa93284969a96208347a647083d)
 2018-07-04 15:07:47 +08:00
Inigo Goiri
3e177a1d62 HDFS-13714. Fix TestNameNodePrunesMissingStorages test failures on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 7296b644f7c44400f03995668573a4cfd2ee552c)
 2018-07-02 16:38:13 -07:00
Eric E Payne
f951d92d46 HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan. 2018-06-29 20:18:55 +00:00
Giovanni Matteo Fumarola
64baa9ec89 YARN-8481. AMRMProxyPolicies should accept heartbeat response from new/unknown subclusters. Contributed by Botong Huang. 2018-06-29 11:51:50 -07:00
Jason Lowe
14c7dc3c1e YARN-8451. Multiple NM heartbeat thread created when a slow NM resync with RM. Contributed by Botong Huang 
(cherry picked from commit 100470140d86eede0fa240a9aa93226f274ee4f5)
 2018-06-29 13:17:14 -05:00
Sunil G
33e6eec7b8 YARN-8401. [UI2] new ui is not accessible with out internet connection. Contributed by Bibin A Chundatt. 
(cherry picked from commit fbaff369e9b9022723a7b2c6f25e71122a8f8a15)
 2018-06-27 10:36:31 -07:00