Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,593 Commits 326 Branches 371 Tags
Commit Graph

14593 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wei-Chiu Chuang
ad5c3300ff HDFS-13813. Exit NameNode if dangling child inode is detected when saving FsImage. Contributed by Siyao Meng. 
(cherry picked from commit 23854443efa62aa70a1c30c32c3816750e5d7a5b)
(cherry picked from commit b38649c59a70b3112811443464b3b3180f4b9873)
(cherry picked from commit 4b606eb9fd5dde5629e791d3338886a70b5aa57c)
 2018-08-13 16:15:36 -07:00
Chao Sun
305b9a652a Remove super user privilege requirement for NameNode.getServiceStatus. Contributed by Chao Sun. 
(cherry picked from commit a2a8c486998b81d2c73804a07cc74f5269bfd904)
 2018-08-10 16:17:04 -07:00
Jonathan Hung
7abffe4529 YARN-8559. Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint. Contributed by Weiwei Yang. 2018-08-10 15:34:21 -07:00
Jason Lowe
2024260af6 YARN-8331. Race condition in NM container launched after done. Contributed by Pradeep Ambati 
(cherry picked from commit cd04e954d2db27f0a15b7d1c492b7cdb656a51db)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java
 2018-08-09 10:35:07 -05:00
Haibo Chen
1991a1d760 YARN-6966. NodeManager metrics may return wrong negative values when NM restart. (Szilard Nemeth via Haibo Chen) 2018-08-02 10:06:16 -07:00
Arun Suresh
e2b82b82e2 YARN-7542. Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED. (Sampada Dehankar via asuresh) 
(cherry picked from commit a55884c68eb175f1c9f61771386c086bf1ee65a9)
(cherry picked from commit bd4dcc7772f9a6786e8ef4ef8fa97dfdd34d64d1)
 2018-08-02 09:59:04 -07:00
Sammi Chen
418e957c64 HADOOP-15607. AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream. Contributed by Jinhu Wu. 2018-08-02 10:14:54 +08:00
Rohith Sharma K S
21e416ad27 YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 2018-08-01 22:25:53 +05:30
Robert Kanter
a7163886ce HADOOP-15644. Hadoop Docker Image Pip Install Fails on branch-2 (haibochen via rkanter) 2018-07-31 13:28:49 -07:00
Chen Liang
d2ec117435 HADOOP-15637. LocalFs#listLocatedStatus does not filter out hidden .crc files. Contributed by Erik Krogen. 2018-07-31 11:44:08 -07:00
Wei-Chiu Chuang
5ef8fd5119 HDFS-11060. make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable. Contributed by Lantao Jin. 
(cherry picked from commit e95c5e9f62452ee848875ec2f8642eab4992cd23)
(cherry picked from commit 964f3454d1237fa25aa828504612b4a11d8e76ff)
(cherry picked from commit 0af04ad89f2e3d867ed4011d519a53b764e3be6e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
 2018-07-25 11:26:19 -07:00
Eric E Payne
3aa17bd737 YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 
(cherry picked from commit 9485c9aee6e9bb935c3e6ae4da81d70b621781de)
 2018-07-25 17:06:47 +00:00
Yiqun Lin
e6fcaccf5a HDFS-13583. RBF: Router admin clrQuota is not synchronized with nameservice. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 17a87977f29ced49724f561a68565217c8cb4e94)
 2018-07-24 11:21:37 +08:00
Robert Kanter
edb9d8b554 YARN-8518. test-container-executor test_is_empty() is broken (Jim_Brennan via rkanter) 
(cherry picked from commit 1bc106a738a6ce4f7ed025d556bb44c1ede022e3)
(cherry picked from commit 6e0db6fe1a8ce50977175567f2ba1f957e7b9c91)
 2018-07-22 05:28:21 +00:00
Robert Kanter
92f02f97fd Only mount non-empty directories for cgroups (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 0838fe833738e04f5e6f6408e97866d77bebbf30)
(cherry picked from commit c1dc4ca2c6080377159157ce97bf5d72fa3285a1)
 2018-07-22 05:28:20 +00:00
Robert Kanter
f5fd5aa025 Disable mounting cgroups by default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 351cf87c92872d90f62c476f85ae4d02e485769c)
(cherry picked from commit d61d84279f7f22867c23dd95e8bfeb70ea7e0690)
 2018-07-22 05:28:20 +00:00
Yiqun Lin
8a68a1590f HDFS-13743. RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver. Contributed by Takanobu Asanuma. 
(cherry picked from commit 7b25fb949bf6f02df997beeca7df46c9e84c8d96)
 2018-07-20 17:33:31 +08:00
Kihwal Lee
cf255dc01f HADOOP-15614. TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails. Contributed by Weiwei Yang. 
(cherry picked from commit ccf2db7fc2688d262df3309007cb12a4dfedc179)
 2018-07-19 11:29:58 -05:00
Yiqun Lin
4c2ab22130 HDFS-13733. RBF: Add Web UI configurations and descriptions to RBF document. Contributed by Takanobu Asanuma. 
(cherry picked from commit 1af87df242c4286474961078d306a5692f85debc)
 2018-07-17 10:49:50 +08:00
Konstantin V Shvachko
fccb3c02f0 Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. 
(cherry picked from commit 0a1e922f3d8eca4e852be57124ec1bcafaadb289)
 2018-07-16 18:44:01 -07:00
Eric E Payne
8ee439d791 YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 
(cherry picked from commit 937ef39b3ff90f72392b7a319e4346344db34e03)
 2018-07-16 17:01:38 +00:00
Inigo Goiri
436fceebf9 HDFS-13475. RBF: Admin cannot enforce Router enter SafeMode. Contributed by Chao Sun. 
(cherry picked from commit 359ea4e18147af5677c6d88265e26de6b6c72999)
 2018-07-16 09:47:57 -07:00
Jason Lowe
0e6efe06ea YARN-8515. container-executor can crash with SIGPIPE after nodemanager restart. Contributed by Jim Brennan 
(cherry picked from commit 17118f446c2387aa796849da8b69a845d9d307d3)
 2018-07-13 10:11:57 -05:00
Akira Ajisaka
ff46db2ea6
HDFS-13729. Fix broken links to RBF documentation. Contributed by Gabor Bota. 2018-07-12 13:53:08 -04:00
Yiqun Lin
fbe7192815 HDFS-13726. RBF: Fix RBF configuration links. Contributed by Takanobu Asanuma. 
(cherry picked from commit 2ae13d41dcd4f49e6b4ebc099e5f8bb8280b9872)
 2018-07-11 22:17:43 +08:00
Sunil G
6cc5d49fa3 YARN-8473. Containers being launched as app tears down can leave containers in NEW state. Contributed by Jason Lowe. 
(cherry picked from commit 705e2c1f7cba51496b0d019ecedffbe5fb55c28b)
 2018-07-10 20:13:56 +05:30
Giovanni Matteo Fumarola
aab9bfc13c YARN-7899. [AMRMProxy] Stateful FederationInterceptor for pending requests. Contributed by Botong Huang. 2018-07-09 16:47:44 -07:00
Hanisha Koneru
4c54ddd1c2 HADOOP-15121. Encounter NullPointerException when using DecayRpcScheduler. Contributed by Tao Jie. 
(cherry picked from commit 3fde0f1db599227773c0cd537b33312d368ad4d9)
 2018-07-09 15:23:33 -05:00
Andrew Wang
913417bbea HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi. 2018-07-09 15:17:26 +02:00
Yiqun Lin
512d20e9c5 HDFS-13710. RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable. Contributed by yanghuafeng. 
(cherry picked from commit 43f7fe8aae0eca89cce4d67bfc4965fe8ce63e38)
 2018-07-09 15:08:39 +08:00
Yiqun Lin
1a96293c63 HDFS-13528. RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 3b637155a47d2aa93284969a96208347a647083d)
 2018-07-04 15:07:47 +08:00
Inigo Goiri
3e177a1d62 HDFS-13714. Fix TestNameNodePrunesMissingStorages test failures on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 7296b644f7c44400f03995668573a4cfd2ee552c)
 2018-07-02 16:38:13 -07:00
Eric E Payne
f951d92d46 HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan. 2018-06-29 20:18:55 +00:00
Giovanni Matteo Fumarola
64baa9ec89 YARN-8481. AMRMProxyPolicies should accept heartbeat response from new/unknown subclusters. Contributed by Botong Huang. 2018-06-29 11:51:50 -07:00
Jason Lowe
14c7dc3c1e YARN-8451. Multiple NM heartbeat thread created when a slow NM resync with RM. Contributed by Botong Huang 
(cherry picked from commit 100470140d86eede0fa240a9aa93226f274ee4f5)
 2018-06-29 13:17:14 -05:00
Sunil G
33e6eec7b8 YARN-8401. [UI2] new ui is not accessible with out internet connection. Contributed by Bibin A Chundatt. 
(cherry picked from commit fbaff369e9b9022723a7b2c6f25e71122a8f8a15)
 2018-06-27 10:36:31 -07:00
Rohith Sharma K S
f359b7e5e9 YARN-8457. Compilation is broken with -Pyarn-ui. 
(cherry picked from commit 4ffe68a6f70ce01a5654da8991b4cdb35ae0bf1f)
 2018-06-25 10:40:40 -07:00
Inigo Goiri
1d457673c8 HADOOP-15458. TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows. Contributed by Xiao Liang. 
(cherry picked from commit abc3e4bad905efde5a4881e8a072c68f6e910ade)
 2018-06-25 09:52:15 -07:00
Steve Loughran
11e7f2e584
HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled. 
Contributed by Anis Elleuch.

(cherry picked from commit 1dedc68f9d8d8544d715e67ee77cd3f017c21699)
(cherry picked from commit cc0f14c13c1699489777b71a5c21bad27f9020ad)
 2018-06-25 17:27:49 +01:00
Eric E Payne
d084c1a4bc YARN-8444: NodeResourceMonitor crashes on bad swapFree value. Contributed by Jim Brennan. 
(cherry picked from commit 6432128622d64f3f9dd638b9c254c77cdf5408aa)
 2018-06-22 17:53:07 +00:00
Inigo Goiri
82874e7895 YARN-8412. Move ResourceRequest.clone logic everywhere into a proper API. Contributed by Botong Huang. 2018-06-21 18:25:30 -07:00
Chris Douglas
4789b8e9c9 HADOOP-15533. Make WASB listStatus messages consistent. Contributed by Esfandiar Manii 
(cherry picked from commit f34744603ee93e082e7ba148df1400af5ac7c30c)
(cherry picked from commit 7e655b642d0ed5b85e0e8c212e004d067318ed4c)
(cherry picked from commit d1dcc39222b6d1d8ba10f38a3f2fb69e4d6548b3)
 2018-06-17 23:14:12 -07:00
Surendra Singh Lilhore
59686179ae HADOOP-15523. Shell command timeout given is in seconds whereas it is taken as millisec while scheduling. Contributed by Bilwa S T. 
(cherry picked from commit 3905fdb793e6370243d05d0c3036ca69898fe3fb)
 2018-06-17 12:20:11 +05:30
Inigo Goiri
9e148aa0bc HDFS-13681. Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 8762e9cf10fa100dd5f7fd695f5e52b75a94c5d4)
 2018-06-15 16:51:03 -07:00
Inigo Goiri
b52316d796 HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 
(cherry picked from commit d31a3ce767d3bb68bdbb4f36d45600eab9f4f8b7)
 2018-06-15 16:00:45 -07:00
Sean Mackrory
b122961bc0 HDFS-13582. Improve backward compatibility for HDFS-13176 (WebHdfs file path gets truncated when having semicolon (;) inside). Contributed by Zsolt Venczel. 2018-06-15 16:28:36 -06:00
Inigo Goiri
fe831cb5b2 HDFS-13676. TestEditLogRace fails on Windows. Contributed by Zuoming Zhang. 2018-06-15 13:15:20 -07:00
Inigo Goiri
55b50eecc4 HDFS-13673. TestNameNodeMetrics fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit 43d994e4a6dfd1c24eafb909d6f8a0663b20769a)
 2018-06-15 10:35:05 -07:00
Inigo Goiri
fcd7ae2745 HDFS-13675. Speed up TestDFSAdminWithHA. Contributed by Lukas Majercak. 2018-06-14 14:12:54 -07:00
Inigo Goiri
06c87edf51 HDFS-13563. TestDFSAdminWithHA times out on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 8d4926f38bf53b32453cd2bc7322c8818f752f85)
 2018-06-14 10:00:26 -07:00