Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,716 Commits 326 Branches 371 Tags
Commit Graph

19716 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Chen Liang
466f77a5cf HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang. 
(cherry picked from commit 226cd3977bfc5948afff6a2d38a41e341089d5fe)
 2020-02-04 12:36:46 -08:00
Kihwal Lee
a55a0a1f6d HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan. 
(cherry picked from commit 10a60fbe20bb08cdd71076ea9bf2ebb3a2f6226e)
 2020-02-04 12:24:58 -06:00
Stephen O'Donnell
8ea4787f9f HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1e3a0b0d931676b191cb4813ed1a283ebb24d4eb)
(cherry picked from commit e89436b1839aa87b4bedfc7e2d558c7541e73540)
 2020-01-31 16:13:53 -08:00
Inigo Goiri
ffaf0d3c82 HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein. 
(cherry picked from commit 1839c467f60cbb8592d446694ec3d7710cda5142)
 2020-01-31 11:25:36 -08:00
Eric Badger
4af7d14ce2 YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne. 2020-01-29 22:45:02 +00:00
Chen Liang
5e1d89b74d [SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Conntributed by Chen Liang 2020-01-29 11:15:35 -08:00
Eric E Payne
d4ecb65629 MAPREDUCE-7079: JobHistory#ServiceStop implementation is incorrect. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit b897f6834ba69d443c3acd1fab52261c00d675a1)
 2020-01-29 17:21:40 +00:00
Kihwal Lee
b8d87cfc32 HDFS-15146. TestBalancerRPCDelay.testBalancerRPCDelay fails 
intermittently. Contributed by Ahmed Hussein.

(cherry picked from commit 799d4c1cf4e8fe78eb9ab607a0449cdd075041fb)
 2020-01-29 11:01:51 -06:00
Ayush Saxena
1293199532 HDFS-14993. checkDiskError doesn't work during datanode startup. Contributed by Yang Yun. 2020-01-28 23:17:21 +05:30
Ahmed Hussein
f3ff61865a MAPREDUCE-7259. testSpeculateSuccessfulWithUpdateEvents fails Intermittently 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 08251538fe2550d9dd86f9daf79994f5b8bdf7fa)
 2020-01-28 11:04:25 -06:00
Ahmed Hussein
1f315c28d9 MAPREDUCE-7262. MRApp helpers block for long intervals (500ms) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 3f01c481060585ccd37be9db8aa4d1e33d2e2d6b)
 2020-01-27 16:23:06 -06:00
Kihwal Lee
7dd43cec7f HDFS-15119. Allow expiration of cached locations in DFSInputStream. 
Contributed by Ahmed Hussein.

(cherry picked from commit d10f77e3c91225f86ed9c0f0e6a9adf2e1434674)
(cherry picked from commit 1bb9667137bc80e63dea3e53bb2746be011a2a9a)
 2020-01-24 09:53:26 -06:00
Abhishek Modi
868a0f5ef0 YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam. 
(cherry picked from commit d2d963f3d4819704351c04dbeb90fc8154488f91)
 2020-01-23 15:50:28 +00:00
Szilard Nemeth
9638985428 YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg 2020-01-22 16:30:59 +01:00
Kihwal Lee
b4e9725955 HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein. 
(cherry picked from commit bd03053ea2f32ef982e37fbf2ffd679cb7dda797)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java

(cherry picked from commit 484270832064c84122348e663aaf5927ed411ebb)
 2020-01-22 09:28:01 -06:00
sunlisheng
96c653d0d5 Remove WARN log when ipc connection interrupted in Client#handleSaslConnectionFailure() 
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
(cherry picked from commit d887e49dd4ed2b94bbb53b7608586f5da6cee037)
 2020-01-21 18:32:14 +09:00
Akira Ajisaka
2c84ea96e3
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li. 
(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)
 2020-01-21 18:05:13 +09:00
Szilard Nemeth
2cdaeca84b YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph 2020-01-20 09:30:48 +01:00
kevinzhao1661
ad34d7032e
MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684) 
(cherry picked from commit b8518e6dabc1cf73fa7c43b3f8904d2e2beb2bf3)
 2020-01-16 12:58:30 +09:00
Sunil G
af89b5b086 YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg. 2020-01-15 17:07:34 +05:30
Masatake Iwasaki
ce52a66fcb
MAPREDUCE-7256. Fix javadoc error in SimpleExponentialSmoothing. (#1804) 
(cherry picked from commit a2fdd7c2b59463c39a4d4250f908a9e2d3653ead)
 2020-01-15 12:12:27 +09:00
Szilard Nemeth
3a09593a73 HADOOP-16683. Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException. Contributed by Adam Antal 2020-01-14 11:19:52 +01:00
Wei-Chiu Chuang
286c94f13d HDFS-15097. Purge log in KMS and HttpFS. Contributed by Doris Gu. 
(cherry picked from commit 6b86a5110e4b3f45bf55f97bd402680bf26cefb9)
(cherry picked from commit aa3c96474f52fba01d999fa75880e0948affbb01)
 2020-01-13 10:49:39 -08:00
Ayush Saxena
12b258115b HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena. 2020-01-11 10:34:37 +05:30
Chen Liang
5490061885 HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang. 
(cherry picked from commit 26a969ec734dbdbf1d544f486dfa33f15c291789)
 2020-01-10 17:19:05 -08:00
Kihwal Lee
82fa8d88ef HDFS-15095. Fix TestDecommissioningStatus. Contributed by Ahmed Hussein. 
(cherry picked from commit 5fb901ac4017b4f13b089ecd920e864cd53ad3a6)
 2020-01-10 12:47:49 -06:00
Daryn Sharp
da45684185 HADOOP-16749. Configuration parsing of CDATA values are blank 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit c4fb43c94259546f2c96b50ceae9cd3adb726166)
 2020-01-10 10:39:18 -06:00
Eric E Payne
ec40c1f400 YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e4d6b8098622a1aa830355cc18d9589)
 2020-01-09 17:41:10 +00:00
Surendra Singh Lilhore
09d3707875 HDFS-14957. INodeReference Space Consumed was not same in QuotaUsage and ContentSummary. Contributed by hemanthboyina. 2020-01-09 12:18:58 +05:30
Eric Badger
cba0bbe98c YARN-8672. TestContainerManager#testLocalingResourceWhileContainerRunning occasionally times out. Contributed by Chandni Singh and Jim Brennan. 2020-01-08 20:02:43 +00:00
Eric E Payne
a110024fcc YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit b1e07d27cc1a26be4e5ebd1ab7b03ef15032bef0)
 2020-01-08 19:41:53 +00:00
Eric E Payne
08a1464444 YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 6899be5a1729e49cff45090acd2cf4f54aeac089)
 2020-01-08 17:50:43 +00:00
Ahmed Hussein
6ad2f662eb MAPREDUCE-7252. Handling 0 progress in SimpleExponential task runtime estimator 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit cdd6efd3ab6917e30b4c5c7b261f61838901bb37)
 2020-01-08 11:14:40 -06:00
Rakesh Radhakrishnan
b1eb47865c HDFS-15080. Fix the issue in reading persistent memory cached data with an offset. Contributed by Feilong He. 
(cherry picked from commit 7030722e5d9f376245a9ab0a6a883538b6c55f82)
 2020-01-08 14:32:45 +05:30
Masatake Iwasaki
c5580af1c4 HDFS-15077. Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout. (#1797) 
(cherry picked from commit aba3f6c3e1fbb150ea7ff0411c41ffd3a2796208)
 2020-01-08 17:06:49 +09:00
Masatake Iwasaki
4c8b01abcb
HDFS-15072. HDFS MiniCluster fails to start when run in directory path with a %. (#1775) 
(cherry picked from commit a43c177f1d4c2b6149a2680dd23d91103eca3be0)
 2020-01-08 12:38:51 +09:00
Sergey Pogorelov
d55d02d25a
MAPREDUCE-7255. Fix typo in MapReduce documentaion example (#1793) 
(cherry picked from commit b343e1533b1f4ebbbf10b063c35d79d53a741321)
 2020-01-06 12:41:03 +09:00
Ayush Saxena
1d3bf33769 HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena. 2020-01-02 21:39:17 +05:30
Prabhu Joseph
807d7aa3af YARN-10053. Use Shared Group Mapping Service in Placement Rules. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 217b56ffdd5fa254f06734bc8cb6f04a02066f1a)
 2020-01-02 14:41:47 +05:30
Rakesh Radhakrishnan
8b748c0308 HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He. 
(cherry picked from commit d79cce20abbbf321f6dcce03f4087544124a7cd2)
 2020-01-02 12:19:38 +05:30
Surendra Singh Lilhore
55510d32bf HDFS-15074. DataNode.DataTransfer thread should catch all the expception and log it. Contributed by hemanthboyina. 2019-12-29 11:25:28 +05:30
Takanobu Asanuma
454a0c2913 HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena. 
(cherry picked from commit dc32f583afffc372f78fb45211c3e7ce13f6a4be)
 2019-12-28 21:35:22 +09:00
Chandra Sanivarapu
4acbe1c05d
HADOOP-16042. Update the link to HadoopJavaVersion (#1784) 
(cherry picked from commit d80d86264d8e4b1a5cfc6fc12eec8f9dfcd76b3f)
 2019-12-27 17:21:29 +09:00
Duo Zhang
83d676ab2e
HADOOP-16610. Upgrade to yetus 0.11.1 and use emoji vote on github pre commit (#1527). Contributed by Duo Zhang. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 3cecb2a46974d446a91435ac49c70742e3cbb6bd)
(cherry picked from commit 440b7abc049a7ea60cc2f72f33a792bdae523e03)
 2019-12-25 19:48:47 +09:00
Akira Ajisaka
5853fa2ea2
HADOOP-16609. Add Jenkinsfile for all active branches (#1557) 
* Backport HADOOP-16035. Jenkinsfile for Hadoop
* Use Yetus 0.10.0

(cherry picked from commit ae8ca70fc0f83384800a249625285b657c3b3057)
 2019-12-25 17:25:14 +09:00
Chandra Sanivarapu
e33651ace7
HDFS-15073. Replace curator-shaded guava import with the standard one (#1782) 
(cherry picked from commit d8cd7098b4bcfbfd76915b9ecefb2c7ea320e149)
 2019-12-25 11:12:29 +09:00
Konstantin V Shvachko
1a22e4036c HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b98ac2a3af50ccf2af07790ab0760d4c51820836)
 2019-12-24 12:50:13 -08:00
Akira Ajisaka
01edc654df
YARN-10055. bower install fails. (#1778) 
(cherry picked from commit 34ff7dbaf53cbbebcb163980f4b45221630c2882)
 2019-12-23 20:27:06 +09:00
Eric Badger
0f01206e15 YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne 
(cherry picked from commit 355ec3341685deb93a40daffa3f28965b7609286)
 2019-12-20 19:34:29 +00:00
Eric Badger
ef00db4ce9 Revert "HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui." 
This reverts commit 27555620fae3791d9fd24168e346714de8467a1d.
 2019-12-20 19:16:22 +00:00