Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,639 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

19639 Commits

Author SHA1 Message Date
Kihwal Lee b92477c638 HDFS-15147. LazyPersistTestCase wait logic is error-prone. Contributed 
by Ahmed Hussein.

(cherry picked from commit 27cfda708e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/LazyPersistTestCase.java
 2020-02-27 09:45:12 -06:00
Elixir Kook 9ccefe2262
YARN-10156. Fix typo 'complaint' which means quite different in Federation.md (#1856) 
(cherry picked from commit d608e94f92)
 2020-02-26 17:32:47 +09:00
Stephen O'Donnell 8aaa8d1b71 HDFS-14861. Reset LowRedundancyBlocks Iterator periodically. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 900430b990)
(cherry picked from commit 2377649cdb)
 2020-02-25 13:29:15 -08:00
Takanobu Asanuma 241c5a14c5 HADOOP-16841. The description of hadoop.http.authentication.signature.secret.file contains outdated information. Contributed by Xieming Li. 
(cherry picked from commit 5cbc4c5461)
 2020-02-25 11:10:22 +09:00
Wei-Chiu Chuang 268e35d3ae HDFS-15174. Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations. Contributed by Lisheng Sun. 
(cherry picked from commit cb4daeaad95470125c59ccb80b988261ae83d534)
(cherry picked from commit f9437b1817)
 2020-02-24 13:37:59 -08:00
Ayush Saxena d318b48e12 HDFS-15166. Remove redundant field fStream in ByteStringLog. Contributed by Xieming Li. 2020-02-24 20:17:52 +05:30
Konstantin V Shvachko cd0525054d HDFS-15185. StartupProgress reports edits segments until the entire startup completes. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 6f84269bcd)
 2020-02-21 10:56:31 -08:00
Masatake Iwasaki 8095e3b149 HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. (#1758) 
(cherry picked from commit 2338d25dc7)

 Conflicts:
        hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java

(cherry picked from commit 610805ec72)
 2020-02-21 13:51:26 +09:00
Akira Ajisaka 3f8ac06cef
HADOOP-16869. Upgrade findbugs-maven-plugin to 3.0.5 to fix mvn findbugs:findbugs failure (#1855) 
(cherry picked from commit 7f35676f90)

 Conflicts:
	hadoop-project/pom.xml

(cherry picked from commit b52f201285)
 2020-02-21 12:24:41 +09:00
Wei-Chiu Chuang b28347d282 HADOOP-16868. ipc.Server readAndProcess threw NullPointerException. Contributed by Tsz-wo Sze. 
(cherry picked from commit cb3f3cca01)
(cherry picked from commit d411a849e1)
 2020-02-18 21:54:00 -08:00
Sunil G a6124cd2b8 YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph. 
(cherry picked from commit 6526f95bd2)
 2020-02-19 11:18:53 +05:30
Kihwal Lee c54051def5 HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3ead525c71)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
 2020-02-17 16:07:03 -06:00
xiaoxiaopan118 b59fa3e571
HDFS-15173. RBF: Delete repeated configuration 'dfs.federation.router.metrics.enable' (#1849) 
(cherry picked from commit 439d935e1d)
 2020-02-17 15:58:35 +09:00
Ayush Saxena 2190fb1fcf HDFS-15115. Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug. Contributed by wangzhixiang 2020-02-16 10:09:40 +05:30
Ayush Saxena cf5a7186e5 HDFS-15164. Fix TestDelegationTokensWithHA. Contributed by Ayush Saxena. 2020-02-14 22:11:39 +05:30
Surendra Singh Lilhore 6a4297adc4 HDFS-15086. Block scheduled counter never get decremet if the block got deleted before replication. Contributed by hemanthboyina. 2020-02-13 23:04:28 +05:30
Ayush Saxena 83ab853733 HDFS-15161. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close(). Contributed by Lisheng Sun 2020-02-12 20:30:13 +05:30
Akira Ajisaka 1409c9da7c
HADOOP-16849. start-build-env.sh behaves incorrectly when username is numeric only. Contributed by Jihyun Cho. 
(cherry picked from commit 9709afe67d)

Conflicts:
	start-build-env.sh
 2020-02-12 14:11:00 +09:00
Kihwal Lee bf09bfa2db HDFS-14758. Make lease hard limit configurable and reduce the default. 
Contributed by hemanthboyina.

(cherry picked from commit 9b8a78d97b)
(cherry picked from commit f1840669cf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
 2020-02-11 14:57:49 -06:00
Prabhu Joseph 00fde836f7 YARN-10022. RM Rest API to validate the CapacityScheduler Configuration change. 
Contributed by Kinga Marton.

(cherry-picked from commit 1ab9c692fa)
 2020-02-11 22:23:07 +05:30
Ayush Saxena 547f1798aa HDFS-15157. Fix compilation failure for branch-3.1. Contributed by Ayush Saxena 2020-02-09 08:14:33 +05:30
Jonathan Hung 5dfd1dcfe3 YARN-10116. Expose diagnostics in RMAppManager summary 
(cherry picked from commit 314e2f9d2e)
(cherry picked from commit 147897da4b420b4749f3c7b410f4c329632c3352)
 2020-02-05 11:17:24 -08:00
Kihwal Lee a628ab0863 HADOOP-15787. TestIPC.testRTEDuringConnectionSetup fails with jdk8u242. Contributed by Zsolt Venczel. 
(cherry picked from commit a463cf75a0)
(cherry picked from commit 0b19c8e086)
 2020-02-05 13:16:34 -06:00
Chen Liang 466f77a5cf HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang. 
(cherry picked from commit 226cd3977b)
 2020-02-04 12:36:46 -08:00
Kihwal Lee a55a0a1f6d HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan. 
(cherry picked from commit 10a60fbe20)
 2020-02-04 12:24:58 -06:00
Stephen O'Donnell 8ea4787f9f HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1e3a0b0d93)
(cherry picked from commit e89436b183)
 2020-01-31 16:13:53 -08:00
Inigo Goiri ffaf0d3c82 HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein. 
(cherry picked from commit 1839c467f6)
 2020-01-31 11:25:36 -08:00
Eric Badger 4af7d14ce2 YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne. 2020-01-29 22:45:02 +00:00
Chen Liang 5e1d89b74d [SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Conntributed by Chen Liang 2020-01-29 11:15:35 -08:00
Eric E Payne d4ecb65629 MAPREDUCE-7079: JobHistory#ServiceStop implementation is incorrect. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit b897f6834b)
 2020-01-29 17:21:40 +00:00
Kihwal Lee b8d87cfc32 HDFS-15146. TestBalancerRPCDelay.testBalancerRPCDelay fails 
intermittently. Contributed by Ahmed Hussein.

(cherry picked from commit 799d4c1cf4)
 2020-01-29 11:01:51 -06:00
Ayush Saxena 1293199532 HDFS-14993. checkDiskError doesn't work during datanode startup. Contributed by Yang Yun. 2020-01-28 23:17:21 +05:30
Ahmed Hussein f3ff61865a MAPREDUCE-7259. testSpeculateSuccessfulWithUpdateEvents fails Intermittently 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 08251538fe)
 2020-01-28 11:04:25 -06:00
Ahmed Hussein 1f315c28d9 MAPREDUCE-7262. MRApp helpers block for long intervals (500ms) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 3f01c48106)
 2020-01-27 16:23:06 -06:00
Kihwal Lee 7dd43cec7f HDFS-15119. Allow expiration of cached locations in DFSInputStream. 
Contributed by Ahmed Hussein.

(cherry picked from commit d10f77e3c9)
(cherry picked from commit 1bb9667137)
 2020-01-24 09:53:26 -06:00
Abhishek Modi 868a0f5ef0 YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam. 
(cherry picked from commit d2d963f3d4)
 2020-01-23 15:50:28 +00:00
Szilard Nemeth 9638985428 YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg 2020-01-22 16:30:59 +01:00
Kihwal Lee b4e9725955 HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein. 
(cherry picked from commit bd03053ea2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java

(cherry picked from commit 4842708320)
 2020-01-22 09:28:01 -06:00
sunlisheng 96c653d0d5 Remove WARN log when ipc connection interrupted in Client#handleSaslConnectionFailure() 
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
(cherry picked from commit d887e49dd4)
 2020-01-21 18:32:14 +09:00
Akira Ajisaka 2c84ea96e3
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li. 
(cherry picked from commit f6d20daf40)
 2020-01-21 18:05:13 +09:00
Szilard Nemeth 2cdaeca84b YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph 2020-01-20 09:30:48 +01:00
kevinzhao1661 ad34d7032e
MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684) 
(cherry picked from commit b8518e6dab)
 2020-01-16 12:58:30 +09:00
Sunil G af89b5b086 YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg. 2020-01-15 17:07:34 +05:30
Masatake Iwasaki ce52a66fcb
MAPREDUCE-7256. Fix javadoc error in SimpleExponentialSmoothing. (#1804) 
(cherry picked from commit a2fdd7c2b5)
 2020-01-15 12:12:27 +09:00
Szilard Nemeth 3a09593a73 HADOOP-16683. Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException. Contributed by Adam Antal 2020-01-14 11:19:52 +01:00
Wei-Chiu Chuang 286c94f13d HDFS-15097. Purge log in KMS and HttpFS. Contributed by Doris Gu. 
(cherry picked from commit 6b86a5110e)
(cherry picked from commit aa3c96474f)
 2020-01-13 10:49:39 -08:00
Ayush Saxena 12b258115b HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena. 2020-01-11 10:34:37 +05:30
Chen Liang 5490061885 HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang. 
(cherry picked from commit 26a969ec73)
 2020-01-10 17:19:05 -08:00
Kihwal Lee 82fa8d88ef HDFS-15095. Fix TestDecommissioningStatus. Contributed by Ahmed Hussein. 
(cherry picked from commit 5fb901ac40)
 2020-01-10 12:47:49 -06:00
Daryn Sharp da45684185 HADOOP-16749. Configuration parsing of CDATA values are blank 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit c4fb43c942)
 2020-01-10 10:39:18 -06:00