Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,881 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

21881 Commits

Author SHA1 Message Date
Masatake Iwasaki 7e5d9983b3 HADOOP-18334. Fix create-release to address removal of GPG_AGENT_INFO in branch-3.2. (#4554) 
(cherry picked from commit 61e3ad6533)
 2022-07-12 11:04:50 +00:00
Masatake Iwasaki 3bca16f4c2 updated contents for 3.2.4 release. 2022-07-12 09:21:23 +09:00
Masatake Iwasaki d7294589ce Preparing for 3.2.4 release 2022-07-11 14:51:12 +09:00
Szilard Nemeth de35dd819a YARN-10997. Revisit allocation and reservation logging. Contributed by Andras Gyori 
(cherry picked from commit 7cb887e6c2)
 2022-07-07 21:15:57 +00:00
Jinhu Wu cdbc2bac02 HADOOP-18313: AliyunOSSBlockOutputStream should not mark the temporary file for deletion (#4502) 
HADOOP-18313: AliyunOSSBlockOutputStream should not mark the temporary file for deletion. Contributed by wujinhu.
(cherry picked from commit 3ec4b932c1)
 2022-07-06 14:35:54 +08:00
Masatake Iwasaki 0fbb62ab8a Revert "HADOOP-17196. Fix C/C++ standard warnings (#2208)" 
This reverts commit ff907b310d.
 2022-06-30 00:48:50 +00:00
Igor Dvorzhak 399580d302 HADOOP-18300. Upgrade Gson dependency to version 2.9.0 (#4454) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit 77d1b194c7)
(cherry picked from commit d41e0a9cc3)
 2022-06-23 00:19:56 +00:00
KevinWikant 4dad353597 HDFS-16064. Determine when to invalidate corrupt replicas based on number of usable replicas (#4410) 
Co-authored-by: Kevin Wikant <wikak@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit cfceaebde6)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommission.java
 2022-06-20 11:40:11 +09:00
xuzq aba4418981 HDFS-16623. Avoid IllegalArgumentException in LifelineSender (#4409) 
* HDFS-16623. Avoid IllegalArgumentException in LifelineSender

Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
(cherry picked from commit af5003a473)
(cherry picked from commit ee3ee98ee5)
 2022-06-10 19:33:36 +00:00
Viraj Jasani 557ab95dc2 HDFS-16618. sync_file_range error should include more volume/file info (#4402) 
Signed-off-by: Tao Li <tomscut@apache.org>
 2022-06-07 17:11:57 +08:00
Wei-Chiu Chuang 4ebdc38634
HDFS-16456. EC: Decommission a rack with only on dn will fail when the rack number is equal with replication. (#4358) 2022-06-04 06:56:07 +08:00
Stephen O'Donnell 58513d367e HDFS-16610. Make fsck read timeout configurable (#4384) 
(cherry picked from commit 34a973a90e)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

(cherry picked from commit 7d6b133af3)
 2022-06-01 21:05:21 +01:00
Stephen O'Donnell 52e405390e
HDFS-16583. DatanodeAdminDefaultMonitor can get stuck in an infinite loop (#4394) 
Co-authored-by: S O'Donnell <sodonnell@cloudera.com>
 2022-06-01 07:55:48 +08:00
Duo Zhang 57fd58bc3c HADOOP-16664 Backport "HADOOP-16561 [MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes" to all active branches 2022-05-29 17:58:31 +09:00
Duo Zhang 1d053b0c65 HADOOP-16663 Backport "HADOOP-16560 [YARN] use protobuf-maven-plugin to generate protobuf classes" to all active branches 2022-05-29 17:54:25 +09:00
Vinayakumar B db4c32b03f HADOOP-16985. Handle release package related issues (#1957) 
(cherry picked from commit 4d24d99e85)
 2022-05-29 17:26:50 +09:00
Ashutosh Gupta cc52627ad4
HDFS-16585.Add @VisibleForTesting in Dispatcher.java (#4337) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>

Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit bee538f785)
(cherry picked from commit d921cc71fd)
 2022-05-26 15:28:42 -07:00
Masatake Iwasaki f9b5366be7 HADOOP-18251. Fix failure of extracting JIRA id from commit message in git_jira_fix_version_check.py. (#4344) 
(cherry picked from commit 6b331dde31)
 2022-05-26 03:30:52 +00:00
Michael Stack ad9b6e8d81
HDFS-16586. Purge FsDatasetAsyncDiskService threadgroup; it causes BP… (#4348) 
Remove the ThreadGroup used by executor factories; they are unused
and ThreadGroups auto-destroy when their Thread-member count goes to zero.
This behavior is incompatible with the configuration we have on the per-volume
executor which is set to let all threads die if no use inside the
keepalive time.
 2022-05-25 16:35:56 -07:00
Ashutosh Gupta 2cc21714bc HADOOP-18240. Upgrade Yetus to 0.14.0 (#4328) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 84b0455cf8)
 2022-05-25 17:33:04 +09:00
Owen O'Malley 2113c84260 YARN-11162. Set the zk acl for nodes created by ZKConfigurationStore. (#4350) 
(cherry picked from commit f390edaec4)
 2022-05-24 14:15:35 +09:00
Ashutosh Gupta 6322a774f3 YARN-10080. Support show app id on localizer thread pool (#4283) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6985f9aabe)
(cherry picked from commit d58f9d438f)
 2022-05-19 12:33:23 +09:00
Szilard Nemeth 45801fba8b YARN-11126. ZKConfigurationStore Java deserialisation vulnerability. Contributed by Tamas Domok 2022-05-18 14:23:56 +02:00
Szilard Nemeth 1ae834fe0b YARN-10850. TimelineService v2 lists containers for all attempts when filtering for one. Contributed by Benjamin Teke 2022-05-18 14:04:51 +02:00
Zilong Zhu e88563336c YARN-11133. YarnClient gets the wrong EffectiveMinCapacity value (#4280) 
(cherry picked from commit 79cc0caf9f)
(cherry picked from commit 2dd5852ad0)
 2022-05-18 03:04:05 +09:00
Ashutosh Gupta 45e5b71a1d YARN-11092. Upgrade jquery ui to 1.13.1 (#4260) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 931abbd927)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/pom.xml
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/JQueryUI.java
 2022-05-16 23:44:52 +09:00
litao 48ca153100
HDFS-16579. Fix build failure for TestBlockManager on branch-3.2 (#4310) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2022-05-15 17:59:23 +09:00
Ashutosh Gupta 92c71b1377
HADOOP-17479. Fix the examples of hadoop config prefix (#4197) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 40a8b9a6a5)
 2022-05-08 08:10:08 +09:00
Ashutosh Gupta 4a532f77a9
HDFS-16185. Fix comment in LowRedundancyBlocks.java (#4194) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 01027e52a9)
 2022-05-08 08:06:39 +09:00
jteagles 6daa369bcf
YARN-11116. Migrate Times util from SimpleDateFormat to thread-safe DateTimeFormatter class (#4242) 
Co-authored-by: Jonathan Eagles <jeagles@verizonmedia.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d4a91bd0c0)
 2022-05-03 03:18:25 +09:00
Ashutosh Gupta 7f08b96bcd
YARN-10187. Removing hadoop-yarn-project/hadoop-yarn/README as it is no longer maintained. (#4222) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a74acc755e)
 2022-05-03 03:11:00 +09:00
Ashutosh Gupta ca9f18b7d6 MAPREDUCE-7246. In MapredAppMasterRest#Mapreduce_Application_Master_Info_API, updating the datatype of appId to "string". (#4223) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fb13c1e4a8)
 2022-04-25 14:31:39 +09:00
Renukaprasad C 5f259ef7fd
HDFS-16551. CryptoInputStream#close() should be syncronized (#4227) 2022-04-25 02:19:12 +08:00
litao 0965822ae1 HDFS-16552. Fix NPE for TestBlockManager (#4210) 
(cherry picked from commit 5ebbacc480)
 2022-04-23 12:18:19 +09:00
Inigo Goiri 27621c9eef HDFS-13983. TestOfflineImageViewer crashes in windows. Contributed by Vinayakumar B. 
(cherry picked from commit f069d38c8d)
 2022-04-22 16:00:40 +09:00
daimin 181efebdfe
HDFS-16546. Fix UT TestOfflineImageViewer#testReverseXmlWithoutSnapshotDiffSection (#4187) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-04-22 15:34:28 +09:00
S O'Donnell bacd0b9a92 Revert "HDFS-16531. Avoid setReplication writing an edit record if old replication equals the new value (#4148). Contributed by Stephen O'Donnell." 
This reverts commit 045d48530c.
 2022-04-20 20:57:23 +01:00
qinyuren 4fed01ef21 HDFS-16544. EC decoding failed due to invalid buffer (#4179) 
(cherry picked from commit 76bbd17374)
 2022-04-20 15:08:05 +09:00
Stephen O'Donnell 045d48530c HDFS-16531. Avoid setReplication writing an edit record if old replication equals the new value (#4148). Contributed by Stephen O'Donnell. 
(cherry picked from commit dbeeee0363)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2022-04-19 11:38:34 +01:00
qinyuren d993d22038 HDFS-16538. EC decoding failed due to not enough valid inputs (#4167) 
Co-authored-by: liubingxing <liubingxing@bigo.sg>
(cherry picked from commit 52e152f8b0)
 2022-04-19 13:40:16 +09:00
jianghuazhu ed83a278c2
HDFS-16389.Improve NNThroughputBenchmark test mkdirs. (#3819) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 900682e712)
(cherry picked from commit 4aa9698ee1e9a10a0f882229036bbb4db1ede939)
 2022-04-19 11:13:29 +08:00
daimin 248c548616
Fix decommission UnsupportedOperationException (#4172) 
Reviewed-by: tomscut <litao@bigo.sg>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-04-18 10:40:12 +08:00
Takanobu Asanuma b8c6ba6046 HDFS-16479. EC: NameNode should not send a reconstruction work when the source datanodes are insufficient (#4138) 
(cherry picked from commit 2efab92959)
 2022-04-14 11:52:12 +09:00
qinyuren cb14e8de81 HDFS-16484. [SPS]: Fix an infinite loop bug in SPSPathIdProcessor thread (#4032) 
(cherry picked from commit 45394433a1)
 2022-04-13 11:53:58 +09:00
Viraj Jasani 2d2631afc0 HADOOP-18191. Log retry count while handling exceptions in RetryInvocationHandler (#4133) 
(cherry picked from commit b69ede7154)
 2022-04-11 15:24:24 +09:00
Hanisha Koneru 9cdf5ca105 HADOOP-17116. Skip Retry INFO logging on first failover from a proxy 
(cherry picked from commit e62d8f8412)
 2022-04-11 15:19:54 +09:00
Stephen O'Donnell 72e48ac84e HDFS-16530. setReplication debug log creates a new string even if debug is disabled (#4142) 
(cherry picked from commit bbfe3500cf)
 2022-04-06 13:06:28 +01:00
wangzhaohui 545d3a6cb7 HDFS-16529. Remove unnecessary setObserverRead in TestConsistentReadsObserver (#4131) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 61bbdfd3a7)
 2022-04-06 17:31:42 +09:00
Masatake Iwasaki 2d770e49f1
HADOOP-18192. Fix multiple_bindings warning about slf4j-reload4j. (#4136) 2022-04-06 14:13:19 +09:00
litao 6b24b85e3f HDFS-16507. [SBN read] Avoid purging edit log which is in progress (#4082) 2022-03-30 23:04:04 -07:00