Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,859 Commits 326 Branches 371 Tags
Commit Graph

21859 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Szilard Nemeth
45801fba8b YARN-11126. ZKConfigurationStore Java deserialisation vulnerability. Contributed by Tamas Domok 2022-05-18 14:23:56 +02:00
Szilard Nemeth
1ae834fe0b YARN-10850. TimelineService v2 lists containers for all attempts when filtering for one. Contributed by Benjamin Teke 2022-05-18 14:04:51 +02:00
Zilong Zhu
e88563336c YARN-11133. YarnClient gets the wrong EffectiveMinCapacity value (#4280) 
(cherry picked from commit 79cc0caf9fde8ad6a9170ed7572a73e01b37d9e0)
(cherry picked from commit 2dd5852ad0af558c9f96e83071f78f11203726df)
 2022-05-18 03:04:05 +09:00
Ashutosh Gupta
45e5b71a1d YARN-11092. Upgrade jquery ui to 1.13.1 (#4260) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 931abbd92706aa5377ba303b7b09f0adf1d8e1c7)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/pom.xml
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/JQueryUI.java
 2022-05-16 23:44:52 +09:00
litao
48ca153100
HDFS-16579. Fix build failure for TestBlockManager on branch-3.2 (#4310) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2022-05-15 17:59:23 +09:00
Ashutosh Gupta
92c71b1377
HADOOP-17479. Fix the examples of hadoop config prefix (#4197) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 40a8b9a6a54181b58d1770dd779d637041a631ce)
 2022-05-08 08:10:08 +09:00
Ashutosh Gupta
4a532f77a9
HDFS-16185. Fix comment in LowRedundancyBlocks.java (#4194) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 01027e52a9789eb5b386729f52b7bb9e52fa5352)
 2022-05-08 08:06:39 +09:00
jteagles
6daa369bcf
YARN-11116. Migrate Times util from SimpleDateFormat to thread-safe DateTimeFormatter class (#4242) 
Co-authored-by: Jonathan Eagles <jeagles@verizonmedia.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d4a91bd0c0d9d438ffe95fafee0212499b0abcfd)
 2022-05-03 03:18:25 +09:00
Ashutosh Gupta
7f08b96bcd
YARN-10187. Removing hadoop-yarn-project/hadoop-yarn/README as it is no longer maintained. (#4222) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a74acc755edcde38203416f9d5225343ea877c08)
 2022-05-03 03:11:00 +09:00
Ashutosh Gupta
ca9f18b7d6 MAPREDUCE-7246. In MapredAppMasterRest#Mapreduce_Application_Master_Info_API, updating the datatype of appId to "string". (#4223) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fb13c1e4a8a945328781fa533b5bf22786b830f1)
 2022-04-25 14:31:39 +09:00
Renukaprasad C
5f259ef7fd
HDFS-16551. CryptoInputStream#close() should be syncronized (#4227) 2022-04-25 02:19:12 +08:00
litao
0965822ae1 HDFS-16552. Fix NPE for TestBlockManager (#4210) 
(cherry picked from commit 5ebbacc48023c0c71dd428df40a4352feede373b)
 2022-04-23 12:18:19 +09:00
Inigo Goiri
27621c9eef HDFS-13983. TestOfflineImageViewer crashes in windows. Contributed by Vinayakumar B. 
(cherry picked from commit f069d38c8d3c0bfa91b70a60e4e556ec708fc411)
 2022-04-22 16:00:40 +09:00
daimin
181efebdfe
HDFS-16546. Fix UT TestOfflineImageViewer#testReverseXmlWithoutSnapshotDiffSection (#4187) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-04-22 15:34:28 +09:00
S O'Donnell
bacd0b9a92 Revert "HDFS-16531. Avoid setReplication writing an edit record if old replication equals the new value (#4148). Contributed by Stephen O'Donnell." 
This reverts commit 045d48530cbbd7baa2b67937e103b9400487f1a4.
 2022-04-20 20:57:23 +01:00
qinyuren
4fed01ef21 HDFS-16544. EC decoding failed due to invalid buffer (#4179) 
(cherry picked from commit 76bbd173749f2af4f17946fb37c4c72e2de26764)
 2022-04-20 15:08:05 +09:00
Stephen O'Donnell
045d48530c HDFS-16531. Avoid setReplication writing an edit record if old replication equals the new value (#4148). Contributed by Stephen O'Donnell. 
(cherry picked from commit dbeeee03639f41a022dd07d5fc04e3aa65a94b5f)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2022-04-19 11:38:34 +01:00
qinyuren
d993d22038 HDFS-16538. EC decoding failed due to not enough valid inputs (#4167) 
Co-authored-by: liubingxing <liubingxing@bigo.sg>
(cherry picked from commit 52e152f8b0d5f522f3b799ea72c6c887d5d2c42d)
 2022-04-19 13:40:16 +09:00
jianghuazhu
ed83a278c2
HDFS-16389.Improve NNThroughputBenchmark test mkdirs. (#3819) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 900682e7120e7af250ebf0b4bafb40b412d3e11c)
(cherry picked from commit 4aa9698ee1e9a10a0f882229036bbb4db1ede939)
 2022-04-19 11:13:29 +08:00
daimin
248c548616
Fix decommission UnsupportedOperationException (#4172) 
Reviewed-by: tomscut <litao@bigo.sg>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-04-18 10:40:12 +08:00
Takanobu Asanuma
b8c6ba6046 HDFS-16479. EC: NameNode should not send a reconstruction work when the source datanodes are insufficient (#4138) 
(cherry picked from commit 2efab92959ca0a68c52ed6a9c721704e57afbcc7)
 2022-04-14 11:52:12 +09:00
qinyuren
cb14e8de81 HDFS-16484. [SPS]: Fix an infinite loop bug in SPSPathIdProcessor thread (#4032) 
(cherry picked from commit 45394433a112334e48087bd60674538af739922a)
 2022-04-13 11:53:58 +09:00
Viraj Jasani
2d2631afc0 HADOOP-18191. Log retry count while handling exceptions in RetryInvocationHandler (#4133) 
(cherry picked from commit b69ede7154d44538a4a66824c34f7ba143deef25)
 2022-04-11 15:24:24 +09:00
Hanisha Koneru
9cdf5ca105 HADOOP-17116. Skip Retry INFO logging on first failover from a proxy 
(cherry picked from commit e62d8f841275ee47a0ba911415aac9e39af291c6)
 2022-04-11 15:19:54 +09:00
Stephen O'Donnell
72e48ac84e HDFS-16530. setReplication debug log creates a new string even if debug is disabled (#4142) 
(cherry picked from commit bbfe3500cf76ce661e0b10bb5d16cb419f9b9b10)
 2022-04-06 13:06:28 +01:00
wangzhaohui
545d3a6cb7 HDFS-16529. Remove unnecessary setObserverRead in TestConsistentReadsObserver (#4131) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 61bbdfd3a73a18cf05aeb2c40429c34c4b9b0732)
 2022-04-06 17:31:42 +09:00
Masatake Iwasaki
2d770e49f1
HADOOP-18192. Fix multiple_bindings warning about slf4j-reload4j. (#4136) 2022-04-06 14:13:19 +09:00
litao
6b24b85e3f HDFS-16507. [SBN read] Avoid purging edit log which is in progress (#4082) 2022-03-30 23:04:04 -07:00
Kengo Seki
5c6547d794 MAPREDUCE-7373. Building MapReduce NativeTask fails on Fedora 34+ (#4120) 
(cherry picked from commit dc4a680da8bcacf152cc8638d86dd171a7901245)
 2022-03-30 22:51:40 +09:00
Masatake Iwasaki
228ad1ac25 Make upstream aware of 3.2.3 release. 
(cherry picked from commit 0fbd96a2449ec49f840d93e1c7d290c5218ef4ea)
 2022-03-28 08:09:58 +00:00
Masatake Iwasaki
5c16db8b17 Make upstream aware of 3.2.3 release. 
(cherry picked from commit 10876333acd4194da3cb545c4f7a2b2cc88be959)
 2022-03-28 08:04:47 +00:00
GuoPhilipse
ff03212145 HDFS-16355. Improve the description of dfs.block.scanner.volume.bytes.per.second (#3724) 
Co-authored-by: gf13871 <gf13871@ly.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 046a6204b4a895b98ccd41dde1c9524a6bb0ea31)
 2022-03-27 22:33:17 +09:00
Akira Ajisaka
73775fe277
YARN-10720. YARN WebAppProxyServlet should support connection timeout to prevent proxy server from hanging. Contributed by Qi Zhu. (#4102) 
(cherry picked from commit a0deda1a777d8967fb8c08ac976543cda895773d)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestWebAppProxyServlet.java

Co-authored-by: Peter Bacsko <pbacsko@cloudera.com>
 2022-03-24 17:36:44 +09:00
qinyuren
66be0a713f HDFS-16501. Print the exception when reporting a bad block (#4062) 
Reviewed-by: tomscut <litao@bigo.sg>
(cherry picked from commit 45ce1cce50c3ff65676d946e96bbc7846ad3131a)
 2022-03-23 15:06:20 +09:00
Masatake Iwasaki
e13408d9d2
HADOOP-18088. Replace log4j 1.x with reload4j. (#4084) 
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-03-23 12:57:27 +09:00
Surendra Singh Lilhore
58fe5b2f35 HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina. 
(cherry picked from commit 5ff76cb8bc69d68ba7c9487d00b1dc753d616bb2)
 2022-03-18 18:39:20 +09:00
Wei-Chiu Chuang
826d13a59a HADOOP-18155. Refactor tests in TestFileUtil (#4063) 
(cherry picked from commit d0fa9b5775185bd83e4a767a7dfc13ef89c5154a)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java

Co-authored-by: Gautham B A <gautham.bangalore@gmail.com>
(cherry picked from commit 743db6e7b4d2acd893f527c9312e6dbe6eefef4f)
 2022-03-14 10:05:01 +09:00
belugabehr
d09ca8bceb HADOOP-16811: Use JUnit TemporaryFolder Rule in TestFileUtils (#1811). Contributed by David Mollitor. 
(cherry picked from commit 1afd54fbbb858a58112e6290b3063216eea82206)
 2022-03-14 10:04:47 +09:00
Thinker313
6f815a546a
HDFS-16428. Source path with storagePolicy cause wrong typeConsumed while rename (#3898). Contributed by lei w. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 43153e80cbfc924fde380e6606c04455341edeab)
 2022-03-12 20:10:44 +08:00
Wei-Chiu Chuang
c531613cc9 HADOOP-17718. Explicitly set locale in the Dockerfile. (#3034) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ad923ad5642b2b11357fbee4277f3435300a19c5)

Conflicts:
	dev-support/docker/Dockerfile_centos_7
	dev-support/docker/Dockerfile_centos_8
(cherry picked from commit 5e09d38b2505db70c712b8b6c93635220325156f)

Conflicts:
	dev-support/docker/Dockerfile_aarch64
 2022-03-10 09:56:04 +00:00
Akira Ajisaka
523bb2efc3 YARN-9783. Remove low-level zookeeper test to be able to build Hadoop against zookeeper 3.5.5. Contributed by Mate Szalay-Beko. 
(cherry picked from commit dc0acceabb6a5189974109cfea09f598c2a99d14)
 2022-03-10 07:41:14 +00:00
srinivasst
c64eda080e YARN-10538: Add RECOMMISSIONING nodes to the list of updated nodes returned to the AM (#2564) 
Contributed by Srinivas S T

(cherry picked from commit 1b1791075a8a45ff8c7d25d52cb014d0ce858cc2)
(cherry picked from commit 98565b6c60c4c1cefaabe4a1fa5fe92158b19b97)
 2022-03-10 16:35:40 +09:00
Szilard Nemeth
9e2e1c795f YARN-11014. YARN incorrectly validates maximum capacity resources on the validation API. Contributed by Benjamin Teke 2022-03-02 14:21:36 +01:00
Szilard Nemeth
4bd67d7516 YARN-11075. Explicitly declare serialVersionUID in LogMutation class. Contributed by Benjamin Teke 2022-03-01 18:06:39 +01:00
Akira Ajisaka
06a2e4ccde HADOOP-16717. Remove GenericsUtil isLog4jLogger dependency on Log4jLoggerAdapter. Contributed by Xieming Li. 
(cherry picked from commit da9435182a8fccec743406e089325655138d9de8)
 2022-02-28 10:54:06 +09:00
Steve Loughran
adfd767225 HADOOP-17112. S3A committers can't handle whitespace in paths. (#2953) 
Contributed by Krzysztof Adamski.

(cherry picked from commit 88a550bc3a73dbbbfb35e2ef29d9c34ad059d1a8)

 Conflicts:
	hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/ITestCommitOperations.java
 2022-02-28 10:41:47 +09:00
Ayush Saxena
551f8d3ddc
HDFS-11041. Unable to unregister FsDatasetState MBean if DataNode is shutdown twice. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit e8cb2ae409bc1d62f23efef485d1c6f1ff21e86c)
(cherry picked from commit fa30224e95d920adaea0ec65c360359044e3f754)
 2022-02-23 11:41:35 +08:00
Viraj Jasani
9d618bcc6b
HADOOP-18125. Utility to identify git commit / Jira fixVersion discrepancies for RC preparation (#3991) 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 697e5d463640a7107a622262eb2d333d0458fd8b)
(cherry picked from commit d763c997071752c351ceb689716c8d44803f29b4)
 2022-02-22 11:01:51 +08:00
Steve Loughran
a8512d6627
HADOOP-18136. Verify FileUtils.unTar() handling of missing .tar files. 
Contributed by Steve Loughran

Change-Id: Id6f3f2b0eeeb186a757d8b6ee5a0584cb4114469
 2022-02-21 17:18:03 +00:00
Chentao Yu
5066722eb7 HADOOP-18109. Ensure that default permissions of directories under internal ViewFS directories are the same as directories on target filesystems. Contributed by Chentao Yu. (3953) 
(cherry picked from commit 19d90e62fb28539f8c79bbb24f703301489825a6)
 2022-02-15 16:07:46 -08:00