Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,742 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

21742 Commits

Author SHA1 Message Date
Shubham Gupta ccb711544b
YARN-10438. Handle null containerId in ClientRMService#getContainerReport() (#2313) 
Co-authored-by: Shubham Gupta <gshubham@microsoft.com>
(cherry picked from commit e3cd627069)
 2021-11-19 00:25:14 +09:00
Takanobu Asanuma 210464b08f
HDFS-15963. Unreleased volume references cause an infinite loop. Contributed by Shuyan Zhang. (#2941, #3668) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 9f2db2c9fd)

Co-authored-by: zhangshuyan0 <81411509+zhangshuyan0@users.noreply.github.com>
 2021-11-18 16:32:54 +09:00
Abhishek Das 0e02f03eae HADOOP-17999. No-op implementation of setWriteChecksum and setVerifyChecksum in ViewFileSystem. Contributed by Abhishek Das. (#3639) 
(cherry picked from commit 54a1d78e16)
 2021-11-16 22:21:14 -08:00
litao 81f6cbc715 HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode (#3666) 
(cherry picked from commit 026d5860cb)
 2021-11-17 13:53:33 +09:00
Neil 78c56f39b4
HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. (#3320) 
(cherry picked from commit 5626734a36)
(cherry picked from commit ce3a7155335416ed4665f90f0ddd7ac3a3edd7e6)
 2021-11-16 22:59:35 +08:00
Gautham B A 980c35ec2d
HADOOP-17993. Disable JIRA plugin for YETUS on Hadoop (#3624) 2021-11-12 19:56:42 +05:30
Wei-Chiu Chuang 002b5fbdc0
HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing (#3340) (#3640) 
(cherry picked from commit 356ebbbe80)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java

Change-Id: If76fd0d77fafc90fe2f2c19ab1d0c43a58510f6b

Co-authored-by: bshashikant <shashikant@apache.org>
(cherry picked from commit 5333e872e2)
 2021-11-12 15:03:15 +08:00
litao c95df2eb6f HDFS-16311. Metric metadataOperationRate calculation error in DataNodeVolumeMetrics (#3636) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 1c1cf64616)
 2021-11-12 10:26:19 +09:00
litao ee871a6afd HDFS-16312. Fix typo for DataNodeVolumeMetrics and ProfilingFileIoEvents (#3637) 
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 2a1a11c039)
 2021-11-10 19:24:32 +09:00
litao ba61dd9b73 HDFS-16298. Improve error msg for BlockMissingException (#3615) 
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit ea65fc26d8)
 2021-11-10 19:18:56 +09:00
jianghuazhu 9ddafb2eee HDFS-16301. Improve BenchmarkThroughput#SIZE naming standardization. (#3622) 
(cherry picked from commit b307f18a58)
 2021-11-10 11:40:28 +09:00
litao b103be470c HDFS-16299. Fix bug for TestDataNodeVolumeMetrics#verifyDataNodeVolumeMetrics (#3616) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit cbc7fb6bca)
 2021-11-09 17:10:01 +09:00
Jinhu Wu 2b6f7bf209 HADOOP-17374. support listObjectV2 (#3587) 
(cherry picked from commit a9c51ea57d)
 2021-11-04 21:44:12 -07:00
jianghuazhu 06c4058a61 HDFS-16294.Remove invalid DataNode#CONFIG_PROPERTY_SIMULATED. (#3605) 
(cherry picked from commit 72718a8dc1)
 2021-11-04 16:51:14 +09:00
daimin 29fd36e2f1 HDFS-16286. Add a debug tool to verify the correctness of erasure coding on file (#3593) 
(cherry picked from commit a21895a5b3)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java
 2021-11-03 21:30:48 +00:00
litao b1801e99de HDFS-16279. Print detail datanode info when process first storage report (#3564) 
(cherry picked from commit 6aa664c624)
 2021-10-29 10:29:32 +09:00
Ananya Singh df9c07fc04
HADOOP-17972. Backport HADOOP-17683 for branch-3.2 (#3573) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-10-28 11:46:12 +09:00
Ahmed Hussein c36d8a1395 YARN-1115: Provide optional means for a scheduler to check real user ACLs. Contributed by Eric Payne (epayne) 2021-10-22 15:54:13 +00:00
Michael Kuchenbecker 7de539f3ca HDFS-7612: Fix default cache directory in TestOfflineEditsViewer.testStored. Contributed by Michael Kuchenbecker (#3571) 
(cherry picked from commit a73ff6915a)
 2021-10-21 13:50:54 -07:00
Szilard Nemeth 7ebc18e8fe HADOOP-17857. Check real user ACLs in addition to proxied user ACLs. Contributed by Eric Payne 
(cherry picked from commit 5428d36b56)
 2021-10-21 17:15:39 +00:00
Nicholas Marion 87869c6c26
HADOOP-17971. Exclude IBM Java security classes from being shaded/relocated. (#3567) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3e1135469e)
 2021-10-20 14:30:29 +09:00
daimin 9f2c89e690 HDFS-16272. Fix int overflow in computing safe length during EC block recovery (#3548) 
(cherry picked from commit 5337bebcc5)
 2021-10-18 12:20:43 +01:00
Xing Lin c8f22fe737 HADOOP-16532. Fix TestViewFsTrash to use the correct homeDir. Contributed by Xing Lin. (#3514) 
(cherry picked from commit 97c0f96879)
 2021-10-13 14:56:53 -07:00
Masatake Iwasaki c7b40e43f5 HADOOP-17925. BUILDING.txt should not encourage to activate docs profile on building binary artifacts. (#3509) 
(cherry picked from commit 65d9523aa2)
 2021-10-11 08:58:09 +00:00
langlaile1221 88744b8a04
HDFS-16241. Standby close reconstruction thread (#3493) 
Co-authored-by: zhanghuazong <Zhz@20068867>
(cherry picked from commit 88d8c3d0c8)
(cherry picked from commit a35a6ec06b)
 2021-10-11 16:22:59 +08:00
Brahma Reddy Battula 675712afa3 HADOOP-17236. Bump up snakeyaml to 1.26 to mitigate CVE-2017-18640. Contributed by Brahma Reddy Battula. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit eb84793af1)
 2021-10-08 16:58:05 +05:30
Siyao Meng c8db6b1eb2
HADOOP-17834. Bump aliyun-sdk-oss to 3.13.0 (#3261) 
Change-Id: I335d4a2cb08c75dc24ef36bdfab51111f87e0762
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3aaac8a1f6)

 Conflicts:
	LICENSE-binary
 2021-10-08 14:58:38 +09:00
Takanobu Asanuma 7a12baeef9 HADOOP-17955. Bump netty to the latest 4.1.68. (#3528) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit cf4d2d826c)
 2021-10-08 10:39:40 +09:00
Ahmed Hussein e99af5cf81 HADOOP-17930. implement non-guava Precondition checkState (#3522) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit c36f9402dc)
 2021-10-07 11:02:51 +09:00
Ahmed Hussein 4b6316a7fd HADOOP-17929. implement non-guava Precondition checkArgument (#3473) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 0c498f21de)
 2021-10-07 11:02:45 +09:00
Rohith Sharma K S 51d17f3cb3 YARN-8936. Bump up Atsv2 hbase versions. Contributed by Vrushali C. 
(cherry picked from commit f63e4e4f2e)

Conflicts:
	hadoop-project/pom.xml
 2021-10-06 20:49:00 +09:00
Viraj Jasani 28d4affea0 HADOOP-17947. Additional element types for VisibleForTesting (ADDENDUM) (#3521) 
(cherry picked from commit 783e4805e7)
 2021-10-06 02:19:44 +09:00
Viraj Jasani ce30f796f6 HADOOP-17947. Provide alternative to Guava VisibleForTesting (#3505) 
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 5b1d594005)
 2021-10-05 10:02:08 +09:00
AngersZhuuuu 1df3b3de46
HDFS-16235. Fix Deadlock in LeaseRenewer for static remove method (#3472) 
(cherry picked from commit 5f9321a5d4)
 2021-09-29 17:44:32 +08:00
He Xiaoqiao cfd0265975
HDFS-14575. LeaseRenewer#daemon threads leak in DFSClient. Contributed by Renukaprasad C. 
Co-authored-by: Tao Yang <taoyang1@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 10b79a26fe)
 2021-09-29 17:44:00 +08:00
Takanobu Asanuma 3851ace026 HADOOP-17940. Upgrade Kafka to 2.8.1 (#3488) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 2068b0041c)
 2021-09-28 13:36:40 +09:00
Inigo Goiri 86028d975c
HDFS-15092. TestRedudantBlocks#testProcessOverReplicatedAndRedudantBlock sometimes fails. Contributed by Fei Hui. 
(cherry picked from commit 8cfc3673dc)
 2021-09-27 14:52:27 +09:00
Wei-Chiu Chuang 978ec57445 HDFS-16233. Do not use exception handler to implement copy-on-write for EnumCounters. (#3468) 
(cherry picked from commit 87632bbacf)
(cherry picked from commit bb08de559a)
 2021-09-24 09:15:04 -07:00
Akira Ajisaka 85346db636
HADOOP-17917. Backport HADOOP-15993 to branch-3.2 which address CVE-2014-4611. Contributed by Brahma Reddy Battula. 2021-09-24 14:59:21 +09:00
Eric Badger ce3def0aa7 YARN-10935. AM Total Queue Limit goes below per-user AM Limit if parent is full. Contributed by Eric Payne. 2021-09-22 20:29:12 +00:00
Rintaro Ikeda 5de18fb483 HADOOP-17926. Maven-eclipse-plugin is no longer needed since Eclipse can import Maven projects by itself. (#3465) 
(cherry picked from commit 962068d2d8)
 2021-09-22 13:09:38 +00:00
He Xiaoqiao 4944dafa29
HDFS-15113. Addendum: Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit af64ce2f4a)
 2021-09-17 22:16:12 +08:00
He Xiaoqiao 50b0f0dc42
HDFS-15113. Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Brahma Reddy Battula <brahma@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
(cherry picked from commit e9955bb8ff)
 2021-09-17 22:16:12 +08:00
He Xiaoqiao fc5fe67b3f
HDFS-15651. Client could not obtain block when DN CommandProcessingThread exit. Contributed by Aiphago. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Yiqun Lin <yqlin@apache.org>
(cherry picked from commit 3067a25fa1)
 2021-09-17 22:16:11 +08:00
Inigo Goiri 5da46e197b
HDFS-15075. Remove process command timing from BPServiceActor. Contributed by Xiaoqiao He. 
(cherry picked from commit cdcb77a2c5)
 2021-09-17 22:16:11 +08:00
Ayush Saxena f77965b29e
HDFS-14997. Addendum: BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He. 
(cherry picked from commit 80f91d14ab)
 2021-09-17 22:16:11 +08:00
Inigo Goiri d5501c125f
HDFS-14997. BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He. 
(cherry picked from commit b86895485d)
 2021-09-17 22:16:07 +08:00
Rintaro Ikeda a777264c7f HADOOP-17919. Fix command line example in Hadoop Cluster Setup documentation. (#3453) 
(cherry picked from commit 607c20c612)
 2021-09-17 13:35:54 +00:00
Steve Loughran 5094dede8e
HADOOP-17126. implement non-guava Precondition checkNotNull 
This adds a new class org.apache.hadoop.util.Preconditions which is

* @Private/@Unstable
* Intended to allow us to move off Google Guava
* Is designed to be trivially backportable
  (i.e contains no references to guava classes internally)

Please use this instead of the guava equivalents, where possible.

Contributed by: Ahmed Hussein

Change-Id: Ic392451bcfe7d446184b7c995734bcca8c07286e
 2021-09-17 11:37:29 +01:00
wangzhaohui 928dbac0a6 HDFS-16181. [SBN Read] Fix display of JournalNode metric RpcRequestCacheMissAmount (#3317) 
Co-authored-by: wangzhaohui8 <wangzhaohui8@jd.com>

(cherry picked from commit 232fd7cae170de8c6b52c14841a47dca8735c6d2)
(cherry picked from commit 2f73ac1c14)
 2021-09-15 10:06:27 -07:00