Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,686 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

21686 Commits

Author SHA1 Message Date
He Xiaoqiao 1d9a6ffae5
HDFS-14575. LeaseRenewer#daemon threads leak in DFSClient. Contributed by Renukaprasad C. 
Co-authored-by: Tao Yang <taoyang1@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 10b79a26fe)
(cherry picked from commit cfd0265975)
 2021-09-29 18:36:10 +08:00
Wei-Chiu Chuang 6ef0b3fc21
HDFS-16233. Do not use exception handler to implement copy-on-write for EnumCounters. (#3468) 
(cherry picked from commit 87632bbacf)
(cherry picked from commit bb08de559a)
(cherry picked from commit 978ec57445)
 2021-09-28 16:26:55 +09:00
Takanobu Asanuma 9b016b401a HADOOP-17940. Upgrade Kafka to 2.8.1 (#3488) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 2068b0041c)
 2021-09-28 13:43:30 +09:00
Inigo Goiri 1f55483339
HDFS-15092. TestRedudantBlocks#testProcessOverReplicatedAndRedudantBlock sometimes fails. Contributed by Fei Hui. 
(cherry picked from commit 8cfc3673dc)
(cherry picked from commit 86028d975c)
 2021-09-27 14:52:51 +09:00
Akira Ajisaka 91acb7b9fe
HADOOP-17917. Backport HADOOP-15993 to branch-3.2 which address CVE-2014-4611. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 85346db636)
 2021-09-24 15:00:12 +09:00
He Xiaoqiao 7fea1dc006
HDFS-15113. Addendum: Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit af64ce2f4a)
(cherry picked from commit 4944dafa29)
 2021-09-22 18:14:51 +08:00
He Xiaoqiao 56f7e52be5
HDFS-15113. Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Brahma Reddy Battula <brahma@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
(cherry picked from commit e9955bb8ff)
(cherry picked from commit 50b0f0dc42)
 2021-09-22 18:14:46 +08:00
He Xiaoqiao c7213e4d2b
HDFS-15651. Client could not obtain block when DN CommandProcessingThread exit. Contributed by Aiphago. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Yiqun Lin <yqlin@apache.org>
(cherry picked from commit 3067a25fa1)
(cherry picked from commit fc5fe67b3f)
 2021-09-22 18:14:41 +08:00
Inigo Goiri 79187b5a5e
HDFS-15075. Remove process command timing from BPServiceActor. Contributed by Xiaoqiao He. 
(cherry picked from commit cdcb77a2c5)
(cherry picked from commit 5da46e197b)
 2021-09-22 18:14:35 +08:00
Ayush Saxena cedb8fec9d
HDFS-14997. Addendum: BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He. 
(cherry picked from commit 80f91d14ab)
(cherry picked from commit f77965b29e)
 2021-09-22 18:14:29 +08:00
Inigo Goiri baa4190eab
HDFS-14997. BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He. 
(cherry picked from commit b86895485d)
(cherry picked from commit d5501c125f)
 2021-09-22 18:14:21 +08:00
Ahmed Hussein 0d0f344d65
HDFS-15160. amend to fix javac error supressing unchecked warning 2021-09-14 14:26:32 +08:00
LeonGao 38973a52f0
HDFS-15818. Fix TestFsDatasetImpl.testReadLockCanBeDisabledByConfig. Contributed by Leon Gao (#2679) 
(cherry picked from commit 9434c1eccc)
 2021-09-14 14:26:27 +08:00
Ahmed Hussein 0879f4ec0f
HDFS-15457. TestFsDatasetImpl fails intermittently (#2407) 
(cherry picked from commit 98097b8f19)
 2021-09-14 14:26:22 +08:00
Stephen O'Donnell 84e16adab3
HDFS-15160. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 2a67e2b1a0)
 2021-09-14 14:26:09 +08:00
Stephen O'Donnell 3c715a272a
HDFS-15150. Introduce read write lock to Datanode. Contributed Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit d7c136b9ed)
 2021-09-14 14:25:57 +08:00
Ahmed Hussein a9dbe47a72
HADOOP-17886. Upgrade ant to 1.10.11 (#3371) 
(cherry picked from commit 051207375b)
(cherry picked from commit 601591c021)
 2021-09-03 10:17:41 +09:00
Szilard Nemeth 867acc2e6d
YARN-10428. Zombie applications in the YARN queue using FAIR + sizebasedweight. Contributed by Guang Yang, Andras Gyori 
(cherry picked from commit 79a46599f7)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java

(cherry picked from commit 7aea2e1b5c)
 2021-09-01 13:16:05 +09:00
Xiaoyu Yao 3c79d65272
HADOOP-15939. Filter overlapping objenesis class in hadoop-client-minicluster. Contributed by Xiaoyu Yao. 
(cherry picked from commit 397f523e22)
(cherry picked from commit 3ea74033bf)
 2021-08-25 21:10:12 +09:00
Masatake Iwasaki bcd81a7fb4 HADOOP-14922. Build of Mapreduce Native Task module fails with unknown opcode "bswap". Contributed by Anup Halarnkar. 
(cherry picked from commit 0d59500e8c)
 2021-08-25 11:50:49 +09:00
Masatake Iwasaki 996cbe453a
HADOOP-17850. Upgrade ZooKeeper to 3.4.14 in branch-3.2. (#3308) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 037875ba79)
 2021-08-17 13:37:18 +09:00
Masatake Iwasaki 462b808d42 HADOOP-17849. Exclude spotbugs-annotations from transitive dependencies on branch-3.2. (#3305) 
(cherry picked from commit 786888b8d3)
 2021-08-16 21:01:33 +00:00
Renukaprasad C fd1d138a62 HADOOP-17844. Upgrade JSON smart to 2.4.7 (#3299) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b90389ae98)

 Conflicts:
	LICENSE-binary

(cherry picked from commit d18c0a1d25)
 2021-08-14 20:28:26 +09:00
Brahma Reddy Battula 182167a9fb Preparing for 3.2.3 release. 2021-08-08 19:17:14 +05:30
Akira Ajisaka 840c6ef7fa
HADOOP-17370. Upgrade commons-compress to 1.21 (#3274) 
(cherry picked from commit 3565c9477d)

 Conflicts:
	LICENSE-binary
 2021-08-08 11:26:03 +09:00
Bryan Beaudreault dab829063d
HADOOP-17840: Backport HADOOP-17837 to branch-3.2 (#3275) 
Reviewed-by: Brahma Reddy Battula <brahma@apache.org>
 2021-08-07 09:49:46 +05:30
Akira Ajisaka 78c5fdd90d
Fix potential heap buffer overflow in hdfs.c. Contributed by Igor Chervatyuk. 
(cherry picked from commit 4972e7a246)
 2021-08-07 11:28:22 +09:00
wangzhaohui b98a8d259b
HDFS-16154. TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely (#3270) 
Co-authored-by: wangzhaohui8 <wangzhaohui8@jd.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit e85c44657c)
(cherry picked from commit 7068e18842)
 2021-08-06 18:25:28 +08:00
wangzhaohui 6b10154123 HDFS-16153. Avoid evaluation of LOG.debug statement in QuorumJournalManager (#3269). Contributed by wangzhaohui. 
(cherry picked from commit a73b64f86b)
 2021-08-06 09:52:51 +01:00
Viraj Jasani 2f5e527910 HADOOP-17808. Avoid excessive logging for interruption (ADDENDUM) (#3267) 
(cherry picked from commit 9fe1f24ec1)
 2021-08-06 09:54:34 +08:00
Akira Ajisaka c40abe10c0
YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3265) 
* YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3081)

Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit ca27744770)

* Use mockito 1.x

Co-authored-by: Viraj Jasani <vjasani@apache.org>
 2021-08-05 10:05:10 +09:00
Abhishek Modi c5e2ec465b
YARN-9990. Testcase fails with Insufficient configured threads: required=16 < max=10. Contributed by Prabhu Joseph. 
(cherry picked from commit a2dadac790)
 2021-08-04 17:05:35 +09:00
Jim Brennan 1f9a315625
YARN-10337. Amendment to fix import as in HADOOP-17100 
(cherry picked from commit 53a1c7653f)
 2021-08-04 16:38:52 +09:00
Prabhu Joseph b9d0412fd5
YARN-10337. Fix failing testcase TestRMHATimelineCollectors. 
Contributed by Bilwa S T.

(cherry picked from commit 2bbd00dff4)
 2021-08-04 16:37:47 +09:00
Akira Ajisaka d830e84217
YARN-9875. Improve fair scheduler configuration store on HDFS. (#3262) 
Contributed by Prabhu Joseph

(cherry picked from commit 155864da00)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/conf/TestFSSchedulerConfigurationStore.java

Co-authored-by: Eric Yang <eyang@apache.org>
 2021-08-04 16:12:54 +09:00
Giovanni Matteo Fumarola 2f3c0b8dd1
HDFS-14189. Fix intermittent failure of TestNameNodeMetrics. Contributed by Ayush Saxena. 
(cherry picked from commit 1a08302897)
 2021-08-04 15:54:08 +09:00
Akira Ajisaka b838647b16
HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data. (#3264) 
* HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data. (#2003)

(cherry picked from commit 328eae9a14)

* Add assertj-core dependency to hadoop-common for test
* Bump assertj-core version to 3.12.2 to use assertThat.hasSizeGreaterThan
 2021-08-04 15:46:00 +09:00
Akira Ajisaka e259d85a99
YARN-8992. Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue. (Contributed by Wilfred Spiegelenburg) (#3257) 
(cherry picked from commit a41b648e98)

Co-authored-by: Haibo Chen <haibochen@apache.org>
 2021-08-04 10:17:51 +09:00
Siyao Meng fe59fe6744 HDFS-16055. Quota is not preserved in snapshot INode (#3078) 
(cherry picked from commit ebee2aed00)
(cherry picked from commit 72508e6430)
 2021-08-03 20:45:17 +01:00
Akira Ajisaka 786a43e729
YARN-8990. Fix fair scheduler race condition in app submit and queue cleanup. (Contributed by Wilfred Spiegelenburg) (#3254) 
(cherry picked from commit 524a7523c4)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java

Co-authored-by: Haibo Chen <haibochen@apache.org>
 2021-08-03 13:12:01 +09:00
Vrushali C 521c1085b4
YARN-9338 Timeline related testcases are failing. Contributed by Abhishek Modi. 
(cherry picked from commit 17a3e14d25)
 2021-08-03 11:42:28 +09:00
jenny 18ba3d6b1e
MAPREDUCE-7258. HistoryServerRest.html#Task_Counters_API, modify the jobTaskCounters's itemName from taskcounterGroup to taskCounterGroup (#1808) 
Co-authored-by: chenjuanni <chenjuanni@inspur.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e31169c864)
 2021-08-02 15:40:14 +09:00
Szilard Nemeth e4df745c13 YARN-6221. Entities missing from ATS when summary log file info got returned to the ATS before the domain log. Contributed by Xiaomin Zhang 2021-07-31 14:54:51 +02:00
Szilard Nemeth 7a8b6265c6 YARN-10789. RM HA startup can fail due to race conditions in ZKConfigurationStore. Contributed by Tarun Parimi 2021-07-29 19:22:57 +02:00
He Xiaoqiao f46a9e229e
HDFS-15175. Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog. Contributed by Wan Chang. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Stephen O'Donnell <sodonnel@apache.org>
(cherry picked from commit 683feaa1d4)
 2021-07-29 11:26:09 +08:00
hchaverr 4f602cb17b HADOOP-17819. Add extensions to ProtobufRpcEngine RequestHeaderProto. Contributed by Hector Sandoval Chaverri. (#3242) 
(cherry picked from commit 3c8a48e681)
 2021-07-28 15:59:45 -07:00
9uapaw 1dc343726c YARN-9551. TestTimelineClientV2Impl.testSyncCall fails intermittent (#3212) 
Co-authored-by: Andras Gyori <gandras@cloudera.com>
(cherry picked from commit b4a524722a)
(cherry picked from commit f001595b47)
 2021-07-28 17:14:33 +02:00
Szilard Nemeth 6bf02a63e7 YARN-10813. Set default capacity of root for node labels. Contributed by Andras Gyori 2021-07-28 14:54:36 +02:00
Akira Ajisaka 7fe4f3cb89 HDFS-12920. HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop. (#3227) 
Revert "HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin"

This reverts commit b6d839a60c.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

(cherry picked from commit 2f2f822488)
 2021-07-26 01:54:57 +09:00
Viraj Jasani 1df581a20c HADOOP-17808. ipc.Client to set interrupt flag after catching InterruptedException (#3219) 
(cherry picked from commit 3a52bfc5db)
 2021-07-23 12:57:36 +08:00