Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,423 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

15423 Commits

Author SHA1 Message Date
S O'Donnell caa59d7a58 HDFS-15150. Introduce read write lock to Datanode. Contributed by Stephen O'Donnell and Ahmed Hussein 2021-06-22 16:24:19 +01:00
Kihwal Lee ab6b5681e8 HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. 2021-06-16 11:38:30 -05:00
Kihwal Lee e7c7fb2896 HDFS-15963. Unreleased volume references cause an infinite loop. (#2941) Contributed by Shuyan Zhang. 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 9f2db2c9fd)
 2021-06-15 19:57:21 -05:00
Takanobu Asanuma a690e8ae59 HDFS-16068. WebHdfsFileSystem has a possible connection leak in connection with HttpFS (#3104) 
(cherry picked from commit 839fcf7682)
 2021-06-15 17:24:27 +09:00
Takanobu Asanuma f22ff63a15 HADOOP-17756. Increase precommit job timeout from 20 hours to 24 hours. (#3090) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit ae9630f580)
 2021-06-14 10:23:00 +09:00
Hideyuki Furue a07293f819
Fix container-executor 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9c7b8cf54e)
(cherry picked from commit f5568e15d0)
 2021-06-06 09:14:18 +09:00
Konstantin V Shvachko 149325484f HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit 8ce30f51f9)
 2021-05-27 20:19:44 -07:00
Konstantin V Shvachko 8715a18f95 HDFS-15915. Race condition with async edits logging due to updating txId outside of the namesystem log. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 1abd03d68f)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java
 2021-05-26 16:00:18 -07:00
Akira Ajisaka 76eea30bdd
YARN-10770. container-executor permission is wrong in SecureContainer.md. Contributed by Siddharth Ahuja. 
(cherry picked from commit 1cbcde04f3)
 2021-05-25 18:30:21 +09:00
lujiefsi 13a2e751e0
YARN-10555. Missing access check before getAppAttempts (#2608) 
Co-authored-by: lujie <lujie@foxmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d92a25b790)
 2021-05-17 19:47:03 +09:00
hchaverri 7f6185b17c
HDFS-15912. Allow ProtobufRpcEngine to be extensible (#2999) 2021-05-17 00:37:07 -07:00
Hui Fei 77d9c6d0f7 HDFS-15652. Make block size from NNThroughputBenchmark configurable (#2416) 
(cherry picked from commit 8a6d5b9151)
 2021-04-29 12:56:57 -07:00
Eric Badger 7b3a6e96d9 YARN-10479. Can't remove all node labels after add node label without 
nodemanager port, broken by YARN-10647. Contributed by D M Murali Krishna Reddy

(cherry picked from commit 6857a05d6a)
 2021-04-23 23:10:50 +00:00
Eric Badger 58b33d6570 MAPREDUCE-7302. Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail. Contributed by Peter Bacsko. Reviewed by Akira Ajisaka. 
Conflicts only in imports

(cherry picked from commit da93cd962c)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java

(cherry picked from commit 7ad844ee9e)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestFetcher.java
 2021-04-23 20:16:11 +00:00
bibinchundatt 94c1cf2ff2 YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie. 
(cherry picked from commit 6d80b9bc3f)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/scheduler/ContainerScheduler.java

(cherry picked from commit 4a39165b41)
 2021-04-23 16:48:04 +00:00
Eric Badger eb176bb2af YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko 2021-04-20 00:31:08 +00:00
Ahmed Hussein 55645d78e1
YARN-10733. TimelineService Hbase tests failing with timeouts (#2899) 2021-04-14 13:44:38 -05:00
Ahmed Hussein 1056ca3725
HADOOP-17601. Upgrade Jackson databind in branch-2.10 to 2.9.10.7. (#2835) 2021-04-12 17:42:40 -07:00
Ahmed Hussein cb5e41ac07
HADOOP-17603. Upgrade tomcat-embed-core to 7.0.108 (#2834). Contributed by Ahmed Hussein 2021-04-12 10:11:36 -07:00
Viraj Jasani 3c1c1b40d8
HDFS-15931 : Fix non-static inner classes for better memory management (#2830). Contributed by Viraj Jasani 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-04-01 17:48:27 -07:00
Akira Ajisaka d822bc3def
YARN-10716. Fix typo in ContainerRuntime. Contributed by xishuhai. 
(cherry picked from commit 8ad77a26c9)
 2021-03-30 17:49:01 +09:00
Eric Badger 616256b400 YARN-10501. Can't remove all node labels after add node label without 
nodemanager port. Contributed by caozhiqiang.
 2021-03-29 18:11:58 +00:00
Akira Ajisaka f207d9e620
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin. Contributed by Ahmed Hussein. 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.
 2021-03-29 09:44:28 +09:00
Jim Brennan 7a3a749663 MAPREDUCE-7332. Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10. Contributed by Ahmed Hussein. 2021-03-26 15:39:59 +00:00
Akira Ajisaka 4829e1b7ed
HADOOP-17602. Upgrade JUnit to 4.13.1. Contributed by Ahmed Hussein. 2021-03-25 20:30:34 +09:00
Jim Brennan 33c4d4570d YARN-10697. Resources are displayed in bytes in UI for schedulers other than capacity. Contributed by Bilwa S T. 
(cherry picked from commit 34e507cb8c)
 2021-03-23 19:05:57 +00:00
Jim Brennan 99c34a3576 MAPREDUCE-7325. Intermediate data encryption is broken in LocalJobRunner. Contributed by Ahmed Hussein 
(cherry picked from commit ede490d131)
 2021-03-22 20:31:56 +00:00
Jim Brennan b3532a7c22 MAPREDUCE-7322. revisiting TestMRIntermediateDataEncryption. Contributed by Ahmed Hussein. 2021-03-22 20:30:19 +00:00
kwangsun e6b206b33d
HADOOP-17952. Fix the wrong CIDR range example in Proxy User documentation. (#2780) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c8d327a4f1)
 2021-03-22 11:47:03 +09:00
touchida 7183e269ac
HDFS-13975. TestBalancer#testMaxIterationTime fails sporadically (#2726) 
Co-authored-by: toshihiko.uchida <toshihiko.uchida@linecorp.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 25390babc7)
 2021-03-17 16:34:52 +09:00
Viraj Jasani 3d0a0ed92b HDFS-15895 : Remove redundant String#format in DFSAdmin#printOpenFiles (#2772) 
(cherry picked from commit 9c43b60348)
 2021-03-17 15:16:18 +09:00
Wangda Tan c6033eeeef YARN-1151. Ability to configure auxiliary services from HDFS-based JAR files. (Xuan Gong via wangda) 
Change-Id: Ied37ff11e507fc86847753ba79486652c8fadfe9
(cherry picked from commit c962371430)
 2021-03-16 11:03:56 -07:00
Eric Payne d53ca0b887 YARN-10588. Percentage of queue and cluster is zero in WebUI . Contributed by Bilwa S T 
(cherry picked from commit aa4c17b9d7)
 2021-03-15 20:12:58 +00:00
Ayush Saxena f0bde198f0 HADOOP-17586. Upgrade org.codehaus.woodstox:stax2-api to 4.2.1. (#2769). Contributed by Ayush Saxena. 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 Conflicts:
	LICENSE-binary
 2021-03-13 15:25:04 +05:30
Viraj Jasani 3bb40d2a56
HADOOP-17571 : Bump up woodstox-core to 5.3.0 due to security concerns (#2757) 
Contributed by Viraj Jasani.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-11 12:48:14 -08:00
Akira Ajisaka 8bd58c7391
HADOOP-17572. [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package (#2755) 
Reviewed-by: Steve Loughran <stevel@apache.org>
 2021-03-11 21:56:19 +09:00
Akira Ajisaka 82b3cafc1c
HADOOP-17557. skip-dir option is not processed by Yetus. Contributed by Ahmed Hussein. 
(cherry picked from commit 63aa142b77)
 2021-03-09 11:11:48 +09:00
tomscut 68cccba893 HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time. (#2668) Contributed by tomscut. 
(cherry picked from commit 9cb51bf106)
 2021-03-06 14:22:51 -08:00
Akira Ajisaka 9c3c753525
HADOOP-17546. Update Description of hadoop-http-auth-signature-secret in HttpAuthentication.md. Contributed by Ravuri Sushma sree. 
(cherry picked from commit 9fd2198daa)
 2021-03-04 14:58:01 +09:00
Eric Badger 0b3b68cfde [MAPREDUCE-7234] ClientHSSecurityInfo class is in wrong META-INF file. 
Contributed by Eric Badger.

(cherry picked from commit a17dc4e821)
 2021-03-02 21:50:41 +00:00
Arpit Agarwal 9f713bb825 HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq and Stephen O'Donnell. 
(cherry picked from commit 102c8fca10f3c626ab8bc47f818c8391a5c35289)
(cherry picked from commit 4a1d51dea2)
 2021-03-02 20:31:33 +00:00
Konstantin V Shvachko cdd3982db4 HDFS-15849. ExpiredHeartbeats metric should be of Type.COUNTER. Contributed by Qi Zhu. 
(cherry picked from commit 9501c698f4)
 2021-03-01 17:30:40 -08:00
Jim Brennan f31c122ec9 MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722). Contributed by Ahmed Hussein 
(cherry picked from commit 4d323cd47c)
 2021-02-26 21:51:47 +00:00
Jonathan Hung 1d76a8e73f YARN-10651. CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource(). Contributed by Haibo Chen 
(cherry picked from commit f348ab3f2f468751af329a1ffce4917cb000fcbf)
(cherry picked from commit be6e99963d)
(cherry picked from commit 6863a5bb8a)
(cherry picked from commit eb6c08e423)
 2021-02-25 15:47:36 -08:00
Jim Brennan 4ed7b80b19 [YARN-10613] Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF. Contributed by Eric Payne 2021-02-25 20:30:42 +00:00
S O'Donnell 628ccf81ef HDFS-15422. Reported IBR is partially replaced with stored info when queuing. Contributed by Stephen O'Donnell and Kihwal Lee 2021-02-24 14:47:36 +00:00
Renukaprasad C 7c83923192 HADOOP-17510. Hadoop prints sensitive Cookie information. (#2673) 
(cherry picked from commit 924fa87010)
(cherry picked from commit 9951ea9fa8)
 2021-02-24 17:34:45 +08:00
Xieming Li ee40e98d10
HADOOP-16052. Remove Subversion and Forrest from Dockerfile 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit bdbca0e53b)
 2021-02-22 23:37:19 +09:00
Akira Ajisaka f7ed17e636
MAPREDUCE-7323. Remove job_history_summary.py. (#2712) 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 8bca105655)
 2021-02-22 23:17:12 +09:00
Kengo Seki 5f969ced67
HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 
(cherry picked from commit fd7de2b82a)
 2021-02-20 19:16:30 +09:00