Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,470 Commits 326 Branches 371 Tags
Commit Graph

15470 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Takanobu Asanuma
a690e8ae59 HDFS-16068. WebHdfsFileSystem has a possible connection leak in connection with HttpFS (#3104) 
(cherry picked from commit 839fcf7682270bc6ba151875549d64d81917f425)
 2021-06-15 17:24:27 +09:00
Takanobu Asanuma
f22ff63a15 HADOOP-17756. Increase precommit job timeout from 20 hours to 24 hours. (#3090) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit ae9630f580d73e81042f3b8b2b337aba86130653)
 2021-06-14 10:23:00 +09:00
Hideyuki Furue
a07293f819
Fix container-executor 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9c7b8cf54ea88833d54fc71a9612c448dc0eb78d)
(cherry picked from commit f5568e15d0d6fdf9593ee869485aa6471b889d52)
 2021-06-06 09:14:18 +09:00
Konstantin V Shvachko
149325484f HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit 8ce30f51f999c0a80db53a2a96b5be5505d4d151)
 2021-05-27 20:19:44 -07:00
Konstantin V Shvachko
8715a18f95 HDFS-15915. Race condition with async edits logging due to updating txId outside of the namesystem log. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 1abd03d68f4f236674ce929164cc460037730abb)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java
 2021-05-26 16:00:18 -07:00
Akira Ajisaka
76eea30bdd
YARN-10770. container-executor permission is wrong in SecureContainer.md. Contributed by Siddharth Ahuja. 
(cherry picked from commit 1cbcde04f3b025e4c0c844e8bfe8d9007af2be3b)
 2021-05-25 18:30:21 +09:00
lujiefsi
13a2e751e0
YARN-10555. Missing access check before getAppAttempts (#2608) 
Co-authored-by: lujie <lujie@foxmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d92a25b790e5ad7d8e21fc3949cdd0f74d496b1b)
 2021-05-17 19:47:03 +09:00
hchaverri
7f6185b17c
HDFS-15912. Allow ProtobufRpcEngine to be extensible (#2999) 2021-05-17 00:37:07 -07:00
Hui Fei
77d9c6d0f7 HDFS-15652. Make block size from NNThroughputBenchmark configurable (#2416) 
(cherry picked from commit 8a6d5b9151cd4a922372835b18a9a031c9d3475e)
 2021-04-29 12:56:57 -07:00
Eric Badger
7b3a6e96d9 YARN-10479. Can't remove all node labels after add node label without 
nodemanager port, broken by YARN-10647. Contributed by D M Murali Krishna Reddy

(cherry picked from commit 6857a05d6ac566a60336c0a28951f09ecda39f24)
 2021-04-23 23:10:50 +00:00
Eric Badger
58b33d6570 MAPREDUCE-7302. Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail. Contributed by Peter Bacsko. Reviewed by Akira Ajisaka. 
Conflicts only in imports

(cherry picked from commit da93cd962c8d19874d09360726628cdde8595a7d)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java

(cherry picked from commit 7ad844ee9ed7eaab4fbb4d38576be10d497460cf)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestFetcher.java
 2021-04-23 20:16:11 +00:00
bibinchundatt
94c1cf2ff2 YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie. 
(cherry picked from commit 6d80b9bc3ff3ba8073e3faf64551b9109d2aa2ad)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/scheduler/ContainerScheduler.java

(cherry picked from commit 4a39165b41dc079dd2de140e610255f363982f59)
 2021-04-23 16:48:04 +00:00
Eric Badger
eb176bb2af YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko 2021-04-20 00:31:08 +00:00
Ahmed Hussein
55645d78e1
YARN-10733. TimelineService Hbase tests failing with timeouts (#2899) 2021-04-14 13:44:38 -05:00
Ahmed Hussein
1056ca3725
HADOOP-17601. Upgrade Jackson databind in branch-2.10 to 2.9.10.7. (#2835) 2021-04-12 17:42:40 -07:00
Ahmed Hussein
cb5e41ac07
HADOOP-17603. Upgrade tomcat-embed-core to 7.0.108 (#2834). Contributed by Ahmed Hussein 2021-04-12 10:11:36 -07:00
Viraj Jasani
3c1c1b40d8
HDFS-15931 : Fix non-static inner classes for better memory management (#2830). Contributed by Viraj Jasani 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-04-01 17:48:27 -07:00
Akira Ajisaka
d822bc3def
YARN-10716. Fix typo in ContainerRuntime. Contributed by xishuhai. 
(cherry picked from commit 8ad77a26c9f674e734e7f4c917ee2d51c77e527d)
 2021-03-30 17:49:01 +09:00
Eric Badger
616256b400 YARN-10501. Can't remove all node labels after add node label without 
nodemanager port. Contributed by caozhiqiang.
 2021-03-29 18:11:58 +00:00
Akira Ajisaka
f207d9e620
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin. Contributed by Ahmed Hussein. 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.
 2021-03-29 09:44:28 +09:00
Jim Brennan
7a3a749663 MAPREDUCE-7332. Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10. Contributed by Ahmed Hussein. 2021-03-26 15:39:59 +00:00
Akira Ajisaka
4829e1b7ed
HADOOP-17602. Upgrade JUnit to 4.13.1. Contributed by Ahmed Hussein. 2021-03-25 20:30:34 +09:00
Jim Brennan
33c4d4570d YARN-10697. Resources are displayed in bytes in UI for schedulers other than capacity. Contributed by Bilwa S T. 
(cherry picked from commit 34e507cb8c11d3b6ee561fd4aabde6dadadcee00)
 2021-03-23 19:05:57 +00:00
Jim Brennan
99c34a3576 MAPREDUCE-7325. Intermediate data encryption is broken in LocalJobRunner. Contributed by Ahmed Hussein 
(cherry picked from commit ede490d13182f0fd3831afe6db97e01917add5b4)
 2021-03-22 20:31:56 +00:00
Jim Brennan
b3532a7c22 MAPREDUCE-7322. revisiting TestMRIntermediateDataEncryption. Contributed by Ahmed Hussein. 2021-03-22 20:30:19 +00:00
kwangsun
e6b206b33d
HADOOP-17952. Fix the wrong CIDR range example in Proxy User documentation. (#2780) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c8d327a4f1a7f15d6be35051414199d1d3fdc5ef)
 2021-03-22 11:47:03 +09:00
touchida
7183e269ac
HDFS-13975. TestBalancer#testMaxIterationTime fails sporadically (#2726) 
Co-authored-by: toshihiko.uchida <toshihiko.uchida@linecorp.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 25390babc74ab8a6bbc53305b7a49df2437406f2)
 2021-03-17 16:34:52 +09:00
Viraj Jasani
3d0a0ed92b HDFS-15895 : Remove redundant String#format in DFSAdmin#printOpenFiles (#2772) 
(cherry picked from commit 9c43b60348bea7f89f7c0fc86f2813fdf2d1929f)
 2021-03-17 15:16:18 +09:00
Wangda Tan
c6033eeeef YARN-1151. Ability to configure auxiliary services from HDFS-based JAR files. (Xuan Gong via wangda) 
Change-Id: Ied37ff11e507fc86847753ba79486652c8fadfe9
(cherry picked from commit c962371430b8957500397fe71e944c953f46ed7d)
 2021-03-16 11:03:56 -07:00
Eric Payne
d53ca0b887 YARN-10588. Percentage of queue and cluster is zero in WebUI . Contributed by Bilwa S T 
(cherry picked from commit aa4c17b9d7af122163789a731ced05f740562e45)
 2021-03-15 20:12:58 +00:00
Ayush Saxena
f0bde198f0 HADOOP-17586. Upgrade org.codehaus.woodstox:stax2-api to 4.2.1. (#2769). Contributed by Ayush Saxena. 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 Conflicts:
	LICENSE-binary
 2021-03-13 15:25:04 +05:30
Viraj Jasani
3bb40d2a56
HADOOP-17571 : Bump up woodstox-core to 5.3.0 due to security concerns (#2757) 
Contributed by Viraj Jasani.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-11 12:48:14 -08:00
Akira Ajisaka
8bd58c7391
HADOOP-17572. [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package (#2755) 
Reviewed-by: Steve Loughran <stevel@apache.org>
 2021-03-11 21:56:19 +09:00
Akira Ajisaka
82b3cafc1c
HADOOP-17557. skip-dir option is not processed by Yetus. Contributed by Ahmed Hussein. 
(cherry picked from commit 63aa142b770eda31e83ca6c6e5e96cb802b8b6a9)
 2021-03-09 11:11:48 +09:00
tomscut
68cccba893 HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time. (#2668) Contributed by tomscut. 
(cherry picked from commit 9cb51bf106802c78b1400fba9f1d1c7e772dd5e7)
 2021-03-06 14:22:51 -08:00
Akira Ajisaka
9c3c753525
HADOOP-17546. Update Description of hadoop-http-auth-signature-secret in HttpAuthentication.md. Contributed by Ravuri Sushma sree. 
(cherry picked from commit 9fd2198daa58610c04bdc1ac7570476accdc25b7)
 2021-03-04 14:58:01 +09:00
Eric Badger
0b3b68cfde [MAPREDUCE-7234] ClientHSSecurityInfo class is in wrong META-INF file. 
Contributed by Eric Badger.

(cherry picked from commit a17dc4e8218b77e7fb5b5e253cb2d20014073b49)
 2021-03-02 21:50:41 +00:00
Arpit Agarwal
9f713bb825 HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq and Stephen O'Donnell. 
(cherry picked from commit 102c8fca10f3c626ab8bc47f818c8391a5c35289)
(cherry picked from commit 4a1d51dea2149e8f458467467798e81b126b7cc5)
 2021-03-02 20:31:33 +00:00
Konstantin V Shvachko
cdd3982db4 HDFS-15849. ExpiredHeartbeats metric should be of Type.COUNTER. Contributed by Qi Zhu. 
(cherry picked from commit 9501c698f4789188f744f3a1fba7f1a9bb9b8aa4)
 2021-03-01 17:30:40 -08:00
Jim Brennan
f31c122ec9 MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722). Contributed by Ahmed Hussein 
(cherry picked from commit 4d323cd47c0787c1bb7592ed56acd4aced3e11e7)
 2021-02-26 21:51:47 +00:00
Jonathan Hung
1d76a8e73f YARN-10651. CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource(). Contributed by Haibo Chen 
(cherry picked from commit f348ab3f2f468751af329a1ffce4917cb000fcbf)
(cherry picked from commit be6e99963ded94adf6f447ff53f2ba66b99120ca)
(cherry picked from commit 6863a5bb8ace591de3374102920bba916dbebfda)
(cherry picked from commit eb6c08e423dd06bf37ff44665ffb98c97e26ad08)
 2021-02-25 15:47:36 -08:00
Jim Brennan
4ed7b80b19 [YARN-10613] Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF. Contributed by Eric Payne 2021-02-25 20:30:42 +00:00
S O'Donnell
628ccf81ef HDFS-15422. Reported IBR is partially replaced with stored info when queuing. Contributed by Stephen O'Donnell and Kihwal Lee 2021-02-24 14:47:36 +00:00
Renukaprasad C
7c83923192 HADOOP-17510. Hadoop prints sensitive Cookie information. (#2673) 
(cherry picked from commit 924fa87010ac193a8242d59b14f9fa9d17957fe5)
(cherry picked from commit 9951ea9fa8427c134861f047acc164b5cee4eef0)
 2021-02-24 17:34:45 +08:00
Xieming Li
ee40e98d10
HADOOP-16052. Remove Subversion and Forrest from Dockerfile 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit bdbca0e53b4e0e728c71bc51f99e035d9081c7b1)
 2021-02-22 23:37:19 +09:00
Akira Ajisaka
f7ed17e636
MAPREDUCE-7323. Remove job_history_summary.py. (#2712) 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 8bca105655d91d949f8aa8daae656f34ed950e78)
 2021-02-22 23:17:12 +09:00
Kengo Seki
5f969ced67
HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 
(cherry picked from commit fd7de2b82a6b9d600b43909a0770a6b0afea899a)
 2021-02-20 19:16:30 +09:00
Jim Brennan
7363e6eee1 [YARN-10626] Log resource allocation in NM log at container start time. Contributed by Eric Badger 
(cherry picked from commit e6f5dbbe7f556e3525293814ad2239e4f04d88ad)
 2021-02-16 17:30:08 +00:00
Jim Brennan
d0562d6cd0 YARN-10500. TestDelegationTokenRenewer fails intermittently. (#2619) Contributed by Masatake Iwasaki 2021-02-11 22:45:08 +00:00
Akira Ajisaka
a5a6c973af
HADOOP-17516. Upgrade ant to 1.10.9. (#2683) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 933fbeffee1e9e2b9d78b5be31466d4d640768c5)
 2021-02-11 08:45:42 +09:00