Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,242 Commits 326 Branches 371 Tags
Commit Graph

15242 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Masatake Iwasaki
e2c7c8d96f HADOOP-17045. Configuration javadoc describes supporting environment variables, but the feature is not available. 2020-05-18 11:11:46 +09:00
Yiqun Lin
e6374f031a HDFS-15264. Backport Datanode detection to branch-2.10. Contributed by Lisheng Sun. 2020-05-17 11:59:10 +08:00
Inigo Goiri
b5eb08a0d9 HDFS-15038. TestFsck testFsckListCorruptSnapshotFiles is failing in trunk. Contributed by hemanthboyina. 2020-05-14 11:26:31 -07:00
Virajith Jalaparti
cf12c1349d Revert "HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov" 
This reverts commit e6b7112c8837d918614fe3571d1fe077cc061aa1.
 2020-05-13 08:18:43 -07:00
Chris Douglas
e6b7112c88 HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov 
(cherry picked from commit 6d3d01d7ad5bff0e057761c288a2e0c9263b9ed3)
 2020-05-13 07:44:50 -07:00
Andrew Wang
ae2bc55bfc HADOOP-8299. ViewFileSystem link slash mount point crashes with IndexOutOfBoundsException. Contributed by Manoj Govindassamy. 
(cherry picked from commit 22ff0eff4d58ac0beda7a5a3ae0e5d108da14f7f)
 2020-05-13 06:56:15 -07:00
Akira Ajisaka
14bd270ca0
Revert "SPNEGO TLS verification" 
This reverts commit 28715b584ab25dedc600cc2d5d22866865026bf7.
 2020-05-13 17:17:07 +09:00
Virajith Jalaparti
284a1e58d7 HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun. 2020-05-12 20:41:19 -07:00
Jonathan Hung
b8c88f6968 YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T 
(cherry picked from commit fff1d2c1226ec23841b04dd478e8b97f31abbeba)
(cherry picked from commit 564d3211f27c35bf3143a4bd1b3f8eeac2c6b01f)
(cherry picked from commit a7ea55e0156299ec8b80af1f3e681a3a7a31a3b4)
(cherry picked from commit b3e9aff5f7bcafea8b82f9b07719ff53d3ab2f12)
 2020-05-12 10:53:37 -07:00
Eric E Payne
d0b0fb6e04 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 2020-05-11 17:07:35 +00:00
Ahmed Hussein
0f0707fb0d YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-05-06 12:48:12 -05:00
Konstantin V Shvachko
4aac3406f6 HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko. 
(cherry picked from commit ebb878bab991c242b5089a18881aa10abf318ea0)
 2020-05-04 10:57:32 -07:00
Eric Yang
28715b584a
SPNEGO TLS verification 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454a5f6ea84f2cf7ac0082c555e2954a7)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
 2020-05-02 22:03:53 +09:00
Chen Liang
4cf7bbf043 HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li. 2020-05-01 11:40:37 -07:00
Akira Ajisaka
b22fb90350
HDFS-15302. Backport HDFS-15286 to branch-2.x. Contributed by hemanthboyina. 2020-05-01 15:11:08 +09:00
Ahmed Hussein
712da57658 YARN-10255. fix intermittent failure TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic in branch-2.10 (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-04-30 17:50:55 -05:00
Kihwal Lee
be6070081d HDFS-10499. TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently. Contributed by Yiqun Lin and Ahmed Hussein. 2020-04-30 16:29:40 -05:00
Jonathan Hung
27ad054696 YARN-8193. YARN RM hangs abruptly (stops allocating resources) when running successive applications. (Zian Chen via wangda) 2020-04-30 12:16:15 -07:00
Eric E Payne
568f185bf8 MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles). 2020-04-28 00:14:21 +00:00
Miklos Szegedi
71063434a6 YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
(cherry picked from commit 21948b048e9b91bb4ec646b7220b7f15deaf2203)
 2020-04-27 14:14:42 -07:00
Mingliang Liu
18f5997b95
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena 2020-04-25 15:50:00 -07:00
Dhiraj
23b15089ad
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-25 13:54:26 -07:00
Mingliang Liu
4ebe78496f
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina 2020-04-22 14:53:15 -07:00
Akira Ajisaka
597de7f802
HADOOP-16944. Use Yetus 0.12.0 in GitHub PR (#1917) 
(cherry picked from commit 5576915236aba172cb5ab49b43111661590058af)
 2020-04-21 21:13:27 +09:00
Masatake Iwasaki
34862e4ff8 HADOOP-16647. Support OpenSSL 1.1.1 LTS. Contributed by Rakesh Radhakrishnan. 
(cherry picked from commit 8f8be6b92a32982c2fd5be4647ebe1610a533dc3)
 2020-04-20 13:22:45 +09:00
Yufei Gu
4f47d9e52b HADOOP-15062. TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9. Contributed by Miklos Szegedi. 
(cherry picked from commit 138c1ed5660f713d24bfebc44ea1846f76c00cb9)
 2020-04-20 13:16:32 +09:00
Ravi Prakash
2096113d06 HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash. 
(cherry picked from commit 94ca52ae9ec0ae04854d726bf2ac1bc457b96a9c)
 2020-04-20 12:53:09 +09:00
Masatake Iwasaki
df7cc1edb0 HADOOP-16739. Fix native build failure of hadoop-pipes on CentOS 8. 2020-04-20 12:50:44 +09:00
Wangda Tan
34804679e3 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
(cherry picked from commit 17aa40f669f197d43387d67dc00040d14cd00948)
 2020-04-19 12:51:56 +09:00
Jonathan Hung
ebce5c74e6 YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T 
(cherry picked from commit cd6c10de442fc3a53c9ed5521ac1d944a6ac95c6)
(cherry picked from commit 2c79865b951d0fdea7f576ce31e310b4074ecedd)
 2020-04-17 10:35:39 -07:00
Stephen O'Donnell
9ec141c77f HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3481895f8a9ea9f6e217a0ba158c48da89b3faf2)
(cherry picked from commit aaad947c740c91c36139f3f0569ae78b53bca682)
(cherry picked from commit 041c93a26cd0ff705bea9d6283951702cf2cf1c2)
(cherry picked from commit 0ea8f3a19cd0b841dda6ba5c4e19887d9fc9cbb4)
 2020-04-16 20:19:38 -07:00
bilaharith
8f60971138 HADOOP-16933. Backport HADOOP-16890 and HADOOP-16825 to branch-2 
Hadoop 16890. Change in expiry calculation for MSI token provider. Contributed by Bilahari T H
(cherry picked from commit 0b931f36ec83dc72729a9e84a0d313f471061c64)

HADOOP-16825: ITestAzureBlobFileSystemCheckAccess failing. Contributed by Bilahari T H
(cherry picked from commit 5944d28130925fe1452f545e96b5e44f064bc69e)
 2020-04-16 11:31:51 -07:00
Sneha Vijayarajan
3bc2e26cd7 HADOOP-16778. ABFS: Backport HADOOP-16660 ABFS: Make RetryCount in ExponentialRetryPolicy Configurable to Branch-2. 
Contributed by Sneha Vijayarajan.
 2020-04-16 11:27:46 -07:00
Bilahari T H
17a0bf8d78 HADOOP-16734. Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to branch-2. 
Contributed by Bilahari T H.
 2020-04-16 11:25:27 -07:00
Chen Liang
70c2a2992f HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang. 2020-04-16 11:24:22 -07:00
Eric Yang
7945ed40f0 HADOOP-16361. Fixed TestSecureLogins#testValidKerberosName on branch-2. 
Contributed by Jim Brennan via eyang
 2020-04-16 12:12:21 -04:00
Eric Badger
0e05b02136 YARN-8680. YARN NM: Implement Iterable Abstraction for LocalResourceTracker 
state. Contributed by Pradeep Ambati.
 2020-04-14 18:30:56 +00:00
Eric E Payne
48a8d7691e MAPREDUCE-7272. TaskAttemptListenerImpl excessive log messages. Contributed by Ahmed Hussein (ahussein) 2020-04-13 19:47:27 +00:00
Masatake Iwasaki
00b313d6ff HDFS-15048. Fix findbug in DirectoryScanner. Contributed by Sean Chow. 2020-04-13 16:00:50 +09:00
Masatake Iwasaki
01ed6832f4 HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow. 2020-04-13 16:00:50 +09:00
Akira Ajisaka
8fae3b1cec
HADOOP-16869. Upgrade findbugs-maven-plugin to 3.0.5 to fix mvn findbugs:findbugs failure (#1855) 
(cherry picked from commit 7f35676f90f0730b8c9844cf00ee5a943f80d48d)

 Conflicts:
	hadoop-project/pom.xml
 2020-04-13 15:17:03 +09:00
Jonathan Hung
c0394c5434 YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T 
(cherry picked from commit 57659422abbf6d9bf52e6e27fca775254bb77a56)
(cherry picked from commit e3a52804b03d646f15048c078f8c5292d5cbecfa)
(cherry picked from commit 54599b177c46ed511e096909bed0c4f17bca1fe0)
(cherry picked from commit 6271a2852ea70c54589ce554e6bfad2eb703fe86)
 2020-04-09 11:07:59 -07:00
Eric E Payne
0ce55c699c YARN-8242. YARN NM: OOM error while reading back the state store on recovery. Contributed by Pradeep Ambati and Kanwaljeet Sachdev 
YARN-10227. Pull YARN-8242 back to branch-2.10. Contributed by Jim Brennan
 2020-04-09 17:18:07 +00:00
Ayush Saxena
10c297af04 HADOOP-16952. Add .diff to gitignore. Contributed by Ayush Saxena. 2020-04-09 19:28:13 +05:30
Eric Badger
a3343c4097 YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed 
Hussein.
 2020-04-09 00:52:45 +00:00
Akira Ajisaka
bfeec47759
HADOOP-16949. pylint fails in the build environment (#1926) 
(cherry picked from commit e6455cc864d932469610e86644162fa324589067)

Conflicts:
	dev-support/docker/Dockerfile_aarch64
(cherry picked from commit 11aff6c269fed38ba68dd69aaf17adba59a6fec8)
 2020-04-04 16:58:24 +09:00
Jonathan Hung
a7556f1ec2 YARN-8213. Add Capacity Scheduler performance metrics. (Weiwei Yang via wangda) 2020-03-27 16:10:39 -07:00
Jonathan Hung
1c8529f030 YARN-10200. Add number of containers to RMAppManager summary 
(cherry picked from commit 2de0572cdc1c6fdbfaab108b169b2d5b0c077e86)
(cherry picked from commit 5d3fb0ebe9d3f3395320b82a76194ba6fad01e00)
(cherry picked from commit 9c6dd8c83a29183d70cd4a69a8317a9303954cc1)
 2020-03-25 10:39:45 -07:00
Jonathan Hung
2c321c3c30
HADOOP-14918. Remove the Local Dynamo DB test option (branch-2.10) (#1864). Contributed by Jonathan Hung and Gabor Bota. 2020-03-23 16:22:26 +01:00
Eric Badger
d01e9c125b Revert "YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed" 
This reverts commit 85eac8b0dfec30c2ecc7310cc65f6023097b2c26.
 2020-03-17 22:15:38 +00:00