Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,359 Commits 326 Branches 371 Tags
Commit Graph

15359 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Eric Yang
e81002bcb1
SPNEGO TLS verification 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 81d8a887b0406380e469c76ed2e41022a6372dd7)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
 2020-06-15 11:15:03 +09:00
Kihwal Lee
17109758dd HDFS-12453. TestDataNodeHotSwapVolumes fails in trunk Jenkins runs. Contributed by Jim Brennan and Lei Xu. 2020-06-12 16:19:36 -05:00
Eric Badger
a11c606910 YARN-10312. Add support for yarn logs -logFile to retain backward compatibility. 
Contributed by Jim Brennan

(cherry picked from commit 73639319422b8443df943cf818b122c1ee2afcf6)
 2020-06-12 19:21:45 +00:00
Tao Yang
a91d4d612f YARN-8011. TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails intermittently. Contributed by Jim Brennan. 2020-06-12 10:56:24 +08:00
Mingliang Liu
8e01bd317b
HADOOP-17059. ArrayIndexOfboundsException in ViewFileSystem#listStatus. Contributed by hemanthboyina 2020-06-10 09:03:57 -07:00
Eric E Payne
af324e3153 YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger). 
(cherry picked from commit 56247db3022705635580c4d2f8b0abde109f954f)
(cherry picked from commit 034d458511692341636f0d2ef0574b7516c01ed6)
(cherry picked from commit 2e4892061a2ff1ae99ef5aacffd7b229dc3dac1b)
 2020-06-09 22:16:16 +00:00
Toshihiro Suzuki
d1f4c8f10f
HDFS-15386. ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories (#2054) 
Contributed by Toshihiro Suzuki.
 2020-06-09 21:44:27 +01:00
Kihwal Lee
14ff6171a5 HADOOP-16255. Add ChecksumFs.rename(path, path, boolean). Contributed by Jungtaek Lim. 2020-06-08 17:31:16 -05:00
Eric Badger
6eb7c28293 Revert "MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles)." 
This reverts commit 568f185bf8c27887d3f4c2559b301275d4bfa486.
 2020-06-08 20:03:02 +00:00
Mingliang Liu
dcc6d63828
HADOOP-17047. TODO comment exist in trunk while related issue HADOOP-6223 is already fixed. Contributed by Rungroj Maipradit 2020-06-08 11:34:01 -07:00
Inigo Goiri
b411fcee86 HDFS-14054. TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky. Contributed by Zsolt Venczel. 
(cherry picked from commit 993c2140ccfb4c6dfd09c0c686b85a4b58cb39a2)
 2020-06-03 14:16:38 -07:00
Akira Ajisaka
75516620c8
HADOOP-17062. Fix shelldocs path in Jenkinsfile (#2049) 
(cherry picked from commit 704409d53bf7ebf717a3c2e988ede80f623bbad3)
 2020-06-04 06:08:28 +09:00
Kihwal Lee
54c7d23e58 HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger 
(cherry picked from commit 9ef6ed9c1c83b9752e772ece7a716a33045752bf)
 2020-06-02 09:52:06 -05:00
Dhiraj
e49573631c
HADOOP-17052. NetUtils.connect() throws unchecked exception (UnresolvedAddressException) causing clients to abort (#2036) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-06-01 11:08:32 -07:00
Jonathan Hung
b9a0f99966 YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R 2020-06-01 10:48:41 -07:00
cnauroth
73ae3cee85 HDFS-9376. TestSeveralNameNodes fails occasionally. Contributed by Masatake Iwasaki. 
(cherry picked from commit 84a81477912644290173518d566b586305b85bf7)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2020-05-20 08:22:47 +09:00
Chen Liang
d0504cf74d HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang 2020-05-18 12:10:05 -07:00
Masatake Iwasaki
e2c7c8d96f HADOOP-17045. Configuration javadoc describes supporting environment variables, but the feature is not available. 2020-05-18 11:11:46 +09:00
Yiqun Lin
e6374f031a HDFS-15264. Backport Datanode detection to branch-2.10. Contributed by Lisheng Sun. 2020-05-17 11:59:10 +08:00
Inigo Goiri
b5eb08a0d9 HDFS-15038. TestFsck testFsckListCorruptSnapshotFiles is failing in trunk. Contributed by hemanthboyina. 2020-05-14 11:26:31 -07:00
Virajith Jalaparti
cf12c1349d Revert "HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov" 
This reverts commit e6b7112c8837d918614fe3571d1fe077cc061aa1.
 2020-05-13 08:18:43 -07:00
Chris Douglas
e6b7112c88 HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov 
(cherry picked from commit 6d3d01d7ad5bff0e057761c288a2e0c9263b9ed3)
 2020-05-13 07:44:50 -07:00
Andrew Wang
ae2bc55bfc HADOOP-8299. ViewFileSystem link slash mount point crashes with IndexOutOfBoundsException. Contributed by Manoj Govindassamy. 
(cherry picked from commit 22ff0eff4d58ac0beda7a5a3ae0e5d108da14f7f)
 2020-05-13 06:56:15 -07:00
Akira Ajisaka
14bd270ca0
Revert "SPNEGO TLS verification" 
This reverts commit 28715b584ab25dedc600cc2d5d22866865026bf7.
 2020-05-13 17:17:07 +09:00
Virajith Jalaparti
284a1e58d7 HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun. 2020-05-12 20:41:19 -07:00
Jonathan Hung
b8c88f6968 YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T 
(cherry picked from commit fff1d2c1226ec23841b04dd478e8b97f31abbeba)
(cherry picked from commit 564d3211f27c35bf3143a4bd1b3f8eeac2c6b01f)
(cherry picked from commit a7ea55e0156299ec8b80af1f3e681a3a7a31a3b4)
(cherry picked from commit b3e9aff5f7bcafea8b82f9b07719ff53d3ab2f12)
 2020-05-12 10:53:37 -07:00
Eric E Payne
d0b0fb6e04 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 2020-05-11 17:07:35 +00:00
Ahmed Hussein
0f0707fb0d YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-05-06 12:48:12 -05:00
Konstantin V Shvachko
4aac3406f6 HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko. 
(cherry picked from commit ebb878bab991c242b5089a18881aa10abf318ea0)
 2020-05-04 10:57:32 -07:00
Eric Yang
28715b584a
SPNEGO TLS verification 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454a5f6ea84f2cf7ac0082c555e2954a7)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
 2020-05-02 22:03:53 +09:00
Chen Liang
4cf7bbf043 HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li. 2020-05-01 11:40:37 -07:00
Akira Ajisaka
b22fb90350
HDFS-15302. Backport HDFS-15286 to branch-2.x. Contributed by hemanthboyina. 2020-05-01 15:11:08 +09:00
Ahmed Hussein
712da57658 YARN-10255. fix intermittent failure TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic in branch-2.10 (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-04-30 17:50:55 -05:00
Kihwal Lee
be6070081d HDFS-10499. TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently. Contributed by Yiqun Lin and Ahmed Hussein. 2020-04-30 16:29:40 -05:00
Jonathan Hung
27ad054696 YARN-8193. YARN RM hangs abruptly (stops allocating resources) when running successive applications. (Zian Chen via wangda) 2020-04-30 12:16:15 -07:00
Eric E Payne
568f185bf8 MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles). 2020-04-28 00:14:21 +00:00
Miklos Szegedi
71063434a6 YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
(cherry picked from commit 21948b048e9b91bb4ec646b7220b7f15deaf2203)
 2020-04-27 14:14:42 -07:00
Mingliang Liu
18f5997b95
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena 2020-04-25 15:50:00 -07:00
Dhiraj
23b15089ad
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-25 13:54:26 -07:00
Mingliang Liu
4ebe78496f
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina 2020-04-22 14:53:15 -07:00
Akira Ajisaka
597de7f802
HADOOP-16944. Use Yetus 0.12.0 in GitHub PR (#1917) 
(cherry picked from commit 5576915236aba172cb5ab49b43111661590058af)
 2020-04-21 21:13:27 +09:00
Masatake Iwasaki
34862e4ff8 HADOOP-16647. Support OpenSSL 1.1.1 LTS. Contributed by Rakesh Radhakrishnan. 
(cherry picked from commit 8f8be6b92a32982c2fd5be4647ebe1610a533dc3)
 2020-04-20 13:22:45 +09:00
Yufei Gu
4f47d9e52b HADOOP-15062. TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9. Contributed by Miklos Szegedi. 
(cherry picked from commit 138c1ed5660f713d24bfebc44ea1846f76c00cb9)
 2020-04-20 13:16:32 +09:00
Ravi Prakash
2096113d06 HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash. 
(cherry picked from commit 94ca52ae9ec0ae04854d726bf2ac1bc457b96a9c)
 2020-04-20 12:53:09 +09:00
Masatake Iwasaki
df7cc1edb0 HADOOP-16739. Fix native build failure of hadoop-pipes on CentOS 8. 2020-04-20 12:50:44 +09:00
Wangda Tan
34804679e3 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
(cherry picked from commit 17aa40f669f197d43387d67dc00040d14cd00948)
 2020-04-19 12:51:56 +09:00
Jonathan Hung
ebce5c74e6 YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T 
(cherry picked from commit cd6c10de442fc3a53c9ed5521ac1d944a6ac95c6)
(cherry picked from commit 2c79865b951d0fdea7f576ce31e310b4074ecedd)
 2020-04-17 10:35:39 -07:00
Stephen O'Donnell
9ec141c77f HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3481895f8a9ea9f6e217a0ba158c48da89b3faf2)
(cherry picked from commit aaad947c740c91c36139f3f0569ae78b53bca682)
(cherry picked from commit 041c93a26cd0ff705bea9d6283951702cf2cf1c2)
(cherry picked from commit 0ea8f3a19cd0b841dda6ba5c4e19887d9fc9cbb4)
 2020-04-16 20:19:38 -07:00
bilaharith
8f60971138 HADOOP-16933. Backport HADOOP-16890 and HADOOP-16825 to branch-2 
Hadoop 16890. Change in expiry calculation for MSI token provider. Contributed by Bilahari T H
(cherry picked from commit 0b931f36ec83dc72729a9e84a0d313f471061c64)

HADOOP-16825: ITestAzureBlobFileSystemCheckAccess failing. Contributed by Bilahari T H
(cherry picked from commit 5944d28130925fe1452f545e96b5e44f064bc69e)
 2020-04-16 11:31:51 -07:00
Sneha Vijayarajan
3bc2e26cd7 HADOOP-16778. ABFS: Backport HADOOP-16660 ABFS: Make RetryCount in ExponentialRetryPolicy Configurable to Branch-2. 
Contributed by Sneha Vijayarajan.
 2020-04-16 11:27:46 -07:00