718c8a5868
HDFS-15369. Refactor method VolumeScanner#runLoop(). Contributed by Yang Yun.
2020-05-24 18:06:19 +05:30
c44d8b8bdf
HDFS-15363. BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup. Contributed by hemanthboyina.
...
(cherry picked from commit 4d22d1c58f
2020-05-23 17:33:39 +09:00
9ce913a7bf
FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
...
(cherry picked from commit ac4540dd8e
2020-05-22 14:48:53 +02:00
4d30c395f7
HADOOP-17040. Fix intermittent failure of ITestBlockingThreadPoolExecutorService. ( #2020 )
...
(cherry picked from commit 9685314633
2020-05-22 18:53:53 +09:00
6b54f259e7
HADOOP-17049. javax.activation-api and jakarta.activation-api define overlapping classes ( #2027 )
...
* Removed javax.activation-api from dependency
(cherry picked from commit 52b21de1d8
2020-05-22 11:20:16 +09:00
d7cf19d7c0
YARN-10228. Relax restriction of file path character in yarn.service.am.java.opts.
...
Contributed by Bilwa S T via eyang
2020-05-20 09:31:40 -07:00
e452163a06
HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang
...
(cherry picked from commit 7bb902bc0d
2020-05-19 09:52:25 +09:00
eb045ea056
HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a3f44dacc1
2020-05-18 09:59:12 -07:00
3915d1afc7
HDFS-15356. Unify configuration `dfs.ha.allow.stale.reads` to DFSConfigKeys. Contributed by Xiaoqiao He.
...
(cherry picked from commit 178336f8a8
2020-05-18 09:58:59 -07:00
240cba7e6d
HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )
...
(cherry picked from commit 2abcf7762a
2020-05-18 09:41:11 -07:00
289f8acc64
Revert "HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )"
...
This reverts commit ad9a6a0ee3
2020-05-18 09:41:03 -07:00
53d22fdb88
Revert "HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He."
...
This reverts commit acae31aa28
2020-05-18 09:39:57 -07:00
032ccba67c
HDFS-15207. VolumeScanner skip to scan blocks accessed during recent scan peroid. Contributed by Yang Yun.
...
(cherry picked from commit 50caba1a92
2020-05-18 08:42:20 -07:00
acae31aa28
HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a3f44dacc1
2020-05-18 07:10:27 -07:00
ad9a6a0ee3
HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )
...
Added parameter dfs.client.short.circuit.num improving HDFS-client's massive reading performance by create few instances ShortCircuit caches instead of one. It helps avoid locks and lets CPU do job.
(cherry picked from commit 86e6aa8eec
2020-05-18 07:05:03 -07:00
77587ffb1e
HADOOP-17042. Hadoop distcp throws 'ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found'. Contributed by Aki Tanaka.
...
(cherry picked from commit 27601fc79e
2020-05-18 15:37:22 +09:00
f2f727359b
Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."
...
This reverts commit 44350fdf49
2020-05-14 19:20:34 +01:00
7f2cf334a8
Revert "HADOOP-8143. Change distcp to have -pb on by default."
...
This reverts commit dd65eea74b
2020-05-14 19:14:39 +01:00
7cef7109de
YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng
2020-05-14 15:17:13 +05:30
74887ab2ad
YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.
2020-05-14 00:37:09 +05:30
ee58c112d0
Revert "SPNEGO TLS verification"
...
This reverts commit 0f27c04c23
2020-05-13 17:14:51 +09:00
564d3211f2
YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T
...
(cherry picked from commit fff1d2c122
2020-05-12 10:49:10 -07:00
433aaeefa4
HDFS-15255. Consider StorageType when DatanodeManager#sortLocatedBlock(). Contributed by Lisheng Sun.
2020-05-12 15:25:05 +01:00
ef41229f70
HDFS-14367. EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads. Contributed by Guo Lei.
2020-05-12 18:43:37 +05:30
95e4ed18a6
YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
2020-05-12 09:03:50 +02:00
d1e5e393c3
HDFS-15270. Account for *env == NULL in hdfsThreadDestructor ( #1951 )
...
OpenJ9 JVM properly terminates the thread before hdfsThreadDestructor is
invoked. JNIEnv is a mirror of J9VMThread in OpenJ9. After proper thread
termination, accessing JNIEnv in hdfsThreadDestructor (*env)->GetJavaVM,
yields a SIGSEGV since *env is NULL after thread cleanup is performed.
The main purpose of hdfsThreadDestructor is to invoke
DetachCurrentThread, which performs thread cleanup in OpenJ9. Since
OpenJ9 performs thread cleanup before hdfsThreadDestructor is invoked,
hdfsThreadDestructor should account for *env == NULL and skip
DetachCurrentThread.
Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>
(cherry picked from commit 1996351b0b
2020-05-11 10:45:57 -07:00
61a4cd5539
HDFS-15338. listOpenFiles() should throw InvalidPathException in case of invalid paths. Contributed by Jinglun.
2020-05-11 16:52:23 +05:30
282427f6d1
HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data. ( #2003 )
...
(cherry picked from commit 328eae9a14
2020-05-11 14:46:43 +09:00
6ccb1cd5f1
HDFS-15250. Setting `dfs.client.use.datanode.hostname` to true can crash the system because of unhandled UnresolvedAddressException. Contributed by Ctest.
2020-05-10 11:44:02 +05:30
763a79916d
HDFS-15343. TestConfiguredFailoverProxyProvider is failing. ( #2001 )
...
(cherry picked from commit c784ba370e
2020-05-08 17:19:48 +09:00
817814105a
HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole.
2020-05-08 00:57:06 +05:30
6a0aeea60f
HADOOP-17025. Fix invalid metastore configuration in S3GuardTool tests. ( #1994 )
...
(cherry picked from commit 99840aaba6
2020-05-07 12:02:15 +09:00
edf52d29f1
HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 9c8236d04d
2020-05-06 15:13:33 -07:00
7aade932c7
YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 92e3ebb401
2020-05-06 12:28:04 -05:00
4e0d99c4d6
HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun.
2020-05-06 17:02:40 +05:30
78c97907cb
HDFS-15334. INodeAttributeProvider's new API checkPermissionWithContext not getting called in for authorization. ( #1998 )
...
Reviewed-by: Arpit Agarwal <arp@apache.org>
(cherry picked from commit adecdb8b53
2020-05-05 21:25:56 -07:00
8809b43fb9
HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit ebb878bab9
2020-05-04 10:45:18 -07:00
1d8350055e
YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit f5081a9a5d
2020-05-04 10:24:01 -05:00
df16146e7f
Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida.
2020-05-03 19:17:14 +05:30
e32e1384d9
HDFS-15320. StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter ( #1992 )
...
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-02 13:59:58 -07:00
0f27c04c23
SPNEGO TLS verification
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454
2020-05-02 21:51:12 +09:00
ccebc9d9d0
HADOOP-16952. Add .diff to gitignore. Contributed by Ayush Saxena.
2020-05-01 14:43:34 +05:30
dfa7f160a5
Preparing for 3.3.1 development
2020-04-30 13:33:42 +09:00
a34174fea3
HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun.
...
(cherry picked from commit 9ca6298a9a
2020-04-29 16:01:48 +01:00
8ffe1f313c
YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G
...
(cherry picked from commit 410c605aec
2020-04-29 15:54:32 +02:00
75433a658d
YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph
2020-04-29 09:36:37 +02:00
08dbe32122
HDFS-15301. statfs function in hdfs-fuse not working. Contributed by Aryan Gupta.( #1980 )
...
(cherry picked from commit 816042e62b
2020-04-29 11:16:56 +05:30
e45faae1d4
YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori
...
(cherry picked from commit ab36429559
2020-04-28 18:56:29 +02:00
d504574d82
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina.
...
(cherry picked from commit 5e0eda5d5f
2020-04-28 06:18:25 +09:00
62ee5970f8
YARN-9848. Revert YARN-4946. Contributed by Steven Rand.
...
(cherry picked from commit 6ef01646ba
2020-04-28 05:45:37 +09:00
Ayush Saxena
Takanobu Asanuma
Szilard Nemeth
Masatake Iwasaki
Akira Ajisaka
Eric Yang
Chen Liang
He Xiaoqiao
Wei-Chiu Chuang
pustota2009
Steve Loughran
Surendra Singh Lilhore
Jonathan Hung
S O'Donnell
Babneet Singh
Uma Maheswara Rao G
Ahmed Hussein
Konstantin V Shvachko
Aryan Gupta