95e4ed18a6
YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
2020-05-12 09:03:50 +02:00
d1e5e393c3
HDFS-15270. Account for *env == NULL in hdfsThreadDestructor ( #1951 )
...
OpenJ9 JVM properly terminates the thread before hdfsThreadDestructor is
invoked. JNIEnv is a mirror of J9VMThread in OpenJ9. After proper thread
termination, accessing JNIEnv in hdfsThreadDestructor (*env)->GetJavaVM,
yields a SIGSEGV since *env is NULL after thread cleanup is performed.
The main purpose of hdfsThreadDestructor is to invoke
DetachCurrentThread, which performs thread cleanup in OpenJ9. Since
OpenJ9 performs thread cleanup before hdfsThreadDestructor is invoked,
hdfsThreadDestructor should account for *env == NULL and skip
DetachCurrentThread.
Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>
(cherry picked from commit 1996351b0b
2020-05-11 10:45:57 -07:00
61a4cd5539
HDFS-15338. listOpenFiles() should throw InvalidPathException in case of invalid paths. Contributed by Jinglun.
2020-05-11 16:52:23 +05:30
282427f6d1
HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data. ( #2003 )
...
(cherry picked from commit 328eae9a14
2020-05-11 14:46:43 +09:00
6ccb1cd5f1
HDFS-15250. Setting `dfs.client.use.datanode.hostname` to true can crash the system because of unhandled UnresolvedAddressException. Contributed by Ctest.
2020-05-10 11:44:02 +05:30
763a79916d
HDFS-15343. TestConfiguredFailoverProxyProvider is failing. ( #2001 )
...
(cherry picked from commit c784ba370e
2020-05-08 17:19:48 +09:00
817814105a
HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole.
2020-05-08 00:57:06 +05:30
6a0aeea60f
HADOOP-17025. Fix invalid metastore configuration in S3GuardTool tests. ( #1994 )
...
(cherry picked from commit 99840aaba6
2020-05-07 12:02:15 +09:00
edf52d29f1
HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 9c8236d04d
2020-05-06 15:13:33 -07:00
7aade932c7
YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 92e3ebb401
2020-05-06 12:28:04 -05:00
4e0d99c4d6
HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun.
2020-05-06 17:02:40 +05:30
78c97907cb
HDFS-15334. INodeAttributeProvider's new API checkPermissionWithContext not getting called in for authorization. ( #1998 )
...
Reviewed-by: Arpit Agarwal <arp@apache.org>
(cherry picked from commit adecdb8b53
2020-05-05 21:25:56 -07:00
8809b43fb9
HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit ebb878bab9
2020-05-04 10:45:18 -07:00
1d8350055e
YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit f5081a9a5d
2020-05-04 10:24:01 -05:00
df16146e7f
Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida.
2020-05-03 19:17:14 +05:30
e32e1384d9
HDFS-15320. StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter ( #1992 )
...
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-02 13:59:58 -07:00
0f27c04c23
SPNEGO TLS verification
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454
2020-05-02 21:51:12 +09:00
ccebc9d9d0
HADOOP-16952. Add .diff to gitignore. Contributed by Ayush Saxena.
2020-05-01 14:43:34 +05:30
dfa7f160a5
Preparing for 3.3.1 development
2020-04-30 13:33:42 +09:00
a34174fea3
HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun.
...
(cherry picked from commit 9ca6298a9a
2020-04-29 16:01:48 +01:00
8ffe1f313c
YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G
...
(cherry picked from commit 410c605aec
2020-04-29 15:54:32 +02:00
75433a658d
YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph
2020-04-29 09:36:37 +02:00
08dbe32122
HDFS-15301. statfs function in hdfs-fuse not working. Contributed by Aryan Gupta.( #1980 )
...
(cherry picked from commit 816042e62b
2020-04-29 11:16:56 +05:30
e45faae1d4
YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori
...
(cherry picked from commit ab36429559
2020-04-28 18:56:29 +02:00
d504574d82
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina.
...
(cherry picked from commit 5e0eda5d5f
2020-04-28 06:18:25 +09:00
62ee5970f8
YARN-9848. Revert YARN-4946. Contributed by Steven Rand.
...
(cherry picked from commit 6ef01646ba
2020-04-28 05:45:37 +09:00
741fcf2c63
MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles).
...
(cherry picked from commit e2322e1117
2020-04-27 19:34:38 +00:00
68d8802624
HDFS-1820. FTPFileSystem attempts to close the outputstream even when it is not initialised. ( #1952 )
...
Contributed by Mikhail Pryakhin.
2020-04-27 14:46:52 +01:00
c0b7b38e22
YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph
...
(cherry picked from commit f91e21ac10
2020-04-27 08:26:46 +02:00
9e79cca171
HADOOP-17007. hadoop-cos fails to build. Contributed by Yang Yu.
2020-04-26 12:55:55 +05:30
1db1992b31
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena
2020-04-25 15:44:01 -07:00
1c19107ce8
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address ( #1964 )
...
Contributed by Dhiraj Hegde.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
2020-04-25 13:06:08 -07:00
c2769384ac
HADOOP-16886. Add hadoop.http.idle_timeout.ms to core-default.xml. Contributed by Lisheng Sun.
2020-04-25 13:23:16 +05:30
d6c48f8100
HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun.
...
(cherry picked from commit 453771f170
2020-04-24 11:54:11 -07:00
7a3f190d89
YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke
2020-04-24 17:09:35 +02:00
61ca459c74
YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke
2020-04-24 11:28:52 +02:00
497c7a1680
YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. ( #1939 )
...
(cherry picked from commit 9827ff2961
2020-04-24 10:27:39 +09:00
2471ba8b5c
HADOOP-16914 Adding Output Stream Counters in ABFS ( #1899 )
...
Contributed by Mehakmeet Singh.There
2020-04-23 14:38:15 +01:00
32fb174da2
Hadoop 16857. ABFS: Stop CustomTokenProvider retry logic to depend on AbfsRestOp retry policy
...
Contributed by Sneha Vijayarajan
2020-04-23 14:37:25 +01:00
1340518cd8
YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja
2020-04-23 14:55:50 +02:00
47b330dc34
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina
2020-04-22 13:41:04 -07:00
5b92d73a74
HADOOP-17001. The suffix name of the unified compression class. Contributed by bianqi
2020-04-22 12:48:39 -07:00
f53ded6185
HADOOP-16922. ABFS: Change User-Agent header ( #1938 )
...
Contributed by Bilahari T H.
2020-04-22 14:32:30 +01:00
98fdbb820e
HADOOP-16965. Refactor abfs stream configuration. ( #1956 )
...
Contributed by Mukund Thakur.
2020-04-22 14:32:01 +01:00
f74a571fdf
HADOOP-16910 : ABFS Streams to update FileSystem.Statistics counters on IO. ( #1918 ). Contributed by Mehakmeet Singh.
2020-04-22 14:30:25 +01:00
54a64e542e
HADOOP-16944. Use Yetus 0.12.0 in GitHub PR ( #1917 )
...
(cherry picked from commit 5576915236
2020-04-21 21:12:02 +09:00
5459dd64fd
HDFS-14742. RBF: TestRouterFaultTolerant tests are flaky
...
(cherry picked from commit e069a06137
2020-04-21 10:27:27 +09:00
0982f56f3a
HADOOP-16953. tuning s3guard disabled warnings ( #1962 )
...
Contributed by Steve Loughran.
The S3Guard absence warning of HADOOP-16484 has been changed
so that by default the S3A connector only logs at debug
when the connection to the S3 Store does not have S3Guard
enabled.
The option to control this log level is now
fs.s3a.s3guard.disabled.warn.level
and can be one of: silent, inform, warn, fail.
On a failure, an ExitException is raised with exit code 49.
For details on this safety feature, consult the s3guard documentation.
Change-Id: If868671c9260977c2b03b3e475b9c9531c98ce79
2020-04-20 15:07:00 +01:00
de9a6b4588
HADOOP-16986. S3A to not need wildfly on the classpath. ( #1948 )
...
Contributed by Steve Loughran.
This is a successor to HADOOP-16346, which enabled the S3A connector
to load the native openssl SSL libraries for better HTTPS performance.
That patch required wildfly.jar to be on the classpath. This
update:
* Makes wildfly.jar optional except in the special case that
"fs.s3a.ssl.channel.mode" is set to "openssl"
* Retains the declaration of wildfly.jar as a compile-time
dependency in the hadoop-aws POM. This means that unless
explicitly excluded, applications importing that published
maven artifact will, transitively, add the specified
wildfly JAR into their classpath for compilation/testing/
distribution.
This is done for packaging and to offer that optional
speedup. It is not mandatory: applications importing
the hadoop-aws POM can exclude it if they choose.
Change-Id: I7ed3e5948d1e10ce21276b3508871709347e113d
2020-04-20 14:42:36 +01:00
9c81b17153
HADOOP-16959. Resolve hadoop-cos dependency conflict. Contributed by Yang Yu.
...
(cherry picked from commit 82ff7bc9ab
2020-04-20 18:57:49 +08:00
Szilard Nemeth
Babneet Singh
Ayush Saxena
Akira Ajisaka
Masatake Iwasaki
Uma Maheswara Rao G
Ahmed Hussein
Wei-Chiu Chuang
Konstantin V Shvachko
Eric Yang
S O'Donnell
Aryan Gupta
Eric E Payne
Mike
Mingliang Liu
Dhiraj
Mehakmeet Singh
Sneha Vijayarajan
bilaharith
Mukund Thakur
Steve Loughran
Sammi Chen