8809b43fb9
HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit ebb878bab9
2020-05-04 10:45:18 -07:00
1d8350055e
YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit f5081a9a5d
2020-05-04 10:24:01 -05:00
df16146e7f
Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida.
2020-05-03 19:17:14 +05:30
e32e1384d9
HDFS-15320. StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter ( #1992 )
...
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-02 13:59:58 -07:00
0f27c04c23
SPNEGO TLS verification
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454
2020-05-02 21:51:12 +09:00
ccebc9d9d0
HADOOP-16952. Add .diff to gitignore. Contributed by Ayush Saxena.
2020-05-01 14:43:34 +05:30
dfa7f160a5
Preparing for 3.3.1 development
2020-04-30 13:33:42 +09:00
a34174fea3
HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun.
...
(cherry picked from commit 9ca6298a9a
2020-04-29 16:01:48 +01:00
8ffe1f313c
YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G
...
(cherry picked from commit 410c605aec
2020-04-29 15:54:32 +02:00
75433a658d
YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph
2020-04-29 09:36:37 +02:00
08dbe32122
HDFS-15301. statfs function in hdfs-fuse not working. Contributed by Aryan Gupta.( #1980 )
...
(cherry picked from commit 816042e62b
2020-04-29 11:16:56 +05:30
e45faae1d4
YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori
...
(cherry picked from commit ab36429559
2020-04-28 18:56:29 +02:00
d504574d82
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina.
...
(cherry picked from commit 5e0eda5d5f
2020-04-28 06:18:25 +09:00
62ee5970f8
YARN-9848. Revert YARN-4946. Contributed by Steven Rand.
...
(cherry picked from commit 6ef01646ba
2020-04-28 05:45:37 +09:00
741fcf2c63
MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles).
...
(cherry picked from commit e2322e1117
2020-04-27 19:34:38 +00:00
68d8802624
HDFS-1820. FTPFileSystem attempts to close the outputstream even when it is not initialised. ( #1952 )
...
Contributed by Mikhail Pryakhin.
2020-04-27 14:46:52 +01:00
c0b7b38e22
YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph
...
(cherry picked from commit f91e21ac10
2020-04-27 08:26:46 +02:00
9e79cca171
HADOOP-17007. hadoop-cos fails to build. Contributed by Yang Yu.
2020-04-26 12:55:55 +05:30
1db1992b31
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena
2020-04-25 15:44:01 -07:00
1c19107ce8
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address ( #1964 )
...
Contributed by Dhiraj Hegde.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
2020-04-25 13:06:08 -07:00
c2769384ac
HADOOP-16886. Add hadoop.http.idle_timeout.ms to core-default.xml. Contributed by Lisheng Sun.
2020-04-25 13:23:16 +05:30
d6c48f8100
HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun.
...
(cherry picked from commit 453771f170
2020-04-24 11:54:11 -07:00
7a3f190d89
YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke
2020-04-24 17:09:35 +02:00
61ca459c74
YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke
2020-04-24 11:28:52 +02:00
497c7a1680
YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. ( #1939 )
...
(cherry picked from commit 9827ff2961
2020-04-24 10:27:39 +09:00
2471ba8b5c
HADOOP-16914 Adding Output Stream Counters in ABFS ( #1899 )
...
Contributed by Mehakmeet Singh.There
2020-04-23 14:38:15 +01:00
32fb174da2
Hadoop 16857. ABFS: Stop CustomTokenProvider retry logic to depend on AbfsRestOp retry policy
...
Contributed by Sneha Vijayarajan
2020-04-23 14:37:25 +01:00
1340518cd8
YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja
2020-04-23 14:55:50 +02:00
47b330dc34
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina
2020-04-22 13:41:04 -07:00
5b92d73a74
HADOOP-17001. The suffix name of the unified compression class. Contributed by bianqi
2020-04-22 12:48:39 -07:00
f53ded6185
HADOOP-16922. ABFS: Change User-Agent header ( #1938 )
...
Contributed by Bilahari T H.
2020-04-22 14:32:30 +01:00
98fdbb820e
HADOOP-16965. Refactor abfs stream configuration. ( #1956 )
...
Contributed by Mukund Thakur.
2020-04-22 14:32:01 +01:00
f74a571fdf
HADOOP-16910 : ABFS Streams to update FileSystem.Statistics counters on IO. ( #1918 ). Contributed by Mehakmeet Singh.
2020-04-22 14:30:25 +01:00
54a64e542e
HADOOP-16944. Use Yetus 0.12.0 in GitHub PR ( #1917 )
...
(cherry picked from commit 5576915236
2020-04-21 21:12:02 +09:00
5459dd64fd
HDFS-14742. RBF: TestRouterFaultTolerant tests are flaky
...
(cherry picked from commit e069a06137
2020-04-21 10:27:27 +09:00
0982f56f3a
HADOOP-16953. tuning s3guard disabled warnings ( #1962 )
...
Contributed by Steve Loughran.
The S3Guard absence warning of HADOOP-16484 has been changed
so that by default the S3A connector only logs at debug
when the connection to the S3 Store does not have S3Guard
enabled.
The option to control this log level is now
fs.s3a.s3guard.disabled.warn.level
and can be one of: silent, inform, warn, fail.
On a failure, an ExitException is raised with exit code 49.
For details on this safety feature, consult the s3guard documentation.
Change-Id: If868671c9260977c2b03b3e475b9c9531c98ce79
2020-04-20 15:07:00 +01:00
de9a6b4588
HADOOP-16986. S3A to not need wildfly on the classpath. ( #1948 )
...
Contributed by Steve Loughran.
This is a successor to HADOOP-16346, which enabled the S3A connector
to load the native openssl SSL libraries for better HTTPS performance.
That patch required wildfly.jar to be on the classpath. This
update:
* Makes wildfly.jar optional except in the special case that
"fs.s3a.ssl.channel.mode" is set to "openssl"
* Retains the declaration of wildfly.jar as a compile-time
dependency in the hadoop-aws POM. This means that unless
explicitly excluded, applications importing that published
maven artifact will, transitively, add the specified
wildfly JAR into their classpath for compilation/testing/
distribution.
This is done for packaging and to offer that optional
speedup. It is not mandatory: applications importing
the hadoop-aws POM can exclude it if they choose.
Change-Id: I7ed3e5948d1e10ce21276b3508871709347e113d
2020-04-20 14:42:36 +01:00
9c81b17153
HADOOP-16959. Resolve hadoop-cos dependency conflict. Contributed by Yang Yu.
...
(cherry picked from commit 82ff7bc9ab
2020-04-20 18:57:49 +08:00
a6c718fd0f
HDFS-15218. RBF: MountTableRefresherService failed to refresh other router MountTableEntries in secure mode. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit 8e6227441a
2020-04-18 20:21:01 +05:30
de5d43300a
HADOOP-16972. Ignore AuthenticationFilterInitializer for KMSWebServer. ( #1961 )
...
(cherry picked from commit ac40daece1
2020-04-18 06:40:18 +09:00
49ae9b2137
YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T
2020-04-17 10:34:59 -07:00
96d7ceb39a
HADOOP-13873. log DNS addresses on s3a initialization.
...
Contributed by Mukund Thakur.
If you set the log org.apache.hadoop.fs.s3a.impl.NetworkBinding
to DEBUG, then when the S3A bucket probe is made -the DNS address
of the S3 endpoint is calculated and printed.
This is useful to see if a large set of processes are all using
the same IP address from the pool of load balancers to which AWS
directs clients when an AWS S3 endpoint is resolved.
This can have implications for performance: if all clients
access the same load balancer performance may be suboptimal.
Note: if bucket probes are disabled, fs.s3a.bucket.probe = 0,
the DNS logging does not take place.
Change-Id: I21b3ac429dc0b543f03e357fdeb94c2d2a328dd8
2020-04-17 14:20:54 +01:00
aaad947c74
HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3481895f8a
2020-04-16 20:19:00 -07:00
b4ba9bed7c
HADOOP-16985. Handle release package related issues ( #1957 )
...
(cherry picked from commit 4d24d99e85
2020-04-15 23:09:41 +05:30
94da630cd2
HADOOP-16465 listLocatedStatus() optimisation ( #1943 )
...
Contributed by Mukund Thakur
Optimize S3AFileSystem.listLocatedStatus() to perform list
operations directly and then fallback to head checks for files
Change-Id: Ia2c0fa6fcc5967c49b914b92f41135d07dab0464
2020-04-15 17:04:55 +01:00
4bd37f2283
YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja
2020-04-15 07:54:53 +02:00
f473473355
YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko
...
(cherry picked from commit 55fcbcb5c2
2020-04-15 07:27:47 +02:00
7aad965198
HADOOP-16982. Update Netty to 4.1.48.Final. Contributed by Lisheng Sun.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit ae474e1ae5
2020-04-15 07:22:54 +09:00
6bae8c46a8
HADOOP-16855. Changing wildfly dependency scope in hadoop-azure to compile
...
Contributed by Biliharith
2020-04-14 19:18:15 +01:00
68a9562848
HADOOP-16941. ITestS3GuardOutOfBandOperations.testListingDelete failing on versioned bucket ( #1919 )
...
Contributed by Steve Loughran.
Removed the failing probe and replacing with two probes which will fail
on both versioned and unversioned buckets.
2020-04-14 10:58:13 +01:00
Konstantin V Shvachko
Ahmed Hussein
Ayush Saxena
Akira Ajisaka
Eric Yang
S O'Donnell
Szilard Nemeth
Aryan Gupta
Eric E Payne
Mike
Mingliang Liu
Dhiraj
Wei-Chiu Chuang
Mehakmeet Singh
Sneha Vijayarajan
bilaharith
Mukund Thakur
Steve Loughran
Sammi Chen
Surendra Singh Lilhore
Masatake Iwasaki
Jonathan Hung
Mukund Thakur
Stephen O'Donnell
Vinayakumar B
sunlisheng