a6c718fd0f
HDFS-15218. RBF: MountTableRefresherService failed to refresh other router MountTableEntries in secure mode. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit 8e6227441a
2020-04-18 20:21:01 +05:30
de5d43300a
HADOOP-16972. Ignore AuthenticationFilterInitializer for KMSWebServer. ( #1961 )
...
(cherry picked from commit ac40daece1
2020-04-18 06:40:18 +09:00
49ae9b2137
YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T
2020-04-17 10:34:59 -07:00
96d7ceb39a
HADOOP-13873. log DNS addresses on s3a initialization.
...
Contributed by Mukund Thakur.
If you set the log org.apache.hadoop.fs.s3a.impl.NetworkBinding
to DEBUG, then when the S3A bucket probe is made -the DNS address
of the S3 endpoint is calculated and printed.
This is useful to see if a large set of processes are all using
the same IP address from the pool of load balancers to which AWS
directs clients when an AWS S3 endpoint is resolved.
This can have implications for performance: if all clients
access the same load balancer performance may be suboptimal.
Note: if bucket probes are disabled, fs.s3a.bucket.probe = 0,
the DNS logging does not take place.
Change-Id: I21b3ac429dc0b543f03e357fdeb94c2d2a328dd8
2020-04-17 14:20:54 +01:00
aaad947c74
HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3481895f8a
2020-04-16 20:19:00 -07:00
b4ba9bed7c
HADOOP-16985. Handle release package related issues ( #1957 )
...
(cherry picked from commit 4d24d99e85
2020-04-15 23:09:41 +05:30
94da630cd2
HADOOP-16465 listLocatedStatus() optimisation ( #1943 )
...
Contributed by Mukund Thakur
Optimize S3AFileSystem.listLocatedStatus() to perform list
operations directly and then fallback to head checks for files
Change-Id: Ia2c0fa6fcc5967c49b914b92f41135d07dab0464
2020-04-15 17:04:55 +01:00
4bd37f2283
YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja
2020-04-15 07:54:53 +02:00
f473473355
YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko
...
(cherry picked from commit 55fcbcb5c2
2020-04-15 07:27:47 +02:00
7aad965198
HADOOP-16982. Update Netty to 4.1.48.Final. Contributed by Lisheng Sun.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit ae474e1ae5
2020-04-15 07:22:54 +09:00
6bae8c46a8
HADOOP-16855. Changing wildfly dependency scope in hadoop-azure to compile
...
Contributed by Biliharith
2020-04-14 19:18:15 +01:00
68a9562848
HADOOP-16941. ITestS3GuardOutOfBandOperations.testListingDelete failing on versioned bucket ( #1919 )
...
Contributed by Steve Loughran.
Removed the failing probe and replacing with two probes which will fail
on both versioned and unversioned buckets.
2020-04-14 10:58:13 +01:00
b397a3a875
MAPREDUCE-7272. TaskAttemptListenerImpl excessive log messages. Contributed by Ahmed Hussein (ahussein)
...
(cherry picked from commit 11d17417ce
2020-04-13 18:51:00 +00:00
e1dd78143b
YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T
...
(cherry picked from commit 57659422abbf6d9bf52e6e27fca775254bb77a56)
2020-04-09 11:06:49 -07:00
e4331a73c9
HADOOP-16932. distcp copy calls getFileStatus() needlessly and can fail against S3 ( #1936 )
...
Contributed by Steve Loughran.
This strips out all the -p preservation options which have already been
processed when uploading a file before deciding whether or not to query
the far end for the status of the (existing/uploaded) file to see if any
other attributes need changing.
This will avoid 404 caching-related issues in S3, wherein a newly created
file can have a 404 entry in the S3 load balancer's cache from the
probes for the file's existence prior to the upload.
It partially addresses a regression caused by HADOOP-8143,
"Change distcp to have -pb on by default" that causes a resurfacing
of HADOOP-13145, "In DistCp, prevent unnecessary getFileStatus call when
not preserving metadata"
Change-Id: Ibc25d19e92548e6165eb8397157ebf89446333f7
2020-04-09 18:23:47 +01:00
749a5b81da
HDFS-15269. NameNode should check the authorization API version only … ( #1945 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 061afcdf30
2020-04-09 09:45:05 -07:00
061a7a8a41
YARN-9995. Code cleanup in TestSchedConfCLI. Contributed by Bilwa S T.
...
(cherry picked from commit c04a484aed
2020-04-09 15:47:57 +02:00
94df6b513b
YARN-10226. NPE in Capacity Scheduler while using %primary_group queue mapping. Contributed by Peter Bacsko.
...
(cherry picked from commit 6fb29d0f17
2020-04-09 18:52:52 +05:30
ef01e9d40e
MAPREDUCE-7266. historyContext doesn't need to be a class attribute inside JobHistoryServer. Contributed by Siddharth Ahuja
...
(cherry picked from commit e2c87b7917
2020-04-09 12:40:22 +02:00
e8520fd132
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 ( #1933 )
...
(cherry picked from commit e53d472bb0
2020-04-09 01:49:01 +09:00
618a9208f7
YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja
...
(cherry picked from commit 2214005c0f
2020-04-08 13:02:51 +10:00
d293e120eb
YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja
...
(cherry picked from commit bffb43b00e
2020-04-07 17:04:01 +02:00
7abc6221a3
YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja
...
(cherry picked from commit 45362a9f4c
2020-04-07 16:01:47 +02:00
cb806221e9
MAPREDUCE-7269. TestNetworkedJob fails ( #1940 )
...
(cherry picked from commit ab7495dc7d
2020-04-06 17:58:25 +09:00
f537a51288
HADOOP-16949. pylint fails in the build environment ( #1926 )
...
(cherry picked from commit e6455cc864
2020-04-04 16:53:26 +09:00
0a90df76bc
HADOOP-16647. Support OpenSSL 1.1.1 LTS. Contributed by Rakesh Radhakrishnan.
...
(cherry picked from commit 8f8be6b92a
2020-04-04 08:16:46 +09:00
d501c6a43d
HDFS-15258. RBF: Mark Router FSCK unstable. ( #1934 )
...
(cherry picked from commit 1695d8d59c
2020-04-04 00:16:52 +09:00
215a62e99f
MAPREDUCE-7268. Fix TestMapreduceConfigFields ( #1935 )
...
(cherry picked from commit f8e598f278
2020-04-03 11:00:02 +09:00
a2f4dcb470
YARN-10202. Fix documentation about NodeAttributes. Contributed by Sen Zhao.
...
(cherry picked from commit c162648aff
2020-04-01 16:06:16 +09:00
f7a94ec0a4
HDFS-15239. Add button to go to the parent directory in the explorer. Contributed by hemanthboyina.
2020-03-29 08:17:02 -07:00
696a663cd0
HDFS-15193. Improving the error message for missing `dfs.namenode.rpc-address.$NAMESERVICE`. Contributed by Ctest.
2020-03-29 00:01:22 +05:30
f531a4a487
HDFS-15191. EOF when reading legacy buffer in BlockTokenIdentifier. Contributed by Steven Rand.
2020-03-27 12:55:38 -07:00
cbe71eaddf
HDFS-15238. RBF: NamenodeHeartbeatService caused memory to grow rapidly. Contributed by xuzq.
2020-03-27 13:42:06 +05:30
eaaaba12b1
HADOOP-16939 fs.s3a.authoritative.path should support multiple FS URIs ( #1914 )
...
add unit test, new ITest and then fix the issue: different schema, bucket == skip
factored out the underlying logic for unit testing; also moved
maybeAddTrailingSlash to S3AUtils (while retaining/forwarnding existing method
in S3AFS).
tested: london, sole failure is
testListingDelete[auth=true](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)
filed HADOOP-16853
Change-Id: I4b8d0024469551eda0ec70b4968cba4abed405ed
2020-03-26 12:59:11 -06:00
745a6c1e69
Revert "HADOOP-16818. ABFS: Combine append+flush calls for blockblob & appendblob"
...
This reverts commit 3612317038
2020-03-26 15:24:37 +00:00
679631b188
HDFS-13470. RBF: Add Browse the Filesystem button to the UI.
2020-03-26 08:23:18 -07:00
50f7f6dfd1
YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover.
2020-03-26 08:14:30 -07:00
3d5ade1839
YARN-10043. FairOrderingPolicy Improvements. Contributed by Manikandan R
2020-03-26 13:23:43 +01:00
9a297ff31d
YARN-10197. FS-CS converter: fix emitted ordering policy string and max-am-resource percent value. Contributed by Peter Bacsko
2020-03-26 12:35:10 +01:00
348685dcb9
YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke
2020-03-26 12:18:15 +01:00
0fa7bf47df
HDFS-15234. Add a default method body for the INodeAttributeProvider#checkPermissionWithContext API. ( #1909 )
2020-03-25 16:03:26 -07:00
cdcb77a2c5
HDFS-15075. Remove process command timing from BPServiceActor. Contributed by Xiaoqiao He.
2020-03-25 11:30:54 -07:00
6ce189c621
YARN-10200. Add number of containers to RMAppManager summary
2020-03-25 10:41:59 -07:00
a700803a18
HDFS-15154. Allow only hdfs superusers the ability to assign HDFS storage policies. Contributed by Siddharth Wagle.
...
Change-Id: I32d6dd2837945b8fc026a759aa367c55daefe348
2020-03-25 10:28:30 -07:00
e3fbdcbc14
HADOOP-16912. Emit per priority RPC queue time and processing time from DecayRpcScheduler. Contributed by Fengnan Li.
2020-03-25 10:21:20 -07:00
2d294bd575
HADOOP-16938. Make non-HA proxy providers pluggable
2020-03-25 08:06:58 -07:00
cdb2107066
YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak.
2020-03-25 16:50:19 +05:30
25a03bfece
HADOOP-16930. Add hadoop-aws documentation for ProfileCredentialsProvider
...
Contributed by Nicholas Chammas.
2020-03-25 10:39:35 +00:00
d353b30baf
HDFS-15215. The Timestamp for longest write/read lock held log is wrong
2020-03-24 14:50:15 -07:00
d9c4f1129c
HDFS-15219. DFS Client will stuck when ResponseProcessor.run throw Error ( #1902 ). Contributed by zhengchenyu.
2020-03-24 23:17:22 +05:30
Surendra Singh Lilhore
Masatake Iwasaki
Jonathan Hung
Mukund Thakur
Stephen O'Donnell
Vinayakumar B
Mukund Thakur
Szilard Nemeth
sunlisheng
bilaharith
Steve Loughran
Eric E Payne
Wei-Chiu Chuang
Sunil G
Akira Ajisaka
Wilfred Spiegelenburg
Wanqiang Ji
Inigo Goiri
Ayush Saxena
Chen Liang
RogPodge
Arpit Agarwal
Chao Sun
Nicholas Chammas
Toshihiro Suzuki
Isa Hekmatizadeh