7fd90cdcbe
YARN-10788. TestCsiClient fails ( #3989 )
...
Create unix domain socket in java.io.tmpdir instead of
test.build.dir to avoid 'File name too long' error.
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2022-02-15 01:13:13 +09:00
b68964336d
HDFS-16449. Fix hadoop web site release notes and changelog not available ( #3967 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-02-14 05:38:28 +09:00
0e74f1e467
Fix thread safety of EC decoding during concurrent preads ( #3881 )
2022-02-11 10:20:00 +08:00
ca8ba24051
HADOOP-18110. ViewFileSystem: Add Support for Localized Trash Root
...
Fixes #3956
2022-02-10 16:43:04 -08:00
fe583c4b63
HADOOP-18096. Distcp: Sync moves filtered file to home directory rather than deleting. ( #3940 ). Contributed by Ayush Saxena.
...
Reviewed-by: Steve Loughran <stevel@apache.org>
Reviewed-by: stack <stack@apache.org>
2022-02-11 01:59:40 +05:30
c777142ed3
RBF: Registry HDFS Router's rpcserver & rpcclient metrics for PrometheusSink. ( #3965 )
2022-02-10 09:17:30 -08:00
efdec92cab
HADOOP-18091. S3A auditing leaks memory through ThreadLocal references ( #3930 )
...
Adds a new map type WeakReferenceMap, which stores weak
references to values, and a WeakReferenceThreadMap subclass
to more closely resemble a thread local type, as it is a
map of threadId to value.
Construct it with a factory method and optional callback
for notification on loss and regeneration.
WeakReferenceThreadMap<WrappingAuditSpan> activeSpan =
new WeakReferenceThreadMap<>(
(k) -> getUnbondedSpan(),
this::noteSpanReferenceLost);
This is used in ActiveAuditManagerS3A for span tracking.
Relates to
* HADOOP-17511. Add an Audit plugin point for S3A
* HADOOP-18094. Disable S3A auditing by default.
Contributed by Steve Loughran.
2022-02-10 12:31:41 +00:00
390967f1f0
RBF: Add search box in Router's tab-mounttable web page ( #3968 )
2022-02-09 09:17:23 -08:00
a08e69d33e
HADOOP-18114. Documentation correction in assumed_roles.md ( #3949 )
...
Fixes typo in hadoop-aws/assumed_roles.md
Contributed by Joey Krabacher
2022-02-09 10:35:11 +00:00
bf0cefb0d8
HDFS-16406. ReadsFromLocalClient counts short-circuit reads ( #3847 )
2022-02-09 10:07:05 +08:00
3e7a7c3c4a
HDFS-16411 RBF: RouterId is NULL when disable RourterRpcServer ( #3878 )
2022-02-08 08:40:57 -08:00
b39b334873
HADOOP-18098. Basic verification for the release candidate vote ( #3944 )
2022-02-07 09:34:58 +08:00
125e3b6160
HDFS-16437 ReverseXML processor doesn't accept XML files without the … ( #3926 )
2022-02-06 13:05:57 +08:00
5e7ce26e66
HADOOP-18085. S3 SDK Upgrade causes AccessPoint ARN endpoint mistranslation ( #3902 )
...
Part of HADOOP-17198. Support S3 Access Points.
HADOOP-18068. "upgrade AWS SDK to 1.12.132" broke the access point endpoint
translation.
Correct endpoints should start with "s3-accesspoint.", after SDK upgrade they start with
"s3.accesspoint-" which messes up tests + region detection by the SDK.
Contributed by Bogdan Stolojan
2022-02-04 15:37:08 +00:00
3684c7f66a
HADOOP-18100: Change scope of inner classes in InodeTree to make them accessible outside package
...
Fixes #3950
Signed-off-by: Owen O'Malley <omalley@apache.org>
2022-02-03 16:28:04 -08:00
41c86b6464
HADOOP-18101. Bump aliyun-sdk-oss to 3.13.2 and jdom2 to 2.0.6.1 ( #3951 )
2022-02-03 15:50:38 -08:00
b5b07af605
HDFS-16435. Remove no need TODO comment for ObserverReadProxyProvider ( #3912 ). Contributed by tomscut.
...
Reviewed-by: Chao Sun <sunchao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-02-03 08:21:06 +05:30
ed44662968
HDFS-16445. Make HDFS count, mkdir, rm cross platform ( #3945 )
2022-02-02 11:12:17 +05:30
ec2fd01333
YARN-10459. containerLaunchedOnNode method not need to hold scheduler… ( #3195 )
2022-02-01 09:37:40 -08:00
87abc437c7
YARN-10822. Containers going from New to Scheduled transition for kil… ( #3632 )
2022-02-01 09:35:59 -08:00
aeae5716cc
Revert "HADOOP-18024. SocketChannel is not closed when IOException happens in Server$Listener.doAccept ( #3719 )"
...
This reverts commit 6ed01585eb
2022-02-01 14:11:25 +05:30
e8f767f2f4
YARN-11026. Make default AppPlacementAllocator configurable in AppSch… ( #3741 )
...
* YARN-11026. Make default AppPlacementAllocator configurable in AppSchedulingInfo
Co-authored-by: Minni Mittal <mimittal@microsoft.com>
2022-02-01 12:18:54 +05:30
089e06de21
HDFS-16443. Fix edge case where DatanodeAdminDefaultMonitor doubly enqueues a DatanodeDescriptor on exception ( #3942 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-31 13:04:32 +09:00
bd50b9117e
HDFS-16444. Show start time of JournalNode on Web ( #3943 )
2022-01-30 16:13:58 +08:00
39cad5f28f
HDFS-16169. Fix TestBlockTokenWithDFSStriped#testEnd2End failure ( #3850 )
...
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-28 17:04:11 +09:00
9cb535caf2
YARN-10561. Upgrade node.js to 12.22.1 and yarn to 1.22.5 in YARN application catalog webapp ( #2591 )
...
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
2022-01-28 15:51:21 +09:00
02f6bad1ff
Revert "YARN-11068. Exclude transitive log4j2 dependency coming from solr 8. ( #3936 )"
...
This reverts commit 1c01944f35
2022-01-28 00:36:25 +09:00
1c01944f35
YARN-11068. Exclude transitive log4j2 dependency coming from solr 8. ( #3936 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-28 00:04:01 +09:00
e17c96a40a
HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl. ( #3900 ). Contributed by limingxiang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-01-27 16:53:21 +08:00
6136d630a3
HDFS-16427. Add debug log for BlockManager#chooseExcessRedundancyStriped ( #3888 )
2022-01-27 13:42:44 +09:00
c2ff39006f
HDFS-16398. Reconfig block report parameters for datanode ( #3831 )
2022-01-26 17:24:09 +09:00
4faac58841
HADOOP-18089. Test coverage for Async profiler servlets ( #3913 )
...
Reviewed-by: Akira Ajisaka <akiraaj@amazon.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-01-26 11:24:16 +08:00
4b26635a34
YARN-11034. Add enhanced headroom in AllocateResponse ( #3766 )
2022-01-25 09:25:18 -08:00
0d17b629ff
HADOOP-18093. Better exception handling for testFileStatusOnMountLink() in ViewFsBaseTest.java ( #3918 ). Contributed by Xing Lin.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-01-25 19:40:18 +05:30
8c7c49d877
HDFS-16401.Remove the worthless DatasetVolumeChecker#numAsyncDatasetChecks. ( #3838 )
2022-01-25 21:51:17 +08:00
94b884ae55
HDFS-16262. Async refresh of cached locations in DFSInputStream ( #3527 )
2022-01-25 11:42:35 +00:00
43153e80cb
HDFS-16428. Source path with storagePolicy cause wrong typeConsumed while rename ( #3898 ). Contributed by lei w.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-01-25 15:26:18 +08:00
d69938994e
HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background ( #3842 )
...
Reviewed-by: Istvan Fajth <pifta@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-01-25 13:02:37 +08:00
3ed3c74a6c
YARN-11015. Decouple queue capacity with ability to run OPPORTUNISTIC container ( #3779 )
2022-01-24 08:03:36 -08:00
b795f6f9a8
HADOOP-18094. Disable S3A auditing by default.
...
See HADOOP-18091. S3A auditing leaks memory through ThreadLocal references
* Adds a new option fs.s3a.audit.enabled to controls whether or not auditing
is enabled. This is false by default.
* When false, the S3A auditing manager is NoopAuditManagerS3A,
which was formerly only used for unit tests and
during filsystem initialization.
* When true, ActiveAuditManagerS3A is used for managing auditing,
allowing auditing events to be reported.
* updates documentation and tests.
This patch does not fix the underlying leak. When auditing is enabled,
long-lived threads will retain references to the audit managers
of S3A filesystem instances which have already been closed.
Contributed by Steve Loughran.
2022-01-24 13:37:33 +00:00
795a5efcc1
HADOOP-17593. hadoop-huaweicloud and hadoop-cloud-storage to remove log4j as transitive dependency
...
Signed-off-by: Akira Ajisaka <akiraaj@amazon.com>
2022-01-24 21:04:58 +09:00
5ef335da1e
HDFS-16430. Add validation to maximum blocks in EC group when adding an EC policy ( #3899 ). Contributed by daimin.
...
Reviewed-by: tomscut <litao@bigo.sg>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-01-24 12:04:26 +05:30
15b820c83c
HDFS-16402. Improve HeartbeatManager logic to avoid incorrect stats. ( #3839 ). Contributed by tomscut.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-01-24 14:26:30 +08:00
dae33cf935
YARN-11065. Bump follow-redirects from 1.13.3 to 1.14.7 in hadoop-yarn-ui ( #3890 )
...
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects ) from 1.13.3 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases )
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.13.3...v1.14.7 )
---
updated-dependencies:
- dependency-name: follow-redirects
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-20 21:44:10 +09:00
7c97c0f969
HADOOP-18084. ABFS: Add testfilePath while verifying test contents are read correctly ( #3903 )
...
Contributed by: Anmol Asrani
2022-01-19 10:13:13 +00:00
db2c3200e6
HDFS-16423. Balancer should not get blocks on stale storages ( #3883 )
...
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-01-19 17:59:42 +09:00
e355646330
HDFS-16399. Reconfig cache report parameters for datanode ( #3841 )
2022-01-19 15:10:39 +09:00
fcb1076699
HDFS-16426. Fix nextBlockReportTime when trigger full block report force ( #3887 )
2022-01-19 13:42:33 +09:00
d8ab84275e
HADOOP-18068. upgrade AWS SDK to 1.12.132 ( #3864 )
...
With this update, the versions of key shaded dependencies are
jackson 2.12.3
httpclient 4.5.13
Contributed by Steve Loughran
2022-01-18 10:31:28 +00:00
14ba19af06
HADOOP-17409. Remove s3guard from S3A module ( #3534 )
...
Completely removes S3Guard support from the S3A codebase.
If the connector is configured to use any metastore other than
the null and local stores (i.e. DynamoDB is selected) the s3a client
will raise an exception and refuse to initialize.
This is to ensure that there is no mix of S3Guard enabled and disabled
deployments with the same configuration but different hadoop releases
-it must be turned off completely.
The "hadoop s3guard" command has been retained -but the supported
subcommands have been reduced to those which are not purely S3Guard
related: "bucket-info" and "uploads".
This is major change in terms of the number of files
changed; before cherry picking subsequent s3a patches into
older releases, this patch will probably need backporting
first.
Goodbye S3Guard, your work is done. Time to die.
Contributed by Steve Loughran.
2022-01-17 18:08:57 +00:00
Akira Ajisaka
GuoPhilipse
daimin
Xing Lin
Ayush Saxena
Neil
Steve Loughran
Joey Krabacher
secfree
YulongZ
Viraj Jasani
singer-bin
Petre Bogdan Stolojan
Abhishek Das
Aswin Shakil Balasubramanian
litao
Gautham B A
minni31
KevinWikant
Akira Ajisaka
Wei-Chiu Chuang
Li MingXiang
Xing Lin
jianghuazhu
Bryan Beaudreault
Thinker313
Andrew Chung
zhongjun2
dependabot[bot]
Anmol Asrani
qinyuren