b5b07af605
HDFS-16435. Remove no need TODO comment for ObserverReadProxyProvider ( #3912 ). Contributed by tomscut.
...
Reviewed-by: Chao Sun <sunchao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-02-03 08:21:06 +05:30
ed44662968
HDFS-16445. Make HDFS count, mkdir, rm cross platform ( #3945 )
2022-02-02 11:12:17 +05:30
ec2fd01333
YARN-10459. containerLaunchedOnNode method not need to hold scheduler… ( #3195 )
2022-02-01 09:37:40 -08:00
87abc437c7
YARN-10822. Containers going from New to Scheduled transition for kil… ( #3632 )
2022-02-01 09:35:59 -08:00
aeae5716cc
Revert "HADOOP-18024. SocketChannel is not closed when IOException happens in Server$Listener.doAccept ( #3719 )"
...
This reverts commit 6ed01585eb
2022-02-01 14:11:25 +05:30
e8f767f2f4
YARN-11026. Make default AppPlacementAllocator configurable in AppSch… ( #3741 )
...
* YARN-11026. Make default AppPlacementAllocator configurable in AppSchedulingInfo
Co-authored-by: Minni Mittal <mimittal@microsoft.com>
2022-02-01 12:18:54 +05:30
089e06de21
HDFS-16443. Fix edge case where DatanodeAdminDefaultMonitor doubly enqueues a DatanodeDescriptor on exception ( #3942 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-31 13:04:32 +09:00
bd50b9117e
HDFS-16444. Show start time of JournalNode on Web ( #3943 )
2022-01-30 16:13:58 +08:00
39cad5f28f
HDFS-16169. Fix TestBlockTokenWithDFSStriped#testEnd2End failure ( #3850 )
...
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-28 17:04:11 +09:00
9cb535caf2
YARN-10561. Upgrade node.js to 12.22.1 and yarn to 1.22.5 in YARN application catalog webapp ( #2591 )
...
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
2022-01-28 15:51:21 +09:00
02f6bad1ff
Revert "YARN-11068. Exclude transitive log4j2 dependency coming from solr 8. ( #3936 )"
...
This reverts commit 1c01944f35
2022-01-28 00:36:25 +09:00
1c01944f35
YARN-11068. Exclude transitive log4j2 dependency coming from solr 8. ( #3936 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-28 00:04:01 +09:00
e17c96a40a
HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl. ( #3900 ). Contributed by limingxiang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-01-27 16:53:21 +08:00
6136d630a3
HDFS-16427. Add debug log for BlockManager#chooseExcessRedundancyStriped ( #3888 )
2022-01-27 13:42:44 +09:00
c2ff39006f
HDFS-16398. Reconfig block report parameters for datanode ( #3831 )
2022-01-26 17:24:09 +09:00
4faac58841
HADOOP-18089. Test coverage for Async profiler servlets ( #3913 )
...
Reviewed-by: Akira Ajisaka <akiraaj@amazon.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-01-26 11:24:16 +08:00
4b26635a34
YARN-11034. Add enhanced headroom in AllocateResponse ( #3766 )
2022-01-25 09:25:18 -08:00
0d17b629ff
HADOOP-18093. Better exception handling for testFileStatusOnMountLink() in ViewFsBaseTest.java ( #3918 ). Contributed by Xing Lin.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-01-25 19:40:18 +05:30
8c7c49d877
HDFS-16401.Remove the worthless DatasetVolumeChecker#numAsyncDatasetChecks. ( #3838 )
2022-01-25 21:51:17 +08:00
94b884ae55
HDFS-16262. Async refresh of cached locations in DFSInputStream ( #3527 )
2022-01-25 11:42:35 +00:00
43153e80cb
HDFS-16428. Source path with storagePolicy cause wrong typeConsumed while rename ( #3898 ). Contributed by lei w.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-01-25 15:26:18 +08:00
d69938994e
HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background ( #3842 )
...
Reviewed-by: Istvan Fajth <pifta@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-01-25 13:02:37 +08:00
3ed3c74a6c
YARN-11015. Decouple queue capacity with ability to run OPPORTUNISTIC container ( #3779 )
2022-01-24 08:03:36 -08:00
b795f6f9a8
HADOOP-18094. Disable S3A auditing by default.
...
See HADOOP-18091. S3A auditing leaks memory through ThreadLocal references
* Adds a new option fs.s3a.audit.enabled to controls whether or not auditing
is enabled. This is false by default.
* When false, the S3A auditing manager is NoopAuditManagerS3A,
which was formerly only used for unit tests and
during filsystem initialization.
* When true, ActiveAuditManagerS3A is used for managing auditing,
allowing auditing events to be reported.
* updates documentation and tests.
This patch does not fix the underlying leak. When auditing is enabled,
long-lived threads will retain references to the audit managers
of S3A filesystem instances which have already been closed.
Contributed by Steve Loughran.
2022-01-24 13:37:33 +00:00
795a5efcc1
HADOOP-17593. hadoop-huaweicloud and hadoop-cloud-storage to remove log4j as transitive dependency
...
Signed-off-by: Akira Ajisaka <akiraaj@amazon.com>
2022-01-24 21:04:58 +09:00
5ef335da1e
HDFS-16430. Add validation to maximum blocks in EC group when adding an EC policy ( #3899 ). Contributed by daimin.
...
Reviewed-by: tomscut <litao@bigo.sg>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-01-24 12:04:26 +05:30
15b820c83c
HDFS-16402. Improve HeartbeatManager logic to avoid incorrect stats. ( #3839 ). Contributed by tomscut.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-01-24 14:26:30 +08:00
dae33cf935
YARN-11065. Bump follow-redirects from 1.13.3 to 1.14.7 in hadoop-yarn-ui ( #3890 )
...
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects ) from 1.13.3 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases )
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.13.3...v1.14.7 )
---
updated-dependencies:
- dependency-name: follow-redirects
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-20 21:44:10 +09:00
7c97c0f969
HADOOP-18084. ABFS: Add testfilePath while verifying test contents are read correctly ( #3903 )
...
Contributed by: Anmol Asrani
2022-01-19 10:13:13 +00:00
db2c3200e6
HDFS-16423. Balancer should not get blocks on stale storages ( #3883 )
...
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-01-19 17:59:42 +09:00
e355646330
HDFS-16399. Reconfig cache report parameters for datanode ( #3841 )
2022-01-19 15:10:39 +09:00
fcb1076699
HDFS-16426. Fix nextBlockReportTime when trigger full block report force ( #3887 )
2022-01-19 13:42:33 +09:00
d8ab84275e
HADOOP-18068. upgrade AWS SDK to 1.12.132 ( #3864 )
...
With this update, the versions of key shaded dependencies are
jackson 2.12.3
httpclient 4.5.13
Contributed by Steve Loughran
2022-01-18 10:31:28 +00:00
14ba19af06
HADOOP-17409. Remove s3guard from S3A module ( #3534 )
...
Completely removes S3Guard support from the S3A codebase.
If the connector is configured to use any metastore other than
the null and local stores (i.e. DynamoDB is selected) the s3a client
will raise an exception and refuse to initialize.
This is to ensure that there is no mix of S3Guard enabled and disabled
deployments with the same configuration but different hadoop releases
-it must be turned off completely.
The "hadoop s3guard" command has been retained -but the supported
subcommands have been reduced to those which are not purely S3Guard
related: "bucket-info" and "uploads".
This is major change in terms of the number of files
changed; before cherry picking subsequent s3a patches into
older releases, this patch will probably need backporting
first.
Goodbye S3Guard, your work is done. Time to die.
Contributed by Steve Loughran.
2022-01-17 18:08:57 +00:00
a94e9fcbde
YARN-11055. Add missing newline in cgroups-operations.c ( #3851 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-17 16:20:48 +09:00
d8862822d2
HDFS-16420. Avoid deleting unique data blocks when deleting redundancy striped blocks. ( #3880 )
...
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-01-14 22:38:11 +09:00
f02374df92
HDFS-16400. Reconfig DataXceiver parameters for datanode ( #3843 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-01-14 14:48:10 +09:00
034dc8d03c
HDFS-16043. Add markedDeleteBlockScrubberThread to delete blocks asynchronously ( #3063 ). Contributed by Xiangyi Zhu.
...
Reviewed-by: tomscut <litao@bigo.sg>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-01-13 12:56:19 +08:00
7542677470
HADOOP-16223. Remove misleading fs.s3a.delegation.tokens.enabled prompt ( #3879 )
...
Contributed by Ahmar Suhail
2022-01-12 17:25:17 +00:00
271ac0e091
HDFS-16419. Make HDFS data transfer tools cross platform ( #3873 )
2022-01-12 19:57:27 +05:30
93294f0329
HADOOP-18077. ProfileOutputServlet unable to proceed due to NPE ( #3875 )
2022-01-12 16:20:34 +08:00
e2d620192a
HADOOP-18044. Hadoop - Upgrade to jQuery 3.6.0 ( #3791 )
...
Co-authored-by: luoyuan <luoyuan@shopee.com>
2022-01-12 11:40:32 +08:00
21fe0e63fd
YARN-10632. Make auto queue creation maximum allowed depth configurable ( #3859 )
2022-01-11 14:38:59 +08:00
b88d5ae1c6
HDFS-16417. StaticRouterRpcFairnessPolicyController init ( #3871 )
2022-01-10 18:28:20 +08:00
b27732c69b
HADOOP-14334. S3 SSEC tests to downgrade when running against a mandatory encryption object store ( #3870 )
...
Contributed by Monthon Klongklaew
2022-01-09 18:01:47 +00:00
39efbc6b6f
HDFS-16404. Fix typo for CachingGetSpaceUsed ( #3844 ). Contributed by tomscut.
2022-01-09 16:41:10 +08:00
08c803ea30
MAPREDUCE-7371. DistributedCache alternative APIs should not use DistributedCache APIs internally ( #3855 )
2022-01-09 00:18:10 +09:00
f64fda0f00
HADOOP-18055. Async Profiler endpoint for Hadoop daemons ( #3824 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-06 17:56:49 +08:00
da0a6ba1ce
HADOOP-18065. ExecutorHelper.logThrowableFromAfterExecute() is too noisy. ( #3860 )
...
Downgrading warn logs to debug in case of InterruptedException
Contributed By: Mukund Thakur
2022-01-06 10:54:27 +05:30
46b02788bd
HDFS-16371. Exclude slow disks when choosing volume ( #3753 )
2022-01-06 09:31:00 +09:00
litao
Gautham B A
minni31
Ayush Saxena
KevinWikant
secfree
Akira Ajisaka
Wei-Chiu Chuang
Li MingXiang
Viraj Jasani
Xing Lin
jianghuazhu
Bryan Beaudreault
Thinker313
daimin
Andrew Chung
Steve Loughran
zhongjun2
dependabot[bot]
Anmol Asrani
qinyuren
Gera Shegalov
Jackson Wang
Xiangyi Zhu
ahmarsuhail
luoyuan3471
Andras Gyori
Felix Nguyen
monthonk
Mukund Thakur