0d06fd77de
HADOOP-18134. Setup Jenkins nightly CI for Windows 10 ( #5062 )
...
This PR gets Yetus to run on Windows 10
against the Hadoop codebase. It introduces
the following changes to allow us to setup
the nightly CI on Jenkins for Hadoop on
Windows 10.
* Hadoop personality changes for Yetus.
Additional arguments have been passed,
which are necessary to build and test
Hadoop on Windows 10.
* Docker image for building Hadoop on
Windows 10.
Installs the necessary tools that are
needed to run Yetus.
* dev-support/jenkins.sh file.
Passing of some flags are handled here
which are needed for the nightly CI.
2023-05-03 22:44:54 +05:30
476f60a806
YARN-10144. Federation: Add missing FederationClientInterceptor APIs. ( #5587 )
2023-05-03 09:21:56 -07:00
c1d10f3872
YARN-9049. Add application submit data to state store. ( #5606 )
2023-05-03 09:19:54 -07:00
0e46388474
HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common ( #5553 )
...
The HDFS lease APIs have been replicated as interfaces in hadoop-common so other filesystems can
also implement them. Applications which use the leasing APIs should migrate to the new
interface where possible.
Contributed by Stephen Wu
2023-05-03 11:05:55 +01:00
668c0a0930
YARN-11379. [Federation] Support mapAttributesToNodes, getGroupsForUser API's for Federation. ( #5596 )
2023-05-02 13:46:06 -07:00
fddc9769a5
HADOOP-18726. Set the locale to avoid printing useless logs. ( #5612 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-03 00:09:36 +08:00
bfcf5dd03b
HADOOP-18697. S3A prefetch: failure of ITestS3APrefetchingInputStream#testRandomReadLargeFile ( #5580 )
...
Contributed by Viraj Jasani
2023-05-02 15:21:46 +01:00
73ca64a3ba
YARN-11450. Improvements for TestYarnConfigurationFields and TestConfigurationFieldsBase ( #5455 )
2023-05-02 15:52:57 +02:00
87e17b2713
YARN-11437. [hotfix][Federation] SQLFederationStateStore Support Version. ( #5598 )
2023-05-01 16:01:50 -07:00
5147106b59
HADOOP-18725. Avoid cross-platform build for irrelevant Dockerfile changes ( #5610 )
2023-05-01 09:35:50 -07:00
1079890ae3
HDFS-16707. RBF: Expose RouterRpcFairnessPolicyController related request record metrics for each nameservice to Prometheus ( #4665 ). Contributed by Jiale Qi.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-01 21:22:26 +05:30
d75c6d9d57
HADOOP-18715. Add debug log for getting details of tokenKindMap ( #5608 ). Contributed by Pralabh Kumar.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-29 17:28:49 +05:30
1a2cd965a7
HDFS-16897. Fix abundant Broken pipe exception in BlockSender ( #5329 ). Contributed by fanluo.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-29 17:24:56 +05:30
70c0aa342e
YARN-11482. Fix bug of DRF comparision DominantResourceFairnessComparator2 in fair scheduler. ( #5607 ). Contributed by Xiaoqiao He.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
2023-04-29 11:18:42 +08:00
5ed7e912dc
YARN-11469. Refactor FederationStateStoreFacade Cache Code. ( #5570 )
...
Co-authored-by: slfan1989 <louj1988@@>
2023-04-28 14:11:13 -07:00
0e63152218
HDFS-16995. Remove unused parameters at NameNodeHttpServer#initWebHdfs ( #5601 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-28 05:25:01 +05:30
d9576bb9ee
HDFS-16972. Delete a snapshot may deleteCurrentFile. ( #5532 )
2023-04-27 09:17:47 -07:00
60a7e8acaa
YARN-11459 Changed label called "max resource" on UIv1 and UIv2 ( #5527 )
2023-04-27 15:25:25 +02:00
245fde17d7
YARN-11474.The yarn queue list is displayed on the CLI ( #5577 )
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-04-27 18:05:48 +08:00
eb749ddd4d
HADOOP-18695. S3A: reject multipart copy requests when disabled ( #5548 )
...
Contributed by Steve Loughran.
2023-04-27 10:59:46 +01:00
55eebcf277
YARN-11378. [Federation] Support checkForDecommissioningNodes、refreshClusterMaxPriority API's for Federation. ( #5551 )
2023-04-25 14:12:38 -07:00
a716459cdf
YARN-11437. [Federation] SQLFederationStateStore Support Version. ( #5589 )
2023-04-25 14:09:45 -07:00
5af0845076
HDFS-16672. Fix lease interval comparison in BlockReportLeaseManager ( #4598 ). Contributed by dzcxzl.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org
2023-04-26 02:11:11 +05:30
b6b9bd67bb
MAPREDUCE-7437. MR Fetcher class to use an AtomicInteger to generate IDs. ( #5579 )
...
...as until now it wasn't thread safe
Contributed by Steve Loughran
2023-04-25 19:53:40 +01:00
6aac6cb212
HADOOP-18660. Filesystem Spelling Mistake ( #5475 ). Contributed by Sebastian Baunsgaard.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-25 21:44:04 +05:30
2f66f0b83a
HADOOP-18694. Client.Connection#updateAddress needs to ensure that address is resolved before updating ( #5542 ). Contributed by dzcxzl.
...
Reviewed-by: Steve Vaughan <email@stevevaughan.me>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org
2023-04-25 03:52:49 +05:30
c9e0af9961
HDFS-16981. Support getFileLinkStatus API in WebHDFS ( #5572 ). Contributed by Hualong Zhang.
...
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-25 03:30:56 +05:30
dc78849f27
HDFS-16975. FileWithSnapshotFeature.isCurrentFileDeleted is not reloaded from FSImage. ( #5546 )
2023-04-24 09:04:28 -07:00
05e6dc19ea
HADOOP-18705. ABFS should exclude incompatible credential providers. ( #5560 )
...
Contributed by Tamas Domok.
2023-04-24 15:46:40 +01:00
6a23c376c9
HDFS-16986. EC: Fix locationBudget in getListing(). ( #5582 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-04-24 18:53:25 +08:00
5b23224970
HADOOP-18714. Wrong StringUtils.join() called in AbstractContractRootDirectoryTest ( #5578 )
2023-04-24 09:17:12 +02:00
51dcbd1d61
HDFS-16988. Improve NameServices info at JournalNode web UI ( #5584 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-24 01:13:02 +05:30
b683769fc9
HADOOP-18712. Upgrade to jetty 9.4.51 due to cve ( #5574 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-24 01:01:51 +05:30
3b7783c549
HADOOP-18689. Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project ( #5502 )
...
Co-authored-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-22 16:19:21 +05:30
ad49ddda0e
HADOOP-18711. upgrade nimbus jwt jar due to issues in its embedded shaded json-smart code. ( #5573 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-22 14:01:09 +05:30
742e07d9c3
HADOOP-18710. Add RPC metrics for response time ( #5545 ). Contributed by liuguanghua.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-22 01:06:08 +05:30
964c1902c8
YARN-11463. Node Labels root directory creation doesn't have a retry logic ( #5562 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2023-04-21 14:03:22 +02:00
9e24ed2196
HADOOP-18691. Add a CallerContext getter on the Schedulable interface ( #5540 )
2023-04-20 10:11:25 -07:00
0918c87fa2
HADOOP-18687. Remove json-smart dependency. ( #5549 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 18:28:09 +05:30
d07356e60e
HADOOP-18597. Simplify single node instructions for creating directories for Map Reduce. ( #5305 )
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 16:12:44 +05:30
1ff7a65b9f
HDFS-16954. RBF: The operation of renaming a multi-subcluster directory to a single-cluster directory should throw ioexception. ( #5483 ). Contributed by Max Xie.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 15:19:18 +05:30
9e3d5c754b
Revert "HADOOP-18687. Remove json-smart dependency. ( #5549 ). Contributed by PJ Fanning."
...
This reverts commit b6c0ec796e
2023-04-20 10:26:08 +05:30
b6c0ec796e
HADOOP-18687. Remove json-smart dependency. ( #5549 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 00:47:22 +05:30
5119d0c72f
HDFS-16982 Use the right Quantiles Array for Inverse Quantiles snapshot ( #5556 )
2023-04-18 10:47:37 -07:00
a258f1f235
YARN-11326. [Federation] Add RM FederationStateStoreService Metrics. ( #4963 )
2023-04-18 09:13:08 -07:00
635521db4c
YARN-11438. [Federation] ZookeeperFederationStateStore Support Version. ( #5537 )
2023-04-18 09:05:52 -07:00
0e3aafe6c0
HADOOP-18399. S3A Prefetch - SingleFilePerBlockCache to use LocalDirAllocator ( #5054 )
...
Contributed by Viraj Jasani
2023-04-18 16:37:48 +01:00
405ed1dde6
HADOOP-18470. Hadoop 3.3.5 release wrap-up ( #5558 )
...
Post-release updates of the branches
* Add jdiff xml files from 3.3.5 release.
* Declare 3.3.5 as the latest stable release.
* Copy release notes.
2023-04-18 10:12:07 +01:00
6ea10cf41b
HADOOP-18696. ITestS3ABucketExistence arn test failures. ( #5557 )
...
Explicitly sets the fs.s3a.endpoint.region to eu-west-1 so
the ARN-referenced fs creation fails with unknown store
rather than IllegalArgumentException.
Steve Loughran
2023-04-17 10:18:33 +01:00
2c4d6bf33d
YARN-11465. Improved YarnClient Log Format ( #5550 )
...
Co-authored-by: yl09099 <shaq376260428@163.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-04-17 09:27:52 +08:00
Gautham B A
slfan1989
Tak Lon (Stephen) Wu
zhangshuyan
Viraj Jasani
Szilard Nemeth
SevenAddSix
Pralabh Kumar
fanluoo
Hexiaoqiao
wangzhaohui
Tsz-Wo Nicholas Sze
Riya Khandelwal
yl09099
Steve Loughran
cxzl25
Sebastian Baunsgaard
zhtttylz
Tamas Domok
Doroszlai, Attila
PJ Fanning
dependabot[bot]
LiuGuH
Ashutosh Gupta
Christos Bisias
Nikita Eshkeev
Neil
Ayush Saxena
rdingankar