Stephen O'Donnell
ca6f5afb6d
HDFS-16942. Send error to datanode if FBR is rejected due to bad lease ( #5460 )
2023-03-11 16:40:07 +00:00
nao
734f7abfb8
HADOOP-18646. Upgrade Netty to 4.1.89.Final to fix CVE-2022-41881 ( #5435 )
...
This fixes CVE-2022-41881.
This also upgrades io.opencensus dependencies to 0.12.3
Contributed by Aleksandr Nikolaev
2023-03-10 15:27:22 +00:00
slfan1989
b406060c6b
YARN-8972. [Router] Add support to prevent DoS attack over ApplicationSubmissionContext size. ( #5382 )
2023-03-08 13:29:30 -08:00
rohit-kb
487368c4b9
HADOOP-18655. Upgrade kerby to 2.0.3 due to CVE-2023-25613 ( #5458 )
...
Upgrade kerby to 2.0.3 due to the CVE https://nvd.nist.gov/vuln/detail/CVE-2023-25613
Contributed by Rohit Kumar Badeau
2023-03-08 15:31:03 +00:00
Pranav Saxena
358bf80c94
HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. ( #5299 )
...
Contributed by Pranav Saxena
2023-03-07 17:02:13 +00:00
slfan1989
927401886a
HDFS-16934. TestDFSAdmin.testAllDatanodesReconfig regression ( #5434 )
...
Contributed by Shilun Fan
2023-03-06 15:26:53 +00:00
zhangshuyan
2cb0c35fc1
HDFS-16939. Fix the thread safety bug in LowRedundancyBlocks. ( #5450 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-06 20:10:31 +08:00
Varun Saxena
2a0dc2ab2f
YARN-11383. Workflow priority mappings is case sensitive ( #5171 )
...
Contributed by Aparajita Choudhary
2023-03-05 21:25:16 +05:30
ZanderXu
6bd2444815
HDFS-16923. [SBN read] getlisting RPC to observer will throw NPE if path does not exist ( #5400 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2023-03-01 16:18:38 -08:00
Viraj Jasani
e1ca466bdb
HADOOP-18648. Avoid loading kms log4j properties dynamically by KMSWebServer ( #5441 )
2023-03-02 08:02:07 +08:00
Tom
162288bc0a
HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref… ( #5322 )
...
HDFS-16896 clear ignoredNodes list when we clear deadnode list on refetchLocations.
ignoredNodes list is only used on hedged read codepath
Co-authored-by: Tom McCormick <tmccormi@linkedin.com>
2023-03-01 19:47:04 +00:00
Viraj Jasani
2ab7eb4caa
HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks ( #5432 )
...
Contributed by Viraj Jasani
2023-03-01 18:53:10 +00:00
Szilard Nemeth
8f6be3678d
MAPREDUCE-7434. Fix ShuffleHandler tests. Contributed by Tamas Domok
2023-03-01 16:10:05 +01:00
Viraj Jasani
28d2753d2f
HADOOP-18645. Provide keytab file key name with ServiceStateException ( #5433 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2023-03-01 09:34:12 +08:00
slfan1989
bcc51ce2c5
YARN-11375. [Federation] Support refreshAdminAcls、refreshServiceAcls API's for Federation. ( #5312 )
2023-02-28 14:44:00 -08:00
Steve Loughran
dcd9dc6983
HADOOP-18641. Cloud connector dependency and LICENSE fixup. ( #5429 )
...
POM and LICENSE fixup of transient dependencies
* Exclude hadoop-cloud-storage imports which come in with hadoop-common
* Add explicit import of hadoop's org.codehaus.jettison declaration
to hadoop-aliyun
* Tune aliyun jars imports
* Update LICENSE-binary for the current set of libraries.
Contributed by Steve Loughran
2023-02-28 10:48:54 +00:00
rdingankar
0ca5686034
HDFS-16917 Add transfer rate quantile metrics for DataNode reads ( #5397 )
...
Co-authored-by: Ravindra Dingankar <rdingankar@linkedin.com>
2023-02-27 18:26:32 +00:00
Simbarashe Dzinamarira
61f369c43e
HDFS-16890: RBF: Ensures router periodically refreshes its record of a namespace's state. ( #5298 )
2023-02-27 17:56:24 +00:00
slfan1989
8798b94ee1
YARN-11221. [Federation] Add replaceLabelsOnNodes, replaceLabelsOnNode REST APIs for Router. ( #5302 )
2023-02-27 09:34:39 -08:00
Viraj Jasani
a90238c0b8
HADOOP-18631. Migrate Async appenders to log4j properties ( #5418 )
2023-02-26 01:47:44 +08:00
slfan1989
25ebd0b8b1
YARN-11222. [Federation] Add addToClusterNodeLabels, removeFromClusterNodeLabels REST APIs for Router. ( #5328 )
2023-02-24 10:52:57 -08:00
slfan1989
27a54955f9
YARN-5604. [Federation] Add versioning for FederationStateStore. ( #5394 )
2023-02-24 10:51:19 -08:00
Steve Loughran
4067facae6
HADOOP-18470. Remove HDFS RBF text in the 3.3.5 index.md file
...
+ add a link to mukund's apachecon talk
Change-Id: I3d04b385ff1312aabf2a81d034f54f124d544a54
2023-02-23 13:23:35 +00:00
Steve Loughran
e2d7919dc1
Revert "HADOOP-18590. Publish SBOM artifacts ( #5281 )"
...
Causes HADOOP-18641. cyclonedx maven plugin breaks on recent maven releases
This reverts commit 6f99558c2e
.
2023-02-23 11:23:53 +00:00
Owen O'Malley
8025a60ae7
HDFS-16901: Minor fix for unit test.
...
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 16:37:49 -08:00
Simbarashe Dzinamarira
4cc33e5e37
HDFS-16901: RBF: Propagates real user's username via the caller context, when a proxy user is being used. ( #5346 )
2023-02-22 21:58:44 +00:00
slfan1989
2e997d818d
YARN-11370. [Federation] Refactor MemoryFederationStateStore code. ( #5126 )
2023-02-22 12:37:35 -08:00
Ayush Saxena
e8a6b2c2c4
HADOOP-18582. Addendum: Skip unnecessary cleanup logic in DistCp. ( #5409 )
...
Followup to the original HADOOP-18582.
Temporary path cleanup is re-enabled for -append jobs
as these will create temporary files when creating or overwriting files.
Contributed by Ayush Saxena
2023-02-22 19:29:41 +00:00
hchaverr
fb31393b65
HADOOP-18535. Implement token storage solution based on MySQL
...
Fixes #1240
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 10:38:50 -08:00
Steve Loughran
11a220c6e7
HADOOP-18636 LocalDirAllocator cannot recover from directory tree deletion ( #5412 )
...
Even though DiskChecker.mkdirsWithExistsCheck() will create the directory tree,
it is only called *after* the enumeration of directories with available
space has completed.
Directories which don't exist are reported as having 0 space, therefore
the mkdirs code is never reached.
Adding a simple mkdirs() -without bothering to check the outcome-
ensures that if a dir has been deleted then it will be reconstructed
if possible. If it can't it will still have 0 bytes of space
reported and so be excluded from the allocation.
Contributed by Steve Loughran
2023-02-22 11:48:12 +00:00
susheel-gupta
49b8ac19f2
YARN-11408. Add a check of autoQueueCreation is disabled for emitDefaultUserLimitFactor method ( #5278 )
...
* added a check of autoQueueCreationV2Disabled
Change-Id: If1e36c5969d270c1b81a4bbd2e883fa819c81f20
* added check of AutoCreateChildQueueDisabled
Change-Id: Ia011b4393ccd8d4d419a2e46b06a5237d050851c
* removed auto-create-child-queue-enabled check and emit
Change-Id: I7a154124519ecbd81379b46a238707c16db1e82a
2023-02-22 09:46:42 +01:00
Ayush Saxena
fe5bb49ad9
Revert "YARN-11404. Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure. Contributed by Susheel Gupta"
...
This reverts commit 8eda456d37
.
2023-02-22 07:28:13 +05:30
slfan1989
4e6e2f318c
YARN-11394. Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. ( #5288 )
...
Contributed by Shilun Fan
2023-02-21 14:39:32 +00:00
nao
acf82d4d55
HADOOP-18622. Upgrade ant to 1.10.13 ( #5360 ). Contributed by Aleksandr Nikolaev.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-02-21 16:48:49 +05:30
Viraj Jasani
88914cada0
HDFS-16925. Namenode audit log to only include IP address of client ( #5407 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-21 08:26:32 +08:00
zhtttylz
a3b500d046
HDFS-16916. Improve the use of JUnit Test in DFSClient ( #5404 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-18 13:17:37 +08:00
slfan1989
7e486038ea
YARN-11439. Fix Typo of hadoop-yarn-ui README.md. ( #5405 )
...
Co-authored-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-18 13:14:01 +08:00
Arnout Engelen
02fd87a4d8
HADOOP-18627. Add stronger wording in 'secure mode' introduction ( #5406 )
...
Make it more clear that when deploying Hadoop 'secure mode' is generally not optional.
Contributed by Arnout Engelen
2023-02-17 16:30:41 +00:00
Steve Loughran
10e7ca481c
YARN-11441. Revert YARN-10495.
...
This reverts commit 7d3c8ef606
.
2023-02-17 15:05:06 +00:00
Bryan Beaudreault
7e19bc31b6
HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers ( #4215 )
2023-02-16 18:13:25 +00:00
Mehakmeet Singh
7a0903b743
HADOOP-18633. fix test AbstractContractDistCpTest#testDistCpUpdateCheckFileSkip ( #5401 )
...
Contributed by: Mehakmeet Singh
2023-02-16 10:09:06 +05:30
hfutatzhanghb
723535b788
HDFS-16914. Add some logs for updateBlockForPipeline RPC. ( #5381 )
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-16 09:25:50 +08:00
slfan1989
a5f48eacca
YARN-11425. [Federation] Router Supports SubClusterCleaner. ( #5326 )
2023-02-15 14:40:34 -08:00
slfan1989
c3706597a3
YARN-11349. [Federation] Router Support DelegationToken With SQL. ( #5244 )
2023-02-15 14:38:41 -08:00
Ankit Saurabh
f4f2793f3b
HADOOP-18351. Reduce excess logging of errors during S3A prefetching reads ( #5274 )
...
Contributed by Ankit Saurabh
2023-02-15 18:28:42 +00:00
Zita Dombi
4cbe19f3a2
HDFS-16761. Namenode UI for Datanodes page not loading if any data node is down ( #5390 )
2023-02-15 16:16:04 +00:00
Steve Loughran
d56977e909
HADOOP-18470. More in the 3.3.5 index.html about security ( #5383 )
...
Expands on the comments in cluster config to tell people
they shouldn't be running a cluster without a private VLAN
in cloud, that Knox is good here, and unsecured clusters
without a VLAN are just computation-as-a-service to crypto miners
Contributed by Steve Loughran
2023-02-14 17:22:59 +00:00
SimhadriGovindappa
e2ab35084a
HADOOP-18630. Add gh-pages in asf.yaml to deploy the current trunk doc ( #5393 ). Contributed by Simhadri Govindappa.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-02-14 18:13:29 +05:30
Viraj Jasani
021fcc6c5e
HADOOP-18628. IPC Server Connection should log host name before returning VersionMismatch error ( #5385 )
...
Contributed by Viraj Jasani
2023-02-14 11:48:48 +00:00
GuoPhilipse
fe0541b58d
HDFS-16913. Fix flaky some unit tests since they offen timeout ( #5377 )
...
Co-authored-by: gf13871 <gf13871@ly.com>
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-14 15:29:38 +08:00