eb1d3ebe2f
YARN-11442. Refactor FederationInterceptorREST Code. ( #5420 )
2023-03-31 15:29:18 -07:00
389b3ea6e3
HADOOP-18012. ABFS: Enable config controlled ETag check for Rename idempotency ( #5488 )
...
To support recovery of network failures during rename, the abfs client
fetches the etag of the source file, and when recovering from a
failure, uses this tag to determine whether the rename succeeded
before the failure happened.
* This works for files, but not directories
* It adds the overhead of a HEAD request before each rename.
* The option can be disabled by setting "fs.azure.enable.rename.resilience"
to false
Contributed by Sree Bhattacharyya
2023-03-31 19:15:15 +01:00
016362a28b
HADOOP-18548. Hadoop Archive tool (HAR) should acquire delegation tokens from source and destination file systems ( #5355 )
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-03-30 07:12:02 +08:00
b4bcbb9515
HDFS-16959. RBF: State store cache loading metrics ( #5497 )
2023-03-29 10:43:13 -07:00
5bc8f25327
YARN-11446. [Federation] Add updateSchedulerConfiguration, getSchedulerConfiguration REST APIs for Router. ( #5476 )
2023-03-28 09:33:19 -07:00
aa602381c5
YARN-11426. Improve YARN NodeLabel Memory Display. ( #5335 )
...
YARN-11426. Improve YARN NodeLabel Memory Display.
Co-authored-by: slfan1989 <louj1988@@>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-03-28 22:48:46 +08:00
700147b4ac
HDFS-16964. Improve processing of excess redundancy after failover. ( #5510 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-28 16:14:59 +08:00
b5e8269d9b
HADOOP-18458: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS ( #4912 )
2023-03-28 14:27:01 +08:00
926993cb73
YARN-11376. [Federation] Support updateNodeResource、refreshNodesResources API's for Federation. ( #5496 )
2023-03-27 09:27:21 -07:00
762d3ddb43
HADOOP-18146: ABFS: Added changes for expect hundred continue header ( #4039 )
...
This change lets the client react pre-emptively to server load without getting to 503 and the exponential backoff
which follows. This stops performance suffering so much as capacity limits are approached for an account.
Contributed by Anmol Asranii
2023-03-27 12:43:34 +01:00
ee01c64c6c
HADOOP-18676. jettison dependency override in hadoop-common lib ( #5513 )
2023-03-27 09:59:02 +02:00
b82bcbd8ad
Revert "HADOOP-18676. Fixing jettison vulnerability of hadoop-common lib ( #5507 )"
...
This reverts commit 72b0122706
2023-03-25 12:04:28 +05:30
72b0122706
HADOOP-18676. Fixing jettison vulnerability of hadoop-common lib ( #5507 )
...
* HADOOP-18587. Fixing jettison vulnerability of hadoop-common lib
* no need for excluding, let it come
Change-Id: Ia6e4ad351158dd4b0510dec34bbde531a60e7654
2023-03-24 16:31:45 +01:00
69748aae32
YARN-11461. fix NPE in determineMissingParents (auto queue creation / CS). ( #5506 )
...
Change-Id: Iaaaf43a545588eaff8a0a20f6f3c27258a45f390
2023-03-24 09:38:53 +01:00
5cf62d1498
HDFS-16948. Update log of BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by balancer. ( #5474 ). Contributed by Kidd53685368.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-23 17:00:23 +08:00
028cde0006
HADOOP-18644. Add bswap support for LoongArch64. ( #5453 ). Contributed by zhaixiaojuan.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-23 11:08:59 +08:00
e3cb9573e1
HADOOP-18662. ListFiles with recursive fails with FNF. ( #5477 ). Contributed by Ayush Saxena.
...
Reviewed-by: Steve Loughran <stevel@apache.org
2023-03-23 08:30:08 +05:30
67e02a92e0
HADOOP-18666. A whitelist of endpoints to skip Kerberos authentication doesn't work for ResourceManager and Job History Server ( #5480 )
2023-03-22 10:54:41 +09:00
0dbe1d3284
HADOOP-18668. Path capability probe for truncate is only honored by RawLocalFileSystem ( #5492 )
2023-03-21 10:23:16 +08:00
9a8287c36f
HADOOP-18669. Remove Log4Json Layout ( #5493 )
2023-03-21 10:07:06 +08:00
f8d0949f7d
HDFS-16953. RBF: Mount table store APIs should update cache only if state store record is successfully updated ( #5482 )
2023-03-18 14:43:25 -07:00
b6a9d7b442
HADOOP-18631. (ADDENDUM) Use LogCapturer to match audit log pattern and remove hdfs async audit log configs ( #5451 )
2023-03-18 06:33:50 +08:00
fa723ae839
YARN-11445. [Federation] Add getClusterInfo, getClusterUserInfo REST APIs for Router. ( #5472 )
2023-03-17 11:59:45 -07:00
759ddebb13
HADOOP-18647. x-ms-client-request-id to identify the retry of an API. ( #5437 )
...
The x-ms-client-request-id now includes a field to indicate a call is a retry of a previous
operation
Contributed by Pranav Saxena
2023-03-15 20:03:22 +00:00
7c42d0f7da
HADOOP-17746. Compatibility table in directory_markers.md doesn't render right. ( #3116 )
...
Contributed by Masatake Iwasaki
2023-03-15 17:10:42 +00:00
15935fa865
HDFS-16947. RBF NamenodeHeartbeatService to report error for not being able to register namenode in state store ( #5470 )
2023-03-16 00:59:55 +08:00
cf4a678ce9
HADOOP-18649. CLA and CRLA appenders to be replaced with RFA ( #5448 )
2023-03-16 00:46:17 +08:00
405bfa2800
HADOOP-18654. Remove unused custom appender TaskLogAppender ( #5457 )
2023-03-16 00:45:37 +08:00
eee2ea075d
HDFS-16942. Addendum. Send error to datanode if FBR is rejected due to bad lease ( #5478 ). Contributed by Stephen O'Donnell/
2023-03-15 10:03:00 +05:30
aff840c59c
HADOOP-18653. LogLevel servlet to determine log impl before using setLevel ( #5456 )
...
The log level can only be set on Log4J log implementations;
probes are used to downgrade to a warning when other
logging back ends are used
Contributed by Viraj Jasani
2023-03-13 12:30:12 +00:00
09469bf47d
HADOOP-18661. Fix bin/hadoop usage script terminology. ( #5473 )
...
Followup to HADOOP-13209: s/slaves/r/workers in
the usage message you get when you type "bin/hadoop"
Contributed by Steve Loughran
2023-03-13 12:24:36 +00:00
476340c699
HADOOP-18658. snakeyaml dependency: upgrade to v2.0 ( #5467 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-03-13 10:08:04 +05:30
ca6f5afb6d
HDFS-16942. Send error to datanode if FBR is rejected due to bad lease ( #5460 )
2023-03-11 16:40:07 +00:00
734f7abfb8
HADOOP-18646. Upgrade Netty to 4.1.89.Final to fix CVE-2022-41881 ( #5435 )
...
This fixes CVE-2022-41881.
This also upgrades io.opencensus dependencies to 0.12.3
Contributed by Aleksandr Nikolaev
2023-03-10 15:27:22 +00:00
b406060c6b
YARN-8972. [Router] Add support to prevent DoS attack over ApplicationSubmissionContext size. ( #5382 )
2023-03-08 13:29:30 -08:00
487368c4b9
HADOOP-18655. Upgrade kerby to 2.0.3 due to CVE-2023-25613 ( #5458 )
...
Upgrade kerby to 2.0.3 due to the CVE https://nvd.nist.gov/vuln/detail/CVE-2023-25613
Contributed by Rohit Kumar Badeau
2023-03-08 15:31:03 +00:00
358bf80c94
HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. ( #5299 )
...
Contributed by Pranav Saxena
2023-03-07 17:02:13 +00:00
927401886a
HDFS-16934. TestDFSAdmin.testAllDatanodesReconfig regression ( #5434 )
...
Contributed by Shilun Fan
2023-03-06 15:26:53 +00:00
2cb0c35fc1
HDFS-16939. Fix the thread safety bug in LowRedundancyBlocks. ( #5450 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-06 20:10:31 +08:00
2a0dc2ab2f
YARN-11383. Workflow priority mappings is case sensitive ( #5171 )
...
Contributed by Aparajita Choudhary
2023-03-05 21:25:16 +05:30
6bd2444815
HDFS-16923. [SBN read] getlisting RPC to observer will throw NPE if path does not exist ( #5400 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2023-03-01 16:18:38 -08:00
e1ca466bdb
HADOOP-18648. Avoid loading kms log4j properties dynamically by KMSWebServer ( #5441 )
2023-03-02 08:02:07 +08:00
162288bc0a
HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref… ( #5322 )
...
HDFS-16896 clear ignoredNodes list when we clear deadnode list on refetchLocations.
ignoredNodes list is only used on hedged read codepath
Co-authored-by: Tom McCormick <tmccormi@linkedin.com>
2023-03-01 19:47:04 +00:00
2ab7eb4caa
HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks ( #5432 )
...
Contributed by Viraj Jasani
2023-03-01 18:53:10 +00:00
8f6be3678d
MAPREDUCE-7434. Fix ShuffleHandler tests. Contributed by Tamas Domok
2023-03-01 16:10:05 +01:00
28d2753d2f
HADOOP-18645. Provide keytab file key name with ServiceStateException ( #5433 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2023-03-01 09:34:12 +08:00
bcc51ce2c5
YARN-11375. [Federation] Support refreshAdminAcls、refreshServiceAcls API's for Federation. ( #5312 )
2023-02-28 14:44:00 -08:00
dcd9dc6983
HADOOP-18641. Cloud connector dependency and LICENSE fixup. ( #5429 )
...
POM and LICENSE fixup of transient dependencies
* Exclude hadoop-cloud-storage imports which come in with hadoop-common
* Add explicit import of hadoop's org.codehaus.jettison declaration
to hadoop-aliyun
* Tune aliyun jars imports
* Update LICENSE-binary for the current set of libraries.
Contributed by Steve Loughran
2023-02-28 10:48:54 +00:00
0ca5686034
HDFS-16917 Add transfer rate quantile metrics for DataNode reads ( #5397 )
...
Co-authored-by: Ravindra Dingankar <rdingankar@linkedin.com>
2023-02-27 18:26:32 +00:00
61f369c43e
HDFS-16890: RBF: Ensures router periodically refreshes its record of a namespace's state. ( #5298 )
2023-02-27 17:56:24 +00:00
slfan1989
sreeb-msft
Galsza
Viraj Jasani
zhangshuyan
Jinhu Wu
Anmol Asrani
Andras Katona
Ayush Saxena
Tamas Domok
Kidd5368
zhaixiaojuan@loongson.cn
Yubi Lee
Pranav Saxena
Masatake Iwasaki
Stephen O'Donnell
Steve Loughran
PJ Fanning
nao
rohit-kb
Varun Saxena
ZanderXu
Tom
Szilard Nemeth
rdingankar
Simbarashe Dzinamarira