Szilard Nemeth
e0a339223a
HADOOP-18709. Add curator based ZooKeeper communication support over SSL/TLS into the common library. Contributed by Ferenc Erdelyi
2023-06-04 14:40:41 -04:00
Viraj Jasani
03a499821c
HADOOP-18207. Introduce hadoop-logging module ( #5503 )
...
Reviewed-by: Duo Zhang <zhangduo@apache.org>
2023-06-02 18:07:34 -07:00
Steve Loughran
160b9fc3c9
HADOOP-18755. openFile builder new optLong() methods break hbase-filesystem ( #5704 )
...
This is a followup to
HADOOP-18724. Open file fails with NumberFormatException for S3AFileSystem
Contributed by Steve Loughran
2023-06-01 14:31:08 +01:00
smarthan
9f1e23cc67
HDFS-17031. Reduce some repeated codes in RouterRpcServer. ( #5701 ). Contributed by Chengwei Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-01 08:32:38 +05:30
NishthaShah
f8b7ddf69c
HDFS-16996. Fix flaky testFsCloseAfterClusterShutdown in TestFileCreation ( #5697 ). Contributed by Nishtha Shah.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-01 08:23:48 +05:30
Simbarashe Dzinamarira
d92a5815f4
HDFS-17027. RBF: Adds auto-msync support for clients connecting to routers. ( #5693 )
2023-05-31 10:20:19 -07:00
Marcono1234
9acf462d26
HDFS-17000. Fix faulty loop condition in TestDFSStripedOutputStreamUpdatePipeline ( #5699 ). Contributed by Marcono1234.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-31 16:10:07 +05:30
hchaverri
124313d215
HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency. ( #5691 ). Contributed by hchaverri.
...
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-31 14:26:31 +08:00
slfan1989
86c250a54a
YARN-7720. Race condition between second app attempt and UAM timeout when first attempt node is down. ( #5672 )
2023-05-29 10:37:08 -07:00
Xianming Lei
97afb33c73
YARN-11276. Add LRU cache for RMWebServices.getApps. ( #4793 )
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-05-26 20:46:00 +08:00
slfan1989
b977065cc4
YARN-11478. [Federation] SQLFederationStateStore Support Store ApplicationSubmitData. ( #5663 )
2023-05-24 11:43:20 -07:00
Steve Loughran
e6b54f7f68
Revert "HADOOP-18706. Improve S3ABlockOutputStream recovery ( #5563 )"
...
This reverts commit 372631c566
.
Reverted due to HADOOP-18744.
2023-05-24 19:22:22 +01:00
hfutatzhanghb
e9740cb17a
HDFS-16908. Fix javadoc of field IncrementalBlockReportManager#readyToSend. ( #5351 ). Contributed by farmmamba.
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-23 16:12:50 +05:30
Tamas Domok
aeb3f6f1a8
YARN-11490. Reverting YARN-11211 and eliminating the use of DefaultMetricsSystem during configuration validation ( #5644 )
2023-05-23 10:36:37 +02:00
Ashutosh Gupta
a98d15804a
MAPREDUCE-7419. Upgrade Junit 4 to 5 in hadoop-mapreduce-client-common ( #5028 ). Contributed by Ashutosh Gupta.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-23 11:37:49 +05:30
Viraj Jasani
3b65b5d68f
HDFS-17020. RBF: mount table addAll should print failed records in std error ( #5674 )
2023-05-22 18:39:52 -07:00
Gautham B A
afe850ca2c
HADOOP-18746. Install Python 3 for Windows 10 docker image ( #5679 )
...
* This PR installs Python 3.10.11 for
Windows 10 Docker image to fix
the issue with building mvnsite.
* After installing Python 3.10.11, it
creates the hardlink python -> python3
as required by the script.
2023-05-21 21:10:04 +05:30
hfutatzhanghb
5b22dc6ace
HDFS-16909. Improve ReplicaMap#mergeAll method. ( #5353 ). Contributed by ZhangHB.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-21 19:08:58 +08:00
huhaiyang
af933f3a4f
HDFS-17017. Fix the issue of arguments number limit in report command in DFSAdmin ( #5667 ). Contributed by Haiyang Hu.
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-21 09:03:34 +05:30
Xianming Lei
0110e24ed8
YARN-11496. Improve TimelineService log format. ( #5677 ). Contributed by Xianming Lei.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 14:57:45 +05:30
NishthaShah
9a524ede87
HDFS-17022. Fix the exception message to print the Identifier pattern ( #5678 ). Contributed by Nishtha Shah.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 14:40:23 +05:30
zhtttylz
408dbf318e
HDFS-17014. HttpFS Add Support getStatus API ( #5660 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 14:37:38 +05:30
NishthaShah
5272ed8670
HADOOP-17518. Update the regex to A-Z ( #5669 ). Contributed by Nishtha Shah.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 06:21:13 +05:30
Keyao Li
0914b3e792
HDFS-16697. Add logs if resources are not available in NameNodeResourcePolicy. ( #5569 ). Contributed by ECFuzz.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 18:38:01 +05:30
Keyao Li
339bc7b3a6
HDFS-16653. Improve error messages in ShortCircuitCache. ( #5568 ). Contributed by ECFuzz.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 07:43:18 +05:30
Xianming Lei
441fb23293
HDFS-17018. Improve dfsclient log format. ( #5668 ). Contributed by Xianming Lei.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 06:25:43 +05:30
Patrick GRANDJEAN
4627242c44
HADOOP-18652. Path.suffix raises NullPointerException ( #5653 ). Contributed by Patrick Grandjean.
...
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 05:16:55 +05:30
LiuGuH
f6770dee47
HDFS-16979. RBF: Add proxyuser port in hdfsauditlog ( #5552 ). Contributed by liuguanghua.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 05:02:16 +05:30
slfan1989
bba663038d
YARN-8898. Fix FederationInterceptor#allocate to set application priority in allocateResponse. ( #5645 )
2023-05-18 11:57:38 -07:00
Peter Szucs
ff8eac517a
YARN-11463. Node Labels root directory creation doesn't have a retry logic - 2nd addendum ( #5670 )
2023-05-18 14:48:43 +02:00
jianghuazhu
78cc528739
HDFS-17012.Remove unused DFSConfigKeys#DFS_DATANODE_PMEM_CACHE_DIRS_DEFAULT. ( #5659 ). Contributed by JiangHua Zhu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-18 14:27:19 +05:30
Tsz-Wo Nicholas Sze
350dcaf616
HDFS-17010. Add a subtree test to TestSnapshotDiffReport. ( #5656 )
2023-05-18 15:53:26 +08:00
wangzhaohui
03163f9de2
HDFS-17011. Fix the metric of "HttpPort" at DataNodeInfo ( #5657 ). Contributed by Zhaohui Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-18 12:12:11 +05:30
slfan1989
5ddaf2e133
YARN-11493. [Federation] ConfiguredRMFailoverProxyProvider Supports Randomly Select an Router. ( #5651 )
2023-05-17 11:09:10 -07:00
Viraj Jasani
8e17385141
HDFS-17009. RBF: state store putAll should also return failed records ( #5664 )
2023-05-17 09:33:34 -07:00
liang3zy22
482897a0f6
Fix typos in HDFS documents. ( #5665 )
2023-05-17 09:28:01 -07:00
Steve Loughran
a90c722143
HADOOP-18724. [FOLLOW-UP] cherrypick changes from branch-3.3 backport ( #5662 )
...
* move FileContext.copy() onto optLong()
* move FileUtil onto optLong()
This brings trunk into sync with the branch-3.3 changes
2023-05-16 18:16:24 +01:00
Viraj Jasani
bef40e9427
HADOOP-18688. S3A audit header to include count of items in delete ops ( #5621 )
...
The auditor-generated http referrer URL now includes the count of keys
to delete in the "ks" query parameter
Contributed by Viraj Jasani
2023-05-16 10:40:16 +01:00
Chun Chen
11af08d67a
YARN-11489. Fix memory leak of DelegationTokenRenewer futures in DelegationTokenRenewerPoolTracker. ( #5629 ). Contributed by Chun Chen.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-14 21:38:04 +08:00
smarthan
251439d769
HDFS-16985. Fix data missing issue when delete local block file. ( #5564 ). Contributed by Chengwei Wang.
...
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-14 21:33:38 +08:00
slfan1989
e0938b4c2a
YARN-11495. Fix typos in hadoop-yarn-server-web-proxy. ( #5652 ). Contributed by Shilun Fan.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-13 11:41:38 +05:30
slfan1989
2f87f716fa
YARN-3660. BackPort [GPG] Federation Global Policy Generator (service hook only). ( #5625 )
2023-05-12 18:12:05 -07:00
Viraj Jasani
5d0cc455f5
HDFS-17008. Fix RBF JDK 11 javadoc warnings ( #5648 )
2023-05-12 17:27:13 -07:00
zhtttylz
0c77629849
HDFS-17001. Support getStatus API in WebHDFS ( #5628 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-13 03:49:13 +05:30
Steve Loughran
ad1e3a0f5b
HADOOP-18724. (followup) remove deprecation on optLong/optDouble methods ( #5650 )
...
Somehow @Deprecated crept in to the declaration of the
new FSBuilder optLong/optDouble methods.
2023-05-12 15:22:37 +01:00
susheel-gupta
0f3406ac34
YARN-11312: [UI2] Refresh buttons don't work after EmberJS upgrade ( #5649 )
2023-05-12 14:20:26 +02:00
WangYuanben
905bfa84a8
HDFS-16965. Add switch to decide whether to enable native codec. ( #5520 ). Contributed by WangYuanben.
...
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-12 04:12:02 +05:30
Steve Loughran
e76c09ac3b
HADOOP-18724. Open file fails with NumberFormatException for S3AFileSystem ( #5611 )
...
This:
1. Adds optLong, optDouble, mustLong and mustDouble
methods to the FSBuilder interface to let callers explicitly
passin long and double arguments.
2. The opt() and must() builder calls which take float/double values
now only set long values instead, so as to avoid problems
related to overloaded methods resulting in a ".0" being appended
to a long value.
3. All of the relevant opt/must calls in the hadoop codebase move to
the new methods
4. And the s3a code is resilient to parse errors in is numeric options
-it will downgrade to the default.
This is nominally incompatible, but the floating-point builder methods
were never used: nothing currently expects floating point numbers.
For anyone who wants to safely set numeric builder options across all compatible
releases, convert the number to a string and then use the opt(String, String)
and must(String, String) methods.
Contributed by Steve Loughran
2023-05-11 17:57:25 +01:00
Viraj Jasani
fe61d8f073
HDFS-16978. RBF: Admin command to support bulk add of mount points ( #5554 ). Contributed by Viraj Jasani.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-11 08:45:34 +05:30
zhtttylz
5084e881ef
HDFS-16990. HttpFS Add Support getFileLinkStatus API ( #5602 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-10 23:10:19 +05:30