905bfa84a8
HDFS-16965. Add switch to decide whether to enable native codec. ( #5520 ). Contributed by WangYuanben.
...
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-12 04:12:02 +05:30
e76c09ac3b
HADOOP-18724. Open file fails with NumberFormatException for S3AFileSystem ( #5611 )
...
This:
1. Adds optLong, optDouble, mustLong and mustDouble
methods to the FSBuilder interface to let callers explicitly
passin long and double arguments.
2. The opt() and must() builder calls which take float/double values
now only set long values instead, so as to avoid problems
related to overloaded methods resulting in a ".0" being appended
to a long value.
3. All of the relevant opt/must calls in the hadoop codebase move to
the new methods
4. And the s3a code is resilient to parse errors in is numeric options
-it will downgrade to the default.
This is nominally incompatible, but the floating-point builder methods
were never used: nothing currently expects floating point numbers.
For anyone who wants to safely set numeric builder options across all compatible
releases, convert the number to a string and then use the opt(String, String)
and must(String, String) methods.
Contributed by Steve Loughran
2023-05-11 17:57:25 +01:00
a2dda0ce03
HADOOP-18359. Update commons-cli from 1.2 to 1.5. ( #5095 ). Contributed by Shilun Fan.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-10 01:42:12 +05:30
c974710d8e
HADOOP-18729. Fix mvnsite on Windows 10 ( #5618 )
2023-05-05 13:08:58 -07:00
0e46388474
HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common ( #5553 )
...
The HDFS lease APIs have been replicated as interfaces in hadoop-common so other filesystems can
also implement them. Applications which use the leasing APIs should migrate to the new
interface where possible.
Contributed by Stephen Wu
2023-05-03 11:05:55 +01:00
fddc9769a5
HADOOP-18726. Set the locale to avoid printing useless logs. ( #5612 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-03 00:09:36 +08:00
bfcf5dd03b
HADOOP-18697. S3A prefetch: failure of ITestS3APrefetchingInputStream#testRandomReadLargeFile ( #5580 )
...
Contributed by Viraj Jasani
2023-05-02 15:21:46 +01:00
73ca64a3ba
YARN-11450. Improvements for TestYarnConfigurationFields and TestConfigurationFieldsBase ( #5455 )
2023-05-02 15:52:57 +02:00
d75c6d9d57
HADOOP-18715. Add debug log for getting details of tokenKindMap ( #5608 ). Contributed by Pralabh Kumar.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-29 17:28:49 +05:30
6aac6cb212
HADOOP-18660. Filesystem Spelling Mistake ( #5475 ). Contributed by Sebastian Baunsgaard.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-25 21:44:04 +05:30
2f66f0b83a
HADOOP-18694. Client.Connection#updateAddress needs to ensure that address is resolved before updating ( #5542 ). Contributed by dzcxzl.
...
Reviewed-by: Steve Vaughan <email@stevevaughan.me>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org
2023-04-25 03:52:49 +05:30
5b23224970
HADOOP-18714. Wrong StringUtils.join() called in AbstractContractRootDirectoryTest ( #5578 )
2023-04-24 09:17:12 +02:00
742e07d9c3
HADOOP-18710. Add RPC metrics for response time ( #5545 ). Contributed by liuguanghua.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-22 01:06:08 +05:30
9e24ed2196
HADOOP-18691. Add a CallerContext getter on the Schedulable interface ( #5540 )
2023-04-20 10:11:25 -07:00
d07356e60e
HADOOP-18597. Simplify single node instructions for creating directories for Map Reduce. ( #5305 )
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 16:12:44 +05:30
5119d0c72f
HDFS-16982 Use the right Quantiles Array for Inverse Quantiles snapshot ( #5556 )
2023-04-18 10:47:37 -07:00
0e3aafe6c0
HADOOP-18399. S3A Prefetch - SingleFilePerBlockCache to use LocalDirAllocator ( #5054 )
...
Contributed by Viraj Jasani
2023-04-18 16:37:48 +01:00
405ed1dde6
HADOOP-18470. Hadoop 3.3.5 release wrap-up ( #5558 )
...
Post-release updates of the branches
* Add jdiff xml files from 3.3.5 release.
* Declare 3.3.5 as the latest stable release.
* Copy release notes.
2023-04-18 10:12:07 +01:00
2b60d0c1f4
[HDFS-16971] Add read metrics for remote reads in FileSystem Statistics #5534 ( #5536 )
2023-04-13 09:07:42 -07:00
3e2ae1da00
HDFS-16949 Introduce inverse quantiles for metrics where higher numer… ( #5495 )
2023-04-10 08:56:00 -07:00
e45451f9c7
HADOOP-18687. hadoop-auth: remove unnecessary dependency on json-smart ( #5524 )
...
Contributed by Michiel de Jong
2023-04-06 16:00:33 +01:00
b4bcbb9515
HDFS-16959. RBF: State store cache loading metrics ( #5497 )
2023-03-29 10:43:13 -07:00
ee01c64c6c
HADOOP-18676. jettison dependency override in hadoop-common lib ( #5513 )
2023-03-27 09:59:02 +02:00
b82bcbd8ad
Revert "HADOOP-18676. Fixing jettison vulnerability of hadoop-common lib ( #5507 )"
...
This reverts commit 72b0122706
2023-03-25 12:04:28 +05:30
72b0122706
HADOOP-18676. Fixing jettison vulnerability of hadoop-common lib ( #5507 )
...
* HADOOP-18587. Fixing jettison vulnerability of hadoop-common lib
* no need for excluding, let it come
Change-Id: Ia6e4ad351158dd4b0510dec34bbde531a60e7654
2023-03-24 16:31:45 +01:00
e3cb9573e1
HADOOP-18662. ListFiles with recursive fails with FNF. ( #5477 ). Contributed by Ayush Saxena.
...
Reviewed-by: Steve Loughran <stevel@apache.org
2023-03-23 08:30:08 +05:30
67e02a92e0
HADOOP-18666. A whitelist of endpoints to skip Kerberos authentication doesn't work for ResourceManager and Job History Server ( #5480 )
2023-03-22 10:54:41 +09:00
9a8287c36f
HADOOP-18669. Remove Log4Json Layout ( #5493 )
2023-03-21 10:07:06 +08:00
405bfa2800
HADOOP-18654. Remove unused custom appender TaskLogAppender ( #5457 )
2023-03-16 00:45:37 +08:00
aff840c59c
HADOOP-18653. LogLevel servlet to determine log impl before using setLevel ( #5456 )
...
The log level can only be set on Log4J log implementations;
probes are used to downgrade to a warning when other
logging back ends are used
Contributed by Viraj Jasani
2023-03-13 12:30:12 +00:00
09469bf47d
HADOOP-18661. Fix bin/hadoop usage script terminology. ( #5473 )
...
Followup to HADOOP-13209: s/slaves/r/workers in
the usage message you get when you type "bin/hadoop"
Contributed by Steve Loughran
2023-03-13 12:24:36 +00:00
e1ca466bdb
HADOOP-18648. Avoid loading kms log4j properties dynamically by KMSWebServer ( #5441 )
2023-03-02 08:02:07 +08:00
28d2753d2f
HADOOP-18645. Provide keytab file key name with ServiceStateException ( #5433 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2023-03-01 09:34:12 +08:00
0ca5686034
HDFS-16917 Add transfer rate quantile metrics for DataNode reads ( #5397 )
...
Co-authored-by: Ravindra Dingankar <rdingankar@linkedin.com>
2023-02-27 18:26:32 +00:00
4cc33e5e37
HDFS-16901: RBF: Propagates real user's username via the caller context, when a proxy user is being used. ( #5346 )
2023-02-22 21:58:44 +00:00
fb31393b65
HADOOP-18535. Implement token storage solution based on MySQL
...
Fixes #1240
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 10:38:50 -08:00
11a220c6e7
HADOOP-18636 LocalDirAllocator cannot recover from directory tree deletion ( #5412 )
...
Even though DiskChecker.mkdirsWithExistsCheck() will create the directory tree,
it is only called *after* the enumeration of directories with available
space has completed.
Directories which don't exist are reported as having 0 space, therefore
the mkdirs code is never reached.
Adding a simple mkdirs() -without bothering to check the outcome-
ensures that if a dir has been deleted then it will be reconstructed
if possible. If it can't it will still have 0 bytes of space
reported and so be excluded from the allocation.
Contributed by Steve Loughran
2023-02-22 11:48:12 +00:00
02fd87a4d8
HADOOP-18627. Add stronger wording in 'secure mode' introduction ( #5406 )
...
Make it more clear that when deploying Hadoop 'secure mode' is generally not optional.
Contributed by Arnout Engelen
2023-02-17 16:30:41 +00:00
7e19bc31b6
HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers ( #4215 )
2023-02-16 18:13:25 +00:00
c3706597a3
YARN-11349. [Federation] Router Support DelegationToken With SQL. ( #5244 )
2023-02-15 14:38:41 -08:00
f4f2793f3b
HADOOP-18351. Reduce excess logging of errors during S3A prefetching reads ( #5274 )
...
Contributed by Ankit Saurabh
2023-02-15 18:28:42 +00:00
d56977e909
HADOOP-18470. More in the 3.3.5 index.html about security ( #5383 )
...
Expands on the comments in cluster config to tell people
they shouldn't be running a cluster without a private VLAN
in cloud, that Knox is good here, and unsecured clusters
without a VLAN are just computation-as-a-service to crypto miners
Contributed by Steve Loughran
2023-02-14 17:22:59 +00:00
021fcc6c5e
HADOOP-18628. IPC Server Connection should log host name before returning VersionMismatch error ( #5385 )
...
Contributed by Viraj Jasani
2023-02-14 11:48:48 +00:00
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future ( #5315 )
2023-02-14 03:24:06 +08:00
26fba8701c
HDFS-18324. Fix race condition in closing IPC connections. ( #5371 )
2023-02-10 17:51:03 +00:00
113a9e40cb
HADOOP-18625. Fix method name of RPC.Builder#setnumReaders ( #5301 )
...
Changes method name of RPC.Builder#setnumReaders to setNumReaders()
The original method is still there, just marked deprecated.
It is the one which should be used when working with older branches.
Contributed by Haiyang Hu
2023-02-09 13:28:34 +00:00
4fcceff535
HADOOP-18620 Avoid using grizzly-http-* APIs ( #5356 )
2023-02-09 10:45:07 +08:00
8714403dc7
HADOOP-18621. Resource leak in CryptoOutputStream.close() ( #5347 )
...
When closing we need to wrap the flush() in a try .. finally, otherwise
when flush throws it will stop completion of the remainder of the
close activities and in particular the close of the underlying wrapped
stream object resulting in a resource leak.
Contributed by Colm Dougan
2023-02-07 12:01:57 +00:00
5f5157ac53
HADOOP-18612. Avoid mixing canonical and non-canonical when performing comparisons ( #5339 )
...
Contributed by Steve Vaughan Jr
2023-02-06 18:28:29 +00:00
aed6fcee5b
HADOOP-18576. Java 11 JavaDoc fails due to missing package comments ( #5344 )
...
Add JavaDoc comments to package-info.java to avoid errors resulting from the use of Hadoop annotations.
Contributed by Steve Vaughan Jr
2023-02-06 18:17:57 +00:00
WangYuanben
Steve Loughran
slfan1989
Gautham B A
Tak Lon (Stephen) Wu
zhangshuyan
Viraj Jasani
Szilard Nemeth
Pralabh Kumar
Sebastian Baunsgaard
cxzl25
Doroszlai, Attila
LiuGuH
Christos Bisias
Nikita Eshkeev
rdingankar
Melissa You
mjwiq
Andras Katona
Ayush Saxena
Yubi Lee
Simbarashe Dzinamarira
hchaverr
Arnout Engelen
Bryan Beaudreault
Ankit Saurabh
Owen O'Malley
huhaiyang
gardenia
Steve Vaughan