Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
26,116 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

5839 Commits

Author SHA1 Message Date
slfan1989 7bfff63774
HADOOP-18289. Remove WhiteBox in hadoop-kms module. (#4433) 
Co-authored-by: slfan1989 <louj1988@@>
 2022-06-17 09:13:16 +08:00
Renukaprasad C 7bd4ac3ce0
HDFS-16563. Namenode WebUI prints sensitive information on Token expiry (#4241) 
Contributed by Renukaprasad C
 2022-06-03 18:37:27 +01:00
Masatake Iwasaki 3228142f53 Make upstream aware of 2.10.2 release. 
(cherry picked from commit 3dcb6367ed)

Conflicts:
	hadoop-project-dist/pom.xml
 2022-06-01 00:53:01 +09:00
Ashutosh Gupta 21fa693d38
HADOOP-18238. Fix reentrancy check in SFTPFileSystem.close() (#4330) 
Contributed by Ashutosh Gupta
 2022-05-30 17:34:06 +01:00
HanleyYang 835f39cefc
HDFS-15878. RBF: Fix TestRouterWebHDFSContractCreate#testSyncable. (#4340). Contributed by Hanley Yang. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-05-28 09:56:07 +05:30
9uapaw 0773fae392 HADOOP-18120. Hadoop auth does not handle HTTP Headers in a case-insensitive way. Contributed by Janos Makai. 2022-05-20 10:43:19 +02:00
Ritesh H Shukla 78008bc0ee
HADOOP-18245 Extend KMS related exceptions that get mapped to ConnectException (#4329) 2022-05-20 04:20:24 +08:00
slfan1989 f6fa5bd1aa
HADOOP-18229. Fix Hadoop-Common JavaDoc Errors (#4292) 
Contributed by slfan1989
 2022-05-18 12:12:04 +01:00
Steve Loughran 6f261ed4a2
HADOOP-18198. Release 3.3.3: release notes and jdiff files. 
* Add the changelog and release notes
* add all jdiff XML files
* update the project pom with the new stable version

Change-Id: Iaea846c3e451bbd446b45de146845a48953d580d
 2022-05-17 19:00:54 +01:00
Lei Yang 6a95c3a039
HADOOP-18193:Support nested mount points in INodeTree 
Fixes #4181

Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
 2022-05-11 17:01:21 -07:00
hchaverr 99a83fd4bd
HADOOP-18222. Prevent DelegationTokenSecretManagerMetrics from registering multiple times 
Fixes #4266

Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
 2022-05-10 13:58:39 -07:00
Ashutosh Gupta 40a8b9a6a5
HADOOP-17479. Fix the examples of hadoop config prefix (#4197) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-08 08:09:24 +09:00
Ashutosh Gupta ae47846a5b
HADOOP-16515. Update the link to compatibility guide (#4226) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-08 07:38:32 +09:00
PJ Fanning 63187083cc
HADOOP-15983. Use jersey-json that is built to use jackson2 (#3988) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-04-28 14:18:19 +09:00
liever18 2632d49282
HADOOP-18216. io.file.buffer.size must be positive (#4220) 
Documents that the buffer size must be > 0

Contributed by Jingxuan Fu.
 2022-04-26 19:53:11 +01:00
hchaverri d60262fe00
HADOOP-18167. Add metrics to track delegation token secret manager op… (#4092) 
* HADOOP-18167. Add metrics to track delegation token secret manager operations
 2022-04-26 16:20:11 +00:00
Steve Loughran 1b4dba99b5
HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1) 
This defines standard option and values for the
openFile() builder API for opening a file:

fs.option.openfile.read.policy
 A list of the desired read policy, in preferred order.
 standard values are
 adaptive, default, random, sequential, vector, whole-file

fs.option.openfile.length
 How long the file is.

fs.option.openfile.split.start
 start of a task's split

fs.option.openfile.split.end
 end of a task's split

These can be used by filesystem connectors to optimize their
reading of the source file, including but not limited to
* skipping existence/length probes when opening a file
* choosing a policy for prefetching/caching data

The hadoop shell commands which read files all declare "whole-file"
and "sequential", as appropriate.

Contributed by Steve Loughran.

Change-Id: Ia290f79ea7973ce8713d4f90f1315b24d7a23da1
 2022-04-24 17:33:04 +01:00
Ashutosh Gupta f84b88dd6b
HADOOP-17564. Fix typo in UnixShellGuide.html (#4195) 
contributed by Ashutosh Gupta
 2022-04-22 17:59:41 +01:00
Renukaprasad C 4ff8a5dc73
HDFS-16526. Addendum Add metrics for slow DataNode (#4191) 2022-04-20 18:57:43 +05:30
Xing Lin 98b9c435f2
HADOOP-18172: Changed scope for isRootInternalDir/getRootFallbackLink for InodeTree (#4106) 
* HADOOP-18172: Change scope of InodeTree and its member methods to make them accessible from outside package.

Co-authored-by: Xing Lin <xinglin@linkedin.com>
 2022-04-19 22:47:02 -07:00
Renukaprasad C f14f305051
HDFS-16526. Add metrics for slow DataNode (#4162) 2022-04-15 21:37:05 +05:30
caozhiqiang cee8c62498
HDFS-16456. EC: Decommission a rack with only on dn will fail when the rack number is equal with replication (#4126) 2022-04-14 18:42:39 +09:00
GuoPhilipse 5de78ceb0e
HDFS-16516. Fix Fsshell wrong params (#4090). Contributed by GuoPhilipse. 2022-04-11 15:54:00 +08:00
Viraj Jasani b69ede7154
HADOOP-18191. Log retry count while handling exceptions in RetryInvocationHandler (#4133) 2022-04-08 21:31:54 +09:00
Viraj Jasani f70935522b
HADOOP-18188. Support touch command for directory (#4135) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-04-07 17:29:45 +09:00
litao 34b3275bf4
HDFS-16477. [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS (#4009). Contributed by tomscut. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-04-03 01:36:03 +05:30
Xing Lin 15a5ea2c95
HADOOP-18169. getDelegationTokens in ViewFs should also fetch the token from fallback FS (#4094) 
HADOOP-18169. getDelegationTokens in ViewFs should also fetch the token from the fallback FS
 2022-03-31 21:59:09 +00:00
litao ac50657c37
HDFS-16413. Reconfig dfs usage parameters for datanode (#3863) 2022-03-31 00:51:17 +09:00
zhongjingxiong 08e6d0ce60
HADOOP-18145. Fileutil's unzip method causes unzipped files to lose their original permissions (#4036) 
Contributed by jingxiong zhong
 2022-03-30 12:42:50 +01:00
Owen O'Malley eb16421386 HDFS-16517 Distance metric is wrong for non-DN machines in 2.10. Fixed in HADOOP-16161, but 
this test case adds value to ensure the two getWeight methods stay in sync.

Fixes #4091

Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
 2022-03-28 12:52:46 -07:00
Masatake Iwasaki 10876333ac Make upstream aware of 3.2.3 release. 2022-03-28 08:02:10 +00:00
PJ Fanning 61e809b245
HADOOP-13386. Upgrade Avro to 1.9.2 (#3990) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-03-26 20:31:16 +09:00
Steve Loughran 708a0ce21b
HADOOP-13704. Optimized S3A getContentSummary() 
Optimize the scan for s3 by performing a deep tree listing,
inferring directory counts from the paths returned.

Contributed by Ahmar Suhail.

Change-Id: I26ffa8c6f65fd11c68a88d6e2243b0eac6ffd024
 2022-03-22 13:21:12 +00:00
Owen O'Malley 1d5650c4d0
HDFS-13248: Namenode needs to use the actual client IP when going through the 
RBF proxy. There is a new configuration knob dfs.namenode.ip-proxy-users that configures
the list of users than can set their client ip address using the client context.

Fixes #4081
 2022-03-21 09:27:35 -07:00
Abhishek Das da9970dd69 HADOOP-18129: Change URI to String in INodeLink to reduce memory footprint of ViewFileSystem 
Fixes #3996
 2022-03-17 17:25:55 -07:00
Steve Loughran 9037f9a334
HADOOP-18162. hadoop-common support for MAPREDUCE-7341 Manifest Committer 
* New statistic names in StoreStatisticNames
  (for joint use with s3a committers)
* Improvements to IOStatistics implementation classes
* RateLimiting wrapper to guava RateLimiter
* S3A committer Tasks moved over as TaskPool and
  added support for RemoteIterator
* JsonSerialization.load() to fail fast if source does not exist

+ tests.

This commit is a prerequisite for the main MAPREDUCE-7341 Manifest Committer
patch.

Contributed by Steve Loughran

Change-Id: Ia92e2ab5083ac3d8d3d713a4d9cb3e9e0278f654
 2022-03-17 11:20:53 +00:00
Xing Lin 8b8158f02d
HADOOP-18144: getTrashRoot in ViewFileSystem should return a path in ViewFS. 
To get the new behavior, define fs.viewfs.trash.force-inside-mount-point to be true.

If the trash root for path p is in the same mount point as path p,
and one of:
* The mount point isn't at the top of the target fs.
* The resolved path of path is root (eg it is the fallback FS).
* The trash root isn't in user's target fs home directory.
get the corresponding viewFS path for the trash root and return it.
Otherwise, use <mnt>/.Trash/<user>.

Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
 2022-03-14 11:29:48 -07:00
Owen O'Malley 7b5eac27ff
HDFS-16495: RBF should prepend the client ip rather than append it. 
Fixes #4054

Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
 2022-03-14 10:21:35 -07:00
Mukund Thakur 672e380c4f
HADOOP-18112: Implement paging during multi object delete. (#4045) 
Multi object delete of size more than 1000 is not supported by S3 and 
fails with MalformedXML error. So implementing paging of requests to 
reduce the number of keys in a single request. Page size can be configured
using "fs.s3a.bulk.delete.page.size" 

 Contributed By: Mukund Thakur
 2022-03-11 13:05:45 +05:30
Gautham B A d0fa9b5775
HADOOP-18155. Refactor tests in TestFileUtil (#4053) 2022-03-10 22:02:38 +05:30
Duo Zhang db36747e83
HADOOP-17526 Use Slf4jRequestLog for HttpRequestLog (#4050) 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-03-10 10:15:09 +08:00
Viraj Jasani 66b72406bd
HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies (#4000) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2022-03-08 17:27:04 +09:00
Viraj Jasani 278568203b
HDFS-16481. Provide support to set Http and Rpc ports in MiniJournalCluster (#4028). Contributed by Viraj Jasani. 2022-03-04 22:17:48 +05:30
Chao Sun f800b65b40 Make upstream aware of 3.3.2 release 2022-03-02 19:14:50 -08:00
ted12138 902a7935e9
HADOOP-18128. Fix typo issues of outputstream.md (#4025) 2022-03-02 18:25:56 +08:00
Ayush Saxena d05655d2ad
Revert "HADOOP-18082.Add debug log when RPC#Reader gets a Call. (#3891). Contributed by JiangHua Zhu." 
Exposes a Race Condition. Which leads to test failures in YARN. (HADOOP-18143)

This reverts commit 2025243fbf.
 2022-02-28 21:44:24 +05:30
Owen O'Malley 12fa38d546
HADOOP-18139: Allow configuration of zookeeper server principal. 
Fixes #4024

Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
 2022-02-24 15:01:50 -08:00
monthonk 1f157f802d
HADOOP-17386. Change default fs.s3a.buffer.dir to be under Yarn container path on yarn applications (#3908) 
Co-authored-by: Monthon Klongklaew <monthonk@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-02-22 13:50:27 +09:00
jianghuazhu 589695c6a9
HDFS-16316.Improve DirectoryScanner: add regular file check related block. (#3861) 2022-02-22 10:15:19 +08:00
Steve Loughran cae749b076
HADOOP-18136. Verify FileUtils.unTar() handling of missing .tar files. 
Contributed by Steve Loughran

Change-Id: I73af19d2e2e41f4ba686c470726a80c3903a1950
 2022-02-21 17:08:56 +00:00
jianghuazhu 2025243fbf
HADOOP-18082.Add debug log when RPC#Reader gets a Call. (#3891). Contributed by JiangHua Zhu. 2022-02-17 01:49:45 +05:30
Chentao Yu 19d90e62fb HADOOP-18109. Ensure that default permissions of directories under internal ViewFS directories are the same as directories on target filesystems. Contributed by Chentao Yu. (3953) 2022-02-15 15:58:24 -08:00
GuoPhilipse b68964336d
HDFS-16449. Fix hadoop web site release notes and changelog not available (#3967) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-02-14 05:38:28 +09:00
daimin 0e74f1e467
Fix thread safety of EC decoding during concurrent preads (#3881) 2022-02-11 10:20:00 +08:00
Xing Lin ca8ba24051 HADOOP-18110. ViewFileSystem: Add Support for Localized Trash Root 
Fixes #3956
 2022-02-10 16:43:04 -08:00
Steve Loughran efdec92cab
HADOOP-18091. S3A auditing leaks memory through ThreadLocal references (#3930) 
Adds a new map type WeakReferenceMap, which stores weak
references to values, and a WeakReferenceThreadMap subclass
to more closely resemble a thread local type, as it is a
map of threadId to value.

Construct it with a factory method and optional callback
for notification on loss and regeneration.

 WeakReferenceThreadMap<WrappingAuditSpan> activeSpan =
      new WeakReferenceThreadMap<>(
          (k) -> getUnbondedSpan(),
          this::noteSpanReferenceLost);

This is used in ActiveAuditManagerS3A for span tracking.

Relates to
* HADOOP-17511. Add an Audit plugin point for S3A
* HADOOP-18094. Disable S3A auditing by default.

Contributed by Steve Loughran.
 2022-02-10 12:31:41 +00:00
Abhishek Das 3684c7f66a
HADOOP-18100: Change scope of inner classes in InodeTree to make them accessible outside package 
Fixes #3950

Signed-off-by: Owen O'Malley <omalley@apache.org>
 2022-02-03 16:28:04 -08:00
Ayush Saxena aeae5716cc
Revert "HADOOP-18024. SocketChannel is not closed when IOException happens in Server$Listener.doAccept (#3719)" 
This reverts commit 6ed01585eb.

Breaks TestIPC#testIOEOnListenerAccept
 2022-02-01 14:11:25 +05:30
Li MingXiang e17c96a40a
HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl. (#3900). Contributed by limingxiang. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2022-01-27 16:53:21 +08:00
Viraj Jasani 4faac58841
HADOOP-18089. Test coverage for Async profiler servlets (#3913) 
Reviewed-by: Akira Ajisaka <akiraaj@amazon.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-01-26 11:24:16 +08:00
Xing Lin 0d17b629ff
HADOOP-18093. Better exception handling for testFileStatusOnMountLink() in ViewFsBaseTest.java (#3918). Contributed by Xing Lin. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-01-25 19:40:18 +05:30
Steve Loughran 14ba19af06
HADOOP-17409. Remove s3guard from S3A module (#3534) 
Completely removes S3Guard support from the S3A codebase.

If the connector is configured to use any metastore other than
the null and local stores (i.e. DynamoDB is selected) the s3a client
will raise an exception and refuse to initialize.

This is to ensure that there is no mix of S3Guard enabled and disabled
deployments with the same configuration but different hadoop releases
-it must be turned off completely.

The "hadoop s3guard" command has been retained -but the supported
subcommands have been reduced to those which are not purely S3Guard
related: "bucket-info" and "uploads".

This is major change in terms of the number of files
changed; before cherry picking subsequent s3a patches into
older releases, this patch will probably need backporting
first.

Goodbye S3Guard, your work is done. Time to die.

Contributed by Steve Loughran.
 2022-01-17 18:08:57 +00:00
ahmarsuhail 7542677470
HADOOP-16223. Remove misleading fs.s3a.delegation.tokens.enabled prompt (#3879) 
Contributed by Ahmar Suhail
 2022-01-12 17:25:17 +00:00
Viraj Jasani 93294f0329
HADOOP-18077. ProfileOutputServlet unable to proceed due to NPE (#3875) 2022-01-12 16:20:34 +08:00
litao 39efbc6b6f
HDFS-16404. Fix typo for CachingGetSpaceUsed (#3844). Contributed by tomscut. 2022-01-09 16:41:10 +08:00
Viraj Jasani f64fda0f00
HADOOP-18055. Async Profiler endpoint for Hadoop daemons (#3824) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2022-01-06 17:56:49 +08:00
Mukund Thakur da0a6ba1ce
HADOOP-18065. ExecutorHelper.logThrowableFromAfterExecute() is too noisy. (#3860) 
Downgrading warn logs to debug in case of InterruptedException

Contributed By: Mukund Thakur
 2022-01-06 10:54:27 +05:30
jianghuazhu 7398a0f1b2
HADOOP-18063. Remove unused import AbstractJavaKeyStoreProvider in Shell class. (#3846) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-01-04 11:25:13 +09:00
jianghuazhu 43afd1753a
HDFS-16394.RPCMetrics increases the number of handlers in processing. (#3822) 2021-12-31 16:40:14 +08:00
Ashutosh Gupta caab29ec88
HDFS-14099. Unknown frame descriptor when decompressing multiple frames (#3836) 
Co-authored-by: xuzq <xuzengqiang@kuaishou.com>
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-28 21:44:38 +09:00
Viraj Jasani 04b6b9a87b
HADOOP-16908. Prune Jackson 1 from the codebase and restrict it's usage for future (#3789) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-20 16:01:34 +09:00
Dhananjay Badaya 4483607a4e
HADOOP-13500. Synchronizing iteration of Configuration properties object (#3775) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-17 16:05:46 +09:00
PHILO-HE 8e08f43e03
HDFS-16014: Fix an issue in checking native pmdk lib by 'hadoop checknative' command (#3762) 2021-12-14 14:45:12 +05:30
Wei-Chiu Chuang d7c5400fbc
HADOOP-17982. OpensslCipher initialization error should log a WARN message. (#3599) 
Change-Id: I070fc4784679b3be73aa3a11201bbae23c20ad4e
 2021-12-10 18:14:04 +09:00
Akira Ajisaka 9b9e2ef87f
HADOOP-18040. Use maven.test.failure.ignore instead of ignoreTestFailure (#3774) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-12-10 01:36:31 +09:00
Haoze Wu 6ed01585eb
HADOOP-18024. SocketChannel is not closed when IOException happens in Server$Listener.doAccept (#3719) 2021-12-08 18:48:43 +09:00
Andras Gyori 47ea0d734f
HADOOP-18021. Provide a public wrapper of Configuration#substituteVars (#3710) 
Contributed by Andras Gyori
 2021-12-03 16:44:58 +00:00
Desmond Sisson df4197592f
HADOOP-18029: Update CompressionCodecFactory to handle uppercase file extensions (#3739) 
Co-authored-by: Desmond Sisson <sissonde@amazon.com>
 2021-12-01 15:36:54 -08:00
smarthan 932a78fe38
HADOOP-18023. Allow cp command to run with multi threads. (#3721) 2021-11-29 12:45:08 +00:00
Takanobu Asanuma 9c887e5b82
HADOOP-18014. CallerContext should not include some characters. (#3698) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
 2021-11-25 14:05:04 +09:00
huhaiyang 99b161dec7
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson (#3708) 2021-11-25 10:20:42 +08:00
Steve Loughran 98fe0d0fc3
HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags (#3633) 
Contributed by Steve Loughran
 2021-11-24 17:33:12 +00:00
Istvan Fajth ae3ba45db5
HADOOP-17975. Fallback to simple auth does not work for a secondary DistributedFileSystem instance. (#3579) 2021-11-24 10:44:57 +00:00
smarthan 63018dc73f
HADOOP-17998. Allow get command to run with multi threads. (#3645) 2021-11-22 11:37:05 +00:00
Hui Fei 8e9ad2d7af
Revert "HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson (#3630)" (#3697) 2021-11-22 10:05:21 +08:00
huhaiyang 91af256a5b
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson (#3630) 2021-11-17 22:41:06 +08:00
Abhishek Das 54a1d78e16 HADOOP-17999. No-op implementation of setWriteChecksum and setVerifyChecksum in ViewFileSystem. Contributed by Abhishek Das. (#3639) 2021-11-16 17:56:30 -08:00
litao c9f95b01ef
HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode (#3643) 
Reviewed-by: Hui Fei <ferhui@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-11-16 11:19:14 +09:00
Viraj Jasani e14a2dcbba
HADOOP-18006. maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms (#3648) 
Reviewed-by: Ahmed Hussein <ahussein@apache.org>
 2021-11-15 22:57:24 +09:00
litao 573b358fce
HADOOP-18003. Add a method appendIfAbsent for CallerContext (#3644) 2021-11-15 22:44:49 +09:00
litao 60acf8434d
HDFS-16319. Add metrics doc for ReadLockLongHoldCount and WriteLockLongHoldCount (#3653). Contributed by tomscut. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-11-14 20:10:24 +05:30
litao 7fb1bb8f35
HADOOP-18005. Correct log format for LdapGroupsMapping (#3647). Contributed by tomscut. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-11-12 20:12:18 +05:30
litao 359b03c83e
HDFS-16266. Add remote port information to HDFS audit log (#3538) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-11-04 10:16:03 +09:00
Steve Loughran 6c6d1b64d4
HADOOP-17928. Syncable: S3A to warn and downgrade (#3585) 
This switches the default behavior of S3A output streams
to warning that Syncable.hsync() or hflush() have been
called; it's not considered an error unless the defaults
are overridden.

This avoids breaking applications which call the APIs,
at the risk of people trying to use S3 as a safe store
of streamed data (HBase WALs, audit logs etc).

Contributed by Steve Loughran.
 2021-11-02 13:26:16 +00:00
Viraj Jasani 516f36c6f1
HADOOP-17967. Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom (#3555) 2021-10-21 16:54:25 +09:00
Szilard Nemeth 414d40155c
YARN-10958. Use correct configuration for Group service init in CSMappingPlacementRule (#3560) 
* YARN-10958. Initial commit

* Fix javadoc + behaviour

* Fix review comments

* fix checkstyle + blanks

* fix checkstyle + blanks

* Fix checkstyle + blanks
 2021-10-20 10:48:42 +02:00
Masatake Iwasaki 46a7117995
HADOOP-17908. Add missing RELEASENOTES and CHANGELOG to upstream. (#3433) 2021-10-20 13:54:46 +09:00
Steve Loughran 2194b9714e
HADOOP-17945. JsonSerialization raises EOFException reading JSON data stored on google GCS (#3501) 
Contributed By: Steve Loughran
 2021-10-19 15:33:37 +05:30
Mehakmeet Singh cb8c98fbb0
HADOOP-17953. S3A: Tests to lookup global or per-bucket configuration for encryption algorithm (#3525) 
Followup to S3-CSE work of HADOOP-13887

Contributed by Mehakmeet Singh
 2021-10-19 10:58:27 +01:00
Ahmed Hussein cb2b7970ee
HADOOP-17123. remove guava Preconditions from Hadoop-common-project modules (#3543) 2021-10-14 19:04:00 +09:00