df4197592f
HADOOP-18029: Update CompressionCodecFactory to handle uppercase file extensions ( #3739 )
...
Co-authored-by: Desmond Sisson <sissonde@amazon.com>
2021-12-01 15:36:54 -08:00
932a78fe38
HADOOP-18023. Allow cp command to run with multi threads. ( #3721 )
2021-11-29 12:45:08 +00:00
9c887e5b82
HADOOP-18014. CallerContext should not include some characters. ( #3698 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-25 14:05:04 +09:00
99b161dec7
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson ( #3708 )
2021-11-25 10:20:42 +08:00
98fe0d0fc3
HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags ( #3633 )
...
Contributed by Steve Loughran
2021-11-24 17:33:12 +00:00
ae3ba45db5
HADOOP-17975. Fallback to simple auth does not work for a secondary DistributedFileSystem instance. ( #3579 )
2021-11-24 10:44:57 +00:00
63018dc73f
HADOOP-17998. Allow get command to run with multi threads. ( #3645 )
2021-11-22 11:37:05 +00:00
8e9ad2d7af
Revert "HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson ( #3630 )" ( #3697 )
2021-11-22 10:05:21 +08:00
91af256a5b
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson ( #3630 )
2021-11-17 22:41:06 +08:00
54a1d78e16
HADOOP-17999. No-op implementation of setWriteChecksum and setVerifyChecksum in ViewFileSystem. Contributed by Abhishek Das. ( #3639 )
2021-11-16 17:56:30 -08:00
c9f95b01ef
HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode ( #3643 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2021-11-16 11:19:14 +09:00
e14a2dcbba
HADOOP-18006. maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms ( #3648 )
...
Reviewed-by: Ahmed Hussein <ahussein@apache.org>
2021-11-15 22:57:24 +09:00
573b358fce
HADOOP-18003. Add a method appendIfAbsent for CallerContext ( #3644 )
2021-11-15 22:44:49 +09:00
60acf8434d
HDFS-16319. Add metrics doc for ReadLockLongHoldCount and WriteLockLongHoldCount ( #3653 ). Contributed by tomscut.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-11-14 20:10:24 +05:30
7fb1bb8f35
HADOOP-18005. Correct log format for LdapGroupsMapping ( #3647 ). Contributed by tomscut.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-11-12 20:12:18 +05:30
359b03c83e
HDFS-16266. Add remote port information to HDFS audit log ( #3538 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-11-04 10:16:03 +09:00
6c6d1b64d4
HADOOP-17928. Syncable: S3A to warn and downgrade ( #3585 )
...
This switches the default behavior of S3A output streams
to warning that Syncable.hsync() or hflush() have been
called; it's not considered an error unless the defaults
are overridden.
This avoids breaking applications which call the APIs,
at the risk of people trying to use S3 as a safe store
of streamed data (HBase WALs, audit logs etc).
Contributed by Steve Loughran.
2021-11-02 13:26:16 +00:00
516f36c6f1
HADOOP-17967. Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom ( #3555 )
2021-10-21 16:54:25 +09:00
414d40155c
YARN-10958. Use correct configuration for Group service init in CSMappingPlacementRule ( #3560 )
...
* YARN-10958. Initial commit
* Fix javadoc + behaviour
* Fix review comments
* fix checkstyle + blanks
* fix checkstyle + blanks
* Fix checkstyle + blanks
2021-10-20 10:48:42 +02:00
46a7117995
HADOOP-17908. Add missing RELEASENOTES and CHANGELOG to upstream. ( #3433 )
2021-10-20 13:54:46 +09:00
2194b9714e
HADOOP-17945. JsonSerialization raises EOFException reading JSON data stored on google GCS ( #3501 )
...
Contributed By: Steve Loughran
2021-10-19 15:33:37 +05:30
cb8c98fbb0
HADOOP-17953. S3A: Tests to lookup global or per-bucket configuration for encryption algorithm ( #3525 )
...
Followup to S3-CSE work of HADOOP-13887
Contributed by Mehakmeet Singh
2021-10-19 10:58:27 +01:00
cb2b7970ee
HADOOP-17123. remove guava Preconditions from Hadoop-common-project modules ( #3543 )
2021-10-14 19:04:00 +09:00
1151edf12e
HADOOP-17956. Replace all default Charset usage with UTF-8 ( #3529 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-10-14 13:07:24 +09:00
97c0f96879
HADOOP-16532. Fix TestViewFsTrash to use the correct homeDir. Contributed by Xing Lin. ( #3514 )
2021-10-13 13:44:00 -07:00
e103c83765
HADOOP-17952. Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-common-project modules ( #3503 )
...
Reviewed-by: Ahmed Hussein <ahussein@apache.org>
2021-10-07 11:23:35 +09:00
8071dbb9c6
HADOOP-17950. Provide replacement for deprecated APIs of commons-io IOUtils ( #3515 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-10-07 10:58:29 +09:00
c36f9402dc
HADOOP-17930. implement non-guava Precondition checkState ( #3522 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-10-07 10:55:00 +09:00
783e4805e7
HADOOP-17947. Additional element types for VisibleForTesting (ADDENDUM) ( #3521 )
2021-10-06 02:17:05 +09:00
5b1d594005
HADOOP-17947. Provide alternative to Guava VisibleForTesting ( #3505 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-10-05 09:58:54 +09:00
68c2accc20
HDFS-16247. RBF: Fix the ProcessingAvgTime and ProxyAvgTime code comments and document metrics describe ms unit ( #3511 )
2021-10-04 23:52:26 +08:00
5f0452602f
HDFS-16222. Fix ViewDFS with mount points for HDFS only API. ( #3422 ). Contributed by Ayush Saxena.
...
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>
2021-10-03 10:02:35 +05:30
bf9106c812
HDFS-16242. JournalMetrics should add JournalId MetricTag. ( #3494 )
...
Contributed by Max Xie
2021-10-01 09:35:14 +02:00
0c498f21de
HADOOP-17929. implement non-guava Precondition checkArgument ( #3473 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
2021-10-01 15:17:10 +08:00
211db3fe08
HADOOP-17938. Print lockWarningThreshold in InstrumentedLock#logWarni… ( #3485 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-10-01 10:21:14 +08:00
7097e5b793
HADOOP-17905. Modify Text.ensureCapacity() to efficiently max out the… ( #3423 )
2021-09-29 17:25:29 -07:00
b7c2864613
HADOOP-17198. Support S3 Access Points ( #3260 )
...
Add support for S3 Access Points. This provides extra security as it
ensures applications are not working with buckets belong to third parties.
To bind a bucket to an access point, set the access point (ap) ARN,
which must be done for each specific bucket, using the pattern
fs.s3a.bucket.$BUCKET.accesspoint.arn = ARN
* The global/bucket option `fs.s3a.accesspoint.required` to
mandate that buckets must declare their access point.
* This is not compatible with S3Guard.
Consult the documentation for further details.
Contributed by Bogdan Stolojan
2021-09-29 10:54:17 +01:00
7ed949600d
HADOOP-17936. Fix test failure after reverting HADOOP-16878 ( #3482 )
2021-09-27 13:56:22 -07:00
ae6cd4cea5
HADOOP-17931. Fix typos in usage message in winutils.exe ( #3490 )
2021-09-27 13:41:55 -07:00
1e8c25d528
Revert "Fix winutils typos ( #3484 )" ( #3489 )
...
Reverting to include JIRA number HADOOP-17931.
2021-09-27 09:40:47 -07:00
0f3734a99e
Fix winutils typos ( #3484 )
2021-09-27 09:37:45 -07:00
9d44f503db
HADOOP-17910. [JDK 17] TestNetUtils fails ( #3481 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-09-27 10:12:47 +09:00
1397cf2496
Revert "HADOOP-16878. FileUtil.copy() to throw IOException if the source and destination are the same"
...
This reverts commit 59f01a548e
2021-09-25 21:32:43 -07:00
4c516536be
HDFS-16236. Example command for daemonlog is not correct ( #3476 )
2021-09-25 18:32:52 +08:00
138add2cb2
HADOOP-17868. Add more tests for BuiltInGzipCompressor ( #3336 )
2021-09-22 07:59:28 -07:00
acffe203b8
HADOOP-17195. ABFS: OutOfMemory error while uploading huge files ( #3446 )
...
Addresses the problem of processes running out of memory when
there are many ABFS output streams queuing data to upload,
especially when the network upload bandwidth is less than the rate
data is generated.
ABFS Output streams now buffer their blocks of data to
"disk", "bytebuffer" or "array", as set in
"fs.azure.data.blocks.buffer"
When buffering via disk, the location for temporary storage
is set in "fs.azure.buffer.dir"
For safe scaling: use "disk" (default); for performance, when
confident that upload bandwidth will never be a bottleneck,
experiment with the memory options.
The number of blocks a single stream can have queued for uploading
is set in "fs.azure.block.upload.active.blocks".
The default value is 20.
Contributed by Mehakmeet Singh.
2021-09-21 12:48:06 +01:00
ae2c5ccfcf
HADOOP-17893. Improve PrometheusSink for Namenode TopMetrics ( #3426 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-09-21 10:43:50 +09:00
4df4389325
YARN-10911. AbstractCSQueue: Create a separate class for usernames and weights that are travelling in a Map. Contributed by Szilard Nemeth
2021-09-20 16:47:46 +02:00
f93e8fbf2d
HDFS-16129. Fixing the signature secret file misusage in HttpFS. Contributed by Tamas Domok
...
* HDFS-16129. Fixing the signature secret file misusage in HttpFS.
The signature secret file was not used in HttpFs.
- if the configuration did not contain the deprecated
httpfs.authentication.signature.secret.file option then it
used the random secret provider
- if both option (httpfs. and hadoop.http.) was set then
the HttpFSAuthenticationFilter could not read the file
because the file path was not substituted properly
!NOTE! behavioral change: the deprecated httpfs. configuration
values are overwritten with the hadoop.http. values.
The commit also contains a follow up change to the YARN-10814,
empty secret files will result in a random secret provider.
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
2021-09-20 14:29:50 +02:00
607c20c612
HADOOP-17919. Fix command line example in Hadoop Cluster Setup documentation. ( #3453 )
2021-09-17 22:24:44 +09:00
Desmond Sisson
smarthan
Takanobu Asanuma
huhaiyang
Steve Loughran
Istvan Fajth
Hui Fei
Abhishek Das
litao
Viraj Jasani
Szilard Nemeth
Masatake Iwasaki
Mehakmeet Singh
Ahmed Hussein
Xing Lin
Ayush Saxena
Neil
pbacsko
Petre Bogdan Stolojan
Chao Sun
Gautham B A
Inigo Goiri
Renukaprasad C
Liang-Chi Hsieh
Tamas Domok
Rintaro Ikeda