65bf544118
HADOOP-16954. Add -S option in "Count" command to show only Snapshot Counts. Contributed by hemanthboyina.
...
(cherry picked from commit b89d875f7b
2021-05-04 17:44:34 +01:00
e944cc0338
HADOOP-16742. NullPointerException in S3A MultiObjectDeleteSupport
...
Contributed by Tor Arvid Lund.
Change-Id: Iadfe9b2f355cf373031075bfbe681705a2c65bdc
2021-05-04 16:15:13 +01:00
00ba886a22
HADOOP-17675. LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException ( #2965 )
...
(cherry picked from commit 0d78d73973
2021-05-04 11:36:15 +01:00
00c3f3c42f
HDFS-15865. Interrupt DataStreamer thread if no ack ( #2728 )
...
(cherry picked from commit bd3da73a0f
2021-05-04 11:11:36 +01:00
98aa4fc32c
HADOOP-17657: implement StreamCapabilities in SequenceFile.Writer and fall back to flush, if hflush is not supported ( #2949 )
...
Co-authored-by: Kishen Das <kishen@cloudera.com>
Reviewed-by: Steve Loughran <stevel@apache.org>
(cherry picked from commit e571025f5b
2021-05-04 16:35:00 +08:00
658bb49e66
HADOOP-17676. Restrict imports from org.apache.curator.shaded ( #2969 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 62bcc79a2c
2021-05-04 01:11:30 +09:00
25244c4eec
HDFS-15952. TestRouterRpcMultiDestination#testProxyGetTransactionID and testProxyVersionRequest are flaky. ( #2970 )
...
(cherry picked from commit 7c561e2a02
2021-05-04 00:26:34 +09:00
43fac739bb
HDFS-15810. RBF: RBFMetrics's TotalCapacity out of bounds ( #2910 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e525ab81c
2021-05-02 19:19:55 +09:00
670205c541
HADOOP-17653. Do not use guava's Files.createTempDir(). ( #2945 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit f1e1809029
2021-05-02 11:12:37 +09:00
9aa6106689
HDFS-15652. Make block size from NNThroughputBenchmark configurable ( #2416 )
...
(cherry picked from commit 8a6d5b9151
2021-04-29 12:10:04 -07:00
da2ebfa8a9
YARN-10707. Support custom resources in ResourceUtilization, and update Node GPU Utilization to use. Contributed by Qi Zhu
...
(cherry picked from commit 803ac4b1a0
2021-04-29 17:52:03 +00:00
0e9042fed5
HDFS-15991. Add location into datanode info for NameNodeMXBean ( #2933 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit b968fa0957
2021-04-28 18:52:05 +09:00
842e05ba53
HDFS-15698. Fix the typo of dfshealth.html after HDFS-15358 ( #2495 )
...
(cherry picked from commit 68442b48c8
2021-04-28 18:51:50 +09:00
f809b863fe
HDFS-15358. RBF: Unify router datanode UI with namenode datanode UI. Contributed by Ayush Saxena.
...
(cherry picked from commit 6e416a83d1
2021-04-28 18:51:43 +09:00
3c7433f2a1
HDFS-15247. RBF: Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.
...
(cherry picked from commit 8d49229c37
2021-04-28 18:51:37 +09:00
187b4a63a4
HDFS-15378. TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk. Contributed by hemanthboyina.
...
(cherry picked from commit 8db38c98a6
2021-04-27 23:18:12 +09:00
f45365f201
HDFS-15975. Use LongAdder instead of AtomicLong for branch-3.3 ( #2940 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-04-27 22:39:25 +09:00
6649e5888b
HADOOP-17536. ABFS: Supporting customer provided encryption key ( #2707 )
...
Contributed by bilahari t h
Change-Id: I86216e755b81e9d14f5e87844d9fd58e8940560c
2021-04-27 13:16:33 +01:00
99e4e9bad6
HADOOP-17661. mvn versions:set fails to parse pom.xml. ( #2956 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Change-Id: I7de255cb9a167e3a5e25ae339b050e82cb4b7c30
(cherry picked from commit 64264d726d
2021-04-27 10:43:18 +09:00
df29aaaf80
YARN-10752. Shaded guava not found when compiling with profile hbase2.0. ( #2957 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Change-Id: Ic9943d18d4086b075ceed7924dcfa566653a16cb
2021-04-27 00:41:42 +09:00
f6efb58b07
HDFS-15621. Datanode DirectoryScanner uses excessive memory ( #2849 ). Contributed by Stephen O'Donnell
...
(cherry picked from commit 605ed85c29
2021-04-26 11:23:56 +01:00
1425512032
Revert "HADOOP-17661. mvn versions:set fails to parse pom.xml. ( #2952 )"
...
This reverts commit d525e1e33d
2021-04-26 16:42:44 +08:00
d525e1e33d
HADOOP-17661. mvn versions:set fails to parse pom.xml. ( #2952 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 66dda86fec
2021-04-26 14:07:35 +08:00
a5f038b3f6
HDFS-15967. Improve the log for Short Circuit Local Reads. Contributed by Bhavik Patel.
...
(cherry picked from commit 01bad0e92a
2021-04-26 13:31:51 +09:00
99dfd3b2d0
HDFS-15566. NN restart fails after RollingUpgrade from 3.1.3/3.2.1 to 3.3.0. Contributed by Brahma Reddy Battula.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 2621d3f15b
2021-04-26 11:40:16 +08:00
8308aab658
HADOOP-17112. S3A committers can't handle whitespace in paths. ( #2953 )
...
Contributed by Krzysztof Adamski.
Change-Id: I2746aabcfeb0fbb138a80b02c4d5bbf2a8cf75da
2021-04-25 18:43:25 +01:00
fb71e6c91e
HADOOP-17597. Optionally downgrade on S3A Syncable calls ( #2801 )
...
Followup to HADOOP-13327, which changed S3A output stream hsync/hflush calls
to raise an exception.
Adds a new option fs.s3a.downgrade.syncable.exceptions
When true, calls to Syncable hsync/hflush on S3A output streams will
log once at warn (for entire process life, not just the stream), then
increment IOStats with the relevant operation counter
With the downgrade option false (default)
* IOStats are incremented
* The UnsupportedOperationException current raised includes a link to the
JIRA.
Contributed by Steve Loughran.
Change-Id: I967e077eda1d1a1a3795b4d22e003fe7997b6679
2021-04-24 18:32:39 +01:00
1b2bc77923
HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0 ( #2593 )
...
Also fixes HADOOP-16995. ITestS3AConfiguration proxy tests failures when bucket probes == 0
The improvement should include the fix, because the test would fail by default otherwise.
Change-Id: I9a7e4b5e6d4391ebba096c15e84461c038a2ec59
2021-04-24 18:28:21 +01:00
33f9ceb3cc
HADOOP-17136. ITestS3ADirectoryPerformance.testListOperations failing ( #2153 )
...
A regression caused by HADOOP-17022: the reduction in LIST calls broken an assertion.
Contributed by Mukund Thakur
Change-Id: Ib9725165906931634567fd1f62a81e3a6ea5620c
2021-04-24 18:28:14 +01:00
389d3034c6
HADOOP-17471. ABFS to collect IOStatistics ( #2731 ) ( #2950 )
...
The ABFS Filesystem and its input and output streams now implement
the IOStatisticSource interface and provide IOStatistics on
their interactions with Azure Storage.
This includes the min/max/mean durations of all REST API calls.
Contributed by Mehakmeet Singh <mehakmeet.singh@cloudera.com>
2021-04-24 17:59:26 +01:00
77fddcfcb1
HADOOP-17535. ABFS: ITestAzureBlobFileSystemCheckAccess test failure if no oauth key. ( #2920 )
...
Contributed by Steve Loughran.
Change-Id: I165f5ed3a8486404403827b5c0338cf7f80c2bb1
2021-04-24 17:24:15 +01:00
003deeeecf
YARN-10479. Can't remove all node labels after add node label without
...
nodemanager port, broken by YARN-10647. Contributed by D M Murali Krishna Reddy
(cherry picked from commit 6857a05d6a
2021-04-23 22:14:57 +00:00
3b1cde2f42
YARN-10743. Add a policy for not aggregating for containers which are killed because exceeding container log size limit. Contributed by Qi Zhu.
...
(cherry picked from commit 1cbe35946b
2021-04-23 21:38:02 +00:00
930f384e30
MAPREDUCE-7302. Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail. Contributed by Peter Bacsko. Reviewed by Akira Ajisaka.
...
(cherry picked from commit da93cd962c
2021-04-23 18:42:50 +00:00
b743d56eb4
HADOOP-17620. DistCp: Use Iterator for listing target directory as well. ( #2861 ). Contributed by Ayush Saxena.
...
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>
2021-04-23 22:49:28 +05:30
842714d99f
HADOOP-17655. Upgrade Jetty to 9.4.40. ( #2947 )
...
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 0b04c9694d
2021-04-23 19:53:30 +09:00
96a1dfa313
HDFS-15989. Split TestBalancer and De-flake testMaxIterationTime() ( #2942 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-04-23 12:57:23 +09:00
2752156e3e
HDFS-15974. RBF: Unable to display the datanode UI of the router ( #2915 )
...
Co-authored-by: zhuxiangyi <zhuxiangyi@360.cn>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit dff95c5eca
2021-04-23 11:22:52 +09:00
9f2db2c9fd
HDFS-15963. Unreleased volume references cause an infinite loop. ( #2941 ) Contributed by Shuyan Zhang.
...
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-04-21 18:32:56 +08:00
29414871bd
HDFS-15879. Exclude slow nodes when choose targets for blocks ( #2928 )
2021-04-21 10:41:13 +09:00
8170a7bb60
HADOOP-16948. Support infinite lease dirs ( #1925 ). Contributed by Billie Rinaldi.
...
(cherry picked from commit c1fde4fe94
2021-04-20 14:36:54 -04:00
1960924d07
YARN-10723. Change CS nodes page in UI to support custom resource. Contributed by Qi Zhu
...
(cherry picked from commit 6cb90005a7
2021-04-20 17:46:05 +00:00
90f63cb12c
HDFS-15569. Speed up the Storage#doRecover during datanode rolling upgrade. Contributed by Hemanth Boyina.
...
(cherry picked from commit 16a20503ca
2021-04-20 16:21:57 +01:00
f30a0debae
HADOOP-17641. ITestWasbUriAndConfiguration failing. ( #2937 )
...
This moves the mock account name --which is required to never exist-- from
"mockAccount" to an account name containing a static UUID.
Contributed by Steve Loughran.
2021-04-20 15:37:18 +01:00
c93a2f6789
HDFS-14999. Avoid Potential Infinite Loop in DFSNetworkTopology. Contributed by Ayush Saxena.
...
(cherry picked from commit c84e6beada
2021-04-20 12:13:39 +01:00
dc830cf277
HDFS-15970. Print network topology on the web ( #2896 )
...
(cherry picked from commit 85a3532849
2021-04-19 21:22:47 +09:00
f7e382c62e
YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko
...
(cherry picked from commit 31477a224f
2021-04-16 20:21:01 +00:00
49f6326a9f
HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs ( #2895 ). Contributed by Viraj Jasani.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-04-16 12:41:06 +05:30
8daa26d2e5
HADOOP-17576. ABFS: Disable throttling update for auth failures ( #2761 ) ( #2885 )
...
Contributed by Sumangala Patki
(cherry picked from commit 6f640abbaf
2021-04-16 10:47:11 +05:30
4c0b833bab
HDFS-15898. Test case TestOfflineImageViewer fails ( #2779 )
...
(cherry picked from commit 0d42104119
2021-04-16 14:07:15 +09:00
Takanobu Asanuma
Steve Loughran
Istvan Fajth
Karthik Palanisamy
kishendas
Viraj Jasani
Akira Ajisaka
lfengnan
Wei-Chiu Chuang
Hui Fei
Eric Badger
litao
Ayush Saxena
bilaharith
Stephen O'Donnell
Brahma Reddy Battula
Gabor Bota
Mukund Thakur
Mehakmeet Singh
Jim Brennan
zhuxiangyi
zhangshuyan0
billierinaldi
hemanthboyina
sumangala-patki