1df3b3de46
HDFS-16235. Fix Deadlock in LeaseRenewer for static remove method ( #3472 )
...
(cherry picked from commit 5f9321a5d4
2021-09-29 17:44:32 +08:00
cfd0265975
HDFS-14575. LeaseRenewer#daemon threads leak in DFSClient. Contributed by Renukaprasad C.
...
Co-authored-by: Tao Yang <taoyang1@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 10b79a26fe
2021-09-29 17:44:00 +08:00
4eeb4d32e5
HDFS-16198. Short circuit read leaks Slot objects when InvalidToken exception is thrown ( #3359 )
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c4c5883d8b51a4a23e37
2021-09-14 16:49:14 +08:00
1944e0d714
HDFS-16207. Remove NN logs stack trace for non-existent xattr query ( #3375 )
...
Change-Id: Ibde523b20a6b8ac92991da52583e625a018d2ee6
2021-09-09 05:28:29 +00:00
3cf2479c04
Preparing for 3.2.4 development.
2021-08-08 18:35:16 +05:30
70c5e9e780
HDFS-16127. Improper pipeline close recovery causes a permanent write failure or data loss. Contributed by Kihwal Lee.
...
(cherry picked from commit 47002719f2
2021-07-16 14:24:52 -05:00
88f3ce1127
HDFS-15650. Make the socket timeout for computing checksum of striped blocks configurable ( #2414 )
...
(cherry picked from commit 4bb25c810b
2021-07-16 11:01:03 +08:00
11034f2488
HDFS-16074. Remove an expensive debug string concatenation ( #3107 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: litao <tomleescut@gmail.com>
(cherry picked from commit 1ad674e548
2021-06-17 11:38:58 +08:00
77f61df352
HDFS-16073. Remove redundant RPC requests for getFileLinkInfo in ClientNamenodeProtocolTranslatorPB. Contributed by lei w.
2021-06-16 15:55:06 +05:30
b8fc956a4d
HDFS-16068. WebHdfsFileSystem has a possible connection leak in connection with HttpFS ( #3104 )
...
(cherry picked from commit 839fcf7682
2021-06-15 17:23:34 +09:00
52f4f9a063
HDFS-15950. Remove unused hdfs.proto import ( #2858 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ca5e13fd74
2021-04-05 11:47:40 +09:00
bc3c4a3268
HDFS-15633. Avoid redundant RPC calls for getDiskStatus. ( #2386 ). Contributed by Ayush Saxena.
2021-03-19 23:14:29 +05:30
929411acce
HDFS-15210. EC : File write hanged when DN is shutdown by admin command. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit db6252b6c3
2021-02-26 10:49:48 +08:00
d74e9453d9
[HDFS-15813] DataStreamer: keep sending heartbeat packets during flush. Contributed by Daryn Sharp and Jim Brennan
...
(cherry picked from commit 62389a5a04
2021-02-05 22:18:52 +00:00
3b2a131885
HDFS-15708. TestURLConnectionFactory fails by NoClassDefFoundError in branch-3.3 and branch-3.2 ( #2517 )
2020-12-04 10:42:46 -08:00
4af5f55623
HDFS-14395. Remove WARN Logging From Interrupts. Contributed by David Mollitor.
2020-11-10 14:44:34 +08:00
0d6958962c
HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client. ( #2394 )
...
(cherry picked from commit 30f06e0c74
2020-10-20 19:13:20 +09:00
3274fd139d
Preparing for 3.2.3 development
2020-10-16 14:52:41 +08:00
69b60ead5b
Addendum to HADOOP-16990. Update Mockserver. Contributed by Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 800b1ed1c2
2020-10-13 18:09:14 +09:00
23b6ab67f7
HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit b3786d6c3c
2020-10-12 17:42:58 -07:00
61563885f8
HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 8675a0d283bcf1f6a0c1e5953d28f7497c63ba66)
2020-10-08 23:44:33 -07:00
544d6da3ab
HDFS-15117. EC: Add getECTopologyResultForPolicies to DistributedFileSystem. Contributed by Ayush Saxena. ( #2261 )
2020-09-06 07:56:16 +05:30
3bacea2e5e
HDFS-14096. [SPS] : Add Support for Storage Policy Satisfier in ViewFs. Contributed by Ayush Saxena.
...
(cherry picked from commit 788e7473a4
2020-08-26 10:31:39 -07:00
0512b27172
HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun. (Backported)
2020-08-26 09:33:51 -07:00
42c71a5790
HADOOP-15691. Add PathCapabilities to FileSystem and FileContext.
...
Contributed by Steve Loughran.
This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.
This is intended to allow applications to determine
* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.
As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
fs. + schema + .
The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.
Note
* The remote store is not expected to be checked for the feature;
It is more a check of client API and the client's configuration/knowledge
of the state of the remote system.
* Permissions are not checked.
2020-08-19 17:15:06 +01:00
27a97e4f28
HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.
2020-07-22 18:39:49 +05:30
4a9ec4d143
HADOOP-17060. Clarify listStatus and getFileStatus behaviors inconsistent in the case of ViewFs implementation for isDirectory. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 93b121a971
2020-06-27 15:54:37 -07:00
e9cf454ac2
SPNEGO TLS verification
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 81d8a887b0
2020-06-15 11:12:40 +09:00
dfb094d581
Revert "SPNEGO TLS verification"
...
This reverts commit 9db6e96db3
2020-05-13 17:15:46 +09:00
06f43fc628
HDFS-15250. Setting `dfs.client.use.datanode.hostname` to true can crash the system because of unhandled UnresolvedAddressException. Contributed by Ctest.
2020-05-10 11:44:43 +05:30
9db6e96db3
SPNEGO TLS verification
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454
2020-05-02 21:51:41 +09:00
10fe37b704
HDFS-15191. EOF when reading legacy buffer in BlockTokenIdentifier. Contributed by Steven Rand.
2020-03-27 13:01:00 -07:00
ba6b3a3848
HDFS-14434. Ignore user.name query parameter in secure WebHDFS.
...
Contributed by KWON BYUNGCHANG
(cherry picked from commit d78854b928
2020-03-25 17:12:01 -07:00
152cbc6457
HDFS-15219. DFS Client will stuck when ResponseProcessor.run throw Error ( #1902 ). Contributed by zhengchenyu.
...
(cherry picked from commit d9c4f1129c
2020-03-25 16:52:52 -07:00
e9916052b1
HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena.
2020-03-15 20:53:45 +05:30
87532b6e3f
HDFS-15197. [SBN read] Change ObserverRetryOnActiveException log to debug. Contributed by Chen Liang.
2020-03-15 16:31:20 +05:30
9acbbd885c
HDFS-12999. When reach the end of the block group, it may not need to flush all the data packets(flushAllInternals) twice. Contributed by lufei and Fei Hui.
...
(cherry picked from commit df622cf4a3
2020-02-27 16:36:53 -08:00
610805ec72
HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. ( #1758 )
...
(cherry picked from commit 2338d25dc7
2020-02-21 13:40:11 +09:00
9a1ce410f6
HDFS-15161. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close(). Contributed by Lisheng Sun
2020-02-12 20:29:09 +05:30
f1840669cf
HDFS-14758. Make lease hard limit configurable and reduce the default.
...
Contributed by hemanthboyina.
(cherry picked from commit 9b8a78d97b
2020-02-11 14:50:10 -06:00
9813ba2ebc
[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Contributed by Chen Liang
2020-01-28 16:36:27 -08:00
1bb9667137
HDFS-15119. Allow expiration of cached locations in DFSInputStream.
...
Contributed by Ahmed Hussein.
(cherry picked from commit d10f77e3c9
2020-01-24 09:21:09 -06:00
3e17b471b6
HDFS-15077. Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout. ( #1797 )
...
(cherry picked from commit aba3f6c3e1
2020-01-08 17:05:20 +09:00
0d94e0b1a0
HDFS-15073. Replace curator-shaded guava import with the standard one ( #1782 )
...
(cherry picked from commit d8cd7098b4
2019-12-25 11:12:03 +09:00
8ba4e13dcc
HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 99af49878cc47f5750b6c0edadc8a4c2fcd7a7c6)
2019-12-11 16:34:28 -08:00
aa1c795dc9
HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit c36014165c
2019-11-06 19:37:41 +05:30
a37a4bc62f
HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches
...
Signed-off-by: stack <stack@apache.org>
2019-10-30 20:26:54 -07:00
9316ca149f
HDFS-14308. DFSStripedInputStream curStripeBuf is not freed by unbuffer() ( #1667 )
...
Reviewed-by: Aravindan Vijayan <avijayan@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 30db895b59
2019-10-25 13:10:56 -07:00
a12ab911ee
HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko.
...
(cherry picked from commit 72ae371e7a
2019-10-08 12:03:04 -07:00
a0cdebc409
HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit 382967be51
2019-10-08 00:26:27 +05:30
AngersZhuuuu
He Xiaoqiao
EungsopYoo
Ahmed Hussein
Brahma Reddy Battula
Kihwal Lee
crossfire
Wei-Chiu Chuang
Ayush Saxena
Takanobu Asanuma
Gautham B A
Surendra Singh Lilhore
Jim Brennan
Chao Sun
Giovanni Matteo Fumarola
Akira Ajisaka
Konstantin V Shvachko
Doroszlai, Attila
Uma Maheswara Rao G
Steve Loughran
Eric Yang
Chen Liang
Isa Hekmatizadeh
Masatake Iwasaki
Chandra Sanivarapu
Duo Zhang
zhaoyim