Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
24,312 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

968 Commits

Author SHA1 Message Date
Takanobu Asanuma 30f06e0c74
HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client. (#2394) 2020-10-20 19:08:59 +09:00
Ayush Saxena 1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. (#2342). Contributed by Ayush Saxena. 2020-10-17 12:01:18 +05:30
Ayush Saxena cc57eebe45
HDFS-15633. Avoid redundant RPC calls for getDiskStatus. (#2386). Contributed by Ayush Saxena. 2020-10-16 10:36:24 +05:30
Konstantin V Shvachko b3786d6c3c HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 2020-10-12 17:26:49 -07:00
bshashikant 16aea11c94
HDFS-15611. Add list Snapshot command in WebHDFS. (#2355) 2020-10-07 10:34:32 +05:30
Doroszlai, Attila 1cfe5916e2 HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-10-06 14:58:42 -07:00
Siyao Meng 43b0c0b054
HDFS-15607. Create trash dir when allowing snapshottable dir (#2352) 2020-10-05 05:02:00 -07:00
Vinayakumar B 82b86e3754
HDFS-15098. Add SM4 encryption method for HDFS. Contributed by liusheng 2020-09-27 19:27:13 +05:30
Uma Maheswara Rao G 899dea2a21
HDFS-15598: ViewHDFS#canonicalizeUri should not be restricted to DFS only API. (#2339). Contributed by Uma Maheswara Rao G. 
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
 2020-09-25 21:21:01 -07:00
Uma Maheswara Rao G 3ccc962b99
HDFS-15596: ViewHDFS#create(f, permission, cflags, bufferSize, replication, blockSize, progress, checksumOpt) should not be restricted to DFS only. (#2333). Contributed by Uma Maheswara Rao G. 
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
 2020-09-24 07:07:48 -07:00
huangtianhua ff59fbb8b0
HDFS-15025. Applying NVDIMM storage media to HDFS (#2189) 
* HDFS-15025. Applying NVDIMM storage media to HDFS
Co-authored-by: YaYun-Wang <yywangyayun@163.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>
 2020-09-24 14:27:04 +05:30
Uma Maheswara Rao G 7bba4c609c
HDFS-15585: ViewDFS#getDelegationToken should not throw UnsupportedOperationException. (#2312). Contributed by Uma Maheswara Rao G. 2020-09-18 15:18:10 +05:30
Takanobu Asanuma 5c5b2ed7c7 HDFS-15576. Erasure Coding: Add rs and rs-legacy codec test for addPolicies. Contributed by Fei Hui. 2020-09-16 01:36:46 +09:00
Siyao Meng 6d3312de47
HDFS-15539. When disallowing snapshot on a dir, throw exception if its trash root is not empty (#2258) 2020-09-14 13:31:34 -07:00
imbajin 89428f142f
HDFS-15551. Tiny Improve for DeadNode detector (#2265) 
Contributed by imbajin.

Reviewed-by: leosunli <lisheng.sun08@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2020-09-11 15:44:03 +08:00
1996fanrui 56ebabd426
HADOOP-17222. Create socket address leveraging URI cache (#2241) 
Contributed by fanrui.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2020-09-10 22:30:52 -07:00
Siyao Meng 43c52d6495
HDFS-15563. Incorrect getTrashRoot return value when a non-snapshottable dir prefix matches the path of a snapshottable dir (#2295) 
Change-Id: Iefcbc14ca939b61f0c1befc9a94d686be1684ddc
 2020-09-10 15:55:23 +05:30
He Xiaoqiao 1d6d0d8207 HDFS-14694. Call recoverLease on DFSOutputStream close exception. Contributed by Lisheng Sun. 
Co-authored-by: Chen Zhang <chzhang1987@gmail.com>

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2020-09-09 20:09:13 +08:00
Uma Maheswara Rao G ac7d4623ae
HDFS-15558: ViewDistributedFileSystem#recoverLease should call super.recoverLease when there are no mounts configured (#2275) Contributed by Uma Maheswara Rao G. 2020-09-07 11:36:13 -07:00
Uma Maheswara Rao G dd013f2fdf
HDFS-15533: Provide DFS API compatible class, but use ViewFileSystemOverloadScheme inside. (#2229). Contributed by Uma Maheswara Rao G. 2020-08-19 09:30:41 -07:00
Vivek Ratnavel Subramanian cb50e3fcf7
HDFS-15496. Add UI for deleted snapshots (#2212) 2020-08-13 22:36:15 +05:30
Siyao Meng 3fd3aeb621
HDFS-15492. Make trash root inside each snapshottable directory (#2176) 2020-08-11 08:52:16 -07:00
Akira Ajisaka 32895f4f7e
HDFS-15507. [JDK 11] Fix javadoc errors in hadoop-hdfs-client module. Contributed by Xieming Li. 2020-08-11 13:49:56 +09:00
Takanobu Asanuma 1d5ccc790b
HDFS-15512. Remove smallBufferSize in DFSClient. (#2191) 2020-08-06 22:47:12 +05:30
Akira Ajisaka c40cbc57fa
HADOOP-17091. [JDK11] Fix Javadoc errors (#2098) 2020-08-03 10:46:51 +09:00
bshashikant d8a2df25ad
HDFS-15498. Show snapshots deletion status in snapList cmd. (#2181) 2020-08-02 21:59:56 +05:30
bshashikant 68287371cc
HDFS-15488. Add a command to list all snapshots for a snaphottable root with snapshot Ids. (#2166) 2020-07-29 21:33:25 +05:30
Ayush Saxena 6bcb24d269 HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein. 2020-07-18 14:33:43 +05:30
Steve Loughran b9fa5e0182
HDFS-13934. Multipart uploaders to be created through FileSystem/FileContext. 
Contributed by Steve Loughran.

Change-Id: Iebd34140c1a0aa71f44a3f4d0fee85f6bdf123a3
 2020-07-13 13:30:02 +01:00
He Xiaoqiao 4f26454a7d HDFS-15425. Review Logging of DFSClient. Contributed by Hongbing Wang. 2020-07-08 00:02:39 +08:00
Yiqun Lin 9cbd76cc77 HDFS-15346. FedBalance tool implementation. Contributed by Jinglun. 2020-06-18 13:33:25 +08:00
Eric Yang 81d8a887b0
SPNEGO TLS verification 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2020-06-15 10:56:16 +09:00
Vinayakumar B e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes (#2026) 2020-06-12 23:16:33 +05:30
Uma Maheswara Rao G 93b121a971
HADOOP-17060. Clarify listStatus and getFileStatus behaviors inconsistent in the case of ViewFs implementation for isDirectory. Contributed by Uma Maheswara Rao G. 2020-06-10 15:00:02 -07:00
Ayush Saxena b735a77717 HDFS-15398. EC: hdfs client hangs due to exception during addBlock. Contributed by Hongbing Wang. 2020-06-10 12:06:16 +05:30
Ayush Saxena f4901d0778 HDFS-15355. Make the default block storage policy ID configurable. Contributed by Yang Yun. 2020-05-24 17:59:16 +05:30
Ayush Saxena e0ae232f66 HDFS-15093. RENAME.TO_TRASH is ignored When RENAME.OVERWRITE is specified. Contributed by Ayush Saxena. 2020-05-23 18:38:32 +05:30
leosunli be374faf42
HDFS-13639. SlotReleaser is not fast enough (#1885) 2020-05-21 13:21:17 -07:00
Wei-Chiu Chuang 2abcf7762a HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016) 2020-05-18 09:23:09 -07:00
Wei-Chiu Chuang 4525292d41 Revert "HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016)" 
This reverts commit 86e6aa8eec.
 2020-05-18 09:22:05 -07:00
pustota2009 86e6aa8eec
HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016) 
Added parameter dfs.client.short.circuit.num improving HDFS-client's massive reading performance by create few instances ShortCircuit caches instead of one. It helps avoid locks and lets CPU do job.
 2020-05-18 07:04:04 -07:00
Surendra Singh Lilhore 1958cb7c2b YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng 2020-05-14 15:09:34 +05:30
Ayush Saxena 0918433b4d YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng. 2020-05-14 00:36:20 +05:30
Akira Ajisaka 8ffc356b1e
Revert "SPNEGO TLS verification" 
This reverts commit ba66f3b454.
 2020-05-13 17:14:14 +09:00
S O'Donnell 29dddb8a14 HDFS-15255. Consider StorageType when DatanodeManager#sortLocatedBlock(). Contributed by Lisheng Sun. 2020-05-12 15:07:51 +01:00
Takanobu Asanuma 928b81a533
HDFS-15350. Set dfs.client.failover.random.order to true as default. (#2008) 2020-05-12 09:04:03 -05:00
Ayush Saxena 4c53fb9ce1 HDFS-15338. listOpenFiles() should throw InvalidPathException in case of invalid paths. Contributed by Jinglun. 2020-05-11 16:48:34 +05:30
Ayush Saxena aab9e0b16e HDFS-15250. Setting `dfs.client.use.datanode.hostname` to true can crash the system because of unhandled UnresolvedAddressException. Contributed by Ctest. 2020-05-10 11:43:30 +05:30
Ayush Saxena 7fddf4855e HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun. 2020-05-06 16:55:04 +05:30
Eric Yang ba66f3b454
SPNEGO TLS verification 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2020-05-02 21:50:13 +09:00