Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,519 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

229 Commits

Author SHA1 Message Date
Wei-Chiu Chuang 8af0fcf628 HDFS-15719. [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout (#2533) 
(cherry picked from commit 2b4febcf57)
(cherry picked from commit 94c126cc9e)
 2021-01-04 20:56:32 -08:00
Jim Brennan 76316c4bc0 HDFS-15711. Add Metrics to HttpFS Server. (#2521) Contributed by Ahmed Hussein and Kihwal Lee 2020-12-10 21:30:03 +00:00
Ahmed Hussein f547cd43d1 HDFS-15706. HttpFS: Log more information on request failures. (#2515) 
(cherry picked from commit 07655a7040)
(cherry picked from commit a7dbd3b6f9)
 2020-12-03 15:15:31 -08:00
He Xiaoqiao 3274fd139d
Preparing for 3.2.3 development 2020-10-16 14:52:41 +08:00
Kihwal Lee 392b26c23b HDFS-15628. HttpFS server throws NPE if a file is a symlink. Contributed by Ahmed Hussein. 
(cherry picked from commit e45407128d)
 2020-10-14 18:19:16 -05:00
Kihwal Lee 25a51f2f39 HDFS-15581. Access Controlled HttpFS Server. Contributed by Richard Ross. 
(cherry picked from commit dfc2682213)
 2020-09-22 11:12:43 -05:00
Steve Loughran 42c71a5790
HADOOP-15691. Add PathCapabilities to FileSystem and FileContext. 
Contributed by Steve Loughran.

This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.

This is intended to allow applications to determine

* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.

As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
 fs. + schema + .

The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.

Note

* The remote store is not expected to be checked for the feature;
  It is more a check of client API and the client's configuration/knowledge
  of the state of the remote system.
* Permissions are not checked.
 2020-08-19 17:15:06 +01:00
Mingliang Liu 41c211d762
HDFS-15499. Clean up httpfs/pom.xml to remove aws-java-sdk-s3 exclusion (#2188) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2020-08-04 23:01:04 -07:00
Akira Ajisaka d4f75e2798
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li. 
(cherry picked from commit f6d20daf40)
 2020-01-21 18:03:56 +09:00
Wei-Chiu Chuang aa3c96474f HDFS-15097. Purge log in KMS and HttpFS. Contributed by Doris Gu. 
(cherry picked from commit 6b86a5110e)
 2020-01-13 10:49:26 -08:00
Siyao Meng e0cf1735e1 HADOOP-16676. Backport HADOOP-16152 to branch-3.2. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-11-12 11:38:42 -08:00
Siyao Meng 673c9d53ca HDFS-14686. HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null (#1192) Contributed by Siyao Meng. 
(cherry picked from commit 17e8cf501b)
 2019-10-04 09:40:22 -07:00
Inigo Goiri 94af96325c HDFS-14850. Optimize FileSystemAccessService#getFileSystemConfiguration. Contributed by Lisheng Sun. 
(cherry picked from commit d8313b2274)
 2019-10-03 18:21:46 -07:00
Rohith Sharma K S 7d5bb2ebb7 Preparing for 3.2.2-SNAPSHOT development. 2019-09-07 08:52:08 +05:30
Akira Ajisaka 2d8799f4bc HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2019-08-27 19:08:39 +00:00
Siyao Meng 88aece4083 HDFS-14665. HttpFS: LISTSTATUS response is missing HDFS-specific fields (#1267) Contributed by Siyao Meng. 
(cherry picked from commit 6ae8bc3a4a)
 2019-08-13 16:36:52 -07:00
Chao Sun d38b617baa HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from 3ae775d740)
 2019-08-08 10:15:48 -07:00
Takanobu Asanuma 6dffad028e HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 98d2065643)
 2019-06-20 20:17:45 -07:00
Takanobu Asanuma a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka 855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Weiwei Yang f5e1bad0f3 HDFS-14063. Support noredirect param for CREATE/APPEND/OPEN/GETFILECHECKSUM in HttpFS. Contributed by Íñigo Goiri. 
(cherry picked from commit ad5256e44d)
 2018-11-17 01:16:23 +08:00
Sunil G bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Siyao Meng 551e911493 HDFS-13878. HttpFS: Implement GETSNAPSHOTTABLEDIRECTORYLIST. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 6dcfef79af)
 2018-10-11 15:04:02 -07:00
Siyao Meng 15e0ba3fb4 HDFS-13877. HttpFS: Implement GETSNAPSHOTDIFF. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 396ce0d9f4)
 2018-10-11 15:03:55 -07:00
Akira Ajisaka 284aca8b63
HDFS-13938. Add a missing break in BaseTestHttpFSWith. Contributed by Siyao Meng. 2018-09-28 15:44:46 +09:00
Akira Ajisaka f9c0221623
HDFS-13932. [JDK 11] Casts to BlockStoragePolicy[] in unit tests raise ClassCastExceptions. Contributed by Andrew Purtell. 2018-09-27 11:44:42 +09:00
Siyao Meng 8de5c923b4 HDFS-13876. HttpFS: Implement ALLOWSNAPSHOT, DISALLOWSNAPSHOT. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-09-24 10:40:30 -07:00
Wei-Chiu Chuang 44857476fa HDFS-13886. HttpFSFileSystem.getFileStatus() doesn't return "snapshot enabled" bit. Contributed by Siyao Meng. 2018-09-18 15:33:02 -07:00
Wei-Chiu Chuang e780556ae9 HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang. 2018-09-05 09:52:35 -07:00
Inigo Goiri 914b98a713 HDFS-13537. TestHdfsHelper does not generate jceks path properly for relative path in Windows. Contributed by Xiao Liang. 2018-05-09 16:52:09 -07:00
fang zhenyi 5e013d50d1
HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-15 16:12:07 +09:00
Xiao Chen 9276ef0665 HDFS-13170. Port webhdfs unmaskedpermission parameter to HTTPFS. Contributed by Stephen O'Donnell. 2018-03-06 09:57:35 -08:00
Robert Kanter 21176a8ba7
HADOOP-15282. HADOOP-15235 broke TestHttpFSServerWebServer 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-04 07:33:47 -08:00
fang zhenyi 4d4dde5112
HADOOP-15223. Replace Collections.EMPTY* with empty* when available 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-18 22:19:39 +09:00
Wangda Tan 60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Chris Douglas 0e560f3b8d HDFS-12681. Make HdfsLocatedFileStatus a subtype of LocatedFileStatus 2017-11-29 20:28:06 -08:00
Arpit Agarwal 8acdf5c274 HDFS-12542. Update javadoc and documentation for listStatus. Contributed by Ajay Kumar. 2017-10-11 12:29:35 -07:00
Xiaoyu Yao 107c177782 HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 2017-10-03 13:02:12 -07:00
Andrew Wang 2f867115a8 Revert "HDFS-6874. Add GETFILEBLOCKLOCATIONS operation to HttpFS. Contributed by Weiwei Yang" 
This reverts commit 931a49800e.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
 2017-09-14 15:02:27 -07:00
ChenSammi e0b3c644e1 HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen 2017-09-14 09:15:29 +08:00
Andrew Wang 0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Allen Wittenauer a3fee475f7
HADOOP-14414. Calling maven-site-plugin directly for docs profile is unnecessary (Andras Bokor via aw) 2017-09-01 08:38:30 -07:00
Andrew Wang dd7916d3cd HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas. 2017-08-17 09:35:36 -07:00
Anu Engineer d72124a442 HDFS-12162. Update listStatus document to describe the behavior when the argument is a file. Contributed by Ajay Kumar. 2017-08-14 11:32:49 -07:00
Wei-Chiu Chuang 8a4bff02c1 HDFS-12117. HttpFS does not seem to support SNAPSHOT related methods for WebHDFS REST Interface. Contributed by Wellington Chevreuil. 2017-08-08 23:58:53 -07:00
Chris Douglas 12e44e7bda HDFS-6984. Serialize FileStatus via protobuf. 2017-08-02 12:12:48 -07:00
Sean Mackrory 1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Ravi Prakash c98201b5d8 HDFS-12193. Fix style issues in HttpFS tests. Contributed by Zoran Dimitrijevic 2017-07-24 19:06:15 -07:00
John Zhuge 3b48f81411 HDFS-12166. Do not deprecate HTTPFS_TEMP. Contributed by John Zhuge. 
Change-Id: Icdd032de9feea3d0dde71bf8145b70497993d97f
 2017-07-20 20:45:09 -07:00
Yongjun Zhang 3556e36be3 HDFS-12139. HTTPFS liststatus returns incorrect pathSuffix for path of file. Contributed by Yongjun Zhang. 2017-07-19 10:56:50 -07:00