Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,663 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

190 Commits

Author SHA1 Message Date
Lei Xu ae98314980 Add missing files from HDFS-9005. (lei) 2016-03-25 18:13:01 -07:00
Lei Xu 493648611d HDFS-9005. Provide support for upgrade domain script. (Ming Ma via Lei Xu) 2016-03-25 17:12:44 -07:00
Sangjin Lee d956e0a0bb HDFS-9579. Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level (Ming Ma via sjlee) 2016-03-21 10:56:32 -07:00
Colin Patrick Mccabe 8ac3fc7540 HDFS-9947. Block#toString should not output information from derived classes (cmccabe) 
(cherry picked from commit 9a43094e12)
 2016-03-14 14:59:46 -07:00
Akira Ajisaka fe0009a2bd HDFS-9812. Streamer threads leak if failure happens when closing DFSOutputStream. Contributed by Lin Yiqun. 
(cherry picked from commit 352d299cf8)
 2016-03-08 10:55:59 +09:00
Arpit Agarwal 7b4a1001ea HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou) 2016-03-02 18:35:38 -08:00
Zhe Zhang b97ac02c4b HDFS-9886. Configuration properties for hedged read is broken. Contributed by Akira AJISAKA. 2016-03-02 10:27:09 -08:00
Xiaoyu Yao 6bffe1a908 HDFS-9887. WebHdfs socket timeouts should be configurable. Contributed by Austin Donnelly and Chris Douglas. 
(cherry picked from commit 5abf051249)
 2016-03-02 09:24:46 -08:00
Andrew Wang 81b7b9de27 HDFS-9881. DistributedFileSystem#getTrashRoot returns incorrect path for encryption zones. 
(cherry picked from commit 4abb2fa687)
 2016-03-01 16:35:43 -08:00
cnauroth 59cd23e279 HDFS-9854. Log cipher suite negotiation more verbosely. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit d1dd248b75)
 2016-02-24 12:11:44 -08:00
Zhe Zhang 9ecb71f92c HDFS-9844. Correct path creation in getTrashRoot to handle root dir. 
Change-Id: I5afe6c7176063892e3e632698c849bd9f4b92906
 2016-02-23 13:30:21 -08:00
cnauroth 779a69e28d HDFS-9711. Integrate CSRF prevention filter in WebHDFS. Contributed by Chris Nauroth. 
(cherry picked from commit 5d1889a66d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
 2016-02-18 10:09:33 -08:00
Zhe Zhang e226dbbc8a HDFS-9799. Reimplement getCurrentTrashDir to remove incompatibility. (zhz) 
Change-Id: I7834bcebffed38cb384db5395ddb8b6dd9e79a0b
 2016-02-17 13:31:48 -08:00
Haohui Mai e7d1507e21 HDFS-9815. Move o.a.h.fs.Hdfs to hadoop-hdfs-client. Contributed by Vinayakumar B. 2016-02-17 10:56:57 -08:00
Masatake Iwasaki f71fa500f9 HDFS-9700. DFSClient and DFSOutputStream should set TCP_NODELAY on sockets for DataTransferProtocol (Gary Helmling via iwasakims) 
(cherry picked from commit 372d1302c6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/DfsClientConf.java
 2016-02-13 03:32:58 +09:00
Akira Ajisaka a714d04f5d HDFS-9768. Reuse ObjectMapper instance in HDFS to improve the performance. Contributed by Lin Yiqun. 
(cherry picked from commit e6a7044b85)
 2016-02-13 02:04:01 +09:00
zhezhang ef9a81e875 HDFS-9244. Support nested encryption zones. 
Change-Id: I43a13035a8b27956e90967ee82058efb647c3415
 2016-02-08 16:31:41 -08:00
Uma Maheswara Rao G 44a020ed81 HDFS-9777. Fix typos in DFSAdmin command line and documentation.(Wei-Chiu Chuang via umamahesh) 
(cherry picked from commit bffaa38a91)
 2016-02-08 14:00:47 -08:00
Kihwal Lee 494cfd1dc4 HDFS-9752. Permanent write failures may happen to slow writers during datanode rolling upgrades. Contributed by Walter Su. 
(cherry picked from commit 193d27de0a)
 2016-02-08 12:17:07 -06:00
Haohui Mai e62ca14856 HDFS-9724. Degraded performance in WebHDFS listing as it does not reuse ObjectMapper. Contributed by Akira Ajisaka. 2016-02-04 11:35:34 -08:00
Vinayakumar B e8a9a69a27 HDFS-9739. DatanodeStorage.isValidStorageId() is broken (Contributed by Mingliang Liu) 
(cherry picked from commit d6b1acb940)
 2016-02-03 10:33:55 +05:30
Tsz-Wo Nicholas Sze b10d8ced21 HDFS-8999. Allow a file to be closed with COMMITTED but not yet COMPLETE blocks. 2016-01-28 10:42:40 +08:00
Arpit Agarwal 0dbaef4377 HDFS-9094. Add command line option to ask NameNode reload configuration. (Contributed by Xiaobing Zhou) 2016-01-25 12:16:45 -08:00
Eric Yang da61953a0b HDFS-9653. Added blocks pending deletion report to dfsadmin. 
(Weiwei Yang via eyang)
 2016-01-24 14:26:46 -08:00
Kihwal Lee b5db1d4410 HDFS-8898. Create API and command-line argument to get quota and quota usage without detailed content summary. Contributed by Ming Ma. 2016-01-22 09:19:23 -06:00
Kihwal Lee 3d8d8f1497 HDFS-9634. webhdfs client side exceptions don't provide enough details. Contributed by Eric Payne. 
(cherry picked from commit 7b70500484574a565dd8cd5c7d8b5bc7c6d91154)
 2016-01-21 09:33:30 -06:00
Colin Patrick Mccabe c56044b9ee HDFS-9576: HTrace: collect position/length information on read operations (zhz via cmccabe) 
(cherry picked from commit 7905788db9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
 2016-01-20 11:27:51 -08:00
Kihwal Lee 8d04b7c272 HDFS-9574. Reduce client failures during datanode restart. Contributed by Kihwal Lee. 
(cherry picked from commit 38c4c14472)
 2016-01-08 11:15:12 -06:00
Uma Mahesh cdd796d199 HADOOP-12658. Clear javadoc and check style issues around DomainSocket. Contributed by Kai Zheng 
(cherry picked from commit 778146eaae)
 2016-01-04 15:04:46 -08:00
Kihwal Lee 131260f0a7 HDFS-7163. WebHdfsFileSystem should retry reads according to the configured retry policy. Contributed by Eric Payne. 
(cherry picked from commit 867048c3e4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
 2015-12-22 14:19:12 -06:00
Colin Patrick Mccabe 31d2fb91fc HDFS-9350. Avoid creating temprorary strings in Block.toString() and getBlockName() (Staffan Friberg via cmccabe) 
(cherry picked from commit e63388fdf2)
 2015-12-18 10:05:12 -08:00
cnauroth e8e3737ce8 HDFS-9557. Reduce object allocation in PB conversion. Contributed by Daryn Sharp. 
(cherry picked from commit c470c8953d)
 2015-12-16 11:11:27 -08:00
Andrew Wang 2796c1099f HDFS-8894. Set SO_KEEPALIVE on DN server sockets. Contributed by Kanaka Kumar Avvaru. 
(cherry picked from commit 49949a4bb0)
 2015-12-15 14:39:04 -08:00
cnauroth 1072be12b9 HDFS-9414. Refactor reconfiguration of ClientDatanodeProtocol for reusability. Contributed by Xiaobing Zhou. 
(cherry picked from commit 303f80e656)
 2015-12-08 10:50:13 -08:00
Arpit Agarwal 6999e764cb HDFS-9414. Add missing license header 2015-12-05 09:24:44 -08:00
Arpit Agarwal 71be31201a HDFS-9414. Refactor reconfiguration of ClientDatanodeProtocol for reusability. (Contributed by Xiaobing Zhou) 2015-12-04 20:53:43 -08:00
Xiaoyu Yao eb7f9901b4 HDFS-8831. Trash Support for deletion in HDFS encryption zone. Contributed by Xiaoyu Yao. 
(cherry picked from commit cbc7b6bf97)
 2015-12-04 16:57:58 -08:00
Tsz-Wo Nicholas Sze bac127a4e2 HDFS-9294. DFSClient deadlock when close file and failed to renew lease. Contributed by Brahma Reddy Battula 2015-12-02 17:49:16 -08:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Xiaoyu Yao 4f1f9f7eff HDFS-8512. WebHDFS : GETFILESTATUS should return LocatedBlock with storage type info. Contributed by Xiaoyu Yao. 
(cherry picked from commit e3d673901b)
 2015-11-25 13:54:22 -08:00
Uma Mahesh 52c889ac14 HDFS-9433. DFS getEZForPath API on a non-existent file should throw FileNotFoundException (Rakesh R via umamahesh) 
(cherry picked from commit 411e2b2e7c)
 2015-11-23 18:46:48 -08:00
Haohui Mai d469e9e4ca HDFS-7988. Replace usage of ExactSizeInputStream with LimitInputStream. Contributed by Walter Su. 2015-11-23 10:50:23 -08:00
Haohui Mai 2c185222f9 HDFS-9443. Disabling HDFS client socket cache causes logging message printed to console for CLI commands. Contributed by Chris Nauroth. 2015-11-22 15:06:36 -08:00
Akira Ajisaka c74e42b4a2 HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed by Kazuho Fujii. 
(cherry picked from commit ac1aa6c819)
 2015-11-19 20:55:36 +09:00
cnauroth 7b94ae17ab HDFS-9400. TestRollingUpgradeRollback fails on branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit bad2afe39a)
 2015-11-18 11:00:18 -08:00
Xiaoyu Yao 48a494cdc3 HDFS-9234. WebHdfs: getContentSummary() should give quota for storage types. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 41d3f8899d)
 2015-11-10 16:48:06 -08:00
Zhe Zhang edd4b2d458 HDFS-9364. Unnecessary DNS resolution attempts when creating NameNodeProxies. Contributed by Xiao Chen. 
Change-Id: I9e42f724f27924cf73891425a832de37ce014a1e
 2015-11-10 09:55:39 -08:00
Kihwal Lee ceb062f093 HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit ef926b2e38)
 2015-11-09 08:57:25 -06:00
cnauroth 385dcc4064 HDFS-9394. branch-2 hadoop-hdfs-client fails during FileSystem ServiceLoader initialization, because HftpFileSystem is missing. Contributed by Mingliang Liu. 
(cherry picked from commit 1ee2db5c41)
 2015-11-07 07:39:45 -08:00
Tsz-Wo Nicholas Sze 9766fc988f HDFS-9398. Make ByteArraryManager log message in one-line format. Contributed by Mingliang Liu 2015-11-07 22:22:36 +08:00
cnauroth 75bcc8bcd8 HDFS-9384. TestWebHdfsContentLength intermittently hangs and fails due to TCP conversation mismatch between client and server. Contributed by Chris Nauroth. 
(cherry picked from commit 66c0967310)
 2015-11-05 22:07:34 -08:00
cnauroth 6edd41ac69 HDFS-9378. hadoop-hdfs-client tests do not write logs. Contributed by Chris Nauroth. 
(cherry picked from commit c563401796)
 2015-11-05 06:48:32 -08:00
Zhe Zhang 4f29f5d192 HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in commitBlock. Contributed by Chang Li. 
Change-Id: I0591b1f9c6a0a8b368408b195151de2e0aa440c4
 2015-11-03 14:23:02 -08:00
Haohui Mai 9e911a2439 HDFS-9168. Move client side unit test to hadoop-hdfs-client. Contributed by Haohui Mai. 2015-10-28 15:54:53 -07:00
Ming Ma 2c335a8434 HDFS-9259. Make SO_SNDBUF size configurable at DFSClient side for hdfs write scenario. (Mingliang Liu via mingma) 
(cherry picked from commit aa09880ab8)
 2015-10-27 09:29:37 -07:00
Yongjun Zhang b50af4e04a HDFS-7284. Add more debug info to BlockInfoUnderConstruction#setGenerationStampAndVerifyReplicas. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit 5e718de522)
 2015-10-26 13:36:25 -07:00
Arpit Agarwal b727dfd61c HDFS-4015. Safemode should count and report orphaned blocks. (Contributed by Anu Engineer) 2015-10-23 21:40:26 -07:00
Haohui Mai 558ae8c2df HDFS-9301. HDFS clients can't construct HdfsConfiguration instances. Contributed by Mingliang Liu. 2015-10-23 14:53:15 -07:00
Kihwal Lee d24d0fd981 HDFS-9290. DFSClient#callAppend() is not backward compatible for slightly older NameNodes. Contributed by Tony Wu. 
(cherry picked from commit b9e0417bdf)
 2015-10-23 16:38:47 -05:00
Andrew Wang c35771c46e HDFS-9250. Add Precondition check to LocatedBlock#addCachedLoc. Contributed by Xiao Chen. 
(cherry picked from commit 8175c4f6b9)
 2015-10-19 16:13:14 -07:00
Andrew Wang 12d3b69217 HDFS-8941. DistributedFileSystem listCorruptFileBlocks API should resolve relative path. Contributed by Rakesh R. 
(cherry picked from commit c32614f410)
 2015-10-09 11:57:07 -07:00
Jing Zhao a4b8dc48f1 HDFS-9196. Fix TestWebHdfsContentLength. Contributed by Masatake Iwasaki. 
(cherry picked from commit 239d119c67)
 2015-10-06 23:55:33 -07:00
Vinayakumar B 96fbe42bd7 HDFS-7899. Improve EOF error message (Contributed by Jagadesh Kiran N) 
(cherry picked from commit 874c8ed239)
 2015-10-06 15:55:12 +05:30
Haohui Mai 1257483ebf HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module. Contributed by Mingliang Liu. 2015-10-03 11:37:19 -07:00
cnauroth 7273ef0590 HDFS-9175. Change scope of 'AccessTokenProvider.getAccessToken()' and 'CredentialBasedAccessTokenProvider.getCredential()' abstract methods to public. Contributed by Santhosh Nayak. 
(cherry picked from commit c7e03c3a6c)
 2015-09-30 16:06:45 -07:00
Haohui Mai 37d42fcec1 HDFS-8971. Remove guards when calling LOG.debug() and LOG.trace() in client package. Contributed by Mingliang Liu. 2015-09-29 17:52:42 -07:00
Haohui Mai 58bdc9c672 Revert "HDFS-9170. Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client. Contributed by Haohui Mai." 
This reverts commit cf873fff9f.
 2015-09-29 17:51:31 -07:00
Haohui Mai cf873fff9f HDFS-9170. Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client. Contributed by Haohui Mai. 2015-09-29 17:48:35 -07:00
Haohui Mai cf6fb09162 HDFS-9166. Move hftp / hsftp filesystem to hfds-client. Contributed by Mingliang Liu. 2015-09-29 10:07:19 -07:00
Haohui Mai c705eab221 HDFS-9165. Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client. Contributed by Mingliang Liu. 2015-09-29 09:51:30 -07:00
Colin Patrick Mccabe 171bd1cb2b HADOOP-12447. Clean up some htrace integration issues (cmccabe) 
(cherry picked from commit 850d679acb)
 2015-09-29 09:38:10 -07:00
Kihwal Lee 95b8f1f3a6 HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu. 
(cherry picked from commit 3abbdc929b)
 2015-09-28 17:33:20 -05:00
Lei Xu 213ff984ee HDFS-9147. Fix the setting of visibleLength in ExternalBlockReader. (Colin P. McCabe via Lei (Eddy) Xu) 
(cherry picked from commit e5992ef4df)
 2015-09-28 13:43:52 -07:00
Kihwal Lee 468b330feb HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. 
(cherry picked from commit 4c9497cbf0)
 2015-09-28 14:30:06 -05:00
Colin Patrick Mccabe 3b9d32627f HDFS-9080. Update htrace version to 4.0.1 (cmccabe) 
(cherry picked from commit 892ade689f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockStorageLocationUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
 2015-09-28 08:15:00 -07:00
Haohui Mai 504d6fd950 HDFS-8740. Move DistributedFileSystem to hadoop-hdfs-client. Contributed by Mingliang Liu. 2015-09-27 10:54:53 -07:00
Haohui Mai 94cbb6d164 HDFS-8053. Move DFSIn/OutputStream and related classes to hadoop-hdfs-client. Contributed by Mingliang Liu. 2015-09-26 11:16:50 -07:00
Lei Xu 82d4c05b2d HDFS-9133. ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF. (Colin Patrick McCabe via Lei (Eddy) Xu) 
(cherry picked from commit 67b0e967f0)
 2015-09-25 16:02:27 -07:00
Lei Xu 1a2870f6ff HDFS-9132. Pass genstamp to ReplicaAccessorBuilder. (Colin Patrick McCabe via Lei (Eddy) Xu) 2015-09-25 15:51:29 -07:00
Yongjun Zhang bc5ee80794 HDFS-5795. RemoteBlockReader2#checkSuccess() shoud print error status. (Xiao Chen via Yongjun Zhang) 
(cherry picked from commit 83e65c5fe8)
 2015-09-24 22:15:28 -07:00
Haohui Mai 85c73d159e HDFS-9134. Move LEASE_{SOFTLIMIT,HARDLIMIT}_PERIOD constants from HdfsServerConstants to HdfsConstants. Contributed by Mingliang Liu. 2015-09-24 20:29:52 -07:00
Haohui Mai d148b6a234 HDFS-9131. Move config keys used by hdfs-client to HdfsClientConfigKeys. Contributed by Mingliang Liu. 2015-09-24 00:30:17 -07:00
Haohui Mai 090b1f6398 HDFS-8733. Keep server related definition in hdfs.proto on server side. Contributed by Mingliang Liu. 2015-09-22 20:57:11 -07:00
Haohui Mai ed7211563c HDFS-9039. Separate client and server side methods of o.a.h.hdfs.NameNodeProxies. Contributed by Mingliang Liu. 2015-09-22 20:52:46 -07:00
Haohui Mai 4e752a37e7 HDFS-9111. Move hdfs-client protobuf convert methods from PBHelper to PBHelperClient. Contributed by Mingliang Liu. 2015-09-21 19:43:14 -07:00
Lei Xu 0095936a6a HDFS-9004. Add upgrade domain to DatanodeInfo. Contributed by Ming Ma (via Lei (Eddy) Xu). 
Change-Id: I887c66578eebd61acc34b94f18da6e6851c609f4
(cherry picked from commit 3a9c7076e8)
 2015-09-19 18:09:56 -07:00
Haohui Mai cadde8c1e5 HDFS-9022. Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client. Contributed by Mingliang Liu. 2015-09-17 14:17:30 -07:00
cnauroth b297a9c95c HDFS-9082. Change the log level in WebHdfsFileSystem.initialize() from INFO to DEBUG. Contributed by Santhosh Nayak. 
(cherry picked from commit 559c09dc0e)
 2015-09-15 15:13:56 -07:00
Haohui Mai 323821b174 HDFS-9041. Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client. Contributed by Mingliang Liu. 2015-09-12 11:02:27 -07:00
Arpit Agarwal d732c7b9de HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method. (Contributed by Mingliang Liu) 2015-09-11 10:12:02 -07:00
Vinayakumar B 03f50de48d HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) 
(cherry picked from commit 0f0e897bf1)
 2015-09-09 22:59:55 +05:30
Akira Ajisaka eb9e06f8e3 HDFS-9033. dfsadmin -metasave prints "NaN" for cache used%. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0113e4528d)
 2015-09-09 17:21:22 +09:00
Haohui Mai 8f712dd2f0 HDFS-9012. Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to hadoop-hdfs-client module. Contributed by Mingliang Liu. 2015-09-04 10:41:28 -07:00
Haohui Mai d99018d6bc HDFS-9002. Move o.a.h.hdfs.net/*Peer classes to hdfs-client. Contributed by Mingliang Liu. 2015-09-03 15:33:22 -07:00
Akira Ajisaka 772fe0f478 HDFS-8885. ByteRangeInputStream used in webhdfs does not override available(). Contributed by Shradha Revankar. 
(cherry picked from commit c92e31bd65)
 2015-09-03 19:41:36 +09:00
Haohui Mai 94ab3b9e8f HDFS-8990. Move RemoteBlockReader to hdfs-client module. Contributed by Mingliang Liu. 2015-08-31 13:54:42 -07:00
Jakob Homan 94944ba4f6 HDFS-8155. Support OAuth2 in WebHDFS. 
(cherry picked from commit 837fb75e8e)
 2015-08-29 18:38:21 -07:00
Haohui Mai a3d0534558 HDFS-8925. Move BlockReaderLocal to hdfs-client. Contributed by Mingliang Liu. 2015-08-28 14:38:23 -07:00
Haohui Mai f4b2348503 HDFS-8961. Investigate lock issue in o.a.h.hdfs.shortcircuit.DfsClientShmManager.EndpointShmManager. Contributed by Mingliang Liu. 2015-08-27 09:13:37 -07:00
Haohui Mai af1e2993c7 HDFS-8962. Clean up checkstyle warnings in o.a.h.hdfs.DfsClientConf. Contributed by Mingliang Liu. 2015-08-27 09:05:28 -07:00