Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
25,151 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1001 Commits

Author SHA1 Message Date
Andrew Wang 66e2cfa1a0 HDFS-8632. Add InterfaceAudience annotation to the erasure coding classes. Contributed by Rakesh R. 2015-10-07 18:12:26 -07:00
Jing Zhao 239d119c67 HDFS-9196. Fix TestWebHdfsContentLength. Contributed by Masatake Iwasaki. 2015-10-06 23:54:49 -07:00
Uma Mahesh 8b7339312c HDFS-9182. Cleanup the findbugs and other issues after HDFS EC merged to trunk. Contributed by Uma Maheswara Rao G. 2015-10-06 22:22:32 -07:00
Jing Zhao a8b4d0ff28 HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers. Contributed by Jing Zhao. 2015-10-06 10:56:10 -07:00
Vinayakumar B 874c8ed239 HDFS-7899. Improve EOF error message (Contributed by Jagadesh Kiran N) 2015-10-06 15:42:36 +05:30
Haohui Mai 7136e8c558 HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module. Contributed by Mingliang Liu. 2015-10-03 11:38:45 -07:00
Jing Zhao c6cafc77e6 HDFS-9185. Fix null tracer in ErasureCodingWorker. Contributed by Rakesh R. 2015-10-02 11:08:17 -07:00
cnauroth c7e03c3a6c HDFS-9175. Change scope of 'AccessTokenProvider.getAccessToken()' and 'CredentialBasedAccessTokenProvider.getCredential()' abstract methods to public. Contributed by Santhosh Nayak. 2015-09-30 16:06:34 -07:00
Zhe Zhang 6c17d31528 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java

Change-Id: I53ec1c426dc988d6c4a2c87b00caef49c4057010
 2015-09-30 08:39:13 -07:00
Haohui Mai 39285e6a19 HDFS-8971. Remove guards when calling LOG.debug() and LOG.trace() in client package. Contributed by Mingliang Liu. 2015-09-29 17:52:36 -07:00
Haohui Mai 6ee0539ede Revert "HDFS-9170. Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client. Contributed by Haohui Mai." 
This reverts commit d5a9a3daa0.
 2015-09-29 17:51:15 -07:00
Haohui Mai d5a9a3daa0 HDFS-9170. Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client. Contributed by Haohui Mai. 2015-09-29 17:48:29 -07:00
Haohui Mai 80d33b589b HDFS-9165. Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client. Contributed by Mingliang Liu. 2015-09-29 09:50:04 -07:00
Colin Patrick Mccabe 850d679acb HADOOP-12447. Clean up some htrace integration issues (cmccabe) 2015-09-29 09:25:11 -07:00
Zhe Zhang 8fd5520246 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java

Change-Id: Ic7946c4ea35bed587fe879ce58b959b25ecc0823
 2015-09-29 01:39:16 -07:00
Kihwal Lee 3abbdc929b HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu. 2015-09-28 17:31:51 -05:00
Jing Zhao 6419900ac2 HDFS-9040. Erasure coding: coordinate data streamers in DFSStripedOutputStream. Contributed by Jing Zhao and Walter Su. 2015-09-28 14:40:27 -07:00
Lei Xu e5992ef4df HDFS-9147. Fix the setting of visibleLength in ExternalBlockReader. (Colin P. McCabe via Lei (Eddy) Xu) 2015-09-28 13:39:00 -07:00
Kihwal Lee 4c9497cbf0 HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. 2015-09-28 13:29:56 -05:00
Colin Patrick Mccabe 892ade689f HDFS-9080. Update htrace version to 4.0.1 (cmccabe) 2015-09-28 07:42:12 -07:00
Haohui Mai 1c030c6e58 HDFS-8740. Move DistributedFileSystem to hadoop-hdfs-client. Contributed by Mingliang Liu. 2015-09-27 10:54:44 -07:00
Haohui Mai bf37d3d80e HDFS-8053. Move DFSIn/OutputStream and related classes to hadoop-hdfs-client. Contributed by Mingliang Liu. 2015-09-26 11:08:25 -07:00
Lei Xu 67b0e967f0 HDFS-9133. ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF. (Colin Patrick McCabe via Lei (Eddy) Xu) 2015-09-25 16:01:41 -07:00
Lei Xu 5eb237d544 HDFS-9132. Pass genstamp to ReplicaAccessorBuilder. (Colin Patrick McCabe via Lei (Eddy) Xu) 2015-09-25 15:50:19 -07:00
Yongjun Zhang 83e65c5fe8 HDFS-5795. RemoteBlockReader2#checkSuccess() shoud print error status. (Xiao Chen via Yongjun Zhang) 2015-09-24 22:13:14 -07:00
Haohui Mai 692b1a45ce HDFS-9134. Move LEASE_{SOFTLIMIT,HARDLIMIT}_PERIOD constants from HdfsServerConstants to HdfsConstants. Contributed by Mingliang Liu. 2015-09-24 20:19:16 -07:00
Haohui Mai ead1b9e680 HDFS-9131. Move config keys used by hdfs-client to HdfsClientConfigKeys. Contributed by Mingliang Liu. 2015-09-24 00:30:01 -07:00
Zhe Zhang c09dc258a8 HDFS-8882. Erasure Coding: Use datablocks, parityblocks and cell size from ErasureCodingPolicy. Contributed by Vinayakumar B. 
Change-Id: Ic56da0b426f47c63dac440aef6f5fc8554f6cf13
 2015-09-23 13:34:09 -07:00
Haohui Mai 7c5c099324 HDFS-8733. Keep server related definition in hdfs.proto on server side. Contributed by Mingliang Liu. 2015-09-22 20:57:05 -07:00
Haohui Mai 63d9f1596c HDFS-9039. Separate client and server side methods of o.a.h.hdfs.NameNodeProxies. Contributed by Mingliang Liu. 2015-09-22 20:52:37 -07:00
Zhe Zhang 1080c37300 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

Change-Id: I8511c4d64b0959e79129febc179845a3892fedcc
 2015-09-22 13:27:09 -07:00
Haohui Mai 06022b8fdc HDFS-9111. Move hdfs-client protobuf convert methods from PBHelper to PBHelperClient. Contributed by Mingliang Liu. 2015-09-21 18:53:13 -07:00
Lei Xu 3a9c7076e8 HDFS-9004. Add upgrade domain to DatanodeInfo. Contributed by Ming Ma (via Lei (Eddy) Xu). 
Change-Id: I887c66578eebd61acc34b94f18da6e6851c609f4
 2015-09-19 18:08:09 -07:00
Haohui Mai 9eee97508f HDFS-9022. Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client. Contributed by Mingliang Liu. 2015-09-17 14:18:00 -07:00
cnauroth 559c09dc0e HDFS-9082. Change the log level in WebHdfsFileSystem.initialize() from INFO to DEBUG. Contributed by Santhosh Nayak. 2015-09-15 15:13:43 -07:00
Haohui Mai 0c7d3f4805 HDFS-9041. Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client. Contributed by Mingliang Liu. 2015-09-12 11:02:51 -07:00
Arpit Agarwal 15a557fcfe HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method. (Contributed by Mingliang Liu) 2015-09-11 10:11:52 -07:00
Zhe Zhang f62237bc2f HDFS-8833. Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones. 2015-09-09 23:07:55 -07:00
Vinayakumar B 0f0e897bf1 HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) 2015-09-09 22:58:50 +05:30
Akira Ajisaka 0113e4528d HDFS-9033. dfsadmin -metasave prints "NaN" for cache used%. Contributed by Brahma Reddy Battula. 2015-09-09 17:20:46 +09:00
Haohui Mai d16c4eee18 HDFS-9012. Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to hadoop-hdfs-client module. Contributed by Mingliang Liu. 2015-09-04 10:41:09 -07:00
Haohui Mai ed78b14ebc HDFS-9002. Move o.a.h.hdfs.net/*Peer classes to hdfs-client. Contributed by Mingliang Liu. 2015-09-03 15:32:53 -07:00
Akira Ajisaka c92e31bd65 HDFS-8885. ByteRangeInputStream used in webhdfs does not override available(). Contributed by Shradha Revankar. 2015-09-03 19:40:34 +09:00
Zhe Zhang 53358fe680 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 2015-09-01 14:48:37 -07:00
Zhe Zhang ab56fcdb12 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 2015-09-01 14:30:25 -07:00
Haohui Mai 826ae1c26d HDFS-8990. Move RemoteBlockReader to hdfs-client module. Contributed by Mingliang Liu. 2015-08-31 13:54:14 -07:00
Jakob Homan 837fb75e8e HDFS-8155. Support OAuth2 in WebHDFS. 2015-08-29 18:37:05 -07:00
Haohui Mai e2c9b288b2 HDFS-8925. Move BlockReaderLocal to hdfs-client. Contributed by Mingliang Liu. 2015-08-28 14:38:36 -07:00
Haohui Mai 1e5f69e85c HDFS-8961. Investigate lock issue in o.a.h.hdfs.shortcircuit.DfsClientShmManager.EndpointShmManager. Contributed by Mingliang Liu. 2015-08-27 09:13:20 -07:00
Haohui Mai 7e971b7315 HDFS-8962. Clean up checkstyle warnings in o.a.h.hdfs.DfsClientConf. Contributed by Mingliang Liu. 2015-08-27 09:02:34 -07:00
Haohui Mai c992bcf9c1 HDFS-8951. Move the shortcircuit package to hdfs-client. Contributed by Mingliang Liu. 2015-08-26 14:02:48 -07:00
Zhe Zhang 6b6a63bbbd Merge commit '456e901a4c5c639267ee87b8e5f1319f256d20c2' (HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.) into HDFS-7285-merge 2015-08-24 12:59:26 -07:00
Haohui Mai 490bb5ebd6 HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang Liu. 2015-08-22 13:31:03 -07:00
Lei Xu 7087e700e0 HDFS-8924. Add pluggable interface for reading replicas in DFSClient. (Colin Patrick McCabe via Lei Xu) 2015-08-21 17:02:00 -07:00
Haohui Mai 3aac4758b0 HDFS-8803. Move DfsClientConf to hdfs-client. Contributed by Mingliang Liu. 2015-08-19 11:28:05 -07:00
Chris Douglas 30e342a5d3 HDFS-8435. Support CreateFlag in WebHDFS. Contributed by Jakob Homan 2015-08-18 17:32:48 -07:00
Andrew Wang eee4d716b4 HDFS-8895. Remove deprecated BlockStorageLocation APIs. 2015-08-17 17:40:37 -07:00
Zhe Zhang 1d37a88121 HDFS-8854. Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs. Contributed by Walter Su. 2015-08-13 10:04:45 -07:00
Andrew Wang 1ea1a8334e HDFS-8887. Expose storage type and storage ID in BlockLocation. 2015-08-11 23:25:33 -07:00
Vinayakumar B cc71ad80e1 HDFS-8815. DFS getStoragePolicy implementation using single RPC call (Contributed by Surendra Singh Lilhore) 2015-08-06 11:10:48 +05:30
Vinayakumar B 50887e5b07 HDFS-8811. Move BlockStoragePolicy name's constants from HdfsServerConstants.java to HdfsConstants.java (Contributed by Vinayakumar B) 2015-07-29 11:45:33 +05:30
Walter Su 4a72be6e0e HDFS-8769. Erasure coding: unit test for SequentialBlockGroupIdGenerator. Contributed by Rakesh R. 2015-07-25 10:07:40 +08:00
Akira Ajisaka f8f6091823 HDFS-8735. Inotify: All events classes should implement toString() API. Contributed by Surendra Singh Lilhore. 2015-07-25 02:56:55 +09:00
Haohui Mai 813cf89bb5 HDFS-8730. Clean up the import statements in ClientProtocol. Contributed by Takanobu Asanuma. 2015-07-23 10:31:11 -07:00
Jing Zhao e91ccfad07 HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. 2015-07-22 17:42:31 -07:00
Walter Su 06394e3760 HDFS-8433. Erasure coding: set blockToken in LocatedStripedBlock. Contributed by Walter Su. 2015-07-20 10:18:34 +08:00
Akira Ajisaka 979c9ca2ca HDFS-8742. Inotify: Support event for OP_TRUNCATE. Contributed by Surendra Singh Lilhore. 2015-07-15 04:42:08 +09:00
Haohui Mai fc6182d5ed HDFS-8726. Move protobuf files that define the client-sever protocols to hdfs-client. Contributed by Haohui Mai. 2015-07-08 10:37:10 -07:00
Haohui Mai c0b8e4e5b5 HDFS-8620. Clean up the checkstyle warinings about ClientProtocol. Contributed by Takanobu Asanuma. 2015-07-07 14:01:19 -07:00
Vinayakumar B fc92d3e651 HDFS-8686. WebHdfsFileSystem#getXAttr(Path p, final String name) doesn't work if namespace is in capitals (Contributed by kanaka kumar avvaru) 2015-07-06 16:09:24 +05:30
Vinayakumar B 0b7af27b9a HDFS-8468. 2 RPC calls for every file read in DFSClient#open(..) resulting in double Audit log entries (Contributed by Vinayakumar B) 2015-06-30 11:02:13 +05:30
Andrew Wang 60b858bfa6 HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization. 2015-06-26 11:30:59 -07:00
Haohui Mai 122cad6aec HDFS-6564. Use slf4j instead of common-logging in hdfs-client. Contributed by Rakesh R. 2015-06-23 11:41:40 -07:00
Jakob Homan fac4e04dd3 HDFS-8542. WebHDFS getHomeDirectory behavior does not match specification. Contributed by Kanaka Kumar Avvaru. 2015-06-22 16:30:45 -07:00
Jing Zhao 3682e01984 HDFS-8254. Avoid assigning a leading streamer in StripedDataStreamer to tolerate datanode failure. Contributed by Tsz Wo Nicholas Sze. 2015-06-19 10:23:45 -07:00
Haohui Mai b832774488 HDFS-8238. Move ClientProtocol to the hdfs-client. Contributed by Takanobu Asanuma. 2015-06-17 16:16:36 -07:00
Jing Zhao 5f15084bd5 HDFS-8453. Erasure coding: properly handle start offset for internal blocks in a block group. Contributed by Zhe Zhang. 2015-06-02 16:14:08 -07:00
Jing Zhao 45db1a0b72 HDFS-8323. Bump GenerationStamp for write faliure in DFSStripedOutputStream. Contributed by Tsz Wo Nicholas Sze. 2015-05-26 12:02:34 -07:00
Zhe Zhang 91c81fdc24 HDFS-8375. Add cellSize as an XAttr to ECZone. Contributed by Vinayakumar B. 2015-05-26 12:02:33 -07:00
boli2 8c95673db4 HDFS-8366. Erasure Coding: Make the timeout parameter of polling blocking queue configurable in DFSStripedOutputStream. Contributed by Li Bo 2015-05-26 12:02:32 -07:00
Jing Zhao 9da927540f HDFS-8289. Erasure Coding: add ECSchema to HdfsFileStatus. Contributed by Yong Zhang. 2015-05-26 12:01:49 -07:00
Jing Zhao ea6c66ed57 HDFS-8316. Erasure coding: refactor EC constants to be consistent with HDFS-8249. Contributed by Zhe Zhang. 2015-05-26 11:59:57 -07:00
Zhe Zhang 922631f04f HDFS-8188. Erasure coding: refactor client-related code to sync with HDFS-8082 and HDFS-8169. Contributed by Zhe Zhang. 2015-05-26 11:59:51 -07:00
Zhe Zhang 91c741a2a1 HDFS-7782. Erasure coding: pread from files in striped layout. Contributed by Zhe Zhang and Jing Zhao 2015-05-26 11:55:15 -07:00
Jing Zhao f05c21285e HDFS-7853. Erasure coding: extend LocatedBlocks to support reading from striped files. Contributed by Jing Zhao. 2015-05-26 11:32:25 -07:00
Arpit Agarwal a2190bf15d HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) 2015-05-18 11:36:29 -07:00
Haohui Mai 4d9f9e546f HDFS-8037. CheckAccess in WebHDFS silently accepts malformed FsActions parameters. Contributed by Walter Su. 2015-05-07 15:25:26 -07:00
Haohui Mai 0d6aa5d609 HDFS-8237. Move all protocol classes used by ClientProtocol to hdfs-client. Contributed by Haohui Mai. 2015-05-04 15:00:29 -07:00
Haohui Mai 6ae2a0d048 HDFS-8249. Separate HdfsConstants into the client and the server side class. Contributed by Haohui Mai. 2015-05-02 10:03:23 -07:00
Haohui Mai bcf89ddc7d HDFS-8052. Move WebHdfsFileSystem into hadoop-hdfs-client. Contributed by Haohui Mai. 2015-04-23 17:33:45 -07:00
Haohui Mai 12f4df043f HDFS-8218. Move classes that used by ClientProtocol into hdfs-client. Contributed by Haohui Mai. 2015-04-22 11:33:21 -07:00
Haohui Mai 6f8003dc7b HDFS-8185. Separate client related routines in HAUtil into a new class. Contributed by Haohui Mai. 2015-04-21 21:59:47 -07:00
Haohui Mai 5c97db07fb HDFS-8169. Move LocatedBlocks and related classes to hdfs-client. Contributed by Haohui Mai. 2015-04-20 00:36:46 -07:00
Haohui Mai 76e7264e8d HDFS-8165. Move GRANDFATHER_GENERATION_STAMP and GRANDFATER_INODE_ID to hdfs-client. Contributed by Haohui Mai. 2015-04-16 22:49:30 -07:00
Tsz-Wo Nicholas Sze 75bbcc8bf3 HDFS-8082. Move dfs.client.read.*, dfs.client.short.circuit.*, dfs.client.mmap.* and dfs.client.hedged.read.* conf from DFSConfigKeys to HdfsClientConfigKeys. 2015-04-16 13:22:31 -07:00
Tsz-Wo Nicholas Sze 7fc50e2525 HDFS-8083. Move dfs.client.write.* conf from DFSConfigKeys to HdfsClientConfigKeys.Write. 2015-04-13 11:43:35 -07:00
Tsz-Wo Nicholas Sze 60da0e49e7 HDFS-8084. Move dfs.client.failover.* confs from DFSConfigKeys to HdfsClientConfigKeys.Failover and fix typos in the dfs.http.client.* configuration keys. 2015-04-10 19:38:43 -07:00
Haohui Mai af9d4fede5 HDFS-8102. Separate webhdfs retry configuration keys from DFSConfigKeys. Contributed by Haohui Mai. 2015-04-09 14:36:27 -07:00
Haohui Mai cc25823546 HDFS-8089. Move o.a.h.hdfs.web.resources.* to the client jars. Contributed by Haohui Mai. 2015-04-08 16:30:08 -07:00
Tsz-Wo Nicholas Sze 4be648b55c HDFS-8079. Move dfs.client.retry.* confs from DFSConfigKeys to HdfsClientConfigKeys.Retry. 2015-04-07 19:48:57 -07:00
Haohui Mai 16b74f978a HDFS-7893. Update the POM to create a separate hdfs-client jar. Contributed by Haohui Mai. 2015-04-02 15:29:18 -07:00