Commit Graph

11116 Commits

Author SHA1 Message Date
Kai Zheng 4970f2a2ab HDFS-7889 Subclass DFSOutputStream to support writing striping layout files. Contributed by Li Bo 2015-05-26 11:58:40 -07:00
Jing Zhao 4d0bc724f2 HDFS-8077. Erasure coding: fix bugs in EC zone and symlinks. Contributed by Jing Zhao and Zhe Zhang. 2015-05-26 11:58:32 -07:00
Kai Zheng 971bd5e885 HADOOP-11818 Minor improvements for erasurecode classes. Contributed by Rakesh R 2015-05-26 11:55:17 -07:00
Kai Zheng c4a0d88366 HDFS-8104 Make hard-coded values consistent with the system default schema first before remove them. Contributed by Kai Zheng 2015-05-26 11:55:17 -07:00
Kai Zheng 1a37b0fe9a HDFS-8074 Define a system-wide default EC schema. Contributed by Kai Zheng 2015-05-26 11:55:16 -07:00
Vinayakumar B 2ca3e38feb HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B)
Added missed file
2015-05-26 11:55:16 -07:00
Vinayakumar B 16484f0030 HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B) 2015-05-26 11:55:16 -07:00
Zhe Zhang bff59392ee HDFS-7782. Erasure coding: pread from files in striped layout. Contributed by Zhe Zhang and Jing Zhao 2015-05-26 11:55:15 -07:00
Zhe Zhang 91c741a2a1 HDFS-7782. Erasure coding: pread from files in striped layout. Contributed by Zhe Zhang and Jing Zhao 2015-05-26 11:55:15 -07:00
Vinayakumar B 99502cbbe2 HADOOP-11645. Erasure Codec API covering the essential aspects for an erasure code ( Contributed by Kai Zheng) 2015-05-26 11:55:15 -07:00
Vinayakumar B aac73c21c3 HADOOP-11740. Combine erasure encoder and decoder interfaces. Contributed by Zhe Zhang.
Updated CHANGES-HDFS-EC-7285.txt
2015-05-26 11:55:15 -07:00
Vinayakumar B d25ca09cf6 HADOOP-11782 Correct two thrown messages in ECSchema class. Contributed by Xinwei Qin
Updated CHANGES-HDFS-EC-7285.txt
2015-05-26 11:55:14 -07:00
Kai Zheng d43102d2c5 Updated CHANGES-HDFS-EC-7285.txt 2015-05-26 11:55:14 -07:00
Kai Zheng 17f7cdc047 HADOOP-11805 Better to rename some raw erasure coders. Contributed by Kai Zheng 2015-05-26 11:55:14 -07:00
Zhe Zhang 146ce7a978 HDFS-7969. Erasure coding: NameNode support for lease recovery of striped block groups. Contributed by Zhe Zhang. 2015-05-26 11:55:14 -07:00
Zhe Zhang c243319eab HDFS-7936. Erasure coding: resolving conflicts in the branch when merging trunk changes (this commit is for HDFS-8035). Contributed by Zhe Zhang 2015-05-26 11:55:13 -07:00
Zhe Zhang e54a74b566 HADOOP-11740. Combine erasure encoder and decoder interfaces. Contributed by Zhe Zhang. 2015-05-26 11:55:13 -07:00
Zhe Zhang 1af8c14862 HDFS-7839. Erasure coding: implement facilities in NameNode to create and manage EC zones. Contributed by Zhe Zhang 2015-05-26 11:55:07 -07:00
Zhe Zhang 578019d6a2 HDFS-7936. Erasure coding: resolving conflicts in the branch when merging (this commit is for conflicts from HDFS-6945). Contributed by Zhe Zhang. 2015-05-26 11:44:47 -07:00
Kai Zheng 98ea3ec0da HADOOP-11782 Correct two thrown messages in ECSchema class. Contributed by Xinwei Qin 2015-05-26 11:44:47 -07:00
Zhe Zhang 9edc73d123 HDFS-7617. Add unit tests for editlog transactions for EC. Contributed by Hui Zheng. 2015-05-26 11:44:47 -07:00
Vinayakumar B af8eaacdb4 HDFS-8027. Erasure Coding: Update CHANGES-HDFS-7285.txt with branch commits (Vinayakumar B) 2015-05-26 11:44:46 -07:00
Jing Zhao cd655ee817 HDFS-8005. Erasure Coding: simplify striped block recovery work computation and add tests. Contributed by Jing Zhao. 2015-05-26 11:44:42 -07:00
Jing Zhao abf833a7b2 HDFS-7907. Erasure Coding: track invalid, corrupt, and under-recovery striped blocks in NameNode. Contributed by Jing Zhao. 2015-05-26 11:43:44 -07:00
Zhe Zhang 97378e4cd0 HDFS-7936. Erasure coding: resolving conflicts in the branch when merging (this commit is for HDFS-7742) 2015-05-26 11:43:43 -07:00
Zhe Zhang d9af36b9bd HADOOP-11664. Loading predefined EC schemas from configuration. Contributed by Kai Zheng. 2015-05-26 11:43:43 -07:00
Tsz-Wo Nicholas Sze 9d1175b8fb HDFS-7716. Add a test for BlockGroup support in FSImage. Contributed by Takuya Fukudome 2015-05-26 11:43:43 -07:00
Zhe Zhang ea2e60fbcc HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903, HDFS-7435, HDFS-7930, HDFS-7960 (this commit is for HDFS-7960) 2015-05-26 11:43:42 -07:00
Jing Zhao d0d75a8339 HDFS-7827. Erasure Coding: support striped blocks in non-protobuf fsimage. Contributed by Hui Zheng. 2015-05-26 11:43:42 -07:00
Jing Zhao 46dac3595f HDFS-7864. Erasure Coding: Update safemode calculation for striped blocks. Contributed by GAO Rui. 2015-05-26 11:43:42 -07:00
Zhe Zhang 544f75d651 HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903, HDFS-7435 and HDFS-7930 (this commit is for HDFS-7930 only) 2015-05-26 11:43:42 -07:00
Kai Zheng df297245a7 HADOOP-11647. Reed-Solomon ErasureCoder. Contributed by Kai Zheng 2015-05-26 11:43:41 -07:00
Kai Zheng 90d332d6be HADOOP-11707. Add factory to create raw erasure coder. Contributed by Kai Zheng 2015-05-26 11:43:41 -07:00
Zhe Zhang 57a84c0d14 HDFS-7369. Erasure coding: distribute recovery work for striped blocks to DataNode. Contributed by Zhe Zhang. 2015-05-26 11:43:37 -07:00
Kai Zheng e3dbfeb138 Updated CHANGES-HDFS-EC-7285.txt accordingly 2015-05-26 11:42:05 -07:00
Kai Zheng c3bc083405 HADOOP-11706 Refine a little bit erasure coder API 2015-05-26 11:42:04 -07:00
Jing Zhao a38a37c634 HDFS-7912. Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks. Contributed by Jing Zhao. 2015-05-26 11:41:59 -07:00
Jing Zhao 26773d9d6c HDFS-7826. Erasure Coding: Update INodeFile quota computation for striped blocks. Contributed by Kai Sasaki. 2015-05-26 11:32:26 -07:00
Jing Zhao 11585883a9 HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903 and HDFS-7435. Contributed by Zhe Zhang. 2015-05-26 11:32:26 -07:00
Kai Zheng 68caf8728e Fixed a compiling issue introduced by HADOOP-11705. 2015-05-26 11:32:26 -07:00
drankye 292e367d07 HADOOP-11705. Make erasure coder configurable. Contributed by Kai Zheng 2015-05-26 11:32:25 -07:00
Jing Zhao f05c21285e HDFS-7853. Erasure coding: extend LocatedBlocks to support reading from striped files. Contributed by Jing Zhao. 2015-05-26 11:32:25 -07:00
Vinayakumar B 8f89d7489d HADOOP-11646. Erasure Coder API for encoding and decoding of block group ( Contributed by Kai Zheng ) 2015-05-26 11:32:25 -07:00
Jing Zhao 39a0a85fb7 HDFS-7872. Erasure Coding: INodeFile.dumpTreeRecursively() supports to print striped blocks. Contributed by Takuya Fukudome. 2015-05-26 11:32:25 -07:00
drankye c0945a8971 HADOOP-11643. Define EC schema API for ErasureCodec. Contributed by Kai Zheng 2015-05-26 11:32:24 -07:00
Jing Zhao 1e1e930407 HDFS-7837. Erasure Coding: allocate and persist striped blocks in NameNode. Contributed by Jing Zhao. 2015-05-26 11:32:20 -07:00
Zhe Zhang 9af1f4779b Addendum fix for HDFS-7749 to be compatible with HDFS-7993 2015-05-26 11:07:16 -07:00
Jing Zhao 9f2f583f40 HDFS-7749. Erasure Coding: Add striped block support in INodeFile. Contributed by Jing Zhao. 2015-05-26 11:07:15 -07:00
drankye dae27f6dd1 HADOOP-11542. Raw Reed-Solomon coder in pure Java. Contributed by Kai Zheng 2015-05-26 11:07:15 -07:00
Jing Zhao ba93714920 HDFS-7716. Erasure Coding: extend BlockInfo to handle EC info. Contributed by Jing Zhao. 2015-05-26 11:07:11 -07:00