6581 Commits

Author SHA1 Message Date
Kai Zheng
cb672a45a0 HDFS-11605. Allow user to customize new erasure code policies. Contributed by Huafeng Wang 2017-04-28 13:18:50 +08:00
Wei-Chiu Chuang
bbf8cac14d HADOOP-13200. Implement customizable and configurable erasure coders. Contributed by Tim Yao. 2017-04-27 12:39:47 -07:00
Wei-Chiu Chuang
ddaeb3e497 Revert "HADOOP-13200. Implement customizable and configurable erasure coders. Contributed by Tim Yao."
This reverts commit 872088c6e7e78534a8ffd1ea6e57b86faea3d6e1.
2017-04-27 12:25:47 -07:00
Wei-Chiu Chuang
872088c6e7 HADOOP-13200. Implement customizable and configurable erasure coders. Contributed by Tim Yao. 2017-04-27 11:59:33 -07:00
Zhe Zhang
d8a3309830 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 2017-04-27 09:50:30 -07:00
Akira Ajisaka
249cb210f5
HDFS-11697. Add javadoc for storage policy and erasure coding policy. Contributed by Kai Sasaki. 2017-04-27 16:12:27 +09:00
Konstantin V Shvachko
28eb2aabeb HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 2017-04-26 17:28:49 -07:00
Chen Liang
50dd3a5cfa HDFS-11627. Block Storage: Cblock cache should register with flusher to upload blocks to containers. Contributed by Mukul Kumar Singh. 2017-04-26 10:36:56 -07:00
Akira Ajisaka
93fa48fcf2
HDFS-11417. Add datanode admin command to get the storage info. Contributed by Surendra Singh Lilhore. 2017-04-26 17:43:29 +09:00
Chris Douglas
2f73396b59 HDFS-6708. StorageType should be encoded in the block token. Contributed by Ewan Higgs 2017-04-25 23:57:00 -07:00
Xiaoyu Yao
4ea2778acb HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore. 2017-04-25 18:45:56 -07:00
Jason Lowe
e4321ec843 HDFS-11691. Add a proper scheme to the datanode links in NN web UI. Contributed by Kihwal Lee 2017-04-25 17:30:42 -05:00
John Zhuge
fda86ef2a3 HDFS-11529. Add libHDFS API to return last exception. Contributed by Sailesh Mukil. 2017-04-22 02:56:48 -07:00
Yongjun Zhang
20e3ae260b HDFS-11402. HDFS Snapshots should capture point-in-time copies of OPEN files. (Manoj Govindassamy via Yongjun Zhang) 2017-04-21 20:35:48 -07:00
Yongjun Zhang
5078df7be3 HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang. 2017-04-21 13:36:31 -07:00
Kai Zheng
b0803388fc HDFS-11604. Define and parse erasure code policies. Contributed by Lin Zeng 2017-04-21 13:33:33 +08:00
Chen Liang
eae8c2a469 HDFS-11667. Block Storage:Handling flushing of incomplete block id buffers during shutdown. Contributed by Mukul Kumar Singh. 2017-04-20 13:50:29 -07:00
Xiaoyu Yao
a2c6f594ba HDFS-11668. Ozone: misc improvements for SCM CLI. Contributed by Weiwei Yang. 2017-04-20 11:32:13 -07:00
Yiqun Lin
9154d77ca8 HDFS-11666. Ozone: Fix compile error due to inconsistent package name. Contributed by Yiqun Lin. 2017-04-20 14:22:59 +08:00
Xiaoyu Yao
8ec13a397f HDFS-11504. Ozone: SCM: Add Block APIs. Contributed by Xiaoyu Yao. 2017-04-19 22:34:49 -07:00
Andrew Wang
74a723852d HDFS-11660. TestFsDatasetCache#testPageRounder fails intermittently with AssertionError. 2017-04-19 18:10:04 -07:00
Andrew Wang
c1549352cf HDFS-11671. TestReconstructStripedBlocks#test2RecoveryTasksForSameBlockGroup fails. 2017-04-19 13:49:56 -07:00
John Zhuge
8c81a16a1f HDFS-11531. Expose hedged read metrics via libHDFS API. Contributed by Sailesh Mukil. 2017-04-18 21:11:07 -07:00
Weiwei Yang
0bf416ac40 HDFS-11649. Ozone: SCM: CLI: Add shell code placeholder classes. Contributed by Chen Liang. 2017-04-19 11:43:14 +08:00
John Zhuge
ca2488c009 HDFS-11665. HttpFSServerWebServer$deprecateEnv may leak secret. Contributed by John Zhuge. 2017-04-18 09:28:52 -07:00
Anu Engineer
d889437c27 HDFS-11658. Ozone: SCM daemon is unable to be started via CLI. Contributed by Weiwei Yang. 2017-04-17 19:09:11 -07:00
Anu Engineer
368424b465 HDFS-11581. Ozone: Support force delete a container. Contributed by Yuanbo Liu. 2017-04-17 17:58:54 -07:00
Konstantin V Shvachko
8dfcd95d58 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko. 2017-04-17 16:56:24 -07:00
Andrew Wang
c0ca785dbb HDFS-11652. Improve ECSchema and ErasureCodingPolicy toString, hashCode, equals. 2017-04-17 16:30:25 -07:00
Zhe Zhang
ad49098eb3 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 2017-04-17 16:22:33 -07:00
Anu Engineer
350220bfb3 HDFS-11631. Block Storage : allow cblock server to be started from hdfs commandline. Contributed by Chen Liang. 2017-04-17 15:54:39 -07:00
Anu Engineer
49c92ea73d HDFS-11657. Ozone: Add unit test for storage container metrics. Contributed by Yiqun Lin. 2017-04-17 12:44:29 -07:00
Anu Engineer
1a418ef2c0 HDFS-11582. Block Storage : add SCSI target access daemon. Contributed by Chen Liang. 2017-04-13 13:25:53 -07:00
Anu Engineer
3231ead84b Merge branch 'trunk' into HDFS-7240 2017-04-12 16:36:41 -07:00
Anu Engineer
6556445792 HDFS-11569. Ozone: Implement listKey function for KeyManager. Contributed by Weiwei Yang. 2017-04-12 15:36:56 -07:00
Andrew Wang
a7312715a6 HDFS-10996. Ability to specify per-file EC policy at create time. Contributed by SammiChen. 2017-04-12 12:27:34 -07:00
Andrew Wang
966b1b5b44 HDFS-11565. Use compact identifiers for built-in ECPolicies in HdfsFileStatus. 2017-04-12 12:24:32 -07:00
Chen Liang
cedacf12c9 HDFS-11642. Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup. Contributed by Xiaoyu Yao. 2017-04-12 12:09:10 -07:00
Anu Engineer
abce613356 HDFS-11645. DataXceiver thread should log the actual error when getting InvalidMagicNumberException. Contributed by Chen Liang. 2017-04-12 11:40:58 -07:00
Kihwal Lee
d4c01dde49 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 2017-04-12 13:29:24 -05:00
Chen Liang
f4dec00896 HDFS-11635. Block Storage: Add metrics for Container Flushes. Contributed by Mukul Kumar Singh. 2017-04-12 10:38:30 -07:00
Chris Nauroth
23b1a7bdf1 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 2017-04-11 22:03:09 -07:00
Hanisha Koneru
62e4573efb HDFS-11630. TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds. Contributed by Hanisha Koneru. 2017-04-11 21:36:24 -07:00
Mingliang Liu
3a91376707 HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou 2017-04-11 15:29:38 -07:00
Xiaoyu Yao
a2ff806d2e HDFS-11062. Ozone:SCM: Remove null command. Contributed by Yuanbo Liu. 2017-04-11 13:33:40 -07:00
Wei-Chiu Chuang
f050afb578 HADOOP-13665. Erasure Coding codec should support fallback coder. Contributed by Kai Sasaki. 2017-04-11 07:32:13 -07:00
Yiqun Lin
2fd568fdd4 HDFS-11637. Fix javac warning caused by the deprecated key used in TestDFSClientRetries#testFailuresArePerOperation. Contributed by Yiqun Lin. 2017-04-11 19:40:09 +08:00
Chris Douglas
a5e57df3c5 HADOOP-14225. Remove xmlenc dependency 2017-04-10 11:48:40 -07:00
Andrew Wang
e8bdad7385 HDFS-11623. Move system erasure coding policies into hadoop-hdfs-client. 2017-04-07 16:46:28 -07:00
Andrew Wang
203edc026c HDFS-11633. FSImage failover disables all erasure coding policies. Contributed by Wei-Chiu Chuang. 2017-04-07 15:46:11 -07:00