65ccf2b125
HDFS-8388. Time and Date format need to be in sync in NameNode UI page. Contributed by Surendra Singh Lilhore.
2015-09-02 14:28:38 +09:00
5652131d2e
HDFS-8995. Flaw in registration bookeeping can make DN die on reconnect. (Kihwal Lee via yliu)
2015-09-02 08:58:51 +08:00
53358fe680
Merge remote-tracking branch 'apache/trunk' into HDFS-7285
2015-09-01 14:48:37 -07:00
ab56fcdb12
Merge remote-tracking branch 'apache/trunk' into HDFS-7285
2015-09-01 14:30:25 -07:00
24f6a7c956
HDFS-8965. Harden edit log reading code against out of memory errors (cmccabe)
2015-08-31 18:06:30 -07:00
8fa41d9dd4
HDFS-8946. Improve choosing datanode storage for block placement. (yliu)
2015-09-01 08:52:50 +08:00
826ae1c26d
HDFS-8990. Move RemoteBlockReader to hdfs-client module. Contributed by Mingliang Liu.
2015-08-31 13:54:14 -07:00
caa04de149
HDFS-8980. Remove unnecessary block replacement in INodeFile. Contributed by Jing Zhao.
2015-08-31 11:48:09 -07:00
837fb75e8e
HDFS-8155. Support OAuth2 in WebHDFS.
2015-08-29 18:37:05 -07:00
bdbe53c676
HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal)
2015-08-29 09:52:37 -07:00
e2c9b288b2
HDFS-8925. Move BlockReaderLocal to hdfs-client. Contributed by Mingliang Liu.
2015-08-28 14:38:36 -07:00
b94b56806d
HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list (Daniel Templeton)
2015-08-28 14:21:46 -07:00
6d12cd8d60
HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu.
2015-08-28 14:14:32 -07:00
b6ceee9bf4
HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee.
2015-08-28 13:14:35 -05:00
e166c038c0
HDFS-8963. Fix incorrect sign extension of xattr length in HDFS-8900. (Colin Patrick McCabe via yliu)
2015-08-28 10:54:55 +08:00
035ed26147
Revert "HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu."
...
This reverts commit 4e9307f26d
2015-08-27 16:09:35 -07:00
4e9307f26d
HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu.
2015-08-27 15:36:44 -07:00
f97a0f8c2c
HDFS-8969. Clean up findbugs warnings for HDFS-8823 and HDFS-8932. Contributed by Anu Engineer.
2015-08-27 13:03:16 -07:00
1e5f69e85c
HDFS-8961. Investigate lock issue in o.a.h.hdfs.shortcircuit.DfsClientShmManager.EndpointShmManager. Contributed by Mingliang Liu.
2015-08-27 09:13:20 -07:00
7e971b7315
HDFS-8962. Clean up checkstyle warnings in o.a.h.hdfs.DfsClientConf. Contributed by Mingliang Liu.
2015-08-27 09:02:34 -07:00
0bf285413f
HDFS-2390. dfsadmin -setBalancerBandwidth does not validate -ve value. Contributed by Gautam Gopalakrishnan.
2015-08-27 16:22:48 +05:30
164cbe6439
HDFS-8909. Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature. Contributed by Jing Zhao.
2015-08-27 16:02:30 +08:00
fdb56f74f3
HDFS-8682. Should not remove decommissioned node,while calculating the number of live/dead decommissioned node. (Contributed by J. Andreina)
2015-08-27 13:06:43 +05:30
4cbbfa2220
HDFS-8248. Store INodeId instead of the INodeFile object in BlockInfoContiguous. Contributed by Haohui Mai.
2015-08-26 18:14:29 -07:00
067ec8c2b1
HDFS-8838. Erasure Coding: Tolerate datanode failures in DFSStripedOutputStream when the data length is small. Contributed by Tsz Wo Nicholas Sze.
2015-08-27 09:11:23 +08:00
f44b599003
HDFS-8896. DataNode object isn't GCed when shutdown, because it has GC root in ShutdownHookManager. Contributed by Walter Su.
2015-08-26 16:27:34 -07:00
c992bcf9c1
HDFS-8951. Move the shortcircuit package to hdfs-client. Contributed by Mingliang Liu.
2015-08-26 14:02:48 -07:00
a4d9acc51d
HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe)
2015-08-25 14:29:53 -07:00
eee0d4563c
HDFS-8900. Compact XAttrs to optimize memory footprint. (yliu)
2015-08-25 16:16:09 +08:00
3b00eaea25
HDFS-8932. NPE thrown in NameNode when try to get TotalSyncCount metric before editLogStream initialization. Contributed by Surendra Singh Lilhore
2015-08-24 16:56:24 -07:00
66d0c81d8f
HDFS-8948. Use GenericTestUtils to set log levels in TestPread and TestReplaceDatanodeOnFailure. Contributed by Mingliang Liu.
2015-08-24 16:18:15 -07:00
bdd79388f3
HDFS-8928. Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as a separate class and replicas as an array. Contributed by Jing Zhao.
2015-08-24 15:53:34 -07:00
6b6a63bbbd
Merge commit '456e901a4c5c639267ee87b8e5f1319f256d20c2' (HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.) into HDFS-7285-merge
2015-08-24 12:59:26 -07:00
b5ce87f84d
HDFS-8930. Block report lease may leak if the 2nd full block report comes when NN is still in safemode (Colin P. McCabe via Jing Zhao)
2015-08-24 11:31:56 -07:00
bcaf83902a
HDFS-8942. Update hyperlink to rack awareness page in HDFS Architecture documentation. Contributed by Masatake Iwasaki.
2015-08-24 13:52:49 +09:00
490bb5ebd6
HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang Liu.
2015-08-22 13:31:03 -07:00
745d04be59
HDFS-8823. Move replication factor into individual blocks. Contributed by Haohui Mai.
2015-08-22 00:09:40 -07:00
7087e700e0
HDFS-8924. Add pluggable interface for reading replicas in DFSClient. (Colin Patrick McCabe via Lei Xu)
2015-08-21 17:02:00 -07:00
b0564c9f3c
HDFS-8891. HDFS concat should keep srcs order. Contributed by Yong Zhang.
...
Moved CHANGES.txt entry to 2.7.2
2015-08-20 20:39:06 -07:00
33474939a0
Creating 2.6.2 entries in CHANGES.txt files.
2015-08-20 17:21:06 -07:00
c8bca62718
HDFS-8809. HDFS fsck reports under construction blocks as CORRUPT. Contributed by Jing Zhao.
2015-08-20 16:31:24 -07:00
7642f64c24
HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe)
2015-08-20 13:57:32 -07:00
0bc15cb6e6
HDFS-8828. Utilize Snapshot diff report to build diff copy list in distcp. (Yufei Gu via Yongjun Zhang)
2015-08-20 08:02:54 -07:00
5e8fe89437
HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu)
2015-08-20 20:15:03 +08:00
80a29906bc
HDFS-8884. Fail-fast check in BlockPlacementPolicyDefault#chooseTarget. (yliu)
2015-08-20 20:07:18 +08:00
4e14f7982a
HDFS-8917. Cleanup BlockInfoUnderConstruction from comments and tests. Contributed by Zhe Zhang.
2015-08-19 15:11:37 -07:00
3aac4758b0
HDFS-8803. Move DfsClientConf to hdfs-client. Contributed by Mingliang Liu.
2015-08-19 11:28:05 -07:00
f61120d964
HDFS-8867. Enable optimized block reports. Contributed by Daryn Sharp.
2015-08-19 10:36:56 -07:00
9c3571ea60
HDFS-8911. NameNode Metric : Add Editlog counters as a JMX metric. (Contributed by Anu Engineer)
2015-08-19 09:42:35 -07:00
2da5aaab33
HDFS-8908. TestAppendSnapshotTruncate may fail with IOException: Failed to replace a bad datanode. (Tsz Wo Nicholas Sze via yliu)
2015-08-19 21:55:43 +08:00
7ecbfd44aa
HDFS-8826. In Balancer, add an option to specify the source node list so that balancer only selects blocks to move from those nodes.
2015-08-18 19:25:50 -07:00
30e342a5d3
HDFS-8435. Support CreateFlag in WebHDFS. Contributed by Jakob Homan
2015-08-18 17:32:48 -07:00
fc509f66d8
HDFS-8852. HDFS architecture documentation of version 2.x is outdated about append write support. Contributed by Ajith S.
2015-08-18 23:31:52 +09:00
71566e2382
HDFS-8862. BlockManager#excessReplicateMap should use a HashMap. (yliu)
2015-08-18 09:23:06 +08:00
51a00964da
HDFS-8278. When computing max-size-to-move in Balancer, count only the storage with remaining >= default block size.
2015-08-17 17:55:25 -07:00
eee4d716b4
HDFS-8895. Remove deprecated BlockStorageLocation APIs.
2015-08-17 17:40:37 -07:00
a88f31ebf3
HDFS-8880. NameNode metrics logging. (Arpit Agarwal)
2015-08-17 16:55:46 -07:00
ec183faadc
HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe)
2015-08-17 15:25:29 -07:00
c77bd6af16
HDFS-8792. BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory (Yi Liu via Colin P. McCabe)
2015-08-17 12:00:45 -07:00
e535e0f05b
HDFS-8801. Convert BlockInfoUnderConstruction as a feature. Contributed by Jing Zhao.
2015-08-17 11:28:28 -07:00
456e901a4c
HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.
2015-08-17 11:04:00 -07:00
2e7b7e2cda
HDFS-8713. Convert DatanodeDescriptor to use SLF4J logging.
2015-08-17 10:17:06 -07:00
a7862d5fe4
HDFS-8883. NameNode Metrics : Add FSNameSystem lock Queue Length. Contributed by Anu Engineer.
2015-08-17 10:15:56 -07:00
b57c9a35f7
HDFS-8220. Erasure Coding: StripedDataStreamer fails to handle the blocklocations which doesn't satisfy BlockGroupSize.
2015-08-14 15:16:58 -07:00
dc7a061668
HDFS-8891. HDFS concat should keep srcs order. Contributed by Yong Zhang.
2015-08-14 14:42:43 -07:00
2bc0a4f299
HDFS-8824. Do not use small blocks for balancing the cluster.
2015-08-14 13:03:19 -07:00
1569228ec9
HDFS-8565. Typo in dfshealth.html - Decomissioning. (nijel via xyao)
2015-08-14 12:34:31 -07:00
84bf71295a
HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina)
...
Moved to 2.6.1
2015-08-14 16:13:30 +05:30
08bd4edf40
HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang)
...
Moved to 2.6.1
2015-08-14 12:38:00 +05:30
fa2641143c
HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo.
...
Moved CHANGES.txt entry to 2.6.1
2015-08-14 11:45:43 +05:30
f2b4bc9b6a
HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe)
...
Moved CHANGES.txt entry to 2.6.1
2015-08-14 11:37:39 +05:30
d25cb8fe12
HDFS-7213. processIncrementalBlockReport performance degradation. Contributed by Eric Payne.
...
Moved CHANGES.TXT entry to 2.6.1
2015-08-14 11:23:51 +05:30
ae57d60d82
HDFS-7649. Multihoming docs should emphasize using hostnames in configurations. (Contributed by Brahma Reddy Battula)
2015-08-13 14:26:40 -07:00
1d37a88121
HDFS-8854. Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs. Contributed by Walter Su.
2015-08-13 10:04:45 -07:00
53bef9c5b9
HADOOP-12295. Improve NetworkTopology#InnerNode#remove logic. (yliu)
2015-08-13 16:45:20 +08:00
40f815131e
HDFS-8622. Implement GETCONTENTSUMMARY operation for WebImageViewer. Contributed by Jagadesh Kiran N.
2015-08-13 11:31:56 +09:00
6cc8e38db5
HADOOP-12258. Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression. Contributed by Zhihai Xu.
2015-08-12 16:44:53 -07:00
3e715a4f4c
HDFS-8879. Quota by storage type usage incorrectly initialized upon namenode restart. Contributed by Xiaoyu Yao.
2015-08-12 15:20:54 -07:00
1ea1a8334e
HDFS-8887. Expose storage type and storage ID in BlockLocation.
2015-08-11 23:25:33 -07:00
1fc3c779a4
HDFS-8805. Archival Storage: getStoragePolicy should not need superuser privilege. Contributed by Brahma Reddy Battula.
2015-08-11 10:28:18 -07:00
b56daff6a1
HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable.
2015-08-10 16:52:02 -07:00
8f73bdd06b
HDFS-8866. Typo in docs: Rumtime -> Runtime. Contributed by Gabor Liptak.
2015-08-07 11:38:31 -07:00
fbf7e81ca0
HDFS-8827. Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks. Contributed by Walter Su and Takuya Fukudome.
2015-08-07 11:25:20 -07:00
98a27d1101
HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su.
2015-08-07 09:51:46 -07:00
6d4eee718a
HDFS-8856. Make LeaseManager#countPath O(1). (Contributed by Arpit Agarwal)
2015-08-06 18:51:28 -07:00
8799363db1
HDFS-8857. Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure. Contributed by Li Bo
2015-08-05 22:11:50 -04:00
4de48211c6
HDFS-8399. Erasure Coding: unit test the behaviour of BlockManager recovery work for the deleted blocks. Contributed by Rakesh R.
2015-08-04 15:23:39 -07:00
9312b168e2
HDFS-8804. Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation. Contributed by Jing Zhao.
2015-08-03 17:03:15 -07:00
f4c523b69b
Revert "HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang."
...
This reverts commit c17439c2dd
2015-08-06 10:21:55 -07:00
663eba0ab1
Revert "HDFS-8623. Refactor NameNode handling of invalid, corrupt, and under-recovery blocks. Contributed by Zhe Zhang."
...
This reverts commit de480d6c89
2015-08-06 10:21:54 -07:00
cc71ad80e1
HDFS-8815. DFS getStoragePolicy implementation using single RPC call (Contributed by Surendra Singh Lilhore)
2015-08-06 11:10:48 +05:30
c95993cbaf
HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe)
2015-08-04 12:22:06 -07:00
9a08999906
HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu)
2015-08-04 15:54:23 +08:00
90b510405a
HDFS-8847. change TestHDFSContractAppend to not override testRenameFileBeingAppended method. Contributed by Zhihai Xu
2015-07-31 21:40:43 -07:00
d311a38a6b
HDFS-6860. BlockStateChange logs are too noisy. Contributed by Chang Li and Xiaoyu Yao.
2015-07-31 16:15:50 -07:00
21d10ccc6e
HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe)
2015-07-31 15:01:34 -07:00
666cafca8d
HADOOP-12249. pull argument parsing into a function (aw)
2015-07-31 14:32:21 -07:00
ba90c02853
HDFS-8202. Improve end to end stirpping file test to add erasure recovering test. Contributed by Xinwei Qin.
2015-07-31 12:16:15 -07:00
c5caa25b8f
HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan.
2015-07-31 08:58:22 +05:30
88d8736dde
HDFS-7192. DN should ignore lazyPersist hint if the writer is not local. (Contributed by Arpit Agarwal)
2015-07-30 13:16:46 -07:00
ddc867ceb9
HDFS-8816. Improve visualization for the Datanode tab in the NN UI. Contributed by Haohui Mai.
2015-07-29 17:14:05 -07:00
6f0a35724f
HADOOP-10945. 4-digit octal umask permissions throws a parse error. Contributed by Chang Li
2015-07-29 15:25:10 +00:00
6374ee0db4
HDFS-8670. Better to exclude decommissioned nodes for namenode NodeUsage JMX (Contributed by J.Andreina)
2015-07-29 14:47:19 +05:30
2a1d656196
Revert "HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka)"
...
This reverts commit 02c01815ec
2015-07-29 16:48:30 +09:00
f4f1b8b267
HDFS-8834. TestReplication is not valid after HDFS-6482. (Contributed by Lei Xu)
2015-07-28 23:39:26 -07:00
975e138df3
HDFS-8822. Add SSD storagepolicy tests in TestBlockStoragePolicy#testDefaultPolicies (Contributed by Vinayakumar B)
2015-07-29 11:50:48 +05:30
50887e5b07
HDFS-8811. Move BlockStoragePolicy name's constants from HdfsServerConstants.java to HdfsConstants.java (Contributed by Vinayakumar B)
2015-07-29 11:45:33 +05:30
0712a8103f
HDFS-8180. AbstractFileSystem Implementation for WebHdfs. Contributed by Sathosh G Nayak.
2015-07-28 21:03:31 -07:00
030fcfa99c
HDFS-7858. Improve HA Namenode Failover detection on the client. (asuresh)
2015-07-27 23:02:03 -07:00
2196e39e14
HDFS-8785. TestDistributedFileSystem is failing in trunk. Contributed by Xiaoyu Yao.
2015-07-27 07:28:41 -07:00
1df78688c6
HDFS-8810. Correct assertions in TestDFSInotifyEventInputStream class. Contributed by Surendra Singh Lilhore.
2015-07-27 13:17:24 +09:00
4a72be6e0e
HDFS-8769. Erasure coding: unit test for SequentialBlockGroupIdGenerator. Contributed by Rakesh R.
2015-07-25 10:07:40 +08:00
95b499a367
HDFS-8798. Erasure Coding: fix DFSStripedInputStream/DFSStripedOutputStream re-fetch token when expired. Contributed by Walter Su.
2015-07-24 13:52:50 -07:00
f8f6091823
HDFS-8735. Inotify: All events classes should implement toString() API. Contributed by Surendra Singh Lilhore.
2015-07-25 02:56:55 +09:00
206d4933a5
HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang.
2015-07-24 18:28:44 +09:00
02c01815ec
HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka)
2015-07-24 11:37:23 +09:00
c2c26e6ea7
HDFS-8813. Erasure Coding: Client no need to decode missing parity blocks. Contributed by Walter Su.
2015-07-23 11:42:02 -07:00
813cf89bb5
HDFS-8730. Clean up the import statements in ClientProtocol. Contributed by Takanobu Asanuma.
2015-07-23 10:31:11 -07:00
adfa34ff99
HADOOP-12161. Add getStoragePolicy API to the FileSystem interface. (Contributed by Brahma Reddy Battula)
2015-07-23 10:13:04 -07:00
e91ccfad07
HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao.
2015-07-22 17:42:31 -07:00
4025326288
HDFS-8795. Improve InvalidateBlocks#node2blocks. (yliu)
2015-07-22 15:16:50 +08:00
31f117138a
HDFS-8495. Consolidate append() related implementation into a single class. Contributed by Rakesh R.
2015-07-21 17:25:23 -07:00
5956d23b64
HDFS-8781. Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block. Contributed by Yi Liu.
2015-07-21 17:00:49 -07:00
f8f7a923b7
HDFS-8760. Erasure Coding: reuse BlockReader when reading the same block in pread. Contributed by Jing Zhao.
2015-07-21 15:47:26 -07:00
a26cc66f38
HDFS-8773. Few FSNamesystem metrics are not documented in the Metrics page. Contributed by Rakesh R.
2015-07-21 14:12:03 -07:00
cb03768b1b
HDFS-8721. Add a metric for number of encryption zones. Contributed by Rakesh R.
2015-07-21 13:55:58 -07:00
5137b388fc
Revert "HDFS-8344. NameNode doesn't recover lease for files with missing blocks (raviprak)"
...
This reverts commit e4f756260f
2015-07-21 11:29:35 -07:00
c9507fe6c1
HDFS-8800. hdfs --daemon stop namenode corrupts logs (John Smith via aw)
2015-07-21 09:41:28 -07:00
29cf887b22
HDFS-7582. Enforce maximum number of ACL entries separately per access and default. (Contributed by Vinayakumar B)
2015-07-21 15:16:52 +05:30
df1e8ce44a
HDFS-7483. Display information per tier on the Namenode UI. Contributed by Benoy Antony and Haohui Mai.
2015-07-20 20:13:12 -07:00
a628f67590
Move HDFS-6945 to 2.7.2 section in CHANGES.txt.
2015-07-21 11:45:00 +09:00
d6d58606b8
HDFS-8794. Improve CorruptReplicasMap#corruptReplicasMap. (yliu)
2015-07-21 09:20:22 +08:00
ed01dc70b2
HDFS-8657. Update docs for mSNN. Contributed by Jesse Yates.
2015-07-20 16:40:06 -07:00
e4f756260f
HDFS-8344. NameNode doesn't recover lease for files with missing blocks (raviprak)
2015-07-20 14:03:34 -07:00
06394e3760
HDFS-8433. Erasure coding: set blockToken in LocatedStripedBlock. Contributed by Walter Su.
2015-07-20 10:18:34 +08:00
ee36f4f9b8
HADOOP-10979. Auto-entries in hadoop_usage (aw)
2015-07-16 16:58:11 -07:00
0bda84fd48
Move HDFS-7314 to 2.8 section in CHANGES.txt
2015-07-16 12:52:27 -07:00
fbd88f1062
HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma)
2015-07-16 12:33:57 -07:00
4fdd9abd7e
HDFS-8787. Erasure coding: rename BlockInfoContiguousUC and BlockInfoStripedUC to be consistent with trunk.
2015-07-15 20:13:04 -07:00
3ec0a0444f
HDFS-8778. TestBlockReportRateLimiting#testLeaseExpiration can deadlock. (Contributed by Arpit Agarwal)
2015-07-15 14:08:58 -07:00
7e091de136
HDFS-8058. Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile. Contributed by Zhe Zhang and Yi Liu.
2015-07-15 09:49:32 -07:00
f32d9a1758
HDFS-8619. Erasure Coding: revisit replica counting for striped blocks. (Jing Zhao via yliu)
2015-07-15 22:35:19 +08:00
0a93712f3b
HDFS-8734. Erasure Coding: fix one cell need two packets. Contributed by Walter Su.
2015-07-14 13:10:51 -07:00
979c9ca2ca
HDFS-8742. Inotify: Support event for OP_TRUNCATE. Contributed by Surendra Singh Lilhore.
2015-07-15 04:42:08 +09:00
59388a8015
HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee
2015-07-14 14:04:06 -05:00
b7fb6ec451
HDFS-7608: add CHANGES.txt
2015-07-14 10:58:57 -07:00
1d74ccecec
HDFS-7608: hdfs dfsclient newConnectedPeer has no write timeout (Xiaoyu Yao via Colin P. McCabe)
2015-07-14 10:57:59 -07:00
6ff957be88
HDFS-8702. Erasure coding: update BlockManager.blockHasEnoughRacks(..) logic for striped block. Contributed by Kai Sasaki.
2015-07-14 10:55:58 -07:00
9ef03a4c5b
HDFS-8541. Mover should exit with NO_MOVE_PROGRESS if there is no move progress. Contributed by Surendra Singh Lilhore
2015-07-13 15:12:26 -07:00
f7c8311e98
Add HDFS-8143 to CHANGES.txt.
2015-07-13 14:59:45 -07:00
b1e6429a6b
HDFS-8669. Erasure Coding: handle missing internal block locations in DFSStripedInputStream. Contributed by Jing Zhao.
2015-07-13 11:41:18 -07:00
f4098dfae4
HDFS-8744. Erasure Coding: the number of chunks in packet is not updated when writing parity data. Contributed by Li Bo
2015-07-13 00:41:36 -04:00
47f4c54106
HDFS-8751. Remove setBlocks API from INodeFile and misc code cleanup. Contributed by Zhe Zhang
2015-07-10 14:15:22 -07:00
b48908033f
HDFS-2956. calling fetchdt without a --renewer argument throws NPE (Contributed by Vinayakumar B)HDFS-2956. calling fetchdt without a --renewer argument throws NPE (Contributed by Vinayakumar B)
2015-07-10 15:47:04 +05:30
d66302ed9b
HDFS-8749. Fix findbugs warnings in BlockManager.java. Contributed by Brahma Reddy Battula.
2015-07-10 15:04:06 +09:00
e692c7dd92
HDFS-8484. Erasure coding: Two contiguous blocks occupy IDs belong to same striped group. Contributed by Walter Su.
2015-07-09 14:07:37 -07:00
f4ca530c1c
HDFS-8729. Fix TestFileTruncate#testTruncateWithDataNodesRestartImmediately which occasionally failed. Contributed by Walter Su.
2015-07-09 13:17:52 -07:00
2470a7bf88
HDFS-8563. Erasure Coding: fsck handles file smaller than a full stripe. Contributed by Walter Su.
2015-07-09 10:55:28 -07:00
48f3830f21
HDFS-8719. Erasure Coding: client generates too many small packets when writing parity data. Contributed by Li Bo.
2015-07-09 16:18:05 +08:00
42a1d0dbde
Addendum fix for HDFS-8323 (adding back a mistakenly deleted line).
2015-07-08 22:52:09 -07:00
fc6182d5ed
HDFS-8726. Move protobuf files that define the client-sever protocols to hdfs-client. Contributed by Haohui Mai.
2015-07-08 10:37:10 -07:00
4119ad3112
HDFS-8642. Make TestFileTruncate more reliable. (Contributed by Rakesh R)
2015-07-08 09:31:02 -07:00
bd4e10900c
HDFS-8712. Remove 'public' and 'abstract' modifiers in FsVolumeSpi and FsDatasetSpi (Contributed by Lei (Eddy) Xu)
2015-07-08 16:34:54 +05:30
c0b8e4e5b5
HDFS-8620. Clean up the checkstyle warinings about ClientProtocol. Contributed by Takanobu Asanuma.
2015-07-07 14:01:19 -07:00
b68701b7b2
HDFS-8711. setSpaceQuota command should print the available storage type when input storage type is wrong. Contributed by Brahma Reddy Battula.
2015-07-07 13:50:49 -07:00
bc99aaffe7
Revert "HDFS-8652. Track BlockInfo instead of Block in CorruptReplicasMap. Contributed by Jing Zhao."
...
This reverts commit d62b63d297
2015-07-07 10:13:41 -07:00
bf89ddb9b8
Release process for 2.7.1: Set the release date for 2.7.1.
2015-07-06 16:39:59 -07:00
2c494a8436
HDFS-8461. Erasure coding: fix priority level of UnderReplicatedBlocks for striped block. Contributed by Walter Su.
2015-07-06 16:39:47 -07:00
d62b63d297
HDFS-8652. Track BlockInfo instead of Block in CorruptReplicasMap. Contributed by Jing Zhao.
2015-07-06 15:54:25 -07:00
ee01a09500
HDFS-8684. Erasure Coding: fix some block number calculation for striped block. (yliu)
2015-07-06 19:14:18 +08:00
fc92d3e651
HDFS-8686. WebHdfsFileSystem#getXAttr(Path p, final String name) doesn't work if namespace is in capitals (Contributed by kanaka kumar avvaru)
2015-07-06 16:09:24 +05:30
2eae130ab9
HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina)
2015-07-03 13:35:48 +05:30
5fddc5177d
HDFS-8709. Clarify automatic sync in FSEditLog#logEdit.
2015-07-02 10:26:40 -07:00
bff5999d07
HDFS-8703. Merge refactor of DFSInputStream from ErasureCoding branch (Contributed by Vinayakumar B)
2015-07-02 16:11:50 +05:30
a78d5074fb
HADOOP-12171. Shorten overly-long htrace span names for server (cmccabe)
2015-07-01 17:57:11 -07:00
152e5df3b6
HDFS-8666. Speedup the TestMover test. Contributed by Walter Su.
2015-07-01 14:59:50 -07:00
ec0dc71cab
HDFS-8706. Fix typo in datanode startup options in HDFSCommands.html. (Controibuted by Brahma Reddy Battula)
2015-07-01 10:36:04 -07:00
68e588cbee
HADOOP-10798. globStatus() should always return a sorted list of files (cmccabe)
2015-06-30 16:40:21 -07:00
29df73dc05
HDFS-8635. Migrate HDFS native build to new CMake framework (Alan Burlison via Colin P. McCabe)
2015-06-30 15:54:38 -07:00
7ba5bbac02
HDFS-8627. NPE thrown if unable to fetch token from Namenode (Contributed by J.Andreina)
2015-06-30 15:42:59 +05:30
6d99017f38
HADOOP-12154. FileSystem#getUsed() returns the file length only from root '/' (Contributed by J.Andreina)
2015-06-30 15:25:20 +05:30
77588e1d32
HDFS-8579. Update HDFS usage with missing options (Contributed by J.Andreina)
2015-06-30 15:00:22 +05:30
0b7af27b9a
HDFS-8468. 2 RPC calls for every file read in DFSClient#open(..) resulting in double Audit log entries (Contributed by Vinayakumar B)
2015-06-30 11:02:13 +05:30
d3797f9f3c
HDFS-8493. Consolidate truncate() related implementation in a single class. Contributed by Rakesh R.
2015-06-29 16:45:35 -07:00
8e333720e1
HDFS-8687. Remove the duplicate usage message from Dfsck.java. (Contributed by Brahma Reddy Battula)
2015-06-29 14:56:42 -07:00
43a1288fff
HDFS-8659. Block scanner INFO message is spamming logs. Contributed by Yongjun Zhang.
2015-06-29 14:43:56 -07:00
2ffd84273a
HDFS-8653. Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo. Contributed by Zhe Zhang.
2015-06-29 12:12:41 -07:00
d3fed8e653
HDFS-7390. Provide JMX metrics per storage type. (Benoy Antony)
2015-06-29 11:00:22 -07:00
fde20ffcef
HDFS-8628. Update missing command option for fetchdt (Contributed by J.Andreina)
2015-06-29 16:50:48 +05:30
88ceb382ef
HDFS-8586. Dead Datanode is allocated for write when client is from deadnode (Contributed by Brahma Reddy Battula)
2015-06-29 15:25:03 +05:30
aad6a7d5db
Adding release 2.7.2 to CHANGES.txt.
2015-06-28 16:29:12 -07:00
c6793dd8cc
HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal)
2015-06-28 14:51:36 -07:00
60b858bfa6
HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization.
2015-06-26 11:30:59 -07:00
de480d6c89
HDFS-8623. Refactor NameNode handling of invalid, corrupt, and under-recovery blocks. Contributed by Zhe Zhang.
2015-06-26 10:49:01 -07:00
1b764a01fd
HDFS-8651. Make hadoop-hdfs-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe)
2015-06-26 10:22:40 -07:00
1403b84b12
HDFS-8546. Use try with resources in DataStorage and Storage.
2015-06-25 17:50:32 -07:00
ff0e5e572f
HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication.
2015-06-25 17:29:24 -07:00
67a62da5c5
HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru)
2015-06-25 10:13:22 -07:00
bc433908d3
HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N.
2015-06-26 00:20:12 +09:00
afe9ea3c12
HDFS-8646. Prune cached replicas from DatanodeDescriptor state on replica invalidation.
2015-06-24 14:42:33 -07:00
093907d728
HDFS-8253. DFSStripedOutputStream.closeThreads releases cellBuffers multiple times. Contributed by Kai Sasaki
2015-06-24 23:39:05 +08:00
d920cdab91
HDFS-8567. Erasure Coding: SafeMode handles file smaller than a full stripe. Contributed by Walter Su.
2015-06-23 22:29:28 -07:00
2ba6465721
HDFS-8639. Add Option for NameNode HTTP port in MiniDFSClusterManager. Contributed by Kai Sasaki.
2015-06-23 22:16:16 -07:00
49dfad9429
HDFS-6440. Support more than 2 NameNodes. Contributed by Jesse Yates.
2015-06-23 17:26:11 -07:00
122cad6aec
HDFS-6564. Use slf4j instead of common-logging in hdfs-client. Contributed by Rakesh R.
2015-06-23 11:41:40 -07:00
fac4e04dd3
HDFS-8542. WebHDFS getHomeDirectory behavior does not match specification. Contributed by Kanaka Kumar Avvaru.
2015-06-22 16:30:45 -07:00
7b424f938c
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe)
2015-06-22 14:37:10 -07:00
5590e914f5
Move HDFS-4366 to 2.8.0 in CHANGES.txt
2015-06-22 14:18:55 -07:00
c7d022b66f
HDFS-8192. Eviction should key off used locked memory instead of ram disk free space. (Contributed by Arpit Agarwal)
2015-06-20 13:27:52 -07:00
658b5c84ae
HDFS-7164. Update CHANGES.txt
2015-06-20 13:08:52 -07:00
bcb3c40bed
Move HDFS-7546 to release 2.7.1 in CHANGES.txt.
2015-06-20 00:41:35 -07:00
20c03c9606
Move HDFS-8337 to branch 2.8 in CHANGES.txt.
2015-06-20 00:25:33 -07:00
971dc838ec
HDFS-8337. Accessing httpfs via webhdfs doesn't work from a jar with kerberos. Contributed by Yongjun Zhang.
2015-06-19 23:39:13 -07:00
d112d18324
HDFS-8582. Support getting a list of reconfigurable config properties and do not generate spurious reconfig warnings (Lei (Eddy) Xu via Colin P. McCabe)
2015-06-19 19:14:05 -07:00
55278c012f
HDFS-8626. Reserved RBW space is not released if creation of RBW File fails. (Contributed by kanaka kumar avvaru)
2015-06-19 18:23:34 -07:00
8c423a8163
HDFS-8602. Erasure Coding: Client can't read(decode) the EC files which have corrupt blocks. Contributed by Jing Zhao and Kai Sasaki.
2015-06-19 14:07:38 -07:00
448cb7df67
HDFS-8543. Erasure Coding: processOverReplicatedBlock() handles striped block. Contributed by Walter Su.
2015-06-19 11:53:05 -07:00
3682e01984
HDFS-8254. Avoid assigning a leading streamer in StripedDataStreamer to tolerate datanode failure. Contributed by Tsz Wo Nicholas Sze.
2015-06-19 10:23:45 -07:00
b42f1ec13d
HDFS-8633. Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys. Contributed by Ray Chiang.
2015-06-19 09:47:31 -07:00
1c13519e1e
HDFS-8605. Merge Refactor of DFSOutputStream from HDFS-7285 branch. (vinayakumarb)
2015-06-18 08:48:09 -07:00
05c696882e
HDFS-8466. Refactor BlockInfoContiguous and fix NPE in TestBlockInfo#testCopyConstructor() (Contributed by Vinayakumar B)
...
Missed CHANGES.txt entry
2015-06-18 14:19:49 +05:30
62c5a879ef
HDFS-8466. Refactor BlockInfoContiguous and fix NPE in TestBlockInfo#testCopyConstructor() (Contributed by Vinayakumar B)
2015-06-18 14:15:01 +05:30
cc432885ad
HDFS-6249. Output AclEntry in PBImageXmlWriter. Contributed by surendra singh lilhore.
2015-06-17 17:41:10 -07:00
1a169a26bc
HDFS-8615. Correct HTTP method in WebHDFS document. Contributed by Brahma Reddy Battula.
2015-06-17 17:13:02 -07:00
45ced38f10
HDFS-8589. Fix unused imports in BPServiceActor and BlockReportLeaseManager (cmccabe)
2015-06-17 17:01:42 -07:00
015535dc0a
HDFS-8446. Separate safemode related operations in GetBlockLocations(). Contributed by Haohui Mai.
2015-06-17 16:38:43 -07:00
2de586f60d
Update CHANGES.txt for HDFS-8238.
2015-06-17 16:19:45 -07:00
b832774488
HDFS-8238. Move ClientProtocol to the hdfs-client. Contributed by Takanobu Asanuma.
2015-06-17 16:16:36 -07:00
c12a974ccf
HDFS-8580. Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature. Contributed by Walter Su.
2015-06-17 11:35:31 -07:00
6e3fcffe29
HDFS-8608. Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks). Contributed by Zhe Zhang.
2015-06-17 08:05:44 -07:00
5dbc8c9cb0
HDFS-7164. Feature documentation for HDFS-6581. (Contributed by Arpit Agarwal)
2015-06-16 19:05:44 -07:00
d4929f448f
HDFS-8606. Cleanup DFSOutputStream by removing unwanted changes from HDFS-8386. Contributed by Rakesh R
2015-06-16 18:08:29 -07:00
9cd5ad9d84
HDFS-8551. Fix hdfs datanode CLI usage message. Contributed by Brahma Reddy Battula.
2015-06-16 14:12:45 -07:00
6a76250b39
HDFS-8548. Minicluster throws NPE on shutdown. Contributed by surendra singh lilhore.
2015-06-16 13:57:02 -07:00
c74517c46b
HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee.
2015-06-16 15:40:16 -05:00
4e88ff5b27
HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao.
2015-06-16 12:38:07 -07:00
a3990ca414
HDFS-8591. Remove support for deprecated configuration key dfs.namenode.decommission.nodes.per.interval.
2015-06-16 10:03:34 -07:00
32e39d8a29
HDFS-8592. SafeModeException never get unwrapped. Contributed by Haohui Mai.
2015-06-15 22:07:24 -07:00
175e6d120f
HDFS-8361. Choose SSD over DISK in block placement.
2015-06-15 17:12:01 -07:00
1b6695a4c0
Move HDFS-8540 to 2.8 in CHANGES.txt.
2015-06-15 17:01:55 -07:00
321940cf19
HDFS-8540. Mover should exit with NO_MOVE_BLOCK if no block can be moved. Contributed by surendra singh lilhore
2015-06-15 16:26:53 -07:00
2cb09e98e3
HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina
2015-06-15 16:07:38 -07:00
32ffda1266
HDFS-8607. TestFileCorruption doesn't work as expected. (Contributed by Walter Su)
2015-06-15 10:11:53 -07:00
4c5da9bd94
HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal)
2015-06-14 16:24:30 -07:00
b0dc291961
HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang.
2015-06-14 11:39:31 -07:00
49d5cff490
HDFS-8559. Erasure Coding: fix non-protobuf fsimage for striped blocks. (Jing Zhao via yliu)
2015-06-14 15:39:19 +08:00
b8341f1cd8
HDFS-8593. Calculation of effective layout version mishandles comparison to current layout version in storage. Contributed by Chris Nauroth.
2015-06-13 10:37:52 -07:00
683332b36d
HDFS-8585. Erasure Coding: Remove dataBlockNum and parityBlockNum from StripedBlockProto. Contributed by Yi Liu.
2015-06-12 14:48:53 -07:00
eef7b50e23
HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai.
2015-06-12 14:45:34 -07:00
c17439c2dd
HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang.
2015-06-12 11:38:39 -07:00
46b0b4179c
Add HDFS-7923 to CHANGES.txt
2015-06-12 11:28:18 -07:00
12b5b06c06
HDFS-7923. The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages (cmccabe)
2015-06-12 11:17:51 -07:00
e4489d97e5
HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to hadoop-common. Contributed by Masatake Iwasaki.
2015-06-12 07:25:15 -07:00
83e8110f87
HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal)
2015-06-11 15:08:56 -07:00
587309bbba
HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe)
2015-06-11 14:26:22 -07:00
b258b344bb
HDFS-8573. Move creation of restartMeta file logic from BlockReceiver to ReplicaInPipeline. Contributed by Eddy Xu.
2015-06-11 10:12:31 -07:00
98d340745b
HDFS-8450. Erasure Coding: Consolidate erasure coding zone related implementation into a single class (Contributed by Rakesh R)
2015-06-11 10:48:02 +05:30
b7d6ea8e91
HDFS-8571. Fix TestErasureCodingCli test. Contributed by Vinayakumar B.
2015-06-11 10:26:07 +08:00
a7a7768341
HDFS-8549. Abort the balancer if an upgrade is in progress.
2015-06-10 13:42:57 -07:00
c5d4652ded
HDFS-8556. Erasure Coding: Fix usage of 'createZone' (Contributed by Vinayakumar B)
2015-06-10 10:42:45 +05:30
e299fe86b8
HDFS-8557 Allow to configure RS and XOR raw coders. Contributed by Kai Zheng
2015-06-10 15:35:26 +08:00
3107434031
HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal)
2015-06-09 20:58:39 -07:00
3c2397cb04
HDFS-8568. TestClusterId#testFormatWithEmptyClusterIdOption is failing. Contributed by Rakesh R.
2015-06-09 12:16:24 -07:00
927577c87c
HDFS-8552. Fix hdfs CLI usage message for namenode and zkfc. Contributed by Brahma Reddy Battula
2015-06-08 21:57:26 -07:00
d2832b3d42
HDFS-8553. Document hdfs class path options. Contributed by Brahma Reddy Battula.
2015-06-08 16:15:38 -07:00
0e80d51983
HDFS-8554. TestDatanodeLayoutUpgrade fails on Windows. Contributed by Chris Nauroth.
2015-06-08 08:39:02 -07:00
a6cb4894b4
HDFS-8539. Hdfs doesnt have class debug in windows. Contributed by Anu Engineer.
2015-06-07 13:01:43 -07:00
b61b489995
HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula.
2015-06-06 18:47:45 -07:00
71de367c5e
HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth.
2015-06-06 09:43:47 -07:00
e46cb80002
HADOOP-12055. Deprecate usage of NativeIO#link. Contributed by Andrew Wang.
2015-06-06 09:17:03 -07:00
3841d09765
HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. (Contributed by Xiaoyu Yao)
2015-06-05 15:09:24 -07:00
2eee19cd1d
HDFS-8460. Erasure Coding: stateful read result doesn't match data occasionally because of flawed test. Contributed by Walter Su.
2015-06-05 13:39:37 -07:00
c0929ab3c2
HDFS-8319. Erasure Coding: support decoding for stateful read. Contributed by Jing Zhao.
2015-06-04 18:16:41 -07:00
b2540f486e
HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all block replicas. Contributed by Eddy Xu.
2015-06-04 15:35:07 -07:00
5149dc7b97
HDFS-8532. Make the visibility of DFSOutputStream#streamer member variable to private. Contributed by Rakesh R.
2015-06-04 11:09:19 -07:00
ade6d9a61e
HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki.
2015-06-04 12:51:00 -05:00
a31eada33a
HDFS-8328. Follow-on to update decode for DataNode striped blocks reconstruction. (yliu)
2015-06-04 14:41:38 +08:00
673280df24
HDFS-7621. Erasure Coding: update the Balancer/Mover data migration logic. Contributed by Walter Su.
2015-06-03 11:51:58 -07:00
c59e745630
HDFS-3716. Purger should remove stale fsimage ckpt files (Contributed by J.Andreina)
2015-06-03 15:30:40 +05:30
444a4510c8
HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina)
...
Merged CHANGES.txt from branch-2.7
(cherry picked from commit 0ed9c2d8fedd98cfd328
2015-06-03 15:20:42 +05:30
e965dcec37
HDFS-8470. fsimage loading progress should update inode, delegation token and cache pool count. (Contributed by surendra singh lilhore)
2015-06-03 14:24:55 +05:30
54f83d9bd9
HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina)
2015-06-03 12:11:46 +05:30
806e407ac8
HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe)
2015-06-02 20:06:28 -07:00
5f15084bd5
HDFS-8453. Erasure coding: properly handle start offset for internal blocks in a block group. Contributed by Zhe Zhang.
2015-06-02 16:14:08 -07:00
c1d50a91f7
HDFS-8513. Rename BlockPlacementPolicyRackFaultTolarent to BlockPlacementPolicyRackFaultTolerant. (wang)
2015-06-02 15:48:26 -07:00
efc510a570
HDFS-8386. Improve synchronization of 'streamer' reference in DFSOutputStream. Contributed by Rakesh R.
2015-06-02 15:39:24 -07:00
71329e817b
HDFS-8517. Fix a decoding issue in stripped block recovering in client side. Contributed by Kai Zheng.
2015-06-02 15:35:49 -07:00
03fb5c6425
HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe)
2015-06-02 11:40:37 -07:00
2d847e7d62
HDFS-8444. Erasure Coding: fix cannot rename a zone dir (Contributed by Walter Su)
2015-06-02 16:41:19 +05:30
9b54e66f3e
HDFS-8336. Expose some administrative erasure coding operations to HdfsAdmin (Contributed by Uma Maheswara Rao G)
2015-06-02 16:38:43 +05:30
a2bd6217eb
HDFS-8256. '-storagepolicies , -blockId ,-replicaDetails ' options are missed out in usage and from documentation (Contributed by J.Andreina)
2015-06-02 16:07:18 +05:30
cdc13efb1a
HDFS-8489. Subclass BlockInfo to represent contiguous blocks. Contributed by Zhe Zhang.
2015-06-01 11:42:58 -07:00
014bd32c58
HDFS-8481. Erasure coding: remove workarounds in client side stripped blocks recovering. Contributed by Zhe Zhang.
2015-06-01 09:37:29 -07:00
c2356cb0bc
HDFS-8490. Typo in trace enabled log in ExceptionHandler of WebHDFS. Contributed by Archana T.
2015-06-01 00:41:50 +09:00
7673d4f205
HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang)
2015-05-29 14:54:18 -07:00
7817674a3a
HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma.
2015-05-29 11:05:13 -07:00
b75df697e0
HDFS-7401. Add block info to DFSInputStream' WARN message when it adds node to deadNodes (Contributed by Arshad Mohammad)
2015-05-29 16:36:19 +05:30
d725dd8af6
HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina.
2015-05-29 11:36:08 +09:00
ae14543420
HADOOP-12022. fix site -Pdocs -Pdist in hadoop-project-dist; cleanout remaining forrest bits (aw)
2015-05-28 13:24:01 -07:00
a1140959da
HADOOP-11894. Bump the version of Apache HTrace to 3.2.0-incubating (Masatake Iwasaki via Colin P. McCabe)
2015-05-28 12:00:55 -07:00
d2d95bfe88
HDFS-8407. libhdfs hdfsListDirectory must set errno to 0 on success (Masatake Iwasaki via Colin P. McCabe)
2015-05-28 11:38:06 -07:00
50eeea1300
HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer.
2015-05-27 22:54:10 -07:00
bf500d9798
HDFS-5033. Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw)
2015-05-27 16:51:34 -07:00
4928f54733
HDFS-8482. Rename BlockInfoContiguous to BlockInfo. Contributed by Zhe Zhang.
2015-05-27 15:42:07 -07:00
c46d4bafe1
Update CHANGES.txt for HDFS-8135.
2015-05-27 11:07:15 -07:00
bb1816328a
HADOOP-11242. Record the time of calling in tracing span of IPC server. Contributed by Mastake Iwasaki.
2015-05-27 18:04:40 +09:00
1299357a05
HDFS-8479. Erasure coding: fix striping related logic in FSDirWriteFileOp to sync with HDFS-8421. Contributed by Zhe Zhang.
2015-05-26 16:06:50 -07:00
c9e0268216
Addendum fix for HDFS-7912.
2015-05-26 12:32:16 -07:00
80c56c2d31
HDFS-8408. Revisit and refactor ErasureCodingInfo (Contributed by Vinayakumar B)
...
Added missed file
2015-05-26 12:07:15 -07:00
9a18598e2d
HDFS-8408. Revisit and refactor ErasureCodingInfo (Contributed by Vinayakumar B)
2015-05-26 12:07:15 -07:00
b30e96bfb4
HDFS-8382 Remove chunkSize and initialize from erasure coder. Contributed by Kai Zheng
2015-05-26 12:07:15 -07:00
0ed92e5b13
HDFS-7768. Change fsck to support EC files. Contributed by Takanobu Asanuma
2015-05-26 12:07:15 -07:00
3d734df24c
HDFS-8441. Erasure Coding: make condition check earlier for setReplication. (waltersu4549)
2015-05-26 12:07:09 -07:00
7af05a3db4
HDFS-8294. Erasure Coding: Fix Findbug warnings present in erasure coding. Contributed by Rakesh R.
2015-05-26 12:02:34 -07:00
e53fa769c9
HDFS-8186. Erasure coding: Make block placement policy for EC file configurable. Contributed by Walter Su.
2015-05-26 12:02:34 -07:00
744ef17792
HDFS-8427. Remove dataBlockNum and parityBlockNum from BlockInfoStriped. Contributed by Kai Sasaki.
2015-05-26 12:02:34 -07:00
45db1a0b72
HDFS-8323. Bump GenerationStamp for write faliure in DFSStripedOutputStream. Contributed by Tsz Wo Nicholas Sze.
2015-05-26 12:02:34 -07:00
47ef869fa7
HDFS-8428. Erasure Coding: Fix the NullPointerException when deleting file. Contributed by Yi Liu.
2015-05-26 12:02:33 -07:00
91c81fdc24
HDFS-8375. Add cellSize as an XAttr to ECZone. Contributed by Vinayakumar B.
2015-05-26 12:02:33 -07:00
914580934c
HDFS-8378. Erasure Coding: Few improvements for the erasure coding worker. Contributed by Rakesh R.
2015-05-26 12:02:33 -07:00
8c95673db4
HDFS-8366. Erasure Coding: Make the timeout parameter of polling blocking queue configurable in DFSStripedOutputStream. Contributed by Li Bo
2015-05-26 12:02:32 -07:00
7434c44b16
HDFS-8320. Erasure coding: consolidate striping-related terminologies. Contributed by Zhe Zhang and Jing Zhao.
2015-05-26 12:02:32 -07:00
b008348dbf
HDFS-8418. Fix the isNeededReplication calculation for Striped block in NN. Contributed by Yi Liu.
2015-05-26 12:02:32 -07:00
6c310db159
HDFS-8417. Erasure Coding: Pread failed to read data starting from not-first stripe. Contributed by Walter Su.
2015-05-26 12:02:32 -07:00
a919726914
HDFS-8352. Erasure Coding: test webhdfs read write stripe file. (waltersu4549)
2015-05-26 12:02:30 -07:00
c9103e9cac
HDFS-8367 BlockInfoStriped uses EC schema. Contributed by Kai Sasaki
2015-05-26 12:02:30 -07:00
d8ea443af0
Merge HDFS-8394 from trunk: Move getAdditionalBlock() and related functionalities into a separate class.
2015-05-26 12:02:26 -07:00
c99c337928
HDFS-8391. NN should consider current EC tasks handling count from DN while assigning new tasks. Contributed by Uma Maheswara Rao G.
2015-05-26 12:01:51 -07:00
bba15e06d2
HDFS-8364. Erasure coding: fix some minor bugs in EC CLI (Contributed by Walter Su)
2015-05-26 12:01:51 -07:00
9798065cbb
HDFS-8195. Erasure coding: Fix file quota change when we complete/commit the striped blocks. Contributed by Takuya Fukudome.
2015-05-26 12:01:50 -07:00
8da9e18dad
HDFS-8363. Erasure Coding: DFSStripedInputStream#seekToNewSource. (yliu)
2015-05-26 12:01:50 -07:00
54d2827522
HDFS-8368. Erasure Coding: DFS opening a non-existent file need to be handled properly. Contributed by Rakesh R.
2015-05-26 12:01:50 -07:00
97a2396af6
HDFS-8372. Erasure coding: compute storage type quotas for striped files, to be consistent with HDFS-8327. Contributed by Zhe Zhang.
2015-05-26 12:01:50 -07:00
8d3030f064
HDFS-7678. Erasure coding: DFSInputStream with decode functionality (pread). Contributed by Zhe Zhang.
2015-05-26 12:01:49 -07:00
6bacaa9a52
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging trunk changes (this commit is for HDFS-8327 and HDFS-8357). Contributed by Zhe Zhang.
2015-05-26 12:01:49 -07:00
51ea117f88
HDFS-8355. Erasure Coding: Refactor BlockInfo and BlockInfoUnderConstruction. Contributed by Tsz Wo Nicholas Sze.
2015-05-26 12:01:49 -07:00
9da927540f
HDFS-8289. Erasure Coding: add ECSchema to HdfsFileStatus. Contributed by Yong Zhang.
2015-05-26 12:01:49 -07:00
a17cedb44c
HDFS-8203. Erasure Coding: Seek and other Ops in DFSStripedInputStream. Contributed by Yi Liu.
2015-05-26 12:01:48 -07:00
ac97edd1ab
HDFS-8129. Erasure Coding: Maintain consistent naming for Erasure Coding related classes - EC/ErasureCoding. Contributed by Uma Maheswara Rao G
2015-05-26 12:01:48 -07:00
cea46f79b0
HDFS-8334. Erasure coding: rename DFSStripedInputStream related test classes. Contributed by Zhe Zhang.
2015-05-26 12:01:48 -07:00
6616de24cb
HDFS-7348. Erasure Coding: DataNode reconstruct striped blocks. Contributed by Yi Liu.
2015-05-26 12:01:47 -07:00
220ca960bc
HDFS-7672. Handle write failure for stripping blocks and refactor the existing code in DFSStripedOutputStream and StripedDataStreamer.
2015-05-26 12:00:45 -07:00
e849be2d31
HDFS-8324. Add trace info to DFSClient#getErasureCodingZoneInfo(..). Contributed by Vinayakumar B
2015-05-26 11:59:58 -07:00
4392325546
HDFS-8242. Erasure Coding: XML based end-to-end test for ECCli commands (Contributed by Rakesh R)
2015-05-26 11:59:58 -07:00
436c14855a
HDFS-8137. Send the EC schema to DataNode via EC encoding/recovering command. Contributed by Uma Maheswara Rao G
2015-05-26 11:59:57 -07:00
6dea01f1ee
HDFS-8281. Erasure Coding: implement parallel stateful reading for striped layout. Contributed by Jing Zhao.
2015-05-26 11:59:57 -07:00
ea6c66ed57
HDFS-8316. Erasure coding: refactor EC constants to be consistent with HDFS-8249. Contributed by Zhe Zhang.
2015-05-26 11:59:57 -07:00
77d94dd5ec
HDFS-7949. WebImageViewer need support file size calculation with striped blocks. Contributed by Rakesh R.
2015-05-26 11:59:56 -07:00
5059958bf4
HDFS-8308. Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog. Contributed by Jing Zhao.
2015-05-26 11:59:56 -07:00
1a31f1c303
HDFS-8183. Erasure Coding: Improve DFSStripedOutputStream closing of datastreamer threads. Contributed by Rakesh R.
2015-05-26 11:59:56 -07:00
f0628280c3
HDFS-8282. Erasure coding: move striped reading logic to StripedBlockUtil. Contributed by Zhe Zhang.
2015-05-26 11:59:56 -07:00
b00c663877
HDFS-8272. Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read). Contributed by Jing Zhao
2015-05-26 11:59:55 -07:00
1a8139e6ad
Fix merge conflicts.
2015-05-26 11:59:55 -07:00
9593776e34
HDFS-8235. Erasure Coding: Create DFSStripedInputStream in DFSClient#open. Contributed by Kai Sasaki.
2015-05-26 11:59:55 -07:00
5eb17e5726
HDFS-8189. ClientProtocol#createErasureCodingZone API was wrongly annotated as Idempotent (Contributed by Vinayakumar B)
2015-05-26 11:59:55 -07:00
f9eb95c31d
HDFS-8230. Erasure Coding: Ignore DatanodeProtocol#DNA_ERASURE_CODING_RECOVERY commands from standbynode if any (Contributed by Vinayakumar B)
2015-05-26 11:59:54 -07:00
89d3378578
HDFS-8033. Erasure coding: stateful (non-positional) read from files in striped layout. Contributed by Zhe Zhang.
2015-05-26 11:59:54 -07:00
f5d4a95ef5
HDFS-8228. Erasure Coding: SequentialBlockGroupIdGenerator#nextValue may cause block id conflicts. Contributed by Jing Zhao.
2015-05-26 11:59:54 -07:00
3f2c6938f1
HDFS-8223. Should calculate checksum for parity blocks in DFSStripedOutputStream. Contributed by Yi Liu.
2015-05-26 11:59:54 -07:00
eb612b0b70
HDFS-8233. Fix DFSStripedOutputStream#getCurrentBlockGroupBytes when the last stripe is at the block group boundary. Contributed by Jing Zhao.
2015-05-26 11:59:53 -07:00
fcd54ecce2
HDFS-8136. Client gets and uses EC schema when reads and writes a stripping file. Contributed by Kai Sasaki
2015-05-26 11:59:53 -07:00
e8df2581c3
HDFS-8156. Add/implement necessary APIs even we just have the system default schema. Contributed by Kai Zheng.
2015-05-26 11:59:53 -07:00
014d8675c5
HDFS-8024. Erasure Coding: ECworker frame, basics, bootstraping and configuration. (Contributed by Uma Maheswara Rao G)
2015-05-26 11:59:52 -07:00
2f11109bb0
HDFS-8212. DistributedFileSystem.createErasureCodingZone should pass schema in FileSystemLinkResolver. Contributed by Tsz Wo Nicholas Sze.
2015-05-26 11:59:52 -07:00
e9d85bbf30
HDFS-8216. TestDFSStripedOutputStream should use BlockReaderTestUtil to create BlockReader. Contributed by Tsz Wo Nicholas Sze.
2015-05-26 11:59:52 -07:00
e107886d6f
HDFS-8190. StripedBlockUtil.getInternalBlockLength may have overflow error.
2015-05-26 11:59:52 -07:00
922631f04f
HDFS-8188. Erasure coding: refactor client-related code to sync with HDFS-8082 and HDFS-8169. Contributed by Zhe Zhang.
2015-05-26 11:59:51 -07:00
dfba46ab57
HDFS-8181. createErasureCodingZone sets retryCache state as false always (Contributed by Uma Maheswara Rao G)
2015-05-26 11:59:51 -07:00
a1c9425265
HDFS-8146. Protobuf changes for BlockECRecoveryCommand and its fields for making it ready for transfer to DN (Contributed by Uma Maheswara Rao G)
2015-05-26 11:59:51 -07:00
f6e1160ef1
HDFS-8145. Fix the editlog corruption exposed by failed TestAddStripedBlocks. Contributed by Jing Zhao.
2015-05-26 11:59:46 -07:00
ab76e1fe36
HDFS-7937. Erasure Coding: INodeFile quota computation unit tests. Contributed by Kai Sasaki.
2015-05-26 11:59:13 -07:00
cd458c38a0
HDFS-8166. DFSStripedOutputStream should not create empty blocks. Contributed by Jing Zhao.
2015-05-26 11:59:13 -07:00
64f8f0a145
HDFS-8167. BlockManager.addBlockCollectionWithCheck should check if the block is a striped block. Contributed by Hui Zheng.
2015-05-26 11:59:12 -07:00
4c039b0876
HDFS-7994. Detect if resevered EC Block ID is already used during namenode startup. Contributed by Hui Zheng
2015-05-26 11:59:12 -07:00
5e8837dd6c
HDFS-8120. Erasure coding: created util class to analyze striped block groups. Contributed by Zhe Zhang and Li Bo.
2015-05-26 11:59:12 -07:00
ceb3d1c170
HDFS-7349. Support DFS command for the EC encoding (Contributed by Vinayakumar B)
2015-05-26 11:59:11 -07:00
2c277802c1
HDFS-8027. Erasure Coding: Update CHANGES-HDFS-7285.txt with branch commits (Vinayakumar B)
2015-05-26 11:59:11 -07:00
04c23d95d0
HDFS-8123. Erasure Coding: Better to move EC related proto messages to a separate erasurecoding proto file (Contrubuted by Rakesh R)
2015-05-26 11:59:11 -07:00
3b56a5aa9e
HDFS-8114. Erasure coding: Add auditlog FSNamesystem#createErasureCodingZone if this operation fails. Contributed by Rakesh R.
2015-05-26 11:59:10 -07:00
a32c4dc38a
HDFS-8122. Erasure Coding: Support specifying ECSchema during creation of ECZone. Contributed by Vinayakumar B.
2015-05-26 11:59:06 -07:00
35797b0889
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging trunk changes (this commit mainly addresses HDFS-8081 and HDFS-8048. Contributed by Zhe Zhang.
2015-05-26 11:58:40 -07:00
f53e402635
HDFS-8090. Erasure Coding: Add RPC to client-namenode to list all ECSchemas loaded in Namenode. (Contributed by Vinayakumar B)
2015-05-26 11:58:40 -07:00
4970f2a2ab
HDFS-7889 Subclass DFSOutputStream to support writing striping layout files. Contributed by Li Bo
2015-05-26 11:58:40 -07:00
4d0bc724f2
HDFS-8077. Erasure coding: fix bugs in EC zone and symlinks. Contributed by Jing Zhao and Zhe Zhang.
2015-05-26 11:58:32 -07:00
c4a0d88366
HDFS-8104 Make hard-coded values consistent with the system default schema first before remove them. Contributed by Kai Zheng
2015-05-26 11:55:17 -07:00
1a37b0fe9a
HDFS-8074 Define a system-wide default EC schema. Contributed by Kai Zheng
2015-05-26 11:55:16 -07:00
2ca3e38feb
HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B)
...
Added missed file
2015-05-26 11:55:16 -07:00
16484f0030
HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B)
2015-05-26 11:55:16 -07:00
bff59392ee
HDFS-7782. Erasure coding: pread from files in striped layout. Contributed by Zhe Zhang and Jing Zhao
2015-05-26 11:55:15 -07:00
91c741a2a1
HDFS-7782. Erasure coding: pread from files in striped layout. Contributed by Zhe Zhang and Jing Zhao
2015-05-26 11:55:15 -07:00
aac73c21c3
HADOOP-11740. Combine erasure encoder and decoder interfaces. Contributed by Zhe Zhang.
...
Updated CHANGES-HDFS-EC-7285.txt
2015-05-26 11:55:15 -07:00
d25ca09cf6
HADOOP-11782 Correct two thrown messages in ECSchema class. Contributed by Xinwei Qin
...
Updated CHANGES-HDFS-EC-7285.txt
2015-05-26 11:55:14 -07:00
146ce7a978
HDFS-7969. Erasure coding: NameNode support for lease recovery of striped block groups. Contributed by Zhe Zhang.
2015-05-26 11:55:14 -07:00
c243319eab
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging trunk changes (this commit is for HDFS-8035). Contributed by Zhe Zhang
2015-05-26 11:55:13 -07:00
e54a74b566
HADOOP-11740. Combine erasure encoder and decoder interfaces. Contributed by Zhe Zhang.
2015-05-26 11:55:13 -07:00
1af8c14862
HDFS-7839. Erasure coding: implement facilities in NameNode to create and manage EC zones. Contributed by Zhe Zhang
2015-05-26 11:55:07 -07:00
578019d6a2
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging (this commit is for conflicts from HDFS-6945). Contributed by Zhe Zhang.
2015-05-26 11:44:47 -07:00
9edc73d123
HDFS-7617. Add unit tests for editlog transactions for EC. Contributed by Hui Zheng.
2015-05-26 11:44:47 -07:00
af8eaacdb4
HDFS-8027. Erasure Coding: Update CHANGES-HDFS-7285.txt with branch commits (Vinayakumar B)
2015-05-26 11:44:46 -07:00
cd655ee817
HDFS-8005. Erasure Coding: simplify striped block recovery work computation and add tests. Contributed by Jing Zhao.
2015-05-26 11:44:42 -07:00
abf833a7b2
HDFS-7907. Erasure Coding: track invalid, corrupt, and under-recovery striped blocks in NameNode. Contributed by Jing Zhao.
2015-05-26 11:43:44 -07:00
97378e4cd0
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging (this commit is for HDFS-7742)
2015-05-26 11:43:43 -07:00
9d1175b8fb
HDFS-7716. Add a test for BlockGroup support in FSImage. Contributed by Takuya Fukudome
2015-05-26 11:43:43 -07:00
ea2e60fbcc
HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903, HDFS-7435, HDFS-7930, HDFS-7960 (this commit is for HDFS-7960)
2015-05-26 11:43:42 -07:00
d0d75a8339
HDFS-7827. Erasure Coding: support striped blocks in non-protobuf fsimage. Contributed by Hui Zheng.
2015-05-26 11:43:42 -07:00
46dac3595f
HDFS-7864. Erasure Coding: Update safemode calculation for striped blocks. Contributed by GAO Rui.
2015-05-26 11:43:42 -07:00
544f75d651
HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903, HDFS-7435 and HDFS-7930 (this commit is for HDFS-7930 only)
2015-05-26 11:43:42 -07:00
57a84c0d14
HDFS-7369. Erasure coding: distribute recovery work for striped blocks to DataNode. Contributed by Zhe Zhang.
2015-05-26 11:43:37 -07:00
a38a37c634
HDFS-7912. Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks. Contributed by Jing Zhao.
2015-05-26 11:41:59 -07:00
26773d9d6c
HDFS-7826. Erasure Coding: Update INodeFile quota computation for striped blocks. Contributed by Kai Sasaki.
2015-05-26 11:32:26 -07:00
11585883a9
HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903 and HDFS-7435. Contributed by Zhe Zhang.
2015-05-26 11:32:26 -07:00
f05c21285e
HDFS-7853. Erasure coding: extend LocatedBlocks to support reading from striped files. Contributed by Jing Zhao.
2015-05-26 11:32:25 -07:00
39a0a85fb7
HDFS-7872. Erasure Coding: INodeFile.dumpTreeRecursively() supports to print striped blocks. Contributed by Takuya Fukudome.
2015-05-26 11:32:25 -07:00
1e1e930407
HDFS-7837. Erasure Coding: allocate and persist striped blocks in NameNode. Contributed by Jing Zhao.
2015-05-26 11:32:20 -07:00
9af1f4779b
Addendum fix for HDFS-7749 to be compatible with HDFS-7993
2015-05-26 11:07:16 -07:00
9f2f583f40
HDFS-7749. Erasure Coding: Add striped block support in INodeFile. Contributed by Jing Zhao.
2015-05-26 11:07:15 -07:00
ba93714920
HDFS-7716. Erasure Coding: extend BlockInfo to handle EC info. Contributed by Jing Zhao.
2015-05-26 11:07:11 -07:00
ec48013426
Fix Compilation Error in TestAddBlockgroup.java after the merge
2015-05-26 11:03:34 -07:00
0c1da5a030
HDFS-7652. Process block reports for erasure coded blocks. Contributed by Zhe Zhang
2015-05-26 11:03:34 -07:00
bc2833b1c9
HDFS-7339. Allocating and persisting block groups in NameNode. Contributed by Zhe Zhang
2015-05-26 11:03:34 -07:00
f166e67a23
HDFS-7347. Configurable erasure coding policy for individual files and directories ( Contributed by Zhe Zhang )
2015-05-26 11:03:34 -07:00
ada233b7cd
HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang.
2015-05-24 22:30:32 -07:00
55ed6558ba
HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin)
2015-05-22 16:45:19 +05:30
0c6638c2ea
HDFS-8268. Port conflict log for data node server is not sufficient (Contributed by Mohammad Shahid Khan)
2015-05-22 16:15:15 +05:30
cf2b5694d6
HDFS-8454. Remove unnecessary throttling in TestDatanodeDeath. (Arpit Agarwal)
2015-05-21 20:43:57 -07:00
05e04f34f2
HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran.
2015-05-21 11:58:00 -07:00
2b6bcfdafa
HDFS-8421. Move startFile() and related functions into FSDirWriteFileOp. Contributed by Haohui Mai.
2015-05-21 08:08:28 -07:00
0e4f1081c7
HADOOP-10366. Add whitespaces between classes for values in core-default.xml to fit better in browser. Contributed by kanaka kumar avvaru.
2015-05-21 17:52:03 +09:00
dc8434ab2b
HDFS-4383. Document the lease limits. Contributed by Arshad Mohammad.
2015-05-21 17:30:43 +09:00
8860e352c3
HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts.
2015-05-19 13:05:15 -05:00
470c87dbc6
HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe)
2015-05-19 10:50:15 -07:00
de30d66b26
HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui.
2015-05-19 08:04:38 -05:00
d4a2830b63
HADOOP-11103. Clean up RemoteException (Contributed by Sean Busbey)
2015-05-19 14:41:05 +05:30
93972a332a
HDFS-6348. SecondaryNameNode not terminating properly on runtime exceptions (Contributed by Rakesh R)
2015-05-19 12:24:25 +05:30
cdfae446ad
HDFS-4185. Add a metric for number of active leases (Rakesh R via raviprak)
2015-05-18 12:37:21 -07:00
0c590e1c09
HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma
2015-05-19 02:57:54 +08:00
a2190bf15d
HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal)
2015-05-18 11:36:29 -07:00
a6af0248e9
HDFS-8412. Fix the test failures in HTTPFS: In some tests setReplication called after fs close. Contributed by Uma Maheswara Rao G.
2015-05-18 19:35:53 +05:30
363c35541d
Updating CHANGES.txt for moving entry of HDFS-8332 from branch-2 to trunk
2015-05-18 14:48:49 +05:30
e453989a57
HDFS-8157. Writes to RAM DISK reserve locked memory for block files. (Arpit Agarwal)
2015-05-16 09:05:35 -07:00
ac742c762d
HDFS-8403. Eliminate retries in TestFileCreation#testOverwriteOpenForWrite. Contributed by Arpit Agarwal.
2015-05-15 19:12:40 -07:00
e5afac5896
HDFS-8394. Move getAdditionalBlock() and related functionalities into a separate class. Contributed by Haohui Mai.
2015-05-15 19:09:59 -07:00
8f37873342
HDFS-8397. Refactor the error handling code in DataStreamer. Contributed by Tsz Wo Nicholas Sze.
2015-05-15 16:14:54 -07:00
3bef7c80a9
HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He)
2015-05-15 11:07:51 +05:30
ee7beda6e3
HDFS-8350. Remove old webhdfs.xml and other outdated documentation stuff. Contributed by Brahma Reddy Battula.
2015-05-15 13:54:35 +09:00
cbc01ed08e
HDFS-8371. Fix test failure in TestHdfsConfigFields for spanreceiver properties. Contributed by Ray Chiang.
2015-05-15 12:14:47 +09:00
def9136e02
HDFS-8150. Make getFileChecksum fail for blocks under construction (Contributed by J.Andreina)
2015-05-14 15:54:51 +05:30
b2c85db86c
HDFS-7728. Avoid updating quota usage while loading edits. Contributed by Jing Zhao.
2015-05-13 21:50:35 -07:00
54fa9b4217
HDFS-8243. Files written by TestHostsFiles and TestNameNodeMXBean are causing Release Audit Warnings. (Contributed by Ruth Wisniewski)
2015-05-13 19:43:53 -07:00
281d47a969
HDFS-8380. Always call addStoredBlock on blocks which have been shifted from one storage to another (cmccabe)
2015-05-13 14:29:05 -07:00
f9a46a00d2
HDFS-8358. TestTraceAdmin fails. Contributed by Masatake Iwasaki.
2015-05-13 14:15:27 -05:00
cdec12d1b8
HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore
2015-05-13 11:57:49 -07:00
065d8f2a34
HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R)
2015-05-13 17:27:34 +05:30
2463666ecb
HDFS-6184. Capture NN's thread dump when it fails over. Contributed by Ming Ma.
2015-05-13 11:37:22 +09:00
6d5da94841
HDFS-8255. Rename getBlockReplication to getPreferredBlockReplication. (Contributed by Zhe Zhang)
2015-05-12 21:29:22 +08:00
987abc99b0
HDFS-8362. Java Compilation Error in TestHdfsConfigFields.java (Contributed by Arshad Mohammad)
2015-05-12 12:09:13 +05:30
ea11590aad
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah.
2015-05-11 14:31:58 -05:00
1dd79ffaca
HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula.
2015-05-12 00:18:18 +09:00
7e543c27fa
YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak.
2015-05-11 05:27:07 -07:00
3fa2efc09f
HDFS-8351. Remove namenode -finalize option from document. (aajisaka)
2015-05-11 15:34:44 +09:00
4536399d47
HDFS-8357. Consolidate parameters of INode.CleanSubtree() into a parameter objects. Contributed by Li Lu.
2015-05-09 22:51:58 -07:00
02a4a22b9c
HDFS-8327. Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate(). Contributed by Haohui Mai.
2015-05-08 23:09:25 -07:00
3becc3af83
Add missing entry in CHANGES.txt for HDFS-6757.
2015-05-08 23:08:05 -07:00
00fe1ed3a4
HDFS-6757. Simplify lease manager with INodeID. Contributed by Haohui Mai.
2015-05-08 23:04:31 -07:00
59995cec4a
HDFS-8097. TestFileTruncate is failing intermittently. (Contributed by Rakesh R)
2015-05-08 16:28:24 -07:00
c945c20483
HADOOP-11590. Update sbin commands and documentation to use new --slaves option (aw)
2015-05-08 15:27:25 -07:00
d0e75e60fb
HDFS-8326. Documentation about when checkpoints are run is out of date. (Misty Stanley-Jones via xyao)
2015-05-08 14:46:37 -07:00
2d4ae3d18b
HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah.
2015-05-08 16:36:41 -05:00
f523e963e4
HDFS-8113. Add check for null BlockCollection pointers in BlockInfoContiguous structures (Chengbing Liu via Colin P. McCabe)
2015-05-08 13:50:51 -07:00
7a7960be41
HDFS-7433. Optimize performance of DatanodeManager's node map. Contributed by Daryn Sharp.
2015-05-08 15:18:29 -05:00
8f7c2364d7
HDFS-8284. Update documentation about how to use HTrace with HDFS (Masatake Iwasaki via Colin P. McCabe)
2015-05-08 12:30:03 -07:00
730f9930a4
HDFS-8311. DataStreamer.transfer() should timeout the socket InputStream. (Esteban Gutierrez via Yongjun Zhang)
2015-05-08 00:11:46 -07:00
26f61d41df
HDFS-5640. Add snapshot methods to FileContext. Contributed by Rakesh R.
2015-05-08 11:20:55 -07:00
3cefc02af7
HDFS-7559. Create unit test to automatically compare HDFS related classes and hdfs-default.xml. (Ray Chiang via asuresh)
2015-05-08 10:29:38 -07:00
a2d40bced9
HDFS-8340. Fix NFS documentation of nfs.wtmax. (Contributed by Ajith S)
2015-05-08 10:15:26 -07:00
cd6b26cce7
HDFS-8274. NFS configuration nfs.dump.dir not working (Contributed by Ajith S)
2015-05-08 10:06:43 -07:00
f4ebbc6afc
HDFS-8346. libwebhdfs build fails during link due to unresolved external symbols. Contributed by Chris Nauroth.
2015-05-08 09:58:17 -07:00
6f622672b6
HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula.
2015-05-08 09:30:38 -05:00
7b1ea9c481
HDFS-6285. tidy an error log inside BlockReceiver. Contributed by Liang Xie.
2015-05-08 17:22:52 +05:30
c648317a68
HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G)
2015-05-08 17:18:14 +05:30
46509617b6
HDFS-6576. Datanode log is generating at root directory in security mode (Contributed by surendra singh lilhore)
...
Just merging CHANGES.txt changes from branch-2 to keep in sync.
(cherry picked from commit a87ab77fec
2015-05-08 16:44:03 +05:30
e01c1a979c
HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina)
2015-05-08 15:24:36 +05:30
4c6816faf8
HDFS-8209. Support different number of datanode directories in MiniDFSCluster. (Contributed by surendra singh lilhore)
2015-05-08 15:03:44 +05:30
888e630990
HDFS-8207. Improper log message when blockreport interval compared with initial delay. Contributed by Brahma Reddy Battula and Ashish Singhi.
2015-05-08 18:13:09 +09:00
6232235df3
HDFS-8175. Provide information on snapshotDiff for supporting the comparison between snapshot and current status (Contributed by J.Andreina)
2015-05-08 14:37:50 +05:30
e642eff0e0
HDFS-8187. Remove usage of '-setStoragePolicy' and '-getStoragePolicy' using dfsadmin cmd (as it is not been supported) (Contributed by J.Andreina)
2015-05-08 14:16:30 +05:30
4b00ae5c11
HDFS-8108. Fsck should provide the info on mandatory option to be used along with -blocks ,-locations and -racks. Contributed by J.Andreina.
2015-05-08 14:00:45 +05:30
fd605997ee
HDFS-8222. Remove usage of "dfsadmin -upgradeProgress" from document which is no longer supported. Contributed by J.Andreina.
2015-05-08 16:27:40 +09:00
26da2ddf38
HDFS-7998. HDFS Federation : Command mentioned to add a NN to existing federated cluster is wrong (Contributed by Ajith S)
2015-05-08 12:33:49 +05:30
c5afe444df
HDFS-8349. Remove .xml and documentation references to dfs.webhdfs.enabled. Contributed by Ray Chiang.
2015-05-08 15:57:13 +09:00
e16f4b7f70
HDFS-8332. DFS client API calls should check filesystem closed. Contributed by Rakesh R.
2015-05-08 12:26:47 +05:30
e5e492a963
HDFS-6291. FSImage may be left unclosed in BootstrapStandby#doRun() ( Contributed by Sanghyun Yun)
2015-05-08 11:25:24 +05:30
2ea0f2fc93
HDFS-8174. Update replication count to live rep count in fsck report. Contributed by J.Andreina
2015-05-08 11:01:51 +05:30
66988476d0
HDFS-8067. haadmin prints out stale help messages (Contributed by Ajith S)
2015-05-08 10:45:21 +05:30
c7c26a1e4a
HDFS-8257. Namenode rollingUpgrade option is incorrect in document (Contributed by J.Andreina)
2015-05-08 10:35:25 +05:30
4d9f9e546f
HDFS-8037. CheckAccess in WebHDFS silently accepts malformed FsActions parameters. Contributed by Walter Su.
2015-05-07 15:25:26 -07:00
767b91cd83
HDFS-8321. CacheDirectives and CachePool operations should throw RetriableException in safemode. Contributed by Haohui Mai.
2015-05-07 15:17:11 -07:00
f9427f1760
HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su
2015-05-07 11:36:35 -07:00
449e4426a5
HDFS-8325. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula.
2015-05-07 15:14:20 +09:00
6633a8474d
HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu.
2015-05-06 21:11:30 -07:00
c75cfa29cf
HDFS-2484. checkLease should throw FileNotFoundException when file does not exist. Contributed by Rakesh R.
2015-05-06 16:06:04 -07:00
7a26d174af
HDFS-8310. Fix TestCLI.testAll 'help: help for find' on Windows. (Kiran Kumar M R via Xiaoyu Yao)
2015-05-06 10:24:46 -07:00
4da8490b51
HDFS-8314. Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users. Contributed by Li Lu.
2015-05-05 15:41:22 -07:00
0100b15501
HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao)
2015-05-05 13:41:14 -07:00
ffce9a3413
HDFS-7847. Modify NNThroughputBenchmark to be able to operate on a remote NameNode (Charles Lamb via Colin P. McCabe)
2015-05-05 11:34:58 -07:00
24d3a2d4fd
HDFS-7758. Retire FsDatasetSpi#getVolumes() and use FsDatasetSpi#getVolumeRefs() instead (Lei (Eddy) Xu via Colin P. McCabe)
2015-05-05 11:08:59 -07:00
fcd4cb7516
HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe)
2015-05-05 10:50:09 -07:00
318081ccd7
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B)
...
Reverted earlier commit
2015-05-05 11:05:37 +05:30
0d6aa5d609
HDFS-8237. Move all protocol classes used by ClientProtocol to hdfs-client. Contributed by Haohui Mai.
2015-05-04 15:00:29 -07:00
3fe79e1db8
HDFS-7397. Add more detail to the documentation for the conf key "dfs.client.read.shortcircuit.streams.cache.size" (Brahma Reddy Battula via Colin P. McCabe)
2015-05-04 12:50:29 -07:00
c4578760b6
HDFS-8290. WebHDFS calls before namesystem initialization can cause NullPointerException. Contributed by Chris Nauroth.
2015-05-04 11:35:04 -07:00
a319771d1d
HDFS-8309. Skip unit test using DataNodeTestUtils#injectDataDirFailure() on Windows. (xyao)
2015-05-02 22:15:24 -07:00
6ae2a0d048
HDFS-8249. Separate HdfsConstants into the client and the server side class. Contributed by Haohui Mai.
2015-05-02 10:03:23 -07:00
6f541edce0
HDFS-8229. LAZY_PERSIST file gets deleted after NameNode restart. (Contributed by Surendra Singh Lilhore)
2015-05-01 16:42:19 -07:00
d3d019c337
HDFS-8086. Move LeaseRenewer to the hdfs.client.impl package. Contributed by Takanobu
2015-05-01 15:12:18 -07:00
64d30a6186
HDFS-8276. LazyPersistFileScrubber should be disabled if scrubber interval configured zero. (Contributed by Surendra Singh Lilhore)
2015-05-01 14:43:09 -07:00
3393461197
Updating CHANGES.txt - Pulling in HDFS-8091 to branch-2.7 (for 2.7.1)
2015-05-01 11:48:54 -07:00
b82567d455
HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe)
2015-05-01 11:19:40 -07:00
2d7363b273
HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles)
2015-05-01 13:11:50 -05:00
279958b772
HDFS-7281. Missing block is marked as corrupted block (Ming Ma via Yongjun Zhang)
2015-05-01 08:42:00 -07:00
98a6176628
HDFS-8300. Fix unit test failures and findbugs warning caused by HDFS-8283. Contributed by Jing Zhao.
2015-04-30 19:27:07 -07:00
87e9978235
HDFS-8292. Move conditional in fmt_time from dfs-dust.js to status.html. Contributed by Charles Lamb.
2015-04-30 18:23:13 -07:00
c55d609053
HDFS-8200. Refactor FSDirStatAndListingOp. Contributed by Haohui Mai.
2015-04-30 13:41:46 -07:00
de9404f02f
HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao.
2015-04-30 20:12:32 +09:00
e89fc53a1d
HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang.
2015-04-30 19:11:22 +09:00
aa22450442
HDFS-8214. Secondary NN Web UI shows wrong date for Last Checkpoint. Contributed by Charles Lamb.
2015-04-29 17:37:56 -07:00
3dd6395bb2
HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai.
2015-04-29 11:12:45 -07:00
7947e5b53b
HDFS-8283. DataStreamer cleanup and some minor improvement. Contributed by Tsz Wo Nicholas Sze.
2015-04-29 10:41:46 -07:00
439614b0c8
HDFS-8280. Code Cleanup in DFSInputStream. Contributed by Jing Zhao.
2015-04-28 18:11:59 -07:00
c79e7f7d99
HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai.
2015-04-28 18:05:46 -07:00
5639bf02da
HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su
2015-04-28 13:05:33 -07:00
feb68cb547
HDFS-8232. Missing datanode counters when using Metrics2 sink interface. Contributed by Anu Engineer.
2015-04-27 16:48:13 -07:00
6bae5962cd
HDFS-8205. Fix CHANGES.txt
2015-04-27 12:30:50 -07:00
0d5b0143cc
HDFS-8205. CommandFormat#parse() should not parse option as value of option. (Contributed by Peter Shi and Xiaoyu Yao)
2015-04-27 12:23:34 -07:00
8b69c825e5
HADOOP-11865. Incorrect path mentioned in document for accessing script files (J.Andreina via aw)
2015-04-26 09:55:46 -07:00
22b70e7c5a
Updated CHANGES.TXT for correct version of HDFS-8206
2015-04-25 22:16:06 -07:00
8f3946cd40
HDFS-8206. Fix the typos in hadoop-hdfs-httpfs. (Brahma Reddy Battula via xyao)
2015-04-25 21:41:35 -07:00
a00e001a1a
HDFS-8247. TestDiskspaceQuotaUpdate#testAppendOverTypeQuota is failing. Contributed by Xiaoyu Yao.
2015-04-25 11:37:27 -07:00
f83c55a6be
HDFS-7673. synthetic load generator docs give incorrect/incomplete commands (Brahma Reddy Battula via aw)
2015-04-25 15:05:43 +01:00
dcc5455e07
HDFS-8211. DataNode UUID is always null in the JMX counter. (Contributed by Anu Engineer)
2015-04-24 16:47:48 -07:00
c7d9ad68e3
HDFS-8191. Fix byte to integer casting in SimulatedFSDataset#simulatedByte. Contributed by Zhe Zhang.
2015-04-24 11:54:25 -07:00
cf6c8a1b4e
HDFS-8176. Record from/to snapshots in audit log for snapshot diff report. Contributed by J. Andreina.
2015-04-24 10:23:32 -07:00
91b97c21c9
HDFS-8110. Remove unsupported 'hdfs namenode -rollingUpgrade downgrade' from document. Contributed by J.Andreina.
2015-04-24 20:32:55 +09:00
c8d72907ff
HDFS-8231. StackTrace displayed at client while QuotaByStorageType exceeds (Contributed by J.Andreina and Xiaoyu Yao)
2015-04-24 12:51:04 +05:30
262c1bc339
HDFS-8217. During block recovery for truncate Log new Block Id in case of copy-on-truncate is true. (Contributed by Vinayakumar B)
2015-04-24 12:16:41 +05:30
a8898445dc
HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe)
2015-04-23 19:13:27 -07:00
a0e0a63209
HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe)
2015-04-23 19:00:24 -07:00
bcf89ddc7d
HDFS-8052. Move WebHdfsFileSystem into hadoop-hdfs-client. Contributed by Haohui Mai.
2015-04-23 17:33:45 -07:00
ef4e9963b2
HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma
2015-04-23 14:19:33 -07:00
416b84354e
HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore
2015-04-23 11:55:06 -07:00
Akira Ajisaka
yliu
Zhe Zhang
Colin Patrick Mccabe
Haohui Mai
Jing Zhao
Jakob Homan
Arpit Agarwal
Kihwal Lee
Harsh J
Walter Su
Vinayakumar B
Xiaoyu Yao
Lei Xu
Chris Douglas
Vinod Kumar Vavilapalli
Yongjun Zhang
Tsz-Wo Nicholas Sze
Andrew Wang
cnauroth
boli2
Zhihai Xu
Allen Wittenauer
Jason Lowe
Arun Suresh
Ravi Prakash
Aaron T. Myers
Ming Ma
Benoy Antony
Kai Zheng
Zhe Zhang
Tsuyoshi Ozawa
Andrew Wang
boli2
Uma Maheswara Rao G
Vinayakumar B
Junping Du
Konstantin V Shvachko
Jonathan Eagles