Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,966 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2271 Commits

Author SHA1 Message Date
cnauroth 57fa43ab0a HDFS-7313. Support optional configuration of AES cipher suite on DataTransferProtocol. Contributed by Chris Nauroth. 
(cherry picked from commit 5573b3476a)
 2014-10-30 22:52:33 -07:00
cnauroth caded2545c HDFS-6385. Show when block deletion will start after NameNode startup in WebUI. Contributed by Chris Nauroth. 
(cherry picked from commit c637d6d9d5)
 2014-10-30 22:29:20 -07:00
Colin Patrick Mccabe ec2621e907 HDFS-7035. Make adding a new data directory to the DataNode an atomic operation and improve error handling (Lei Xu via Colin P. McCabe) 
(cherry picked from commit a9331fe9b0)
 2014-10-30 17:32:23 -07:00
Andrew Wang ec19c85e65 HDFS-3342. SocketTimeoutException in BlockSender.sendChunks could have a better error message. Contributed by Yongjun Zhang. 
(cherry picked from commit 16c0f04c50822a39610e34e0715d3d4a44be5a5f)
 2014-10-30 11:03:30 -07:00
Jing Zhao fe1f4c64d1 HDFS-7230. Add rolling downgrade documentation. Contributed by Tsz Wo Nicholas Sze. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-30 10:48:36 -07:00
Andrew Wang ca6a21f7d1 HDFS-7165. Separate block metrics for files with replication count 1. Contributed by Zhe Zhang. 2014-10-30 10:21:12 -07:00
Tao Luo 8bfef59029 HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 2014-10-29 20:20:11 -07:00
Jing Zhao d25f8a12c6 HDFS-7305. NPE seen in wbhdfs FS while running SLive. Contributed by Jing Zhao. 2014-10-29 16:26:20 -07:00
Kihwal Lee 79c57e5913 HDFS-7300. HDFS-7300. The getMaxNodesPerRack() method in 
BlockPlacementPolicyDefault is flawed. contributed by Kihwal Lee
(cherry picked from commit 3ae84e1ba8)
 2014-10-29 17:25:05 -05:00
Jason Lowe 0c15130db2 HADOOP-9740. Fix FsShell '-text' command to be able to read Avro files stored in HDFS and other filesystems. Contributed by Allan Yan 2014-10-29 20:23:17 +00:00
Colin Patrick Mccabe 112d694086 HDFS-7287. The OfflineImageViewer (OIV) can output invalid XML depending on the filename (Ravi Prakash via Colin P. McCabe) 
(cherry picked from commit d33e07dc49)
 2014-10-29 12:39:00 -07:00
Brandon Li 6caf36f9eb HADOOP-11195. Move Id-Name mapping in NFS to the hadoop-common area for better maintenance. Contributed by Yongjun Zhang 
(cherry picked from commit 72a556d3b0)
 2014-10-29 11:06:44 -07:00
Haohui Mai bbdd990864 HDFS-7280. Use netty 4 in WebImageViewer. Contributed by Haohui Mai. 2014-10-28 16:54:04 -07:00
Colin Patrick Mccabe 842a54a5f6 HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
(cherry picked from commit ac9ab037e9)
 2014-10-28 16:42:13 -07:00
Haohui Mai d51ea6a248 HDFS-7301. TestMissingBlocksAlert should use MXBeans instead of old web UI. Contributed by Zhe Zhang. 2014-10-28 16:28:17 -07:00
cnauroth c2790932bd HDFS-7291. Persist in-memory replicas with appropriate unbuffered copy API on POSIX and Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit c6f04f391b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-28 14:17:35 -07:00
Kihwal Lee 946463efef HDFS-7213. processIncrementalBlockReport performance degradation. 
Contributed by Eric Payne.
(cherry picked from commit e226b5b40d)
 2014-10-28 14:55:16 -05:00
Kihwal Lee dabaf582eb HDFS-6663. Admin command to track file and locations from block id. 
Contributed by Chen He.
(cherry picked from commit 371a3b87ed)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
 2014-10-28 14:32:18 -05:00
yliu 8b812a35b8 HDFS-6606. Optimize HDFS Encrypted Transport performance. (yliu) 2014-10-28 21:16:34 +08:00
Harsh J 9548bb360b HDFS-6741. Improve permission denied message when FSPermissionChecker#checkOwner fails. Contributed by Stephen Chu and Harsh J. (harsh) 2014-10-28 12:24:14 +05:30
Colin Patrick Mccabe 296c4064fc HDFS-7282. Fix intermittent TestShortCircuitCache and TestBlockReaderFactory failures resulting from TemporarySocketDirectory GC (Jinghui Wang via Colin P. McCabe) 
(cherry picked from commit 518a7f4af3)
 2014-10-27 17:45:15 -07:00
Haohui Mai 0e05624618 HDFS-5928. Show namespace and namenode ID on NN dfshealth page. Contributed by Siqi Li. 2014-10-27 16:25:03 -07:00
cnauroth 3d67da502a HDFS-6934. Move checksum computation off the hot path when writing to RAM disk. Contributed by Chris Nauroth. 
(cherry picked from commit 463aec1171)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2014-10-27 11:50:47 -07:00
Jing Zhao 57e82e647b HDFS-7296. HdfsConstants#MEMORY_STORAGE_POLICY_ID and HdfsConstants#MEMORY_STORAGE_POLICY_ID are missing in branch-2. Contributed by Jing Zhao. 2014-10-27 11:20:57 -07:00
Colin Patrick Mccabe 5f3d967aae HDFS-7278. Add a command that allows sysadmins to manually trigger full block reports from a DN (cmccabe) 
(cherry picked from commit baf794dc40)
 2014-10-27 10:13:32 -07:00
Harsh J 8d781aaad3 HDFS-6538. Comment format error in ShortCircuitRegistry javadoc. Contributed by David Luo. (harsh) 2014-10-27 19:26:02 +05:30
Byron Wong d991cd52d8 HADOOP-6857. FsShell should report raw disk usage including replication factor. Contributed by Byron Wong. 2014-10-25 12:39:03 -07:00
Colin Patrick Mccabe 2bcda17ee7 HDFS-6988. Improve HDFS-6581 eviction configuration (Xiaoyu Yao via Colin P. McCabe) 
(cherry picked from commit a52eb4bc5f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-10-24 13:17:59 -07:00
Jitendra Pandey 5fb7660ad6 HDFS-6904. Added files missing in previous commit. 2014-10-24 12:01:54 -07:00
Jitendra Pandey b576890f35 HDFS-6904. YARN unable to renew delegation token fetched via webhdfs due to incorrect service port. 2014-10-24 11:36:54 -07:00
Haohui Mai 1f6258f535 HDFS-7283. Bump DataNode OOM log from WARN to ERROR. Contributed by Stephen Chu. 2014-10-24 10:50:28 -07:00
Haohui Mai a4bb56b93b HDFS-7258. CacheReplicationMonitor rescan schedule log should use DEBUG level instead of INFO level. Contributed by Xiaoyu Yao. 2014-10-23 23:58:49 -07:00
yliu 193724ccaf HDFS-7243. HDFS concat operation should not be allowed in Encryption Zone. (clamb via yliu) 2014-10-24 11:41:49 +08:00
Colin Patrick Mccabe caa658a015 HDFS-7223. Tracing span description of IPC client is too long (iwasakims via cmccabe) 
(cherry picked from commit 5b56ac4c72)
 2014-10-23 19:15:08 -07:00
Haohui Mai 92ad210699 HDFS-7257. Add the time of last HA state transition to NN's /jmx page. Contributed by Charles Lamb. 2014-10-23 16:53:17 -07:00
Andrew Wang d3b9e9be24 HDFS-7222. Expose DataNode network errors as a metric. (Charles Lamb via wang) 
(cherry picked from commit 86cad007d7)
 2014-10-23 12:53:39 -07:00
Haohui Mai 59e0ea8779 HDFS-7277. Remove explicit dependency on netty 3.2 in BKJournal. Contributed by Haohui Mai. 2014-10-22 17:32:30 -07:00
Andrew Wang c8d0758656 HDFS-6824. Additional user documentation for HDFS encryption. 
(cherry picked from commit a36399e09c)
 2014-10-22 14:39:00 -07:00
Tsz-wo Sze 3e2889f3c7 HDFS-2486. Remove unnecessary priority level checks in UnderReplicatedBlocks. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1242636 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit dc847b39fd)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/UnderReplicatedBlocks.java
 2014-10-22 14:37:15 -07:00
Colin Patrick Mccabe caaf778768 HDFS-6877. Avoid calling checkDisk when an HDFS volume is removed during a write. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 7b0f9bb258)
 2014-10-22 13:38:56 -07:00
Colin Patrick Mccabe c212347d32 HDFS-7254. Add documentation for hot swaping DataNode drives (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 66e8187ea1)
 2014-10-22 13:28:34 -07:00
Jing Zhao 75a9c894b3 HDFS-7228. Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. 2014-10-22 10:49:06 -07:00
Andrew Wang 2d0996ae6d HDFS-7221. TestDNFencingWithReplication fails consistently. Contributed by Charles Lamb. 
(cherry picked from commit ac56b0637e)
 2014-10-21 12:55:48 -07:00
Brandon Li c800e058ea HDFS-7215.Add JvmPauseMonitor to NFS gateway. Contributed by Brandon Li 
(cherry picked from commit 4e134a02a4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-21 11:55:18 -07:00
Colin Patrick Mccabe 8c3b007236 HDFS-7154. Fix returning value of starting reconfiguration task (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 7aab5fa1bd)
 2014-10-20 18:33:44 -07:00
Colin Patrick Mccabe 4eef6dcb58 HDFS-7266. HDFS Peercache enabled check should not lock on object (awang via cmccabe) 
(cherry picked from commit 4799570dfd)
 2014-10-20 18:25:36 -07:00
Tsz-Wo Nicholas Sze 4aae0fe976 HDFS-7260. Change DFSOutputStream.MAX_PACKETS to be configurable. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2014-10-17 18:35:14 -07:00
arp 0ee251ea26 HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsLocatedFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2014-10-17 13:42:04 -07:00
cnauroth a181daf458 HDFS-7090. Use unbuffered writes when persisting in-memory replicas. Contributed by Xiaoyu Yao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-17 13:42:03 -07:00
cnauroth 69aacf19c1 HDFS-7112. LazyWriter should use either async IO or one thread per physical disk. Contributed by Xiaoyu Yao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-17 13:42:03 -07:00
arp ae8c9cdb18 HDFS-7171. Fix Jenkins failures in HDFS-6581 branch. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
 2014-10-17 13:42:03 -07:00
arp c865c93dc1 HDFS-7129. Metrics to track usage of memory for writes. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:03 -07:00
arp 6916d41fe6 HDFS-7159. Use block storage policy to set lazy persist preference. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileAttributes.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FSImageFormatPBSnapshot.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2014-10-17 13:42:03 -07:00
arp ca6ac1997f HDFS-7144. Fix findbugs warnings in RamDiskReplicaTracker. (Contributed by Tsz Wo Nicholas Sze) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:03 -07:00
arp 081244fc8b HDFS-7155. Bugfix in createLocatedFileStatus caused by bad merge. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:03 -07:00
arp 6bb27f0ff6 HDFS-7143. Fix findbugs warnings in HDFS-6581 branch. (Contributed by Tsz Wo Nicholas Sze) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:03 -07:00
arp 69828a9bf0 HDFS-6932. Balancer and Mover tools should ignore replicas on RAM_DISK. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/StorageType.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
 2014-10-17 13:42:02 -07:00
arp 70f70f845f HDFS-6990. Add unit test for evict/delete RAM_DISK block with open handle. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp bf1b84abe1 HDFS-7100. Make eviction scheme pluggable. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-10-17 13:42:02 -07:00
arp f6903ca945 HDFS-7108. Fix unit test failures in SimulatedFsDataset. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp a973b2f55e HDFS-7091. Add forwarding constructor for INodeFile for existing callers. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSPermissionChecker.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
 2014-10-17 13:42:02 -07:00
arp f193d96627 HDFS-7084. FsDatasetImpl#copyBlockFiles debug log can be improved. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp d5940dfe4b HDFS-7080. Fix finalize and upgrade unit test failures. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp 5f86ce7ea9 HDFS-7064. Fix unit test failures in HDFS-6581 branch. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp 14edbc9419 HDFS-6978. Directory scanner should correctly reconcile blocks on RAM disk. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 6906eccec4 HDFS-7066. LazyWriter#evictBlocks misses a null check for replicaState. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp b8a2eb793a HDFS-6991. Notify NN of evicted block before deleting it from RAM disk. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
 2014-10-17 13:42:01 -07:00
arp 3abf34af9b HDFS-6977. Delete all copies when a block is deleted from the block space. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 21046d8310 HDFS-6930. Improve replica eviction from RAM disk. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 225ffdb6d8 HDFS-6950. Add Additional unit tests for HDFS-6581. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
 2014-10-17 13:42:01 -07:00
arp 43ea145c39 HDFS-6931. Move lazily persisted replicas to finalized directory on DN startup. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-10-17 13:42:01 -07:00
arp d0e6dd88fb HDFS-6960. Bugfix in LazyWriter, fix test case and some refactoring. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 08cc08adb3 HDFS-6928. 'hdfs put' command should accept lazyPersist flag for testing. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 0aaa2b9ba7 HDFS-6929. NN periodically unlinks lazy persist files with missing replicas from namespace. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:00 -07:00
arp 49a72e1360 HDFS-6927. Initial unit tests for Lazy Persist files. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:00 -07:00
arp 3b262caf11 HDFS-6926. DN support for saving replicas to persistent storage and evicting in-memory replicas. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-10-17 13:42:00 -07:00
arp 193528c84d HDFS-6925. DataNode should attempt to place replicas on transient storage first if lazyPersist flag is received. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
 2014-10-17 13:42:00 -07:00
arp 3b7d4715a1 HDFS-6923. Propagate LazyPersist flag to DNs via DataTransferProtocol. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
 2014-10-17 13:42:00 -07:00
arp 183f4a4dfb HDFS-6922. Add LazyPersist flag to INodeFile, save it in FsImage and edit logs. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileAttributes.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FSImageFormatPBSnapshot.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSPermissionChecker.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
 2014-10-17 13:42:00 -07:00
arp b1d085e7e3 HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/StorageType.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
 2014-10-17 13:42:00 -07:00
Vinayakumar B 6f19ba7688 HDFS-7252. small refinement to the use of isInAnEZ in FSNamesystem. (Yi Liu via vinayakumarb) 
(cherry picked from commit 368743140d)
 2014-10-17 18:05:16 +05:30
Vinayakumar B 7c87c2b42e HDFS-7242. Code improvement for FSN#checkUnreadableBySuperuser. (Contributed by Yi Liu) 
(cherry picked from commit 1c3ff0b7c8)
 2014-10-17 17:49:37 +05:30
Vinayakumar B fea721daf9 HDFS-6995. Block should be placed in the client's 'rack-local' node if 'client-local' node is not available (vinayakumarb) 
Adding missed test file

(cherry picked from commit cba1f9e389)
 2014-10-17 17:34:54 +05:30
Tsz-Wo Nicholas Sze aef8dbde91 HDFS-5089. When a LayoutVersion support SNAPSHOT, it must support FSIMAGE_NAME_OPTIMIZATION. 2014-10-15 20:58:01 -07:00
Tsz-Wo Nicholas Sze 603293da92 HDFS-7208. NN doesn't schedule replication when a DN storage fails. Contributed by Ming Ma 2014-10-15 20:49:54 -07:00
Jing Zhao e68fef4d81 HDFS-7185. The active NameNode will not accept an fsimage sent from the standby during rolling upgrade. Contributed by Jing Zhao. 2014-10-15 10:28:02 -07:00
Haohui Mai 3ee0a5892b HDFS-7190. Bad use of Preconditions in startFileInternal(). Contributed by Dawson Choong. 2014-10-14 23:42:36 -07:00
Haohui Mai 174138ec26 HDFS-7201. Fix typos in hdfs-default.xml. Contributed by Dawson Choong. 2014-10-14 17:36:23 -07:00
Jing Zhao 6c3ac6a485 HDFS-7228. Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
 2014-10-14 10:33:51 -07:00
Tsz-Wo Nicholas Sze dd0857fd70 HDFS-7237. The command "hdfs namenode -rollingUpgrade" throws ArrayIndexOutOfBoundsException. 2014-10-13 18:30:21 -07:00
Haohui Mai c3dfc71193 HDFS-6544. Broken Link for GFS in package.html. Contributed by Suraj Nayak M. 2014-10-13 10:33:31 -07:00
Jing Zhao 715f222a80 HDFS-7236. Fix TestOpenFilesWithSnapshot#testOpenFilesWithMultipleSnapshots. Contributed by Yongjun Zhang. 2014-10-13 10:28:12 -07:00
Haohui Mai 059f774f90 HDFS-6657. Remove link to 'Legacy UI' in trunk's Namenode UI. Contributed by Vinayakumar B. 2014-10-10 17:27:30 -07:00
Haohui Mai f11c8eb096 HDFS-6252. Phase out the old web UI in HDFS. Contributed by Haohui Mai. 2014-10-10 17:10:54 -07:00
Andrew Wang 217c0405d9 HDFS-7209. Populate EDEK cache when creating encryption zone. (Yi Liu via wang) 
(cherry picked from commit d3d3d47202)
 2014-10-10 13:40:53 -07:00
Colin Patrick Mccabe 4c0bc45978 HDFS-7198. Fix "unchecked conversion" warning in DFSClient#getPathTraceScope (cmccabe) 
(cherry picked from commit 1ceb326933)
 2014-10-10 13:04:16 -07:00
Aaron T. Myers 8cbacb37e9 HDFS-7026. Introduce a string constant for "Failed to obtain user group info...". Contributed by Yongjun Zhang. 
(cherry picked from commit cbd21fd13b)
 2014-10-09 18:53:03 -07:00
Andrew Wang 6cc8ec980c HADOOP-11174. Delegation token for KMS should only be got once if it already exists. (Yi Liu via wang) 
(cherry picked from commit 8d94114495)
 2014-10-09 13:21:32 -07:00
Kihwal Lee 7056e50c21 HDFS-7217. Better batching of IBRs. Contributed by Kihwal Lee. 
(cherry picked from commit db71bb54bc)
 2014-10-09 12:09:11 -05:00
Aaron T. Myers afaadd6535 HADOOP-11161. Expose close method in KeyProvider to give clients of Provider implementations a hook to release resources. Contribued by Arun Suresh. 
(cherry picked from commit d9556e873ef4d3e68c4f0c991f856d1faa747f07)
 2014-10-08 18:02:00 -07:00
Colin Patrick Mccabe f1feaae1dd HDFS-7202. Should be able to omit package name of SpanReceiver on "hadoop trace -add" (iwasakims via cmccabe) 
(cherry picked from commit d996235285)
 2014-10-08 18:01:22 -07:00
Kihwal Lee b39c8c3128 HDFS-7203. Concurrent appending to the same file can cause data 
corruption. Contributed by Kihwal Lee.
 2014-10-08 15:07:12 -05:00
cnauroth 7e949744b1 HDFS-7128. Decommission slows way down when it gets towards the end. Contributed by Ming Ma. 
(cherry picked from commit 80cb472620f0a7889e311ca32212a42b95c6794e)
 2014-10-07 15:01:54 -07:00
Kihwal Lee 7910c6d035 HADOOP-11007. Reinstate building of ant tasks support. Contributed by 
Jason Lowe.
(cherry picked from commit 99d7a452be)
 2014-10-07 16:23:52 -05:00
Colin Patrick Mccabe eeb39dc218 HDFS-7194 Fix findbugs "inefficient new String constructor" warning in DFSClient#PATH (yzhang via cmccabe) 
(cherry picked from commit 8dc6abf2f4)
 2014-10-06 15:29:45 -07:00
Vinayakumar B 5f58886770 HDFS-6995. Block should be placed in the client's 'rack-local' node if 'client-local' node is not available (vinayakumarb) 
(cherry picked from commit ed841dd9a9)
 2014-10-06 14:32:06 +05:30
Colin Patrick Mccabe b335df9a9b HDFS-7055. Add tracing to DFSInputStream (cmccabe) 
(cherry picked from commit 7f6ed7fe36)
 2014-10-03 13:38:15 -07:00
Andrew Wang 6ddd9eff78 HDFS-7179. DFSClient should instantiate a KeyProvider, not a KeyProviderCryptoExtension. (wang) 
(cherry picked from commit d2d5a0ea03)
 2014-10-02 13:54:50 -07:00
Colin Patrick Mccabe ab1afc4e1c HDFS-7162. Wrong path when deleting through fuse-dfs a file which already exists in trash (cmccabe) 
(cherry picked from commit 03db9cc839)
 2014-10-02 12:42:47 -07:00
arp d8e6481395 HDFS-7178. Additional unit test for replica write with full disk. (Arpit Agarwal) 2014-10-02 09:28:41 -07:00
Colin Patrick Mccabe 1725d56132 HDFS-7176. The namenode usage message doesn't include "-rollingupgrade started" (cmccabe) 
(cherry picked from commit dd1b8f2ed8)
 2014-10-01 14:54:12 -07:00
Haohui Mai 46d4f4c4ab HDFS-7158. Reduce the memory usage of WebImageViewer. Contributed by Haohui Mai. 2014-10-01 10:54:02 -07:00
Kihwal Lee c9c5b982b7 HDFS-6754. TestNamenodeCapacityReport.testXceiverCount may sometimes fail due to lack of retry. Contributed by Mit Desai. 
(cherry picked from commit 3f25d916d5)
 2014-10-01 09:58:22 -05:00
arp b680efa1e8 HDFS-7153. Add storagePolicy to NN edit log during file creation. (Arpit Agarwal) 2014-09-30 18:11:18 -07:00
Jing Zhao 635fdcbe5c HDFS-7167. NPE while running Mover if the given path is for a file. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-30 11:19:57 -07:00
Haohui Mai eea383ff2e HDFS-6779. Add missing version subcommand for hdfs. Contributed by Sasaki Toru. 2014-09-29 18:20:44 -07:00
Andrew Wang 38f3b11c29 HDFS-7124. Remove EncryptionZoneManager.NULL_EZ. (clamb via wang) 
(cherry picked from commit 7f0efe96f8)
 2014-09-29 14:14:49 -07:00
Andrew Wang 58bdba9dc0 HDFS-7104. Fix and clarify INodeInPath getter functions. Contributed by Zhe Zhang. 
(cherry picked from commit f0293f11a8)
 2014-09-29 12:58:45 -07:00
Suresh Srinivas c7cf212b2a HDFS-4165. Faulty sanity check in FsDirectory.unprotectedSetQuota. Contributed by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407245 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 0f7a4337ac)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-29 11:06:17 -07:00
Allen Wittenauer dacf2b5515 HDFS-4227. Document dfs.namenode.resource.* (Daisuke Kobayashi via aw) 2014-09-29 09:01:59 -07:00
Allen Wittenauer 96fc90a9c4 HDFS-6519. Document oiv_legacy command (Akira AJISAKA via aw) 2014-09-29 08:51:55 -07:00
arp 8ebea28b40 HDFS-7093. Add config key to restrict setStoragePolicy. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2014-09-28 19:34:25 -07:00
Konstantin V Shvachko 3472221989 HDFS-7156. Update fsck documentation. Contributed by Masahiro Yamaguch. 2014-09-27 21:26:38 -07:00
Allen Wittenauer 3c86c68078 HDFS-6664. HDFS permissions guide documentation states incorrect default group mapping class. (Ray Chiang via aw) 2014-09-26 12:21:37 -07:00
Colin Patrick Mccabe b5b4cb4d64 HDFS-6956. Allow dynamically changing the tracing level in Hadoop servers (cmccabe) 
(cherry picked from commit 55302ccfba)
 2014-09-26 10:34:42 -07:00
Andrew Wang 09c324a4a8 HDFS-7148. TestEncryptionZones#testIsEncryptedMethod fails on branch-2 after archival storage merge. (wang) 2014-09-25 21:50:12 -07:00
Andrew Wang 3f71a15813 HDFS-7077. Separate CipherSuite from crypto protocol version. (wang) 
(cherry picked from commit e96ce6f3e3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2014-09-25 18:42:17 -07:00
Jing Zhao 9ad9e51cd9 HDFS-7131. During HA upgrade, JournalNode should create a new committedTxnId file in the current directory. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-25 17:18:02 -07:00
Jing Zhao 87f07e67b1 HDFS-7140. Add a tool to list all the existing block storage policies. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
 2014-09-26 05:16:44 +08:00
Jing Zhao f83096d49c HDFS-7081. Add new DistributedFileSystem API for getting all the existing storage policies. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-26 05:08:32 +08:00
Jing Zhao ba3182c60c HDFS-7095. Archival Storage: TestStorageMover often fails in Jenkins. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-26 05:02:21 +08:00
Tsz-Wo Nicholas Sze 022474c807 Merge branch 'HDFS-6584' into trunk 
Conflicts:
	hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyConsiderLoad.java
 2014-09-26 04:47:25 +08:00
cnauroth fd3aa38231 HDFS-7119. Split error checks in AtomicFileOutputStream#close into separate conditions to improve diagnostics. Contributed by Chris Nauroth. 
(cherry picked from commit 9f9a2222a2)
 2014-09-25 13:35:40 -07:00
Colin Patrick Mccabe cb9283549f HDFS-6808. Add command line option to ask DataNode reload configuration. (Lei Xu via Colin McCabe) 
(cherry picked from commit 1861b32eb5)
 2014-09-25 13:22:35 -07:00
cnauroth 6bace90b29 HDFS-7127. TestLeaseRecovery leaks MiniDFSCluster instances. Contributed by Chris Nauroth. 
(cherry picked from commit b607c3c363)
 2014-09-25 11:29:15 -07:00
cnauroth 94517982a9 HDFS-7111. TestSafeMode assumes Unix line endings in safe mode tip. Contributed by Chris Nauroth. 
(cherry picked from commit dc2ebf88a8)
 2014-09-25 11:21:07 -07:00
cnauroth 707a7dbd24 HDFS-7118. Improve diagnostics on storage directory rename operations by using NativeIO#renameTo in Storage#rename. Contributed by Chris Nauroth. 
(cherry picked from commit e0b1dc514f)
 2014-09-25 10:06:09 -07:00
Andrew Wang 835ade7064 HDFS-7138. Fix hftp to work with encryption. (clamb via wang) 2014-09-24 16:45:29 -07:00
Andrew Wang a17544e5e3 HDFS-7139. Unit test for creating encryption zone on root path. (Zhe Zhang via wang) 
(cherry picked from commit 63198bfba7)
 2014-09-24 16:40:40 -07:00
Andrew Wang 3cf28210ec HDFS-6987. Move CipherSuite xattr information up to the encryption zone root. (Zhe Zhang via wang) 2014-09-24 12:11:45 -07:00
Haohui Mai 53c2288dc6 HDFS-7049. TestByteRangeInputStream.testPropagatedClose fails and throw NPE on branch-2. Contributed by Juan Yu. 2014-09-24 12:07:26 -07:00
Allen Wittenauer 151281bfca HDFS-6534. Fix build on macosx: HDFS parts (Binglin Chang via aw) 2014-09-24 08:35:31 -07:00
cnauroth fd3003d63b HDFS-7130. TestDataTransferKeepalive fails intermittently on Windows. Contributed by Chris Nauroth. 
(cherry picked from commit b93d9603a2)
 2014-09-23 14:44:26 -07:00
Andrew Wang 0d116b18de HDFS-7132. hdfs namenode -metadataVersion command does not honor configured name dirs. Contributed by Charles Lamb. 
(cherry picked from commit f48686a1ad)
 2014-09-23 14:10:40 -07:00
cnauroth ac3b62dae7 HDFS-7126. TestEncryptionZonesWithHA assumes Unix path separator for KMS key store path. Contributed by Xiaoyu Yao. 
(cherry picked from commit a1fd804a31)
 2014-09-23 08:22:04 -07:00
Colin Patrick Mccabe cac7063b31 HDFS-7001. Tests in TestTracing should not depend on the order of execution. (iwasakims via cmccabe) 
(cherry picked from commit 7b8df93ce1)
 2014-09-22 17:22:42 -07:00
Colin Patrick Mccabe d7b0e9382d HDFS-7106. Reconfiguring DataNode volumes does not release the lock files in removed volumes. (cnauroth via cmccabe) 
(cherry picked from commit 912ad32b03)
 2014-09-22 11:52:01 -07:00
cnauroth 39bcc57c3e HDFS-7115. TestEncryptionZones assumes Unix path separator for KMS key store path. Contributed by Xiaoyu Yao. 
(cherry picked from commit 26cba7f35f)
 2014-09-22 09:07:41 -07:00
cnauroth 4a77729a1c HDFS-7110. Skip tests related to short-circuit read on platforms that do not currently implement short-circuit read. Contributed by Chris Nauroth. 
(cherry picked from commit b6fd5ccbce)
 2014-09-21 12:37:37 -07:00
cnauroth 465ba7fe31 HDFS-7109. TestDataStorage does not release file locks between tests. Contributed by Chris Nauroth. 
(cherry picked from commit 0795fe3721)
 2014-09-20 21:01:52 -07:00