Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,287 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2718 Commits

Author SHA1 Message Date
Byron Wong 693939782b HDFS-7329. Improve logging when MiniDFSCluster fails to start. Contributed by Byron Wong. 2014-11-05 20:52:34 -08:00
Haohui Mai ee44d8fb3f HDFS-7367. HDFS short-circuit read cannot negotiate shared memory slot and file descriptors when SASL is enabled on DataTransferProtocol. Contributed by Chris Nauroth. 2014-11-05 20:34:11 -08:00
Konstantin V Shvachko 74d40b3d9e HDFS-7361. TestCheckpoint fails after change of log message related to locking violation. Contributed by Konstantin Shvachko. 2014-11-05 20:23:13 -08:00
Haohui Mai 0f9d0d9d8f HDFS-7366. BlockInfo should take replication as an short in the constructor. Contributed by Li Lu. 2014-11-05 18:13:55 -08:00
cnauroth f172a08238 HDFS-7359. NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address. Contributed by Chris Nauroth. 
(cherry picked from commit ba1d4ad25b)
 2014-11-05 15:27:27 -08:00
Colin Patrick Mccabe af3694b392 CHANGES.txt. Move HDFS-7199 to branch-2.6 2014-11-05 10:55:36 -08:00
Colin Patrick Mccabe 6f7ce842c0 HDFS-7199. DFSOutputStream should not silently drop data if DataStreamer crashes with an unchecked exception (rushabhs via cmccabe) 
(cherry picked from commit 56257fab1d)
 2014-11-05 10:52:19 -08:00
Konstantin V Shvachko 1ef616ceb7 HDFS-7333. Improve logging in Storage.tryLock(). Contributed by Konstantin Shvachko. 2014-11-05 09:50:48 -08:00
Konstantin V Shvachko 1436e89fb7 HDFS-7335. Redundant checkOperation() in FSN.analyzeFileState(). Contributed by Milan Desai. 2014-11-05 09:39:02 -08:00
yliu 46c29b988d HDFS-7218. FSNamesystem ACL operations should write to audit log on failure. (clamb via yliu) 2014-11-05 15:53:08 +08:00
Haohui Mai b96fefc032 HDFS-7357. FSNamesystem.checkFileProgress should log file path. Contributed by Tsz Wo Nicholas Sze. 2014-11-04 18:11:45 -08:00
Haohui Mai fa92b40f09 HADOOP-11230. Add missing dependency of bouncycastle for kms, httpfs, hdfs, MR and YARN. Contributed by Robert Kanter. 2014-11-04 17:53:00 -08:00
Karthik Kambatla 6f9484221e Adding release 2.5.2 to CHANGES.txt 
(cherry picked from commit 4de56d27fb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-04 16:51:54 -08:00
Karthik Kambatla 395d4ba766 HADOOP-11260. Patch up Jetty to disable SSLv3. (Mike Yoder via kasha) 
(cherry picked from commit dbf30e3c0e)
 2014-11-04 16:22:09 -08:00
Jing Zhao 9d697e3038 HDFS-7233. NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException. Contributed by Rushabh S Shah. 2014-11-04 16:05:04 -08:00
Haohui Mai 4ca67c09fc HDFS-7355. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner. Contributed by Chris Nauroth. 2014-11-04 15:19:06 -08:00
Jing Zhao dbe4b20d23 HDFS-7356. Use DirectoryListing.hasMore() directly in nfs. Contributed by Li Lu. 2014-11-04 15:05:55 -08:00
Haohui Mai 7e2bdd5255 HADOOP-10717. HttpServer2 should load jsp DTD from local jars instead of going remote. Contributed by Dapeng Sun. 2014-11-04 15:02:25 -08:00
Haohui Mai aa09c4c342 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 2014-11-04 14:52:51 -08:00
Haohui Mai e5e3f86c64 HDFS-7334. Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures. Contributed by Charles Lamb. 2014-11-04 14:35:10 -08:00
Jing Zhao b1fd0b6678 HDFS-7340. Make rollingUpgrade start/finalize idempotent. Contributed by Jing Zhao. 2014-11-04 10:17:11 -08:00
cnauroth 5720cc9d2d HDFS-7328. TestTraceAdmin assumes Unix line endings. Contributed by Chris Nauroth. 
(cherry picked from commit 2bb327eb93)
 2014-11-03 20:04:25 -08:00
Haohui Mai d2d2c84a10 HDFS-7147. Update archival storage user documentation. Contributed by Tsz Wo Nicholas Sze. 2014-11-03 15:10:37 -08:00
Suresh Srinivas c1ba223009 HDFS-7324. haadmin command usage prints incorrect command name. Contributed by Brahma Reddy Battula. 2014-11-03 13:33:33 -08:00
Kihwal Lee c8648bd27e HADOOP-10987. Provide an iterator-based listing API for FileSystem. Contributed by Kihwal Lee. 2014-11-03 08:23:34 -06:00
Tsz-Wo Nicholas Sze bac064de62 HDFS-7276. Limit the number of byte arrays used by DFSOutputStream and provide a mechanism for recycling arrays. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2014-11-01 11:33:22 -07:00
Colin Patrick Mccabe 60acff3e60 HDFS-6917. Add an hdfs debug command to validate blocks, call recoverlease, etc. (cmccabe) 
(cherry picked from commit 7b026c50f1)
 2014-10-31 13:35:00 -07:00
Haohui Mai 31612ed64d HDFS-7315. DFSTestUtil.readFileBuffer opens extra FSDataInputStream. Contributed by Plamen Jeliazkov. 2014-10-31 11:50:57 -07:00
Ravi Prakash fe8cf32493 HDFS-7309. XMLUtils.mangleXmlString doesn't seem to handle less than sign. (Colin Patrick McCabe via raviprak) 
(cherry picked from commit c7f81dad30)
 2014-10-31 11:24:00 -07:00
cnauroth 57fa43ab0a HDFS-7313. Support optional configuration of AES cipher suite on DataTransferProtocol. Contributed by Chris Nauroth. 
(cherry picked from commit 5573b3476a)
 2014-10-30 22:52:33 -07:00
cnauroth caded2545c HDFS-6385. Show when block deletion will start after NameNode startup in WebUI. Contributed by Chris Nauroth. 
(cherry picked from commit c637d6d9d5)
 2014-10-30 22:29:20 -07:00
Colin Patrick Mccabe ec2621e907 HDFS-7035. Make adding a new data directory to the DataNode an atomic operation and improve error handling (Lei Xu via Colin P. McCabe) 
(cherry picked from commit a9331fe9b0)
 2014-10-30 17:32:23 -07:00
Andrew Wang ec19c85e65 HDFS-3342. SocketTimeoutException in BlockSender.sendChunks could have a better error message. Contributed by Yongjun Zhang. 
(cherry picked from commit 16c0f04c50822a39610e34e0715d3d4a44be5a5f)
 2014-10-30 11:03:30 -07:00
Jing Zhao fe1f4c64d1 HDFS-7230. Add rolling downgrade documentation. Contributed by Tsz Wo Nicholas Sze. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-30 10:48:36 -07:00
Andrew Wang ca6a21f7d1 HDFS-7165. Separate block metrics for files with replication count 1. Contributed by Zhe Zhang. 2014-10-30 10:21:12 -07:00
Tao Luo 8bfef59029 HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 2014-10-29 20:20:11 -07:00
Haohui Mai c7d1db7495 HADOOP-11247. Fix a couple javac warnings in NFS. Contributed by Brandon Li. 2014-10-29 17:54:37 -07:00
Jing Zhao d25f8a12c6 HDFS-7305. NPE seen in wbhdfs FS while running SLive. Contributed by Jing Zhao. 2014-10-29 16:26:20 -07:00
Kihwal Lee 79c57e5913 HDFS-7300. HDFS-7300. The getMaxNodesPerRack() method in 
BlockPlacementPolicyDefault is flawed. contributed by Kihwal Lee
(cherry picked from commit 3ae84e1ba8)
 2014-10-29 17:25:05 -05:00
Jason Lowe 0c15130db2 HADOOP-9740. Fix FsShell '-text' command to be able to read Avro files stored in HDFS and other filesystems. Contributed by Allan Yan 2014-10-29 20:23:17 +00:00
Colin Patrick Mccabe 112d694086 HDFS-7287. The OfflineImageViewer (OIV) can output invalid XML depending on the filename (Ravi Prakash via Colin P. McCabe) 
(cherry picked from commit d33e07dc49)
 2014-10-29 12:39:00 -07:00
Brandon Li 6caf36f9eb HADOOP-11195. Move Id-Name mapping in NFS to the hadoop-common area for better maintenance. Contributed by Yongjun Zhang 
(cherry picked from commit 72a556d3b0)
 2014-10-29 11:06:44 -07:00
Karthik Kambatla f2679ae22e HDFS-7274. Disable SSLv3 in HttpFS. (Robert Kanter via kasha) 
(cherry picked from commit 3f48493bed)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-28 17:33:52 -07:00
Haohui Mai bbdd990864 HDFS-7280. Use netty 4 in WebImageViewer. Contributed by Haohui Mai. 2014-10-28 16:54:04 -07:00
Colin Patrick Mccabe 842a54a5f6 HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
(cherry picked from commit ac9ab037e9)
 2014-10-28 16:42:13 -07:00
Haohui Mai d51ea6a248 HDFS-7301. TestMissingBlocksAlert should use MXBeans instead of old web UI. Contributed by Zhe Zhang. 2014-10-28 16:28:17 -07:00
cnauroth c2790932bd HDFS-7291. Persist in-memory replicas with appropriate unbuffered copy API on POSIX and Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit c6f04f391b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-28 14:17:35 -07:00
Kihwal Lee 946463efef HDFS-7213. processIncrementalBlockReport performance degradation. 
Contributed by Eric Payne.
(cherry picked from commit e226b5b40d)
 2014-10-28 14:55:16 -05:00
Kihwal Lee dabaf582eb HDFS-6663. Admin command to track file and locations from block id. 
Contributed by Chen He.
(cherry picked from commit 371a3b87ed)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
 2014-10-28 14:32:18 -05:00
yliu 8b812a35b8 HDFS-6606. Optimize HDFS Encrypted Transport performance. (yliu) 2014-10-28 21:16:34 +08:00
Harsh J 9548bb360b HDFS-6741. Improve permission denied message when FSPermissionChecker#checkOwner fails. Contributed by Stephen Chu and Harsh J. (harsh) 2014-10-28 12:24:14 +05:30
yliu 2e156f01a2 HADOOP-11233. hadoop.security.kms.client.encrypted.key.cache.expiry property spelled wrong in core-default. (Stephen Chu via yliu) 2014-10-28 14:44:13 +08:00
Colin Patrick Mccabe 296c4064fc HDFS-7282. Fix intermittent TestShortCircuitCache and TestBlockReaderFactory failures resulting from TemporarySocketDirectory GC (Jinghui Wang via Colin P. McCabe) 
(cherry picked from commit 518a7f4af3)
 2014-10-27 17:45:15 -07:00
Haohui Mai 0e05624618 HDFS-5928. Show namespace and namenode ID on NN dfshealth page. Contributed by Siqi Li. 2014-10-27 16:25:03 -07:00
cnauroth 3d67da502a HDFS-6934. Move checksum computation off the hot path when writing to RAM disk. Contributed by Chris Nauroth. 
(cherry picked from commit 463aec1171)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2014-10-27 11:50:47 -07:00
Jing Zhao 57e82e647b HDFS-7296. HdfsConstants#MEMORY_STORAGE_POLICY_ID and HdfsConstants#MEMORY_STORAGE_POLICY_ID are missing in branch-2. Contributed by Jing Zhao. 2014-10-27 11:20:57 -07:00
Colin Patrick Mccabe 5f3d967aae HDFS-7278. Add a command that allows sysadmins to manually trigger full block reports from a DN (cmccabe) 
(cherry picked from commit baf794dc40)
 2014-10-27 10:13:32 -07:00
Harsh J 8d781aaad3 HDFS-6538. Comment format error in ShortCircuitRegistry javadoc. Contributed by David Luo. (harsh) 2014-10-27 19:26:02 +05:30
Byron Wong d991cd52d8 HADOOP-6857. FsShell should report raw disk usage including replication factor. Contributed by Byron Wong. 2014-10-25 12:39:03 -07:00
Colin Patrick Mccabe 2bcda17ee7 HDFS-6988. Improve HDFS-6581 eviction configuration (Xiaoyu Yao via Colin P. McCabe) 
(cherry picked from commit a52eb4bc5f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-10-24 13:17:59 -07:00
Jitendra Pandey 5fb7660ad6 HDFS-6904. Added files missing in previous commit. 2014-10-24 12:01:54 -07:00
Jitendra Pandey b576890f35 HDFS-6904. YARN unable to renew delegation token fetched via webhdfs due to incorrect service port. 2014-10-24 11:36:54 -07:00
Haohui Mai 1f6258f535 HDFS-7283. Bump DataNode OOM log from WARN to ERROR. Contributed by Stephen Chu. 2014-10-24 10:50:28 -07:00
Haohui Mai a4bb56b93b HDFS-7258. CacheReplicationMonitor rescan schedule log should use DEBUG level instead of INFO level. Contributed by Xiaoyu Yao. 2014-10-23 23:58:49 -07:00
yliu 193724ccaf HDFS-7243. HDFS concat operation should not be allowed in Encryption Zone. (clamb via yliu) 2014-10-24 11:41:49 +08:00
Colin Patrick Mccabe caa658a015 HDFS-7223. Tracing span description of IPC client is too long (iwasakims via cmccabe) 
(cherry picked from commit 5b56ac4c72)
 2014-10-23 19:15:08 -07:00
Haohui Mai 92ad210699 HDFS-7257. Add the time of last HA state transition to NN's /jmx page. Contributed by Charles Lamb. 2014-10-23 16:53:17 -07:00
Allen Wittenauer a58f88056c HDFS-7232. Populate hostname in httpfs audit log (Zoran Dimitrijevic via aw) 2014-10-23 14:07:07 -07:00
Andrew Wang d3b9e9be24 HDFS-7222. Expose DataNode network errors as a metric. (Charles Lamb via wang) 
(cherry picked from commit 86cad007d7)
 2014-10-23 12:53:39 -07:00
Brandon Li d27cd5ad77 HDFS-7180. NFSv3 gateway frequently gets stuck due to GC. Contributed by Brandon Li 
(cherry picked from commit d71d40a63d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-22 21:34:15 -07:00
Haohui Mai 59e0ea8779 HDFS-7277. Remove explicit dependency on netty 3.2 in BKJournal. Contributed by Haohui Mai. 2014-10-22 17:32:30 -07:00
Andrew Wang c8d0758656 HDFS-6824. Additional user documentation for HDFS encryption. 
(cherry picked from commit a36399e09c)
 2014-10-22 14:39:00 -07:00
Tsz-wo Sze 3e2889f3c7 HDFS-2486. Remove unnecessary priority level checks in UnderReplicatedBlocks. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1242636 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit dc847b39fd)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/UnderReplicatedBlocks.java
 2014-10-22 14:37:15 -07:00
Colin Patrick Mccabe caaf778768 HDFS-6877. Avoid calling checkDisk when an HDFS volume is removed during a write. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 7b0f9bb258)
 2014-10-22 13:38:56 -07:00
Colin Patrick Mccabe c212347d32 HDFS-7254. Add documentation for hot swaping DataNode drives (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 66e8187ea1)
 2014-10-22 13:28:34 -07:00
Jing Zhao 75a9c894b3 HDFS-7228. Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. 2014-10-22 10:49:06 -07:00
Andrew Wang 2d0996ae6d HDFS-7221. TestDNFencingWithReplication fails consistently. Contributed by Charles Lamb. 
(cherry picked from commit ac56b0637e)
 2014-10-21 12:55:48 -07:00
Brandon Li c800e058ea HDFS-7215.Add JvmPauseMonitor to NFS gateway. Contributed by Brandon Li 
(cherry picked from commit 4e134a02a4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-21 11:55:18 -07:00
Brandon Li 33e020cb19 HDFS-7259. Unresponseive NFS mount point due to deferred COMMIT response. Contributed by Brandon Li 2014-10-21 11:01:27 -07:00
Colin Patrick Mccabe 8c3b007236 HDFS-7154. Fix returning value of starting reconfiguration task (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 7aab5fa1bd)
 2014-10-20 18:33:44 -07:00
Colin Patrick Mccabe 4eef6dcb58 HDFS-7266. HDFS Peercache enabled check should not lock on object (awang via cmccabe) 
(cherry picked from commit 4799570dfd)
 2014-10-20 18:25:36 -07:00
Tsz-Wo Nicholas Sze 4aae0fe976 HDFS-7260. Change DFSOutputStream.MAX_PACKETS to be configurable. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2014-10-17 18:35:14 -07:00
arp 0ee251ea26 HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsLocatedFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2014-10-17 13:42:04 -07:00
cnauroth a181daf458 HDFS-7090. Use unbuffered writes when persisting in-memory replicas. Contributed by Xiaoyu Yao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-17 13:42:03 -07:00
cnauroth 69aacf19c1 HDFS-7112. LazyWriter should use either async IO or one thread per physical disk. Contributed by Xiaoyu Yao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-17 13:42:03 -07:00
arp ae8c9cdb18 HDFS-7171. Fix Jenkins failures in HDFS-6581 branch. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
 2014-10-17 13:42:03 -07:00
arp c865c93dc1 HDFS-7129. Metrics to track usage of memory for writes. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:03 -07:00
arp 6916d41fe6 HDFS-7159. Use block storage policy to set lazy persist preference. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileAttributes.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FSImageFormatPBSnapshot.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2014-10-17 13:42:03 -07:00
arp ca6ac1997f HDFS-7144. Fix findbugs warnings in RamDiskReplicaTracker. (Contributed by Tsz Wo Nicholas Sze) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:03 -07:00
arp 081244fc8b HDFS-7155. Bugfix in createLocatedFileStatus caused by bad merge. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:03 -07:00
arp 6bb27f0ff6 HDFS-7143. Fix findbugs warnings in HDFS-6581 branch. (Contributed by Tsz Wo Nicholas Sze) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:03 -07:00
arp 69828a9bf0 HDFS-6932. Balancer and Mover tools should ignore replicas on RAM_DISK. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/StorageType.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
 2014-10-17 13:42:02 -07:00
arp 70f70f845f HDFS-6990. Add unit test for evict/delete RAM_DISK block with open handle. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp bf1b84abe1 HDFS-7100. Make eviction scheme pluggable. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-10-17 13:42:02 -07:00
arp f6903ca945 HDFS-7108. Fix unit test failures in SimulatedFsDataset. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp a973b2f55e HDFS-7091. Add forwarding constructor for INodeFile for existing callers. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSPermissionChecker.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
 2014-10-17 13:42:02 -07:00
arp f193d96627 HDFS-7084. FsDatasetImpl#copyBlockFiles debug log can be improved. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp d5940dfe4b HDFS-7080. Fix finalize and upgrade unit test failures. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp 5f86ce7ea9 HDFS-7064. Fix unit test failures in HDFS-6581 branch. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:02 -07:00
arp 14edbc9419 HDFS-6978. Directory scanner should correctly reconcile blocks on RAM disk. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 6906eccec4 HDFS-7066. LazyWriter#evictBlocks misses a null check for replicaState. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp b8a2eb793a HDFS-6991. Notify NN of evicted block before deleting it from RAM disk. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
 2014-10-17 13:42:01 -07:00
arp 3abf34af9b HDFS-6977. Delete all copies when a block is deleted from the block space. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 21046d8310 HDFS-6930. Improve replica eviction from RAM disk. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 225ffdb6d8 HDFS-6950. Add Additional unit tests for HDFS-6581. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
 2014-10-17 13:42:01 -07:00
arp 43ea145c39 HDFS-6931. Move lazily persisted replicas to finalized directory on DN startup. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-10-17 13:42:01 -07:00
arp d0e6dd88fb HDFS-6960. Bugfix in LazyWriter, fix test case and some refactoring. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 08cc08adb3 HDFS-6928. 'hdfs put' command should accept lazyPersist flag for testing. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:01 -07:00
arp 0aaa2b9ba7 HDFS-6929. NN periodically unlinks lazy persist files with missing replicas from namespace. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:00 -07:00
arp 49a72e1360 HDFS-6927. Initial unit tests for Lazy Persist files. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 13:42:00 -07:00
arp 3b262caf11 HDFS-6926. DN support for saving replicas to persistent storage and evicting in-memory replicas. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-10-17 13:42:00 -07:00
arp 193528c84d HDFS-6925. DataNode should attempt to place replicas on transient storage first if lazyPersist flag is received. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
 2014-10-17 13:42:00 -07:00
arp 3b7d4715a1 HDFS-6923. Propagate LazyPersist flag to DNs via DataTransferProtocol. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
 2014-10-17 13:42:00 -07:00
arp 183f4a4dfb HDFS-6922. Add LazyPersist flag to INodeFile, save it in FsImage and edit logs. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileAttributes.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FSImageFormatPBSnapshot.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSPermissionChecker.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
 2014-10-17 13:42:00 -07:00
arp b1d085e7e3 HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/StorageType.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
 2014-10-17 13:42:00 -07:00
Vinayakumar B 6f19ba7688 HDFS-7252. small refinement to the use of isInAnEZ in FSNamesystem. (Yi Liu via vinayakumarb) 
(cherry picked from commit 368743140d)
 2014-10-17 18:05:16 +05:30
Vinayakumar B 7c87c2b42e HDFS-7242. Code improvement for FSN#checkUnreadableBySuperuser. (Contributed by Yi Liu) 
(cherry picked from commit 1c3ff0b7c8)
 2014-10-17 17:49:37 +05:30
Vinayakumar B fea721daf9 HDFS-6995. Block should be placed in the client's 'rack-local' node if 'client-local' node is not available (vinayakumarb) 
Adding missed test file

(cherry picked from commit cba1f9e389)
 2014-10-17 17:34:54 +05:30
Tsz-Wo Nicholas Sze aef8dbde91 HDFS-5089. When a LayoutVersion support SNAPSHOT, it must support FSIMAGE_NAME_OPTIMIZATION. 2014-10-15 20:58:01 -07:00
Tsz-Wo Nicholas Sze 603293da92 HDFS-7208. NN doesn't schedule replication when a DN storage fails. Contributed by Ming Ma 2014-10-15 20:49:54 -07:00
Jing Zhao e68fef4d81 HDFS-7185. The active NameNode will not accept an fsimage sent from the standby during rolling upgrade. Contributed by Jing Zhao. 2014-10-15 10:28:02 -07:00
Haohui Mai 3ee0a5892b HDFS-7190. Bad use of Preconditions in startFileInternal(). Contributed by Dawson Choong. 2014-10-14 23:42:36 -07:00
Haohui Mai 174138ec26 HDFS-7201. Fix typos in hdfs-default.xml. Contributed by Dawson Choong. 2014-10-14 17:36:23 -07:00
Jing Zhao 6c3ac6a485 HDFS-7228. Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
 2014-10-14 10:33:51 -07:00
Tsz-Wo Nicholas Sze dd0857fd70 HDFS-7237. The command "hdfs namenode -rollingUpgrade" throws ArrayIndexOutOfBoundsException. 2014-10-13 18:30:21 -07:00
Haohui Mai c3dfc71193 HDFS-6544. Broken Link for GFS in package.html. Contributed by Suraj Nayak M. 2014-10-13 10:33:31 -07:00
Jing Zhao 715f222a80 HDFS-7236. Fix TestOpenFilesWithSnapshot#testOpenFilesWithMultipleSnapshots. Contributed by Yongjun Zhang. 2014-10-13 10:28:12 -07:00
Haohui Mai 059f774f90 HDFS-6657. Remove link to 'Legacy UI' in trunk's Namenode UI. Contributed by Vinayakumar B. 2014-10-10 17:27:30 -07:00
Haohui Mai f11c8eb096 HDFS-6252. Phase out the old web UI in HDFS. Contributed by Haohui Mai. 2014-10-10 17:10:54 -07:00
Andrew Wang 217c0405d9 HDFS-7209. Populate EDEK cache when creating encryption zone. (Yi Liu via wang) 
(cherry picked from commit d3d3d47202)
 2014-10-10 13:40:53 -07:00
Colin Patrick Mccabe 4c0bc45978 HDFS-7198. Fix "unchecked conversion" warning in DFSClient#getPathTraceScope (cmccabe) 
(cherry picked from commit 1ceb326933)
 2014-10-10 13:04:16 -07:00
Aaron T. Myers 8cbacb37e9 HDFS-7026. Introduce a string constant for "Failed to obtain user group info...". Contributed by Yongjun Zhang. 
(cherry picked from commit cbd21fd13b)
 2014-10-09 18:53:03 -07:00
Andrew Wang 6cc8ec980c HADOOP-11174. Delegation token for KMS should only be got once if it already exists. (Yi Liu via wang) 
(cherry picked from commit 8d94114495)
 2014-10-09 13:21:32 -07:00
cnauroth 4e544c84e9 HDFS-7195. Update user doc of secure mode about Datanodes don't require root or jsvc. Contributed by Chris Nauroth. 2014-10-09 11:55:59 -07:00
Kihwal Lee 7056e50c21 HDFS-7217. Better batching of IBRs. Contributed by Kihwal Lee. 
(cherry picked from commit db71bb54bc)
 2014-10-09 12:09:11 -05:00
Aaron T. Myers afaadd6535 HADOOP-11161. Expose close method in KeyProvider to give clients of Provider implementations a hook to release resources. Contribued by Arun Suresh. 
(cherry picked from commit d9556e873ef4d3e68c4f0c991f856d1faa747f07)
 2014-10-08 18:02:00 -07:00
Colin Patrick Mccabe f1feaae1dd HDFS-7202. Should be able to omit package name of SpanReceiver on "hadoop trace -add" (iwasakims via cmccabe) 
(cherry picked from commit d996235285)
 2014-10-08 18:01:22 -07:00
Kihwal Lee b39c8c3128 HDFS-7203. Concurrent appending to the same file can cause data 
corruption. Contributed by Kihwal Lee.
 2014-10-08 15:07:12 -05:00
cnauroth 7e949744b1 HDFS-7128. Decommission slows way down when it gets towards the end. Contributed by Ming Ma. 
(cherry picked from commit 80cb472620f0a7889e311ca32212a42b95c6794e)
 2014-10-07 15:01:54 -07:00
Andrew Wang 6546c59745 Fixup for CHANGES.txt 
(cherry picked from commit 0263656247)
 2014-10-07 14:59:17 -07:00
Kihwal Lee 7910c6d035 HADOOP-11007. Reinstate building of ant tasks support. Contributed by 
Jason Lowe.
(cherry picked from commit 99d7a452be)
 2014-10-07 16:23:52 -05:00
Colin Patrick Mccabe eeb39dc218 HDFS-7194 Fix findbugs "inefficient new String constructor" warning in DFSClient#PATH (yzhang via cmccabe) 
(cherry picked from commit 8dc6abf2f4)
 2014-10-06 15:29:45 -07:00
Tsz-Wo Nicholas Sze e5e9cf219a HDFS-7169. Add SE_BAD_FIELD to findbugsExcludeFile.xml. 2014-10-06 13:31:46 -07:00
Vinayakumar B 5f58886770 HDFS-6995. Block should be placed in the client's 'rack-local' node if 'client-local' node is not available (vinayakumarb) 
(cherry picked from commit ed841dd9a9)
 2014-10-06 14:32:06 +05:30
Colin Patrick Mccabe b335df9a9b HDFS-7055. Add tracing to DFSInputStream (cmccabe) 
(cherry picked from commit 7f6ed7fe36)
 2014-10-03 13:38:15 -07:00
Andrew Wang 6dbba19469 HDFS-7181. Remove incorrect precondition check on key length in FileEncryptionInfo. (wang) 
(cherry picked from commit 6ac10516e7)
 2014-10-02 13:54:56 -07:00
Andrew Wang 6ddd9eff78 HDFS-7179. DFSClient should instantiate a KeyProvider, not a KeyProviderCryptoExtension. (wang) 
(cherry picked from commit d2d5a0ea03)
 2014-10-02 13:54:50 -07:00
Colin Patrick Mccabe fb0107ef0d HDFS-7162. Fix CHANGES.txt (should have Chenging Liu's name) 2014-10-02 12:44:23 -07:00
Colin Patrick Mccabe ab1afc4e1c HDFS-7162. Wrong path when deleting through fuse-dfs a file which already exists in trash (cmccabe) 
(cherry picked from commit 03db9cc839)
 2014-10-02 12:42:47 -07:00
arp d8e6481395 HDFS-7178. Additional unit test for replica write with full disk. (Arpit Agarwal) 2014-10-02 09:28:41 -07:00
Haohui Mai ddb46c1c28 Addendum patch for HDFS-7172. Contributed by Chris Nauroth. 2014-10-01 16:10:37 -07:00
Colin Patrick Mccabe 1725d56132 HDFS-7176. The namenode usage message doesn't include "-rollingupgrade started" (cmccabe) 
(cherry picked from commit dd1b8f2ed8)
 2014-10-01 14:54:12 -07:00
Haohui Mai 63ad22be48 HDFS-6894. Add XDR parser method for each NFS response. Contributed by Brandon Li. 2014-10-01 13:19:05 -07:00
Haohui Mai 46d4f4c4ab HDFS-7158. Reduce the memory usage of WebImageViewer. Contributed by Haohui Mai. 2014-10-01 10:54:02 -07:00
Haohui Mai 5db04a5a1a HDFS-7172. Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. 2014-10-01 10:46:51 -07:00
Kihwal Lee c9c5b982b7 HDFS-6754. TestNamenodeCapacityReport.testXceiverCount may sometimes fail due to lack of retry. Contributed by Mit Desai. 
(cherry picked from commit 3f25d916d5)
 2014-10-01 09:58:22 -05:00
arp b680efa1e8 HDFS-7153. Add storagePolicy to NN edit log during file creation. (Arpit Agarwal) 2014-09-30 18:11:18 -07:00
Jing Zhao 635fdcbe5c HDFS-7167. NPE while running Mover if the given path is for a file. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-30 11:19:57 -07:00
Arun C. Murthy 6a60fa279a Preparing to branch hadoop-2.6 2014-09-30 10:37:03 -07:00
Haohui Mai eea383ff2e HDFS-6779. Add missing version subcommand for hdfs. Contributed by Sasaki Toru. 2014-09-29 18:20:44 -07:00
Andrew Wang 47ddaee1db HDFS-7122. Use of ThreadLocal<Random> results in poor block placement. (wang) 
(cherry picked from commit d7086c563ff2847c415913ac625b2a557eeccbdd)
 2014-09-29 15:00:28 -07:00
Andrew Wang 38f3b11c29 HDFS-7124. Remove EncryptionZoneManager.NULL_EZ. (clamb via wang) 
(cherry picked from commit 7f0efe96f8)
 2014-09-29 14:14:49 -07:00
Andrew Wang d551a95fc4 Move HDFS-7157 to HDFS CHANGES.txt 
(cherry picked from commit a45b331cc9)
 2014-09-29 13:17:28 -07:00
Andrew Wang 58bdba9dc0 HDFS-7104. Fix and clarify INodeInPath getter functions. Contributed by Zhe Zhang. 
(cherry picked from commit f0293f11a8)
 2014-09-29 12:58:45 -07:00
Suresh Srinivas c7cf212b2a HDFS-4165. Faulty sanity check in FsDirectory.unprotectedSetQuota. Contributed by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407245 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 0f7a4337ac)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-29 11:06:17 -07:00
Allen Wittenauer dacf2b5515 HDFS-4227. Document dfs.namenode.resource.* (Daisuke Kobayashi via aw) 2014-09-29 09:01:59 -07:00
Allen Wittenauer 96fc90a9c4 HDFS-6519. Document oiv_legacy command (Akira AJISAKA via aw) 2014-09-29 08:51:55 -07:00
arp 8ebea28b40 HDFS-7093. Add config key to restrict setStoragePolicy. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2014-09-28 19:34:25 -07:00
Konstantin V Shvachko 3472221989 HDFS-7156. Update fsck documentation. Contributed by Masahiro Yamaguch. 2014-09-27 21:26:38 -07:00
Allen Wittenauer 3c86c68078 HDFS-6664. HDFS permissions guide documentation states incorrect default group mapping class. (Ray Chiang via aw) 2014-09-26 12:21:37 -07:00
Colin Patrick Mccabe b5b4cb4d64 HDFS-6956. Allow dynamically changing the tracing level in Hadoop servers (cmccabe) 
(cherry picked from commit 55302ccfba)
 2014-09-26 10:34:42 -07:00
Andrew Wang 09c324a4a8 HDFS-7148. TestEncryptionZones#testIsEncryptedMethod fails on branch-2 after archival storage merge. (wang) 2014-09-25 21:50:12 -07:00
Andrew Wang 3f71a15813 HDFS-7077. Separate CipherSuite from crypto protocol version. (wang) 
(cherry picked from commit e96ce6f3e3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2014-09-25 18:42:17 -07:00
Jing Zhao 9ad9e51cd9 HDFS-7131. During HA upgrade, JournalNode should create a new committedTxnId file in the current directory. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-25 17:18:02 -07:00
Jing Zhao 87f07e67b1 HDFS-7140. Add a tool to list all the existing block storage policies. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
 2014-09-26 05:16:44 +08:00
Jing Zhao f83096d49c HDFS-7081. Add new DistributedFileSystem API for getting all the existing storage policies. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-26 05:08:32 +08:00
Jing Zhao ba3182c60c HDFS-7095. Archival Storage: TestStorageMover often fails in Jenkins. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-26 05:02:21 +08:00
Tsz-Wo Nicholas Sze 788ba5e4c4 Revise the HDFS-6584 entry CHANGES.txt. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-26 04:59:00 +08:00
Tsz-Wo Nicholas Sze 80bd4d88eb Fix hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt for HDFS-6584 after 
merge.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-26 04:57:45 +08:00
Tsz-Wo Nicholas Sze 022474c807 Merge branch 'HDFS-6584' into trunk 
Conflicts:
	hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyConsiderLoad.java
 2014-09-26 04:47:25 +08:00
cnauroth fd3aa38231 HDFS-7119. Split error checks in AtomicFileOutputStream#close into separate conditions to improve diagnostics. Contributed by Chris Nauroth. 
(cherry picked from commit 9f9a2222a2)
 2014-09-25 13:35:40 -07:00
Colin Patrick Mccabe cb9283549f HDFS-6808. Add command line option to ask DataNode reload configuration. (Lei Xu via Colin McCabe) 
(cherry picked from commit 1861b32eb5)
 2014-09-25 13:22:35 -07:00
cnauroth 6bace90b29 HDFS-7127. TestLeaseRecovery leaks MiniDFSCluster instances. Contributed by Chris Nauroth. 
(cherry picked from commit b607c3c363)
 2014-09-25 11:29:15 -07:00
cnauroth 94517982a9 HDFS-7111. TestSafeMode assumes Unix line endings in safe mode tip. Contributed by Chris Nauroth. 
(cherry picked from commit dc2ebf88a8)
 2014-09-25 11:21:07 -07:00
cnauroth 707a7dbd24 HDFS-7118. Improve diagnostics on storage directory rename operations by using NativeIO#renameTo in Storage#rename. Contributed by Chris Nauroth. 
(cherry picked from commit e0b1dc514f)
 2014-09-25 10:06:09 -07:00
Andrew Wang 835ade7064 HDFS-7138. Fix hftp to work with encryption. (clamb via wang) 2014-09-24 16:45:29 -07:00
Andrew Wang a17544e5e3 HDFS-7139. Unit test for creating encryption zone on root path. (Zhe Zhang via wang) 
(cherry picked from commit 63198bfba7)
 2014-09-24 16:40:40 -07:00
Andrew Wang 3cf28210ec HDFS-6987. Move CipherSuite xattr information up to the encryption zone root. (Zhe Zhang via wang) 2014-09-24 12:11:45 -07:00
Haohui Mai 53c2288dc6 HDFS-7049. TestByteRangeInputStream.testPropagatedClose fails and throw NPE on branch-2. Contributed by Juan Yu. 2014-09-24 12:07:26 -07:00
Allen Wittenauer 151281bfca HDFS-6534. Fix build on macosx: HDFS parts (Binglin Chang via aw) 2014-09-24 08:35:31 -07:00
cnauroth fd3003d63b HDFS-7130. TestDataTransferKeepalive fails intermittently on Windows. Contributed by Chris Nauroth. 
(cherry picked from commit b93d9603a2)
 2014-09-23 14:44:26 -07:00
Andrew Wang 0d116b18de HDFS-7132. hdfs namenode -metadataVersion command does not honor configured name dirs. Contributed by Charles Lamb. 
(cherry picked from commit f48686a1ad)
 2014-09-23 14:10:40 -07:00
cnauroth ac3b62dae7 HDFS-7126. TestEncryptionZonesWithHA assumes Unix path separator for KMS key store path. Contributed by Xiaoyu Yao. 
(cherry picked from commit a1fd804a31)
 2014-09-23 08:22:04 -07:00
Colin Patrick Mccabe cac7063b31 HDFS-7001. Tests in TestTracing should not depend on the order of execution. (iwasakims via cmccabe) 
(cherry picked from commit 7b8df93ce1)
 2014-09-22 17:22:42 -07:00
Colin Patrick Mccabe d7b0e9382d HDFS-7106. Reconfiguring DataNode volumes does not release the lock files in removed volumes. (cnauroth via cmccabe) 
(cherry picked from commit 912ad32b03)
 2014-09-22 11:52:01 -07:00
cnauroth 39bcc57c3e HDFS-7115. TestEncryptionZones assumes Unix path separator for KMS key store path. Contributed by Xiaoyu Yao. 
(cherry picked from commit 26cba7f35f)
 2014-09-22 09:07:41 -07:00
cnauroth 4a77729a1c HDFS-7110. Skip tests related to short-circuit read on platforms that do not currently implement short-circuit read. Contributed by Chris Nauroth. 
(cherry picked from commit b6fd5ccbce)
 2014-09-21 12:37:37 -07:00
cnauroth 465ba7fe31 HDFS-7109. TestDataStorage does not release file locks between tests. Contributed by Chris Nauroth. 
(cherry picked from commit 0795fe3721)
 2014-09-20 21:01:52 -07:00
cnauroth 3ad3f034bb HDFS-7107. Avoid Findbugs warning for synchronization on AbstractNNFailoverProxyProvider#fallbackToSimpleAuth. Contributed by Chris Nauroth. 
(cherry picked from commit 47491f9a11)
 2014-09-20 20:53:38 -07:00
cnauroth cadba3067f HDFS-7073. Allow falling back to a non-SASL connection on DataTransferProtocol in several edge cases. Contributed by Chris Nauroth. 
(cherry picked from commit f85cc14eb4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
 2014-09-19 22:04:13 -07:00
cnauroth 9ee5e26cbb HDFS-7105. Fix TestJournalNode#testFailToStartWithBadConfig to match log output change. Contributed by Ray Chiang. 
(cherry picked from commit 332e2e23ba)
 2014-09-19 21:06:04 -07:00
Kihwal Lee b55aa50ec6 HDFS-7046. HA NN can NPE upon transition to active. Contributed by 
Kihwal Lee.
(cherry picked from commit 9e35571965)
 2014-09-19 17:08:30 -05:00
Colin Patrick Mccabe 5520ca6ec3 HDFS-7096. Fix TestRpcProgramNfs3 to use DFS_ENCRYPTION_KEY_PROVIDER_URI (clamb via cmccabe) 
(cherry picked from commit 25fd69a8f9)
 2014-09-19 09:20:55 -07:00
Kihwal Lee 35fccfda3e HDFS-7065. Pipeline close recovery race can cause block corruption. 
Contributed by Kihwal Lee.
(cherry picked from commit bf27b9ca57)
 2014-09-19 09:01:27 -05:00
Kihwal Lee 2a6c9f0725 HDFS-6948. DN rejects blocks if it has older UC block. Contributed by 
Eric Payne.
(cherry picked from commit f02d934fed)
 2014-09-19 08:52:16 -05:00
Andrew Wang 58d9cc0914 HDFS-6840. Clients are always sent to the same datanode when read is off rack. (wang) 
(cherry picked from commit 8e73084491)
 2014-09-18 18:14:10 -07:00
Andrew Wang ef693b541c HDFS-6970. Move startFile EDEK retries to the DFSClient. (wang) 
(cherry picked from commit 20a076bafc)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2014-09-18 17:41:22 -07:00
Colin Patrick Mccabe d329237990 HDFS-6727. Refresh data volumes on DataNode based on configuration changes (Lei Xu via Colin Patrick McCabe) 
(cherry picked from commit fe38d2e9b5)
 2014-09-18 16:53:07 -07:00
Andrew Wang 47c9dcfe8e HDFS-7003. Add NFS Gateway support for reading and writing to encryption zones. (clamb via wang) 
(cherry picked from commit 70be56d093)
 2014-09-18 14:57:33 -07:00
Colin Patrick Mccabe 2b1c9c6210 HDFS-7047. Expose FileStatus#isEncrypted in libhdfs (cmccabe) 
(cherry picked from commit 570b8b468e)
 2014-09-18 11:42:53 -07:00
Andrew Wang 008e2f68f1 HDFS-7078. Fix listEZs to work correctly with snapshots. (wang) 
(cherry picked from commit 0ecefe6017)
 2014-09-17 21:28:15 -07:00
Andrew Wang b477d30e63 HDFS-7004. Update KeyProvider instantiation to create by URI. (wang) 
(cherry picked from commit 10e8602f32)
 2014-09-17 20:15:42 -07:00
Andrew Wang b05da10d0d Move some HDFS JIRAs to the correct CHANGES.txt 2014-09-17 18:08:24 -07:00
Colin Patrick Mccabe 6cb8ed0d22 HDFS-6843. Create FileStatus isEncrypted() method (clamb via cmccabe) 
(cherry picked from commit e3803d002c)
 2014-09-17 13:36:41 -07:00
Andrew Wang 5e54aae62b HDFS-6705. Create an XAttr that disallows the HDFS admin from accessing a file. (clamb via wang) 
(cherry picked from commit ea4e2e843e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2014-09-17 11:26:17 -07:00
Alejandro Abdelnur 1c847fdd61 HADOOP-11062. CryptoCodec testcases requiring OpenSSL should be run only if -Pnative is used. (asuresh via tucu) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/pom.xml
 2014-09-16 23:37:21 -07:00
Alejandro Abdelnur 94a1e68aa5 HADOOP-11096. KMS: KeyAuthorizationKeyProvider should verify the keyversion belongs to the keyname on decrypt. (tucu) 2014-09-16 23:21:17 -07:00
Andrew Wang fcd181bde1 HDFS-6789. TestDFSClientFailover.testFileContextDoesntDnsResolveLogicalURI and TestDFSClientFailover.testDoesntDnsResolveLogicalURI failing on jdk7. Contributed by Akira Ajisaka. 
(cherry picked from commit a044407b5c)
 2014-09-16 17:03:52 -07:00
Andrew Wang 23f34b75e4 HDFS-6851. Refactor EncryptionZoneWithId and EncryptionZone. (clamb via wang) 
(cherry picked from commit 33ce887c20)
 2014-09-16 16:57:06 -07:00
Benoy Antony 215216abeb HDFS-6799. The invalidate method in SimulatedFSDataset failed to remove (invalidate) blocks from the file system. Contributed by Megasthenis Asteris. 2014-09-16 16:27:25 -07:00
Alejandro Abdelnur 5d897026e4 HDFS-7006. Test encryption zones with KMS. (Anthony Young-Garner and tucu) 2014-09-16 14:37:04 -07:00
Colin Patrick Mccabe 9be338911b HDFS-6880. Adding tracing to DataNode data transfer protocol (iwasakims via cmccabe) 
(cherry picked from commit 56119fec96)
 2014-09-16 13:59:58 -07:00
Kihwal Lee 08abb320a7 HDFS-6965. NN continues to issue block locations for DNs with full 
disks. Contributed by Rushabh Shah.
(cherry picked from commit 0c26412be4)
 2014-09-16 09:06:43 -05:00
Alejandro Abdelnur e59f6771e8 HADOOP-10868. AuthenticationFilter should support externalizing the secret for signing and provide rotation support. (rkanter via tucu) 2014-09-15 17:10:43 -07:00
Kihwal Lee e6e31fbbba HDFS-7059. HAadmin transtionToActive with forceActive option can show 
confusing message. Contributed by Rushabh Shah.
(cherry picked from commit 2f847b3357)
 2014-09-15 16:30:22 -05:00
Andrew Wang d595557859 HDFS-7032. Add WebHDFS support for reading and writing to encryption zones. Contributed by Charles Lamb. 
(cherry picked from commit 43b0303008)
 2014-09-15 10:24:38 -07:00
Aaron T. Myers a0c54aeb00 HADOOP-10400. Incorporate new S3A FileSystem implementation. Contributed by Jordan Mendelson and Dave Wang. 2014-09-15 08:30:42 -07:00
Andrew Wang dd3e28d434 HDFS-7061. Add test to verify encryption zone creation after NameNode restart without saving namespace. Contributed by Stephen Chu. 
(cherry picked from commit fc741b5d78)
 2014-09-14 23:49:26 -07:00
cnauroth 993ede4db3 HDFS-7051. TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator. Contributed by Chris Nauroth. 
(cherry picked from commit d6ce26986e)
 2014-09-11 20:58:54 -07:00
Karthik Kambatla d22cd065ed Update CHANGES.txt to set the release date for 2.5.1 
(cherry picked from commit 17ffbe0864)
 2014-09-11 17:36:06 -07:00
cnauroth ac296f9522 HDFS-7042. Upgrade fails for Windows HA cluster due to file locks held during rename in JournalNode. Contributed by Chris Nauroth. 
(cherry picked from commit 80ac6aabce)
 2014-09-11 12:17:19 -07:00
Andrew Wang 6151c2bea7 HDFS-7045. Fix NameNode deadlock when opening file under /.reserved path. Contributed by Yi Liu. 
(cherry picked from commit 1e684995d7)
 2014-09-11 10:59:24 -07:00
Andrew Wang c90e15c559 HDFS-6621. Hadoop Balancer prematurely exits iterations. Contributed by Rafal Wodjdyla and Benjamin Bowman. 
(cherry picked from commit d1dab6cc63)
 2014-09-11 10:32:57 -07:00
Andrew Wang 0fae8a0e14 HDFS-6966. Add additional unit tests for encryption zones. Contributed by Stephen Chu. 
(cherry picked from commit bf64fce78c)
 2014-09-11 10:26:21 -07:00
Alejandro Abdelnur 16a4558fda HDFS-6776. Using distcp to copy data between insecure and secure cluster via webdhfs doesn't work. (yzhangal via tucu) 2014-09-09 22:20:36 -07:00
Konstantin V Shvachko c1df6f3b40 Revert HDFS-6940. 2014-09-09 17:34:25 -07:00
cnauroth c6fabbb9e9 Newly moved block replica been invalidated and deleted in TestBalancer. Contributed by Binglin Chang. 
(cherry picked from commit 2749fc677c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-09 11:43:21 -07:00
arp e736018601 HDFS-6981. Fix DN upgrade with layout version change. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-08 22:06:51 -07:00
arp 3259e8954f HDFS-6800. Support Datanode layout changes with rolling upgrade. (Contributed by James Thomas) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-09-08 22:06:12 -07:00
Colin McCabe da118bb113 HDFS-6482. Use block ID-based block layout on datanodes (James Thomas via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615223 13f79535-47bb-0310-9956-ffa450edef68

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2014-09-08 22:00:51 -07:00
Andrew Wang 176b07b480 Update CHANGES.txt for HDFS-6951 
(cherry picked from commit 7498dd790e)
 2014-09-08 17:01:53 -07:00
Andrew Wang dc4da242f2 HDFS-6951. Correctly persist raw namespace xattrs to edit log and fsimage. Contributed by Charles Lamb. 
(cherry picked from commit 04915a0814)
 2014-09-08 17:01:47 -07:00
Colin Patrick Mccabe bdcf5e940f HDFS-6036. Forcibly timeout misbehaving DFSClients that try to do no-checksum reads that extend too long. (cmccabe) 
(cherry picked from commit cad14aa916)
 2014-09-08 13:02:54 -07:00
Kihwal Lee d20047edda HDFS-7005. DFS input streams do not timeout. Contributed by Daryn Sharp. 
(cherry picked from commit 6a84f88c11)
 2014-09-08 14:44:47 -05:00
Haohui Mai 876062ac20 HDFS-6943. Improve NN allocateBlock log to include replicas' datanode IPs. Contributed by Ming Ma. 2014-09-08 11:07:56 -07:00
cnauroth b87d1d7d0a HDFS Credential Provider related Unit Test Failure. Contributed by Xiaoyu Yao. 
(cherry picked from commit a23144fd8a)
 2014-09-07 08:41:08 -07:00
arp eba06e61ce HDFS-6898. DN must reserve space for a full block when an RBW block is created. (Contributed by Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2014-09-06 21:04:21 -07:00
Konstantin V Shvachko 1ea3883553 HDFS-6940. Refactoring to allow ConsensusNode implementation. 
Contributed by Konstantin Shvachko.
 2014-09-06 12:04:49 -07:00
Alejandro Abdelnur 035112f251 HDFS-6986. DistributedFileSystem must get delegation tokens from configured KeyProvider. (zhz via tucu) 2014-09-05 22:33:58 -07:00
arp 27c53a1d67 HADOOP-11067 [HDFS-6998]. Fix CHANGES.txt 2014-09-05 11:18:30 -07:00