Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,372 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3162 Commits

Author SHA1 Message Date
yliu d4b6c4a3cd HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu) 2015-08-20 20:43:26 +08:00
Jing Zhao 4c005a4e16 HDFS-8867. Enable optimized block reports. Contributed by Daryn Sharp. 
(cherry picked from commit f61120d964)
(cherry picked from commit c0a4cd978a)
 2015-08-19 10:41:30 -07:00
Akira Ajisaka f98b5ae3bb HDFS-8852. HDFS architecture documentation of version 2.x is outdated about append write support. Contributed by Ajith S. 
(cherry picked from commit fc509f66d8)
(cherry picked from commit 38b344adfa)
 2015-08-18 23:33:28 +09:00
Vinayakumar B 4d94bc1d95 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
Moved to 2.6.1

(cherry picked from commit 84bf71295a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-08-14 16:15:10 +05:30
Vinayakumar B 90f3641728 HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang) 
Moved to 2.6.1

(cherry picked from commit 08bd4edf40)
 2015-08-14 12:38:59 +05:30
Vinayakumar B b06d342749 HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit fa2641143c)
 2015-08-14 11:46:58 +05:30
Vinayakumar B f40714f8d8 HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit f2b4bc9b6a)
 2015-08-14 11:38:28 +05:30
Vinayakumar B 5ba4065345 HDFS-7213. processIncrementalBlockReport performance degradation. Contributed by Eric Payne. 
Moved CHANGES.TXT entry to 2.6.1

(cherry picked from commit d25cb8fe12)
(cherry picked from commit 2c0063d420)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-08-14 11:27:46 +05:30
Akira Ajisaka b919c1c044 HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang. 
(cherry picked from commit 206d4933a5)
(cherry picked from commit b2a532f61c)
 2015-07-24 18:38:44 +09:00
Akira Ajisaka 3e793224f9 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 
(cherry picked from commit 18a91fe4df)
(cherry picked from commit b85bbca745)
 2015-07-21 11:42:21 +09:00
Kihwal Lee e263219829 HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee 
(cherry picked from commit 59388a8015)
 2015-07-14 14:08:15 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) dcafd3ccd2 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8)
 2015-07-06 16:41:38 -07:00
Yongjun Zhang 849b47525d HDFS-8659. Block scanner INFO message is spamming logs. Contributed by Yongjun Zhang. 
(cherry picked from commit 43a1288fff)
 2015-06-29 14:56:14 -07:00
Vinod Kumar Vavilapalli a36f090c55 Preparing for 2.7.2 development: mvn versions:set -DnewVersion=2.7.2 2015-06-28 16:41:12 -07:00
Vinod Kumar Vavilapalli 2e99675c72 Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5db)
 2015-06-28 16:33:49 -07:00
Arpit Agarwal f6550361f7 HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal) 2015-06-28 14:58:07 -07:00
Arpit Agarwal 4a8db40f3b HDFS-8626. Add test case for branch-2.7 2015-06-23 10:03:44 -07:00
Colin Patrick Mccabe 86c01d90ba HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 7b424f938c)
(cherry picked from commit cbd11681ce)

Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
 2015-06-22 14:46:57 -07:00
Arpit Agarwal 25b99e4816 HDFS-7164. Feature documentation for HDFS-6581. (Contributed by Arpit Agarwal) 2015-06-20 13:08:18 -07:00
Allen Wittenauer 0adb6d8292 HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) 
(cherry picked from commit 63613c79c1)
 2015-06-20 07:45:39 -07:00
Arpit Agarwal dc4171cb66 HDFS-8626. Reserved RBW space is not released if creation of RBW File fails. (Contributed by kanaka kumar avvaru) 2015-06-19 18:30:53 -07:00
Yongjun Zhang 8c03a079df HDFS-8633. Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys. Contributed by Ray Chiang. 
(cherry picked from commit b42f1ec13d)
 2015-06-19 09:51:48 -07:00
Kihwal Lee 6de4d52c31 HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 
(cherry picked from commit c74517c46b)
 2015-06-16 15:44:38 -05:00
Xiaoyu Yao fe35ba1df9 HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4e88ff5b27)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-16 12:51:34 -07:00
Yongjun Zhang f4ae56d605 HDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit b1d3ed02c1)
 2015-06-15 21:51:21 -07:00
Tsz-Wo Nicholas Sze 632cb542f6 HDFS-8361. Choose SSD over DISK in block placement. 2015-06-15 17:29:33 -07:00
Tsz-Wo Nicholas Sze 7a248f2e9e HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina 2015-06-15 16:13:56 -07:00
Arpit Agarwal a1810d75f0 HDFS-8600. TestWebHdfsFileSystemContract.testGetFileBlockLocations fails in branch-2.7. (Contributed by Arpit Agarwal) 2015-06-14 20:18:31 -07:00
Arpit Agarwal 1145d09e0a HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal) 2015-06-14 16:24:42 -07:00
Yongjun Zhang 6dda57a817 HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang. 
(cherry picked from commit b0dc291961)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java

(cherry picked from commit cd578fa289)
 2015-06-14 12:04:16 -07:00
Haohui Mai 3651feb49d HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai. 2015-06-11 19:14:48 -07:00
Arpit Agarwal 3ec9b63586 HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal) 2015-06-11 15:08:15 -07:00
Colin Patrick Mccabe 0bcc68337b HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit 587309bbba)
(cherry picked from commit 754dd3496c)
 2015-06-11 14:30:21 -07:00
Arpit Agarwal 105cadb421 HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:55 -07:00
Xiaoyu Yao 8c1a46ca4f HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:13:40 -07:00
Vinayakumar B dd98cfd328 HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 
(cherry picked from commit 0ed9c2d8fe)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
 2015-06-03 15:18:42 +05:30
Vinayakumar B 8090a6ee63 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java
 2015-06-03 12:20:54 +05:30
Colin Patrick Mccabe a7058caa72 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8)
(cherry picked from commit 12497b0543)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2015-06-02 20:07:35 -07:00
Colin Patrick Mccabe 323ffccfe9 HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c6425)
(cherry picked from commit 17e369511d)
 2015-06-02 11:46:12 -07:00
Kihwal Lee 1ff3f16ed0 HDFS-5215. dfs.datanode.du.reserved is not considered while computing 
available space ( Brahma Reddy Battula via Yongjun Zhang)
(cherry picked from commit 66763bb06f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-02 08:14:57 -05:00
Xiaoyu Yao c550825673 HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran. 
(cherry picked from commit 05e04f34f2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
 2015-05-21 12:49:49 -07:00
Kihwal Lee 7ac66f8712 HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts. 
(cherry picked from commit 8860e352c3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2015-05-19 13:17:45 -05:00
Tsz-Wo Nicholas Sze 59d1b4a323 HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma 2015-05-19 03:17:51 +08:00
cnauroth 82530ba0f1 HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth. 
(cherry picked from commit ddc4afeff2)
 2015-05-13 12:26:02 -07:00
Tsz-Wo Nicholas Sze 2113e0a3f2 HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore 2015-05-13 11:59:01 -07:00
Vinayakumar B ef2488e7f0 HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R) 
(cherry picked from commit 065d8f2a34)
 2015-05-13 17:30:03 +05:30
Kihwal Lee 1e4a686ea9 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah. 
(cherry picked from commit ea11590aad)
 2015-05-11 17:11:51 -05:00
Kihwal Lee f57b1bfbdd HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah. 
(cherry picked from commit 2d4ae3d18b)
 2015-05-08 16:38:43 -05:00
Kihwal Lee bbb9f6aafe HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6f622672b6)
 2015-05-08 09:32:44 -05:00
Vinayakumar B 1fbefe586b HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
Added one missed change, missed while resolving conflict

(cherry picked from commit 3ee1b50dad)
 2015-05-08 15:42:17 +05:30
Vinayakumar B 99309a9fec HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
(cherry picked from commit e01c1a979c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md

(cherry picked from commit 3a894b4563)
 2015-05-08 15:42:02 +05:30
Tsz-Wo Nicholas Sze 74bd9666d1 HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 2015-05-07 11:38:15 -07:00
Colin Patrick Mccabe e056e0a3c1 HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe) 
(cherry picked from commit fcd4cb7516)
(cherry picked from commit 1f01d8347a)
 2015-05-05 11:07:13 -07:00
Vinayakumar B 69d837428c HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Reverted earlier commit

(cherry picked from commit 318081ccd7)
 2015-05-05 11:07:59 +05:30
Colin Patrick Mccabe 472fd563e4 HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe) 
(cherry picked from commit b82567d455)
(cherry picked from commit 9edea9507d)
 2015-05-01 11:52:08 -07:00
Arun Suresh 4e6ff8c78b HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh) 
(cherry picked from commit 922b7ed21d)
 2015-05-01 11:42:29 -07:00
Akira Ajisaka 185a1ff275 HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao. 
(cherry picked from commit de9404f02f)
(cherry picked from commit fdb886d656)
 2015-04-30 20:13:47 +09:00
Haohui Mai 6ac2b5712b HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. 2015-04-29 11:13:03 -07:00
Haohui Mai 4b9147ef8c HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai. 2015-04-28 18:06:22 -07:00
Tsz-Wo Nicholas Sze 6b20b325f4 HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su 2015-04-28 13:07:20 -07:00
Colin Patrick Mccabe 176d22b440 HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) 
(cherry picked from commit a8898445dc)
(cherry picked from commit 3d0385c3c8)
 2015-04-23 19:28:38 -07:00
Colin Patrick Mccabe c2e6d2589a HADOOP-11802. Fix renamed constant in test file in backport to 2.7.1 (cmccabe) 2015-04-23 19:16:28 -07:00
Colin Patrick Mccabe 496afb5e1a HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe) 
(cherry picked from commit a0e0a63209)
(cherry picked from commit 788b76761d)
 2015-04-23 19:09:50 -07:00
Tsz-Wo Nicholas Sze 5b4eda7573 HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore 2015-04-23 11:57:16 -07:00
Arpit Agarwal b17d365fa1 HDFS-8163. Using monotonicNow for block report scheduling causes test failures on recently restarted systems. (Arpit Agarwal) 2015-04-21 11:17:27 -07:00
Vinod Kumar Vavilapalli 7dd5f42a76 Set the release date for 2.7.0 
(cherry picked from commit d52de61544)
 2015-04-20 20:18:38 -07:00
Vinayakumar B 1c8d54bc04 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
(cherry picked from commit 867d5d2675)
 2015-04-21 07:45:59 +05:30
Arpit Agarwal cc42ad03d9 HDFS-8179. DFSClient#getServerDefaults returns null within 1 hour of system start. (Contributed by Xiaoyu Yao) 2015-04-20 15:42:58 -07:00
Jitendra Pandey 0743f128a0 HDFS-8153. Error Message points to wrong parent directory in case of path component name length error. Contributed by Anu Engineer. 
cherry picked from 369ddc67bd
 2015-04-16 22:23:35 -07:00
Akira Ajisaka cffc801ecb Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0feb.
 2015-04-16 16:46:19 -05:00
Akira Ajisaka 35e1936f84 HDFS-8149. The footer of the Web UI "Hadoop, 2014" is old. Contributed by Brahma Reddy Battula. 
(cherry picked from commit de0f1700c1)
(cherry picked from commit 6d183561ff)
 2015-04-15 15:32:29 -05:00
Jing Zhao e1ce0700eb HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina. 
(cherry picked from commit b172d03595)
(cherry picked from commit 725621d95a)
 2015-04-15 13:11:45 -07:00
Jing Zhao 9e11ae684a HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao. 
(cherry picked from commit 4c097e473b)
(cherry picked from commit d4dd97eabd)
 2015-04-15 12:41:34 -07:00
Jing Zhao 50c8d06314 HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao. 
(cherry picked from commit fddd55279d)
(cherry picked from commit 38b031d6ba)
 2015-04-14 15:15:36 -07:00
Arun Suresh 0a95b4f8aa HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh) 
(cherry picked from commit 7660da95cb)
 2015-04-10 11:43:49 -07:00
Konstantin V Shvachko e964bbc856 HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko 2015-04-09 22:04:48 -07:00
Vinod Kumar Vavilapalli a7fa46c2e0 Preparing for 2.7.1 development: mvn versions:set 2015-04-09 14:13:30 -07:00
Vinod Kumar Vavilapalli 2b44fcaa53 Adding release 2.7.1 to CHANGES.txt 
(cherry picked from commit 623fd46c1e)
 2015-04-09 14:01:23 -07:00
Colin Patrick Mccabe 4d5b1fbdec HDFS-8063: Fix intermittent test failures in TestTracing (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 61dc2ea3fe)
(cherry picked from commit 8dac245920)
 2015-04-09 11:42:29 -07:00
Kihwal Lee 9a111fcd1d HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma. 
(cherry picked from commit 5a540c3d31)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-09 10:01:44 -05:00
Haohui Mai 84ba4db2a0 Revert HDFS-7813. 2015-04-08 16:03:35 -07:00
Haohui Mai eb53d6b967 Revert HDFS-7808. 2015-04-08 16:00:26 -07:00
Arpit Agarwal 12739b541b HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal) 2015-04-08 11:38:42 -07:00
cnauroth 5021f52f7b HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao. 
(cherry picked from commit 672ed462965ea7da10ce3df48c02e2a3bd13f0ae)
(cherry picked from commit 950dc1b1995191327e6b919ee17a87e59b749264)
 2015-04-07 13:33:38 -07:00
Konstantin V Shvachko 2cb9dac9a3 HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko. 2015-04-06 23:35:01 -07:00
Colin Patrick Mccabe 3f1d1d0b48 HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe) 
(cherry picked from commit 28bebc81db)
(cherry picked from commit a827089905)
 2015-04-06 09:10:34 -07:00
Vinod Kumar Vavilapalli 5387a289fc HADOOP-11776. Fixed the broken JDiff support in Hadoop 2. Contributed by Li Lu. 
(cherry picked from commit 4b3948ea36)
 2015-04-04 13:53:34 -07:00
Vinod Kumar Vavilapalli 996b0ca173 HADOOP-11377. Added Null.java without which jdiff completely flops. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit 5370e7128b)
 2015-04-04 13:48:58 -07:00
Colin Patrick Mccabe aa5eab8ec4 HDFS-8051. Fix missing import statement from backport. 2015-04-03 19:07:25 -07:00
Colin Patrick Mccabe 54b88943ff HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit ef591b1d6a)
(cherry picked from commit b26ba22a90)
 2015-04-03 16:57:06 -07:00
Colin Patrick Mccabe 520043b438 HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 023133cef9)
(cherry picked from commit 0c5069c432)
 2015-04-03 14:27:09 -07:00
cnauroth 6855bbc6ca HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit db80e42891)
(cherry picked from commit 28e0602e7b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-03 10:45:15 -07:00
Jing Zhao b08bc43c9b HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao. 
(cherry picked from commit 72f6bd4893)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 2c1f33d178)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-02 21:27:32 -07:00
Tsz-Wo Nicholas Sze acfe44e5b0 HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao 2015-04-02 11:14:04 -07:00
Brandon Li a445f8b675 HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot 
(cherry picked from commit 4d14816c26)
(cherry picked from commit 27970b3f46)
 2015-04-01 17:30:00 -07:00
Haohui Mai 8ebbbc6ebd HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:51:25 -07:00
Haohui Mai 86c0c6b044 HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. 2015-03-30 12:16:25 -07:00
Kihwal Lee a0ed29a058 HDFS-7742. Favoring decommissioning node for replication can cause a block to stay 
underreplicated for long periods. Contributed by Nathan Roberts.
(cherry picked from commit 04ee18ed48)
 2015-03-30 10:11:47 -05:00
Haohui Mai d481a8d79b HADOOP-11748. The secrets of auth cookies should not be specified in configuration in clear text. Contributed by Li Lu and Haohui Mai. 2015-03-26 17:35:54 -07:00
Colin Patrick Mccabe 9411cf9b65 HDFS-7410. Support CreateFlags with append() to support hsync() for appending streams (Vinayakumar B via Colin P. McCabe) 
(cherry picked from commit 61df1b27a7)
(cherry picked from commit 2a750c9aa6)
 2015-03-26 13:29:17 -07:00
Arpit Agarwal c808d5fe55 HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao) 2015-03-26 10:48:03 -07:00
Kihwal Lee 370c91461f HDFS-7963. Fix expected tracing spans in TestTracing. Contributed by Masatake Iwasaki. 2015-03-26 08:45:39 -05:00
Jitendra Pandey d286673c60 HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh. 
(cherry picked from commit 456cec127b)
 2015-03-24 16:17:06 -07:00
Brandon Li 14d1cccfad HDFS-7977. NFS couldn't take percentile intervals. Contributed by Brandon Li 
(cherry picked from commit 570a83ae80)
(cherry picked from commit 260dbe96c3)
 2015-03-24 10:53:20 -07:00
Brandon Li cc5922607a HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li 
(cherry picked from commit 51f1f49374)
(cherry picked from commit 95bde88986)
 2015-03-24 10:30:19 -07:00
Andrew Wang a7f447d460 HDFS-7961. Trigger full block report after hot swapping disk. Contributed by Eddy Xu. 
(cherry picked from commit 6413d34986)
 2015-03-24 09:07:27 -07:00
Plamen Jeliazkov da318cbbde HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov. 2015-03-23 23:09:52 -07:00
Andrew Wang af0af28afc HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu. 
(cherry picked from commit 50ee8f4e67)
 2015-03-23 22:01:37 -07:00
Tsz-Wo Nicholas Sze 87079cde7d HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula 2015-03-24 13:52:18 +09:00
cnauroth 8e1c33e703 HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 2c238ae4e0)
(cherry picked from commit 01c0bcb176)
 2015-03-23 16:30:48 -07:00
Akira Ajisaka 2742f12b58 HDFS-7881. TestHftpFileSystem#testSeek fails in branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit fad8c78173)
 2015-03-24 06:25:21 +09:00
Brandon Li 6b9f2d9f39 HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts. Contributed by Brandon Li 
(cherry picked from commit 36af4a913c)
(cherry picked from commit 503d8e4164)
 2015-03-23 10:13:11 -07:00
Kihwal Lee 08f2f7ed34 HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B 
(cherry picked from commit 99a8dcd19528b265d4fda9ae09a17e4af52f2782)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
(cherry picked from commit 63b7870aec007081c82427587cb6e1d38e2c70f8)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-20 14:06:02 -05:00
Jing Zhao 03fb9b4ce0 HDFS-7957. Truncate should verify quota before making changes. Contributed by Jing Zhao. 
(cherry picked from commit d368d3647a)
(cherry picked from commit 6be52e42a9)
 2015-03-20 11:56:39 -07:00
yliu faeb8b9b53 HDFS-7962. Remove duplicated logs in BlockManager. (yliu) 2015-03-20 05:32:19 +08:00
yliu 0da3fcb4b0 HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu) 2015-03-19 23:26:26 +08:00
Kihwal Lee 4e967d0bcd HDFS-7932. Speed up the shutdown of datanode during rolling upgrade. Contributed by Kihwal Lee. 
(cherry picked from commit 61a4c7fc98)
 2015-03-19 12:29:16 -05:00
Vinayakumar B 76ce2e8ea0 HDFS.7849. Update documentation for enabling a new feature in rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 5be5129d93)
 2015-03-19 19:10:23 +05:30
Kihwal Lee b8f269af9d HDFS-7816. Unable to open webhdfs paths with "+". Contributed by Haohui Mai 
(cherry picked from commit e79be0ee12)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit ceb39c1cc6)
 2015-03-19 08:04:19 -05:00
Colin Patrick Mccabe 9e61835678 HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 43b41f2241)
(cherry picked from commit 219eb22c15)
 2015-03-18 19:08:53 -07:00
Jing Zhao 0bf549728d HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao. 
(cherry picked from commit c7c71cdba5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 5a5b244648)
 2015-03-18 19:00:06 -07:00
Jing Zhao 4b8de154aa HDFS-7943. Append cannot handle the last block with length greater than the preferred block size. Contributed by Jing Zhao. 
(cherry picked from commit bee5a6a64a)
(cherry picked from commit 6dcc79507d)
 2015-03-18 18:47:21 -07:00
Colin Patrick Mccabe 94976cb369 HDFS-7054. Make DFSOutputStream tracing more fine-grained (cmccabe) 
(cherry picked from commit 8234fd0e10)
(cherry picked from commit 79c07bbaca)
 2015-03-18 18:15:34 -07:00
Haohui Mai 388696c089 HDFS-7945. The WebHdfs system on DN does not honor the length parameter. Contributed by Haohui Mai. 2015-03-18 16:20:07 -07:00
Arpit Agarwal bb986988d4 HDFS-7914. TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use. (Contributed by Xiaoyu Yao) 2015-03-18 14:05:37 -07:00
Haohui Mai 482aeee784 HDFS-7697. Mark the PB OIV tool as experimental. Contributed by Lei (Eddy) Xu. 2015-03-18 13:34:21 -07:00
Arpit Agarwal 5f7926dcf0 Fix CHANGES.txt for HDFS-7722. 2015-03-18 12:53:02 -07:00
Arpit Agarwal 2b589da779 HDFS-7951. Fix NPE for TestFsDatasetImpl#testAddVolumeFailureReleasesInUseLock on Linux. (Contributed by Xiaoyu Yao) 2015-03-18 12:46:17 -07:00
Arpit Agarwal ae6d273d63 HDFS-7950. Fix TestFsDatasetImpl#testAddVolumes failure on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:35:41 -07:00
Arpit Agarwal 8944bb3d2e HDFS-7948. TestDataNodeHotSwapVolumes#testAddVolumeFailures failed on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:26:22 -07:00
Haohui Mai 5293ae0311 HDFS-7953. NN Web UI fails to navigate to paths that contain #. Contributed by kanaka kumar avvaru. 2015-03-18 11:08:49 -07:00
Arpit Agarwal 26c35438f3 HDFS-7946. TestDataNodeVolumeFailureReporting NPE on Windows. (Contributed by Xiaoyu Yao) 2015-03-17 21:30:13 -07:00
Colin Patrick Mccabe d2dad74421 HDFS-7940. Add tracing to DFSClient#setQuotaByStorageType (Rakesh R via Colin P. McCabe) 
(cherry picked from commit d8846707c5)
(cherry picked from commit 455d4aa8a1)
 2015-03-17 11:02:05 -07:00
yliu ef9d46dcb6 HDFS-7838. Expose truncate API for libhdfs. (yliu) 2015-03-17 07:25:58 +08:00
Colin Patrick Mccabe 861cc05092 HDFS-7722. DataNode#checkDiskError should also remove Storage when error is found. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit b49c3a1813)
(cherry picked from commit 7455412a24)
 2015-03-16 18:42:21 -07:00
Colin Patrick Mccabe 647d9ae27a HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe) 
(cherry picked from commit bc9cb3e271)
(cherry picked from commit c71b54fa32)
 2015-03-16 18:39:58 -07:00
Konstantin V Shvachko ad4d704fbb HDFS-7886. Fix TestFileTruncate falures. Contributed by Plamen Jeliazkov and Konstantin Shvachko. 2015-03-16 12:43:46 -07:00
Konstantin V Shvachko 41530b4173 HDFS-2605. Remove redundant "Release 0.21.1" section from CHANGES.txt. Contributed by Allen Wittenauer. 2015-03-13 13:45:47 -07:00
Konstantin V Shvachko f6bdcd938e HDFS-7903. Cannot recover block after truncate and delete snapshot. Contributed by Plamen Jeliazkov. 2015-03-13 12:45:47 -07:00
Kihwal Lee 116a7f1a16 HDFS-7435. PB encoding of block reports is very inefficient. Contributed by Daryn Sharp. 
(cherry picked from commit d324164a51)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
(cherry picked from commit 464271a5ed)
 2015-03-13 14:42:02 -05:00
Brandon Li cdeb1079ea HDFS-7926. NameNode implementation of ClientProtocol.truncate(..) is not idempotent. Contributed by Tsz Wo Nicholas Sze 
(cherry picked from commit f446669afb)
(cherry picked from commit af80a98ace)
 2015-03-13 10:47:25 -07:00
Tsz-Wo Nicholas Sze a1fffc3fcc HDFS-6833. DirectoryScanner should not register a deleting block with memory of DataNode. Contributed by Shinichi Yamashita 2015-03-13 02:29:24 +08:00
Akira Ajisaka 718cd0f1f4 HDFS-7880. Remove the tests for legacy Web UI in branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6c80a3d34a)
 2015-03-12 10:55:47 +09:00
Colin Patrick Mccabe 8241228db7 HDFS-7830. DataNode does not release the volume lock when adding a volume fails. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 5c1036d598)
(cherry picked from commit eefca23e8c)
 2015-03-12 10:49:48 +09:00
Haohui Mai 245f7b2a77 HADOOP-11633. Convert remaining branch-2 .apt.vm files to markdown. Contributed by Masatake Iwasaki. 2015-03-11 14:31:16 -07:00
cnauroth 0fe5f5b5b8 HDFS-7491. Add incremental blockreport latency to DN metrics. Contributed by Ming Ma. 
(cherry picked from commit fb34f45727)
(cherry picked from commit 5e99b7cd2f)
 2015-03-11 14:12:06 -07:00
Jing Zhao 368ab2cd37 HDFS-7898. Change TestAppendSnapshotTruncate to fail-fast. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit e43882e84a)
(cherry picked from commit c7105fcff0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-10 10:31:41 -07:00
Akira Ajisaka c9a2e6f5db HDFS-6806. HDFS Rolling upgrade document should mention the versions available. Contributed by J.Andreina. 
(cherry picked from commit 82db3341bf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-10 11:27:30 +09:00
Tsuyoshi Ozawa 28f4e6b222 HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 
(cherry picked from commit b46f9e72db)
 2015-03-09 19:54:45 +09:00
Chris Douglas 3bc4f3502a HDFS-7411. Change decommission logic to throttle by blocks rather 
than nodes in each interval. Contributed by Andrew Wang
 2015-03-08 18:32:20 -07:00
Brandon Li c8af3b2768 HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li 
(cherry picked from commit 0f8ecb1d0c)
 2015-03-06 15:22:20 -08:00
Haohui Mai ab251fd355 HDFS-7818. OffsetParam should return the default value instead of throwing NPE when the value is unspecified. Contributed by Eric Payne. 2015-03-06 14:26:47 -08:00
Jing Zhao 994dadb9ba HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 24db0812be)
 2015-03-06 10:57:38 -08:00
Jing Zhao c5d52c7b52 HDFS-7855. Separate class Packet from DFSOutputStream. Contributed by Li Bo. 
(cherry picked from commit 952640fa4c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-05 11:11:08 -08:00
Tsuyoshi Ozawa f805d48b19 HADOOP-11648. Set DomainSocketWatcher thread name explicitly. Contributed by Liang Xie. 
(cherry picked from commit 74a4754d1c)
 2015-03-05 16:06:37 +09:00
Tsz-Wo Nicholas Sze f9a2007aff HDFS-7746. Add a test randomly mixing append, truncate and snapshot operations. 2015-03-05 10:23:14 +08:00
Dongming Liang ada545d3b7 HDFS-1522. Combine two BLOCK_FILE_PREFIX constants into one. Contributed by Dongming Liang. 2015-03-04 17:52:25 -08:00
Kihwal Lee f85530f649 HDFS-7434. DatanodeID hashCode should not be mutable. Contributed by Daryn Sharp. 
(cherry picked from commit 722b479469)
 2015-03-04 17:23:00 -06:00
Jing Zhao b2ccf54c14 HDFS-7535. Utilize Snapshot diff report for distcp. Contributed by Jing Zhao. 
(cherry picked from commit ed70fa142c)
 2015-03-04 10:32:39 -08:00
Haohui Mai d020544ef6 HDFS-7879. hdfs.dll does not export functions of the public libhdfs API. Contributed by Chris Nauroth. 2015-03-04 09:17:29 -08:00
Vinayakumar B c24448c27b HDFS-7869. Inconsistency in the return information while performing rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 3560180b6e)
 2015-03-04 14:39:51 +05:30
Akira Ajisaka 31b3f84601 HADOOP-11666. Revert the format change of du output introduced by HADOOP-6857. Contributed by Byron Wong. 2015-03-03 21:46:28 -08:00
Aaron T. Myers bce3d442ff HDFS-7682. {{DistributedFileSystem#getFileChecksum}} of a snapshotted file includes non-snapshotted content. Contributed by Charles Lamb. 
(cherry picked from commit f2d7a67a2c)
 2015-03-03 18:09:31 -08:00
Haohui Mai 65bfde552a HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. 2015-03-03 17:58:21 -08:00
Arpit Agarwal 7e6624c218 HDFS-7757. Misleading error messages in FSImage.java. (Contributed by Brahma Reddy Battula) 2015-03-03 10:55:32 -08:00
Jing Zhao 6090f51725 HDFS-7871. NameNodeEditLogRoller can keep printing 'Swallowing exception' message. Contributed by Jing Zhao. 
(cherry picked from commit b442aeec95)
 2015-03-02 20:23:10 -08:00
Haohui Mai 651aa8761e HDFS-7785. Improve diagnostics information for HttpPutFailedException. Contributed by Chengbing Liu. 2015-03-02 15:35:19 -08:00
Gera Shegalov 52d0724a20 HDFS-7789. DFSck should resolve the path to support cross-FS symlinks. (gera) 
(cherry picked from commit cbb492578e)
 2015-03-02 01:05:15 -08:00
Tsz-Wo Nicholas Sze a5f3156b30 HDFS-7439. Add BlockOpResponseProto's message to the exception messages. Contributed by Takanobu Asanuma 2015-03-02 15:05:06 +08:00
Akira Ajisaka d664705fd1 HDFS-5853. Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml (aajisaka) 
(cherry picked from commit aa55fd3096)
 2015-03-01 01:18:36 -08:00
Tsz-Wo Nicholas Sze c52636df3f recommit "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir." 
(cherry picked from commit acc172e371)
 2015-02-27 14:20:26 -08:00
Akira Ajisaka 0f9289e848 HDFS-7685. Document dfs.namenode.heartbeat.recheck-interval in hdfs-default.xml. Contributed by Kai Sasaki. 
(cherry picked from commit 8719cdd4f6)
 2015-02-27 12:18:46 -08:00
Tsz-Wo Nicholas Sze 657b027bb2 HDFS-7308. Change the packet chunk size computation in DFSOutputStream in order to enforce packet size <= 64kB. Contributed by Takuya Fukudome 2015-02-27 23:46:42 +08:00
Vinayakumar B bc60404eaf HDFS-6753. Initialize checkDisk when DirectoryScanner not able to get files list for scanning (Contributed by J.Andreina) 
(cherry picked from commit 4f75b15628)
 2015-02-27 16:37:03 +05:30
Tsz-Wo Nicholas Sze d223a4a594 Revert "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir." 
This reverts commit acc172e371.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-27 18:27:19 +08:00
cnauroth 5dfb793b05 HDFS-7774. Unresolved symbols error while compiling HDFS on Windows 7/32 bit. Contributed by Kiran Kumar M R. 
(cherry picked from commit c6d5b37a1c)
 2015-02-26 13:37:58 -08:00
Colin Patrick Mccabe 8b3b9568b6 HDFS-7819. Log WARN message for the blocks which are not in Block ID based layout (Rakesh R via Colin P. McCabe) 
(cherry picked from commit f0c980abed)
 2015-02-26 12:03:11 -08:00
Vinayakumar B f0f3d1937d HDFS-7832. Show 'Last Modified' in Namenode's 'Browse Filesystem' (Contributed by Vinayakumar B) 
(cherry picked from commit 166eecf687)
 2015-02-26 14:37:03 +05:30
Tsz-Wo Nicholas Sze 22b13a0ddb HDFS-7537. Add "UNDER MIN REPL'D BLOCKS" count to fsck. Contributed by GAO Rui 2015-02-26 11:47:04 +08:00
Tsz-Wo Nicholas Sze 12fe3afcd3 HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade. 2015-02-26 10:15:49 +08:00
Benoy Antony 4e400030f6 HDFS-7467. Provide storage tier information for a directory via fsck. (Benoy Antony) 2015-02-25 16:46:41 -08:00
Colin Patrick Mccabe d389a1ae98 HDFS-7495. Remove updatePosition argument from DFSInputStream#getBlockAt() (cmccabe) 
(cherry picked from commit caa42adf20)
 2015-02-25 13:39:02 -08:00
Colin Patrick Mccabe 5c0073e706 HDFS-7668. Backport "Convert site documentation from apt to markdown" to branch-2 (Masatake Iwasaki via Colin P. McCabe) 
(backported from commit 2f1e5dc628)

Conflicts:
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithNFS.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithQJM.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsDesign.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsQuotaAdminGuide.apt.vm
 2015-02-24 16:18:18 -08:00
Andrew Wang efb7e287f4 HDFS-7763. fix zkfc hung issue due to not catching exception in a corner case. Contributed by Liang Xie. 
(cherry picked from commit 7105ebaa9f)
 2015-02-24 15:31:18 -08:00
Jing Zhao 8346427929 HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. 
(cherry picked from commit 73bcfa99af)
 2015-02-24 10:34:42 -08:00
Tsuyoshi Ozawa 280dda5bcd HDFS-7008. xlator should be closed upon exit from DFSAdmin#genericRefresh(). (ozawa) 
(cherry picked from commit b53fd7163b)
 2015-02-24 23:59:59 +09:00
Colin Patrick Mccabe 453b210d0a HDFS-7805. NameNode recovery prompt should be printed on console (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit faaddb6ecb)
 2015-02-23 19:06:01 -08:00
Colin Patrick Mccabe f4da6f7193 HDFS-7807. libhdfs htable.c: fix htable resizing, add unit test (cmccabe) 
(cherry picked from commit 585768667e)
 2015-02-23 18:45:44 -08:00
cnauroth 657a6e389b HDFS-7009. Active NN and standby NN have different live nodes. Contributed by Ming Ma. 
(cherry picked from commit 769507bd7a)
 2015-02-23 15:12:36 -08:00
Arpit Agarwal 83f779e49a HDFS-7806. Refactor: move StorageType from hadoop-hdfs to hadoop-common. (Contributed by Xiaoyu Yao) 2015-02-21 15:48:59 -08:00
yliu eca1588db8 HDFS-7740. Test truncate with DataNodes restarting. (yliu) 2015-02-21 06:33:58 +08:00
cnauroth 9f06fb3aef HDFS-7813. TestDFSHAAdminMiniCluster#testFencer testcase is failing frequently. Contributed by Rakesh R. 
(cherry picked from commit 0d6af574e0)
 2015-02-20 17:01:22 -08:00
cnauroth e09ba94f28 HDFS-7773. Additional metrics in HDFS to be accessed via jmx. Contributed by Anu Engineer. 2015-02-20 12:38:34 -08:00
cnauroth ad77dfc3e4 HDFS-7814. Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota". Contributed by Xiaoyu Yao. 
(cherry picked from commit 8c6ae0d619)
 2015-02-20 12:22:00 -08:00
Kihwal Lee b9157f92fc HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. 
(cherry picked from commit 7ae5255a16)
 2015-02-20 09:09:56 -06:00
Harsh J 5c2c6b00dd HDFS-7752. Improve description for "dfs.namenode.num.extra.edits.retained" and "dfs.namenode.num.checkpoints.retained" properties on hdfs-default.xml. Contributed by Wellington Chevreuil. 
(cherry picked from commit b9a17909ba)
 2015-02-20 19:21:34 +05:30
yliu fee29e4a4f HDFS-7656. Expose truncate API for HDFS httpfs. (yliu) 2015-02-19 08:38:25 +08:00
Haohui Mai 3605928bd0 HDFS-7808. Remove obsolete -ns options in in DFSHAAdmin.java. Contributed by Arshad Mohammad. 2015-02-18 15:14:53 -08:00
cnauroth b1b1e05e18 HDFS-7772. Document hdfs balancer -exclude/-include option in HDFSCommands.html. Contributed by Xiaoyu Yao. 2015-02-18 12:03:00 -08:00
Akira Ajisaka 6d6c684567 HDFS-7780. Update use of Iterator to Iterable in DataXceiverServer and SnapshotDiffInfo. Contributed by Ray Chiang. 
(cherry picked from commit 6dc8812a95)
 2015-02-17 14:49:03 -08:00
Haohui Mai f92a4904bf HDFS-6662. WebHDFS cannot open a file if its path contains "%". Contributed by Gerson Carlos. 2015-02-17 13:05:00 -08:00
Kihwal Lee 8b37b4a78b HDFS-7795. Show warning if not all favored nodes were chosen by namenode. Contributed by Kihwal Lee. 
(cherry picked from commit db6606223c)
 2015-02-17 13:06:41 -06:00
Uma Maheswara Rao G 2cbac36fd3 HDFS-4266. BKJM: Separate write and ack quorum (Rakesh R via umamahesh) 
(cherry picked from commit f0412de1c1)
 2015-02-17 21:31:43 +05:30
Uma Maheswara Rao G 35fecb5306 HDFS-7797. Add audit log for setQuota operation (Rakesh R via umamahesh) 
(cherry picked from commit f24a56787a)
 2015-02-17 20:16:34 +05:30
Tsuyoshi Ozawa 00fb0710b6 HADOOP-11600. Fix up source codes to be compiled with Guava 17.0. (ozawa) 
(cherry picked from commit 2f0f756b26)
 2015-02-17 21:57:58 +09:00
Vinayakumar B 91a5d92916 HDFS-7703. Support favouredNodes for the append for new blocks ( Contributed by Vinayakumar B) 
(cherry picked from commit 89a5449280)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-17 15:19:03 +05:30
yliu 54e33baaf6 HDFS-7798. Checkpointing failure caused by shared KerberosAuthenticator. (Chengbing Liu via yliu) 2015-02-17 07:50:14 +08:00
cnauroth 441dfa4867 HDFS-7604. Track and display failed DataNode storage locations in NameNode. Contributed by Chris Nauroth. 
(cherry picked from commit 9729b244de)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
 2015-02-16 14:44:49 -08:00
Arpit Agarwal c6aa6cbfe5 HDFS-7775. Use consistent naming for NN-internal quota related types and functions. (Contributed bu Xiaoyu Yao) 2015-02-13 21:01:54 -08:00
cnauroth 751b834ad1 HDFS-7778. Rename FsVolumeListTest to TestFsVolumeList and commit it to branch-2. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 2efb2347a9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsVolumeList.java
 2015-02-13 16:30:53 -08:00
Akira Ajisaka 3ad1d8d533 HDFS-4265. BKJM doesn't take advantage of speculative reads. Contributed by Rakesh R. 
(cherry picked from commit 0d521e3326)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-13 15:25:13 -08:00
Colin Patrick Mccabe 7b9c978df2 HDFS-7686. Re-add rapid rescan of possibly corrupt block feature to the block scanner (cmccabe) 
(cherry picked from commit 8bb9a5000e)
 2015-02-13 15:19:04 -08:00
Colin Patrick Mccabe 6edcb9a503 HDFS-7721. The HDFS BlockScanner may run fast during the first hour (cmccabe) 
(cherry picked from commit 115428176e)
 2015-02-13 15:18:10 -08:00
Colin Patrick Mccabe a54b4a7fc6 HDFS-7430 backport addendum. Add missing change. (cmccabe) 2015-02-13 15:11:14 -08:00
Colin Patrick Mccabe a8c6a96f3b HDFS-7430. Refactor the BlockScanner to use O(1) memory and use multiple threads (cmccabe) 
(cherry picked from commit 6e62a1a672)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeListTest.java
 2015-02-13 15:04:06 -08:00
Arpit Agarwal f507791554 HDFS-7776. Adding additional unit tests for Quota By Storage Type. (Contributed by Xiaoyu Yao) 2015-02-13 12:02:54 -08:00
Allen Wittenauer 730597c20c HDFS-316. Balancer should run for a configurable # of iterations (Xiaoyu Yao via aw) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
 2015-02-12 18:02:23 -08:00
Colin Patrick Mccabe a9a361c549 HDFS-7790. Do not create optional fields in DFSInputStream unless they are needed (cmccabe) 
(cherry picked from commit 871cb56152)
 2015-02-12 17:49:11 -08:00
Akira Ajisaka 5696376e29 HDFS-7684. The host:port settings of the deamons should be trimmed before use. Contributed by Anu Engineer. 
(cherry picked from commit 9235e2fb4e94a86671e1494f293c973cf572281c)
 2015-02-12 17:40:48 -08:00
Arpit Agarwal 676fc2d1dc HDFS-7723. Quota By Storage Type namenode implemenation. (Contributed by Xiaoyu Yao) 2015-02-12 10:50:14 -08:00
Colin Patrick Mccabe e35788aa5a HDFS-7694. FSDataInputStream should support "unbuffer" (cmccabe) 
(cherry picked from commit 6b39ad0865)
 2015-02-12 10:42:14 -08:00
Jing Zhao 2156e38d5e HDFS-3689. Add support for variable length block. Contributed by Jing Zhao. 
(cherry picked from commit 2848db814a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirConcatOp.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
 2015-02-12 10:23:13 -08:00
Kihwal Lee 146f559097 Adding missing files from HDFS-7404 
(cherry picked from commit 8d7215d40f)
 2015-02-12 11:18:41 -06:00
Kihwal Lee 39594301e1 HDFS-7704. DN heartbeat to Active NN may be blocked and expire if connection to Standby NN continues to time out. Contributed by Rushabh Shah. 
(cherry picked from commit 38262779bb)
 2015-02-12 09:17:10 -06:00
yliu f8d0825f39 HDFS-7761. cleanup unnecssary code logic in LocatedBlock. (yliu) 2015-02-12 03:09:38 +08:00
Tsz-Wo Nicholas Sze 65a6cf47ec HDFS-6133. Add a feature for replica pinning so that a pinned replica will not be moved by Balancer/Mover. Contributed by zhaoyunjiong 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
 2015-02-11 15:16:34 -08:00
Colin Patrick Mccabe 2ca76df21a HDFS-7771. fuse_dfs should permit FILE: on the front of KRB5CCNAME (cmccabe) 
(cherry picked from commit 50625e660a)
 2015-02-11 12:14:56 -08:00
Haohui Mai 744db47872 HDFS-7736. Fix typos in dfsadmin/fsck/snapshotDiff usage messages. Contributed by Brahma Reddy Battula. 2015-02-11 11:24:44 -08:00
Tsz-Wo Nicholas Sze b49eeddbaf HDFS-7751. Fix TestHDFSCLI for quota with storage type. Contributed by Xiaoyu Yao 2015-02-11 11:10:05 -08:00
Arpit Agarwal 8fa0fc9574 HDFS-7720. Quota by Storage Type API, tools and ClientNameNode Protocol changes. (Contributed by Xiaoyu Yao) 2015-02-11 11:07:53 -08:00
Konstantin V Shvachko 04789db74c HDFS-7760. Document truncate for WebHDFS. Contributed by Konstantin Shvachko. 2015-02-11 10:05:53 -08:00
Konstantin V Shvachko 7f18176cd7 HDFS-7753. Fix Multithreaded correctness Warnings in BackupImage. Contributed by Rakesh R and Konstantin Shvachko. 2015-02-11 00:52:12 -08:00
Tsz-Wo Nicholas Sze acc172e371 HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir. 2015-02-10 17:50:00 -08:00
Konstantin V Shvachko 2b722b904e HDFS-7058. Tests for truncate CLI. Contributed by Dasha Boudnik. 2015-02-09 21:31:02 -08:00
Vinayakumar B a1bf7aecf7 HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode.(Contributed by Vinayakumar B) 
(cherry picked from commit 3d15728ff5)
 2015-02-10 10:45:14 +05:30
Colin Patrick Mccabe 643a8892d1 HDFS-7718. Store KeyProvider in ClientContext to avoid leaking key provider threads when using FileContext (Arun Suresh via Colin P. McCabe) 
(cherry picked from commit 02340a24f2)
 2015-02-09 20:25:31 -08:00
Colin Patrick Mccabe 1465393d14 HDFS-7744. Fix potential NPE in DFSInputStream after setDropBehind or setReadahead is called (cmccabe) 
(cherry picked from commit a9dc5cd706)
 2015-02-09 20:19:48 -08:00
yliu cc0668ebe6 HDFS-7756. Restore method signature for LocatedBlock#getLocations(). (Ted Yu via yliu) 2015-02-10 02:00:32 +08:00
yliu ae316705bb HADOOP-11510. Expose truncate API via FileContext. (yliu) 2015-02-10 01:43:08 +08:00
Arpit Agarwal ff900eb64a HDFS-7647. DatanodeManager.sortLocatedBlocks sorts DatanodeInfos but not StorageIDs. (Contributed by Milan Desai) 2015-02-09 12:17:53 -08:00
Jing Zhao 1e49b40f1f HDFS-7743. Code cleanup of BlockInfo and rename BlockInfo to BlockInfoContiguous. Contributed by Jing Zhao. 
(cherry picked from commit 1382ae525c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockCollection.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
 2015-02-08 12:16:09 -08:00
Tsz-Wo Nicholas Sze c5f18ba65b HDFS-7738. Revise the exception message for recover lease; add more truncate tests such as truncate with HA setup, negative tests, truncate with other operations and multiple truncates. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHAAppend.java
 2015-02-07 15:26:06 -08:00
Akira Ajisaka 964a3b0c3e HDFS-7710. Remove dead code in BackupImage.java. Contributed by Xiaoyu Yao. 
(cherry picked from commit cfb829ecd5)
 2015-02-06 19:00:58 -08:00
Akira Ajisaka 2ec796ba80 HDFS-7732. Fix the order of the parameters in DFSConfigKeys. Contributed by Brahma Reddy Battula. 
(cherry picked from commit eaab959a3e)
 2015-02-06 11:28:18 -08:00
yliu e2803f5a51 HDFS-7741. Remove unnecessary synchronized in FSDataInputStream and HdfsDataInputStream. (yliu) 2015-02-06 06:29:52 +08:00
yliu 44d800b353 HDFS-7698. Fix locking on HDFS read statistics and add a method for clearing them. (Colin P. McCabe via yliu) 2015-02-05 23:57:36 +08:00
yliu 3ffe5a1ed2 HDFS-7655. Expose truncate API for Web HDFS. (yliu) 2015-02-05 23:48:55 +08:00
Haohui Mai 67002a5fc0 HDFS-7270. Add congestion signaling capability to DataNode write protocol. Contributed by Haohui Mai. 2015-02-05 11:03:43 -08:00
Tsuyoshi Ozawa 970fdc3ad9 HDFS-7709. Fix findbug warnings in httpfs. Contributed by Rakesh R. 
(cherry picked from commit 20660b7a67)
 2015-02-05 19:57:40 +09:00
Arpit Agarwal 5807afed0a HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. (Contributed by Arpit Agarwal) 2015-02-04 16:25:40 -08:00