Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,122 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3048 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli 2e99675c72 Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5db)
 2015-06-28 16:33:49 -07:00
Arpit Agarwal f6550361f7 HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal) 2015-06-28 14:58:07 -07:00
Arpit Agarwal 4a8db40f3b HDFS-8626. Add test case for branch-2.7 2015-06-23 10:03:44 -07:00
Colin Patrick Mccabe 86c01d90ba HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 7b424f938c)
(cherry picked from commit cbd11681ce)

Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
 2015-06-22 14:46:57 -07:00
Arpit Agarwal 25b99e4816 HDFS-7164. Feature documentation for HDFS-6581. (Contributed by Arpit Agarwal) 2015-06-20 13:08:18 -07:00
Allen Wittenauer 0adb6d8292 HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) 
(cherry picked from commit 63613c79c1)
 2015-06-20 07:45:39 -07:00
Arpit Agarwal dc4171cb66 HDFS-8626. Reserved RBW space is not released if creation of RBW File fails. (Contributed by kanaka kumar avvaru) 2015-06-19 18:30:53 -07:00
Yongjun Zhang 8c03a079df HDFS-8633. Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys. Contributed by Ray Chiang. 
(cherry picked from commit b42f1ec13d)
 2015-06-19 09:51:48 -07:00
Kihwal Lee 6de4d52c31 HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 
(cherry picked from commit c74517c46b)
 2015-06-16 15:44:38 -05:00
Xiaoyu Yao fe35ba1df9 HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4e88ff5b27)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-16 12:51:34 -07:00
Yongjun Zhang f4ae56d605 HDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit b1d3ed02c1)
 2015-06-15 21:51:21 -07:00
Tsz-Wo Nicholas Sze 632cb542f6 HDFS-8361. Choose SSD over DISK in block placement. 2015-06-15 17:29:33 -07:00
Tsz-Wo Nicholas Sze 7a248f2e9e HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina 2015-06-15 16:13:56 -07:00
Arpit Agarwal a1810d75f0 HDFS-8600. TestWebHdfsFileSystemContract.testGetFileBlockLocations fails in branch-2.7. (Contributed by Arpit Agarwal) 2015-06-14 20:18:31 -07:00
Arpit Agarwal 1145d09e0a HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal) 2015-06-14 16:24:42 -07:00
Yongjun Zhang 6dda57a817 HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang. 
(cherry picked from commit b0dc291961)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java

(cherry picked from commit cd578fa289)
 2015-06-14 12:04:16 -07:00
Haohui Mai 3651feb49d HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai. 2015-06-11 19:14:48 -07:00
Arpit Agarwal 3ec9b63586 HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal) 2015-06-11 15:08:15 -07:00
Colin Patrick Mccabe 0bcc68337b HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit 587309bbba)
(cherry picked from commit 754dd3496c)
 2015-06-11 14:30:21 -07:00
Arpit Agarwal 105cadb421 HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:55 -07:00
Xiaoyu Yao 8c1a46ca4f HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:13:40 -07:00
Vinayakumar B dd98cfd328 HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 
(cherry picked from commit 0ed9c2d8fe)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
 2015-06-03 15:18:42 +05:30
Vinayakumar B 8090a6ee63 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java
 2015-06-03 12:20:54 +05:30
Colin Patrick Mccabe a7058caa72 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8)
(cherry picked from commit 12497b0543)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2015-06-02 20:07:35 -07:00
Colin Patrick Mccabe 323ffccfe9 HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c6425)
(cherry picked from commit 17e369511d)
 2015-06-02 11:46:12 -07:00
Kihwal Lee 1ff3f16ed0 HDFS-5215. dfs.datanode.du.reserved is not considered while computing 
available space ( Brahma Reddy Battula via Yongjun Zhang)
(cherry picked from commit 66763bb06f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-02 08:14:57 -05:00
Xiaoyu Yao c550825673 HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran. 
(cherry picked from commit 05e04f34f2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
 2015-05-21 12:49:49 -07:00
Kihwal Lee 7ac66f8712 HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts. 
(cherry picked from commit 8860e352c3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2015-05-19 13:17:45 -05:00
Tsz-Wo Nicholas Sze 59d1b4a323 HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma 2015-05-19 03:17:51 +08:00
cnauroth 82530ba0f1 HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth. 
(cherry picked from commit ddc4afeff2)
 2015-05-13 12:26:02 -07:00
Tsz-Wo Nicholas Sze 2113e0a3f2 HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore 2015-05-13 11:59:01 -07:00
Vinayakumar B ef2488e7f0 HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R) 
(cherry picked from commit 065d8f2a34)
 2015-05-13 17:30:03 +05:30
Kihwal Lee 1e4a686ea9 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah. 
(cherry picked from commit ea11590aad)
 2015-05-11 17:11:51 -05:00
Kihwal Lee f57b1bfbdd HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah. 
(cherry picked from commit 2d4ae3d18b)
 2015-05-08 16:38:43 -05:00
Kihwal Lee bbb9f6aafe HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6f622672b6)
 2015-05-08 09:32:44 -05:00
Vinayakumar B 1fbefe586b HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
Added one missed change, missed while resolving conflict

(cherry picked from commit 3ee1b50dad)
 2015-05-08 15:42:17 +05:30
Vinayakumar B 99309a9fec HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
(cherry picked from commit e01c1a979c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md

(cherry picked from commit 3a894b4563)
 2015-05-08 15:42:02 +05:30
Tsz-Wo Nicholas Sze 74bd9666d1 HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 2015-05-07 11:38:15 -07:00
Colin Patrick Mccabe e056e0a3c1 HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe) 
(cherry picked from commit fcd4cb7516)
(cherry picked from commit 1f01d8347a)
 2015-05-05 11:07:13 -07:00
Vinayakumar B 69d837428c HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Reverted earlier commit

(cherry picked from commit 318081ccd7)
 2015-05-05 11:07:59 +05:30
Colin Patrick Mccabe 472fd563e4 HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe) 
(cherry picked from commit b82567d455)
(cherry picked from commit 9edea9507d)
 2015-05-01 11:52:08 -07:00
Arun Suresh 4e6ff8c78b HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh) 
(cherry picked from commit 922b7ed21d)
 2015-05-01 11:42:29 -07:00
Akira Ajisaka 185a1ff275 HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao. 
(cherry picked from commit de9404f02f)
(cherry picked from commit fdb886d656)
 2015-04-30 20:13:47 +09:00
Haohui Mai 6ac2b5712b HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. 2015-04-29 11:13:03 -07:00
Haohui Mai 4b9147ef8c HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai. 2015-04-28 18:06:22 -07:00
Tsz-Wo Nicholas Sze 6b20b325f4 HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su 2015-04-28 13:07:20 -07:00
Colin Patrick Mccabe 176d22b440 HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) 
(cherry picked from commit a8898445dc)
(cherry picked from commit 3d0385c3c8)
 2015-04-23 19:28:38 -07:00
Colin Patrick Mccabe c2e6d2589a HADOOP-11802. Fix renamed constant in test file in backport to 2.7.1 (cmccabe) 2015-04-23 19:16:28 -07:00
Colin Patrick Mccabe 496afb5e1a HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe) 
(cherry picked from commit a0e0a63209)
(cherry picked from commit 788b76761d)
 2015-04-23 19:09:50 -07:00
Tsz-Wo Nicholas Sze 5b4eda7573 HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore 2015-04-23 11:57:16 -07:00
Arpit Agarwal b17d365fa1 HDFS-8163. Using monotonicNow for block report scheduling causes test failures on recently restarted systems. (Arpit Agarwal) 2015-04-21 11:17:27 -07:00
Vinod Kumar Vavilapalli 7dd5f42a76 Set the release date for 2.7.0 
(cherry picked from commit d52de61544)
 2015-04-20 20:18:38 -07:00
Vinayakumar B 1c8d54bc04 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
(cherry picked from commit 867d5d2675)
 2015-04-21 07:45:59 +05:30
Arpit Agarwal cc42ad03d9 HDFS-8179. DFSClient#getServerDefaults returns null within 1 hour of system start. (Contributed by Xiaoyu Yao) 2015-04-20 15:42:58 -07:00
Jitendra Pandey 0743f128a0 HDFS-8153. Error Message points to wrong parent directory in case of path component name length error. Contributed by Anu Engineer. 
cherry picked from 369ddc67bd
 2015-04-16 22:23:35 -07:00
Akira Ajisaka cffc801ecb Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0feb.
 2015-04-16 16:46:19 -05:00
Akira Ajisaka 35e1936f84 HDFS-8149. The footer of the Web UI "Hadoop, 2014" is old. Contributed by Brahma Reddy Battula. 
(cherry picked from commit de0f1700c1)
(cherry picked from commit 6d183561ff)
 2015-04-15 15:32:29 -05:00
Jing Zhao e1ce0700eb HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina. 
(cherry picked from commit b172d03595)
(cherry picked from commit 725621d95a)
 2015-04-15 13:11:45 -07:00
Jing Zhao 9e11ae684a HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao. 
(cherry picked from commit 4c097e473b)
(cherry picked from commit d4dd97eabd)
 2015-04-15 12:41:34 -07:00
Jing Zhao 50c8d06314 HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao. 
(cherry picked from commit fddd55279d)
(cherry picked from commit 38b031d6ba)
 2015-04-14 15:15:36 -07:00
Arun Suresh 0a95b4f8aa HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh) 
(cherry picked from commit 7660da95cb)
 2015-04-10 11:43:49 -07:00
Konstantin V Shvachko e964bbc856 HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko 2015-04-09 22:04:48 -07:00
Vinod Kumar Vavilapalli a7fa46c2e0 Preparing for 2.7.1 development: mvn versions:set 2015-04-09 14:13:30 -07:00
Vinod Kumar Vavilapalli 2b44fcaa53 Adding release 2.7.1 to CHANGES.txt 
(cherry picked from commit 623fd46c1e)
 2015-04-09 14:01:23 -07:00
Colin Patrick Mccabe 4d5b1fbdec HDFS-8063: Fix intermittent test failures in TestTracing (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 61dc2ea3fe)
(cherry picked from commit 8dac245920)
 2015-04-09 11:42:29 -07:00
Kihwal Lee 9a111fcd1d HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma. 
(cherry picked from commit 5a540c3d31)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-09 10:01:44 -05:00
Haohui Mai 84ba4db2a0 Revert HDFS-7813. 2015-04-08 16:03:35 -07:00
Haohui Mai eb53d6b967 Revert HDFS-7808. 2015-04-08 16:00:26 -07:00
Arpit Agarwal 12739b541b HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal) 2015-04-08 11:38:42 -07:00
cnauroth 5021f52f7b HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao. 
(cherry picked from commit 672ed462965ea7da10ce3df48c02e2a3bd13f0ae)
(cherry picked from commit 950dc1b1995191327e6b919ee17a87e59b749264)
 2015-04-07 13:33:38 -07:00
Konstantin V Shvachko 2cb9dac9a3 HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko. 2015-04-06 23:35:01 -07:00
Colin Patrick Mccabe 3f1d1d0b48 HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe) 
(cherry picked from commit 28bebc81db)
(cherry picked from commit a827089905)
 2015-04-06 09:10:34 -07:00
Vinod Kumar Vavilapalli 5387a289fc HADOOP-11776. Fixed the broken JDiff support in Hadoop 2. Contributed by Li Lu. 
(cherry picked from commit 4b3948ea36)
 2015-04-04 13:53:34 -07:00
Vinod Kumar Vavilapalli 996b0ca173 HADOOP-11377. Added Null.java without which jdiff completely flops. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit 5370e7128b)
 2015-04-04 13:48:58 -07:00
Colin Patrick Mccabe aa5eab8ec4 HDFS-8051. Fix missing import statement from backport. 2015-04-03 19:07:25 -07:00
Colin Patrick Mccabe 54b88943ff HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit ef591b1d6a)
(cherry picked from commit b26ba22a90)
 2015-04-03 16:57:06 -07:00
Colin Patrick Mccabe 520043b438 HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 023133cef9)
(cherry picked from commit 0c5069c432)
 2015-04-03 14:27:09 -07:00
cnauroth 6855bbc6ca HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit db80e42891)
(cherry picked from commit 28e0602e7b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-03 10:45:15 -07:00
Jing Zhao b08bc43c9b HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao. 
(cherry picked from commit 72f6bd4893)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 2c1f33d178)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-02 21:27:32 -07:00
Tsz-Wo Nicholas Sze acfe44e5b0 HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao 2015-04-02 11:14:04 -07:00
Brandon Li a445f8b675 HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot 
(cherry picked from commit 4d14816c26)
(cherry picked from commit 27970b3f46)
 2015-04-01 17:30:00 -07:00
Haohui Mai 8ebbbc6ebd HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:51:25 -07:00
Haohui Mai 86c0c6b044 HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. 2015-03-30 12:16:25 -07:00
Kihwal Lee a0ed29a058 HDFS-7742. Favoring decommissioning node for replication can cause a block to stay 
underreplicated for long periods. Contributed by Nathan Roberts.
(cherry picked from commit 04ee18ed48)
 2015-03-30 10:11:47 -05:00
Haohui Mai d481a8d79b HADOOP-11748. The secrets of auth cookies should not be specified in configuration in clear text. Contributed by Li Lu and Haohui Mai. 2015-03-26 17:35:54 -07:00
Colin Patrick Mccabe 9411cf9b65 HDFS-7410. Support CreateFlags with append() to support hsync() for appending streams (Vinayakumar B via Colin P. McCabe) 
(cherry picked from commit 61df1b27a7)
(cherry picked from commit 2a750c9aa6)
 2015-03-26 13:29:17 -07:00
Arpit Agarwal c808d5fe55 HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao) 2015-03-26 10:48:03 -07:00
Kihwal Lee 370c91461f HDFS-7963. Fix expected tracing spans in TestTracing. Contributed by Masatake Iwasaki. 2015-03-26 08:45:39 -05:00
Jitendra Pandey d286673c60 HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh. 
(cherry picked from commit 456cec127b)
 2015-03-24 16:17:06 -07:00
Brandon Li 14d1cccfad HDFS-7977. NFS couldn't take percentile intervals. Contributed by Brandon Li 
(cherry picked from commit 570a83ae80)
(cherry picked from commit 260dbe96c3)
 2015-03-24 10:53:20 -07:00
Brandon Li cc5922607a HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li 
(cherry picked from commit 51f1f49374)
(cherry picked from commit 95bde88986)
 2015-03-24 10:30:19 -07:00
Andrew Wang a7f447d460 HDFS-7961. Trigger full block report after hot swapping disk. Contributed by Eddy Xu. 
(cherry picked from commit 6413d34986)
 2015-03-24 09:07:27 -07:00
Plamen Jeliazkov da318cbbde HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov. 2015-03-23 23:09:52 -07:00
Andrew Wang af0af28afc HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu. 
(cherry picked from commit 50ee8f4e67)
 2015-03-23 22:01:37 -07:00
Tsz-Wo Nicholas Sze 87079cde7d HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula 2015-03-24 13:52:18 +09:00
cnauroth 8e1c33e703 HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 2c238ae4e0)
(cherry picked from commit 01c0bcb176)
 2015-03-23 16:30:48 -07:00
Akira Ajisaka 2742f12b58 HDFS-7881. TestHftpFileSystem#testSeek fails in branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit fad8c78173)
 2015-03-24 06:25:21 +09:00
Brandon Li 6b9f2d9f39 HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts. Contributed by Brandon Li 
(cherry picked from commit 36af4a913c)
(cherry picked from commit 503d8e4164)
 2015-03-23 10:13:11 -07:00
Kihwal Lee 08f2f7ed34 HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B 
(cherry picked from commit 99a8dcd19528b265d4fda9ae09a17e4af52f2782)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
(cherry picked from commit 63b7870aec007081c82427587cb6e1d38e2c70f8)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-20 14:06:02 -05:00
Jing Zhao 03fb9b4ce0 HDFS-7957. Truncate should verify quota before making changes. Contributed by Jing Zhao. 
(cherry picked from commit d368d3647a)
(cherry picked from commit 6be52e42a9)
 2015-03-20 11:56:39 -07:00
yliu faeb8b9b53 HDFS-7962. Remove duplicated logs in BlockManager. (yliu) 2015-03-20 05:32:19 +08:00
yliu 0da3fcb4b0 HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu) 2015-03-19 23:26:26 +08:00
Kihwal Lee 4e967d0bcd HDFS-7932. Speed up the shutdown of datanode during rolling upgrade. Contributed by Kihwal Lee. 
(cherry picked from commit 61a4c7fc98)
 2015-03-19 12:29:16 -05:00
Vinayakumar B 76ce2e8ea0 HDFS.7849. Update documentation for enabling a new feature in rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 5be5129d93)
 2015-03-19 19:10:23 +05:30
Kihwal Lee b8f269af9d HDFS-7816. Unable to open webhdfs paths with "+". Contributed by Haohui Mai 
(cherry picked from commit e79be0ee12)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit ceb39c1cc6)
 2015-03-19 08:04:19 -05:00
Colin Patrick Mccabe 9e61835678 HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 43b41f2241)
(cherry picked from commit 219eb22c15)
 2015-03-18 19:08:53 -07:00
Jing Zhao 0bf549728d HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao. 
(cherry picked from commit c7c71cdba5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 5a5b244648)
 2015-03-18 19:00:06 -07:00
Jing Zhao 4b8de154aa HDFS-7943. Append cannot handle the last block with length greater than the preferred block size. Contributed by Jing Zhao. 
(cherry picked from commit bee5a6a64a)
(cherry picked from commit 6dcc79507d)
 2015-03-18 18:47:21 -07:00
Colin Patrick Mccabe 94976cb369 HDFS-7054. Make DFSOutputStream tracing more fine-grained (cmccabe) 
(cherry picked from commit 8234fd0e10)
(cherry picked from commit 79c07bbaca)
 2015-03-18 18:15:34 -07:00
Haohui Mai 388696c089 HDFS-7945. The WebHdfs system on DN does not honor the length parameter. Contributed by Haohui Mai. 2015-03-18 16:20:07 -07:00
Arpit Agarwal bb986988d4 HDFS-7914. TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use. (Contributed by Xiaoyu Yao) 2015-03-18 14:05:37 -07:00
Haohui Mai 482aeee784 HDFS-7697. Mark the PB OIV tool as experimental. Contributed by Lei (Eddy) Xu. 2015-03-18 13:34:21 -07:00
Arpit Agarwal 5f7926dcf0 Fix CHANGES.txt for HDFS-7722. 2015-03-18 12:53:02 -07:00
Arpit Agarwal 2b589da779 HDFS-7951. Fix NPE for TestFsDatasetImpl#testAddVolumeFailureReleasesInUseLock on Linux. (Contributed by Xiaoyu Yao) 2015-03-18 12:46:17 -07:00
Arpit Agarwal ae6d273d63 HDFS-7950. Fix TestFsDatasetImpl#testAddVolumes failure on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:35:41 -07:00
Arpit Agarwal 8944bb3d2e HDFS-7948. TestDataNodeHotSwapVolumes#testAddVolumeFailures failed on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:26:22 -07:00
Haohui Mai 5293ae0311 HDFS-7953. NN Web UI fails to navigate to paths that contain #. Contributed by kanaka kumar avvaru. 2015-03-18 11:08:49 -07:00
Arpit Agarwal 26c35438f3 HDFS-7946. TestDataNodeVolumeFailureReporting NPE on Windows. (Contributed by Xiaoyu Yao) 2015-03-17 21:30:13 -07:00
Colin Patrick Mccabe d2dad74421 HDFS-7940. Add tracing to DFSClient#setQuotaByStorageType (Rakesh R via Colin P. McCabe) 
(cherry picked from commit d8846707c5)
(cherry picked from commit 455d4aa8a1)
 2015-03-17 11:02:05 -07:00
yliu ef9d46dcb6 HDFS-7838. Expose truncate API for libhdfs. (yliu) 2015-03-17 07:25:58 +08:00
Colin Patrick Mccabe 861cc05092 HDFS-7722. DataNode#checkDiskError should also remove Storage when error is found. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit b49c3a1813)
(cherry picked from commit 7455412a24)
 2015-03-16 18:42:21 -07:00
Colin Patrick Mccabe 647d9ae27a HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe) 
(cherry picked from commit bc9cb3e271)
(cherry picked from commit c71b54fa32)
 2015-03-16 18:39:58 -07:00
Konstantin V Shvachko ad4d704fbb HDFS-7886. Fix TestFileTruncate falures. Contributed by Plamen Jeliazkov and Konstantin Shvachko. 2015-03-16 12:43:46 -07:00
Konstantin V Shvachko 41530b4173 HDFS-2605. Remove redundant "Release 0.21.1" section from CHANGES.txt. Contributed by Allen Wittenauer. 2015-03-13 13:45:47 -07:00
Konstantin V Shvachko f6bdcd938e HDFS-7903. Cannot recover block after truncate and delete snapshot. Contributed by Plamen Jeliazkov. 2015-03-13 12:45:47 -07:00
Kihwal Lee 116a7f1a16 HDFS-7435. PB encoding of block reports is very inefficient. Contributed by Daryn Sharp. 
(cherry picked from commit d324164a51)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
(cherry picked from commit 464271a5ed)
 2015-03-13 14:42:02 -05:00
Brandon Li cdeb1079ea HDFS-7926. NameNode implementation of ClientProtocol.truncate(..) is not idempotent. Contributed by Tsz Wo Nicholas Sze 
(cherry picked from commit f446669afb)
(cherry picked from commit af80a98ace)
 2015-03-13 10:47:25 -07:00
Tsz-Wo Nicholas Sze a1fffc3fcc HDFS-6833. DirectoryScanner should not register a deleting block with memory of DataNode. Contributed by Shinichi Yamashita 2015-03-13 02:29:24 +08:00
Akira Ajisaka 718cd0f1f4 HDFS-7880. Remove the tests for legacy Web UI in branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6c80a3d34a)
 2015-03-12 10:55:47 +09:00
Colin Patrick Mccabe 8241228db7 HDFS-7830. DataNode does not release the volume lock when adding a volume fails. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 5c1036d598)
(cherry picked from commit eefca23e8c)
 2015-03-12 10:49:48 +09:00
Haohui Mai 245f7b2a77 HADOOP-11633. Convert remaining branch-2 .apt.vm files to markdown. Contributed by Masatake Iwasaki. 2015-03-11 14:31:16 -07:00
cnauroth 0fe5f5b5b8 HDFS-7491. Add incremental blockreport latency to DN metrics. Contributed by Ming Ma. 
(cherry picked from commit fb34f45727)
(cherry picked from commit 5e99b7cd2f)
 2015-03-11 14:12:06 -07:00
Jing Zhao 368ab2cd37 HDFS-7898. Change TestAppendSnapshotTruncate to fail-fast. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit e43882e84a)
(cherry picked from commit c7105fcff0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-10 10:31:41 -07:00
Akira Ajisaka c9a2e6f5db HDFS-6806. HDFS Rolling upgrade document should mention the versions available. Contributed by J.Andreina. 
(cherry picked from commit 82db3341bf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-10 11:27:30 +09:00
Tsuyoshi Ozawa 28f4e6b222 HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 
(cherry picked from commit b46f9e72db)
 2015-03-09 19:54:45 +09:00
Chris Douglas 3bc4f3502a HDFS-7411. Change decommission logic to throttle by blocks rather 
than nodes in each interval. Contributed by Andrew Wang
 2015-03-08 18:32:20 -07:00
Brandon Li c8af3b2768 HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li 
(cherry picked from commit 0f8ecb1d0c)
 2015-03-06 15:22:20 -08:00
Haohui Mai ab251fd355 HDFS-7818. OffsetParam should return the default value instead of throwing NPE when the value is unspecified. Contributed by Eric Payne. 2015-03-06 14:26:47 -08:00
Jing Zhao 994dadb9ba HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 24db0812be)
 2015-03-06 10:57:38 -08:00
Jing Zhao c5d52c7b52 HDFS-7855. Separate class Packet from DFSOutputStream. Contributed by Li Bo. 
(cherry picked from commit 952640fa4c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-05 11:11:08 -08:00
Tsuyoshi Ozawa f805d48b19 HADOOP-11648. Set DomainSocketWatcher thread name explicitly. Contributed by Liang Xie. 
(cherry picked from commit 74a4754d1c)
 2015-03-05 16:06:37 +09:00
Tsz-Wo Nicholas Sze f9a2007aff HDFS-7746. Add a test randomly mixing append, truncate and snapshot operations. 2015-03-05 10:23:14 +08:00
Dongming Liang ada545d3b7 HDFS-1522. Combine two BLOCK_FILE_PREFIX constants into one. Contributed by Dongming Liang. 2015-03-04 17:52:25 -08:00
Kihwal Lee f85530f649 HDFS-7434. DatanodeID hashCode should not be mutable. Contributed by Daryn Sharp. 
(cherry picked from commit 722b479469)
 2015-03-04 17:23:00 -06:00
Jing Zhao b2ccf54c14 HDFS-7535. Utilize Snapshot diff report for distcp. Contributed by Jing Zhao. 
(cherry picked from commit ed70fa142c)
 2015-03-04 10:32:39 -08:00
Haohui Mai d020544ef6 HDFS-7879. hdfs.dll does not export functions of the public libhdfs API. Contributed by Chris Nauroth. 2015-03-04 09:17:29 -08:00
Vinayakumar B c24448c27b HDFS-7869. Inconsistency in the return information while performing rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 3560180b6e)
 2015-03-04 14:39:51 +05:30
Akira Ajisaka 31b3f84601 HADOOP-11666. Revert the format change of du output introduced by HADOOP-6857. Contributed by Byron Wong. 2015-03-03 21:46:28 -08:00
Aaron T. Myers bce3d442ff HDFS-7682. {{DistributedFileSystem#getFileChecksum}} of a snapshotted file includes non-snapshotted content. Contributed by Charles Lamb. 
(cherry picked from commit f2d7a67a2c)
 2015-03-03 18:09:31 -08:00
Haohui Mai 65bfde552a HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. 2015-03-03 17:58:21 -08:00