Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,053 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2995 Commits

Author SHA1 Message Date
Vinayakumar B ef2488e7f0 HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R) 
(cherry picked from commit 065d8f2a34)
 2015-05-13 17:30:03 +05:30
Kihwal Lee 1e4a686ea9 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah. 
(cherry picked from commit ea11590aad)
 2015-05-11 17:11:51 -05:00
Kihwal Lee f57b1bfbdd HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah. 
(cherry picked from commit 2d4ae3d18b)
 2015-05-08 16:38:43 -05:00
Kihwal Lee bbb9f6aafe HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6f622672b6)
 2015-05-08 09:32:44 -05:00
Vinayakumar B 1fbefe586b HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
Added one missed change, missed while resolving conflict

(cherry picked from commit 3ee1b50dad)
 2015-05-08 15:42:17 +05:30
Vinayakumar B 99309a9fec HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
(cherry picked from commit e01c1a979c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md

(cherry picked from commit 3a894b4563)
 2015-05-08 15:42:02 +05:30
Tsz-Wo Nicholas Sze 74bd9666d1 HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 2015-05-07 11:38:15 -07:00
Colin Patrick Mccabe e056e0a3c1 HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe) 
(cherry picked from commit fcd4cb7516)
(cherry picked from commit 1f01d8347a)
 2015-05-05 11:07:13 -07:00
Vinayakumar B 69d837428c HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Reverted earlier commit

(cherry picked from commit 318081ccd7)
 2015-05-05 11:07:59 +05:30
Colin Patrick Mccabe 472fd563e4 HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe) 
(cherry picked from commit b82567d455)
(cherry picked from commit 9edea9507d)
 2015-05-01 11:52:08 -07:00
Arun Suresh 4e6ff8c78b HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh) 
(cherry picked from commit 922b7ed21d)
 2015-05-01 11:42:29 -07:00
Akira Ajisaka 185a1ff275 HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao. 
(cherry picked from commit de9404f02f)
(cherry picked from commit fdb886d656)
 2015-04-30 20:13:47 +09:00
Haohui Mai 6ac2b5712b HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. 2015-04-29 11:13:03 -07:00
Haohui Mai 4b9147ef8c HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai. 2015-04-28 18:06:22 -07:00
Tsz-Wo Nicholas Sze 6b20b325f4 HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su 2015-04-28 13:07:20 -07:00
Colin Patrick Mccabe 176d22b440 HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) 
(cherry picked from commit a8898445dc)
(cherry picked from commit 3d0385c3c8)
 2015-04-23 19:28:38 -07:00
Colin Patrick Mccabe c2e6d2589a HADOOP-11802. Fix renamed constant in test file in backport to 2.7.1 (cmccabe) 2015-04-23 19:16:28 -07:00
Colin Patrick Mccabe 496afb5e1a HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe) 
(cherry picked from commit a0e0a63209)
(cherry picked from commit 788b76761d)
 2015-04-23 19:09:50 -07:00
Tsz-Wo Nicholas Sze 5b4eda7573 HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore 2015-04-23 11:57:16 -07:00
Arpit Agarwal b17d365fa1 HDFS-8163. Using monotonicNow for block report scheduling causes test failures on recently restarted systems. (Arpit Agarwal) 2015-04-21 11:17:27 -07:00
Vinod Kumar Vavilapalli 7dd5f42a76 Set the release date for 2.7.0 
(cherry picked from commit d52de61544)
 2015-04-20 20:18:38 -07:00
Vinayakumar B 1c8d54bc04 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
(cherry picked from commit 867d5d2675)
 2015-04-21 07:45:59 +05:30
Arpit Agarwal cc42ad03d9 HDFS-8179. DFSClient#getServerDefaults returns null within 1 hour of system start. (Contributed by Xiaoyu Yao) 2015-04-20 15:42:58 -07:00
Jitendra Pandey 0743f128a0 HDFS-8153. Error Message points to wrong parent directory in case of path component name length error. Contributed by Anu Engineer. 
cherry picked from 369ddc67bd
 2015-04-16 22:23:35 -07:00
Akira Ajisaka cffc801ecb Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0feb.
 2015-04-16 16:46:19 -05:00
Akira Ajisaka 35e1936f84 HDFS-8149. The footer of the Web UI "Hadoop, 2014" is old. Contributed by Brahma Reddy Battula. 
(cherry picked from commit de0f1700c1)
(cherry picked from commit 6d183561ff)
 2015-04-15 15:32:29 -05:00
Jing Zhao e1ce0700eb HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina. 
(cherry picked from commit b172d03595)
(cherry picked from commit 725621d95a)
 2015-04-15 13:11:45 -07:00
Jing Zhao 9e11ae684a HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao. 
(cherry picked from commit 4c097e473b)
(cherry picked from commit d4dd97eabd)
 2015-04-15 12:41:34 -07:00
Jing Zhao 50c8d06314 HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao. 
(cherry picked from commit fddd55279d)
(cherry picked from commit 38b031d6ba)
 2015-04-14 15:15:36 -07:00
Arun Suresh 0a95b4f8aa HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh) 
(cherry picked from commit 7660da95cb)
 2015-04-10 11:43:49 -07:00
Konstantin V Shvachko e964bbc856 HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko 2015-04-09 22:04:48 -07:00
Vinod Kumar Vavilapalli a7fa46c2e0 Preparing for 2.7.1 development: mvn versions:set 2015-04-09 14:13:30 -07:00
Vinod Kumar Vavilapalli 2b44fcaa53 Adding release 2.7.1 to CHANGES.txt 
(cherry picked from commit 623fd46c1e)
 2015-04-09 14:01:23 -07:00
Colin Patrick Mccabe 4d5b1fbdec HDFS-8063: Fix intermittent test failures in TestTracing (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 61dc2ea3fe)
(cherry picked from commit 8dac245920)
 2015-04-09 11:42:29 -07:00
Kihwal Lee 9a111fcd1d HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma. 
(cherry picked from commit 5a540c3d31)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-09 10:01:44 -05:00
Haohui Mai 84ba4db2a0 Revert HDFS-7813. 2015-04-08 16:03:35 -07:00
Haohui Mai eb53d6b967 Revert HDFS-7808. 2015-04-08 16:00:26 -07:00
Arpit Agarwal 12739b541b HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal) 2015-04-08 11:38:42 -07:00
cnauroth 5021f52f7b HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao. 
(cherry picked from commit 672ed462965ea7da10ce3df48c02e2a3bd13f0ae)
(cherry picked from commit 950dc1b1995191327e6b919ee17a87e59b749264)
 2015-04-07 13:33:38 -07:00
Konstantin V Shvachko 2cb9dac9a3 HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko. 2015-04-06 23:35:01 -07:00
Colin Patrick Mccabe 3f1d1d0b48 HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe) 
(cherry picked from commit 28bebc81db)
(cherry picked from commit a827089905)
 2015-04-06 09:10:34 -07:00
Vinod Kumar Vavilapalli 5387a289fc HADOOP-11776. Fixed the broken JDiff support in Hadoop 2. Contributed by Li Lu. 
(cherry picked from commit 4b3948ea36)
 2015-04-04 13:53:34 -07:00
Vinod Kumar Vavilapalli 996b0ca173 HADOOP-11377. Added Null.java without which jdiff completely flops. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit 5370e7128b)
 2015-04-04 13:48:58 -07:00
Colin Patrick Mccabe aa5eab8ec4 HDFS-8051. Fix missing import statement from backport. 2015-04-03 19:07:25 -07:00
Colin Patrick Mccabe 54b88943ff HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit ef591b1d6a)
(cherry picked from commit b26ba22a90)
 2015-04-03 16:57:06 -07:00
Colin Patrick Mccabe 520043b438 HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 023133cef9)
(cherry picked from commit 0c5069c432)
 2015-04-03 14:27:09 -07:00
cnauroth 6855bbc6ca HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit db80e42891)
(cherry picked from commit 28e0602e7b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-03 10:45:15 -07:00
Jing Zhao b08bc43c9b HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao. 
(cherry picked from commit 72f6bd4893)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 2c1f33d178)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-02 21:27:32 -07:00
Tsz-Wo Nicholas Sze acfe44e5b0 HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao 2015-04-02 11:14:04 -07:00
Brandon Li a445f8b675 HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot 
(cherry picked from commit 4d14816c26)
(cherry picked from commit 27970b3f46)
 2015-04-01 17:30:00 -07:00
Haohui Mai 8ebbbc6ebd HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:51:25 -07:00
Haohui Mai 86c0c6b044 HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. 2015-03-30 12:16:25 -07:00
Kihwal Lee a0ed29a058 HDFS-7742. Favoring decommissioning node for replication can cause a block to stay 
underreplicated for long periods. Contributed by Nathan Roberts.
(cherry picked from commit 04ee18ed48)
 2015-03-30 10:11:47 -05:00
Colin Patrick Mccabe 9411cf9b65 HDFS-7410. Support CreateFlags with append() to support hsync() for appending streams (Vinayakumar B via Colin P. McCabe) 
(cherry picked from commit 61df1b27a7)
(cherry picked from commit 2a750c9aa6)
 2015-03-26 13:29:17 -07:00
Arpit Agarwal c808d5fe55 HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao) 2015-03-26 10:48:03 -07:00
Kihwal Lee 370c91461f HDFS-7963. Fix expected tracing spans in TestTracing. Contributed by Masatake Iwasaki. 2015-03-26 08:45:39 -05:00
Jitendra Pandey d286673c60 HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh. 
(cherry picked from commit 456cec127b)
 2015-03-24 16:17:06 -07:00
Brandon Li 14d1cccfad HDFS-7977. NFS couldn't take percentile intervals. Contributed by Brandon Li 
(cherry picked from commit 570a83ae80)
(cherry picked from commit 260dbe96c3)
 2015-03-24 10:53:20 -07:00
Brandon Li cc5922607a HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li 
(cherry picked from commit 51f1f49374)
(cherry picked from commit 95bde88986)
 2015-03-24 10:30:19 -07:00
Andrew Wang a7f447d460 HDFS-7961. Trigger full block report after hot swapping disk. Contributed by Eddy Xu. 
(cherry picked from commit 6413d34986)
 2015-03-24 09:07:27 -07:00
Plamen Jeliazkov da318cbbde HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov. 2015-03-23 23:09:52 -07:00
Andrew Wang af0af28afc HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu. 
(cherry picked from commit 50ee8f4e67)
 2015-03-23 22:01:37 -07:00
Tsz-Wo Nicholas Sze 87079cde7d HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula 2015-03-24 13:52:18 +09:00
cnauroth 8e1c33e703 HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 2c238ae4e0)
(cherry picked from commit 01c0bcb176)
 2015-03-23 16:30:48 -07:00
Akira Ajisaka 2742f12b58 HDFS-7881. TestHftpFileSystem#testSeek fails in branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit fad8c78173)
 2015-03-24 06:25:21 +09:00
Brandon Li 6b9f2d9f39 HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts. Contributed by Brandon Li 
(cherry picked from commit 36af4a913c)
(cherry picked from commit 503d8e4164)
 2015-03-23 10:13:11 -07:00
Kihwal Lee 08f2f7ed34 HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B 
(cherry picked from commit 99a8dcd19528b265d4fda9ae09a17e4af52f2782)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
(cherry picked from commit 63b7870aec007081c82427587cb6e1d38e2c70f8)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-20 14:06:02 -05:00
Jing Zhao 03fb9b4ce0 HDFS-7957. Truncate should verify quota before making changes. Contributed by Jing Zhao. 
(cherry picked from commit d368d3647a)
(cherry picked from commit 6be52e42a9)
 2015-03-20 11:56:39 -07:00
yliu faeb8b9b53 HDFS-7962. Remove duplicated logs in BlockManager. (yliu) 2015-03-20 05:32:19 +08:00
yliu 0da3fcb4b0 HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu) 2015-03-19 23:26:26 +08:00
Kihwal Lee 4e967d0bcd HDFS-7932. Speed up the shutdown of datanode during rolling upgrade. Contributed by Kihwal Lee. 
(cherry picked from commit 61a4c7fc98)
 2015-03-19 12:29:16 -05:00
Vinayakumar B 76ce2e8ea0 HDFS.7849. Update documentation for enabling a new feature in rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 5be5129d93)
 2015-03-19 19:10:23 +05:30
Kihwal Lee b8f269af9d HDFS-7816. Unable to open webhdfs paths with "+". Contributed by Haohui Mai 
(cherry picked from commit e79be0ee12)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit ceb39c1cc6)
 2015-03-19 08:04:19 -05:00
Colin Patrick Mccabe 9e61835678 HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 43b41f2241)
(cherry picked from commit 219eb22c15)
 2015-03-18 19:08:53 -07:00
Jing Zhao 0bf549728d HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao. 
(cherry picked from commit c7c71cdba5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 5a5b244648)
 2015-03-18 19:00:06 -07:00
Jing Zhao 4b8de154aa HDFS-7943. Append cannot handle the last block with length greater than the preferred block size. Contributed by Jing Zhao. 
(cherry picked from commit bee5a6a64a)
(cherry picked from commit 6dcc79507d)
 2015-03-18 18:47:21 -07:00
Colin Patrick Mccabe 94976cb369 HDFS-7054. Make DFSOutputStream tracing more fine-grained (cmccabe) 
(cherry picked from commit 8234fd0e10)
(cherry picked from commit 79c07bbaca)
 2015-03-18 18:15:34 -07:00
Haohui Mai 388696c089 HDFS-7945. The WebHdfs system on DN does not honor the length parameter. Contributed by Haohui Mai. 2015-03-18 16:20:07 -07:00
Arpit Agarwal bb986988d4 HDFS-7914. TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use. (Contributed by Xiaoyu Yao) 2015-03-18 14:05:37 -07:00
Haohui Mai 482aeee784 HDFS-7697. Mark the PB OIV tool as experimental. Contributed by Lei (Eddy) Xu. 2015-03-18 13:34:21 -07:00
Arpit Agarwal 5f7926dcf0 Fix CHANGES.txt for HDFS-7722. 2015-03-18 12:53:02 -07:00
Arpit Agarwal 2b589da779 HDFS-7951. Fix NPE for TestFsDatasetImpl#testAddVolumeFailureReleasesInUseLock on Linux. (Contributed by Xiaoyu Yao) 2015-03-18 12:46:17 -07:00
Arpit Agarwal ae6d273d63 HDFS-7950. Fix TestFsDatasetImpl#testAddVolumes failure on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:35:41 -07:00
Arpit Agarwal 8944bb3d2e HDFS-7948. TestDataNodeHotSwapVolumes#testAddVolumeFailures failed on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:26:22 -07:00
Haohui Mai 5293ae0311 HDFS-7953. NN Web UI fails to navigate to paths that contain #. Contributed by kanaka kumar avvaru. 2015-03-18 11:08:49 -07:00
Arpit Agarwal 26c35438f3 HDFS-7946. TestDataNodeVolumeFailureReporting NPE on Windows. (Contributed by Xiaoyu Yao) 2015-03-17 21:30:13 -07:00
Colin Patrick Mccabe d2dad74421 HDFS-7940. Add tracing to DFSClient#setQuotaByStorageType (Rakesh R via Colin P. McCabe) 
(cherry picked from commit d8846707c5)
(cherry picked from commit 455d4aa8a1)
 2015-03-17 11:02:05 -07:00
yliu ef9d46dcb6 HDFS-7838. Expose truncate API for libhdfs. (yliu) 2015-03-17 07:25:58 +08:00
Colin Patrick Mccabe 861cc05092 HDFS-7722. DataNode#checkDiskError should also remove Storage when error is found. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit b49c3a1813)
(cherry picked from commit 7455412a24)
 2015-03-16 18:42:21 -07:00
Colin Patrick Mccabe 647d9ae27a HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe) 
(cherry picked from commit bc9cb3e271)
(cherry picked from commit c71b54fa32)
 2015-03-16 18:39:58 -07:00
Konstantin V Shvachko ad4d704fbb HDFS-7886. Fix TestFileTruncate falures. Contributed by Plamen Jeliazkov and Konstantin Shvachko. 2015-03-16 12:43:46 -07:00
Konstantin V Shvachko 41530b4173 HDFS-2605. Remove redundant "Release 0.21.1" section from CHANGES.txt. Contributed by Allen Wittenauer. 2015-03-13 13:45:47 -07:00
Konstantin V Shvachko f6bdcd938e HDFS-7903. Cannot recover block after truncate and delete snapshot. Contributed by Plamen Jeliazkov. 2015-03-13 12:45:47 -07:00
Kihwal Lee 116a7f1a16 HDFS-7435. PB encoding of block reports is very inefficient. Contributed by Daryn Sharp. 
(cherry picked from commit d324164a51)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
(cherry picked from commit 464271a5ed)
 2015-03-13 14:42:02 -05:00
Brandon Li cdeb1079ea HDFS-7926. NameNode implementation of ClientProtocol.truncate(..) is not idempotent. Contributed by Tsz Wo Nicholas Sze 
(cherry picked from commit f446669afb)
(cherry picked from commit af80a98ace)
 2015-03-13 10:47:25 -07:00
Tsz-Wo Nicholas Sze a1fffc3fcc HDFS-6833. DirectoryScanner should not register a deleting block with memory of DataNode. Contributed by Shinichi Yamashita 2015-03-13 02:29:24 +08:00
Akira Ajisaka 718cd0f1f4 HDFS-7880. Remove the tests for legacy Web UI in branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6c80a3d34a)
 2015-03-12 10:55:47 +09:00
Colin Patrick Mccabe 8241228db7 HDFS-7830. DataNode does not release the volume lock when adding a volume fails. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 5c1036d598)
(cherry picked from commit eefca23e8c)
 2015-03-12 10:49:48 +09:00
cnauroth 0fe5f5b5b8 HDFS-7491. Add incremental blockreport latency to DN metrics. Contributed by Ming Ma. 
(cherry picked from commit fb34f45727)
(cherry picked from commit 5e99b7cd2f)
 2015-03-11 14:12:06 -07:00
Jing Zhao 368ab2cd37 HDFS-7898. Change TestAppendSnapshotTruncate to fail-fast. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit e43882e84a)
(cherry picked from commit c7105fcff0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-10 10:31:41 -07:00
Akira Ajisaka c9a2e6f5db HDFS-6806. HDFS Rolling upgrade document should mention the versions available. Contributed by J.Andreina. 
(cherry picked from commit 82db3341bf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-10 11:27:30 +09:00
Tsuyoshi Ozawa 28f4e6b222 HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 
(cherry picked from commit b46f9e72db)
 2015-03-09 19:54:45 +09:00
Chris Douglas 3bc4f3502a HDFS-7411. Change decommission logic to throttle by blocks rather 
than nodes in each interval. Contributed by Andrew Wang
 2015-03-08 18:32:20 -07:00
Brandon Li c8af3b2768 HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li 
(cherry picked from commit 0f8ecb1d0c)
 2015-03-06 15:22:20 -08:00
Haohui Mai ab251fd355 HDFS-7818. OffsetParam should return the default value instead of throwing NPE when the value is unspecified. Contributed by Eric Payne. 2015-03-06 14:26:47 -08:00
Jing Zhao 994dadb9ba HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 24db0812be)
 2015-03-06 10:57:38 -08:00
Jing Zhao c5d52c7b52 HDFS-7855. Separate class Packet from DFSOutputStream. Contributed by Li Bo. 
(cherry picked from commit 952640fa4c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-05 11:11:08 -08:00
Tsuyoshi Ozawa f805d48b19 HADOOP-11648. Set DomainSocketWatcher thread name explicitly. Contributed by Liang Xie. 
(cherry picked from commit 74a4754d1c)
 2015-03-05 16:06:37 +09:00
Tsz-Wo Nicholas Sze f9a2007aff HDFS-7746. Add a test randomly mixing append, truncate and snapshot operations. 2015-03-05 10:23:14 +08:00
Dongming Liang ada545d3b7 HDFS-1522. Combine two BLOCK_FILE_PREFIX constants into one. Contributed by Dongming Liang. 2015-03-04 17:52:25 -08:00
Kihwal Lee f85530f649 HDFS-7434. DatanodeID hashCode should not be mutable. Contributed by Daryn Sharp. 
(cherry picked from commit 722b479469)
 2015-03-04 17:23:00 -06:00
Jing Zhao b2ccf54c14 HDFS-7535. Utilize Snapshot diff report for distcp. Contributed by Jing Zhao. 
(cherry picked from commit ed70fa142c)
 2015-03-04 10:32:39 -08:00
Haohui Mai d020544ef6 HDFS-7879. hdfs.dll does not export functions of the public libhdfs API. Contributed by Chris Nauroth. 2015-03-04 09:17:29 -08:00
Vinayakumar B c24448c27b HDFS-7869. Inconsistency in the return information while performing rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 3560180b6e)
 2015-03-04 14:39:51 +05:30
Akira Ajisaka 31b3f84601 HADOOP-11666. Revert the format change of du output introduced by HADOOP-6857. Contributed by Byron Wong. 2015-03-03 21:46:28 -08:00
Aaron T. Myers bce3d442ff HDFS-7682. {{DistributedFileSystem#getFileChecksum}} of a snapshotted file includes non-snapshotted content. Contributed by Charles Lamb. 
(cherry picked from commit f2d7a67a2c)
 2015-03-03 18:09:31 -08:00
Haohui Mai 65bfde552a HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. 2015-03-03 17:58:21 -08:00
Arpit Agarwal 7e6624c218 HDFS-7757. Misleading error messages in FSImage.java. (Contributed by Brahma Reddy Battula) 2015-03-03 10:55:32 -08:00
Jing Zhao 6090f51725 HDFS-7871. NameNodeEditLogRoller can keep printing 'Swallowing exception' message. Contributed by Jing Zhao. 
(cherry picked from commit b442aeec95)
 2015-03-02 20:23:10 -08:00
Haohui Mai 651aa8761e HDFS-7785. Improve diagnostics information for HttpPutFailedException. Contributed by Chengbing Liu. 2015-03-02 15:35:19 -08:00
Gera Shegalov 52d0724a20 HDFS-7789. DFSck should resolve the path to support cross-FS symlinks. (gera) 
(cherry picked from commit cbb492578e)
 2015-03-02 01:05:15 -08:00
Tsz-Wo Nicholas Sze a5f3156b30 HDFS-7439. Add BlockOpResponseProto's message to the exception messages. Contributed by Takanobu Asanuma 2015-03-02 15:05:06 +08:00
Akira Ajisaka d664705fd1 HDFS-5853. Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml (aajisaka) 
(cherry picked from commit aa55fd3096)
 2015-03-01 01:18:36 -08:00
Tsz-Wo Nicholas Sze c52636df3f recommit "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir." 
(cherry picked from commit acc172e371)
 2015-02-27 14:20:26 -08:00
Akira Ajisaka 0f9289e848 HDFS-7685. Document dfs.namenode.heartbeat.recheck-interval in hdfs-default.xml. Contributed by Kai Sasaki. 
(cherry picked from commit 8719cdd4f6)
 2015-02-27 12:18:46 -08:00
Tsz-Wo Nicholas Sze 657b027bb2 HDFS-7308. Change the packet chunk size computation in DFSOutputStream in order to enforce packet size <= 64kB. Contributed by Takuya Fukudome 2015-02-27 23:46:42 +08:00
Vinayakumar B bc60404eaf HDFS-6753. Initialize checkDisk when DirectoryScanner not able to get files list for scanning (Contributed by J.Andreina) 
(cherry picked from commit 4f75b15628)
 2015-02-27 16:37:03 +05:30
Tsz-Wo Nicholas Sze d223a4a594 Revert "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir." 
This reverts commit acc172e371.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-27 18:27:19 +08:00
cnauroth 5dfb793b05 HDFS-7774. Unresolved symbols error while compiling HDFS on Windows 7/32 bit. Contributed by Kiran Kumar M R. 
(cherry picked from commit c6d5b37a1c)
 2015-02-26 13:37:58 -08:00
Colin Patrick Mccabe 8b3b9568b6 HDFS-7819. Log WARN message for the blocks which are not in Block ID based layout (Rakesh R via Colin P. McCabe) 
(cherry picked from commit f0c980abed)
 2015-02-26 12:03:11 -08:00
Vinayakumar B f0f3d1937d HDFS-7832. Show 'Last Modified' in Namenode's 'Browse Filesystem' (Contributed by Vinayakumar B) 
(cherry picked from commit 166eecf687)
 2015-02-26 14:37:03 +05:30
Tsz-Wo Nicholas Sze 22b13a0ddb HDFS-7537. Add "UNDER MIN REPL'D BLOCKS" count to fsck. Contributed by GAO Rui 2015-02-26 11:47:04 +08:00
Tsz-Wo Nicholas Sze 12fe3afcd3 HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade. 2015-02-26 10:15:49 +08:00
Benoy Antony 4e400030f6 HDFS-7467. Provide storage tier information for a directory via fsck. (Benoy Antony) 2015-02-25 16:46:41 -08:00
Colin Patrick Mccabe d389a1ae98 HDFS-7495. Remove updatePosition argument from DFSInputStream#getBlockAt() (cmccabe) 
(cherry picked from commit caa42adf20)
 2015-02-25 13:39:02 -08:00
Colin Patrick Mccabe 5c0073e706 HDFS-7668. Backport "Convert site documentation from apt to markdown" to branch-2 (Masatake Iwasaki via Colin P. McCabe) 
(backported from commit 2f1e5dc628)

Conflicts:
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithNFS.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithQJM.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsDesign.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsQuotaAdminGuide.apt.vm
 2015-02-24 16:18:18 -08:00
Andrew Wang efb7e287f4 HDFS-7763. fix zkfc hung issue due to not catching exception in a corner case. Contributed by Liang Xie. 
(cherry picked from commit 7105ebaa9f)
 2015-02-24 15:31:18 -08:00
Jing Zhao 8346427929 HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. 
(cherry picked from commit 73bcfa99af)
 2015-02-24 10:34:42 -08:00
Tsuyoshi Ozawa 280dda5bcd HDFS-7008. xlator should be closed upon exit from DFSAdmin#genericRefresh(). (ozawa) 
(cherry picked from commit b53fd7163b)
 2015-02-24 23:59:59 +09:00
Colin Patrick Mccabe 453b210d0a HDFS-7805. NameNode recovery prompt should be printed on console (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit faaddb6ecb)
 2015-02-23 19:06:01 -08:00
Colin Patrick Mccabe f4da6f7193 HDFS-7807. libhdfs htable.c: fix htable resizing, add unit test (cmccabe) 
(cherry picked from commit 585768667e)
 2015-02-23 18:45:44 -08:00
cnauroth 657a6e389b HDFS-7009. Active NN and standby NN have different live nodes. Contributed by Ming Ma. 
(cherry picked from commit 769507bd7a)
 2015-02-23 15:12:36 -08:00
Arpit Agarwal 83f779e49a HDFS-7806. Refactor: move StorageType from hadoop-hdfs to hadoop-common. (Contributed by Xiaoyu Yao) 2015-02-21 15:48:59 -08:00
yliu eca1588db8 HDFS-7740. Test truncate with DataNodes restarting. (yliu) 2015-02-21 06:33:58 +08:00
cnauroth 9f06fb3aef HDFS-7813. TestDFSHAAdminMiniCluster#testFencer testcase is failing frequently. Contributed by Rakesh R. 
(cherry picked from commit 0d6af574e0)
 2015-02-20 17:01:22 -08:00
cnauroth e09ba94f28 HDFS-7773. Additional metrics in HDFS to be accessed via jmx. Contributed by Anu Engineer. 2015-02-20 12:38:34 -08:00
cnauroth ad77dfc3e4 HDFS-7814. Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota". Contributed by Xiaoyu Yao. 
(cherry picked from commit 8c6ae0d619)
 2015-02-20 12:22:00 -08:00
Kihwal Lee b9157f92fc HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. 
(cherry picked from commit 7ae5255a16)
 2015-02-20 09:09:56 -06:00
Harsh J 5c2c6b00dd HDFS-7752. Improve description for "dfs.namenode.num.extra.edits.retained" and "dfs.namenode.num.checkpoints.retained" properties on hdfs-default.xml. Contributed by Wellington Chevreuil. 
(cherry picked from commit b9a17909ba)
 2015-02-20 19:21:34 +05:30
yliu fee29e4a4f HDFS-7656. Expose truncate API for HDFS httpfs. (yliu) 2015-02-19 08:38:25 +08:00