7100c0da35
HDFS-9588. DiskBalancer: Add submitDiskbalancer RPC. (Anu Engineer via Arpit Agarwal)
2016-06-23 18:18:48 -07:00
5724a10316
HDFS-9469. DiskBalancer: Add Planner. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
e325c6ade9
HDFS-9611. DiskBalancer : Replace htrace json imports with jackson. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
599eca07b4
HDFS-9526. Fix jackson annotation imports. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
30c6ebd699
HDFS-9449. DiskBalancer: Add connectors. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
91a5c48143
HDFS-9420. Add DataModels for DiskBalancer. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
dca298d79e
HDFS-10469. Add number of active xceivers to datanode metrics. Contributed by Hanisha Koneru.
2016-06-23 14:13:39 -07:00
e6cb07520f
HDFS-10460. Recompute block checksum for a particular range less than file size on the fly by reconstructing missed block. Contributed by Rakesh R
2016-06-24 17:39:32 +08:00
ff07b10803
HDFS-10561. test_native_mini_dfs fails by NoClassDefFoundError. (aajisaka)
2016-06-23 15:48:18 +09:00
01643020d2
HDFS-10555: Unable to loadFSEdits due to a failure in readCachePoolInfo. Contributed by Uma Maheswara Rao G
2016-06-22 22:06:13 -07:00
17eae9ebb3
HDFS-10473: Allow only suitable storage policies to be set on striped files. Contributed by Uma Maheswara Rao G
2016-06-22 11:17:43 -07:00
d433b16ce6
HDFS-7597. DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping. Contributed by Daryn Sharp, Bob Hansen, and Xiao Chen.
2016-06-22 11:00:16 +09:00
f2ac132d6a
HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula)
2016-06-21 15:42:28 +05:30
46f1602e89
HDFS-10448. CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 (Yiqun Lin via cmccabe)
2016-06-20 18:25:09 -07:00
7adc4d1769
HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey.
2016-06-20 17:22:55 -07:00
b7c4cf7129
HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu.
2016-06-20 16:25:30 -07:00
aa1b583cf9
HDFS-10423. Increase default value of httpfs maxHttpHeaderSize. Contributed by Nicolae Popa.
2016-06-20 13:46:56 -07:00
5f6b4157a4
HDFS-10328. Add per-cache-pool default replication num configuration (xupeng via cmccabe)
2016-06-20 10:42:50 -07:00
0319d73c3b
HDFS-10437. ReconfigurationProtocol not covered by HDFSPolicyProvider. (Arpit Agarwal)
2016-06-18 16:16:54 -07:00
280069510b
HDFS-10493. Add links to datanode web UI in namenode datanodes page. Contributed by Weiwei Yang.
2016-06-17 11:05:44 -05:00
09e82acaf9
HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang.
2016-06-17 16:20:38 +09:00
e983eafb44
HDFS-10532. Typo in RollingUpgrade docs. (Contributed by Yiqun Lin)
2016-06-16 10:10:33 -07:00
cc6c265171
HDFS-10256. Use GenericTestUtils.getTestDir method in tests for temporary directories (Contributed by Vinayakumar B)
2016-06-16 16:47:06 +05:30
2ca73445f5
HDFS-10525. Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap (Xiao Chen via cmccabe)
2016-06-15 22:47:01 -07:00
2449db507d
HDFS-10505. OIV's ReverseXML processor should support ACLs (Surendra Singh Lilhore via cmccabe)
2016-06-15 22:35:19 -07:00
b48f27e794
HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma)
2016-06-15 22:00:52 -07:00
7d521a29ee
HDFS-9016. Display upgrade domain information in fsck. (mingma)
2016-06-14 20:05:50 -07:00
709a814fe0
HDFS-10516. Fix bug when warming up EDEK cache of more than one encryption zone. Contributed by Xiao Chen.
2016-06-13 11:50:19 -07:00
d44f4745b4
HDFS-7987. Allow files / directories to be moved (Ravi Prakash via aw)
2016-06-10 09:02:28 -07:00
8ea9bbce26
HDFS-10508. DFSInputStream should set thread's interrupt status after catching InterruptException from sleep. Contributed by Jing Zhao.
2016-06-09 14:52:29 +09:00
ae047655f4
HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak)
2016-06-08 13:44:22 -07:00
1ee9ea0026
Remove redundant TestMiniDFSCluster.testDualClusters. Contributed by Jiayi Zhou.
2016-06-08 12:58:56 -07:00
be34e85e68
HDFS-10468. HDFS read ends up ignoring an interrupt. Contributed by Jing Zhao
2016-06-07 10:48:21 -07:00
e620530301
HDFS-10485. Fix findbugs warning in FSEditLog.java. (aajisaka)
2016-06-07 17:52:55 +09:00
6de9213df1
HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster.
2016-06-06 15:52:39 -07:00
35f255b03b
Revert "Revert "HADOOP-13226 Support async call retry and failover.""
...
This reverts commit 5360da8bd9
2016-06-06 16:31:43 +08:00
7e7b1ae037
Revert "Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.""
...
This reverts commit 8cf47d8589
2016-06-06 16:31:38 +08:00
db41e6d285
Revert "Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou""
...
This reverts commit 5ee5912ebd
2016-06-06 16:31:34 +08:00
b3d81f38da
Revert "Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit b82c74b910
2016-06-06 16:31:30 +08:00
574dcd34c0
Revert "Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls.""
...
This reverts commit e4450d47f1
2016-06-06 16:31:23 +08:00
cba9a01889
Revert "Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit f23d5dfc60
2016-06-06 16:29:38 +08:00
aa20fa150d
Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou""
...
This reverts commit 4d36b221a2
2016-06-06 16:28:47 +08:00
eded3d109e
Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit 106234d873
2016-06-06 16:28:21 +08:00
106234d873
Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit fc94810d3f
2016-06-03 18:09:55 -07:00
4d36b221a2
Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"
...
This reverts commit 1b9f18623a
2016-06-03 18:09:18 -07:00
f23d5dfc60
Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 7251bb922b
2016-06-03 18:09:17 -07:00
e4450d47f1
Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls."
...
This reverts commit 42c22f7e3d
2016-06-03 18:09:16 -07:00
b82c74b910
Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 02d4e478a3
2016-06-03 18:09:14 -07:00
5ee5912ebd
Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou"
...
This reverts commit f4b9bcd87c
2016-06-03 18:09:13 -07:00
8cf47d8589
Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou."
...
This reverts commit 21890c4239
2016-06-03 18:09:12 -07:00
5360da8bd9
Revert "HADOOP-13226 Support async call retry and failover."
...
This reverts commit 83f2f78c11
2016-06-03 18:09:09 -07:00
47e0321ee9
HDFS-10481. HTTPFS server should correctly impersonate as end user to open file. Contributed by Xiao Chen.
2016-06-03 17:21:17 -07:00
99a771cd7a
HDFS-7767. Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI (Ravi Prakash via aw)
2016-06-03 17:07:39 -07:00
15f018434c
HDFS-9877. HDFS Namenode UI: Fix browsing directories that need to be encoded (Ravi Prakash via aw)
2016-06-03 17:06:29 -07:00
713cb71820
HADOOP-13155. Implement TokenRenewer to renew and cancel delegation tokens in KMS. Contributed by Xiao Chen.
2016-06-03 16:48:54 -07:00
97e2449477
HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (Contributed by Akira Ajisaka)
2016-06-02 13:14:45 -07:00
1df6f5735c
HDFS-10471. DFSAdmin#SetQuotaCommand's help msg is not correct. Contributed by Yiqun Lin.
2016-06-03 04:10:32 +09:00
aadb77e412
HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula.
2016-06-02 20:55:04 +09:00
69555fca06
HDFS-9476. TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail. Contributed by Masatake Iwasaki.
2016-06-02 18:52:47 +09:00
d749cf65e1
HDFS-9833. Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data. Contributed by Rakesh R.
2016-06-02 12:56:21 +08:00
8ceb06e239
HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:54:34 -07:00
c7921c9bdd
HDFS-9466. TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:37:52 -07:00
29d6cadc52
HDFS-10415. TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called (Mingliang Liu via cmccabe)
2016-05-31 16:54:52 -07:00
83f2f78c11
HADOOP-13226 Support async call retry and failover.
2016-05-31 16:30:11 -07:00
93d8a7f2a2
Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng"
...
This reverts commit a6c79f92d5
2016-06-01 08:41:15 +08:00
21890c4239
HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.
2016-05-27 14:20:18 -07:00
5ea6fd85c7
HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang)
2016-05-27 10:02:02 -07:00
916140604f
HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li
2016-05-28 14:23:39 +08:00
8c84a2a93c
HDFS-10236. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-3]. Contributed by Rakesh R.
2016-05-26 16:50:55 -07:00
55c3e2de3d
HDFS-10463. TestRollingFileSystemSinkWithHdfs needs some cleanup. (Daniel Templeton via kasha)
2016-05-26 14:24:31 -07:00
f4b9bcd87c
HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou
2016-05-26 12:15:17 -07:00
77202fa103
HADOOP-13010. Refactor raw erasure coders. Contributed by Kai Zheng
2016-05-27 13:23:34 +08:00
f69f5ab3b6
HDFS-10434. Fix intermittent test failure of TestDataNodeErasureCodingMetrics. Contributed by Rakesh R
2016-05-27 04:42:53 +08:00
a6c79f92d5
HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng
2016-05-27 04:29:42 +08:00
2dab64da2f
HDFS-10217. show 'blockScheduled' tooltip in datanodes table.
2016-05-25 12:34:52 +05:30
02d4e478a3
HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-24 14:51:27 -07:00
15ed080e36
HDFS-9365. Balaner does not work with the HDFS-6376 HA setup.
2016-05-24 12:49:48 -07:00
57c31a3fef
HDFS-9782. RollingFileSystemSink should have configurable roll interval. (Daniel Templeton via kasha)
2016-05-24 10:37:00 -07:00
4b0f55b6ea
HDFS-7766. Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect (Ravi Prakash via aw)
2016-05-23 15:52:01 -07:00
12eb5a7b11
HDFS-10439. Update setOwner doc in HdfsPermissionsGuide. Contributed by John Zhuge.
2016-05-22 16:53:30 +09:00
500e946729
HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth.
2016-05-20 13:01:43 -07:00
bcde1562d2
HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth.
2016-05-20 12:47:41 -07:00
42c22f7e3d
HADOOP-13168. Support Future.get with timeout in ipc async calls.
2016-05-19 15:34:04 -07:00
204de430bf
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-19 15:52:32 -05:00
010e6ac328
HDFS-2173. saveNamespace should not throw IOE when only one storage directory fails to write VERSION file. Contributed by Andras Bokor.
2016-05-18 14:13:41 -07:00
cf552aa87b
HDFS-10360. DataNode may format directory and lose blocks if current/VERSION is missing. (Wei-Chiu Chuang via lei)
2016-05-18 09:38:37 -07:00
f4d8fde822
HDFS-9926. MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils. (Contributed by Josh Elser)
2016-05-18 09:20:57 -07:00
dd99f5fc79
HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu.
2016-05-17 16:12:00 -07:00
16c07cc68a
HDFS-10381, DataStreamer DataNode exclusion log message should be warning. (John Zhuge via Yongjun Zhang)
2016-05-17 15:57:33 -07:00
f6ef876fe1
MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen.
2016-05-17 14:41:47 -07:00
e24fe2641b
HDFS-9732, Improve DelegationTokenIdentifier.toString() for better logging. Contributed by Yongjun Zhang.
2016-05-17 14:03:08 -07:00
1356cbe994
Revert "HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu."
...
This reverts commit 0942954e8a
2016-05-17 15:20:03 -05:00
0942954e8a
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-17 15:06:25 -05:00
7cd5ae62f6
HDFS-10404. Fix formatting of CacheAdmin command usage help text (Yiqun Lin via cmccabe)
2016-05-17 11:10:48 -07:00
4a5819dae2
HDFS-10303. DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. Contributed by Surendra Singh Lilhore.
2016-05-17 08:53:31 -05:00
9fe5828f05
HDFS-10242. Cannot create space quota of zero. Contributed by Takashi Ohnishi.
2016-05-17 15:53:09 +09:00
61f46be071
HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee)
2016-05-16 18:49:47 -07:00
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
6a6e74acf5
HDFS-10410. RedundantEditLogInputStream.LOG is set to wrong class. (John Zhuge via lei)
2016-05-16 17:05:46 -07:00
ad9441122f
HDFS-8449. Add tasks count metrics to datanode for ECWorker. Contributed by Bo Li.
2016-05-15 23:39:09 -07:00
45788204ae
HDFS-10333. Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk. Contributed by Yiqun Lin.
2016-05-15 22:15:05 -07:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
a2774debf7
HDFS-9389. Add maintenance states to AdminStates. (Ming Ma via lei)
2016-05-12 15:42:54 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
7251bb922b
HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-11 17:53:40 -07:00
687233f20d
HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe)
2016-05-11 13:45:39 -07:00
307db61aac
HDFS-10172. hdfs erasurecode command should remove the redundant -usage option. Contributed by Yuanbo Liu.
2016-05-10 15:20:17 -07:00
f0252ad252
HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe)
2016-05-10 11:15:05 -07:00
b9e5a32fa1
HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah.
2016-05-09 15:50:09 -05:00
411fb4b2b7
HDFS-10337. OfflineEditsViewer stats option should print 0 instead of null for the count of operations. Contributed by Yiqun Lin.
2016-05-08 15:49:13 -07:00
6957e45699
HDFS-10374. typo in HDFSHighAvailabilityWithQJM doc. Contributed by Takashi Ohnishi.
2016-05-07 11:26:42 +09:00
2835f146b5
HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun.
2016-05-06 15:36:47 +09:00
dacd1f50fe
HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang.
2016-05-05 18:08:47 -07:00
d8faf47f32
HADOOP-13051. Test for special characters in path being respected during globPaths. Contributed by Harsh J.
2016-05-05 14:18:39 -07:00
35cf503149
HADOOP-13077. Handle special characters in passwords in httpfs.sh (Xiao Chen via aw)
2016-05-05 11:33:06 -07:00
1268cf5fbe
HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma)
2016-05-04 17:02:26 -07:00
6d77d6eab7
HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. (Contributed by Brahma Reddy Battula)
2016-05-03 16:52:43 -07:00
45a753ccf7
HDFS-10344. DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash. Contributed by Xiaoyu Yao.
2016-05-02 19:30:47 -07:00
3ff0510ffd
Fix hadoop-hdfs-native-client compilation on Windows. Contributed by Brahma Reddy Battula.
2016-05-02 18:15:51 -07:00
1b9f18623a
HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou
2016-05-02 11:16:30 -07:00
2753185a01
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-29 22:32:27 -07:00
7da540d03e
HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah.
2016-04-29 15:50:43 -05:00
4da6f69ca1
HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu
2016-04-29 11:22:33 -07:00
af9b000535
HDFS-10260. TestFsDatasetImpl#testCleanShutdownOfVolume often fails. Contributed by Rushabh Shah.
2016-04-29 09:58:44 -05:00
6243eabb48
HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla
2016-04-28 12:44:53 -05:00
8bd0bca0b1
HDFS-3702. Fix missing imports from HDFS-3702 trunk patch.
2016-04-27 15:37:41 -07:00
0a152103f1
HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu)
2016-04-27 14:22:51 -07:00
7b1c37a13a
HADOOP-12892. fix/rewrite create-release (aw)
2016-04-27 08:38:22 -07:00
919a1d824a
HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla.
2016-04-27 08:19:48 -05:00
fc94810d3f
HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-04-26 17:10:13 -07:00
6be22ddbf1
HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge.
2016-04-26 13:04:18 -07:00
c6a2430b47
HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun.
2016-04-26 11:45:29 -05:00
5865fe2bf0
HDFS-9869. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-2]. Contributed by Rakesh R.
2016-04-25 22:01:54 -07:00
8eadd7145e
HDFS-10325. TestDFSShell#testMoveWithTargetPortEmpty fails. Contributed by Takanobu Asanuma.
2016-04-26 07:11:13 +09:00
f308561f1d
HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma
2016-04-25 12:01:48 -07:00
10f0f7851a
HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor.
2016-04-25 11:38:14 -05:00
6fcde2e38d
HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims)
2016-04-23 23:37:56 +09:00
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
d6402faded
Revert "HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano"
...
This reverts commit 4838b735f0
2016-04-22 10:17:20 -07:00
5d4255a801
HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang.
2016-04-22 11:08:44 -05:00
4a043b3fec
HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou.
2016-04-21 20:17:27 -07:00
14ab7a81e2
HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand
2016-04-21 15:41:50 -07:00
bbce1d525e
HDFS-10319. Balancer should not try to pair storages with different types.
2016-04-21 15:20:12 -07:00
4838b735f0
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-21 11:32:39 -07:00
b4be288c5d
HDFS-10207. Support enable Hadoop IPC backoff without namenode restart. Contributed by Xiaobing Zhou.
2016-04-21 10:18:48 -07:00
5bd7b592e5
HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao.
2016-04-20 23:10:52 -07:00
63ac2db59a
HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth.
2016-04-20 13:39:44 -07:00
ad36fa6f42
HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe)
2016-04-20 11:33:11 -07:00
af9bdbe447
HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou)
2016-04-19 11:26:30 -07:00
90d1af1416
HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun.
2016-04-19 18:54:11 +08:00
4b4df2cd48
HDFS-10284. o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently. Contributed by Mingliang Liu.
2016-04-19 18:07:46 +08:00
438efc3634
HDFS-10291 TestShortCircuitLocalRead failing (stevel)
...
(cherry picked from commit dd64769699
2016-04-19 17:48:20 +08:00
b5d4c7dc76
HADOOP-12924. Configure raw erasure coders for supported codecs. Contributed by Rui Li.
2016-04-20 15:47:01 +08:00
be0bce1b71
HDFS-10306. SafeModeMonitor should not leave safe mode if name system is starting active service. Contributed by Mingliang Liu.
2016-04-18 18:05:49 -07:00
cb3ca460ef
HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe)
2016-04-18 13:47:56 -07:00
d8b729e16f
HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun.
2016-04-18 07:58:55 -05:00
ab903029a9
HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun.
2016-04-18 20:29:29 +08:00
67523ffcf4
HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi.
2016-04-18 09:28:02 +08:00
55e19b7f0c
HDFS-10293. StripedFileTestUtil#readAll flaky. Contributed by Mingliang Liu.
2016-04-15 10:53:40 -07:00
89a838769f
HDFS-10283. o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently. Contributed by Mingliang Liu.
2016-04-15 10:49:21 -07:00
b9c9d03591
HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu.
2016-04-14 15:24:39 -05:00
2c155afe27
HDFS-10292. Add block id when client got Unable to close file exception. Contributed by Brahma Reddy Battula.
2016-04-14 14:25:11 -05:00
809226752d
HDFS-10286. Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties. Contributed by Xiaobing Zhou.
2016-04-14 10:57:02 -07:00
c970f1d005
HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang.
2016-04-14 12:45:47 -05:00
0d1c1152f1
HDFS-10282. The VolumeScanner should warn about replica files which are misplaced. Contributed by Colin Patrick McCabe.
2016-04-14 07:58:24 -05:00
5566177c9a
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode. Contributed by Xiaobing Zhou.
2016-04-13 16:51:13 -07:00
4895c73dd4
Revert "HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode"
...
This reverts commit 192112d5a2
2016-04-13 16:45:18 -07:00
314aa21a89
HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun.
2016-04-13 16:39:50 -07:00
192112d5a2
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode
2016-04-13 15:34:41 -07:00
d2f3bbc290
HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák
2016-04-13 11:22:36 -05:00
903428bf94
HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun.
2016-04-13 17:49:45 +08:00
35f0770555
HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe)
2016-04-12 16:28:42 -07:00
6ef42873a0
HDFS-9918. Erasure Coding: Sort located striped blocks based on decommissioned states. Contributed by Rakesh R.
2016-04-12 13:38:58 -07:00
600d129bb8
HDFS-10273. Remove duplicate logSync() and log message in FSN#enterSafemode() (Vinayakumar B via cmccabe)
2016-04-12 10:52:39 -07:00
209303be3a
HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang.
2016-04-12 15:26:50 +09:00
a9a607f8fc
HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula)
2016-04-12 11:44:34 +05:30
a409508b3f
HDFS-10277. PositionedReadable test testReadFullyZeroByteFile failing in HDFS. Contributed by Steve Loughran.
2016-04-12 14:55:01 +09:00
843ee8d59d
HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran.
2016-04-08 13:36:58 -07:00
9c32f8785e
HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang.
2016-04-08 15:32:13 +09:00
3c18a53cbd
HDFS-9719. Refactoring ErasureCodingWorker into smaller reusable constructs. Contributed by Kai Zheng.
2016-04-06 22:50:24 -07:00
654cd1d0c0
HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R
2016-04-07 12:38:42 +08:00
4bd7cbc29d
HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose
2016-04-06 21:07:31 -07:00
aede8c10ec
HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee
2016-04-06 20:20:14 +00:00
188f65287d
HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe)
2016-04-06 11:28:34 -07:00
221b3a8722
HDFS-10192. Namenode safemode not coming out during failover. Contributed by Brahma Reddy Battula.
2016-04-06 10:42:59 -07:00
9ba1e5af06
HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger.
2016-04-05 16:26:18 -05:00
0cd320a846
HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula.
2016-04-05 13:41:19 -07:00
85ec5573eb
HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims)
2016-04-06 03:22:48 +09:00
917464505c
HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla.
2016-04-05 09:07:24 -05:00
818d6b799e
HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula)
2016-04-05 09:49:39 +08:00
f6b1a81812
HDFS-8496. Calling stopWriter() with FSDatasetImpl lock held may block other threads (cmccabe)
2016-04-04 18:02:15 -07:00
a7d1fb0cd2
HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee.
2016-04-04 16:40:00 -05:00
154d2532cf
HDFS-9599. TestDecommissioningStatus.testDecommissionStatus occasionally fails (Lin Yiqun via iwasakims)
2016-04-05 03:19:48 +09:00
54b2e78fd2
HDFS-10253. Fix TestRefreshCallQueue failure (Contributed by Xiaoyu Yao)
2016-04-03 13:27:49 +05:30
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
32c0c3ecdf
HDFS-10221. Add .json to the rat exclusions. Contributed by Ming Ma.
2016-03-31 09:06:14 +09:00
37e23ce45c
HDFS-10223. peerFromSocketAndKey performs SASL exchange before setting connection timeouts (cmccabe)
2016-03-30 13:37:37 -07:00
09d63d5a19
HDFS-5177. blocksScheduled count should be decremented for abandoned blocks (Contributed by Vinayakumar B)
2016-03-30 14:22:00 +08:00
ddfe6774c2
HDFS-9439. Support reconfiguring fs.protected.directories without NN restart. (Contributed by Xiaobing Zhou)
2016-03-29 13:55:00 -07:00
c9307e48b7
HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA)
2016-03-29 13:50:08 -07:00
f2aec4eb82
HDFS-10197. TestFsDatasetCache failing intermittently due to timeout. Contributed by Lin Yiqun.
2016-03-29 11:54:42 -07:00
1f004b3367
HDFS-9871. "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. (Contributed by Brahma Reddy Battulla)
2016-03-29 11:25:41 +08:00
d8383c687c
HDFS-10182. Hedged read might overwrite user's buf. Contributed by zhouyingchao.
2016-03-28 15:44:25 +08:00
3a4ff7776e
HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng
2016-03-26 19:58:09 -07:00
a337ceb74e
Revert "HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng"
...
This reverts commit e5ff0ea7ba
2016-03-26 09:20:01 -07:00
e5ff0ea7ba
HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng
2016-03-26 00:52:50 -07:00
fde8ac5d85
Add missing files from HDFS-9005. (lei)
2016-03-25 17:11:35 -07:00
4fcfea71bf
HDFS-9005. Provide support for upgrade domain script. (Ming Ma via Lei Xu)
2016-03-25 17:09:12 -07:00
738155063e
HADOOP-12857. rework hadoop-tools (aw)
2016-03-23 13:46:38 -07:00
8f85e5d212
HDFS-10200. Docs for WebHDFS still describe GETDELEGATIONTOKENS operation. Contributed by Wellington Chevreuil.
2016-03-23 12:55:15 -07:00
0d19a0ce98
HDFS-10193. fuse_dfs segfaults if uid cannot be resolved to a username (John Thiltges via cmccabe)
2016-03-23 11:19:18 -07:00
a7d8f2b396
HDFS-10189. PacketResponder#toString should include the downstreams for PacketResponderType.HAS_DOWNSTREAM_IN_PIPELINE (Joe Pallas via cmccabe)
2016-03-22 14:56:02 -07:00
680716f31e
HDFS-9951. Use string constants for XML tags in OfflineImageReconstructor (Lin Yiqun via cmccabe)
2016-03-21 11:40:37 -07:00
e3bb38d625
HDFS-9405. Warmup NameNode EDEK caches in background thread. Contributed by Xiao Chen.
2016-03-21 11:39:05 -07:00
8a3f0cb255
HDFS-7166. SbNN Web UI shows #Under replicated blocks and #pending deletion blocks. Contributed by Wei-Chiu Chuang.
2016-03-20 17:21:25 -07:00
cd8b6889a7
HDFS-9579. Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level (Ming Ma via sjlee)
2016-03-19 14:02:04 -07:00
9b623fbaf7
HDFS-3677. dfs.namenode.edits.dir.required is missing from hdfs-default.xml. Contributed by Mark Yang.
2016-03-19 01:11:15 +09:00
63c966a3fb
HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah.
2016-03-18 10:24:59 -05:00
dc951e606f
HDFS-9949. Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared (Harsh J via cmccabe)
2016-03-17 10:37:42 -07:00
02a250db9f
HDFS-10173. Typo in DataXceiverServer. Contributed by Michael Han.
2016-03-16 18:49:57 -07:00
32d043d9c5
HDFS-9857. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-1]. Contributed by Rakesh R.
2016-03-16 16:53:58 -07:00
d457401784
HDFS-9904. testCheckpointCancellationDuringUpload occasionally fails. Contributed by Lin Yiqun.
2016-03-15 10:52:47 -05:00
5de848cd5d
HDFS-9928. Make HDFS commands guide up to date (Wei-Chiu Chuang via iwasakims)
2016-03-15 20:42:05 +09:00
9a43094e12
HDFS-9947. Block#toString should not output information from derived classes (cmccabe)
2016-03-14 14:59:11 -07:00
bd5556ba12
Download File from UI broken after pagination. Contributed by Brahma Reddy Battula
2016-03-14 14:30:13 -07:00
5644137ada
HDFS-9941. Do not log StandbyException on NN, other minor logging fixes. Contributed by Arpit Agarwal.
2016-03-14 09:54:54 -07:00
e01c6ea688
HDFS-1477. Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart. (Contributed by Xiaobing Zhou)
2016-03-10 19:03:55 -08:00
79961ecea8
HDFS-9933. ReverseXML should be capitalized in oiv usage message (cmccabe)
2016-03-10 13:43:27 -08:00
bd49354c6d
HDFS-9934. ReverseXML oiv processor should bail out if the XML file's layoutVersion doesn't match oiv's (cmccabe)
2016-03-10 13:41:06 -08:00
500875dfcc
HDFS-9927. Document the new OIV ReverseXML processor (Wei-Chiu Chuang via cmccabe)
2016-03-10 13:38:00 -08:00
7600e3c48f
HDFS-7866. Erasure coding: NameNode manages multiple erasure coding policies. Contributed by Rui Li.
2016-03-08 22:30:30 -08:00
89b16d27e2
HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth.
2016-03-08 21:22:06 -08:00
a14a6f08ee
HDFS-9919. TestStandbyCheckpoints#testNonPrimarySBNUploadFSImage waitForCheckpoint incorrectly. Contributed by Lin Yiqun.
2016-03-08 11:54:50 -08:00
743a99f2db
HDFS-8786. Erasure coding: use simple replication for internal blocks on decommissioning datanodes. Contributed by Rakesh R.
2016-03-08 10:24:57 -08:00
c2140d05ef
HDFS-9882. Add heartbeatsTotal in Datanode metrics. (Contributed by Hua Liu)
2016-03-07 21:10:24 -08:00
352d299cf8
HDFS-9812. Streamer threads leak if failure happens when closing DFSOutputStream. Contributed by Lin Yiqun.
2016-03-08 10:43:17 +09:00
724d2299cd
HDFS-9906. Remove spammy log spew when a datanode is restarted. (Contributed by Brahma Reddy Battula)
2016-03-07 12:19:53 -08:00
d718fc1ee5
HDFS-9865. TestBlockReplacement fails intermittently in trunk (Lin Yiqun via iwasakims)
2016-03-08 02:07:18 +09:00
fd1c09be3e
HDFS-9521. TransferFsImage.receiveFile should account and log separate times for image download and fsync to disk. Contributed by Wellington Chevreuil
2016-03-07 17:14:51 +05:30
2759689d7d
HDFS-9239. DataNode Lifeline Protocol: an alternative protocol for reporting DataNode liveness. Contributed by Chris Nauroth.
2016-03-04 15:29:50 -08:00
8e08861a14
HDFS-9889. Update balancer/mover document about HDFS-6133 feature. Contributed by Yongjun Zhang.
2016-03-04 13:25:29 -08:00
3e8099a45a
HDFS-9888. Allow reseting KerberosName in unit tests. Contributed by Xiao Chen.
2016-03-04 09:48:05 -08:00
cbd31328a6
HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)
2016-03-04 14:11:36 +09:00
1bc454615e
HADOOP-11792. Remove all of the CHANGES.txt files.
2016-03-03 21:02:31 -08:00
33a412e8a4
HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)
2016-03-03 18:57:23 +09:00
f70e317ccc
HADOOP-12793. Write a new group mapping service guide (Wei-Chiu Chuang via iwasakims)
2016-03-03 17:14:32 +09:00
27941a1811
HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou)
2016-03-02 18:35:28 -08:00
700b0e4019
HDFS-9835. OIV: add ReverseXML processor which reconstructs an fsimage from an XML file (cmccabe)
2016-03-02 17:56:11 -08:00
67880ccae6
HDFS-9886. Configuration properties for hedged read is broken. Contributed by Akira AJISAKA.
2016-03-02 10:26:06 -08:00
Arpit Agarwal
Tsz-Wo Nicholas Sze
Xiaoyu Yao
Kai Zheng
Akira Ajisaka
Uma Maheswara Rao G
Brahma Reddy Battula
Colin Patrick Mccabe
Jitendra Pandey
Aaron T. Myers
Kihwal Lee
Vinayakumar B
Ming Ma
Andrew Wang
Allen Wittenauer
Masatake Iwasaki
Ravi Prakash
Jing Zhao
Zhe Zhang
Chris Nauroth
Yongjun Zhang
Karthik Kambatla
Karthik Kambatla
Lei Xu
Junping Du
Sangjin Lee
Ravi Prakash
Walter Su
Steve Loughran
Xiaobing Zhou
Eric Payne
Haohui Mai
Zhe Zhang
Harsh J