dd99f5fc79
HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu.
2016-05-17 16:12:00 -07:00
16c07cc68a
HDFS-10381, DataStreamer DataNode exclusion log message should be warning. (John Zhuge via Yongjun Zhang)
2016-05-17 15:57:33 -07:00
f6ef876fe1
MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen.
2016-05-17 14:41:47 -07:00
e24fe2641b
HDFS-9732, Improve DelegationTokenIdentifier.toString() for better logging. Contributed by Yongjun Zhang.
2016-05-17 14:03:08 -07:00
1356cbe994
Revert "HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu."
...
This reverts commit 0942954e8a
2016-05-17 15:20:03 -05:00
0942954e8a
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-17 15:06:25 -05:00
7cd5ae62f6
HDFS-10404. Fix formatting of CacheAdmin command usage help text (Yiqun Lin via cmccabe)
2016-05-17 11:10:48 -07:00
4a5819dae2
HDFS-10303. DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. Contributed by Surendra Singh Lilhore.
2016-05-17 08:53:31 -05:00
9fe5828f05
HDFS-10242. Cannot create space quota of zero. Contributed by Takashi Ohnishi.
2016-05-17 15:53:09 +09:00
61f46be071
HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee)
2016-05-16 18:49:47 -07:00
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
6a6e74acf5
HDFS-10410. RedundantEditLogInputStream.LOG is set to wrong class. (John Zhuge via lei)
2016-05-16 17:05:46 -07:00
ad9441122f
HDFS-8449. Add tasks count metrics to datanode for ECWorker. Contributed by Bo Li.
2016-05-15 23:39:09 -07:00
45788204ae
HDFS-10333. Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk. Contributed by Yiqun Lin.
2016-05-15 22:15:05 -07:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
a2774debf7
HDFS-9389. Add maintenance states to AdminStates. (Ming Ma via lei)
2016-05-12 15:42:54 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
7251bb922b
HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-11 17:53:40 -07:00
687233f20d
HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe)
2016-05-11 13:45:39 -07:00
307db61aac
HDFS-10172. hdfs erasurecode command should remove the redundant -usage option. Contributed by Yuanbo Liu.
2016-05-10 15:20:17 -07:00
f0252ad252
HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe)
2016-05-10 11:15:05 -07:00
b9e5a32fa1
HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah.
2016-05-09 15:50:09 -05:00
411fb4b2b7
HDFS-10337. OfflineEditsViewer stats option should print 0 instead of null for the count of operations. Contributed by Yiqun Lin.
2016-05-08 15:49:13 -07:00
6957e45699
HDFS-10374. typo in HDFSHighAvailabilityWithQJM doc. Contributed by Takashi Ohnishi.
2016-05-07 11:26:42 +09:00
2835f146b5
HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun.
2016-05-06 15:36:47 +09:00
dacd1f50fe
HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang.
2016-05-05 18:08:47 -07:00
d8faf47f32
HADOOP-13051. Test for special characters in path being respected during globPaths. Contributed by Harsh J.
2016-05-05 14:18:39 -07:00
35cf503149
HADOOP-13077. Handle special characters in passwords in httpfs.sh (Xiao Chen via aw)
2016-05-05 11:33:06 -07:00
1268cf5fbe
HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma)
2016-05-04 17:02:26 -07:00
6d77d6eab7
HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. (Contributed by Brahma Reddy Battula)
2016-05-03 16:52:43 -07:00
45a753ccf7
HDFS-10344. DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash. Contributed by Xiaoyu Yao.
2016-05-02 19:30:47 -07:00
3ff0510ffd
Fix hadoop-hdfs-native-client compilation on Windows. Contributed by Brahma Reddy Battula.
2016-05-02 18:15:51 -07:00
1b9f18623a
HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou
2016-05-02 11:16:30 -07:00
2753185a01
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-29 22:32:27 -07:00
7da540d03e
HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah.
2016-04-29 15:50:43 -05:00
4da6f69ca1
HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu
2016-04-29 11:22:33 -07:00
af9b000535
HDFS-10260. TestFsDatasetImpl#testCleanShutdownOfVolume often fails. Contributed by Rushabh Shah.
2016-04-29 09:58:44 -05:00
6243eabb48
HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla
2016-04-28 12:44:53 -05:00
8bd0bca0b1
HDFS-3702. Fix missing imports from HDFS-3702 trunk patch.
2016-04-27 15:37:41 -07:00
0a152103f1
HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu)
2016-04-27 14:22:51 -07:00
7b1c37a13a
HADOOP-12892. fix/rewrite create-release (aw)
2016-04-27 08:38:22 -07:00
919a1d824a
HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla.
2016-04-27 08:19:48 -05:00
fc94810d3f
HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-04-26 17:10:13 -07:00
6be22ddbf1
HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge.
2016-04-26 13:04:18 -07:00
c6a2430b47
HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun.
2016-04-26 11:45:29 -05:00
5865fe2bf0
HDFS-9869. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-2]. Contributed by Rakesh R.
2016-04-25 22:01:54 -07:00
8eadd7145e
HDFS-10325. TestDFSShell#testMoveWithTargetPortEmpty fails. Contributed by Takanobu Asanuma.
2016-04-26 07:11:13 +09:00
f308561f1d
HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma
2016-04-25 12:01:48 -07:00
10f0f7851a
HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor.
2016-04-25 11:38:14 -05:00
6fcde2e38d
HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims)
2016-04-23 23:37:56 +09:00
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
d6402faded
Revert "HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano"
...
This reverts commit 4838b735f0
2016-04-22 10:17:20 -07:00
5d4255a801
HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang.
2016-04-22 11:08:44 -05:00
4a043b3fec
HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou.
2016-04-21 20:17:27 -07:00
14ab7a81e2
HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand
2016-04-21 15:41:50 -07:00
bbce1d525e
HDFS-10319. Balancer should not try to pair storages with different types.
2016-04-21 15:20:12 -07:00
4838b735f0
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-21 11:32:39 -07:00
b4be288c5d
HDFS-10207. Support enable Hadoop IPC backoff without namenode restart. Contributed by Xiaobing Zhou.
2016-04-21 10:18:48 -07:00
5bd7b592e5
HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao.
2016-04-20 23:10:52 -07:00
63ac2db59a
HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth.
2016-04-20 13:39:44 -07:00
ad36fa6f42
HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe)
2016-04-20 11:33:11 -07:00
af9bdbe447
HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou)
2016-04-19 11:26:30 -07:00
90d1af1416
HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun.
2016-04-19 18:54:11 +08:00
4b4df2cd48
HDFS-10284. o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently. Contributed by Mingliang Liu.
2016-04-19 18:07:46 +08:00
438efc3634
HDFS-10291 TestShortCircuitLocalRead failing (stevel)
...
(cherry picked from commit dd64769699
2016-04-19 17:48:20 +08:00
b5d4c7dc76
HADOOP-12924. Configure raw erasure coders for supported codecs. Contributed by Rui Li.
2016-04-20 15:47:01 +08:00
be0bce1b71
HDFS-10306. SafeModeMonitor should not leave safe mode if name system is starting active service. Contributed by Mingliang Liu.
2016-04-18 18:05:49 -07:00
cb3ca460ef
HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe)
2016-04-18 13:47:56 -07:00
d8b729e16f
HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun.
2016-04-18 07:58:55 -05:00
ab903029a9
HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun.
2016-04-18 20:29:29 +08:00
67523ffcf4
HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi.
2016-04-18 09:28:02 +08:00
55e19b7f0c
HDFS-10293. StripedFileTestUtil#readAll flaky. Contributed by Mingliang Liu.
2016-04-15 10:53:40 -07:00
89a838769f
HDFS-10283. o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently. Contributed by Mingliang Liu.
2016-04-15 10:49:21 -07:00
b9c9d03591
HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu.
2016-04-14 15:24:39 -05:00
2c155afe27
HDFS-10292. Add block id when client got Unable to close file exception. Contributed by Brahma Reddy Battula.
2016-04-14 14:25:11 -05:00
809226752d
HDFS-10286. Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties. Contributed by Xiaobing Zhou.
2016-04-14 10:57:02 -07:00
c970f1d005
HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang.
2016-04-14 12:45:47 -05:00
0d1c1152f1
HDFS-10282. The VolumeScanner should warn about replica files which are misplaced. Contributed by Colin Patrick McCabe.
2016-04-14 07:58:24 -05:00
5566177c9a
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode. Contributed by Xiaobing Zhou.
2016-04-13 16:51:13 -07:00
4895c73dd4
Revert "HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode"
...
This reverts commit 192112d5a2
2016-04-13 16:45:18 -07:00
314aa21a89
HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun.
2016-04-13 16:39:50 -07:00
192112d5a2
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode
2016-04-13 15:34:41 -07:00
d2f3bbc290
HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák
2016-04-13 11:22:36 -05:00
903428bf94
HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun.
2016-04-13 17:49:45 +08:00
35f0770555
HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe)
2016-04-12 16:28:42 -07:00
6ef42873a0
HDFS-9918. Erasure Coding: Sort located striped blocks based on decommissioned states. Contributed by Rakesh R.
2016-04-12 13:38:58 -07:00
600d129bb8
HDFS-10273. Remove duplicate logSync() and log message in FSN#enterSafemode() (Vinayakumar B via cmccabe)
2016-04-12 10:52:39 -07:00
209303be3a
HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang.
2016-04-12 15:26:50 +09:00
a9a607f8fc
HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula)
2016-04-12 11:44:34 +05:30
a409508b3f
HDFS-10277. PositionedReadable test testReadFullyZeroByteFile failing in HDFS. Contributed by Steve Loughran.
2016-04-12 14:55:01 +09:00
843ee8d59d
HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran.
2016-04-08 13:36:58 -07:00
9c32f8785e
HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang.
2016-04-08 15:32:13 +09:00
3c18a53cbd
HDFS-9719. Refactoring ErasureCodingWorker into smaller reusable constructs. Contributed by Kai Zheng.
2016-04-06 22:50:24 -07:00
654cd1d0c0
HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R
2016-04-07 12:38:42 +08:00
4bd7cbc29d
HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose
2016-04-06 21:07:31 -07:00
aede8c10ec
HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee
2016-04-06 20:20:14 +00:00
188f65287d
HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe)
2016-04-06 11:28:34 -07:00
221b3a8722
HDFS-10192. Namenode safemode not coming out during failover. Contributed by Brahma Reddy Battula.
2016-04-06 10:42:59 -07:00
9ba1e5af06
HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger.
2016-04-05 16:26:18 -05:00
Xiaoyu Yao
Yongjun Zhang
Junping Du
Kihwal Lee
Colin Patrick Mccabe
Akira Ajisaka
Sangjin Lee
Allen Wittenauer
Lei Xu
Kai Zheng
Andrew Wang
Tsz-Wo Nicholas Sze
Zhe Zhang
Masatake Iwasaki
Ravi Prakash
Ming Ma
Arpit Agarwal
Ravi Prakash
Jing Zhao
Chris Nauroth
Walter Su
Steve Loughran
Xiaobing Zhou
Vinayakumar B
Uma Maheswara Rao G
Eric Payne