0424056a77
HDFS-10534. NameNode WebUI should display DataNode usage rate with a certain percentile. Contributed by Kai Sasaki.
2016-06-24 08:58:29 -07:00
b2584bee45
HDFS-10571. TestDiskBalancerCommand#testPlanNode failed with IllegalArgumentException. Contributed by Xiaobing Zhou.
2016-06-23 18:27:02 -07:00
8a6e354122
HDFS-10552. DiskBalancer "-query" results in NPE if no plan for the node. Contributed by Anu Engineer.
2016-06-23 18:27:02 -07:00
e8de28181a
HDFS-10551. o.a.h.h.s.diskbalancer.command.Command does not actually verify options as expected. Contributed by Anu Engineer.
2016-06-23 18:27:02 -07:00
66fa34c839
HDFS-10557. Fix handling of the -fs Generic option. (Arpit Agarwal)
2016-06-23 18:27:01 -07:00
7b23ad1ef7
HDFS-10550. DiskBalancer: fix issue of order dependency in iteration in ReportCommand test. Contributed by Xiaobing Zhou.
2016-06-23 18:27:01 -07:00
c6ed54808d
HDFS-10547. DiskBalancer: fix whitespace issue in doc files. Contributed by Anu Engineer.
2016-06-23 18:27:01 -07:00
0774412e41
HDFS-10545. DiskBalancer: PlanCommand should use -fs instead of -uri to be consistent with other hdfs commands. Contributed by Anu Engineer.
2016-06-23 18:27:01 -07:00
5b8e1c26d7
HDFS-10541. Diskbalancer: When no actions in plan, error message says "Plan was generated more than 24 hours ago". Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
cb68e5b3bd
HDFS-10540. Diskbalancer: The CLI error message for disk balancer is not enabled is not clear. Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
3225c24e0e
HDFS-10520. DiskBalancer: Fix Checkstyle issues in test code. Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
7e2be5c4a0
HDFS-10518. DiskBalancer: Pretty-print json in Query command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
af11ab34d0
HDFS-10517. DiskBalancer: Support help command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
b502102bb1
HDFS-9461. DiskBalancer: Add Report Command. Contributed by Xiaobing Zhou.
2016-06-23 18:21:08 -07:00
121142cf95
HDFS-10501. DiskBalancer: Use the default datanode port if port is not provided. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
78a1032b71
HDFS-10500. Diskbalancer: Print out information when a plan is not generated. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
d2ff793ae4
HDFS-10496. DiskBalancer: ExecuteCommand checks planFile in a wrong way. Contributed by Lei (Eddy) Xu.
2016-06-23 18:21:08 -07:00
64ccb232cc
HDFS-10478. DiskBalancer: resolve volume path names. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
47dcb0f952
HDFS-10476. DiskBalancer: Plan command output directory should be a sub-directory. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
06a9799d84
HDFS-9547. DiskBalancer: Add user documentation. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
43eee50966
HDFS-10403. DiskBalancer: Add cancel command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
9e5fcb5e40
HDFS-10402. DiskBalancer: Add QueryStatus command. (Contributed by Anu Engineer)
2016-06-23 18:21:08 -07:00
5df2d2b8fd
HDFS-10399. DiskBalancer: Add JMX for DiskBalancer. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
1b39b283c7
HDFS-9546: DiskBalancer: Add Execute command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
75882ec0b0
HDFS-9545: DiskBalancer: Add Plan Command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
1594b472bb
HDFS-9543. DiskBalancer: Add Data mover. Contributed by Anu Engineer.
2016-06-23 18:20:24 -07:00
7820737cfa
HDFS-9735. DiskBalancer : Refactor moveBlockAcrossStorage to be used by disk balancer. Contributed by Anu Engineer.
2016-06-23 18:20:24 -07:00
050677077b
HDFS-9720. DiskBalancer : Add configuration parameters. Contributed by Anu Engineer.
2016-06-23 18:18:48 -07:00
6c606bf5c8
HDFS-9709. DiskBalancer : Add tests for disk balancer using a Mock Mover class. Contributed by Anu Engineer.
2016-06-23 18:18:48 -07:00
9be9703716
Fix a build break in HDFS-1312
2016-06-23 18:18:48 -07:00
ec601673a8
HDFS-10170. DiskBalancer: Force rebase diskbalancer branch. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
75a711a2d5
HDFS-9703. DiskBalancer: getBandwidth implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
918722bdd2
HDFS-9702. DiskBalancer: getVolumeMap implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
4b93ddae07
HDFS-8821. Stop tracking CHANGES.txt in the HDFS-1312 feature branch. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
747227e9de
HDFS-9817. Use SLF4J in new classes. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
3df0781aa7
HDFS-9856. Suppress Jenkins warning for sample JSON file. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
9847640603
HDFS-9683. DiskBalancer: Add cancelPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
e646c2eb50
HDFS-9681. DiskBalancer: Add QueryPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
2b1b2faf76
HDFS-9671. DiskBalancer: SubmitPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
66f0bb646d
HDFS-9647. DiskBalancer: Add getRuntimeSettings. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
96fe685b7a
HDFS-9645. DiskBalancer: Add Query RPC. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
0501d430e2
HDFS-9595. DiskBalancer: Add cancelPlan RPC. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
7100c0da35
HDFS-9588. DiskBalancer: Add submitDiskbalancer RPC. (Anu Engineer via Arpit Agarwal)
2016-06-23 18:18:48 -07:00
5724a10316
HDFS-9469. DiskBalancer: Add Planner. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
e325c6ade9
HDFS-9611. DiskBalancer : Replace htrace json imports with jackson. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
599eca07b4
HDFS-9526. Fix jackson annotation imports. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
30c6ebd699
HDFS-9449. DiskBalancer: Add connectors. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
91a5c48143
HDFS-9420. Add DataModels for DiskBalancer. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
dca298d79e
HDFS-10469. Add number of active xceivers to datanode metrics. Contributed by Hanisha Koneru.
2016-06-23 14:13:39 -07:00
e6cb07520f
HDFS-10460. Recompute block checksum for a particular range less than file size on the fly by reconstructing missed block. Contributed by Rakesh R
2016-06-24 17:39:32 +08:00
01643020d2
HDFS-10555: Unable to loadFSEdits due to a failure in readCachePoolInfo. Contributed by Uma Maheswara Rao G
2016-06-22 22:06:13 -07:00
17eae9ebb3
HDFS-10473: Allow only suitable storage policies to be set on striped files. Contributed by Uma Maheswara Rao G
2016-06-22 11:17:43 -07:00
d433b16ce6
HDFS-7597. DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping. Contributed by Daryn Sharp, Bob Hansen, and Xiao Chen.
2016-06-22 11:00:16 +09:00
f2ac132d6a
HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula)
2016-06-21 15:42:28 +05:30
46f1602e89
HDFS-10448. CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 (Yiqun Lin via cmccabe)
2016-06-20 18:25:09 -07:00
7adc4d1769
HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey.
2016-06-20 17:22:55 -07:00
5f6b4157a4
HDFS-10328. Add per-cache-pool default replication num configuration (xupeng via cmccabe)
2016-06-20 10:42:50 -07:00
0319d73c3b
HDFS-10437. ReconfigurationProtocol not covered by HDFSPolicyProvider. (Arpit Agarwal)
2016-06-18 16:16:54 -07:00
280069510b
HDFS-10493. Add links to datanode web UI in namenode datanodes page. Contributed by Weiwei Yang.
2016-06-17 11:05:44 -05:00
09e82acaf9
HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang.
2016-06-17 16:20:38 +09:00
b5f772abda
Merge branch 'trunk' into HDFS-7240
2016-06-16 11:29:40 -07:00
e983eafb44
HDFS-10532. Typo in RollingUpgrade docs. (Contributed by Yiqun Lin)
2016-06-16 10:10:33 -07:00
cc6c265171
HDFS-10256. Use GenericTestUtils.getTestDir method in tests for temporary directories (Contributed by Vinayakumar B)
2016-06-16 16:47:06 +05:30
2ca73445f5
HDFS-10525. Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap (Xiao Chen via cmccabe)
2016-06-15 22:47:01 -07:00
2449db507d
HDFS-10505. OIV's ReverseXML processor should support ACLs (Surendra Singh Lilhore via cmccabe)
2016-06-15 22:35:19 -07:00
b48f27e794
HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma)
2016-06-15 22:00:52 -07:00
7d521a29ee
HDFS-9016. Display upgrade domain information in fsck. (mingma)
2016-06-14 20:05:50 -07:00
709a814fe0
HDFS-10516. Fix bug when warming up EDEK cache of more than one encryption zone. Contributed by Xiao Chen.
2016-06-13 11:50:19 -07:00
d44f4745b4
HDFS-7987. Allow files / directories to be moved (Ravi Prakash via aw)
2016-06-10 09:02:28 -07:00
23923086d1
Merge branch 'trunk' into HDFS-7240
2016-06-09 20:34:39 -07:00
ae047655f4
HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak)
2016-06-08 13:44:22 -07:00
1ee9ea0026
Remove redundant TestMiniDFSCluster.testDualClusters. Contributed by Jiayi Zhou.
2016-06-08 12:58:56 -07:00
be34e85e68
HDFS-10468. HDFS read ends up ignoring an interrupt. Contributed by Jing Zhao
2016-06-07 10:48:21 -07:00
e620530301
HDFS-10485. Fix findbugs warning in FSEditLog.java. (aajisaka)
2016-06-07 17:52:55 +09:00
6de9213df1
HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster.
2016-06-06 15:52:39 -07:00
35f255b03b
Revert "Revert "HADOOP-13226 Support async call retry and failover.""
...
This reverts commit 5360da8bd9
2016-06-06 16:31:43 +08:00
7e7b1ae037
Revert "Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.""
...
This reverts commit 8cf47d8589
2016-06-06 16:31:38 +08:00
db41e6d285
Revert "Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou""
...
This reverts commit 5ee5912ebd
2016-06-06 16:31:34 +08:00
b3d81f38da
Revert "Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit b82c74b910
2016-06-06 16:31:30 +08:00
cba9a01889
Revert "Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit f23d5dfc60
2016-06-06 16:29:38 +08:00
aa20fa150d
Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou""
...
This reverts commit 4d36b221a2
2016-06-06 16:28:47 +08:00
eded3d109e
Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit 106234d873
2016-06-06 16:28:21 +08:00
106234d873
Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit fc94810d3f
2016-06-03 18:09:55 -07:00
4d36b221a2
Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"
...
This reverts commit 1b9f18623a
2016-06-03 18:09:18 -07:00
f23d5dfc60
Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 7251bb922b
2016-06-03 18:09:17 -07:00
b82c74b910
Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 02d4e478a3
2016-06-03 18:09:14 -07:00
5ee5912ebd
Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou"
...
This reverts commit f4b9bcd87c
2016-06-03 18:09:13 -07:00
8cf47d8589
Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou."
...
This reverts commit 21890c4239
2016-06-03 18:09:12 -07:00
5360da8bd9
Revert "HADOOP-13226 Support async call retry and failover."
...
This reverts commit 83f2f78c11
2016-06-03 18:09:09 -07:00
99a771cd7a
HDFS-7767. Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI (Ravi Prakash via aw)
2016-06-03 17:07:39 -07:00
15f018434c
HDFS-9877. HDFS Namenode UI: Fix browsing directories that need to be encoded (Ravi Prakash via aw)
2016-06-03 17:06:29 -07:00
97e2449477
HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (Contributed by Akira Ajisaka)
2016-06-02 13:14:45 -07:00
1df6f5735c
HDFS-10471. DFSAdmin#SetQuotaCommand's help msg is not correct. Contributed by Yiqun Lin.
2016-06-03 04:10:32 +09:00
aadb77e412
HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula.
2016-06-02 20:55:04 +09:00
69555fca06
HDFS-9476. TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail. Contributed by Masatake Iwasaki.
2016-06-02 18:52:47 +09:00
d749cf65e1
HDFS-9833. Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data. Contributed by Rakesh R.
2016-06-02 12:56:21 +08:00
8ceb06e239
HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:54:34 -07:00
c7921c9bdd
HDFS-9466. TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:37:52 -07:00
29d6cadc52
HDFS-10415. TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called (Mingliang Liu via cmccabe)
2016-05-31 16:54:52 -07:00
83f2f78c11
HADOOP-13226 Support async call retry and failover.
2016-05-31 16:30:11 -07:00
93d8a7f2a2
Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng"
...
This reverts commit a6c79f92d5
2016-06-01 08:41:15 +08:00
21890c4239
HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.
2016-05-27 14:20:18 -07:00
5ea6fd85c7
HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang)
2016-05-27 10:02:02 -07:00
916140604f
HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li
2016-05-28 14:23:39 +08:00
8c84a2a93c
HDFS-10236. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-3]. Contributed by Rakesh R.
2016-05-26 16:50:55 -07:00
55c3e2de3d
HDFS-10463. TestRollingFileSystemSinkWithHdfs needs some cleanup. (Daniel Templeton via kasha)
2016-05-26 14:24:31 -07:00
f4b9bcd87c
HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou
2016-05-26 12:15:17 -07:00
77202fa103
HADOOP-13010. Refactor raw erasure coders. Contributed by Kai Zheng
2016-05-27 13:23:34 +08:00
f69f5ab3b6
HDFS-10434. Fix intermittent test failure of TestDataNodeErasureCodingMetrics. Contributed by Rakesh R
2016-05-27 04:42:53 +08:00
a6c79f92d5
HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng
2016-05-27 04:29:42 +08:00
2dab64da2f
HDFS-10217. show 'blockScheduled' tooltip in datanodes table.
2016-05-25 12:34:52 +05:30
02d4e478a3
HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-24 14:51:27 -07:00
15ed080e36
HDFS-9365. Balaner does not work with the HDFS-6376 HA setup.
2016-05-24 12:49:48 -07:00
57c31a3fef
HDFS-9782. RollingFileSystemSink should have configurable roll interval. (Daniel Templeton via kasha)
2016-05-24 10:37:00 -07:00
4b0f55b6ea
HDFS-7766. Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect (Ravi Prakash via aw)
2016-05-23 15:52:01 -07:00
12eb5a7b11
HDFS-10439. Update setOwner doc in HdfsPermissionsGuide. Contributed by John Zhuge.
2016-05-22 16:53:30 +09:00
54f7975dfc
HDFS-10420. Fix Ozone unit tests to use MiniOzoneCluster. Contributed by Arpit Agarwal.
2016-05-20 15:19:09 -07:00
500e946729
HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth.
2016-05-20 13:01:43 -07:00
bcde1562d2
HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth.
2016-05-20 12:47:41 -07:00
204de430bf
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-19 15:52:32 -05:00
76279f5877
Merge branch 'trunk' into HDFS-7240
2016-05-19 12:44:16 -07:00
4770ae7e33
HDFS-10363. Ozone: Introduce new config keys for SCM service endpoints. Contributed by Arpit Agarwal.
2016-05-19 11:02:40 -07:00
010e6ac328
HDFS-2173. saveNamespace should not throw IOE when only one storage directory fails to write VERSION file. Contributed by Andras Bokor.
2016-05-18 14:13:41 -07:00
cf552aa87b
HDFS-10360. DataNode may format directory and lose blocks if current/VERSION is missing. (Wei-Chiu Chuang via lei)
2016-05-18 09:38:37 -07:00
f4d8fde822
HDFS-9926. MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils. (Contributed by Josh Elser)
2016-05-18 09:20:57 -07:00
dd99f5fc79
HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu.
2016-05-17 16:12:00 -07:00
f6ef876fe1
MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen.
2016-05-17 14:41:47 -07:00
e24fe2641b
HDFS-9732, Improve DelegationTokenIdentifier.toString() for better logging. Contributed by Yongjun Zhang.
2016-05-17 14:03:08 -07:00
1356cbe994
Revert "HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu."
...
This reverts commit 0942954e8a
2016-05-17 15:20:03 -05:00
0942954e8a
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-17 15:06:25 -05:00
7cd5ae62f6
HDFS-10404. Fix formatting of CacheAdmin command usage help text (Yiqun Lin via cmccabe)
2016-05-17 11:10:48 -07:00
4a5819dae2
HDFS-10303. DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. Contributed by Surendra Singh Lilhore.
2016-05-17 08:53:31 -05:00
9fe5828f05
HDFS-10242. Cannot create space quota of zero. Contributed by Takashi Ohnishi.
2016-05-17 15:53:09 +09:00
61f46be071
HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee)
2016-05-16 18:49:47 -07:00
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
6a6e74acf5
HDFS-10410. RedundantEditLogInputStream.LOG is set to wrong class. (John Zhuge via lei)
2016-05-16 17:05:46 -07:00
ad9441122f
HDFS-8449. Add tasks count metrics to datanode for ECWorker. Contributed by Bo Li.
2016-05-15 23:39:09 -07:00
45788204ae
HDFS-10333. Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk. Contributed by Yiqun Lin.
2016-05-15 22:15:05 -07:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
a2774debf7
HDFS-9389. Add maintenance states to AdminStates. (Ming Ma via lei)
2016-05-12 15:42:54 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
7251bb922b
HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-11 17:53:40 -07:00
687233f20d
HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe)
2016-05-11 13:45:39 -07:00
307db61aac
HDFS-10172. hdfs erasurecode command should remove the redundant -usage option. Contributed by Yuanbo Liu.
2016-05-10 15:20:17 -07:00
f0252ad252
HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe)
2016-05-10 11:15:05 -07:00
b9e5a32fa1
HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah.
2016-05-09 15:50:09 -05:00
411fb4b2b7
HDFS-10337. OfflineEditsViewer stats option should print 0 instead of null for the count of operations. Contributed by Yiqun Lin.
2016-05-08 15:49:13 -07:00
6957e45699
HDFS-10374. typo in HDFSHighAvailabilityWithQJM doc. Contributed by Takashi Ohnishi.
2016-05-07 11:26:42 +09:00
2835f146b5
HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun.
2016-05-06 15:36:47 +09:00
dacd1f50fe
HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang.
2016-05-05 18:08:47 -07:00
63a6a1d732
HDFS-10361. Support starting StorageContainerManager as a daemon. (Contributed by Arpit Agarwal)
2016-05-05 15:09:59 -07:00
d8faf47f32
HADOOP-13051. Test for special characters in path being respected during globPaths. Contributed by Harsh J.
2016-05-05 14:18:39 -07:00
1268cf5fbe
HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma)
2016-05-04 17:02:26 -07:00
78bd1b2ab2
Merge trunk into HDFS-7240
2016-05-04 10:01:39 -07:00
6d77d6eab7
HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. (Contributed by Brahma Reddy Battula)
2016-05-03 16:52:43 -07:00
45a753ccf7
HDFS-10344. DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash. Contributed by Xiaoyu Yao.
2016-05-02 19:30:47 -07:00
05d7a838ee
Merge branch 'trunk' into HDFS-7240
2016-05-02 14:55:23 -07:00
183175e60f
HDFS-10351. Ozone: Optimize key writes to chunks by providing a bulk write implementation in ChunkOutputStream. Contributed by Chris Nauroth.
2016-05-02 14:50:40 -07:00
1b9f18623a
HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou
2016-05-02 11:16:30 -07:00
2753185a01
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-29 22:32:27 -07:00
8264b61656
HDFS-10349. Ozone: StorageContainerManager fails to compile after merge of HDFS-10312 maxDataLength enforcement. Contributed by Chris Nauroth.
2016-04-29 20:44:38 -07:00
7da540d03e
HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah.
2016-04-29 15:50:43 -05:00
3dcbbdc1d1
Merge trunk into HDFS-7240
2016-04-29 13:18:40 -07:00
4da6f69ca1
HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu
2016-04-29 11:22:33 -07:00
af9b000535
HDFS-10260. TestFsDatasetImpl#testCleanShutdownOfVolume often fails. Contributed by Rushabh Shah.
2016-04-29 09:58:44 -05:00
6243eabb48
HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla
2016-04-28 12:44:53 -05:00
0a152103f1
HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu)
2016-04-27 14:22:51 -07:00
919a1d824a
HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla.
2016-04-27 08:19:48 -05:00
fc94810d3f
HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-04-26 17:10:13 -07:00
6be22ddbf1
HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge.
2016-04-26 13:04:18 -07:00
c6a2430b47
HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun.
2016-04-26 11:45:29 -05:00
5865fe2bf0
HDFS-9869. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-2]. Contributed by Rakesh R.
2016-04-25 22:01:54 -07:00
8eadd7145e
HDFS-10325. TestDFSShell#testMoveWithTargetPortEmpty fails. Contributed by Takanobu Asanuma.
2016-04-26 07:11:13 +09:00
f308561f1d
HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma
2016-04-25 12:01:48 -07:00
10f0f7851a
HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor.
2016-04-25 11:38:14 -05:00
6fcde2e38d
HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims)
2016-04-23 23:37:56 +09:00
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
d6402faded
Revert "HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano"
...
This reverts commit 4838b735f0
2016-04-22 10:17:20 -07:00
5d4255a801
HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang.
2016-04-22 11:08:44 -05:00
14ab7a81e2
HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand
2016-04-21 15:41:50 -07:00
bbce1d525e
HDFS-10319. Balancer should not try to pair storages with different types.
2016-04-21 15:20:12 -07:00
4838b735f0
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-21 11:32:39 -07:00
b4be288c5d
HDFS-10207. Support enable Hadoop IPC backoff without namenode restart. Contributed by Xiaobing Zhou.
2016-04-21 10:18:48 -07:00
5bd7b592e5
HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao.
2016-04-20 23:10:52 -07:00
63ac2db59a
HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth.
2016-04-20 13:39:44 -07:00
ad36fa6f42
HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe)
2016-04-20 11:33:11 -07:00
af9bdbe447
HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou)
2016-04-19 11:26:30 -07:00
90d1af1416
HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun.
2016-04-19 18:54:11 +08:00
4b4df2cd48
HDFS-10284. o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently. Contributed by Mingliang Liu.
2016-04-19 18:07:46 +08:00
438efc3634
HDFS-10291 TestShortCircuitLocalRead failing (stevel)
...
(cherry picked from commit dd64769699
2016-04-19 17:48:20 +08:00
b5d4c7dc76
HADOOP-12924. Configure raw erasure coders for supported codecs. Contributed by Rui Li.
2016-04-20 15:47:01 +08:00
be0bce1b71
HDFS-10306. SafeModeMonitor should not leave safe mode if name system is starting active service. Contributed by Mingliang Liu.
2016-04-18 18:05:49 -07:00
c884170c93
Merge branch 'trunk' into HDFS-7240
2016-04-18 16:58:45 -07:00
ce524c584b
HDFS-10232. Ozone: Make config key naming consistent. Contributed by Anu Engineer.
2016-04-18 16:52:40 -07:00
cb3ca460ef
HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe)
2016-04-18 13:47:56 -07:00
d8b729e16f
HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun.
2016-04-18 07:58:55 -05:00
ab903029a9
HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun.
2016-04-18 20:29:29 +08:00
67523ffcf4
HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi.
2016-04-18 09:28:02 +08:00
55e19b7f0c
HDFS-10293. StripedFileTestUtil#readAll flaky. Contributed by Mingliang Liu.
2016-04-15 10:53:40 -07:00
89a838769f
HDFS-10283. o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently. Contributed by Mingliang Liu.
2016-04-15 10:49:21 -07:00
b9c9d03591
HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu.
2016-04-14 15:24:39 -05:00
809226752d
HDFS-10286. Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties. Contributed by Xiaobing Zhou.
2016-04-14 10:57:02 -07:00
c970f1d005
HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang.
2016-04-14 12:45:47 -05:00
0d1c1152f1
HDFS-10282. The VolumeScanner should warn about replica files which are misplaced. Contributed by Colin Patrick McCabe.
2016-04-14 07:58:24 -05:00
ae6c0e839b
Merge branch 'trunk' into HDFS-7240
2016-04-13 23:13:11 -07:00
5566177c9a
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode. Contributed by Xiaobing Zhou.
2016-04-13 16:51:13 -07:00
4895c73dd4
Revert "HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode"
...
This reverts commit 192112d5a2
2016-04-13 16:45:18 -07:00
314aa21a89
HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun.
2016-04-13 16:39:50 -07:00
192112d5a2
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode
2016-04-13 15:34:41 -07:00
d2f3bbc290
HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák
2016-04-13 11:22:36 -05:00
903428bf94
HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun.
2016-04-13 17:49:45 +08:00
eee7269748
HDFS-10278. Ozone: Add paging support to list Volumes. Contributed by Anu Engineer.
2016-04-12 18:08:48 -07:00
33b51da8c5
HDFS-10251. Ozone: Shutdown cleanly. Contributed by Anu Engineer
2016-04-12 18:00:24 -07:00
35f0770555
HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe)
2016-04-12 16:28:42 -07:00
6ef42873a0
HDFS-9918. Erasure Coding: Sort located striped blocks based on decommissioned states. Contributed by Rakesh R.
2016-04-12 13:38:58 -07:00
600d129bb8
HDFS-10273. Remove duplicate logSync() and log message in FSN#enterSafemode() (Vinayakumar B via cmccabe)
2016-04-12 10:52:39 -07:00
209303be3a
HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang.
2016-04-12 15:26:50 +09:00
a9a607f8fc
HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula)
2016-04-12 11:44:34 +05:30
9c32f8785e
HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang.
2016-04-08 15:32:13 +09:00
3f62ba558d
Merge branch 'trunk' into HDFS-7240
2016-04-07 14:43:39 -07:00
fedb22d9b6
HDFS-10268. Ozone: end-to-end integration for create/get volumes, buckets and keys. Contributed by Chris Nauroth.
2016-04-07 14:38:54 -07:00
3c18a53cbd
HDFS-9719. Refactoring ErasureCodingWorker into smaller reusable constructs. Contributed by Kai Zheng.
2016-04-06 22:50:24 -07:00
654cd1d0c0
HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R
2016-04-07 12:38:42 +08:00
4bd7cbc29d
HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose
2016-04-06 21:07:31 -07:00
aede8c10ec
HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee
2016-04-06 20:20:14 +00:00
188f65287d
HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe)
2016-04-06 11:28:34 -07:00
221b3a8722
HDFS-10192. Namenode safemode not coming out during failover. Contributed by Brahma Reddy Battula.
2016-04-06 10:42:59 -07:00
9ba1e5af06
HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger.
2016-04-05 16:26:18 -05:00
0cd320a846
HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula.
2016-04-05 13:41:19 -07:00
b3044db407
HDFS-10250. Ozone: Add key Persistence. Contributed by Anu Engineer.
2016-04-05 12:59:45 -07:00
917464505c
HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla.
2016-04-05 09:07:24 -05:00
818d6b799e
HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula)
2016-04-05 09:49:39 +08:00
f6b1a81812
HDFS-8496. Calling stopWriter() with FSDatasetImpl lock held may block other threads (cmccabe)
2016-04-04 18:02:15 -07:00
a7d1fb0cd2
HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee.
2016-04-04 16:40:00 -05:00
154d2532cf
HDFS-9599. TestDecommissioningStatus.testDecommissionStatus occasionally fails (Lin Yiqun via iwasakims)
2016-04-05 03:19:48 +09:00
54b2e78fd2
HDFS-10253. Fix TestRefreshCallQueue failure (Contributed by Xiaoyu Yao)
2016-04-03 13:27:49 +05:30
4ac97b1815
HDFS-10238. Ozone : Add chunk persistance. Contributed by Anu Engineer.
2016-04-01 10:52:38 -07:00
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
32c0c3ecdf
HDFS-10221. Add .json to the rat exclusions. Contributed by Ming Ma.
2016-03-31 09:06:14 +09:00
55d4ee71e2
HDFS-8210. Ozone: Implement storage container manager. Contributed by Chris Nauroth.
2016-03-30 14:49:55 -07:00
37e23ce45c
HDFS-10223. peerFromSocketAndKey performs SASL exchange before setting connection timeouts (cmccabe)
2016-03-30 13:37:37 -07:00
09d63d5a19
HDFS-5177. blocksScheduled count should be decremented for abandoned blocks (Contributed by Vinayakumar B)
2016-03-30 14:22:00 +08:00
ddfe6774c2
HDFS-9439. Support reconfiguring fs.protected.directories without NN restart. (Contributed by Xiaobing Zhou)
2016-03-29 13:55:00 -07:00
c9307e48b7
HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA)
2016-03-29 13:50:08 -07:00
f2aec4eb82
HDFS-10197. TestFsDatasetCache failing intermittently due to timeout. Contributed by Lin Yiqun.
2016-03-29 11:54:42 -07:00
1f004b3367
HDFS-9871. "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. (Contributed by Brahma Reddy Battulla)
2016-03-29 11:25:41 +08:00
30473ecfb4
Merge branch 'trunk' into HDFS-7240
2016-03-28 10:34:52 -07:00
3a4ff7776e
HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng
2016-03-26 19:58:09 -07:00
643c5e5bdc
HDFS-10195. Ozone: Add container persistence. Contributed by Anu Engineer.
2016-03-26 11:40:00 -07:00
Zhe Zhang
Anu Engineer
Arpit Agarwal
Tsz-Wo Nicholas Sze
Xiaoyu Yao
Kai Zheng
Uma Maheswara Rao G
Akira Ajisaka
Brahma Reddy Battula
Colin Patrick Mccabe
Jitendra Pandey
Kihwal Lee
Vinayakumar B
Ming Ma
Andrew Wang
Allen Wittenauer
Ravi Prakash
Jing Zhao
Masatake Iwasaki
Chris Nauroth
Yongjun Zhang
Karthik Kambatla
Karthik Kambatla
Lei Xu
Junping Du
Sangjin Lee
Ravi Prakash
Walter Su
Steve Loughran
Xiaobing Zhou
Eric Payne