58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
cc01ed7026
HDFS-10855. Fix typos in HDFS documents. Contributed by Yiqun Lin.
2016-09-10 23:24:49 -07:00
a99bf26a08
HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy)
2016-09-10 18:22:25 -07:00
bee9f57f5c
HDFS-10808. DiskBalancer does not execute multi-steps plan-redux. Contributed by Anu Engineer.
2016-09-09 15:00:39 -07:00
cba973f036
HDFS-10832. Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissions.
2016-09-09 12:12:29 -07:00
a0b0383677
Addendum patch for HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-09 10:54:14 -07:00
baab48922a
HDFS-9849. DiskBalancer: reduce lock path in shutdown code. Contributed by Yuanbo Liu.
2016-09-08 20:00:42 -07:00
35c5943b8b
HDFS-10553. DiskBalancer: Rename Tools/DiskBalancer class to Tools/DiskBalancerCLI. Contributed by Manoj Govindassamy.
2016-09-08 19:26:56 -07:00
b07c266dca
HDFS-10831. Add log when URLConnectionFactory.openConnection failed. Contributed by yunjiong zhao.
2016-09-08 18:30:18 -07:00
011f3b24d4
HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang
2016-09-08 17:53:40 -07:00
b6d839a60c
HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin
2016-09-08 13:29:30 -07:00
401db4fc65
HDFS-8901. Use ByteBuffer in striping positional read. Contributed by Sammi Chen and Kai Zheng.
2016-09-08 11:54:33 -07:00
20a20c2f6e
HDFS-10847. Complete the document for FileDistribution processor in OfflineImageViewer. Contributed by Yiqun Lin.
2016-09-09 01:34:34 +09:00
2d1bf53c7e
HDFS-10844. test_libhdfs_threaded_hdfs_static and test_libhdfs_zerocopy_hdfs_static are failing.
2016-09-09 00:49:22 +09:00
63f594892e
HDFS-10778. Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer.
2016-09-08 15:13:43 +09:00
d355573f56
Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng"
...
This reverts commit 62a9667136
2016-09-08 05:50:17 +08:00
5f23abfa30
HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula)
2016-09-06 13:37:21 -07:00
f6c0b7543f
HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee.
2016-09-06 11:48:35 -07:00
7c213c341d
HDFS-10835. Fix typos in httpfs.sh. Contributed by John Zhuge.
2016-09-06 11:44:26 -07:00
f0d5382ff3
HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth.
2016-09-06 11:02:39 -07:00
d37dc5d1b8
HDFS-9847. HDFS configuration should accept time units. Contributed by Yiqun Lin
2016-09-06 10:38:04 -07:00
62a9667136
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-07 17:05:33 +08:00
07650bc37a
HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-02 15:33:11 -07:00
cbd909ce2a
HDFS-10833. Fix JSON errors in WebHDFS.md examples.
2016-09-02 14:50:34 -07:00
5a8c5064d1
HDFS-10822. Log DataNodes in the write pipeline. John Zhuge via Lei Xu
2016-09-02 11:01:13 -07:00
0690f0969e
HDFS-10820. Reuse closeResponder to reset the response variable in DataStreamer#run. Contributed by Yiqun Lin.
2016-09-01 15:35:25 -07:00
6c600360ca
Addendum fix for HDFS-10817 to fix failure of the added testFSReadLockLongHoldingReport in branch-2.
2016-08-31 23:43:59 -07:00
6f4b0d33ca
HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen.
2016-08-31 15:40:09 -07:00
85bab5fb57
HDFS-10784. Implement WebHdfsFileSystem#listStatusIterator.
2016-08-31 14:29:37 -07:00
01721dd88e
HDFS-10729. Improve log message for edit loading failures caused by FS limit checks. Contributed by Wei-Chiu Chuang.
2016-08-31 14:02:37 -05:00
20ae1fa259
HDFS-10813. DiskBalancer: Add the getNodeList method in Command. Contributed by Yiqun Lin.
2016-08-30 18:42:55 -07:00
9dcbdbdb5a
HDFS-9392. Admins support for maintenance state. Contributed by Ming Ma.
2016-08-30 14:00:13 -07:00
c4ee6915a1
HDFS-10760. DataXceiver#run() should not log InvalidToken exception as an error. Contributed by Pan Yuxuan.
2016-08-30 10:43:20 -07:00
4bd45f54ee
HDFS-10814. Add assertion for getNumEncryptionZones when no EZ is created. Contributed by Vinitha Reddy Gankidi.
2016-08-29 23:37:26 -07:00
cd5e10ccca
HDFS-4210. Throw helpful exception when DNS entry for JournalNode cannot be resolved. Contributed by Charles Lamb and John Zhuge.
2016-08-29 17:48:08 -07:00
5d1609ddf2
HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin.
2016-08-29 13:59:54 -07:00
6742fb6e68
HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu)
2016-08-29 10:15:34 -07:00
c25817159a
HDFS-10652. Add a unit test for HDFS-4660. Contributed by Vinayakumar B., Wei-Chiu Chuang, Yongjun Zhang.
2016-08-27 22:51:31 -07:00
e806db7190
HDFS-10584. Allow long-running Mover tool to login with keytab. Contributed by Rakesh R.
2016-08-26 16:43:45 -07:00
a445b82baa
HDFS-10793. Fix HdfsAuditLogger binary incompatibility introduced by HDFS-9184. Contributed by Manoj Govindassamy.
2016-08-26 15:39:18 -07:00
407b519fb1
HDFS-10798. Make the threshold of reporting FSNamesystem lock contention configurable. Contributed by Erik Krogen.
2016-08-26 14:19:55 -07:00
8b7adf4ddf
HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp.
2016-08-26 15:39:56 -05:00
13fb1b50e6
HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki.
2016-08-26 09:53:10 -05:00
f4a21d3aba
HDFS-10795. Fix an error in ReaderStrategy#ByteBufferStrategy. Contributed by Sammi Chen
2016-08-27 10:54:25 +08:00
4da5000dd3
HDFS-10748. TestFileTruncate#testTruncateWithDataNodesRestart runs sometimes timeout. Contributed by Yiqun Lin.
2016-08-25 09:00:44 -07:00
a1f3293762
HDFS-10772. Reduce byte/string conversions for get listing. Contributed by Daryn Sharp.
2016-08-24 15:21:08 -05:00
793447f799
HDFS-8905. Refactor DFSInputStream#ReaderStrategy. Contributed by Kai Zheng and Sammi Chen
2016-08-24 21:57:23 +08:00
ec252ce0fc
HDFS-10762. Pass IIP for file status related methods
2016-08-24 08:46:47 -05:00
126d165efd
HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen
2016-08-23 14:46:58 +00:00
f0efea490e
HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen.
2016-08-23 04:14:24 -07:00
e90f3359de
HDFS-10783. The option '-maxSize' and '-step' fail in OfflineImageViewer. Contributed by Yiqun Lin.
2016-08-23 19:57:23 +09:00
c49333becf
HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang)
2016-08-22 18:29:56 -07:00
3ca4d6ddfd
Revert "HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp."
...
This reverts commit 22fc46d765
2016-08-22 16:57:45 -05:00
22fc46d765
HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp.
2016-08-22 15:37:02 -05:00
dc7a1c54f4
HDFS-10692. Update JDiff report's base version for HDFS from 2.6.0 to 2.7.2. Contributed by Wangda Tan.
2016-08-22 11:35:18 -07:00
0faee62a0c
HDFS-10764. Fix INodeFile#getBlocks to not return null. Contributed by Arpit Agarwal.
2016-08-19 22:13:36 -07:00
03a9343d57
HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen.
2016-08-19 09:22:49 -07:00
2550371f66
HDFS-10711. Optimize FSPermissionChecker group membership check. Contributed by Daryn Sharp.
2016-08-19 09:12:17 -05:00
8179f9a493
HDFS-10645. Make block report size as a metric and add this metric to datanode web ui. Contributed by Yuanbo Liu.
2016-08-19 16:15:00 +09:00
ca13e7971d
HDFS-10773. BlockSender should not synchronize on the dataset object. (Contributed by Chen Liang)
2016-08-17 16:29:08 -07:00
2aa5e2c403
HDFS-10549. Correctly revoke file leases when closing files. Contributed by Yiqun Lin.
2016-08-17 15:52:38 -07:00
869393643d
HDFS-10745. Directly resolve paths into INodesInPath. Contributed by Daryn Sharp.
2016-08-17 15:53:03 -05:00
b047bc7270
HDFS-10560. DiskBalancer: Reuse ObjectMapper instance to improve the performance. Contributed by Yiqun Lin.
2016-08-16 10:20:08 -07:00
4d4d95fdd5
HDFS-10641. TestBlockManager#testBlockReportQueueing fails intermittently. (Contributed by Daryn Sharp)
2016-08-15 20:28:40 -07:00
ef55fe1716
HDFS-10747. o.a.h.hdfs.tools.DebugAdmin usage message is misleading. (Contributed by Mingliang Liu)
2016-08-15 20:23:47 -07:00
382d615260
HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu)
2016-08-15 20:22:14 -07:00
5628b36c08
HDFS-10559. DiskBalancer: Use SHA1 for Plan ID. Contributed by Xiaobing Zhou.
2016-08-15 20:10:21 -07:00
02abd131b8
HDFS-10567. Improve plan command help message. Contributed by Xiaobing Zhou.
2016-08-15 19:58:57 -07:00
864f878d59
HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee.
2016-08-15 17:28:09 -05:00
03dea65e0b
HDFS-10744. Internally optimize path component resolution. Contributed by Daryn Sharp.
2016-08-15 16:45:44 -05:00
bed69d18e6
HDFS-10580. DiskBalancer: Make use of unused methods in GreedyPlanner to print debug info. Contributed by Yiqun Lin
2016-08-15 12:40:29 -07:00
83e57e083f
HDFS-9696. Garbage snapshot records linger forever. Contributed by Kihwal Lee
2016-08-15 13:01:23 -05:00
9f29f423e4
HDFS-10737. disk balancer add volume path to report command. Contributed by Yuanbo Liu.
2016-08-15 09:47:30 -07:00
9019606b69
HDFS-10731. FSDirectory#verifyMaxDirItems does not log path name. Contributed by Wei-Chiu Chuang.
2016-08-12 05:58:13 -07:00
b5af9be72c
HDFS-8668. Erasure Coding: revisit buffer used for encoding and decoding. Contributed by Sammi Chen
2016-08-13 13:52:37 +08:00
a428d4f50e
HDFS-8897. Balancer should handle fs.defaultFS trailing slash in HA. Contributed by John Zhuge.
2016-08-10 22:17:35 -07:00
e7e8aed208
HDFS-10720. Fix intermittent test failure of TestDataNodeErasureCodingMetrics. Contributed by Rakesh R
2016-08-12 11:15:05 +08:00
ec289bbece
HDFS-10643. Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao.
2016-08-10 15:49:36 -07:00
fdafd0cb4f
Revert "Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao."
...
This reverts commit 9f816d7ca7
2016-08-10 15:47:55 -07:00
9f816d7ca7
Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao.
2016-08-10 15:14:38 -07:00
10e84c6a6e
HDFS-10694. processReport() should print blockReportId in each log message. Contributed by Yuanbo Liu.
2016-08-10 11:07:42 -07:00
d00d3add9e
HDFS-8224. Schedule a block for scanning if its metadata file is corrupt. Contributed by Rushabh S Shah.
2016-08-09 18:11:34 -07:00
9c6a4383ca
HDFS-10681. DiskBalancer: query command should report Plan file path apart from PlanID. (Manoj Govindassamy via lei)
2016-08-09 15:59:22 -07:00
cc48251bfd
HDFS-10457. DataNode should not auto-format block pool directory if VERSION is missing. (Wei-Chiu Chuang via lei)
2016-08-09 15:51:19 -07:00
0f701f433d
HDFS-10738. Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure. Contributed by Rakesh R.
2016-08-09 14:54:11 -05:00
b10c936020
HDFS-10342. BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt. Contributed by Kuhu Shukla.
2016-08-09 11:56:21 -05:00
8c0638471f
HADOOP-10682. Replace FsDatasetImpl object lock with a separate lock object. (Chen Liang)
2016-08-08 12:02:53 -07:00
4d3af47f27
HDFS-10693. metaSave should print blocks, not LightWeightHashSet. Contributed by Yuanbo Liu.
2016-08-07 14:29:26 -07:00
ef432579a7
HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu.
2016-08-05 17:31:11 +09:00
18d9e6ec0b
HDFS-10588. False alarm in datanode log - ERROR - Disk Balancer is not enabled. Contributed by Weiwei Yang.
2016-08-04 19:40:33 -07:00
438a9f047e
HDFS-10673. Optimize FSPermissionChecker's internal path usage. Contributed by Daryn Sharp.
2016-08-04 16:14:13 -05:00
57369a678c
HDFS-10343. BlockManager#createLocatedBlocks may return blocks on failed storages. Contributed by Kuhu Shukla.
2016-08-04 14:55:21 -05:00
4a8e6dc02f
HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp.
2016-08-04 11:53:44 -05:00
8c9b44eaff
In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin
2016-08-04 09:45:40 -07:00
6ae39199da
HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp.
2016-08-04 09:07:34 -05:00
6f63566694
HDFS-10569. A bug causes OutOfIndex error in BlockListAsLongs. Contributed by Weiwei Yang.
2016-08-03 15:20:20 -05:00
f4ba5ff1d7
HDFS-10710. In BlockManager#rescanPostponedMisreplicatedBlocks(), postponed misreplicated block counts should be retrieved within the NN lock protection.Contributed by GAO Rui.
2016-08-03 12:42:05 -07:00
22ef5286bc
HDFS-10674. Optimize creating a full path from an inode. Contributed by Daryn Sharp.
2016-08-03 13:12:43 -05:00
58db263e93
HDFS-742. A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list. Contributed by Mit Desai.
2016-08-03 12:14:59 -05:00
bebf10d245
HDFS-10656. Optimize conversion of byte arrays back to path string. Contributed by Daryn Sharp.
2016-08-03 11:53:41 -05:00
5e5b8793fb
HDFS-5805. TestCheckpoint.testCheckpoint fails intermittently on branch-2. Contributed by Eric Badger.
2016-08-02 15:37:36 -05:00
a5fb298e56
HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon.
2016-08-02 17:07:59 +09:00
c4463f2ef2
Revert "HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi."
...
This reverts commit 85a20508bd
2016-08-01 22:34:40 -07:00
9f473cf903
HDFS-10655. Fix path related byte array conversion bugs. (daryn)
2016-08-01 10:14:28 -05:00
8d32bd884d
HDFS-10703. HA NameNode Web UI should show last checkpoint time. (John Zhuge via Yongjun Zhang)
2016-07-29 23:20:14 -07:00
204a2055b1
HDFS-10691. FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException. Contributed by Yiqun Lin.
2016-07-29 15:39:48 +09:00
d9aae22fdf
HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge
2016-07-28 16:29:40 -07:00
ce3d68e9c3
HDFS-10676. Add namenode metric to measure time spent in generating EDEKs. Contributed by Hanisha Koneru.
2016-07-28 16:07:45 -07:00
a3d0cba810
HDFS-10689. Hdfs dfs chmod should reset sticky bit permission when the bit is omitted in the octal mode. (Manoj Govindassamy via Lei Xu)
2016-07-28 13:52:14 -07:00
67406460f0
HDFS-4176. EditLogTailer should call rollEdits with a timeout. (Lei Xu)
2016-07-28 13:32:00 -07:00
328c855a57
HDFS-10650. DFSClient#mkdirs and DFSClient#primitiveMkdir should use default directory permission. Contributed by John Zhuge.
2016-07-28 13:15:02 -07:00
414fbfab41
HDFS-10696. TestHDFSCLI fails. Contributed by Kai Sasaki.
2016-07-28 16:58:23 +09:00
098ec2b11f
HDFS-10519. Add a configuration option to enable in-progress edit log tailing. Contributed by Jiayi Zhou.
2016-07-27 17:55:41 -07:00
eb7ff0c992
HDFS-10667. Report more accurate info about data corruption location. (Yuanbo Liu via Yongjun Zhang)
2016-07-27 13:58:04 -07:00
54fe17a607
HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák.
2016-07-27 19:41:09 +09:00
55d5993a8e
HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu.
2016-07-27 10:26:42 +05:30
d84ab8a578
HDFS-10598. DiskBalancer does not execute multi-steps plan. Contributed by Lei (Eddy) Xu.
2016-07-26 16:01:50 -07:00
d2cf8b54c5
HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu)
2016-07-26 12:27:46 -07:00
25f0a9b3ab
HDFS-9937. Update dfsadmin command line help and HdfsQuotaAdminGuide. Contributed by Kai Sasaki.
2016-07-26 08:01:42 -07:00
2aa18e1925
Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu."
...
This reverts commit 7cac7655fd
2016-07-26 14:30:02 +01:00
7cac7655fd
HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu.
2016-07-26 10:54:24 +05:30
85a20508bd
HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi.
2016-07-25 18:50:59 -07:00
59466b8c18
HDFS-10671. Fix typo in HdfsRollingUpgrade.md. Contributed by Yiqun Lin.
2016-07-26 10:48:21 +09:00
0cde9e12a7
HDFS-10688. BPServiceActor may run into a tight loop for sending block report when hitting IOException. Contributed by Chen Liang.
2016-07-25 18:41:13 -07:00
b79ba4fe96
HDFS-10651. Clean up some configuration related codes about legacy block reader. Contributed by Youwei Wang
2016-07-23 17:45:33 +08:00
c6e3a0020f
HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R)
2016-07-22 12:33:50 -07:00
77ac04efe4
HDFS-10565: Erasure Coding: Document about the current allowed storage policies for EC Striped mode files
2016-07-22 11:08:20 -07:00
132deb4cac
HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu.
2016-07-21 16:41:02 -07:00
fcde6940e0
HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor.
2016-07-21 11:26:08 -07:00
bd3dcf46e2
HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp.
2016-07-21 11:14:39 -07:00
38128baff4
HDFS-10425. Clean up NNStorage and TestSaveNamespace. Contributed by Andras Bokor.
2016-07-20 10:44:11 -07:00
f7dabe3add
HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg.
2016-07-19 15:13:01 -07:00
fc570b55b9
HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin.
2016-07-19 13:49:24 -07:00
5b4a708704
HDFS-10639. Fix typos in HDFSDiskbalancer.md. Contributed by Yiqun Lin.
2016-07-15 21:41:57 -07:00
da456ffd62
Preparing for 3.0.0-alpha2 development
2016-07-15 19:04:17 -07:00
c2f9cd584c
HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu.
2016-07-15 14:28:53 -07:00
c48e9d608c
HDFS-10628. Log HDFS Balancer exit message to its own log. Contributed by Jiayi Zhou.
2016-07-15 11:42:44 -07:00
5537c6b234
HADOOP-13351. TestDFSClientSocketSize buffer size tests are flaky. Contributed by Aaron Fabbri and Mingliang Liu.
2016-07-14 14:40:58 -07:00
382dff7475
HDFS-10600. PlanCommand#getThrsholdPercentage should not use throughput value. (Yiqun Lin via lei)
2016-07-14 10:40:34 -07:00
1fa1fab695
HDFS-10570. Remove classpath conflicts of netty-all jar in hadoop-hdfs-client. Contributed by Vinayakumar B.
2016-07-14 10:20:18 +01:00
2bbc3ea1b5
HDFS-10617. PendingReconstructionBlocks.size() should be synchronized. Contributed by Eric Badger.
2016-07-13 21:59:35 -05:00
438b7c5935
HDFS-10590: Fix TestReconstructStripedBlocks.testCountLiveReplicas test failures. Contributed by Rakesh R
2016-07-13 00:07:11 -07:00
087290e6b1
HDFS-10544. Balancer doesn't work with IPFailoverProxyProvider.
2016-07-12 23:18:37 -07:00
d5f5a6475b
HDFS-10387. DataTransferProtocol#writeBlock missing some javadocs (John Zhuge via lei)
2016-07-12 16:16:03 -07:00
c447efebdb
HDFS-10579. HDFS web interfaces lack configs for X-FRAME-OPTIONS protection. Contributed by Anu Engineer.
2016-07-11 14:55:33 -07:00
da6f1b88dd
HDFS-10512. VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks. Contributed by Wei-Chiu Chuang and Yiqun Lin.
2016-07-08 19:40:44 -07:00
932aed64d7
HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin.
2016-07-08 14:39:11 -07:00
6e597600f7
HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu.
2016-07-07 14:58:19 -07:00
ab092c56c2
HDFS-10592:Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning. Contributed by Rakesh R
2016-07-07 13:52:35 -07:00
b8f93cd275
HDFS-10488. Update WebHDFS documentation regarding CREATE and MKDIR default permissions. Contributed by Wellington Chevreuil.
2016-07-07 10:36:55 -07:00
a3f93beaa1
HDFS-10582. Change deprecated configuration fs.checkpoint.dir to dfs.namenode.checkpoint.dir in HDFS Commands Doc. Contributed by Pan Yuxuan.
2016-07-06 17:51:42 -07:00
8b4b5259d5
HDFS-10589. Javadoc for HAState#HAState and HAState#setStateInternal contains non-existent params. Contributed by shenyinjie.
2016-07-04 07:15:04 -07:00
2e835d986e
HDFS-10572. Fix TestOfflineEditsViewer#testGenerated. Contributed by Surendra Singh Lilhore.
2016-07-03 21:11:56 -07:00
8b281bce85
HDFS-10548. Remove the long deprecated BlockReaderRemote. Contributed by Kai Zheng
2016-07-03 11:56:23 +08:00
aa42c7a6dd
HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu.
2016-07-01 15:34:03 -07:00
e4a2545620
HDFS-9805. Add server-side configuration for enabling TCP_NODELAY for DataTransferProtocol and default it to true (Gary Helmling via cmccabe)
2016-06-29 12:41:42 -07:00
991c946593
HADOOP-10568. Reuse ObjectMapper instance in CombinedHostsFileReader and CombinedHostsFileWriter. Contributed by Yiqun Lin.
2016-06-29 10:10:19 -07:00
c0829f4493
HDFS-6434. Default permission for creating file should be 644 for WebHdfs/HttpFS. Contributed by Wellington Chevreuil.
2016-06-28 13:55:26 -07:00
23c3ff85a9
HADOOP-13209. replace slaves with workers (John Smith via aw)
2016-06-28 05:53:03 -07:00
2a0082c51d
HDFS-10440. Improve DataNode web UI (Contributed by Weiwei Yang)
2016-06-28 16:49:39 +05:30
73615a789d
HDFS-10536. Standby NN can not trigger log roll after EditLogTailer thread failed 3 times in EditLogTailer.triggerActiveLogRoll method (Contributed by XingFeng Shen
2016-06-26 15:45:32 +05:30
bf74dbf80d
HDFS-7959. WebHdfs logging is missing on Datanode (Kihwal Lee via sjlee)
2016-06-24 14:44:15 -07:00
22be604fa3
Revert "HDFS-10534. NameNode WebUI should display DataNode usage rate with a certain percentile. Contributed by Kai Sasaki."
...
This reverts commit 0424056a77
2016-06-24 09:36:03 -07:00
0424056a77
HDFS-10534. NameNode WebUI should display DataNode usage rate with a certain percentile. Contributed by Kai Sasaki.
2016-06-24 08:58:29 -07:00
b2584bee45
HDFS-10571. TestDiskBalancerCommand#testPlanNode failed with IllegalArgumentException. Contributed by Xiaobing Zhou.
2016-06-23 18:27:02 -07:00
8a6e354122
HDFS-10552. DiskBalancer "-query" results in NPE if no plan for the node. Contributed by Anu Engineer.
2016-06-23 18:27:02 -07:00
e8de28181a
HDFS-10551. o.a.h.h.s.diskbalancer.command.Command does not actually verify options as expected. Contributed by Anu Engineer.
2016-06-23 18:27:02 -07:00
66fa34c839
HDFS-10557. Fix handling of the -fs Generic option. (Arpit Agarwal)
2016-06-23 18:27:01 -07:00
7b23ad1ef7
HDFS-10550. DiskBalancer: fix issue of order dependency in iteration in ReportCommand test. Contributed by Xiaobing Zhou.
2016-06-23 18:27:01 -07:00
c6ed54808d
HDFS-10547. DiskBalancer: fix whitespace issue in doc files. Contributed by Anu Engineer.
2016-06-23 18:27:01 -07:00
0774412e41
HDFS-10545. DiskBalancer: PlanCommand should use -fs instead of -uri to be consistent with other hdfs commands. Contributed by Anu Engineer.
2016-06-23 18:27:01 -07:00
5b8e1c26d7
HDFS-10541. Diskbalancer: When no actions in plan, error message says "Plan was generated more than 24 hours ago". Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
cb68e5b3bd
HDFS-10540. Diskbalancer: The CLI error message for disk balancer is not enabled is not clear. Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
3225c24e0e
HDFS-10520. DiskBalancer: Fix Checkstyle issues in test code. Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
7e2be5c4a0
HDFS-10518. DiskBalancer: Pretty-print json in Query command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
af11ab34d0
HDFS-10517. DiskBalancer: Support help command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
b502102bb1
HDFS-9461. DiskBalancer: Add Report Command. Contributed by Xiaobing Zhou.
2016-06-23 18:21:08 -07:00
121142cf95
HDFS-10501. DiskBalancer: Use the default datanode port if port is not provided. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
78a1032b71
HDFS-10500. Diskbalancer: Print out information when a plan is not generated. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
d2ff793ae4
HDFS-10496. DiskBalancer: ExecuteCommand checks planFile in a wrong way. Contributed by Lei (Eddy) Xu.
2016-06-23 18:21:08 -07:00
64ccb232cc
HDFS-10478. DiskBalancer: resolve volume path names. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
47dcb0f952
HDFS-10476. DiskBalancer: Plan command output directory should be a sub-directory. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
06a9799d84
HDFS-9547. DiskBalancer: Add user documentation. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
43eee50966
HDFS-10403. DiskBalancer: Add cancel command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
9e5fcb5e40
HDFS-10402. DiskBalancer: Add QueryStatus command. (Contributed by Anu Engineer)
2016-06-23 18:21:08 -07:00
5df2d2b8fd
HDFS-10399. DiskBalancer: Add JMX for DiskBalancer. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
1b39b283c7
HDFS-9546: DiskBalancer: Add Execute command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
75882ec0b0
HDFS-9545: DiskBalancer: Add Plan Command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
1594b472bb
HDFS-9543. DiskBalancer: Add Data mover. Contributed by Anu Engineer.
2016-06-23 18:20:24 -07:00
7820737cfa
HDFS-9735. DiskBalancer : Refactor moveBlockAcrossStorage to be used by disk balancer. Contributed by Anu Engineer.
2016-06-23 18:20:24 -07:00
050677077b
HDFS-9720. DiskBalancer : Add configuration parameters. Contributed by Anu Engineer.
2016-06-23 18:18:48 -07:00
6c606bf5c8
HDFS-9709. DiskBalancer : Add tests for disk balancer using a Mock Mover class. Contributed by Anu Engineer.
2016-06-23 18:18:48 -07:00
9be9703716
Fix a build break in HDFS-1312
2016-06-23 18:18:48 -07:00
ec601673a8
HDFS-10170. DiskBalancer: Force rebase diskbalancer branch. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
75a711a2d5
HDFS-9703. DiskBalancer: getBandwidth implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
918722bdd2
HDFS-9702. DiskBalancer: getVolumeMap implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
4b93ddae07
HDFS-8821. Stop tracking CHANGES.txt in the HDFS-1312 feature branch. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
747227e9de
HDFS-9817. Use SLF4J in new classes. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
3df0781aa7
HDFS-9856. Suppress Jenkins warning for sample JSON file. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
9847640603
HDFS-9683. DiskBalancer: Add cancelPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
Allen Wittenauer
Xiao Chen
Arpit Agarwal
Anu Engineer
Andrew Wang
Chris Douglas
Zhe Zhang
Akira Ajisaka
Kai Zheng
Chris Nauroth
Lei Xu
Kihwal Lee
Ming Ma
Wei-Chiu Chuang
Yongjun Zhang
Mingliang Liu
Xiaoyu Yao
Daryn Sharp
Jason Lowe
Eric Yang
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
Jing Zhao
Yuanbo Liu
Konstantin V Shvachko
Tsz-Wo Nicholas Sze
Brahma Reddy Battula
Steve Loughran
Vinitha Reddy Gankidi
Masatake Iwasaki
Uma Maheswara Rao G
Jitendra Pandey
Colin Patrick Mccabe
Vinayakumar B
Sangjin Lee