48b9d5fd2a
HDFS-10955. Pass IIP for FSDirAttr methods. Contributed by Daryn Sharp.
2016-10-06 16:33:46 -05:00
1d330fbaf6
HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth.
2016-10-06 13:19:16 -07:00
f32e9fc8f7
HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn sharp.
2016-10-06 15:11:14 -05:00
311954883f
HDFS-10957. Retire BKJM from trunk (Vinayakumar B)
2016-10-06 19:28:25 +05:30
b90fc70d67
HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen
2016-10-06 17:47:31 +05:30
202325485d
HDFS-10893. Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test. Contributed by Mingliang Liu
2016-10-05 22:33:29 -07:00
c5ca216915
HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge.
2016-10-05 17:36:50 -07:00
e68c7b96c7
HDFS-10962. TestRequestHedgingProxyProvider is flaky.
2016-10-05 15:08:56 -07:00
8867762256
HDFS-10826. Correctly report missing EC blocks in FSCK. Contributed by Takanobu Asanuma.
2016-10-05 10:52:50 -07:00
44f48ee96e
HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp.
2016-10-04 15:05:09 -05:00
382307cbdd
HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah.
2016-10-04 11:09:03 -05:00
ef7f06f7d1
HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore
2016-10-04 18:46:42 +05:30
5ea0210c7f
HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N
2016-10-04 18:33:22 +05:30
f61e3d13e9
HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger
2016-10-03 22:34:41 -04:00
8078a5efd0
HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula
2016-10-03 21:50:16 -04:00
853d65a157
HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen.
2016-10-03 16:01:54 -07:00
607705c488
HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu.
2016-10-03 10:53:21 -07:00
744208431f
HDFS-10940. Reduce performance penalty of block caching when not used. Contributed by Daryn Sharp.
2016-10-03 11:27:23 -05:00
90020624b0
HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp.
2016-10-03 09:13:04 -05:00
fe9ebe20ab
Revert "HDFS-10923. Make InstrumentedLock require ReentrantLock."
...
This reverts commit c7ce6fdc20
2016-09-30 23:11:51 -07:00
c7ce6fdc20
HDFS-10923. Make InstrumentedLock require ReentrantLock.
2016-09-30 23:00:06 -07:00
4217f85202
HDFS-10932. Ozone: fix XceiverClient slow shutdown. Contributed by Chen Liang.
2016-09-30 17:12:43 -07:00
434c5ea75d
HDFS-10896. Move lock logging logic from FSNamesystem into FSNamesystemLock. Contributed by Erik Krogen.
2016-09-30 13:16:08 -07:00
7fad1221d6
HDFS-10907. Fix Erasure Coding documentation. Contributed by Manoj Govindassamy.
2016-09-30 12:51:27 -07:00
ee33a02234
HDFS-10910. HDFS Erasure Coding doc should state its currently supported erasure coding policies. Contributed by Yiqun Lin.
2016-09-30 12:48:11 -07:00
2ab1ef15c5
HDFS-10908. Improve StripedBlockReader#createBlockReader error logging. Contributed by Manoj Govindassamy.
2016-09-30 12:35:47 -07:00
a0730aa5ce
HDFS-10851. FSDirStatAndListingOp: stop passing path as string. Contributed by Daryn Sharp.
2016-09-30 13:03:24 -05:00
0670149c88
HDFS-10850. getEZForPath should NOT throw FNF. Contributed by Andrew Wang.
2016-09-30 08:44:18 -05:00
595257e3f4
Merge branch 'trunk' into HDFS-7240
2016-09-29 11:12:53 -07:00
71c9b5be7b
HDFS-10897. Ozone: SCM: Add NodeManager. Contributed by Anu Engineer.
2016-09-29 10:36:25 -07:00
84c6264079
HDFS-10892. Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'. Contributed by Mingliang Liu
2016-09-28 16:03:51 -07:00
92e5e91598
HDFS-10914. Move remnants of oah.hdfs.client to hadoop-hdfs-client.
2016-09-28 16:01:03 -07:00
5f34402ada
HDFS-10779. Rename does not need to re-solve destination. Contributed by Daryn Sharp.
2016-09-28 17:57:23 -05:00
c3b235e565
HDFS-10824. MiniDFSCluster#storageCapacities has no effects on real capacity. Contributed by Xiaobing Zhou.
2016-09-28 11:47:37 -07:00
03f519a757
HDFS-9850. DiskBalancer: Explore removing references to FsVolumeSpi. Contributed by Manoj Govindassamy.
2016-09-27 21:35:06 -07:00
6437ba18c5
HDFS-10915. Fix time measurement bug in TestDatanodeRestart. Contributed by Xiaobing Zhou
2016-09-27 20:40:17 -07:00
9c9736463b
HDFS-10900. DiskBalancer: Complete the documents for the report command. Contributed by Yiqun Lin.
2016-09-27 19:28:41 -07:00
2acfb1e1e4
HDFS-10376. Enhance setOwner testing. (John Zhuge via Yongjun Zhang)
2016-09-27 14:55:28 -07:00
875062b5bc
HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda)
2016-09-27 11:26:45 -07:00
8ae4729107
HDFS-10828. Fix usage of FsDatasetImpl object lock in ReplicaMap. (Arpit Agarwal)
2016-09-27 10:02:15 -07:00
f55eb981dd
HDFS-10426. TestPendingInvalidateBlock failed in trunk. Contributed by Yiqun Lin.
2016-09-28 01:29:24 +09:00
df1d0f5d56
HDFS-10889. Remove outdated Fault Injection Framework documentaion. Contributed by Brahma Reddy Battula
2016-09-27 19:33:10 +05:30
edf0d0f8b2
HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki
2016-09-27 07:04:37 +05:30
059058f961
HDFS-10713. Throttle FsNameSystem lock warnings. Contributed by Hanisha Koneru.
2016-09-26 17:09:47 -07:00
8e06d865c4
Addendum patch for HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications.
2016-09-26 14:41:04 -07:00
3ae652f821
HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications. Contributed by Wei-Chiu Chuang.
2016-09-26 13:11:32 -07:00
3e37e243ee
HDFS-10869. Remove the unused method InodeId#checkId(). Contributed by Jagadesh Kiran N
2016-09-25 11:02:46 +05:30
6eb700ee74
HDFS-10866. Fix Eclipse Java compile errors related to generic parameters. Contributed by Konstantin Shvachko.
2016-09-23 14:10:11 -07:00
74b3dd514c
HDFS-10876. Dispatcher#dispatch should log IOException stacktrace. Contributed by Manoj Govindassamy.
2016-09-23 13:26:57 -07:00
a5bb88c8e0
HDFS-10843. Update space quota when a UC block is completed rather than committed. Contributed by Erik Krogen.
2016-09-23 10:37:46 -07:00
bbdf350ff9
HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document.
2016-09-23 19:21:34 +05:30
c4480f7f42
HDFS-10894. Remove the redundant charactors for command -saveNamespace in HDFSCommands.md Contributed by Yiqun Lin.
2016-09-23 19:11:02 +05:30
d85d9b2e7b
HDFS-10871. DiskBalancerWorkItem should not import jackson relocated by htrace. Contributed by Manoj Govindassamy.
2016-09-22 19:36:16 -07:00
40acacee08
HDFS-10877. Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers. Contributed by Sean Mackrory.
2016-09-22 11:43:11 -07:00
964e546ab1
HDFS-9333. Some tests using MiniDFSCluster errored complaining port in use. (iwasakims)
2016-09-21 10:35:25 +09:00
0e918dff59
HDFS-10879. TestEncryptionZonesWithKMS#testReadWrite fails intermittently. Contributed by Xiao Chen.
2016-09-20 16:56:52 -07:00
734d54c1a8
HDFS-10861. Refactor StripeReaders and use ECChunk version decode API. Contributed by Sammi Chen
2016-09-21 21:34:48 +08:00
e52d6e7a46
HDFS-10875. Optimize du -x to cache intermediate result. Contributed by Xiao Chen.
2016-09-19 21:44:42 -07:00
c54f6ef30f
HDFS-10868. Remove stray references to DFS_HDFS_BLOCKS_METADATA_ENABLED.
2016-09-19 11:17:03 -07:00
ea839bd48e
HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen.
2016-09-17 22:25:39 -07:00
b09a03cd7d
HDFS-10862. Typos in 4 log messages. Contributed by Mehran Hassani.
2016-09-16 16:08:47 +09:00
ec3ea18875
HDFS-9895. Remove unnecessary conf cache from DataNode. Contributed by Xiaobing Zhou.
2016-09-15 15:25:47 -07:00
2a8f55a0cf
HDFS-10805. Reduce runtime for append test. Contributed by Gergely Novak.
2016-09-14 09:31:17 -07:00
ea0c2b8b05
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-15 10:46:00 +08:00
86c9862bec
HDFS-10636. Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File. (Virajith Jalaparti via lei)
2016-09-13 12:54:14 -07:00
1c0d18f322
HDFS-10562. DiskBalancer: update documentation on how to report issues and debug. Contributed by Anu Engineer.
2016-09-13 12:42:10 -07:00
db6d243cf8
HDFS-10837. Standardize serializiation of WebHDFS DirectoryListing.
2016-09-13 11:02:36 -07:00
e3f7f58a5f
HDFS-10599. DiskBalancer: Execute CLI via Shell. Contributed by Manoj Govindassamy.
2016-09-13 09:38:12 -07:00
f0876b8b60
HDFS-10856. Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReport. Contributed by Yiqun Lin.
2016-09-13 14:50:03 +09:00
d2466ec3e0
HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang)
2016-09-12 21:25:06 -07:00
72dfb048a9
HDFS-10858. FBR processing may generate incorrect reportedBlock-blockGroup mapping. Contributed by Jing Zhao.
2016-09-12 16:40:11 -07:00
8a93f45a80
HDFS-10821. DiskBalancer: Report command support with multiple nodes. Contributed by Yiqun Lin.
2016-09-12 15:45:26 -07:00
58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
cc01ed7026
HDFS-10855. Fix typos in HDFS documents. Contributed by Yiqun Lin.
2016-09-10 23:24:49 -07:00
a99bf26a08
HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy)
2016-09-10 18:22:25 -07:00
bee9f57f5c
HDFS-10808. DiskBalancer does not execute multi-steps plan-redux. Contributed by Anu Engineer.
2016-09-09 15:00:39 -07:00
a0b0383677
Addendum patch for HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-09 10:54:14 -07:00
baab48922a
HDFS-9849. DiskBalancer: reduce lock path in shutdown code. Contributed by Yuanbo Liu.
2016-09-08 20:00:42 -07:00
0f73cd7ff6
Merge branch 'trunk' into HDFS-7240
2016-09-08 19:34:51 -07:00
35c5943b8b
HDFS-10553. DiskBalancer: Rename Tools/DiskBalancer class to Tools/DiskBalancerCLI. Contributed by Manoj Govindassamy.
2016-09-08 19:26:56 -07:00
011f3b24d4
HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang
2016-09-08 17:53:40 -07:00
b6d839a60c
HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin
2016-09-08 13:29:30 -07:00
401db4fc65
HDFS-8901. Use ByteBuffer in striping positional read. Contributed by Sammi Chen and Kai Zheng.
2016-09-08 11:54:33 -07:00
20a20c2f6e
HDFS-10847. Complete the document for FileDistribution processor in OfflineImageViewer. Contributed by Yiqun Lin.
2016-09-09 01:34:34 +09:00
63f594892e
HDFS-10778. Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer.
2016-09-08 15:13:43 +09:00
d355573f56
Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng"
...
This reverts commit 62a9667136
2016-09-08 05:50:17 +08:00
66921274db
Merge branch 'trunk' into HDFS-7240
2016-09-07 11:19:03 -07:00
5f23abfa30
HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula)
2016-09-06 13:37:21 -07:00
f6c0b7543f
HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee.
2016-09-06 11:48:35 -07:00
f0d5382ff3
HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth.
2016-09-06 11:02:39 -07:00
d37dc5d1b8
HDFS-9847. HDFS configuration should accept time units. Contributed by Yiqun Lin
2016-09-06 10:38:04 -07:00
62a9667136
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-07 17:05:33 +08:00
07650bc37a
HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-02 15:33:11 -07:00
cbd909ce2a
HDFS-10833. Fix JSON errors in WebHDFS.md examples.
2016-09-02 14:50:34 -07:00
1dc5c7ebfc
Merge branch 'trunk' into HDFS-7240
2016-09-01 08:44:50 -07:00
6c600360ca
Addendum fix for HDFS-10817 to fix failure of the added testFSReadLockLongHoldingReport in branch-2.
2016-08-31 23:43:59 -07:00
6f4b0d33ca
HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen.
2016-08-31 15:40:09 -07:00
85bab5fb57
HDFS-10784. Implement WebHdfsFileSystem#listStatusIterator.
2016-08-31 14:29:37 -07:00
01721dd88e
HDFS-10729. Improve log message for edit loading failures caused by FS limit checks. Contributed by Wei-Chiu Chuang.
2016-08-31 14:02:37 -05:00
20ae1fa259
HDFS-10813. DiskBalancer: Add the getNodeList method in Command. Contributed by Yiqun Lin.
2016-08-30 18:42:55 -07:00
9dcbdbdb5a
HDFS-9392. Admins support for maintenance state. Contributed by Ming Ma.
2016-08-30 14:00:13 -07:00
c4ee6915a1
HDFS-10760. DataXceiver#run() should not log InvalidToken exception as an error. Contributed by Pan Yuxuan.
2016-08-30 10:43:20 -07:00
4bd45f54ee
HDFS-10814. Add assertion for getNumEncryptionZones when no EZ is created. Contributed by Vinitha Reddy Gankidi.
2016-08-29 23:37:26 -07:00
cd5e10ccca
HDFS-4210. Throw helpful exception when DNS entry for JournalNode cannot be resolved. Contributed by Charles Lamb and John Zhuge.
2016-08-29 17:48:08 -07:00
5d1609ddf2
HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin.
2016-08-29 13:59:54 -07:00
6742fb6e68
HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu)
2016-08-29 10:15:34 -07:00
c25817159a
HDFS-10652. Add a unit test for HDFS-4660. Contributed by Vinayakumar B., Wei-Chiu Chuang, Yongjun Zhang.
2016-08-27 22:51:31 -07:00
e806db7190
HDFS-10584. Allow long-running Mover tool to login with keytab. Contributed by Rakesh R.
2016-08-26 16:43:45 -07:00
a445b82baa
HDFS-10793. Fix HdfsAuditLogger binary incompatibility introduced by HDFS-9184. Contributed by Manoj Govindassamy.
2016-08-26 15:39:18 -07:00
407b519fb1
HDFS-10798. Make the threshold of reporting FSNamesystem lock contention configurable. Contributed by Erik Krogen.
2016-08-26 14:19:55 -07:00
8b7adf4ddf
HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp.
2016-08-26 15:39:56 -05:00
13fb1b50e6
HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki.
2016-08-26 09:53:10 -05:00
4da5000dd3
HDFS-10748. TestFileTruncate#testTruncateWithDataNodesRestart runs sometimes timeout. Contributed by Yiqun Lin.
2016-08-25 09:00:44 -07:00
a1f3293762
HDFS-10772. Reduce byte/string conversions for get listing. Contributed by Daryn Sharp.
2016-08-24 15:21:08 -05:00
793447f799
HDFS-8905. Refactor DFSInputStream#ReaderStrategy. Contributed by Kai Zheng and Sammi Chen
2016-08-24 21:57:23 +08:00
ec252ce0fc
HDFS-10762. Pass IIP for file status related methods
2016-08-24 08:46:47 -05:00
126d165efd
HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen
2016-08-23 14:46:58 +00:00
f0efea490e
HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen.
2016-08-23 04:14:24 -07:00
e90f3359de
HDFS-10783. The option '-maxSize' and '-step' fail in OfflineImageViewer. Contributed by Yiqun Lin.
2016-08-23 19:57:23 +09:00
c49333becf
HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang)
2016-08-22 18:29:56 -07:00
3ca4d6ddfd
Revert "HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp."
...
This reverts commit 22fc46d765
2016-08-22 16:57:45 -05:00
22fc46d765
HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp.
2016-08-22 15:37:02 -05:00
dc7a1c54f4
HDFS-10692. Update JDiff report's base version for HDFS from 2.6.0 to 2.7.2. Contributed by Wangda Tan.
2016-08-22 11:35:18 -07:00
0faee62a0c
HDFS-10764. Fix INodeFile#getBlocks to not return null. Contributed by Arpit Agarwal.
2016-08-19 22:13:36 -07:00
db22affd79
Merge branch 'trunk' into HDFS-7240
2016-08-19 13:32:24 -07:00
2550371f66
HDFS-10711. Optimize FSPermissionChecker group membership check. Contributed by Daryn Sharp.
2016-08-19 09:12:17 -05:00
8179f9a493
HDFS-10645. Make block report size as a metric and add this metric to datanode web ui. Contributed by Yuanbo Liu.
2016-08-19 16:15:00 +09:00
ca13e7971d
HDFS-10773. BlockSender should not synchronize on the dataset object. (Contributed by Chen Liang)
2016-08-17 16:29:08 -07:00
2aa5e2c403
HDFS-10549. Correctly revoke file leases when closing files. Contributed by Yiqun Lin.
2016-08-17 15:52:38 -07:00
869393643d
HDFS-10745. Directly resolve paths into INodesInPath. Contributed by Daryn Sharp.
2016-08-17 15:53:03 -05:00
b047bc7270
HDFS-10560. DiskBalancer: Reuse ObjectMapper instance to improve the performance. Contributed by Yiqun Lin.
2016-08-16 10:20:08 -07:00
4d4d95fdd5
HDFS-10641. TestBlockManager#testBlockReportQueueing fails intermittently. (Contributed by Daryn Sharp)
2016-08-15 20:28:40 -07:00
ef55fe1716
HDFS-10747. o.a.h.hdfs.tools.DebugAdmin usage message is misleading. (Contributed by Mingliang Liu)
2016-08-15 20:23:47 -07:00
382d615260
HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu)
2016-08-15 20:22:14 -07:00
5628b36c08
HDFS-10559. DiskBalancer: Use SHA1 for Plan ID. Contributed by Xiaobing Zhou.
2016-08-15 20:10:21 -07:00
02abd131b8
HDFS-10567. Improve plan command help message. Contributed by Xiaobing Zhou.
2016-08-15 19:58:57 -07:00
864f878d59
HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee.
2016-08-15 17:28:09 -05:00
03dea65e0b
HDFS-10744. Internally optimize path component resolution. Contributed by Daryn Sharp.
2016-08-15 16:45:44 -05:00
bed69d18e6
HDFS-10580. DiskBalancer: Make use of unused methods in GreedyPlanner to print debug info. Contributed by Yiqun Lin
2016-08-15 12:40:29 -07:00
83e57e083f
HDFS-9696. Garbage snapshot records linger forever. Contributed by Kihwal Lee
2016-08-15 13:01:23 -05:00
9f29f423e4
HDFS-10737. disk balancer add volume path to report command. Contributed by Yuanbo Liu.
2016-08-15 09:47:30 -07:00
9019606b69
HDFS-10731. FSDirectory#verifyMaxDirItems does not log path name. Contributed by Wei-Chiu Chuang.
2016-08-12 05:58:13 -07:00
b5af9be72c
HDFS-8668. Erasure Coding: revisit buffer used for encoding and decoding. Contributed by Sammi Chen
2016-08-13 13:52:37 +08:00
a428d4f50e
HDFS-8897. Balancer should handle fs.defaultFS trailing slash in HA. Contributed by John Zhuge.
2016-08-10 22:17:35 -07:00
e7e8aed208
HDFS-10720. Fix intermittent test failure of TestDataNodeErasureCodingMetrics. Contributed by Rakesh R
2016-08-12 11:15:05 +08:00
ec289bbece
HDFS-10643. Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao.
2016-08-10 15:49:36 -07:00
fdafd0cb4f
Revert "Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao."
...
This reverts commit 9f816d7ca7
2016-08-10 15:47:55 -07:00
9f816d7ca7
Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao.
2016-08-10 15:14:38 -07:00
10e84c6a6e
HDFS-10694. processReport() should print blockReportId in each log message. Contributed by Yuanbo Liu.
2016-08-10 11:07:42 -07:00
d00d3add9e
HDFS-8224. Schedule a block for scanning if its metadata file is corrupt. Contributed by Rushabh S Shah.
2016-08-09 18:11:34 -07:00
9c6a4383ca
HDFS-10681. DiskBalancer: query command should report Plan file path apart from PlanID. (Manoj Govindassamy via lei)
2016-08-09 15:59:22 -07:00
cc48251bfd
HDFS-10457. DataNode should not auto-format block pool directory if VERSION is missing. (Wei-Chiu Chuang via lei)
2016-08-09 15:51:19 -07:00
0f701f433d
HDFS-10738. Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure. Contributed by Rakesh R.
2016-08-09 14:54:11 -05:00
b10c936020
HDFS-10342. BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt. Contributed by Kuhu Shukla.
2016-08-09 11:56:21 -05:00
8c0638471f
HADOOP-10682. Replace FsDatasetImpl object lock with a separate lock object. (Chen Liang)
2016-08-08 12:02:53 -07:00
4d3af47f27
HDFS-10693. metaSave should print blocks, not LightWeightHashSet. Contributed by Yuanbo Liu.
2016-08-07 14:29:26 -07:00
ef432579a7
HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu.
2016-08-05 17:31:11 +09:00
18d9e6ec0b
HDFS-10588. False alarm in datanode log - ERROR - Disk Balancer is not enabled. Contributed by Weiwei Yang.
2016-08-04 19:40:33 -07:00
438a9f047e
HDFS-10673. Optimize FSPermissionChecker's internal path usage. Contributed by Daryn Sharp.
2016-08-04 16:14:13 -05:00
57369a678c
HDFS-10343. BlockManager#createLocatedBlocks may return blocks on failed storages. Contributed by Kuhu Shukla.
2016-08-04 14:55:21 -05:00
4a8e6dc02f
HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp.
2016-08-04 11:53:44 -05:00
8c9b44eaff
In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin
2016-08-04 09:45:40 -07:00
6ae39199da
HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp.
2016-08-04 09:07:34 -05:00
6f63566694
HDFS-10569. A bug causes OutOfIndex error in BlockListAsLongs. Contributed by Weiwei Yang.
2016-08-03 15:20:20 -05:00
f4ba5ff1d7
HDFS-10710. In BlockManager#rescanPostponedMisreplicatedBlocks(), postponed misreplicated block counts should be retrieved within the NN lock protection.Contributed by GAO Rui.
2016-08-03 12:42:05 -07:00
22ef5286bc
HDFS-10674. Optimize creating a full path from an inode. Contributed by Daryn Sharp.
2016-08-03 13:12:43 -05:00
58db263e93
HDFS-742. A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list. Contributed by Mit Desai.
2016-08-03 12:14:59 -05:00
bebf10d245
HDFS-10656. Optimize conversion of byte arrays back to path string. Contributed by Daryn Sharp.
2016-08-03 11:53:41 -05:00
5e5b8793fb
HDFS-5805. TestCheckpoint.testCheckpoint fails intermittently on branch-2. Contributed by Eric Badger.
2016-08-02 15:37:36 -05:00
a5fb298e56
HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon.
2016-08-02 17:07:59 +09:00
c4463f2ef2
Revert "HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi."
...
This reverts commit 85a20508bd
2016-08-01 22:34:40 -07:00
9f473cf903
HDFS-10655. Fix path related byte array conversion bugs. (daryn)
2016-08-01 10:14:28 -05:00
8d32bd884d
HDFS-10703. HA NameNode Web UI should show last checkpoint time. (John Zhuge via Yongjun Zhang)
2016-07-29 23:20:14 -07:00
204a2055b1
HDFS-10691. FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException. Contributed by Yiqun Lin.
2016-07-29 15:39:48 +09:00
d9aae22fdf
HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge
2016-07-28 16:29:40 -07:00
ce3d68e9c3
HDFS-10676. Add namenode metric to measure time spent in generating EDEKs. Contributed by Hanisha Koneru.
2016-07-28 16:07:45 -07:00
a3d0cba810
HDFS-10689. Hdfs dfs chmod should reset sticky bit permission when the bit is omitted in the octal mode. (Manoj Govindassamy via Lei Xu)
2016-07-28 13:52:14 -07:00
67406460f0
HDFS-4176. EditLogTailer should call rollEdits with a timeout. (Lei Xu)
2016-07-28 13:32:00 -07:00
328c855a57
HDFS-10650. DFSClient#mkdirs and DFSClient#primitiveMkdir should use default directory permission. Contributed by John Zhuge.
2016-07-28 13:15:02 -07:00
414fbfab41
HDFS-10696. TestHDFSCLI fails. Contributed by Kai Sasaki.
2016-07-28 16:58:23 +09:00
098ec2b11f
HDFS-10519. Add a configuration option to enable in-progress edit log tailing. Contributed by Jiayi Zhou.
2016-07-27 17:55:41 -07:00
eb7ff0c992
HDFS-10667. Report more accurate info about data corruption location. (Yuanbo Liu via Yongjun Zhang)
2016-07-27 13:58:04 -07:00
54fe17a607
HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák.
2016-07-27 19:41:09 +09:00
55d5993a8e
HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu.
2016-07-27 10:26:42 +05:30
d84ab8a578
HDFS-10598. DiskBalancer does not execute multi-steps plan. Contributed by Lei (Eddy) Xu.
2016-07-26 16:01:50 -07:00
d2cf8b54c5
HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu)
2016-07-26 12:27:46 -07:00
25f0a9b3ab
HDFS-9937. Update dfsadmin command line help and HdfsQuotaAdminGuide. Contributed by Kai Sasaki.
2016-07-26 08:01:42 -07:00
2aa18e1925
Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu."
...
This reverts commit 7cac7655fd
2016-07-26 14:30:02 +01:00
7cac7655fd
HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu.
2016-07-26 10:54:24 +05:30
85a20508bd
HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi.
2016-07-25 18:50:59 -07:00
59466b8c18
HDFS-10671. Fix typo in HdfsRollingUpgrade.md. Contributed by Yiqun Lin.
2016-07-26 10:48:21 +09:00
0cde9e12a7
HDFS-10688. BPServiceActor may run into a tight loop for sending block report when hitting IOException. Contributed by Chen Liang.
2016-07-25 18:41:13 -07:00
b79ba4fe96
HDFS-10651. Clean up some configuration related codes about legacy block reader. Contributed by Youwei Wang
2016-07-23 17:45:33 +08:00
c6e3a0020f
HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R)
2016-07-22 12:33:50 -07:00
77ac04efe4
HDFS-10565: Erasure Coding: Document about the current allowed storage policies for EC Striped mode files
2016-07-22 11:08:20 -07:00
132deb4cac
HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu.
2016-07-21 16:41:02 -07:00
fcde6940e0
HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor.
2016-07-21 11:26:08 -07:00
bd3dcf46e2
HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp.
2016-07-21 11:14:39 -07:00
38128baff4
HDFS-10425. Clean up NNStorage and TestSaveNamespace. Contributed by Andras Bokor.
2016-07-20 10:44:11 -07:00
f7dabe3add
HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg.
2016-07-19 15:13:01 -07:00
fc570b55b9
HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin.
2016-07-19 13:49:24 -07:00
5b4a708704
HDFS-10639. Fix typos in HDFSDiskbalancer.md. Contributed by Yiqun Lin.
2016-07-15 21:41:57 -07:00
da456ffd62
Preparing for 3.0.0-alpha2 development
2016-07-15 19:04:17 -07:00
c48e9d608c
HDFS-10628. Log HDFS Balancer exit message to its own log. Contributed by Jiayi Zhou.
2016-07-15 11:42:44 -07:00
5537c6b234
HADOOP-13351. TestDFSClientSocketSize buffer size tests are flaky. Contributed by Aaron Fabbri and Mingliang Liu.
2016-07-14 14:40:58 -07:00
382dff7475
HDFS-10600. PlanCommand#getThrsholdPercentage should not use throughput value. (Yiqun Lin via lei)
2016-07-14 10:40:34 -07:00
2bbc3ea1b5
HDFS-10617. PendingReconstructionBlocks.size() should be synchronized. Contributed by Eric Badger.
2016-07-13 21:59:35 -05:00
438b7c5935
HDFS-10590: Fix TestReconstructStripedBlocks.testCountLiveReplicas test failures. Contributed by Rakesh R
2016-07-13 00:07:11 -07:00
087290e6b1
HDFS-10544. Balancer doesn't work with IPFailoverProxyProvider.
2016-07-12 23:18:37 -07:00
c447efebdb
HDFS-10579. HDFS web interfaces lack configs for X-FRAME-OPTIONS protection. Contributed by Anu Engineer.
2016-07-11 14:55:33 -07:00
da6f1b88dd
HDFS-10512. VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks. Contributed by Wei-Chiu Chuang and Yiqun Lin.
2016-07-08 19:40:44 -07:00
932aed64d7
HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin.
2016-07-08 14:39:11 -07:00
6e597600f7
HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu.
2016-07-07 14:58:19 -07:00
ab092c56c2
HDFS-10592:Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning. Contributed by Rakesh R
2016-07-07 13:52:35 -07:00
b8f93cd275
HDFS-10488. Update WebHDFS documentation regarding CREATE and MKDIR default permissions. Contributed by Wellington Chevreuil.
2016-07-07 10:36:55 -07:00
a3f93beaa1
HDFS-10582. Change deprecated configuration fs.checkpoint.dir to dfs.namenode.checkpoint.dir in HDFS Commands Doc. Contributed by Pan Yuxuan.
2016-07-06 17:51:42 -07:00
8b4b5259d5
HDFS-10589. Javadoc for HAState#HAState and HAState#setStateInternal contains non-existent params. Contributed by shenyinjie.
2016-07-04 07:15:04 -07:00
2e835d986e
HDFS-10572. Fix TestOfflineEditsViewer#testGenerated. Contributed by Surendra Singh Lilhore.
2016-07-03 21:11:56 -07:00
8b281bce85
HDFS-10548. Remove the long deprecated BlockReaderRemote. Contributed by Kai Zheng
2016-07-03 11:56:23 +08:00
e4a2545620
HDFS-9805. Add server-side configuration for enabling TCP_NODELAY for DataTransferProtocol and default it to true (Gary Helmling via cmccabe)
2016-06-29 12:41:42 -07:00
c0829f4493
HDFS-6434. Default permission for creating file should be 644 for WebHdfs/HttpFS. Contributed by Wellington Chevreuil.
2016-06-28 13:55:26 -07:00
23c3ff85a9
HADOOP-13209. replace slaves with workers (John Smith via aw)
2016-06-28 05:53:03 -07:00
2a0082c51d
HDFS-10440. Improve DataNode web UI (Contributed by Weiwei Yang)
2016-06-28 16:49:39 +05:30
73615a789d
HDFS-10536. Standby NN can not trigger log roll after EditLogTailer thread failed 3 times in EditLogTailer.triggerActiveLogRoll method (Contributed by XingFeng Shen
2016-06-26 15:45:32 +05:30
bf74dbf80d
HDFS-7959. WebHdfs logging is missing on Datanode (Kihwal Lee via sjlee)
2016-06-24 14:44:15 -07:00
22be604fa3
Revert "HDFS-10534. NameNode WebUI should display DataNode usage rate with a certain percentile. Contributed by Kai Sasaki."
...
This reverts commit 0424056a77
2016-06-24 09:36:03 -07:00
0424056a77
HDFS-10534. NameNode WebUI should display DataNode usage rate with a certain percentile. Contributed by Kai Sasaki.
2016-06-24 08:58:29 -07:00
b2584bee45
HDFS-10571. TestDiskBalancerCommand#testPlanNode failed with IllegalArgumentException. Contributed by Xiaobing Zhou.
2016-06-23 18:27:02 -07:00
8a6e354122
HDFS-10552. DiskBalancer "-query" results in NPE if no plan for the node. Contributed by Anu Engineer.
2016-06-23 18:27:02 -07:00
e8de28181a
HDFS-10551. o.a.h.h.s.diskbalancer.command.Command does not actually verify options as expected. Contributed by Anu Engineer.
2016-06-23 18:27:02 -07:00
66fa34c839
HDFS-10557. Fix handling of the -fs Generic option. (Arpit Agarwal)
2016-06-23 18:27:01 -07:00
7b23ad1ef7
HDFS-10550. DiskBalancer: fix issue of order dependency in iteration in ReportCommand test. Contributed by Xiaobing Zhou.
2016-06-23 18:27:01 -07:00
c6ed54808d
HDFS-10547. DiskBalancer: fix whitespace issue in doc files. Contributed by Anu Engineer.
2016-06-23 18:27:01 -07:00
0774412e41
HDFS-10545. DiskBalancer: PlanCommand should use -fs instead of -uri to be consistent with other hdfs commands. Contributed by Anu Engineer.
2016-06-23 18:27:01 -07:00
5b8e1c26d7
HDFS-10541. Diskbalancer: When no actions in plan, error message says "Plan was generated more than 24 hours ago". Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
cb68e5b3bd
HDFS-10540. Diskbalancer: The CLI error message for disk balancer is not enabled is not clear. Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
3225c24e0e
HDFS-10520. DiskBalancer: Fix Checkstyle issues in test code. Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
7e2be5c4a0
HDFS-10518. DiskBalancer: Pretty-print json in Query command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
af11ab34d0
HDFS-10517. DiskBalancer: Support help command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
b502102bb1
HDFS-9461. DiskBalancer: Add Report Command. Contributed by Xiaobing Zhou.
2016-06-23 18:21:08 -07:00
121142cf95
HDFS-10501. DiskBalancer: Use the default datanode port if port is not provided. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
78a1032b71
HDFS-10500. Diskbalancer: Print out information when a plan is not generated. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
d2ff793ae4
HDFS-10496. DiskBalancer: ExecuteCommand checks planFile in a wrong way. Contributed by Lei (Eddy) Xu.
2016-06-23 18:21:08 -07:00
64ccb232cc
HDFS-10478. DiskBalancer: resolve volume path names. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
47dcb0f952
HDFS-10476. DiskBalancer: Plan command output directory should be a sub-directory. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
06a9799d84
HDFS-9547. DiskBalancer: Add user documentation. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
43eee50966
HDFS-10403. DiskBalancer: Add cancel command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
9e5fcb5e40
HDFS-10402. DiskBalancer: Add QueryStatus command. (Contributed by Anu Engineer)
2016-06-23 18:21:08 -07:00
5df2d2b8fd
HDFS-10399. DiskBalancer: Add JMX for DiskBalancer. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
1b39b283c7
HDFS-9546: DiskBalancer: Add Execute command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
75882ec0b0
HDFS-9545: DiskBalancer: Add Plan Command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
1594b472bb
HDFS-9543. DiskBalancer: Add Data mover. Contributed by Anu Engineer.
2016-06-23 18:20:24 -07:00
7820737cfa
HDFS-9735. DiskBalancer : Refactor moveBlockAcrossStorage to be used by disk balancer. Contributed by Anu Engineer.
2016-06-23 18:20:24 -07:00
050677077b
HDFS-9720. DiskBalancer : Add configuration parameters. Contributed by Anu Engineer.
2016-06-23 18:18:48 -07:00
6c606bf5c8
HDFS-9709. DiskBalancer : Add tests for disk balancer using a Mock Mover class. Contributed by Anu Engineer.
2016-06-23 18:18:48 -07:00
9be9703716
Fix a build break in HDFS-1312
2016-06-23 18:18:48 -07:00
ec601673a8
HDFS-10170. DiskBalancer: Force rebase diskbalancer branch. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
75a711a2d5
HDFS-9703. DiskBalancer: getBandwidth implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
918722bdd2
HDFS-9702. DiskBalancer: getVolumeMap implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
4b93ddae07
HDFS-8821. Stop tracking CHANGES.txt in the HDFS-1312 feature branch. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
747227e9de
HDFS-9817. Use SLF4J in new classes. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
3df0781aa7
HDFS-9856. Suppress Jenkins warning for sample JSON file. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
9847640603
HDFS-9683. DiskBalancer: Add cancelPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
e646c2eb50
HDFS-9681. DiskBalancer: Add QueryPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
2b1b2faf76
HDFS-9671. DiskBalancer: SubmitPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
66f0bb646d
HDFS-9647. DiskBalancer: Add getRuntimeSettings. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
96fe685b7a
HDFS-9645. DiskBalancer: Add Query RPC. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
0501d430e2
HDFS-9595. DiskBalancer: Add cancelPlan RPC. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
7100c0da35
HDFS-9588. DiskBalancer: Add submitDiskbalancer RPC. (Anu Engineer via Arpit Agarwal)
2016-06-23 18:18:48 -07:00
5724a10316
HDFS-9469. DiskBalancer: Add Planner. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
e325c6ade9
HDFS-9611. DiskBalancer : Replace htrace json imports with jackson. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
599eca07b4
HDFS-9526. Fix jackson annotation imports. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
30c6ebd699
HDFS-9449. DiskBalancer: Add connectors. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
91a5c48143
HDFS-9420. Add DataModels for DiskBalancer. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
dca298d79e
HDFS-10469. Add number of active xceivers to datanode metrics. Contributed by Hanisha Koneru.
2016-06-23 14:13:39 -07:00
e6cb07520f
HDFS-10460. Recompute block checksum for a particular range less than file size on the fly by reconstructing missed block. Contributed by Rakesh R
2016-06-24 17:39:32 +08:00
01643020d2
HDFS-10555: Unable to loadFSEdits due to a failure in readCachePoolInfo. Contributed by Uma Maheswara Rao G
2016-06-22 22:06:13 -07:00
17eae9ebb3
HDFS-10473: Allow only suitable storage policies to be set on striped files. Contributed by Uma Maheswara Rao G
2016-06-22 11:17:43 -07:00
d433b16ce6
HDFS-7597. DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping. Contributed by Daryn Sharp, Bob Hansen, and Xiao Chen.
2016-06-22 11:00:16 +09:00
f2ac132d6a
HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula)
2016-06-21 15:42:28 +05:30
46f1602e89
HDFS-10448. CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 (Yiqun Lin via cmccabe)
2016-06-20 18:25:09 -07:00
7adc4d1769
HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey.
2016-06-20 17:22:55 -07:00
5f6b4157a4
HDFS-10328. Add per-cache-pool default replication num configuration (xupeng via cmccabe)
2016-06-20 10:42:50 -07:00
0319d73c3b
HDFS-10437. ReconfigurationProtocol not covered by HDFSPolicyProvider. (Arpit Agarwal)
2016-06-18 16:16:54 -07:00
280069510b
HDFS-10493. Add links to datanode web UI in namenode datanodes page. Contributed by Weiwei Yang.
2016-06-17 11:05:44 -05:00
09e82acaf9
HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang.
2016-06-17 16:20:38 +09:00
b5f772abda
Merge branch 'trunk' into HDFS-7240
2016-06-16 11:29:40 -07:00
e983eafb44
HDFS-10532. Typo in RollingUpgrade docs. (Contributed by Yiqun Lin)
2016-06-16 10:10:33 -07:00
cc6c265171
HDFS-10256. Use GenericTestUtils.getTestDir method in tests for temporary directories (Contributed by Vinayakumar B)
2016-06-16 16:47:06 +05:30
2ca73445f5
HDFS-10525. Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap (Xiao Chen via cmccabe)
2016-06-15 22:47:01 -07:00
2449db507d
HDFS-10505. OIV's ReverseXML processor should support ACLs (Surendra Singh Lilhore via cmccabe)
2016-06-15 22:35:19 -07:00
b48f27e794
HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma)
2016-06-15 22:00:52 -07:00
7d521a29ee
HDFS-9016. Display upgrade domain information in fsck. (mingma)
2016-06-14 20:05:50 -07:00
709a814fe0
HDFS-10516. Fix bug when warming up EDEK cache of more than one encryption zone. Contributed by Xiao Chen.
2016-06-13 11:50:19 -07:00
d44f4745b4
HDFS-7987. Allow files / directories to be moved (Ravi Prakash via aw)
2016-06-10 09:02:28 -07:00
23923086d1
Merge branch 'trunk' into HDFS-7240
2016-06-09 20:34:39 -07:00
ae047655f4
HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak)
2016-06-08 13:44:22 -07:00
1ee9ea0026
Remove redundant TestMiniDFSCluster.testDualClusters. Contributed by Jiayi Zhou.
2016-06-08 12:58:56 -07:00
be34e85e68
HDFS-10468. HDFS read ends up ignoring an interrupt. Contributed by Jing Zhao
2016-06-07 10:48:21 -07:00
e620530301
HDFS-10485. Fix findbugs warning in FSEditLog.java. (aajisaka)
2016-06-07 17:52:55 +09:00
6de9213df1
HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster.
2016-06-06 15:52:39 -07:00
35f255b03b
Revert "Revert "HADOOP-13226 Support async call retry and failover.""
...
This reverts commit 5360da8bd9
2016-06-06 16:31:43 +08:00
7e7b1ae037
Revert "Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.""
...
This reverts commit 8cf47d8589
2016-06-06 16:31:38 +08:00
db41e6d285
Revert "Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou""
...
This reverts commit 5ee5912ebd
2016-06-06 16:31:34 +08:00
b3d81f38da
Revert "Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit b82c74b910
2016-06-06 16:31:30 +08:00
cba9a01889
Revert "Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit f23d5dfc60
2016-06-06 16:29:38 +08:00
aa20fa150d
Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou""
...
This reverts commit 4d36b221a2
2016-06-06 16:28:47 +08:00
eded3d109e
Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit 106234d873
2016-06-06 16:28:21 +08:00
106234d873
Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit fc94810d3f
2016-06-03 18:09:55 -07:00
4d36b221a2
Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"
...
This reverts commit 1b9f18623a
2016-06-03 18:09:18 -07:00
f23d5dfc60
Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 7251bb922b
2016-06-03 18:09:17 -07:00
b82c74b910
Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 02d4e478a3
2016-06-03 18:09:14 -07:00
5ee5912ebd
Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou"
...
This reverts commit f4b9bcd87c
2016-06-03 18:09:13 -07:00
8cf47d8589
Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou."
...
This reverts commit 21890c4239
2016-06-03 18:09:12 -07:00
5360da8bd9
Revert "HADOOP-13226 Support async call retry and failover."
...
This reverts commit 83f2f78c11
2016-06-03 18:09:09 -07:00
99a771cd7a
HDFS-7767. Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI (Ravi Prakash via aw)
2016-06-03 17:07:39 -07:00
15f018434c
HDFS-9877. HDFS Namenode UI: Fix browsing directories that need to be encoded (Ravi Prakash via aw)
2016-06-03 17:06:29 -07:00
97e2449477
HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (Contributed by Akira Ajisaka)
2016-06-02 13:14:45 -07:00
1df6f5735c
HDFS-10471. DFSAdmin#SetQuotaCommand's help msg is not correct. Contributed by Yiqun Lin.
2016-06-03 04:10:32 +09:00
aadb77e412
HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula.
2016-06-02 20:55:04 +09:00
69555fca06
HDFS-9476. TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail. Contributed by Masatake Iwasaki.
2016-06-02 18:52:47 +09:00
d749cf65e1
HDFS-9833. Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data. Contributed by Rakesh R.
2016-06-02 12:56:21 +08:00
8ceb06e239
HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:54:34 -07:00
c7921c9bdd
HDFS-9466. TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:37:52 -07:00
29d6cadc52
HDFS-10415. TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called (Mingliang Liu via cmccabe)
2016-05-31 16:54:52 -07:00
83f2f78c11
HADOOP-13226 Support async call retry and failover.
2016-05-31 16:30:11 -07:00
93d8a7f2a2
Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng"
...
This reverts commit a6c79f92d5
2016-06-01 08:41:15 +08:00
21890c4239
HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.
2016-05-27 14:20:18 -07:00
5ea6fd85c7
HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang)
2016-05-27 10:02:02 -07:00
916140604f
HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li
2016-05-28 14:23:39 +08:00
8c84a2a93c
HDFS-10236. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-3]. Contributed by Rakesh R.
2016-05-26 16:50:55 -07:00
55c3e2de3d
HDFS-10463. TestRollingFileSystemSinkWithHdfs needs some cleanup. (Daniel Templeton via kasha)
2016-05-26 14:24:31 -07:00
f4b9bcd87c
HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou
2016-05-26 12:15:17 -07:00
77202fa103
HADOOP-13010. Refactor raw erasure coders. Contributed by Kai Zheng
2016-05-27 13:23:34 +08:00
f69f5ab3b6
HDFS-10434. Fix intermittent test failure of TestDataNodeErasureCodingMetrics. Contributed by Rakesh R
2016-05-27 04:42:53 +08:00
a6c79f92d5
HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng
2016-05-27 04:29:42 +08:00
2dab64da2f
HDFS-10217. show 'blockScheduled' tooltip in datanodes table.
2016-05-25 12:34:52 +05:30
02d4e478a3
HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-24 14:51:27 -07:00
15ed080e36
HDFS-9365. Balaner does not work with the HDFS-6376 HA setup.
2016-05-24 12:49:48 -07:00
57c31a3fef
HDFS-9782. RollingFileSystemSink should have configurable roll interval. (Daniel Templeton via kasha)
2016-05-24 10:37:00 -07:00
4b0f55b6ea
HDFS-7766. Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect (Ravi Prakash via aw)
2016-05-23 15:52:01 -07:00
12eb5a7b11
HDFS-10439. Update setOwner doc in HdfsPermissionsGuide. Contributed by John Zhuge.
2016-05-22 16:53:30 +09:00
54f7975dfc
HDFS-10420. Fix Ozone unit tests to use MiniOzoneCluster. Contributed by Arpit Agarwal.
2016-05-20 15:19:09 -07:00
500e946729
HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth.
2016-05-20 13:01:43 -07:00
bcde1562d2
HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth.
2016-05-20 12:47:41 -07:00
204de430bf
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-19 15:52:32 -05:00
76279f5877
Merge branch 'trunk' into HDFS-7240
2016-05-19 12:44:16 -07:00
4770ae7e33
HDFS-10363. Ozone: Introduce new config keys for SCM service endpoints. Contributed by Arpit Agarwal.
2016-05-19 11:02:40 -07:00
010e6ac328
HDFS-2173. saveNamespace should not throw IOE when only one storage directory fails to write VERSION file. Contributed by Andras Bokor.
2016-05-18 14:13:41 -07:00
cf552aa87b
HDFS-10360. DataNode may format directory and lose blocks if current/VERSION is missing. (Wei-Chiu Chuang via lei)
2016-05-18 09:38:37 -07:00
f4d8fde822
HDFS-9926. MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils. (Contributed by Josh Elser)
2016-05-18 09:20:57 -07:00
dd99f5fc79
HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu.
2016-05-17 16:12:00 -07:00
f6ef876fe1
MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen.
2016-05-17 14:41:47 -07:00
e24fe2641b
HDFS-9732, Improve DelegationTokenIdentifier.toString() for better logging. Contributed by Yongjun Zhang.
2016-05-17 14:03:08 -07:00
1356cbe994
Revert "HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu."
...
This reverts commit 0942954e8a
2016-05-17 15:20:03 -05:00
0942954e8a
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-17 15:06:25 -05:00
7cd5ae62f6
HDFS-10404. Fix formatting of CacheAdmin command usage help text (Yiqun Lin via cmccabe)
2016-05-17 11:10:48 -07:00
4a5819dae2
HDFS-10303. DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. Contributed by Surendra Singh Lilhore.
2016-05-17 08:53:31 -05:00
9fe5828f05
HDFS-10242. Cannot create space quota of zero. Contributed by Takashi Ohnishi.
2016-05-17 15:53:09 +09:00
61f46be071
HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee)
2016-05-16 18:49:47 -07:00
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
6a6e74acf5
HDFS-10410. RedundantEditLogInputStream.LOG is set to wrong class. (John Zhuge via lei)
2016-05-16 17:05:46 -07:00
ad9441122f
HDFS-8449. Add tasks count metrics to datanode for ECWorker. Contributed by Bo Li.
2016-05-15 23:39:09 -07:00
45788204ae
HDFS-10333. Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk. Contributed by Yiqun Lin.
2016-05-15 22:15:05 -07:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
a2774debf7
HDFS-9389. Add maintenance states to AdminStates. (Ming Ma via lei)
2016-05-12 15:42:54 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
7251bb922b
HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-11 17:53:40 -07:00
687233f20d
HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe)
2016-05-11 13:45:39 -07:00
307db61aac
HDFS-10172. hdfs erasurecode command should remove the redundant -usage option. Contributed by Yuanbo Liu.
2016-05-10 15:20:17 -07:00
f0252ad252
HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe)
2016-05-10 11:15:05 -07:00
b9e5a32fa1
HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah.
2016-05-09 15:50:09 -05:00
411fb4b2b7
HDFS-10337. OfflineEditsViewer stats option should print 0 instead of null for the count of operations. Contributed by Yiqun Lin.
2016-05-08 15:49:13 -07:00
6957e45699
HDFS-10374. typo in HDFSHighAvailabilityWithQJM doc. Contributed by Takashi Ohnishi.
2016-05-07 11:26:42 +09:00
2835f146b5
HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun.
2016-05-06 15:36:47 +09:00
dacd1f50fe
HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang.
2016-05-05 18:08:47 -07:00
63a6a1d732
HDFS-10361. Support starting StorageContainerManager as a daemon. (Contributed by Arpit Agarwal)
2016-05-05 15:09:59 -07:00
d8faf47f32
HADOOP-13051. Test for special characters in path being respected during globPaths. Contributed by Harsh J.
2016-05-05 14:18:39 -07:00
1268cf5fbe
HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma)
2016-05-04 17:02:26 -07:00
78bd1b2ab2
Merge trunk into HDFS-7240
2016-05-04 10:01:39 -07:00
6d77d6eab7
HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. (Contributed by Brahma Reddy Battula)
2016-05-03 16:52:43 -07:00
45a753ccf7
HDFS-10344. DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash. Contributed by Xiaoyu Yao.
2016-05-02 19:30:47 -07:00
05d7a838ee
Merge branch 'trunk' into HDFS-7240
2016-05-02 14:55:23 -07:00
183175e60f
HDFS-10351. Ozone: Optimize key writes to chunks by providing a bulk write implementation in ChunkOutputStream. Contributed by Chris Nauroth.
2016-05-02 14:50:40 -07:00
1b9f18623a
HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou
2016-05-02 11:16:30 -07:00
2753185a01
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-29 22:32:27 -07:00
8264b61656
HDFS-10349. Ozone: StorageContainerManager fails to compile after merge of HDFS-10312 maxDataLength enforcement. Contributed by Chris Nauroth.
2016-04-29 20:44:38 -07:00
7da540d03e
HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah.
2016-04-29 15:50:43 -05:00
3dcbbdc1d1
Merge trunk into HDFS-7240
2016-04-29 13:18:40 -07:00
4da6f69ca1
HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu
2016-04-29 11:22:33 -07:00
af9b000535
HDFS-10260. TestFsDatasetImpl#testCleanShutdownOfVolume often fails. Contributed by Rushabh Shah.
2016-04-29 09:58:44 -05:00
6243eabb48
HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla
2016-04-28 12:44:53 -05:00
0a152103f1
HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu)
2016-04-27 14:22:51 -07:00
919a1d824a
HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla.
2016-04-27 08:19:48 -05:00
fc94810d3f
HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-04-26 17:10:13 -07:00
6be22ddbf1
HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge.
2016-04-26 13:04:18 -07:00
c6a2430b47
HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun.
2016-04-26 11:45:29 -05:00
5865fe2bf0
HDFS-9869. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-2]. Contributed by Rakesh R.
2016-04-25 22:01:54 -07:00
8eadd7145e
HDFS-10325. TestDFSShell#testMoveWithTargetPortEmpty fails. Contributed by Takanobu Asanuma.
2016-04-26 07:11:13 +09:00
f308561f1d
HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma
2016-04-25 12:01:48 -07:00
10f0f7851a
HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor.
2016-04-25 11:38:14 -05:00
6fcde2e38d
HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims)
2016-04-23 23:37:56 +09:00
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
d6402faded
Revert "HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano"
...
This reverts commit 4838b735f0
2016-04-22 10:17:20 -07:00
5d4255a801
HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang.
2016-04-22 11:08:44 -05:00
14ab7a81e2
HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand
2016-04-21 15:41:50 -07:00
bbce1d525e
HDFS-10319. Balancer should not try to pair storages with different types.
2016-04-21 15:20:12 -07:00
4838b735f0
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-21 11:32:39 -07:00
b4be288c5d
HDFS-10207. Support enable Hadoop IPC backoff without namenode restart. Contributed by Xiaobing Zhou.
2016-04-21 10:18:48 -07:00
5bd7b592e5
HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao.
2016-04-20 23:10:52 -07:00
63ac2db59a
HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth.
2016-04-20 13:39:44 -07:00
ad36fa6f42
HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe)
2016-04-20 11:33:11 -07:00
af9bdbe447
HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou)
2016-04-19 11:26:30 -07:00
90d1af1416
HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun.
2016-04-19 18:54:11 +08:00
4b4df2cd48
HDFS-10284. o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently. Contributed by Mingliang Liu.
2016-04-19 18:07:46 +08:00
438efc3634
HDFS-10291 TestShortCircuitLocalRead failing (stevel)
...
(cherry picked from commit dd64769699
2016-04-19 17:48:20 +08:00
b5d4c7dc76
HADOOP-12924. Configure raw erasure coders for supported codecs. Contributed by Rui Li.
2016-04-20 15:47:01 +08:00
be0bce1b71
HDFS-10306. SafeModeMonitor should not leave safe mode if name system is starting active service. Contributed by Mingliang Liu.
2016-04-18 18:05:49 -07:00
c884170c93
Merge branch 'trunk' into HDFS-7240
2016-04-18 16:58:45 -07:00
ce524c584b
HDFS-10232. Ozone: Make config key naming consistent. Contributed by Anu Engineer.
2016-04-18 16:52:40 -07:00
cb3ca460ef
HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe)
2016-04-18 13:47:56 -07:00
d8b729e16f
HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun.
2016-04-18 07:58:55 -05:00
ab903029a9
HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun.
2016-04-18 20:29:29 +08:00
67523ffcf4
HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi.
2016-04-18 09:28:02 +08:00
55e19b7f0c
HDFS-10293. StripedFileTestUtil#readAll flaky. Contributed by Mingliang Liu.
2016-04-15 10:53:40 -07:00
89a838769f
HDFS-10283. o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently. Contributed by Mingliang Liu.
2016-04-15 10:49:21 -07:00
b9c9d03591
HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu.
2016-04-14 15:24:39 -05:00
809226752d
HDFS-10286. Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties. Contributed by Xiaobing Zhou.
2016-04-14 10:57:02 -07:00
c970f1d005
HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang.
2016-04-14 12:45:47 -05:00
0d1c1152f1
HDFS-10282. The VolumeScanner should warn about replica files which are misplaced. Contributed by Colin Patrick McCabe.
2016-04-14 07:58:24 -05:00
ae6c0e839b
Merge branch 'trunk' into HDFS-7240
2016-04-13 23:13:11 -07:00
5566177c9a
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode. Contributed by Xiaobing Zhou.
2016-04-13 16:51:13 -07:00
4895c73dd4
Revert "HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode"
...
This reverts commit 192112d5a2
2016-04-13 16:45:18 -07:00
314aa21a89
HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun.
2016-04-13 16:39:50 -07:00
192112d5a2
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode
2016-04-13 15:34:41 -07:00
d2f3bbc290
HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák
2016-04-13 11:22:36 -05:00
903428bf94
HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun.
2016-04-13 17:49:45 +08:00
eee7269748
HDFS-10278. Ozone: Add paging support to list Volumes. Contributed by Anu Engineer.
2016-04-12 18:08:48 -07:00
33b51da8c5
HDFS-10251. Ozone: Shutdown cleanly. Contributed by Anu Engineer
2016-04-12 18:00:24 -07:00
35f0770555
HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe)
2016-04-12 16:28:42 -07:00
6ef42873a0
HDFS-9918. Erasure Coding: Sort located striped blocks based on decommissioned states. Contributed by Rakesh R.
2016-04-12 13:38:58 -07:00
600d129bb8
HDFS-10273. Remove duplicate logSync() and log message in FSN#enterSafemode() (Vinayakumar B via cmccabe)
2016-04-12 10:52:39 -07:00
209303be3a
HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang.
2016-04-12 15:26:50 +09:00
a9a607f8fc
HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula)
2016-04-12 11:44:34 +05:30
9c32f8785e
HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang.
2016-04-08 15:32:13 +09:00
3f62ba558d
Merge branch 'trunk' into HDFS-7240
2016-04-07 14:43:39 -07:00
fedb22d9b6
HDFS-10268. Ozone: end-to-end integration for create/get volumes, buckets and keys. Contributed by Chris Nauroth.
2016-04-07 14:38:54 -07:00
3c18a53cbd
HDFS-9719. Refactoring ErasureCodingWorker into smaller reusable constructs. Contributed by Kai Zheng.
2016-04-06 22:50:24 -07:00
654cd1d0c0
HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R
2016-04-07 12:38:42 +08:00
4bd7cbc29d
HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose
2016-04-06 21:07:31 -07:00
aede8c10ec
HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee
2016-04-06 20:20:14 +00:00
188f65287d
HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe)
2016-04-06 11:28:34 -07:00
221b3a8722
HDFS-10192. Namenode safemode not coming out during failover. Contributed by Brahma Reddy Battula.
2016-04-06 10:42:59 -07:00
9ba1e5af06
HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger.
2016-04-05 16:26:18 -05:00
0cd320a846
HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula.
2016-04-05 13:41:19 -07:00
b3044db407
HDFS-10250. Ozone: Add key Persistence. Contributed by Anu Engineer.
2016-04-05 12:59:45 -07:00
917464505c
HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla.
2016-04-05 09:07:24 -05:00
818d6b799e
HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula)
2016-04-05 09:49:39 +08:00
f6b1a81812
HDFS-8496. Calling stopWriter() with FSDatasetImpl lock held may block other threads (cmccabe)
2016-04-04 18:02:15 -07:00
a7d1fb0cd2
HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee.
2016-04-04 16:40:00 -05:00
154d2532cf
HDFS-9599. TestDecommissioningStatus.testDecommissionStatus occasionally fails (Lin Yiqun via iwasakims)
2016-04-05 03:19:48 +09:00
54b2e78fd2
HDFS-10253. Fix TestRefreshCallQueue failure (Contributed by Xiaoyu Yao)
2016-04-03 13:27:49 +05:30
4ac97b1815
HDFS-10238. Ozone : Add chunk persistance. Contributed by Anu Engineer.
2016-04-01 10:52:38 -07:00
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
32c0c3ecdf
HDFS-10221. Add .json to the rat exclusions. Contributed by Ming Ma.
2016-03-31 09:06:14 +09:00
55d4ee71e2
HDFS-8210. Ozone: Implement storage container manager. Contributed by Chris Nauroth.
2016-03-30 14:49:55 -07:00
37e23ce45c
HDFS-10223. peerFromSocketAndKey performs SASL exchange before setting connection timeouts (cmccabe)
2016-03-30 13:37:37 -07:00
09d63d5a19
HDFS-5177. blocksScheduled count should be decremented for abandoned blocks (Contributed by Vinayakumar B)
2016-03-30 14:22:00 +08:00
ddfe6774c2
HDFS-9439. Support reconfiguring fs.protected.directories without NN restart. (Contributed by Xiaobing Zhou)
2016-03-29 13:55:00 -07:00
c9307e48b7
HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA)
2016-03-29 13:50:08 -07:00
f2aec4eb82
HDFS-10197. TestFsDatasetCache failing intermittently due to timeout. Contributed by Lin Yiqun.
2016-03-29 11:54:42 -07:00
1f004b3367
HDFS-9871. "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. (Contributed by Brahma Reddy Battulla)
2016-03-29 11:25:41 +08:00
30473ecfb4
Merge branch 'trunk' into HDFS-7240
2016-03-28 10:34:52 -07:00
3a4ff7776e
HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng
2016-03-26 19:58:09 -07:00
643c5e5bdc
HDFS-10195. Ozone: Add container persistence. Contributed by Anu Engineer.
2016-03-26 11:40:00 -07:00
a337ceb74e
Revert "HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng"
...
This reverts commit e5ff0ea7ba
2016-03-26 09:20:01 -07:00
e5ff0ea7ba
HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng
2016-03-26 00:52:50 -07:00
fde8ac5d85
Add missing files from HDFS-9005. (lei)
2016-03-25 17:11:35 -07:00
4fcfea71bf
HDFS-9005. Provide support for upgrade domain script. (Ming Ma via Lei Xu)
2016-03-25 17:09:12 -07:00
2dc48b7f1e
HDFS-10202. ozone : Add key commands to CLI. Contributed by Anu Engineer.
2016-03-25 09:50:58 -07:00
58cdb715fb
HDFS-10196. Ozone : Enable better error reporting for failed commands in ozone shell. Contributed by Anu Engineer.
2016-03-24 09:27:10 -07:00
738155063e
HADOOP-12857. rework hadoop-tools (aw)
2016-03-23 13:46:38 -07:00
8f85e5d212
HDFS-10200. Docs for WebHDFS still describe GETDELEGATIONTOKENS operation. Contributed by Wellington Chevreuil.
2016-03-23 12:55:15 -07:00
a7d8f2b396
HDFS-10189. PacketResponder#toString should include the downstreams for PacketResponderType.HAS_DOWNSTREAM_IN_PIPELINE (Joe Pallas via cmccabe)
2016-03-22 14:56:02 -07:00
680716f31e
HDFS-9951. Use string constants for XML tags in OfflineImageReconstructor (Lin Yiqun via cmccabe)
2016-03-21 11:40:37 -07:00
e3bb38d625
HDFS-9405. Warmup NameNode EDEK caches in background thread. Contributed by Xiao Chen.
2016-03-21 11:39:05 -07:00
8a3f0cb255
HDFS-7166. SbNN Web UI shows #Under replicated blocks and #pending deletion blocks. Contributed by Wei-Chiu Chuang.
2016-03-20 17:21:25 -07:00
cd8b6889a7
HDFS-9579. Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level (Ming Ma via sjlee)
2016-03-19 14:02:04 -07:00
3f708bf711
HDFS-10179. Ozone: Adding logging support. Contributed by Anu Engineer.
2016-03-19 13:59:00 -07:00
2e517a64d8
HDFS-9960. OzoneHandler : Add localstorage support for keys. Contributed by Anu Engineer.
2016-03-19 13:36:18 -07:00
c73a32c21c
HDFS-10180. Ozone: Refactor container Namespace. Contributed by Anu Engineer.
2016-03-18 13:46:56 -07:00
37e3a36a3f
Merge branch 'trunk' into HDFS-7240
2016-03-18 11:18:48 -07:00
9b623fbaf7
HDFS-3677. dfs.namenode.edits.dir.required is missing from hdfs-default.xml. Contributed by Mark Yang.
2016-03-19 01:11:15 +09:00
63c966a3fb
HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah.
2016-03-18 10:24:59 -05:00
dc951e606f
HDFS-9949. Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared (Harsh J via cmccabe)
2016-03-17 10:37:42 -07:00
02a250db9f
HDFS-10173. Typo in DataXceiverServer. Contributed by Michael Han.
2016-03-16 18:49:57 -07:00
32d043d9c5
HDFS-9857. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-1]. Contributed by Rakesh R.
2016-03-16 16:53:58 -07:00
4143f0362f
HDFS-9961. Ozone: Add buckets commands to CLI. Contributed by Anu Engineer.
2016-03-15 21:27:53 -07:00
d457401784
HDFS-9904. testCheckpointCancellationDuringUpload occasionally fails. Contributed by Lin Yiqun.
2016-03-15 10:52:47 -05:00
Kihwal Lee
Chris Nauroth
Vinayakumar B
Brahma Reddy Battula
Mingliang Liu
Wei-Chiu Chuang
Andrew Wang
Jing Zhao
Xiao Chen
Xiaoyu Yao
Arpit Agarwal
Anu Engineer
Zhe Zhang
Yongjun Zhang
Wangda Tan
Masatake Iwasaki
Konstantin V Shvachko
Kai Zheng
Akira Ajisaka
Lei Xu
Allen Wittenauer
Chris Douglas
Ming Ma
Daryn Sharp
Jason Lowe
Eric Yang
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
Yuanbo Liu
Tsz-Wo Nicholas Sze
Steve Loughran
Vinitha Reddy Gankidi
Uma Maheswara Rao G
Jitendra Pandey
Colin Patrick Mccabe
Sangjin Lee
Ravi Prakash
Karthik Kambatla
Karthik Kambatla
Junping Du
Ravi Prakash
Walter Su
Xiaobing Zhou
Eric Payne
Haohui Mai
Zhe Zhang