Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,798 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

6672 Commits

Author SHA1 Message Date
Inigo Goiri b3290c41a9 HDFS-12808. Add LOG.isDebugEnabled() guard for LOG.debug(...). Contributed by Bharat Viswanadham. 2018-01-08 17:51:09 -08:00
Manoj Govindassamy 73ff09b79a HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 2018-01-08 15:34:00 -08:00
Wei Yan 2ee0d64ace HDFS-12945. Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods. Contributed by Wei Yan. 2018-01-08 14:25:39 -08:00
Yiqun Lin bf5c948995 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-06 14:31:08 +08:00
Inigo Goiri 83b513ac6d HDFS-12988. RBF: Mount table entries not properly updated in the local cache. Contributed by Inigo Goiri. 2018-01-05 09:11:38 -08:00
Lei Xu dc735b286b HDFS-12860. StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB. (Contributed by Lei (Eddy) Xu) 2018-01-04 10:54:56 -08:00
Arpit Agarwal 739d3c394d HDFS-12987. Document - Disabling the Lazy persist file scrubber.. Contributed by Karthik Palanisamy. 2018-01-04 10:32:00 -08:00
Sean Mackrory 45a47198b4 HDFS-12913. TestDNFencingWithReplication sometimes fails because of NameNode in standby state. Contributed by Zsolt Venczel. 2018-01-04 08:37:21 -07:00
Yiqun Lin 2a48b3594c HDFS-12948. DiskBalancer report command top option should only take positive numeric values. Contributed by Shashikant Banerjee. 2018-01-04 10:48:44 +08:00
Xiaoyu Yao 3ba985997d HDFS-12931. Handle InvalidEncryptionKeyException during DistributedFileSystem#getFileChecksum. Contributed by Mukul Kumar Singh. 2018-01-03 14:54:20 -08:00
Manoj Govindassamy 5c28804b95 HDFS-12629. NameNode UI should report total blocks count by type - replicated and erasure coded. 2018-01-02 15:05:27 -08:00
Manoj Govindassamy 42a1c98597 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-02 14:59:36 -08:00
Arpit Agarwal b4d11337c9 HDFS-12351. Explicitly describe the minimal number of DataNodes required to support an EC policy in EC document.. Contributed by Hanisha Koneru. 2018-01-02 12:54:40 -08:00
Chris Douglas 7fe6f83c8f HADOOP-15117. open(PathHandle) contract test should be exhaustive for default options 2017-12-30 17:58:35 -08:00
Lei Xu 6e3e1b8cde HDFS-12915. Fix findbugs warning in INodeFile$HeaderFormat.getBlockLayoutRedundancy. (Contributed by Chris Douglas) 2017-12-29 12:21:57 -08:00
Xiao Chen 5bf7e594d7 HDFS-9023. When NN is not able to identify DN for replication, reason behind it can be logged. 2017-12-28 11:54:08 -08:00
Manoj Govindassamy 76e664e931 HDFS-12959. Fix TestOpenFilesWithSnapshot redundant configurations. 2017-12-21 15:47:15 -08:00
Chen Liang 826507c41b HDFS-12951. Incorrect javadoc in SaslDataTransferServer.java#receive. Contributed by Mukul Kumar Singh. 2017-12-21 11:20:30 -08:00
Lei Xu b318bed01a HDFS-12938. TestErasureCodigCLI testAll failing consistently. (Contributed by Ajay Kumar) 2017-12-21 10:28:24 -08:00
Wei Yan a78db99190 HDFS-12932. Fix confusing LOG message for block replication. Contributed by Chao Sun. 2017-12-20 08:55:46 -08:00
Yiqun Lin e040c97b77 HDFS-12937. RBF: Add more unit tests for router admin commands. Contributed by Yiqun Lin. 2017-12-19 15:31:34 +08:00
Junping Du a7f8caf58e Add 2.8.3 release jdiff files. 
(cherry picked from commit c89f99aade)
 2017-12-18 22:39:01 -08:00
Yiqun Lin 25a36b7452 HDFS-12930. Remove the extra space in HdfsImageViewer.md. Contributed by Rahul Pathak. 2017-12-19 11:23:16 +08:00
Tsz-Wo Nicholas Sze c7499f2d24 HDFS-12347. TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently. Contributed by Bharat Viswanadham 2017-12-19 10:02:30 +08:00
Erik Krogen 94576b17fb HDFS-12818. Support multiple storages in DataNodeCluster / SimulatedFSDataset. Contributed by Erik Krogen. 2017-12-18 11:36:22 -08:00
Virajith Jalaparti 8239e3afb3 HDFS-12712. [9806] Code style cleanup 2017-12-15 17:51:41 -08:00
Virajith Jalaparti 80c3fec3a1 HDFS-12912. [READ] Fix configuration and implementation of LevelDB-based alias maps 2017-12-15 17:51:41 -08:00
Virajith Jalaparti c89b29bd42 HDFS-12893. [READ] Support replication of Provided blocks with non-default topologies. 2017-12-15 17:51:41 -08:00
Virajith Jalaparti 0f6aa9564c HDFS-12905. [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage. 2017-12-15 17:51:41 -08:00
Chris Douglas 2298f2d76b HDFS-12874. Documentation for provided storage. Contributed by Virajith Jalaparti 2017-12-15 17:51:41 -08:00
Virajith Jalaparti 71ec170107 HDFS-12887. [READ] Allow Datanodes with Provided volumes to start when blocks with the same id exist locally 2017-12-15 17:51:41 -08:00
Virajith Jalaparti 4531588a94 HDFS-11640. [READ] Datanodes should use a unique identifier when reading from external stores 2017-12-15 17:51:41 -08:00
Virajith Jalaparti fb996a32a9 HDFS-12894. [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update 2017-12-15 17:51:40 -08:00
Virajith Jalaparti 9c35be86e1 HDFS-12713. [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata. Contributed by Ewan Higgs 2017-12-15 17:51:40 -08:00
Virajith Jalaparti a027055dd2 HDFS-12885. Add visibility/stability annotations. Contributed by Chris Douglas 2017-12-15 17:51:40 -08:00
Virajith Jalaparti b634053c4d HDFS-12591. [READ] Implement LevelDBFileRegionFormat. Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti 352f994b64 HDFS-12665. [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb). Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti cc933cba77 HDFS-12685. [READ] FsVolumeImpl exception when scanning Provided storage volume 2017-12-15 17:51:40 -08:00
Virajith Jalaparti 4d59dabb7f HDFS-12809. [READ] Fix the randomized selection of locations in {{ProvidedBlocksBuilder}}. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti 3d3be87e30 HDFS-12778. [READ] Report multiple locations for PROVIDED blocks 2017-12-15 17:51:39 -08:00
Virajith Jalaparti 3b1d30301b HDFS-12775. [READ] Fix reporting of Provided volumes 2017-12-15 17:51:39 -08:00
Virajith Jalaparti e1a28f95b8 HDFS-12777. [READ] Reduce memory and CPU footprint for PROVIDED volumes. 2017-12-15 17:51:39 -08:00
Virajith Jalaparti 6cd80b2521 HDFS-12779. [READ] Allow cluster id to be specified to the Image generation tool 2017-12-15 17:51:39 -08:00
Virajith Jalaparti 90d1b47a2a HDFS-12776. [READ] Increasing replication for PROVIDED files should create local replicas 2017-12-15 17:51:39 -08:00
Virajith Jalaparti 71d0a82571 HDFS-12607. [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED 2017-12-15 17:51:39 -08:00
Virajith Jalaparti 98f5ed5aa3 HDFS-11902. [READ] Merge BlockFormatProvider and FileRegionProvider. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti d6a9a89973 HDFS-12605. [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase 2017-12-15 17:51:38 -08:00
Virajith Jalaparti aca023b72c HDFS-12289. [READ] HDFS-12091 breaks the tests for provided block reads 2017-12-15 17:51:38 -08:00
Virajith Jalaparti 2407c9b93a HDFS-12093. [READ] Share remoteFS between ProvidedReplica instances. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti 663b3c08b1 HDFS-12091. [READ] Check that the replicas served from a ProvidedVolumeImpl belong to the correct external storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti 546b95f484 HDFS-11673. [READ] Handle failures of Datanode with PROVIDED storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti 55ade54b8e HDFS-11792. [READ] Test cases for ProvidedVolumeDF and ProviderBlockIteratorImpl 2017-12-15 17:51:38 -08:00
Virajith Jalaparti 89b9faf529 HDFS-11703. [READ] Tests for ProvidedStorageMap 2017-12-15 17:51:37 -08:00
Virajith Jalaparti aa5ec85f7f HDFS-11663. [READ] Fix NullPointerException in ProvidedBlocksBuilder 2017-12-15 17:51:37 -08:00
Virajith Jalaparti 1108cb7691 HDFS-11653. [READ] ProvidedReplica should return an InputStream that is bounded by its length 2017-12-15 17:51:37 -08:00
Virajith Jalaparti d65df0f273 HDFS-11190. [READ] Namenode support for data stored in external stores. 2017-12-15 17:51:37 -08:00
Virajith Jalaparti b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Chen Liang aa503a29d0 HDFS-12917. Fix description errors in testErasureCodingConf.xml. Contributed by chencan. 2017-12-15 13:45:37 -08:00
Lei Xu 949be14b08 HDFS-12927. Update erasure coding doc to address unsupported APIs. (Contributed by Lei (Eddy) Xu) 2017-12-15 10:20:29 -08:00
Lei Xu 1c15b1751c HDFS-12819. Setting/Unsetting EC policy shows warning if the directory is not empty. (Contributed by Lei (Eddy) Xu) 2017-12-15 10:04:43 -08:00
Yiqun Lin ee028bfdf1 HDFS-12895. RBF: Add ACL support for mount table. Contributed by Yiqun Lin. 2017-12-15 14:09:24 +08:00
Xiao Chen e1cb278cd0 HDFS-12910. Secure Datanode Starter should log the port when it fails to bind. Contributed by Stephen O'Donnell and Nanda kumar. 2017-12-14 19:22:51 -08:00
Kihwal Lee f5a72424c0 HDFS-12907. Allow read-only access to reserved raw for non-superusers. Contributed by Rushabh S Shah. 2017-12-14 14:58:47 -06:00
Jason Lowe 80db744ee5 HDFS-12881. Output streams closed with IOUtils suppressing write errors. Contributed by Ajay Kumar 2017-12-14 09:45:44 -06:00
Yiqun Lin 91c96bdf8e HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 2017-12-14 16:00:53 +08:00
Chen Liang 46e18c8da7 HADOOP-14914. Change to a safely casting long to int. Contributed by Ajay Kumar. 2017-12-13 14:56:14 -08:00
Andrew Wang d447152d49 Update CHANGES, RELEASENOTES, jdiff for 3.0.0 release. 2017-12-13 13:05:32 -08:00
Chris Douglas 693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Wei-Chiu Chuang 55fc2d6485 HDFS-12891. Do not invalidate blocks if toInvalidate is empty. Contributed by Zsolt Venczel. 2017-12-11 16:43:03 -08:00
Inigo Goiri 5cd1056ad7 HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 2017-12-11 15:14:57 -08:00
Manoj Govindassamy ef7d334d36 HDFS-12825. Fsck report shows config key name for min replication issues (Contributed by Gabor Bota). 2017-12-08 11:46:30 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) f19638333b HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 2017-12-08 08:00:21 -08:00
Xiao Chen 19e0894209 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 2017-12-07 21:05:55 -08:00
Wei-Chiu Chuang d6c31a3e6b HDFS-11915. Sync rbw dir on the first hsync() to avoid file lost on power failure. Contributed by Vinayakumar B. 2017-12-07 14:50:30 -08:00
Lei Xu 67662e2ac9 HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 2017-12-07 11:15:40 -08:00
Akira Ajisaka 6cca5b3bcb
HDFS-12889. Addendum patch to add missing file. 2017-12-07 13:23:37 +09:00
Xiao Chen 56b1ff80dd HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 2017-12-05 20:48:16 -08:00
Akira Ajisaka 44b06d34a5
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham. 2017-12-06 11:40:33 +09:00
Brahma Reddy Battula 6d16a99ae8 HDFS-11751. DFSZKFailoverController daemon exits with wrong status code. Contributed by Bharat Viswanadham 2017-12-05 19:53:46 +05:30
Arpit Agarwal e00c7f78c1 HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts. 2017-12-04 21:02:04 -08:00
Xiao Chen 404eab4dc0 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:11 -08:00
Chris Douglas 42307e3c3a HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 22:34:30 -08:00
Wei-Chiu Chuang 0faf506245 HDFS-12836. startTxId could be greater than endTxId when tailing in-progress edit log. Contributed by Chao Sun. 2017-12-01 12:01:21 -08:00
Chris Douglas 53bbef3802 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 5304698dc8.
 2017-12-01 11:19:38 -08:00
Chris Douglas 5304698dc8 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 10:29:30 -08:00
Konstantin V Shvachko 60fd0d7fd7 HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 2017-11-30 18:18:28 -08:00
Tsz-Wo Nicholas Sze b1c7654ee4 HDFS-12594. snapshotDiff fails if the report exceeds the RPC response limit. Contributed by Shashikant Banerjee 2017-11-30 12:18:29 -08:00
Chris Douglas 0e560f3b8d HDFS-12681. Make HdfsLocatedFileStatus a subtype of LocatedFileStatus 2017-11-29 20:28:06 -08:00
Kihwal Lee 53509f295b HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 2017-11-29 15:11:14 -06:00
Wei Yan 301641811d HDFS-12835. Fix the javadoc errors in Router-based federation. 2017-11-29 09:43:03 -08:00
Konstantin V Shvachko d331762f24 HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 2017-11-28 17:14:23 -08:00
Surendra Singh Lilhore 30941d99c9 HDFS-12857. StoragePolicyAdmin should support schema based path. Contributed by Surendra Singh Lilhore. 2017-11-28 23:57:03 +05:30
Yiqun Lin 94bed50471 HDFS-12858. Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 2017-11-28 11:48:55 +08:00
Kihwal Lee 738d1a206a HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 2017-11-22 13:01:55 -06:00
Lei Xu 785732c13e HDFS-12847. Regenerate editsStored and editsStored.xml in HDFS tests. Contributed by Lei (Eddy) Xu. 2017-11-22 10:22:32 -08:00
Chen Liang 60fc2a1388 HDFS-12804. Use slf4j instead of log4j in FSEditLog. Contributed by Mukul Kumar Singh. 2017-11-20 12:49:53 -08:00
Manoj Govindassamy 9fb4effd2c HDFS-12730. Verify open files captured in the snapshots across config disable and enable. 2017-11-20 10:28:12 -08:00
Inigo Goiri e182e77794 HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 2017-11-16 16:58:47 -08:00
Weiwei Yang 462e25a3b2 HDFS-12814. Add blockId when warning slow mirror/disk in BlockReceiver. Contributed by Jiandan Yang. 2017-11-16 16:19:53 +08:00
Chris Douglas 675e9a8f57 Revert "HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus." 
This reverts commit b85603e3f8.
 2017-11-15 19:20:37 -08:00