Commit Graph

2921 Commits

Author SHA1 Message Date
cnauroth fef596df03 HDFS-8055. NullPointerException when topology script is missing. Contributed by Anu Engineer. 2015-04-14 10:19:30 -07:00
cnauroth d45aa7647b HDFS-6666. Abort NameNode and DataNode startup if security is enabled but block access token is not enabled. Contributed by Vijay Bhat. 2015-04-14 09:59:01 -07:00
Arpit Agarwal 18a3dad44a HDFS-7701. Support reporting per storage type quota and usage with hadoop/hdfs shell. (Contributed by Peter Shi) 2015-04-13 21:01:15 -07:00
Andrew Wang d60e22152a HDFS-8117. More accurate verification in SimulatedFSDataset: replace DEFAULT_DATABYTE with patterned data. Contributed by Zhe Zhang. 2015-04-13 13:01:10 -07:00
Tsz-Wo Nicholas Sze 7fc50e2525 HDFS-8083. Move dfs.client.write.* conf from DFSConfigKeys to HdfsClientConfigKeys.Write. 2015-04-13 11:43:35 -07:00
Vinayakumar B 14384f5b51 HDFS-8111. NPE thrown when invalid FSImage filename given for 'hdfs oiv_legacy' cmd ( Contributed by surendra singh lilhore ) 2015-04-13 17:16:05 +05:30
cnauroth f8f5887209 HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. 2015-04-11 13:23:18 -07:00
Tsz-Wo Nicholas Sze 60da0e49e7 HDFS-8084. Move dfs.client.failover.* confs from DFSConfigKeys to HdfsClientConfigKeys.Failover and fix typos in the dfs.http.client.* configuration keys. 2015-04-10 19:38:43 -07:00
Haohui Mai 36e4cd3be6 HDFS-8103. Move BlockTokenSecretManager.AccessMode into BlockTokenIdentifier. Contributed by Haohui Mai. 2015-04-10 16:36:45 -07:00
Tsz-Wo Nicholas Sze 2cc9514ad6 HDFS-8100. Refactor DFSClient.Conf to a standalone class and separates short-circuit related conf to ShortCircuitConf. 2015-04-10 14:48:45 -07:00
Arun Suresh 7660da95cb HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh) 2015-04-10 11:40:25 -07:00
Vinayakumar B 987c9e12e1 HDFS-7939. Two fsimage_rollback_* files are created which are not deleted after rollback. (Contributed by J.Andreina) 2015-04-10 11:12:07 +05:30
Konstantin V Shvachko 0959b67f1a HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko 2015-04-09 22:00:20 -07:00
Haohui Mai af9d4fede5 HDFS-8102. Separate webhdfs retry configuration keys from DFSConfigKeys. Contributed by Haohui Mai. 2015-04-09 14:36:27 -07:00
Vinod Kumar Vavilapalli 623fd46c1e Adding release 2.7.1 to CHANGES.txt 2015-04-09 13:54:20 -07:00
Arun Suresh a813db0b1b Fix CHANGES.txt for HDFS-8091 2015-04-09 13:51:00 -07:00
Arun Suresh 922b7ed21d HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh) 2015-04-09 12:38:33 -07:00
Colin Patrick Mccabe 61dc2ea3fe HDFS-8063: Fix intermittent test failures in TestTracing (Masatake Iwasaki via Colin P. McCabe) 2015-04-09 11:28:02 -07:00
Colin Patrick Mccabe 30acb7372a HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) 2015-04-09 11:22:39 -07:00
Aaron T. Myers 3fe61e0bb0 HDFS-8101. DFSClient use of non-constant DFSConfigKeys pulls in WebHDFS classes at runtime. Contributed by Sean Busbey. 2015-04-09 09:40:08 -07:00
Vinayakumar B 9d8952f97f HDFS-8096. DatanodeMetrics#blocksReplicated will get incremented early and even for failed transfers (Contributed by Vinayakumar B) 2015-04-09 11:58:00 +05:30
Andrew Wang b1e059089d HDFS-7979. Initialize block report IDs with a random number. 2015-04-08 21:43:42 -07:00
Haohui Mai cc25823546 HDFS-8089. Move o.a.h.hdfs.web.resources.* to the client jars. Contributed by Haohui Mai. 2015-04-08 16:30:08 -07:00
Andrew Wang 5a540c3d31 HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma. 2015-04-08 16:09:17 -07:00
Haohui Mai 82d56b337d Revert HDFS-7813. 2015-04-08 16:02:45 -07:00
Haohui Mai bd4c99bece Revert HDFS-7808. 2015-04-08 15:59:55 -07:00
Andrew Wang 6af0d74a75 HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma. 2015-04-08 15:52:06 -07:00
Andrew Wang a42bb1cd91 HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. 2015-04-08 15:41:48 -07:00
Kihwal Lee 285b31e75e HDFS-8046. Allow better control of getContentSummary. Contributed by Kihwal Lee. 2015-04-08 15:38:29 -05:00
Tsz-Wo Nicholas Sze c931a3c776 HDFS-8079. Move CorruptFileBlockIterator to a new hdfs.client.impl package. 2015-04-08 11:50:52 -07:00
Arpit Agarwal 608c499841 HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal) 2015-04-08 11:38:21 -07:00
Haohui Mai ab04ff9efe HDFS-8080. Separate JSON related routines used by WebHdfsFileSystem to a package local class. Contributed by Haohui Mai. 2015-04-07 21:30:57 -07:00
Vinayakumar B d505c8acd3 HDFS-8073. Split BlockPlacementPolicyDefault.chooseTarget(..) so it can be easily overrided. (Contributed by Walter Su) 2015-04-08 09:56:37 +05:30
Tsz-Wo Nicholas Sze 4be648b55c HDFS-8079. Move dfs.client.retry.* confs from DFSConfigKeys to HdfsClientConfigKeys.Retry. 2015-04-07 19:48:57 -07:00
Yongjun Zhang 66763bb06f HDFS-5215. dfs.datanode.du.reserved is not considered while computing
available space ( Brahma Reddy Battula via Yongjun Zhang)
2015-04-07 19:31:58 -07:00
cnauroth 1e72d98c69 HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao. 2015-04-07 14:23:01 -07:00
Tsz-Wo Nicholas Sze 571a1ce9d0 HDFS-8049. Add @InterfaceAudience.Private annotation to hdfs client implementation. Contributed by Takuya Fukudome 2015-04-07 13:59:48 -07:00
Konstantin V Shvachko 75c5454860 HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko. 2015-04-06 22:20:44 -07:00
Colin Patrick Mccabe 28bebc81db HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe) 2015-04-06 08:56:52 -07:00
Colin Patrick Mccabe ef591b1d6a HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe) 2015-04-03 16:34:23 -07:00
Colin Patrick Mccabe 023133cef9 HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe) 2015-04-03 14:19:46 -07:00
cnauroth db80e42891 HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao. 2015-04-03 10:44:02 -07:00
cnauroth dc5596c36a HDFS-8034. Fix TestDFSClientRetries#testDFSClientConfigurationLocateFollowingBlockInitialDelay for Windows. Contributed by Xiaoyu Yao. 2015-04-03 10:10:11 -07:00
Jing Zhao 72f6bd4893 HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao. 2015-04-02 21:20:30 -07:00
Tsz-Wo Nicholas Sze 4f412f487c HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao 2015-04-02 11:11:34 -07:00
Tsz-Wo Nicholas Sze 9ed43f2189 HDFS-7888. Change DFSOutputStream and DataStreamer for convenience of subclassing. Contributed by Li Bo 2015-04-02 10:59:26 -07:00
Vinayakumar B 867d5d2675 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 2015-04-02 08:12:00 +05:30
Brandon Li 4d14816c26 HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot 2015-04-01 17:20:24 -07:00
Haohui Mai 6ccf4fbf8a HDFS-8008. Support client-side back off when the datanodes are congested. Contributed by Haohui Mai. 2015-04-01 16:54:46 -07:00
Haohui Mai 75cb1d42ab HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:50:59 -07:00
Colin Patrick Mccabe 3c7adaaf35 HDFS-7922. ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors (Rakesh R via Colin P. McCabe) 2015-04-01 16:02:39 -07:00
Colin Patrick Mccabe c94d594a57 HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk (cmccabe) 2015-04-01 14:10:10 -07:00
Andrew Wang ed72daa5df HDFS-7978. Add LOG.isDebugEnabled() guard for some LOG.debug(..). Contributed by Walter Su. 2015-04-01 12:54:28 -07:00
Haohui Mai 53471d462c HDFS-8009. Signal congestion on the DataNode. Contributed by Haohui Mai. 2015-04-01 10:56:53 -07:00
Akira Ajisaka 18a91fe4df HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 2015-04-01 09:07:28 +09:00
Akira Ajisaka 859cab2f22 HDFS-7671. hdfs user guide should point to the common rack awareness doc. Contributed by Kai Sasaki. 2015-04-01 00:26:16 +09:00
yliu 3df61f303a HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) 2015-03-31 21:17:44 +08:00
Arpit Agarwal 85dc3c14b2 HDFS-7944. Minor cleanup of BlockPoolManager#getAllNamenodeThreads. (Arpit Agarwal) 2015-03-30 21:24:51 -07:00
Harsh J cce66ba3c9 HDFS-3918. EditLogTailer shouldn't log WARN when other node is in standby mode. Contributed by Todd Lipcon. 2015-03-31 08:04:18 +05:30
Arpit Agarwal 1a495fbb48 HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 2015-03-30 15:25:16 -07:00
Haohui Mai b80457158d HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. 2015-03-30 11:59:21 -07:00
Colin Patrick Mccabe 1feb9569f3 HDFS-7261. storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState() (Brahma Reddy Battula via Colin P. McCabe) 2015-03-30 10:46:21 -07:00
Akira Ajisaka e7ea2a8e8f HDFS-8002. Website refers to /trash directory. Contributd by Brahma Reddy Battula. 2015-03-31 00:27:50 +09:00
Kihwal Lee 04ee18ed48 HDFS-7742. Favoring decommissioning node for replication can cause a block to stay
underreplicated for long periods. Contributed by Nathan Roberts.
2015-03-30 10:10:11 -05:00
Harsh J ae3e8c61ff HDFS-4396. Add START_MSG/SHUTDOWN_MSG for ZKFC. Contributed by Liang Xie. 2015-03-30 15:21:18 +05:30
Vinayakumar B 1ed9fb7664 HDFS-7890. Improve information on Top users for metrics in RollingWindowsManager and lower log level (Contributed by J.Andreina) 2015-03-30 10:02:48 +05:30
Akira Ajisaka 232eca944a HDFS-6408. Remove redundant definitions in log4j.properties. Contributed by Abhiraj Butala. 2015-03-30 11:25:35 +09:00
Akira Ajisaka 257c77f895 HDFS-6263. Remove DRFA.MaxBackupIndex config from log4j.properties. Contributed by Abhiraj Butala. 2015-03-30 10:52:15 +09:00
Harsh J 3d9132d434 HDFS-7501. TransactionsSinceLastCheckpoint can be negative on SBNs. Contributed by Gautam Gopalakrishnan. 2015-03-29 00:45:01 +05:30
Arpit Agarwal 89fb0f57ef HDFS-7700. Document quota support for storage types. (Contributed by Xiaoyu Yao) 2015-03-27 19:49:26 -07:00
Arun Suresh e97f8e44af HDFS-8004. Use KeyProviderCryptoExtension#warmUpEncryptedKeys when creating an encryption zone. (awang via asuresh) 2015-03-27 19:23:45 -07:00
Kihwal Lee 60882ab26d HDFS-7990. IBR delete ack should not be delayed. Contributed by Daryn Sharp. 2015-03-27 09:05:17 -05:00
Colin Patrick Mccabe 61df1b27a7 HDFS-7410. Support CreateFlags with append() to support hsync() for appending streams (Vinayakumar B via Colin P. McCabe) 2015-03-26 13:21:09 -07:00
Arpit Agarwal 3d0708bdb0 HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao) 2015-03-26 10:24:11 -07:00
Kihwal Lee 222845632b HDFS-7963. Fix expected tracing spans in TestTracing. Contributed by Masatake Iwasaki. 2015-03-26 08:44:58 -05:00
Kihwal Lee fc1031af74 HDFS-7928. Scanning blocks from disk during rolling upgrade startup takes a lot of time if disks are busy. Contributed by Rushabh Shah. 2015-03-25 14:42:59 -05:00
Jing Zhao 97e2aa2551 Move HDFS-6353 to the trunk section in CHANGES.txt 2015-03-25 11:15:44 -07:00
Jing Zhao 5e21e4ca37 HDFS-6353. Check and make checkpoint before stopping the NameNode. Contributed by Jing Zhao. 2015-03-25 10:38:00 -07:00
Haohui Mai 80278a5f85 HDFS-7985. WebHDFS should be always enabled. Contributed by Li Lu. 2015-03-24 21:55:56 -07:00
Jitendra Pandey 53a28afe29 HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh. 2015-03-24 16:02:40 -07:00
Haohui Mai e38ef70fbc HDFS-7713. Implement mkdirs in the HDFS Web UI. Contributed by Ravi Prakash. 2015-03-24 15:48:52 -07:00
Jing Zhao a16bfff71b HDFS-7854. Separate class DataStreamer out of DFSOutputStream. Contributed by Li Bo. 2015-03-24 11:06:13 -07:00
Brandon Li 570a83ae80 HDFS-7977. NFS couldn't take percentile intervals. Contributed by Brandon Li 2015-03-24 10:49:16 -07:00
Harsh J eda02540ce HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. 2015-03-24 23:06:18 +05:30
Brandon Li 51f1f49374 HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li 2015-03-24 10:28:38 -07:00
Andrew Wang 6413d34986 HDFS-7961. Trigger full block report after hot swapping disk. Contributed by Eddy Xu. 2015-03-24 09:07:02 -07:00
Vinayakumar B c6c396fcd6 HDFS-3325. When configuring 'dfs.namenode.safemode.threshold-pct' to a value greater or equal to 1 there is mismatch in the UI report (Contributed by J.Andreina) 2015-03-24 12:12:01 +05:30
Plamen Jeliazkov 970ee3fc56 HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov. 2015-03-23 23:05:21 -07:00
Andrew Wang 50ee8f4e67 HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu. 2015-03-23 22:00:34 -07:00
Tsz-Wo Nicholas Sze d7e3c3364e HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula 2015-03-24 13:49:17 +09:00
cnauroth 2c238ae4e0 HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu. 2015-03-23 16:29:51 -07:00
Brandon Li 36af4a913c HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts. Contributed by Brandon Li 2015-03-23 10:06:47 -07:00
Kihwal Lee 75ead273be HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B 2015-03-20 14:02:09 -05:00
Jing Zhao d368d3647a HDFS-7957. Truncate should verify quota before making changes. Contributed by Jing Zhao. 2015-03-20 11:50:24 -07:00
Jing Zhao a6a5aae472 HDFS-7829. Code clean up for LocatedBlock. Contributed by Takanobu Asanuma. 2015-03-20 10:50:19 -07:00
Yongjun Zhang 15612313f5 HDFS-7835. make initial sleeptime in locateFollowingBlock configurable for DFSClient. Contributed by Zhihai Xu. 2015-03-20 09:12:42 -07:00
yliu 978ef11f26 HDFS-7962. Remove duplicated logs in BlockManager. (yliu) 2015-03-20 05:29:25 +08:00
yliu e37ca221bf HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu) 2015-03-19 23:23:19 +08:00
Kihwal Lee 61a4c7fc98 HDFS-7932. Speed up the shutdown of datanode during rolling upgrade. Contributed by Kihwal Lee. 2015-03-19 12:27:03 -05:00
Vinayakumar B 5be5129d93 HDFS.7849. Update documentation for enabling a new feature in rolling upgrade ( Contributed by J.Andreina ) 2015-03-19 19:05:42 +05:30