Commit Graph

5471 Commits

Author SHA1 Message Date
Kihwal Lee 0d1c1152f1 HDFS-10282. The VolumeScanner should warn about replica files which are misplaced. Contributed by Colin Patrick McCabe. 2016-04-14 07:58:24 -05:00
Anu Engineer ae6c0e839b Merge branch 'trunk' into HDFS-7240 2016-04-13 23:13:11 -07:00
Xiaoyu Yao 5566177c9a HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode. Contributed by Xiaobing Zhou. 2016-04-13 16:51:13 -07:00
Xiaoyu Yao 4895c73dd4 Revert "HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode"
This reverts commit 192112d5a2.
2016-04-13 16:45:18 -07:00
Andrew Wang 314aa21a89 HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun. 2016-04-13 16:39:50 -07:00
Xiaobing Zhou 192112d5a2 HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode 2016-04-13 15:34:41 -07:00
Kihwal Lee d2f3bbc290 HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák 2016-04-13 11:22:36 -05:00
Walter Su 903428bf94 HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 2016-04-13 17:49:45 +08:00
Anu Engineer eee7269748 HDFS-10278. Ozone: Add paging support to list Volumes. Contributed by Anu Engineer. 2016-04-12 18:08:48 -07:00
Anu Engineer 33b51da8c5 HDFS-10251. Ozone: Shutdown cleanly. Contributed by Anu Engineer 2016-04-12 18:00:24 -07:00
Colin Patrick Mccabe 35f0770555 HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe) 2016-04-12 16:28:42 -07:00
Zhe Zhang 6ef42873a0 HDFS-9918. Erasure Coding: Sort located striped blocks based on decommissioned states. Contributed by Rakesh R. 2016-04-12 13:38:58 -07:00
Colin Patrick Mccabe 600d129bb8 HDFS-10273. Remove duplicate logSync() and log message in FSN#enterSafemode() (Vinayakumar B via cmccabe) 2016-04-12 10:52:39 -07:00
Akira Ajisaka 209303be3a HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang. 2016-04-12 15:26:50 +09:00
Vinayakumar B a9a607f8fc HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 2016-04-12 11:44:34 +05:30
Akira Ajisaka 9c32f8785e HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang. 2016-04-08 15:32:13 +09:00
Anu Engineer 3f62ba558d Merge branch 'trunk' into HDFS-7240 2016-04-07 14:43:39 -07:00
Anu Engineer fedb22d9b6 HDFS-10268. Ozone: end-to-end integration for create/get volumes, buckets and keys. Contributed by Chris Nauroth. 2016-04-07 14:38:54 -07:00
Uma Maheswara Rao G 3c18a53cbd HDFS-9719. Refactoring ErasureCodingWorker into smaller reusable constructs. Contributed by Kai Zheng. 2016-04-06 22:50:24 -07:00
Tsz-Wo Nicholas Sze 654cd1d0c0 HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:38:42 +08:00
Colin Patrick Mccabe 4bd7cbc29d HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose 2016-04-06 21:07:31 -07:00
Eric Payne aede8c10ec HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee 2016-04-06 20:20:14 +00:00
Colin Patrick Mccabe 188f65287d HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe) 2016-04-06 11:28:34 -07:00
Jing Zhao 221b3a8722 HDFS-10192. Namenode safemode not coming out during failover. Contributed by Brahma Reddy Battula. 2016-04-06 10:42:59 -07:00
Kihwal Lee 9ba1e5af06 HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger. 2016-04-05 16:26:18 -05:00
Ravi Prakash 0cd320a846 HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula. 2016-04-05 13:41:19 -07:00
Chris Nauroth b3044db407 HDFS-10250. Ozone: Add key Persistence. Contributed by Anu Engineer. 2016-04-05 12:59:45 -07:00
Kihwal Lee 917464505c HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla. 2016-04-05 09:07:24 -05:00
Vinayakumar B 818d6b799e HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula) 2016-04-05 09:49:39 +08:00
Colin Patrick Mccabe f6b1a81812 HDFS-8496. Calling stopWriter() with FSDatasetImpl lock held may block other threads (cmccabe) 2016-04-04 18:02:15 -07:00
Kihwal Lee a7d1fb0cd2 HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee. 2016-04-04 16:40:00 -05:00
Masatake Iwasaki 154d2532cf HDFS-9599. TestDecommissioningStatus.testDecommissionStatus occasionally fails (Lin Yiqun via iwasakims) 2016-04-05 03:19:48 +09:00
Vinayakumar B 54b2e78fd2 HDFS-10253. Fix TestRefreshCallQueue failure (Contributed by Xiaoyu Yao) 2016-04-03 13:27:49 +05:30
Chris Nauroth 4ac97b1815 HDFS-10238. Ozone : Add chunk persistance. Contributed by Anu Engineer. 2016-04-01 10:52:38 -07:00
Allen Wittenauer 0a74610d1c HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw) 2016-03-31 07:51:05 -07:00
Akira Ajisaka 32c0c3ecdf HDFS-10221. Add .json to the rat exclusions. Contributed by Ming Ma. 2016-03-31 09:06:14 +09:00
Anu Engineer 55d4ee71e2 HDFS-8210. Ozone: Implement storage container manager. Contributed by Chris Nauroth. 2016-03-30 14:49:55 -07:00
Colin Patrick Mccabe 37e23ce45c HDFS-10223. peerFromSocketAndKey performs SASL exchange before setting connection timeouts (cmccabe) 2016-03-30 13:37:37 -07:00
Vinayakumar B 09d63d5a19 HDFS-5177. blocksScheduled count should be decremented for abandoned blocks (Contributed by Vinayakumar B) 2016-03-30 14:22:00 +08:00
Arpit Agarwal ddfe6774c2 HDFS-9439. Support reconfiguring fs.protected.directories without NN restart. (Contributed by Xiaobing Zhou) 2016-03-29 13:55:00 -07:00
Arpit Agarwal c9307e48b7 HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA) 2016-03-29 13:50:08 -07:00
Andrew Wang f2aec4eb82 HDFS-10197. TestFsDatasetCache failing intermittently due to timeout. Contributed by Lin Yiqun. 2016-03-29 11:54:42 -07:00
Vinayakumar B 1f004b3367 HDFS-9871. "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. (Contributed by Brahma Reddy Battulla) 2016-03-29 11:25:41 +08:00
Anu Engineer 30473ecfb4 Merge branch 'trunk' into HDFS-7240 2016-03-28 10:34:52 -07:00
Uma Maheswara Rao G 3a4ff7776e HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng 2016-03-26 19:58:09 -07:00
Chris Nauroth 643c5e5bdc HDFS-10195. Ozone: Add container persistence. Contributed by Anu Engineer. 2016-03-26 11:40:00 -07:00
Arpit Agarwal a337ceb74e Revert "HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng"
This reverts commit e5ff0ea7ba.
2016-03-26 09:20:01 -07:00
Uma Maheswara Rao G e5ff0ea7ba HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng 2016-03-26 00:52:50 -07:00
Lei Xu fde8ac5d85 Add missing files from HDFS-9005. (lei) 2016-03-25 17:11:35 -07:00
Lei Xu 4fcfea71bf HDFS-9005. Provide support for upgrade domain script. (Ming Ma via Lei Xu) 2016-03-25 17:09:12 -07:00
Chris Nauroth 2dc48b7f1e HDFS-10202. ozone : Add key commands to CLI. Contributed by Anu Engineer. 2016-03-25 09:50:58 -07:00
Chris Nauroth 58cdb715fb HDFS-10196. Ozone : Enable better error reporting for failed commands in ozone shell. Contributed by Anu Engineer. 2016-03-24 09:27:10 -07:00
Allen Wittenauer 738155063e HADOOP-12857. rework hadoop-tools (aw) 2016-03-23 13:46:38 -07:00
Andrew Wang 8f85e5d212 HDFS-10200. Docs for WebHDFS still describe GETDELEGATIONTOKENS operation. Contributed by Wellington Chevreuil. 2016-03-23 12:55:15 -07:00
Colin Patrick Mccabe a7d8f2b396 HDFS-10189. PacketResponder#toString should include the downstreams for PacketResponderType.HAS_DOWNSTREAM_IN_PIPELINE (Joe Pallas via cmccabe) 2016-03-22 14:56:02 -07:00
Colin Patrick Mccabe 680716f31e HDFS-9951. Use string constants for XML tags in OfflineImageReconstructor (Lin Yiqun via cmccabe) 2016-03-21 11:40:37 -07:00
Andrew Wang e3bb38d625 HDFS-9405. Warmup NameNode EDEK caches in background thread. Contributed by Xiao Chen. 2016-03-21 11:39:05 -07:00
Haohui Mai 8a3f0cb255 HDFS-7166. SbNN Web UI shows #Under replicated blocks and #pending deletion blocks. Contributed by Wei-Chiu Chuang. 2016-03-20 17:21:25 -07:00
Sangjin Lee cd8b6889a7 HDFS-9579. Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level (Ming Ma via sjlee) 2016-03-19 14:02:04 -07:00
Chris Nauroth 3f708bf711 HDFS-10179. Ozone: Adding logging support. Contributed by Anu Engineer. 2016-03-19 13:59:00 -07:00
Chris Nauroth 2e517a64d8 HDFS-9960. OzoneHandler : Add localstorage support for keys. Contributed by Anu Engineer. 2016-03-19 13:36:18 -07:00
Chris Nauroth c73a32c21c HDFS-10180. Ozone: Refactor container Namespace. Contributed by Anu Engineer. 2016-03-18 13:46:56 -07:00
Anu Engineer 37e3a36a3f Merge branch 'trunk' into HDFS-7240 2016-03-18 11:18:48 -07:00
Akira Ajisaka 9b623fbaf7 HDFS-3677. dfs.namenode.edits.dir.required is missing from hdfs-default.xml. Contributed by Mark Yang. 2016-03-19 01:11:15 +09:00
Kihwal Lee 63c966a3fb HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah. 2016-03-18 10:24:59 -05:00
Colin Patrick Mccabe dc951e606f HDFS-9949. Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared (Harsh J via cmccabe) 2016-03-17 10:37:42 -07:00
Andrew Wang 02a250db9f HDFS-10173. Typo in DataXceiverServer. Contributed by Michael Han. 2016-03-16 18:49:57 -07:00
Zhe Zhang 32d043d9c5 HDFS-9857. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-1]. Contributed by Rakesh R. 2016-03-16 16:53:58 -07:00
Chris Nauroth 4143f0362f HDFS-9961. Ozone: Add buckets commands to CLI. Contributed by Anu Engineer. 2016-03-15 21:27:53 -07:00
Kihwal Lee d457401784 HDFS-9904. testCheckpointCancellationDuringUpload occasionally fails. Contributed by Lin Yiqun. 2016-03-15 10:52:47 -05:00
Masatake Iwasaki 5de848cd5d HDFS-9928. Make HDFS commands guide up to date (Wei-Chiu Chuang via iwasakims) 2016-03-15 20:42:05 +09:00
Chris Nauroth 4683d68ee6 HDFS-9944. Ozone : Add container dispatcher. Contributed by Anu Engineer. 2016-03-14 16:16:35 -07:00
Ravi Prakash bd5556ba12 Download File from UI broken after pagination. Contributed by Brahma Reddy Battula 2016-03-14 14:30:13 -07:00
Chris Nauroth 657bc690c2 HDFS-9926. ozone : Add volume commands to CLI. Contributed by Anu Engineer. 2016-03-14 10:50:54 -07:00
Chris Nauroth 5644137ada HDFS-9941. Do not log StandbyException on NN, other minor logging fixes. Contributed by Arpit Agarwal. 2016-03-14 09:54:54 -07:00
Chris Nauroth 64add6756c HDFS-9925. Ozone: Add Ozone Client lib for bucket handling. Contributed by Anu Engineer.
Fix build break
2016-03-11 13:03:15 -08:00
Chris Nauroth e74f1d4728 HDFS-9916. OzoneHandler : Add Key handler. Contributed by Anu Engineer. 2016-03-11 12:57:10 -08:00
Chris Nauroth e882d6273c HDFS-9916. OzoneHandler : Add Key handler. Contributed by Anu Engineer. 2016-03-11 12:57:10 -08:00
Anu Engineer ff4ea053d3 HDFS-9920. Stop tracking CHANGES.txt in the HDFS-7240 feature branch. Contributed by Chris Nauroth. 2016-03-11 12:57:09 -08:00
Chris Nauroth 979dfe4c2e HDFS-9891. Ozone: Add container transport client. Contributed by Anu Engineer. 2016-03-11 12:57:09 -08:00
Chris Nauroth b31a5d67f1 HDFS-9873. Ozone: Add container transport server. Contributed by Anu Engineer. 2016-03-11 12:57:09 -08:00
Anu Engineer 2fff7d58da HDFS-9907. Exclude Ozone protobuf-generated classes from Findbugs analysis. Contributed by Chris Nauroth 2016-03-11 12:57:09 -08:00
Chris Nauroth d4402c5123 HDFS-9848. Ozone: Add Ozone Client lib for volume handling. Contributed by Anu Engineer. 2016-03-11 12:57:08 -08:00
Anu Engineer 45e360f54e HDFS-9853. Ozone: Add container definitions. Contributed by Anu Engineer. 2016-03-11 12:57:08 -08:00
cnauroth 40211dd4fe HDFS-9845. OzoneHandler : Support List and Info Volumes. Contributed by Anu Engineer. 2016-03-11 12:57:08 -08:00
cnauroth ed258dc633 HDFS-9834. OzoneHandler : Enable MiniDFSCluster based testing for Ozone. Contributed by Anu Engineer. 2016-03-11 12:57:08 -08:00
Anu Engineer 14b85da0ea HDFS-8757 : OzoneHandler : Add localStorageHandler support for Buckets. Contributed by Anu Engineer 2016-03-11 12:57:08 -08:00
Anu Engineer 312a01ea3c HDFS-8527. OzoneHandler: Integration of REST interface and container data pipeline back-end. Contributed by Chris Nauroth 2016-03-11 12:57:07 -08:00
Anu Engineer 114ef3000c HDFS-8695. OzoneHandler : Add Bucket REST Interface. (aengineer) 2016-03-11 12:57:07 -08:00
Anu Engineer d309930374 HDFS-8753. Ozone: Unify StorageContainerConfiguration with ozone-default.xml & ozone-site.xml. Contributed by kanaka kumar avvaru 2016-03-11 12:57:07 -08:00
Arpit Agarwal 06c5b2500a HDFS-8717. OzoneHandler : Add common bucket objects. (Contributed by Anu Engineer) 2016-03-11 12:57:07 -08:00
Arpit Agarwal acb949e6f9 HDFS-8680. OzoneHandler : Add Local StorageHandler support for volumes. (Contributed by Anu Engineer) 2016-03-11 12:57:07 -08:00
Arpit Agarwal 5d70ed415c HDFS-8654. OzoneHandler : Add ACL support. (Contributed by Anu Engineer) 2016-03-11 12:57:06 -08:00
Arpit Agarwal 476d22dfa1 HDFS-8644. OzoneHandler : Add volume handler. (Contributed by Anu Engineer) 2016-03-11 12:57:06 -08:00
Arpit Agarwal d9d9504102 HDFS-8634. OzoneHandler: Add userAuth Interface and Simple userAuth handler. (Contributed by Anu Engineer) 2016-03-11 12:57:06 -08:00
Arpit Agarwal 7452cc5d8b HDFS-8637. OzoneHandler : Add Error Table. (Contributed by Anu Engineer) 2016-03-11 12:57:06 -08:00
cnauroth f0b2b26c34 HDFS-8448. Create REST Interface for Volumes. Contributed by Anu Engineer. 2016-03-11 12:57:05 -08:00
Arpit Agarwal 15a3e75854 HDFS-8614. OzoneHandler : Add Quota Support. (Contributed by Anu Engineer) 2016-03-11 12:56:44 -08:00
Arpit Agarwal e2f494c2c4 HDFS-8456. Introduce STORAGE_CONTAINER_SERVICE as a new NodeType. (Contributed by Arpit Agarwal) 2016-03-11 12:56:29 -08:00
Arpit Agarwal e01c6ea688 HDFS-1477. Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart. (Contributed by Xiaobing Zhou) 2016-03-10 19:03:55 -08:00