Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,917 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4331 Commits

Author SHA1 Message Date
Nanda kumar 9734f505ea HDFS-12739. Add Support for SCM --init command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 709d56fdc9 HDFS-12474. Ozone: SCM: Handling container report with key count and container usage. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 2ec24a7542 HDFS-12774. Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Yiqun Lin 132f30c8ea HDFS-12443. Ozone: Improve SCM block deletion throttling algorithm. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh 7ebe79e879 HDFS-12720. Ozone: Ratis options are not passed from KSM Client protobuf helper correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang 0559265822 HDFS-12750. Ozone: Fix TestStorageContainerManager#testBlockDeletionTransactions. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer 76d34bca62 HDFS-12521. Ozone: SCM should read all Container info into memory when booting up. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer 6291ca1e32 HDFS-12663. Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer 1b56a73a4f HDFS-12680. Ozone: SCM: Lease support for container creation. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Yiqun Lin 6783dad766 HDFS-11468. Ozone: SCM: Add Node Metrics for SCM. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin 393a02d8e1 HDFS-12692. Ozone: fix javadoc/unit test for calling MetadataStore.getRangeKVs with non existing key. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang 6d9f914b55 HDFS-12684. Ozone: SCMMXBean NodeCount is overlapping with NodeManagerMXBean. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Nanda kumar cc86719451 HDFS-12675. Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao a467017068 HDFS-12411. Ozone: Add container usage information to DN container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Nanda kumar 9bb516118d HDFS-12490. Ozone: OzoneClient: Add creation/modification time information in OzoneVolume/OzoneBucket/OzoneKey. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nandakumar 8881309002 HDFS-12519. Ozone: Lease Manager framework. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh c85d3b2d25 HDFS-12593. Ozone: update Ratis to the latest snapshot. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 9445a9267f HDFS-12572. Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
yuanbo e76e10fd7a HDFS-12609. Ozone: SCM: Refactoring of chill mode logic in NodeManager. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze e3b51d9074 HDFS-12598. Ozone: Fix 3 node ratis replication in Ozone. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nandakumar 3504af9411 HDFS-12537. Ozone: Reduce key creation overhead in Corona. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Nandakumar 9630621be7 HDFS-12610. Ozone: OzoneClient: RpcClient list calls throw NPE when iterating over empty list. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Nandakumar c3ef381011 HDFS-12543. Ozone : allow create key without specifying size. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer c17521b1bd HDFS-12387. Ozone: Support Ratis as a first class replication mechanism. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer 0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer ceec14bff5 HDFS-12467. Ozone: SCM: NodeManager should log when it comes out of chill mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer 6a754caca0 HDFS-12568. Ozone: Cleanup the ozone-default.xml. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 05246e2b32 HDFS-12060. Ozone: OzoneClient: Add list calls in RpcClient. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer aaec2689f4 HDFS-12511. Ozone: Add tags to config. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang 7363c8fc09 HDFS-12540. Ozone: node status text reported by SCM is a bit confusing. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Chen Liang 9530153f33 HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer a44f3a777d HDFS-12501. Ozone: Cleanup javac issues. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin 6b10723ec3 HDFS-12525. Ozone: OzoneClient: Verify bucket/volume name in create calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer a1a3ba6529 HDFS-12454. Ozone : the sample ozone-site.xml in OzoneGettingStarted does not work. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer 2804435386 HDFS-12017. Ozone: Container: Move IPC port to 98xx range. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Chen Liang 9ff136bb02 HDFS-12210. Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang dddded0016 Revert "Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh." to fix commit message. 
This reverts commit 087c69ba24.
 2018-04-26 05:36:04 -07:00
Weiwei Yang d303b7f40f HDFS-12524. Ozone: Record number of keys scanned and hinted for getRangeKVs call. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Yiqun Lin 712bd70ac7 HDFS-12205. Ozone: List Key on an empty ozone bucket fails with command failed error. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang 8052374e79 Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang fd1564b87e HDFS-12506. Ozone: ListBucket is too slow. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang c329d3b4b4 HDFS-12329. Ozone: Ratis: Readonly calls in XceiverClientRatis should use sendReadOnly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 607d1fcd31 HDFS-12256. Ozone : handle inactive containers on DataNode. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 77d4b18700 HDFS-12385. Ozone: OzoneClient: Refactoring OzoneClient API. Contributed by Nadakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao d9c3d9a5ef HDFS-11873. Ozone: Object store handler supports reusing http client for multiple requests. Contributed by Xiaoyu Yao and Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer fe4139bdc3 HDFS-11612. Ozone: Cleanup Checkstyle issues. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Chen Liang b127ecd1ba Revert "HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin." as it uses class not part of JRE. 
This reverts commit c29aff44cf.
 2018-04-26 05:36:04 -07:00
Chen Liang f66a522695 HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang 80c50a1eac HDFS-12423. Ozone: TopN container choosing policy should ignore containers that has no pending deletion blocks. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang 1087ce4cd2 HDFS-11676. Ozone: SCM CLI: Implement close container command. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Weiwei Yang 0c17776646 HDFS-12389. Ozone: oz commandline list calls should return valid JSON format output. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang fec04f1158 HDFS-12370. Ozone: Implement TopN container choosing policy for BlockDeletionService. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang 743be0d7c0 HDFS-12235. Ozone: DeleteKey-3: KSM SCM block deletion message and ACK interactions. Contributed by Weiwei Yang and Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Chen Liang e06a9a5bb1 HDFS-12391. Ozone: TestKSMSQLCli is not working as expected. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer ca72c2342c HDFS-12366. Ozone: Refactor KSM metadata class names to avoid confusion. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang 7b3e7eb565 HDFS-12354. Ozone: Shuffle container list for datanode BlockDeletingService. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang eef437d5e2 HDFS-12282. Ozone: DeleteKey-4: Block delete via HB between SCM and DN. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer 2f4dfbc8fb HDFS-12275. Ozone: Corona: Support for random validation of writes. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer 532ced18fc HDFS-12280. Ozone: TestOzoneContainer#testCreateOzoneContainer fails. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Weiwei Yang 4d3b8d332d HDFS-12283. Ozone: DeleteKey-5: Implement SCM DeletedBlockLog. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao f51189e2bb HDFS-12324. Ozone: TestQueryNode#testStaleNodesCount is failing. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 02f8c2224f HDFS-11888. Ozone: SCM: use state machine for open containers allocated for key/blocks. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer f352b104f7 HDFS-12337. Ozone: Concurrent RocksDB open calls fail because of "No locks available". Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang 9e733215a8 HDFS-12327. Ozone: support setting timeout in background service. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer 38bc1d1097 HDFS-12216. Ozone: TestKeys is failing consistently. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer bd33038bf4 HDFS-12159. Ozone: SCM: Add create replication pipeline RPC. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer 3a661b7f82 HDFS-12238. Ozone: Add valid trace ID check in sendCommandAsync. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang 25586607a1 HDFS-12230. Ozone: KSM: Add creation time field in bucket info. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer dcc21a4f93 HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang 639b4fb8a9 HDFS-12196. Ozone: DeleteKey-2: Implement block deleting service to delete stale blocks at background. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang 285d61cd8d HDFS-12274. Ozone: Corona: move corona from test to tools package. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao bb9a4793d3 HDFS-12259. Ozone: OzoneClient: Refactor move ozone client from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer bb5f1c4228 HDFS-12231. Ozone: KSM: Add creation time field in volume info. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 6f83e9c51e HDFS-12163. Ozone: Reduce MiniOzoneCluster handler thread count. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer 9cf40547ce HDFS-11580. Ozone: Support asynchronus client API for SCM and containers. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Weiwei Yang 1ad95cf2a9 HDFS-11984. Ozone: Ensures listKey lists all required key fields. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Chen Liang 070f7e2e59 HDFS-11920. Ozone : add key partition. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer bc413a69cc HDFS-12195. Ozone: DeleteKey-1: KSM replies delete key request asynchronously. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Chen Liang 7907fe1c80 HDFS-12059. Ozone: OzoneClient: OzoneClientImpl Add setBucketProperty and delete calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer fcd4537e63 HDFS-12180. Ozone: Corona: Add stats and progress bar to corona. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang a4b3160efb HDFS-12145. Ozone: OzoneFileSystem: Ozone & KSM should support / delimited key names. Contributed by Mukul Kumar Singh. 
(cherry picked from commit fe0a957e92d506330ba005abba18f665eb5ac1cc)
 2018-04-26 05:36:04 -07:00
Yiqun Lin 8cf0c864c4 HDFS-12170. Ozone: OzoneFileSystem: KSM should maintain key creation time and modification time. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang 349a87ac9c HDFS-12192. Ozone: Fix the remaining failure tests for Windows caused by incorrect path generated. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang d50a743015 HDFS-12155. Ozone : add RocksDB support to DEBUG CLI. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer 0cc166c053 HDFS-12115. Ozone: SCM: Add queryNode RPC Call. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Weiwei Yang 539842ed8b HDFS-12187. Ozone : add support to DEBUG CLI for ksm.db. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Weiwei Yang b00830142b HDFS-12167. Ozone: Intermittent failure TestContainerPersistence#testListKey. Contributed by Surendra Singh Lilhore. 2018-04-26 05:36:04 -07:00
Anu Engineer 487aa2f7bf HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer d02a4e9dd1 HDFS-12175. Ozone: Fix Leaking in TestStorageContainerManager#testRpcPermission. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Chen Liang d05b44f417 Revert "HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao" 
This reverts commit f801454ad3.
 2018-04-26 05:36:04 -07:00
Xiaoyu Yao eda559ffd2 HDFS-12181. Ozone: Fix TestContainerReplicationManager by setting proper log level for LogCapturer. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang ca096501bf HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao 2018-04-26 05:36:04 -07:00
Weiwei Yang f1e2592bf8 HDFS-12127. Ozone: Ozone shell: Add more testing for key shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang a652a72286 HDFS-11936. Ozone: TestNodeManager times out before it is able to find all nodes. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer bbbfc885e7 HDFS-12149. Ozone: RocksDB implementation of ozone metadata store. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer 6225622dee HDFS-12071. Ozone: Corona: Implementation of Corona. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang d2ef3e40bc HDFS-12126. Ozone: Ozone shell: Add more testing for bucket shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang 5604c3788f HDFS-11996. Ozone : add an UT to test partial read of chunks. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer 4d333f8db1 HDFS-12058. Ozone: OzoneClient: OzoneClientImpl add getDetails calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Chen Liang 27262d8889 HDFS-12118. Ozone: Ozone shell: Add more testing for volume shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00