Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,478 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

813 Commits

Author SHA1 Message Date
Hanisha Koneru 44e19fc7f7 HDDS-213. Single lock to synchronize KeyValueContainer#update. 2018-07-09 09:33:09 -07:00
Anu Engineer 790c563511 HDDS-235. Fix TestOzoneAuditLogger#verifyDefaultLogLevel. 
Contributed by Xiaoyu Yao.
 2018-07-08 11:27:54 -07:00
Anu Engineer 2f51cd60ef HDDS-217. Move all SCMEvents to a package. 
Contributed by Anu Engineer.
 2018-07-08 11:11:21 -07:00
Bharat Viswanadham cb9574a337 HDDS-237. Add updateDeleteTransactionId. Contributed by Bharat Viswanadham and Lokesh Jain 2018-07-06 16:26:40 -07:00
Bharat Viswanadham e899c4cc01 HDDS-211. Add a create container Lock. Contributed by Bharat Viswanadham 2018-07-06 15:02:40 -07:00
Arpit Agarwal 061b168529 HDDS-167. Rename KeySpaceManager to OzoneManager. Contributed by Arpit Agarwal. 2018-07-06 12:09:05 -07:00
Bharat Viswanadham 7dcf5878a4 HDDS-204. Modify Integration tests for new ContainerIO classes. Contributed by Bharat Viswanadham 2018-07-06 11:01:33 -07:00
Bharat Viswanadham 2c2351e87b HDDS-182:CleanUp Reimplemented classes. Contributed by Hansiha Koneru 2018-07-05 15:23:39 -07:00
Bharat Viswanadham a404164939 HDDS-229:Remove singleton for Handler. Contributed by Bharat Viswanadham 2018-07-05 11:26:00 -07:00
Nanda kumar 71df8c27c9 HDDS-212. Introduce NodeStateManager to manage the state of Datanodes in SCM. Contributed by Nanda kumar. 2018-07-05 02:11:10 +05:30
Anu Engineer 7ca4f0cefa HDDS-175. Refactor ContainerInfo to remove Pipeline object from it. 
Contributed by Ajay Kumar.
 2018-07-03 14:11:52 -07:00
Anu Engineer c0ef7e7680 HDDS-198. Create AuditLogger mechanism to be used by OM, SCM and Datanode. 
Contributed by Dinesh Chitlangia.
 2018-07-03 11:39:50 -07:00
Hanisha Koneru e1f4b3b560 HDDS-176. Add keyCount and container maximum size to ContainerData. Contributed by Bharat Viswanadham. 2018-07-03 09:53:41 -07:00
Hanisha Koneru 44b091a8d7 HDDS-205. Add metrics to HddsDispatcher. Contributed by Bharat Viswanadham. 2018-07-03 09:50:25 -07:00
Bharat Viswanadham 23ebe0f4f0 HDDS-169:Add Volume IO Stats. Contributed by Bharat Viswanadham 2018-06-28 17:10:10 -07:00
Nanda kumar e4d7227aad HDDS-178: DN should update transactionId on block delete. Contributed by Lokesh Jain. 2018-06-29 04:37:16 +05:30
Bharat Viswanadham c104525168 Merge remote-tracking branch 'apache-commit/trunk' into HDDS-48 2018-06-28 14:28:23 -07:00
Bharat Viswanadham 52d1d9603e HDDS-183:Integrate Volumeset, ContainerSet and HddsDispatcher. Contributed by Bharat Viswanadham 2018-06-28 14:07:52 -07:00
Hanisha Koneru 13579f9296 HDDS-173. Refactor Dispatcher and implement Handler for new ContainerIO design. 2018-06-28 10:13:30 -07:00
Anu Engineer 85627e2cba HDDS-195. Create generic CommandWatcher utility. 
Contributed by Elek, Marton.
 2018-06-27 23:39:33 -07:00
Anu Engineer 8752a48564 HDDS-193. Make Datanode heartbeat dispatcher in SCM event based. 
Contributed by Elek, Marton.
 2018-06-27 14:18:25 -07:00
Anu Engineer 18932717c4 HDDS-94. Change ozone datanode command to start the standalone datanode plugin. 
Contributed by  Sandeep Nemuri.
 2018-06-27 14:15:15 -07:00
Xiaoyu Yao 1e30547642 HDDS-170. Fix TestBlockDeletingService#testBlockDeletionTimeout. Contributed by Lokesh Jain. 2018-06-27 13:56:45 -07:00
Xiaoyu Yao e9ec3d78f5 HDDS-186. Create under replicated queue. Contributed by Ajay Kumar. 2018-06-27 13:35:30 -07:00
Xiaoyu Yao 56a4cdb980 HDDS-194. Remove NodePoolManager and node pool handling from SCM. Contributed by Elek Marton. 2018-06-27 13:28:00 -07:00
Xiaoyu Yao 0d6fe5f36b Revert "HDDS-194. Remove NodePoolManager and node pool handling from SCM. Contributed by Elek Marton" 
This reverts commit aaf03cc459.
 2018-06-27 13:25:45 -07:00
Xiaoyu Yao aaf03cc459 HDDS-194. Remove NodePoolManager and node pool handling from SCM. Contributed by Elek Marton 2018-06-27 12:39:15 -07:00
Bharat Viswanadham 238fe00ad2 HDDS-192:Create new SCMCommand to request a replication of a container. Contributed by Elek Marton 2018-06-25 21:12:05 -07:00
Anu Engineer a55d6bba71 HDDS-191. Queue SCMCommands via EventQueue in SCM. 
Contributed by Elek, Marton.
 2018-06-25 13:05:22 -07:00
Mukul Kumar Singh 2a4632d3d7 HDDS-141. Remove PipeLine Class from SCM and move the data field in the Pipeline to ContainerInfo. Contributed by Shashikant Banerjee. 2018-06-17 23:48:49 -07:00
Bharat Viswanadham ca192cb7c9 HDDS-160:Refactor KeyManager, ChunkManager. Contributed by Bharat Viswanadham 2018-06-15 14:35:33 -07:00
Bharat Viswanadham 998e2850a3 HDDS-155:Implement KeyValueContainer and adopt new disk layout for the containers. Contributed by Bharat Viswanadham 2018-06-14 20:54:54 -07:00
Hanisha Koneru 9a5552bf76 HDDS-156. Implement HDDSVolume to manage volume state 2018-06-14 13:28:41 -07:00
Arpit Agarwal 418cff4820 Merge remote-tracking branch 'apache-commit/trunk' into HDDS-48 2018-06-14 09:38:20 -07:00
Anu Engineer ddd09d59f3 HDDS-163. Add Datanode heartbeat dispatcher in SCM. 
Contributed by Nandakumar.
 2018-06-13 20:18:22 -07:00
Anu Engineer 7547740e5c HDDS-161. Add functionality to queue ContainerClose command from SCM Heartbeat Response to Ratis. 
Contributed by Shashikant Banerjee.
 2018-06-13 18:48:59 -07:00
Anu Engineer 43baa036ae HDDS-109. Add reconnect logic for XceiverClientGrpc. 
Contributed by Lokesh Jain.
 2018-06-13 09:50:10 -07:00
Jitendra Pandey f4c7c91123 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan. 2018-06-13 00:36:02 -07:00
Mukul Kumar Singh aa7614c5f3 HDDS-158. DatanodeStateMachine endPoint task throws NullPointerException. Contributed by Nanda Kumar. 2018-06-12 20:36:23 +05:30
Xiaoyu Yao 23bfd9f7e4 HDDS-72. Add deleteTransactionId field in ContainerInfo. Contributed by Lokesh Jain. 2018-06-11 16:02:32 -07:00
Bharat Viswanadham 7e228e54c5 HDDS-136:Rename dbPath,containerFilePath. Contributed by Bharat Viswanadham 2018-06-11 13:14:03 -07:00
Hanisha Koneru 0e437f9b17 Revert "Create Version File in Datanode. Contributed by Bharat Viswanadham." 
This reverts commit f26d3466d7.
 2018-06-11 12:15:39 -07:00
Bharat Viswanadham 143dd560bf HDDS-133:Change format of .container files to Yaml. Contributed by Bharat Viswanadham 2018-06-11 09:04:54 -07:00
Anu Engineer a1272448bf HDDS-157. Upgrade common-langs version to 3.7 in HDDS and Ozone. 
Contributed by Takanobu Asanuma.
 2018-06-08 10:27:01 -07:00
Anu Engineer 78761e87a7 HDDS-127. Add CloseContainerEventHandler in SCM. 
Contributed by Shashikant Banerjee.
 2018-06-07 14:35:22 -07:00
Hanisha Koneru 772c95395b HDDS-140. Add DU usage to VolumeInfo. 2018-06-06 16:21:21 -07:00
Bharat Viswanadham 977c8cd166 HDDS-123:ContainerSet class to manage ContainerMap. Contributed by Bharat Viswanadham 2018-06-06 13:59:20 -07:00
Anu Engineer 30f0582f2b HDDS-107. TestOzoneConfigurationFields is failing. 
Contributed by LiXin Ge & Mukul Kumar Singh.
 2018-06-06 08:39:18 -07:00
Anu Engineer baebe4d52b HDDS-129. Support for ReportManager in Datanode. 
Contributed by Nanda Kumar.
 2018-06-05 10:31:42 -07:00
Xiaoyu Yao 920d154997 HDDS-148. Remove ContainerReportManager and ContainerReportManagerImpl. Contributed by Nanda kumar. 2018-06-05 08:34:44 -07:00
Bharat Viswanadham f26d3466d7 Create Version File in Datanode. Contributed by Bharat Viswanadham. 2018-06-04 15:28:19 -07:00
Xiaoyu Yao ea7b53fb44 HDDS-145. Freon times out because of because of wrong ratis port number in datanode details. Contributed by Mukul Kumar Singh. 2018-06-04 09:51:04 -07:00
Xiaoyu Yao ab3885f2c8 HDDS-126. Fix findbugs warning in MetadataKeyFilters.java. Contributed by Akira Ajisaka. 2018-06-04 08:55:01 -07:00
Anu Engineer 9e50dce46c HDDS-144. Fix TestEndPoint#testHeartbeat and TestEndPoint#testRegister. 
Contributed by Shashikant Banerjee.
 2018-06-01 15:36:30 -07:00
Anu Engineer 1be05a3623 HDDS-142. TestMetadataStore fails on Windows. 
Contributed by  Íñigo Goiri.
 2018-06-01 14:21:35 -07:00
Anu Engineer 19560bb704 HDDS-137. Cleanup Hdds-ozone CheckStyle Issues. 
Contributed by Anu Engineer.
 2018-06-01 12:54:47 -07:00
Arpit Agarwal 79b298111f Merge remote-tracking branch 'apache-commit/trunk' into HDDS-48 2018-05-31 11:45:01 -07:00
Arpit Agarwal 59777185fc HDDS-116. Implement VolumeSet to manage disk volumes. Contributed by Hanisha Koneru. 2018-05-31 10:29:25 -07:00
Anu Engineer 02c4b89f99 HDDS-128. Support for DN to SCM signaling. 
Contributed by Nanda Kumar.
 2018-05-30 14:21:13 -07:00
Bharat Viswanadham 6cd19b45ef HDDS-90: Create ContainerData, Container classes. Contributed by Bharat Viswanadham 2018-05-30 14:04:19 -07:00
Xiaoyu Yao ee1e0e2036 HDDS-92. Use DBType during parsing datanode .container files. Contributed by Bharat Viswanadham. 2018-05-30 14:04:19 -07:00
Anu Engineer 3b34148c4f HDDS-88. Create separate message structure to represent ports in DatanodeDetails. 
Contributed by Nanda Kumar.
 2018-05-30 08:52:07 -07:00
Anu Engineer 24169062e5 HDDS-114. Ozone Datanode mbean registration fails for StorageLocation. 
Contributed by Elek, Marton.
 2018-05-29 13:48:55 -07:00
Anu Engineer 201440b987 HDDS-81. Moving ContainerReport inside Datanode heartbeat. 
Contributed by Nanda Kumar.
 2018-05-29 12:48:50 -07:00
Anu Engineer 9502b47bd2 HDDS-125. Cleanup HDDS CheckStyle issues. 
Contributed by Anu Engineer.
 2018-05-29 09:54:06 -07:00
Anu Engineer 0cf6e87f92 HDDS-78. Add per volume level storage stats in SCM. 
Contributed by  Shashikant Banerjee.
 2018-05-26 11:11:14 -07:00
Anu Engineer 2a9652e696 HDDS-113. Rest and Rpc Client should verify resource name using HddsClientUtils. 
Contributed by Lokesh Jain.
 2018-05-25 15:45:50 -07:00
Anu Engineer 8733012ae3 HDDS-96. Add an option in ozone script to generate a site file with minimally required ozone configs. 
Contributed by Dinesh Chitlangia.
 2018-05-25 13:06:14 -07:00
Xiaoyu Yao 2d19e7d08f HDDS-80. Remove SendContainerCommand from SCM. Contributed by Nanda Kumar. 2018-05-24 11:10:30 -07:00
Mukul Kumar Singh 774daa8d53 HDDS-45. Removal of old OzoneRestClient. Contributed by Lokesh Jain. 2018-05-24 15:53:42 +05:30
Mukul Kumar Singh fed2bef647 HDDS-85. Send Container State Info while sending the container report from Datanode to SCM. Contributed by Shashikant Banerjee. 2018-05-23 14:15:35 +05:30
Anu Engineer 5a9140690a HDDS-49. Standalone protocol should use grpc in place of netty. 
Contributed by Mukul Kumar Singh.
 2018-05-22 19:56:15 -07:00
Xiaoyu Yao 68c7fd8e60 HDDS-79. Remove ReportState from SCMHeartbeatRequestProto. Contributed by Nanda kumar. 2018-05-22 15:46:59 -07:00
Anu Engineer 60821fb20e HDDS-74. Rename name of properties related to configuration tags. 
Contributed by Sandeep Nemuri.
 2018-05-22 11:38:11 -07:00
Xiaoyu Yao 5e88126776 HDDS-82. Merge ContainerData and ContainerStatus classes. Contributed by Bharat Viswanadham. 2018-05-21 16:09:24 -07:00
Mukul Kumar Singh 132a547dea HDDS-71. Send ContainerType to Datanode during container creation. Contributed by Bharat Viswanadham. 2018-05-21 23:08:20 +05:30
Bharat Viswanadham 3d2d9dbcaa HDDS-87:Fix test failures with uninitialized storageLocation field in storageReport Contributed by Shashikant Banerjee 2018-05-21 08:01:51 -07:00
Mukul Kumar Singh c97df7712c HDDS-57. TestContainerCloser#testRepeatedClose and TestContainerCloser#testCleanupThreadRuns fail consistently. Contributed by Shashikant Banerjee. 2018-05-21 19:06:26 +05:30
Mukul Kumar Singh 6e996867f6 HDDS-76. Modify SCMStorageReportProto to include the data dir paths as well as the StorageType info. Contributed by Shashikant Banerjee. 2018-05-18 14:08:46 +05:30
Anu Engineer 7c485a6701 HDDS-38. Add SCMNodeStorage map in SCM class to store storage statistics per Datanode. 
Contributed by Shashikant Banerjee.
 2018-05-17 16:13:28 -07:00
Xiaoyu Yao 59bde09920 HDDS-3. Send NodeReport and ContainerReport when datanodes register. Contributed by Bharat Viswanadham. 2018-05-17 08:52:00 -07:00
Mukul Kumar Singh 6653f4ba2e HDDS-19. Update ozone to latest ratis snapshot build (0.1.1-alpha-d7d7061-SNAPSHOT). Contributed by Lokesh Jain. 2018-05-14 22:25:03 +05:30
Xiaoyu Yao fc5d49c202 HDDS-29. Fix TestStorageContainerManager#testRpcPermission. Contributed by Mukul Kumar Singh. 2018-05-14 09:10:03 -07:00
Anu Engineer 32cbd0cfd6 HDDS-51. Fix TestDeletedBlockLog#testDeletedBlockTransactions. 
Contributed by Mukul Kumar Singh.
 2018-05-12 13:37:34 -07:00
Anu Engineer 1194ec31d7 HDDS-17. Add node to container map class to simplify state in SCM. 
Contributed by Anu Engineer.
 2018-05-12 09:58:20 -07:00
Anu Engineer ba12e8805e HDDS-25. Simple async event processing for SCM. 
Contributed by Elek, Marton.
 2018-05-11 11:36:52 -07:00
Anu Engineer c1d64d60f6 HDDS-39. Ozone: Compile Ozone/HDFS/Cblock protobuf files with proto3 compiler using maven protoc plugin. 
Contributed by Mukul Kumar Singh.
 2018-05-11 11:08:45 -07:00
Anu Engineer 30293f6065 HDDS-34. Remove .meta file during creation of container 
Contributed by Bharat Viswanadham.
 2018-05-10 17:08:26 -07:00
Xiaoyu Yao 7369f41020 HDDS-16. Remove Pipeline from Datanode Container Protocol protobuf definition. Contributed by Mukul Kumar Singh. 2018-05-10 14:49:58 -07:00
Anu Engineer 48d0b54849 HDDS-31. Fix TestSCMCli. Contributed by Lokesh Jain. 2018-05-10 12:43:13 -07:00
Anu Engineer f8b540049d HDDS-42. Inconsistent module names and descriptions. Contributed by Tsz Wo Nicholas Sze. 2018-05-10 11:44:14 -07:00
Anu Engineer 46e0f27862 HDDS-18. Ozone Shell should use RestClient and RpcClient. Contributed by Lokesh Jain. 2018-05-09 23:00:13 -07:00
Mukul Kumar Singh d72c165161 HDDS-23. Remove SCMNodeAddressList from SCMRegisterRequestProto. Contributed by Nanda Kumar. 2018-05-08 18:11:33 +05:30
Anu Engineer 3a43ac2851 HDDS-1. Remove SCM Block DB. Contributed by Xiaoyu Yao. 2018-05-07 14:58:52 -07:00
Anu Engineer f0c3dc4cf4 HDDS-13. Refactor StorageContainerManager into seperate RPC endpoints. Contributed by Anu Engineer. 2018-04-30 21:41:10 -07:00
Anu Engineer 3d43474f75 HDDS-11. Fix findbugs exclude rules for ozone and hdds projects. Contributed by Elek, Marton. 2018-04-30 09:20:58 -07:00
Owen O'Malley 2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Xiaoyu Yao 9da4caee89 HDFS-13422. Ozone: Fix whitespaces and license issues in HDFS-7240 branch. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 20dab05564 HDFS-13407. Ozone: Use separated version schema for Hdds/Ozone projects. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar 17974ba3a6 HDFS-13444. Ozone: Fix checkstyle issues in HDFS-7240. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh ae8ac7f082 HDFS-13424. Ozone: Refactor MiniOzoneClassicCluster. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar 025058f251 HDFS-13446. Ozone: Fix OzoneFileSystem contract test failures. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 908ddab55b HDFS-13197. Ozone: Fix ConfServlet#getOzoneTags cmd. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh 979bbb4019 HDFS-13423. Ozone: Clean-up of ozone related change from hadoop-hdfs-project. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar d5a8e60256 HDFS-13413. Ozone: ClusterId and DatanodeUuid should be marked mandatory fields in SCMRegisteredCmdResponseProto. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh 8cc13ff962 HDFS-13425. Ozone: Clean-up of ozone related change from hadoop-common-project. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Nanda kumar aae3ba24ca HDFS-13348. Ozone: Update IP and hostname in Datanode from SCM's response to the register call. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Nanda kumar 12d80f8cae HDFS-13416. Ozone: TestNodeManager tests fail. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Nanda kumar 3779fadc5c HDFS-13324. Ozone: Remove InfoPort and InfoSecurePort from DatanodeDetails. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao 25f2398bbd HDFS-13395. Ozone: Plugins support in HDSL Datanode Service. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh 82eb5da2b6 HDFS-13342. Ozone: Rename and fix ozone CLI scripts. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer e1470c8e9a HDFS-13405. Addendum: Ozone: Rename HDSL to HDDS. 
Contributed by Ajay Kumar.
 2018-04-26 05:36:04 -07:00
Anu Engineer 8b832f3c35 HDFS-13405. Ozone: Rename HDSL to HDDS. 
Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.
 2018-04-26 05:36:04 -07:00