Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,934 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

6608 Commits

Author SHA1 Message Date
Pankaj Kumar 39e191e559 HBASE-19979 ReplicationSyncUp tool may leak Zookeeper connection 2018-02-13 08:59:02 -08:00
Michael Stack ba402b1e7b Revert "HBASE-19970 Remove unused functions from TableAuthManager." 
This reverts commit 7cc239fb5a.
 2018-02-13 06:19:08 -08:00
Chia-Ping Tsai 2f48fdbb26 HBASE-19876 The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner 2018-02-13 21:08:59 +08:00
Vasudevan 16f1f5b494 HBASE-19977 FileMmapEngine allocation of byte buffers should be 
synchronized (Ram)
 2018-02-13 15:49:37 +05:30
Michael Stack b4622ffad7 HBASE-19965 Fix flaky TestAsyncRegionAdminApi; ADDENDUM do same to TestAsyncTableAdminAPI 2018-02-12 22:52:56 -08:00
Michael Stack cf57ea15f1 HBASE-19965 Fix flaky TestAsyncRegionAdminApi 2018-02-12 22:27:12 -08:00
Apekshit Sharma 7cc239fb5a HBASE-19970 Remove unused functions from TableAuthManager. 
Functions deleted: setTableUserPermissions, setTableGroupPermissions, setNamespaceUserPermissions,
setNamespaceGroupPermissions, writeTableToZooKeeper, writeNamespaceToZooKeeper
 2018-02-12 16:01:29 -08:00
Michael Stack c2ee82c909 HBASE-19986 If HBaseTestClassRule timesout a test, thread dump 2018-02-12 15:28:40 -08:00
Michael Stack 00f8877323 HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink; ADDENDUM to fix TestReplicationSink 2018-02-12 10:47:25 -08:00
Michael Stack 67b69fb2c7 HBASE-16060 1.x clients cannot access table state talking to 2.0 cluster 
This patch adds mirroring of table state out to zookeeper. HBase-1.x
clients look for table state in zookeeper, not in hbase:meta where
hbase-2.x maintains table state.

The patch also moves and refactors the 'migration' code that was put in
place by HBASE-13032.

D hbase-client/src/main/java/org/apache/hadoop/hbase/CoordinatedStateException.java
 Unused.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Move table state migration code from Master startup out to
TableStateManager where it belongs. Also start
MirroringTableStateManager dependent on config.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MirroringTableStateManager.java

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
 Move migration from zookeeper of table state in here. Also plumb in
mechanism so subclass can get a chance to look at table state as we do
the startup fixup full-table scan of meta.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Bug-fix. Now we create regions in CLOSED state but we fail to check
table state; were presuming table always enabled. Meant on startup
there'd be an unassigned region that never got assigned.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMirroringTableStateManager.java
 Test migration and mirroring.
 2018-02-12 08:47:02 -08:00
zhangduo 8ff783f76d HBASE-19554 Dump thread info before failure 2018-02-12 16:03:52 +08:00
huzheng 629eaf8b74 HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink 2018-02-11 14:48:21 +08:00
Michael Stack 0593dda663 HBASE-19951 Cleanup the explicit timeout value for test method 2018-02-10 09:24:31 -08:00
zhangduo 32f235abee HBASE-19966 The WriteEntry for WALKey maybe null if we failed to call WAL.append 2018-02-09 19:09:40 -08:00
Guanghao Zhang 2b63af376e HBASE-19923 Reset peer state and config when refresh replication source failed 2018-02-09 18:54:03 +08:00
Guanghao Zhang 0a5f26324c HBASE-19942 (addendum) Fix flaky TestSimpleRpcScheduler 2018-02-09 13:37:06 +08:00
Aman Poonia f1a81618fd HBASE-19835 Use explicit casting to avoid side effects 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-02-08 12:32:36 -08:00
Michael Stack 277ce3d8a3
HBASE-19956 Remove category as a consideration timing out tests; set all test to timeout at 10minutes regardless 2018-02-07 23:10:12 -08:00
Mike Drob f5dbdf0dab HBASE-19947 ITU should overwrite HTU local FS 2018-02-07 16:53:54 -06:00
zhangduo dcbb331792 HBASE-19929 Call RS.stop on a session expired RS may hang 2018-02-07 15:52:04 +08:00
Chia-Ping Tsai d8b999e695 HBASE-19900 Region-level exception destroy the result of batch 2018-02-07 08:41:56 +08:00
Guanghao Zhang a5b86dd77a HBASE-19942 Fix flaky TestSimpleRpcScheduler 2018-02-07 07:54:39 +08:00
Balazs Meszaros 380083e7d3 HBASE-19886 Display maintenance mode in shell, web UI 2018-02-06 14:53:32 -08:00
Mike Drob b5ccfecc96 HBASE-19946 More test clean up for local-mode HTU 2018-02-06 15:20:39 -06:00
Michael Stack 001f9cc5ea HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: yet more debug 2018-02-06 09:47:30 -08:00
Michael Stack 590bee78dc HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: more debug 2018-02-06 06:35:36 -08:00
Guanghao Zhang 77153d4dc6 HBASE-19944 Fix timeout TestVisibilityLabelsWithCustomVisLabService 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-02-06 20:30:08 +08:00
Umesh Agashe 3fa27c2382 HBASE-19941 Flaky TestCreateTableProcedure times out in nightly, needs to LargeTests 2018-02-05 22:13:53 -08:00
tedyu 11e3a83870 HBASE-19934 HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting (Toshihiro Suzuki) 2018-02-05 21:03:56 -08:00
Umesh Agashe 3bb8daa605
HBASE-19939 Fixed NPE in tests TestSplitTableRegionProcedure#testSplitWithoutPONR() and testRecoveryAndDoubleExecution() 
Value of 'htd' is null as it is initialized in the constructor but when the object is deserialized its null. Got rid of member variable htd and made it local to method.
 2018-02-05 20:48:56 -08:00
zhangduo f5197979aa HBASE-19927 Addendum join on RegionServerThread instead of HRegionServer 2018-02-06 10:01:23 +08:00
Michael Stack 6307689a6e HBASE-19940 TestMetaShutdownHandler flakey 2018-02-05 12:37:48 -08:00
Michael Stack 9f2149f171 HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM Adding debug 2018-02-05 11:01:09 -08:00
Michael Stack c245bd5c03 HBASE-19837 Flakey TestRegionLoad; ADDENDUM Report more often and wait less time on change (also add some debug on TestMetaShutdown test) 2018-02-05 08:40:26 -08:00
Rajeshbabu Chintaguntla c5f86f2ce4 HBASE-19703 Functionality added as part of HBASE-12583 is not working after moving the split code to master 
Co-authored-by: Michael Stack <stack@apache.org>

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-02-05 23:56:18 +08:00
zhangduo ad3a1ba495 HBASE-19936 Introduce a new base class for replication peer procedure 2018-02-05 20:23:19 +08:00
zhangduo 88d6e06a1f HBASE-19554 reenable TestDLSAsyncFSWAL/TestDLSFSHLog for debugging 2018-02-05 08:58:16 +08:00
zhangduo e1cd10b002 HBASE-19927 TestFullLogReconstruction flakey 2018-02-05 08:41:32 +08:00
Michael Stack ab5a26ad5e HBASE-19931 TestMetaWithReplicas failing 100% of the time in testHBaseFsckWithMetaReplicas 2018-02-04 14:38:21 -08:00
anastas 170ffbba68 HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress 2018-02-04 14:59:10 +02:00
zhangduo 14420e1b41 HBASE-19926 Use a separated class to implement the WALActionListener for Replication 2018-02-04 20:36:43 +08:00
Ashish Singhi 397d34736e HBASE-19905 ReplicationSyncUp tool will not exit if a peer replication is disabled 
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
 2018-02-04 17:52:38 +05:30
zhangduo 2e1ec3d3d8 HBASE-19914 Refactor TestVisibilityLabelsOnNewVersionBehaviorTable 2018-02-04 13:27:28 +08:00
Michael Stack bfd74686c7 HBASE-19928 TestVisibilityLabelsOnNewVersionBehaviorTable fails 2018-02-03 10:17:40 -08:00
Michael Stack 12f3c82a86 HBASE-19919 Tidying up logging; ADDENDUM Fix tests w/ mocked Servers 2018-02-03 09:25:16 -08:00
Michael Stack 06dec20582
HBASE-19919 Tidying up logging 2018-02-03 08:42:02 -08:00
zhangduo 6519b98ac3 Revert "HBASE-19919 Tidying up logging" 
This reverts commit 40250f8c5f.

Wrong patch. Revert to fix compile error.
 2018-02-03 18:18:09 +08:00
Michael Stack 40250f8c5f HBASE-19919 Tidying up logging 2018-02-02 22:52:41 -08:00
Michael Stack 41974efa85
HBASE-19726 Failed to start HMaster due to infinite retrying on meta assign 2018-02-02 18:29:33 -08:00
Guanghao Zhang ad580acc89 HBASE-19918 Promote TestAsyncClusterAdminApi to LargeTests 2018-02-03 09:39:40 +08:00
Chia-Ping Tsai 2f4d0b94bc HBASE-19720 Rename WALKey#getTabnename to WALKey#getTableName 2018-02-02 19:43:26 +08:00
zhangduo fc6d140adf HBASE-19904 Break dependency of WAL constructor on Replication 2018-02-02 14:10:29 +08:00
tedyu a11258599e HBASE-19895 Add keepDeletedCells option in ScanOptions for customizing scanInfo in pre-hooks (Ankit Singhal) 2018-02-01 18:27:18 -08:00
Mike Drob 99b9fff07b HBASE-19841 Every HTU should be local until DFS starts 2018-02-01 20:24:11 -06:00
zhangduo 94dad844af HBASE-19913 Split TestStochasticLoadBalancer2 2018-02-02 08:29:28 +08:00
Michael Stack 3622bb0333 HBASE-19147 TestCacheOnWrite Times Out 2018-02-01 16:14:51 -08:00
Chia-Ping Tsai adccbb7edf HBASE-19897 RowMutations should follow the fluent pattern 2018-02-02 05:33:08 +08:00
Peter Somogyi d472422475 HBASE-19884 BucketEntryGroup's equals, hashCode and compareTo methods are not consistent 
Move back to default equals and hashCode.
Remove compareTo and Comparator to PriorityQueue.

Signed-off-by: Michael Stack <stack@apache.org>
 2018-02-01 13:21:58 -08:00
kewang 38c8144a06 HBASE-19912 Remove useless 'writeToWAL' flag of Region#checkAndRowMutate 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-02-01 12:33:48 -08:00
Umesh Agashe 57911d02c6 HBASE-19839 Fixed flakey tests TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution 
* Added a comment in MergeTableRegionsProcedure and SplitTableRegionProcedure explaining specific rollbacks has side effect that AssignProcedure/s are submitted asynchronously and those procedures may continue to execute even after rollback() is done.
* Updated comments in tests with correct rollback state to abort
* Added overloaded method MasterProcedureTestingUtility#testRollbackAndDoubleExecution which takes additional argument for waiting on all procedures to finish before asserting conditions
* Updated TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution to use newly added method

Signed-off-by: Michael Stack <stack@apache.org>
 2018-02-01 12:01:30 -08:00
Michael Stack b21b8bfb91 HBASE-19906 TestZooKeeper Timeout Includes move of TestQoSFunction from smalltests to mediumtests; it timedout in a run. 2018-02-01 10:53:18 -08:00
Michael Stack 9272f40a5c HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus; ADDENDUM changes TestCheckTestClasses to be medium sized test 2018-01-31 20:27:34 -08:00
Michael Stack b9cb1187b5 HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus 2018-01-31 20:23:14 -08:00
Michael Stack e17529ba73 HBASE-19910 TestBucketCache TimesOut 2018-01-31 18:17:01 -08:00
Michael Stack 28ee4be0fe HBASE-19909 TestRegionLocationFinder Timeout 2018-01-31 17:55:16 -08:00
Michael Stack bc0e9ce8f4 HBASE-19908 TestCoprocessorShortCircuitRPC Timeout.... 2018-01-31 17:32:39 -08:00
Michael Stack 414b2d0889 HBASE-19907 TestMetaWithReplicas still flakey 2018-01-31 15:17:38 -08:00
Michael Stack 60827fc1ea HBASE-19528 Major Compaction Tool; ADDENDUM 
Add @ClassRule to new tests and rename those added here so they
align w/ how we name other Test classes.
 2018-01-31 14:18:24 -08:00
Rahul Gidwani 4b3b627abe HBASE-19528 - Major Compaction Tool 2018-01-31 10:18:03 -08:00
zhangduo 7c318cead9 HBASE-19887 Do not overwrite the surefire junit listener property in the pom of sub modules 2018-01-31 20:49:18 +08:00
binlijin f7faad0477 HBASE-19728 Add lock to filesCompacting in all place. 2018-01-31 14:05:30 +08:00
Peter Somogyi 194890b694
HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey 
Recategorize TestCoprocessorWhitelistMasterObserver as Large test
 2018-01-30 06:01:24 -08:00
Michael Stack 98c9d8f7a1 HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey 2018-01-29 21:35:28 -08:00
Peter Somogyi 34c6c99041
HBASE-19811 Fix findbugs and error-prone warnings in hbase-server (branch-2) - addendum 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-01-29 14:03:08 -08:00
Michael Stack 0b9a0dc951
HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM to fix Checksyte 2018-01-29 09:36:30 -08:00
Michael Stack 4f547b3817
HBASE-19840 Flakey TestMetaWithReplicas 
Fix two issues:

 # Meta Replicas can all be assigned to the same server. This
 will call the test to hang when we do our kill of the server
 hosting meta because there'll be no replicas to read from
 as test intends. Check is to look for this condition on
 startup and adjust if we come across it. Replicas cross-cut
 assignment. They need work.
 # Other issue was shutdown. The master started toward the
 end of the test may not have come up fully by the time
 shutdown is called. We could be stuck assigning the
 meta replicas. Have shutdown shutdown the procedure
 executor engine.

There is other cleanup and notes in the below.

M HMaster
 Remove the silly stops in startup now we have real
 means of shutting down Master during init.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
 This replica stuff was doing stuff it shouldn't be doing
 like setting core Master state flags. It may have made
 sense once but now meta is assigned by a Pv2 Procedure
 so the flag setting in here is meddlesome. Clear out
 methods no longer needed.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Remove unused methods.
 Changes local variable names so they align w/ our naming elsewhere in
 code base.

M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
 Check for all replicas on the one server.
 2018-01-29 09:21:30 -08:00
zhangduo dab51c7900 HBASE-19885 Promote TestAssignmentManager to LargeTests 2018-01-30 00:24:45 +08:00
zhangduo 5b93ea1556 HBASE-19866 TestRegionServerReportForDuty doesn't timeout 
Synchronized on authManager instead of RpcServer
 2018-01-29 21:23:47 +08:00
zhangduo 305da1098a HBASE-19882 Promote TestProcedureManager to MediumTests 2018-01-29 17:48:44 +08:00
zhangduo f9480a56c7 HBASE-19881 Promote TestRegionReplicaReplicationEndpoint to LargeTests 2018-01-29 16:15:45 +08:00
zhangduo 94dfc7d60a HBASE-19880 Promote TestFuzzyRowFilterEndToEnd to LargeTests 2018-01-29 16:15:18 +08:00
zhangduo 80438924ff HBASE-19879 Promote TestAcidGuaranteesXXX to LargeTests 2018-01-29 14:55:04 +08:00
zhangduo 918599ef12 HBASE-19873 Add a CategoryBasedTimeout ClassRule for all UTs 2018-01-29 08:43:56 +08:00
Jan Hentschel 851e17987d HBASE-19869 TestLockManager now uses the right class for logging 2018-01-28 13:54:57 +01:00
zhangduo f1502a3aca HBASE-19867 Split TestStochasticLoadBalancer into several small tests 2018-01-27 18:25:46 +08:00
Michael Stack a5a8c4f3f2
Revert "HBASE-19841 LocalHTU to not enforce stream capabilities" 
At Mike Drobs' request!
 2018-01-26 15:11:50 -08:00
Mike Drob 9ea152d235 HBASE-19841 LocalHTU to not enforce stream capabilities 2018-01-26 10:50:16 -06:00
haxiaolin 94a1c051ba HBASE-19861 - Avoid using RPCs when querying table infos for master status pages 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-01-26 08:02:14 -08:00
Guanghao Zhang 39b912aae9 HBASE-19818 Scan time limit not work if the filter always filter row key 2018-01-26 17:02:22 +08:00
Apekshit Sharma 01c34243fe HBASE-19866 Add cateogry based timeout using Timeout Rule to TestRegionServerReportForDuty. 2018-01-26 00:30:56 -08:00
Apekshit Sharma 1beb687f4e HBASE-19862 addendum: Checkstyle fixes, remove unused imports and variables. Add @Rule for timeout. 2018-01-26 00:24:35 -08:00
Apekshit Sharma 61358985b1 HBASE-19862 Fix TestTokenAuthentication - mock RegionCoprocessorEnvironment should be of type HasRegionServerServices also. 2018-01-25 23:15:45 -08:00
Thiruvel Thirumoolan ce50830a0a HBASE-19756 Master NPE during completed failed proc eviction 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-01-24 16:42:58 -08:00
Michael Stack 970636c5af HBASE-19838 Can not shutdown backup master cleanly when it has already tried to become the active master 
On Master@shutdown, close the shared Master connection to kill any
ongoing RPCs by hosted clients.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Call close ont the Master shared clusterconnection to kill any ongoing
rpcs.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
Remove guts of close; we were closing the Masters connection....not
our responsibility.

Added unit test written by Duo Zhang which demonstrates the case where
Master will not go down.

Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-01-23 11:07:00 +08:00
Chia-Ping Tsai 541f8ad8a8 HBASE-4224 Need a flush by regionserver rather than by table option 2018-01-23 09:51:05 +08:00
Peter Somogyi 93a182f286 HBASE-19811 Fix findbugs and error-prone warnings in hbase-server (branch-2) 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-01-22 17:20:59 -08:00
Michael Stack d49357f265 HBASE-19840 Flakey TestMetaWithReplicas 
Adding debuging
 2018-01-22 12:28:56 -08:00
Rahul Gidwani b577fa0814 HBASE-15321 - Ability to open a HRegion from hdfs snapshot. 2018-01-22 12:09:38 -08:00
Sergey Soldatov be496ba8ea HBASE-19774 incorrect behavior of locateRegionInMeta 2018-01-22 11:48:55 -08:00
Michael Stack c60359959c HBASE-19837 Flakey TestRegionLoad 
Add debug, change name of the rows so readable.
 2018-01-22 08:01:57 -08:00