d40348e8cd
HBASE-21098 Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
...
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-11 16:31:56 -07:00
c0acad92d2
HBASE-21021 Result returned by Append operation should be ordered
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-11 14:55:32 -07:00
6e699b0694
HBASE-21172 Addendum fix comment
2018-09-11 17:32:38 +08:00
55e1297d96
HBASE-21172 Reimplement the retry backoff logic for ReopenTableRegionsProcedure
2018-09-11 15:14:02 +08:00
52d50cc426
HBASE-21173 Remove the duplicate HRegion#close in TestHRegion
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-11 11:20:23 +08:00
1a7dba44ce
HBASE-21158 Empty qualifier cell is always returned when using QualifierFilter
2018-09-10 21:51:48 +08:00
2be9196ba7
HBASE-21102 - ServerCrashProcedure should select target server where no
...
other replicas exist for the current region (ram)
2018-09-10 13:31:36 +05:30
d35e2f35a4
HBASE-21144 AssignmentManager.waitForAssignment is not stable
2018-09-10 12:19:04 +08:00
49501cd5b5
HBASE-21052 After restoring a snapshot, table.jsp page for the table gets stuck
2018-09-09 10:25:10 +09:00
1b65c2f161
HBASE-21001 ReplicationObserver fails to load in HBase 2.0.0
2018-09-07 23:44:03 +08:00
3f8c15ad2b
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir; ADDENDUM
...
Address review comments.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-09-06 17:25:20 -07:00
62919a3791
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
2018-09-06 16:45:53 -07:00
bbd9131c59
HBASE-21138 Close HRegion instance at the end of every test in TestHRegion
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-09-06 15:01:19 -07:00
1798032ace
HBASE-21161 Enable the test added in HBASE-20741 that was removed
...
accidentally (Ram)
2018-09-06 16:53:16 +05:30
83c871add8
HBASE-21107 add a metrics for netty direct memory
2018-09-05 16:10:17 -07:00
dee13ea401
HBASE-20892 [UI] Start / End keys are empty on table.jsp
2018-09-05 09:21:38 +08:00
1a5aedab34
HBASE-20741 Split of a region with replicas creates all daughter regions
...
and its replica in same server (Ram)
Signed-off-by: Huaxiang Sun, Michael Stack, Duo Zhang
2018-09-04 16:26:11 +05:30
0409c54ba7
HBASE-21129 Clean up duplicate codes in #equals and #hashCode methods of Filter
...
Signed-off-by Ted Yu <yuzhihong@gmail.com>
2018-09-04 11:15:57 +08:00
58365c8dd2
HBASE-21126 Configurable number of allowed failures for ZooKeeper Canary
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-09-03 19:35:28 -07:00
68c5313ca4
HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime
2018-09-02 18:32:24 +08:00
cf59f2a1d2
HBASE-21128 TestAsyncRegionAdminApi.testAssignRegionAndUnassignRegion is broken
2018-08-30 10:09:49 +08:00
72a29211cc
HBASE-15728 Add Per-Table metrics back
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableSource.java
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperAggregate.java
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableSourceImpl.java
hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperStub.java
hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsTableSourceImpl.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperAggregateImpl.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsTableAggregate.java
2018-08-29 16:27:15 -07:00
737ac48473
HBASE-21083 Introduce a mechanism to bypass the execution of a stuck procedure
2018-08-28 20:19:55 -07:00
22254cf4ce
HBASE-20942 Fix ArrayIndexOutOfBoundsException for RpcServer TRACE logging
...
Also makes the trace log message length configurable.
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-28 11:02:18 -04:00
fc21dc854b
HBASE-21017 Revisit the expected states for open/close
2018-08-28 14:49:47 +08:00
cdf5cfa396
HBASE-20941 Created and implemented HbckService in master
...
Added API setTableStateInMeta() to update table state only in Meta. This will be used by hbck2 tool.
2018-08-27 12:10:33 -07:00
a0cbfb8ea6
HBASE-21071 HBaseTestingUtility::startMiniCluster() to use builder pattern
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-08-27 10:22:04 -07:00
f9790b01d9
HBASE-21088 HStoreFile should be closed in HStore#hasReferences
2018-08-27 09:38:03 -07:00
4f32883e95
HBASE-21072 Addendum do not write lock file when running TestHBaseFsckReplication
2018-08-27 21:04:48 +08:00
2369afd7e5
HBASE-21084 When cloning a snapshot including a split parent region, the split parent region of the cloned table will be online
2018-08-27 09:58:52 +09:00
404d4d56cc
HBASE-21095 The timeout retry logic for several procedures are broken after master restarts
2018-08-26 18:17:20 +08:00
f533f01a3a
HBASE-20881 Introduce a region transition procedure to handle all the state transition for a region
2018-08-26 18:08:06 +08:00
2b7ef2e623
HBASE-20772 Controlled shutdown fills Master log with the disturbing message 'No matching procedure found for rit=OPEN, location=ZZZZ, table=YYYYY, region=XXXX transition to CLOSED'
...
Look for the particular case where RS does the close of region w/o
involving Master and log special message in this case. Dodgy. But
until we have Master run shutdown of all regions, better than
the message we currently show.
2018-08-25 14:30:27 +08:00
a83073aff0
HBASE-21078 [amv2] CODE-BUG NPE in RTP doing Unassign
2018-08-24 13:22:45 -07:00
6047fb05f5
HBASE-21113 Apply the branch-2 version of HBASE-21095, The timeout retry logic for several procedures are broken after master restarts
...
I applied the patch HBASE-21095 and then reverted it so could apply the
patch as HBASE-21113 (by reverting the HBASE-21095 revert but pushing
with this message!).
Revert "Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts""
This reverts commit a220566b98
2018-08-24 12:32:33 -07:00
a220566b98
Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts"
...
This reverts commit 09be1d0d57
2018-08-24 12:24:17 -07:00
09be1d0d57
HBASE-21095 The timeout retry logic for several procedures are broken after master restarts
2018-08-24 12:21:01 -07:00
2ea45e67d0
HBASE-21072 Block out HBCK1 in hbase2
...
Write the hbase-1.x hbck1 lock file to block out hbck1 instances writing
state to an hbase-2.x cluster (could do damage).
Set hbase.write.hbck1.lock.file to false to disable this writing.
2018-08-24 09:23:54 -07:00
c33af1e854
HBASE-21101 Remove the waitUntilAllRegionsAssigned call after split in TestTruncateTableProcedure
2018-08-24 10:35:05 +08:00
b318311dfd
HBASE-20193 Move TestCreateTableProcedure.testMRegions to a separated file
2018-08-24 10:09:23 +08:00
87f9b4accd
HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning
2018-08-23 11:49:47 -07:00
a7a281a644
HBASE-19008 Add missing equals or hashCode method(s) to stock Filter implementations
...
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-08-24 00:39:35 +08:00
cc6fc1f04f
HBASE-20194 Remove the explicit timeout config for TestTruncateTableProcedure
2018-08-23 06:27:37 +08:00
a7da458c5b
HBASE-21041 Memstore's heap size will be decreased to minus zero after flush
2018-08-22 23:03:20 +08:00
77e1a66f1e
HBASE-21031 Memory leak if replay edits failed during region opening
2018-08-22 22:22:35 +08:00
b5ae4e5f9e
HBASE-20666 Unsuccessful table creation leaves entry in hbase:rsgroup table
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-08-21 18:55:46 -07:00
d1b5113511
HBASE-21032 ScanResponses contain only one cell each
...
Amending-Author: Duo Zhang <zhangduo@apache.org>
2018-08-21 13:31:49 -07:00
3787c72953
HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region - addendum
2018-08-19 14:25:12 -07:00
52ba33aa73
HBASE-20940 HStore.cansplit should not allow split to happen if it has references (Vishal Khandelwal)
2018-08-17 15:02:22 -07:00
1fa67725c5
HBASE-21047 Object creation of StoreFileScanner thru constructor and close may leave refCount to -1 (Vishal Khandelwal)
2018-08-16 11:42:49 -07:00
75939775af
HBASE-20705 Having RPC quota on a table now no longer prevents Space Quota to be recreate/removed
...
Just added 2 test cases as the subtasks of this jira solves the issue
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-16 11:16:56 -04:00
cbe2fc113d
HBASE-21062 Correctly use the defaultProvider value on the Providers enum when constructing a WALProvider
2018-08-16 10:18:39 -04:00
d5abb92088
HBASE-20469 Directory used for sidelining old recovered edits files should be made configurable
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-08-15 18:01:24 -07:00
bee8566f2c
HBASE-20985 add two attributes when we do normalization
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 16:49:37 +08:00
d979f70142
HBASE-21025 Addendum missed a 'succ = true'
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 10:54:49 +08:00
51df2887c4
HBASE-21012 Revert the change of serializing TimeRangeTracker
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-08-11 22:25:10 +08:00
2311fa86d4
HBASE-21025 Add cache for TableStateManager
2018-08-10 21:11:45 +08:00
fb9aa8a95a
HBASE-18201 add UT and docs for DataBlockEncodingTool
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-08-10 11:16:36 +08:00
28c3336bfe
HBASE-20965 Separate region server report requests to new handlers
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-09 18:15:22 +08:00
6a8bd0fe84
HBASE-20813 Removed RPC quotas when the associated table/Namespace is dropped off
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-07 13:43:27 -04:00
1aacde4839
HBASE-20986 Separate the config of block size when we do log splitting and write Hlog
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-07 14:00:31 +08:00
b2cefab0e1
HBASE-20885 Removed entry for RPC quota from hbase:quota when RPC quota is removed
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-08-03 11:06:56 -04:00
06a92a3d20
HBASE-19036 Add action in Chaos Monkey to restart Active Namenode
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-08-02 05:00:16 -07:00
690d29bae7
HBASE-20856 PITA having to set WAL provider in two places
...
With this change if hbase.wal.meta_provider is not explicitly set,
it uses whatever set with hbase.wal.provider. this change avoids a use
case of unexpectedly using two different providers when only
hbase.wal.provider is set to non-default but not hbase.wal.meta_provider.
This change also include document (architecture.adoc) update
Also, this is a port from master to branch-2
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <Apache9@apache.org>
2018-08-01 14:36:29 -07:00
fc0c4660fa
HBASE-20989 Minor, miscellaneous logging fixes
...
Signed-off-by: Zach York <zyork@amazon.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-08-01 11:20:26 -07:00
bfb9c71e3b
HBASE-20794 add INFO level log to createTable operation
2018-08-01 11:04:18 -07:00
477d567fd7
HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing ADDENDUM: Rather than rollback, just do region reopens.
...
In split, reopen the parent if recovered.edits and in merge, reopen the
parent region or regions that happened to have recovered.edits on close.
2018-08-01 00:33:33 -07:00
777f3e88e2
HBASE-20935 HStore.removeCompactedFiles should log in case it is unable to delete a file (Vishal Khandelwal)
...
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
2018-07-31 15:59:53 -07:00
99714da30f
HBASE-20950 Helper method to configure secure DFS cluster for tests
...
Create a helper method HBaseKerberosUtils#setSecuredConfiguration().
TestSecureExport, TestSaslFanOutOneBlockAsyncDFSOutput,
SecureTestCluster and TestThriftSpnegoHttpServer uses this new helper
method.
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-31 13:37:10 -07:00
e14b60a539
HBASE-20886 [Auth] Support keytab login in hbase client
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-07-31 11:10:34 +08:00
584093c23f
HBASE-20538 Upgrade our hadoop versions to 2.7.7 and 3.0.3
2018-07-29 20:20:01 +08:00
5957fceda4
Revert "HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3"
...
This reverts commit 71a4c17d99
2018-07-29 20:19:16 +08:00
71a4c17d99
HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3
2018-07-29 20:04:53 +08:00
ab18440107
HBASE-19369 Switch to Builder Pattern In WAL
...
This patch switches to the builder pattern by adding a helper method.
It also checks to ensure that the pattern is available (i.e. that
HBase is running on a hadoop version that supports it).
Amending-Author: Mike Drob <mdrob@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-27 23:42:53 -05:00
51b0d0edd0
HBASE-20939 There will be race when we call suspendIfNotReady and then throw ProcedureSuspendedException
2018-07-27 21:25:22 +08:00
611ca11c9f
HBASE-20921 Possible NPE in ReopenTableRegionsProcedure
2018-07-27 09:34:25 +08:00
bb6f24cd59
HBASE-20932 Effective MemStoreSize::hashCode()
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-25 14:42:47 -07:00
3b84c9c8d2
HBASE-20649 Validate HFiles do not have PREFIX_TREE DataBlockEncoding
...
Amending-Author: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-07-25 09:12:17 -05:00
ac4b789f5f
HBASE-20867 RS may get killed while master restarts
2018-07-25 18:13:50 +08:00
0b283dae0d
HBASE-20846 Restore procedure locks when master restarts
2018-07-25 14:37:31 +08:00
097ae55708
HBASE-20928 Rewrite calculation of midpoint in binarySearch functions to prevent overflow
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-24 08:17:16 -07:00
99701968c8
HBASE-20565 ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result
2018-07-24 10:33:33 +08:00
aaf5c78445
HBASE-20878 Data loss if merging regions while ServerCrashProcedure executing
2018-07-24 09:57:47 +08:00
786fd9137a
HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing
2018-07-23 14:46:32 +08:00
654480326b
HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region
2018-07-22 20:51:58 -07:00
dd0325a740
HBASE-20401 Make MAX_WAIT and waitIfNotFinished in CleanerContext configurable (Contributed by Stephen Wu)
2018-07-23 10:32:17 +08:00
b15efbf8ed
HBASE-19893 restore_snapshot is broken in master branch when region splits
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-21 07:47:50 -07:00
e8a7a4472f
HBASE-20908 Infinite loop on regionserver if region replica are reduced
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-20 13:05:43 -07:00
810473b277
HBASE-20914 Trim Master memory usage
...
Add (weak reference) interning of ServerNames.
Correct Balancer regions x racks matrix.
Make smaller defaults when creating ArrayDeques.
2018-07-20 10:08:37 -07:00
1a6fae74b5
HBASE-20870 Wrong HBase root dir in ITBLL's Search Tool
2018-07-20 11:22:03 +08:00
37e5f5fb5f
HBASE-20672 New metrics ReadRequestRate and WriteRequestRate
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-07-19 15:41:53 -07:00
cf1f63bf91
HBASE-20901 Reducing region replica has no effect
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-19 13:39:44 -07:00
35a3c605f2
HBASE-6028 Start/Stop compactions at region server level
...
Add switching on/off of compactions.
Switching off compactions will also interrupt any currently ongoing compactions.
Adds a "compaction_switch" to hbase shell. Switching off compactions will
interrupt any currently ongoing compactions. State set from shell will be
lost on restart. To persist the changes across region servers modify
hbase.regionserver.compaction.enabled in hbase-site.xml and restart.
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-07-19 06:20:12 -07:00
e464230bde
HBASE-20907 Fix Intermittent failure on TestProcedurePriority
2018-07-19 12:01:16 +08:00
532d84d6d0
HBASE-20875 MemStoreLABImp::copyIntoCell uses 7% CPU when writing
...
Make the #copyCellInto method smaller so it inlines; we do it by
checking for the common type early and then taking a code path
that presumes ByteBufferExtendedCell -- avoids checks.
2018-07-18 20:41:05 -07:00
3078acb83c
HBASE-20860 Merged region's RIT state may not be cleaned after master restart
2018-07-17 17:33:11 +08:00
07fcdf9418
HBASE-20879 Compacting memstore config should handle lower case
2018-07-13 11:22:06 -07:00
d3bb364595
HBASE-20865 CreateTableProcedure is stuck in retry loop in CREATE_TABLE_WRITE_FS_LAYOUT state
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-13 08:26:05 -07:00
d7f4b06206
HBASE-19572 RegionMover should use the configured default port number and not the one from HConstants
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-07-13 10:46:01 +08:00
59830b0078
HBASE-20847 Addendum use addFront instead of addBack to add sub procedure
2018-07-12 08:31:21 +08:00
430b056ade
HBASE-20847 The parent procedure of RegionTransitionProcedure may not have the table lock
2018-07-11 17:37:21 +08:00
176afd998a
HBASE-20542-ADDENDUM: fix TestHStore
2018-07-11 11:31:28 +03:00
7990283d50
HBASE-20697 Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations()
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-07-11 11:07:04 +08:00
8decae9b92
HBASE-20784 Will lose the SNAPSHOT suffix if we get the version of RS from ServerManager
2018-07-10 10:00:11 +08:00
e6403c18ff
HBASE-20833 Modify pre-upgrade coprocessor validator to support table level coprocessors
...
- -jar parameter now accepts multiple jar files and directories of jar files.
- observer classes can be verified by -class option.
- -table parameter was added to check table level coprocessors.
- -config parameter was added to obtain the coprocessor classes from
HBase cofiguration.
- -scan option was removed.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-07-09 11:15:18 -05:00
ae22d57201
HBASE-20806 Split style journal for flushes and compactions
2018-07-09 18:42:30 +05:30
8a394ade4c
HBASE-20822 TestAsyncNonMetaRegionLocator is flakey
2018-07-09 14:56:42 +08:00
1da2fa3baa
HBASE-20808 (Addendum) Remove duplicate calls for cancelling of chores
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-07-07 00:18:22 +08:00
473c71edb1
HBASE-20808 Wrong shutdown order between Chores and ChoreService
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-07-06 11:37:13 +08:00
4653d4ac6b
HBASE-20691 Change the default WAL storage policy back to "NONE""
...
This reverts commit 564c193d61
2018-07-04 13:45:16 +08:00
341fa55367
HBASE-20474 Show non-RPC tasks on master/regionserver Web UI by default
2018-07-04 10:56:46 +08:00
5ef9d8f451
HBASE-20839 Fallback to FSHLog if we can not instantiated AsyncFSWAL when user does not specify AsyncFSWAL explicitly
2018-07-04 10:29:30 +08:00
c612642b2b
HBASE-20829 Remove the addFront assertion in MasterProcedureScheduler.doAdd
2018-07-04 09:40:52 +08:00
021eae48ca
HBASE-20244 NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-03 22:15:48 +08:00
8b1e6a81ff
HBASE-20789 TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky
2018-07-03 17:59:25 +08:00
30664b1a57
HBASE-20193 Basic Replication Web UI - Regionserver
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-03 16:02:28 +08:00
33d002f445
HBASE-20826 Truncate really long RpcServer warnings unless TRACE is on
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-07-03 10:14:52 +08:00
ad7e453817
HBASE-20450 Provide metrics for number of total active, priority and replication rpc handlers
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-02 11:38:02 -07:00
3847def1c2
HBASE-20817 Infinite loop when executing ReopenTableRegionsProcedure
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-02 21:28:53 +08:00
7868f0ef97
HBASE-20542: Better heap utilization for IMC with MSLABs
2018-07-01 15:52:30 +03:00
490ca0ce20
HBASE-20825 Fix pre and post hooks of CloneSnapshot and RestoreSnapshot for Access checks
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
2018-06-29 16:33:34 -04:00
a16e5523fc
HBASE-20357 AccessControlClient API Enhancement
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-06-29 07:54:14 -07:00
02235f4e32
HBASE-20792 info:servername and info:sn inconsistent for OPEN region
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-06-29 11:11:09 +08:00
58ccd3dc7e
HBASE-19722 Meta query statistics metrics source
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-06-28 17:17:20 -07:00
f264b79ec1
HBASE-20781 Save recalculating families in a WALEdit batch of Cells
...
Pass the Set of families through to the WAL rather than recalculate
a Set already known.
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-27 22:04:40 -07:00
32e0767dfd
HBASE-20732 Shutdown scan pool when master is stopped
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-28 11:51:52 +08:00
e61507b9a0
HBASE-19164: Remove UUID.randomUUID in tests.
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-27 10:36:48 -05:00
557985ee47
HBASE-20194 Basic Replication WebUI - Master
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-26 18:27:50 +08:00
a1b297e673
HBASE-20777 RpcConnection could still remain opened after we shutdown the NettyRpcServer
2018-06-26 09:07:58 +08:00
40cd76af8a
HBASE-20780 ServerRpcConnection logging cleanup Get rid of one of the logging lines in ServerRpcConnection by amalgamating all into one new-style log line.
2018-06-25 16:43:43 -07:00
4e16ecd7f4
HBASE-20770 WAL cleaner logs way too much; gets clogged when lots of work to do
...
General log cleanup; setting stuff that can flood the log to TRACE.
2018-06-25 12:12:43 -07:00
bf65e95466
HBASE-20403. Fix race between prefetch task and non-pread HFile reads
...
With prefetch-on-open enabled, the task doing the prefetching was using
non-positional (i.e. streaming) reads. If the main (non-prefetch) thread
was also using non-positional reads, these two would conflict, because
inputstreams are not thread-safe for non-positional reads.
In the case of an encrypted filesystem, this could cause JVM crashes,
etc, as underlying cipher buffers were freed underneath the racing
threads. In the case of a non-encrypted filesystem, less severe errors
would be thrown. The included unit test reproduces the latter case.
(cherry picked from commit 025ddce868
2018-06-25 12:11:36 -07:00
e915024e3c
HBASE-20778 Make it so WALPE runs on DFS
2018-06-23 23:34:17 -07:00
2f4b0a4a42
HBASE-20775 TestMultiParallel is flakey
2018-06-24 08:26:26 +08:00
dde042cc93
HBASE-20776 Update branch-2 version to 2.2.0-SNAPSHOT
2018-06-22 22:15:18 +08:00
108e636247
HBASE-18569 Add prefetch support for async region locator
2018-06-22 18:20:08 +08:00
a86141b625
HBASE-20752 Make sure the regions are truly reopened after ReopenTableRegionsProcedure
2018-06-22 14:06:29 +08:00
6cebe06225
HBASE-20767 Always close hbaseAdmin along with connection in HBTU
2018-06-22 10:20:06 +08:00
28d0d8c5cc
HBASE-20642 Clients should re-use the same nonce across DDL operations
...
Also changes modify table operations to help the case where a MTP spans
two master, avoiding the sanity-checks propagating back to the client
unnecessarily.
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-06-20 15:10:52 -07:00
c8b76eb3f1
HBASE-20706 Prevent MTP from trying to reopen non-OPEN regions
...
ModifyTableProcedure is using MoveRegionProcedure in a way
that was unintended from the original implementation. As such,
we have to guard against certain usages of it. We know we can
re-open OPEN regions, but regions in OPENING will similarly
soon be OPEN (thus, we want to reopen those regions too).
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-20 14:30:36 -07:00
5c2cb15e0b
HBASE-20739 Add priority for SCP
2018-06-20 15:18:47 +08:00
83969b0da1
HBASE-20742 Always create WAL directory for region server
2018-06-20 14:21:28 +08:00
9eeb501825
HBASE-20745 Log when master proc wal rolls
2018-06-19 19:53:29 -07:00
3e33aecea2
HBASE-20708 Remove the usage of RecoverMetaProcedure in master startup
2018-06-19 15:09:11 +08:00
ee84a8f243
HBASE-20332 shaded mapreduce module shouldn't include hadoop
...
* modify the jar checking script to take args; make hadoop stuff optional
* separate out checking the artifacts that have hadoop vs those that don't.
* * Unfortunately means we need two modules for checking things
* * put in a safety check that the support script for checking jar contents is maintained in both modules
* * have to carve out an exception for o.a.hadoop.metrics2. :(
* fix duplicated class warning
* clean up dependencies in hbase-server and some modules that depend on it.
* allow Hadoop to have its own htrace where it needs it
* add a precommit check to make sure we're not using old htrace imports
Conflicts:
hbase-backup/pom.xml
hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-18 14:02:48 -07:00
b04c976fe6
HBASE-20478 Update checkstyle to v8.2
...
Cannot go to latest (8.9) yet due to
https://github.com/checkstyle/checkstyle/issues/5279
* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-18 14:02:40 -07:00
8edd5d948a
HBASE-20737 put collection into ArrayList instead of addAll function
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-17 11:16:16 +08:00
b2afba580b
HBASE-20723 Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them
2018-06-16 01:34:53 -07:00
b68746c0b2
HBASE-20695 Implement table level RegionServer replication metrics
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-06-15 10:45:13 +08:00
bde9f08a83
HBASE-20625 refactor some WALCellCodec related code
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-06-14 19:46:33 +08:00
161dc7c7f3
HBASE-20722 Make RegionServerTracker only depend on children changed event
2018-06-14 08:38:53 +08:00
075523dd1e
HBASE-20561 The way we stop a ReplicationSource may cause the RS down
2018-06-13 18:05:27 +08:00
d44e8a7aff
HBASE-20656 Validate pre-2.0 coprocessors against HBase 2.0+
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-11 10:32:40 -05:00
4b0bbd839e
HBASE-20707 Move MissingSwitchDefault case check
...
Perform this check using error-prone instead of checkstyle because the
former can handle enum switches somewhat more intelligently.
2018-06-11 10:13:29 -05:00
6befdc43ba
HBASE-20700 Move meta region when server crash can cause the procedure to be stuck
2018-06-11 15:28:21 +08:00
4d971d0f48
HBASE-20698 (addendum) Master don't record right server version until new started region server call regionServerReport method
2018-06-10 08:32:01 +08:00
9d15e16946
HBASE-20698 Master don't record right server version until new started region server call regionServerReport method
2018-06-09 14:47:07 +08:00
4a5fe54d94
HBASE-20699 QuotaCache should cancel the QuotaRefresherChore service inside its stop()
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-06-08 07:22:52 -07:00
858eee20ec
HBASE-20702 Processing crash, skip ONLINE'ing empty rows
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-06-07 09:54:32 -07:00
271d93dc73
HBASE-20665: Changed log level of HBASE-8547 warning to debug
...
Closes #77
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-07 11:34:52 -04:00
00289b8ffa
HBASE-20683 Incorrect return value for PreUpgradeValidator
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-06 20:04:22 +02:00
d7b09de854
HBASE-20670 NPE in HMaster#isInMaintenanceMode
2018-06-04 15:19:45 -07:00
063eefe3b0
HBASE-20634 Reopen region while server crash can cause the procedure to be stuck; ADDENDUM
2018-06-04 12:38:56 -07:00
27e2c8c86b
HBASE-20628 SegmentScanner does over-comparing when one flushing
...
Signed-off-by: eshcar <eshcar@oath.com>
Signed-off-by: anoopsjohn <anoopsamjohn@gmail.com>
2018-06-04 09:50:13 -07:00
d834859404
HBASE-20634 Reopen region while server crash can cause the procedure to be stuck
...
A reattempt at fixing HBASE-20173 [AMv2] DisableTableProcedure concurrent to ServerCrashProcedure can deadlock
The scenario is a SCP after processing WALs, goes to assign regions that
were on the crashed server but a concurrent Procedure gets in there
first and tries to unassign a region that was on the crashed server
(could be part of a move procedure or a disable table, etc.). The
unassign happens to run AFTER SCP has released all RPCs that
were going against the crashed server. The unassign fails because the
server is crashed. The unassign used to suspend itself only it would
never be woken up because the server it was going against had already
been processed. Worse, the SCP could not make progress because the
unassign was suspended with the lock on a region that it wanted to
assign held making it so it could make no progress.
In here, we add to the unassign recognition of the state where it is
running post SCP cleanup of RPCs. If present, unassign moves to finish
instead of suspending itself.
Includes a nice unit test made by Duo Zhang that reproduces nicely the
hung scenario.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/FailedRemoteDispatchException.java
Moved this class back to hbase-procedure where it belongs.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/NoNodeDispatchException.java
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/NoServerDispatchException.java
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/NullTargetServerDispatchException.java
Specializiations on FRDE so we can be more particular when we say there
was a problem.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
Change addOperationToNode so we throw exceptions that give more detail
on issue rather than a mysterious true/false
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Undo SERVER_CRASH_HANDLE_RIT2. Bad idea (from HBASE-20173)
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
Have expireServer return true if it actually queued an expiration. Used
later in this patch.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Hide methods that shouldn't be public. Add a particular check used out
in unassign procedure failure processing.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
Check that server we're to move from is actually online (might
catch a few silly move requests early).
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
Add doc on ServerState. Wasn't being used really. Now we actually stamp
a Server OFFLINE after its WAL has been split. Means its safe to assign
since all WALs have been processed. Add methods to update SPLITTING
and to set it to OFFLINE after splitting done.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Change logging to be new-style and less repetitive of info.
Cater to new way in which .addOperationToNode returns info (exceptions
rather than true/false).
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Add looking for the case where we failed assign AND we should not
suspend because we will never be woken up because SCP is beyond
doing this for all stuck RPCs.
Some cleanup of the failure processing grouping where we can proceed.
TODOs have been handled in this refactor including the TODO that
wonders if it possible that there are concurrent fails coming in
(Yes).
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
Doc and removing the old HBASE-20173 'fix'.
Also updating ServerStateNode post WAL splitting so it gets marked
OFFLINE.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestServerCrashProcedureStuck.java
Nice test by Duo Zhang.
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Duo Zhang <palomino219@gmail.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-04 09:26:36 -07:00
4c95b82b61
HBASE-19761:Fix Checkstyle errors in hbase-zookeeper
...
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2018-06-02 10:17:27 +02:00
f46569a742
HBASE-20667 Rename TestGlobalThrottler to TestReplicationGlobalThrottler
2018-06-01 17:01:14 -07:00
d3e2248f12
HBASE-18116 Replication source in-memory accounting should not include bulk transfer hfiles
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-06-01 11:16:16 -07:00
53d29d53c4
HBASE-20592 Create a tool to verify tables do not have prefix tree encoding
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-01 19:22:49 +02:00
b22409d51d
Revert "HBASE-18116 fix replication source in-memory calculation by excluding bulk load file"
...
This reverts commit 050fae501a
2018-05-31 15:28:37 -07:00
050fae501a
HBASE-18116 fix replication source in-memory calculation by excluding bulk load file
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-31 14:22:12 -07:00
fc9743c17a
HBASE-20444 Addendum keep folks from looking at raw version component array.
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-31 14:17:41 -05:00
aaec02e0f5
HBASE-20646 TestWALProcedureStoreOnHDFS failing on branch-1
2018-05-30 14:44:54 -07:00
15bb234d51
Revert "TestWALProcedureStoreOnHDFS failing on branch-1"
...
This reverts commit 694e79a67e
2018-05-30 14:44:49 -07:00
694e79a67e
TestWALProcedureStoreOnHDFS failing on branch-1
2018-05-30 13:46:08 -07:00
b785896cbd
HBASE-20659 Implement a reopen table regions procedure
2018-05-30 20:03:35 +08:00
856a3ac154
HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint - revert due to pending discussion
2018-05-29 19:58:32 -07:00
2dc51934f4
HBASE-20597 Serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
2018-05-29 11:29:12 -07:00
7f154dc20e
Revert "HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint"
...
This reverts commit 60dcef289b
2018-05-29 11:24:30 -07:00
d36cce1574
HBASE-20633 Dropping a table containing a disable violation policy fails to remove the quota upon table delete
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-05-29 11:50:40 -04:00
aa00391140
HBASE-20390 ADDENDUM 2: fix TestHRegionWithInMemoryFlush OOME
2018-05-29 16:24:27 +03:00
cf1928aaca
HBASE-20390-ADDENDUM: fix TestHRegionWithInMemoryFlush OOME
2018-05-29 13:01:07 +03:00
c8fd6e0fb6
HBASE-20533 Fix the flaky TestAssignmentManagerMetrics
2018-05-29 09:50:04 +08:00
0455e75edd
HBASE-20648 HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-28 08:27:41 -07:00
bc72fcd8c5
HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-27 18:43:08 -07:00
f40c10a211
HBASE-20518 Need to serialize the enabled field for UpdatePeerConfigProcedure
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-25 14:45:49 +08:00
d1cbd561df
HBASE-20548 Master fails to startup on large clusters, refreshing block distribution
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-24 15:47:22 -07:00
db8789ab22
HBASE-20616 TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-24 15:17:01 -07:00
55c4791a63
HBASE-20588 Space quota change after quota violation doesn't seem to take in effect
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-05-24 12:50:22 -04:00
079e08d7c5
HBASE-20390: IMC Default Parameters for 2.0.0
2018-05-24 18:10:10 +03:00
bfab1e2f92
HBASE-20589 Don't need to assign meta to a new RS when standby master become active
2018-05-24 11:45:59 +08:00
a94c6dbadb
HBASE-20624 Race in ReplicationSource which causes walEntryFilter being null when creating new shipper
2018-05-24 10:48:35 +08:00
60dcef289b
HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
2018-05-23 16:46:20 -07:00
afddf6b1c2
HBASE-20620 HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; part 2
...
Adds new stripped-down, faster ByteBufferKeyValue comparator
(BBKV is the base Cell-type in hbase2). Creates an instance
of new Comparator each time we create new memstore rather
than use the universal CellComparator.
Remove unused and unneeded Interfaces from Cell base type.
2018-05-23 13:20:29 -07:00
fbda502435
HBASE-20612 TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn
2018-05-23 12:07:54 +08:00
c3c9a4a595
HBASE-20579 Improve snapshot manifest copy in ExportSnapshot
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-18 06:43:53 -07:00
d06673cf3e
HBASE-20583 SplitLogWorker should handle FileNotFoundException when split a wal
2018-05-18 14:30:40 +08:00
39ea1efa88
HBASE-20571 JMXJsonServlet generates invalid JSON if it has NaN in metrics
...
- CacheStats won't generate NaN metrics.
- JSONBean class will serialize special floating point values as
"NaN", "Infinity" or "-Infinity"
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-16 12:20:37 -07:00
61f2b5f071
HBASE-20567 Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace.
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-05-16 14:03:36 -05:00
77eaff0e10
HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; ADDENDUM
...
Add method the CellComparator Interface. Add implementation to
meta comparator so we don't fall back to the default comparator.
Includes a nothing change to hbase-server/pom.xml just to provoke
build.
2018-05-16 09:42:51 -07:00
Tyler Mi
Nihal Jain
Duo Zhang
Guangxu Cheng
Vasudevan
Toshihiro Suzuki
Michael Stack
Mingliang Liu
Huaxiang Sun
Reid Chan
David Manning
Xu Cang
Allan Yang
krish.dey
Umesh Agashe
tedyu
liubangchen
Andrey Elenskiy
Andrew Purtell
Sakthi
Josh Elser
jingyuntian
brandboat
meiyi
Monani Mihir
TAK LON WU
Wei-Chiu Chuang
Alex Leblang
Peter Somogyi
Saurabh Singh
huzheng
Ankit Singhal
Ankit
Mohit Goel
Yu Li
eshcar
zhaoyuan
Balazs Meszaros
Abhishek Singh Chouhan
Ted Yu
huzheng
Pankaj
Sahil Aggarwal
Todd Lipcon
Sean Busbey
Mike Drob
taiynlee
Guanghao Zhang
eric-maynard
maoling
Thiruvel Thirumoolan
Apekshit Sharma