38d6829a62
HBASE-21232 Show table state in Tables view on Master home page
2018-09-26 10:57:53 -07:00
51854a41fc
HBASE-21227 Implement exponential retrying backoff for Assign/UnassignRegionHandler introduced in HBASE-21217
2018-09-26 20:54:06 +08:00
f3c3c7c422
HBASE-21212 Wrong flush time when update flush metric
2018-09-26 19:17:35 +08:00
21fafbaf53
HBASE-20734 Colocate recovered edits directory with hbase.wal.dir
...
Amending-Author: Reid Chan <reidchan@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-09-26 11:49:21 +08:00
0f514ab75a
HBASE-21164 reportForDuty should do backoff rather than retry
...
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.
2018-09-25 11:32:15 -07:00
fa2888ebf0
HBASE-21217 Revisit the executeProcedure method for open/close region
2018-09-25 18:10:47 +08:00
20974513d9
Amend HBASE-20704 Sometimes some compacted storefiles are not archived on region close
...
Forward port small logging improvements from branch-1 version of this change.
2018-09-21 16:12:44 -07:00
fd68e7593e
HBASE-20636 Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
2018-09-21 16:06:34 -07:00
29214c77a6
HBASE-21214 [hbck2] setTableState just sets hbase:meta state, not in-memory state
2018-09-21 16:04:42 -07:00
648d58bc23
HBASE-21206 Scan with batch size may return incomplete cells
2018-09-20 22:24:17 +08:00
460cf6ff47
HBASE-21204 NPE when scan raw DELETE_FAMILY_VERSION and codec is not set
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-09-20 06:58:53 -07:00
899fddb4e7
HBASE-21023 Added bypassProcedure() API to HbckService
2018-09-19 15:17:52 -07:00
8876f12c0c
HBASE-21156 [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
...
Adds 'raw' assigns and unassigns methods to Hbck Service.
Fixes HbckService so it works when cluster is Kerberized.
2018-09-19 09:47:32 -07:00
84896deb1c
HBASE-21102 ServerCrashProcedure should select target server where no other replicas exist for the current region - addendum fixes NPE
2018-09-19 07:17:55 -07:00
13581e527d
HBASE-21102 - ServerCrashProcedure should select target server where no
...
other replicas exist for the current region - fix flaky test (Ram)
2018-09-18 11:30:49 +05:30
6616fd2e0f
HBASE-21191 Add a holding-pattern if no assign for meta or namespace (Can happen if masterprocwals have been cleared).
...
Add a check for hbase:meta being online before we go to read it.
If not online, move into a holding-pattern until rectified, probably
by external operator.
Incorporates bulk of patch made by Allan Yang over on HBASE-21035.
M hbase-common/src/main/java/org/apache/hadoop/hbase/util/RetryCounterFactory.java
Add a Constructor for case where retries are for ever.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Move stuff around so that the first hbase:meta read is the AM#loadMeta.
Previously, checking table state and/or favored nodes could end up
trying to read a meta that was not onlined holding up master startup.
Do similar for the namespace table. Adds new methods isMeta and
isNamespace which check that the regions/tables are online.. if not,
we wait logging with a back-off that assigns need to be run.
Signed-off-by: Allan Yang <allan163@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-16 21:43:48 -07:00
7a1b4d449d
HBASE-20704 Sometimes some compacted storefiles are not archived on region close
2018-09-16 18:28:24 -07:00
966e3751c2
HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning
...
Amending-Author: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-15 18:36:33 +08:00
e9d23d5d25
HBASE-21168 Insecure Randomness in BloomFilterUtil
...
Flagged by Fortify static analysis
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-12 09:51:57 -05:00
d40348e8cd
HBASE-21098 Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
...
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-11 16:31:56 -07:00
c0acad92d2
HBASE-21021 Result returned by Append operation should be ordered
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-11 14:55:32 -07:00
6e699b0694
HBASE-21172 Addendum fix comment
2018-09-11 17:32:38 +08:00
55e1297d96
HBASE-21172 Reimplement the retry backoff logic for ReopenTableRegionsProcedure
2018-09-11 15:14:02 +08:00
52d50cc426
HBASE-21173 Remove the duplicate HRegion#close in TestHRegion
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-11 11:20:23 +08:00
1a7dba44ce
HBASE-21158 Empty qualifier cell is always returned when using QualifierFilter
2018-09-10 21:51:48 +08:00
2be9196ba7
HBASE-21102 - ServerCrashProcedure should select target server where no
...
other replicas exist for the current region (ram)
2018-09-10 13:31:36 +05:30
d35e2f35a4
HBASE-21144 AssignmentManager.waitForAssignment is not stable
2018-09-10 12:19:04 +08:00
49501cd5b5
HBASE-21052 After restoring a snapshot, table.jsp page for the table gets stuck
2018-09-09 10:25:10 +09:00
1b65c2f161
HBASE-21001 ReplicationObserver fails to load in HBase 2.0.0
2018-09-07 23:44:03 +08:00
3f8c15ad2b
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir; ADDENDUM
...
Address review comments.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-09-06 17:25:20 -07:00
62919a3791
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
2018-09-06 16:45:53 -07:00
bbd9131c59
HBASE-21138 Close HRegion instance at the end of every test in TestHRegion
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-09-06 15:01:19 -07:00
1798032ace
HBASE-21161 Enable the test added in HBASE-20741 that was removed
...
accidentally (Ram)
2018-09-06 16:53:16 +05:30
83c871add8
HBASE-21107 add a metrics for netty direct memory
2018-09-05 16:10:17 -07:00
dee13ea401
HBASE-20892 [UI] Start / End keys are empty on table.jsp
2018-09-05 09:21:38 +08:00
1a5aedab34
HBASE-20741 Split of a region with replicas creates all daughter regions
...
and its replica in same server (Ram)
Signed-off-by: Huaxiang Sun, Michael Stack, Duo Zhang
2018-09-04 16:26:11 +05:30
0409c54ba7
HBASE-21129 Clean up duplicate codes in #equals and #hashCode methods of Filter
...
Signed-off-by Ted Yu <yuzhihong@gmail.com>
2018-09-04 11:15:57 +08:00
58365c8dd2
HBASE-21126 Configurable number of allowed failures for ZooKeeper Canary
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-09-03 19:35:28 -07:00
68c5313ca4
HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime
2018-09-02 18:32:24 +08:00
cf59f2a1d2
HBASE-21128 TestAsyncRegionAdminApi.testAssignRegionAndUnassignRegion is broken
2018-08-30 10:09:49 +08:00
72a29211cc
HBASE-15728 Add Per-Table metrics back
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableSource.java
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperAggregate.java
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableSourceImpl.java
hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperStub.java
hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsTableSourceImpl.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperAggregateImpl.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsTableAggregate.java
2018-08-29 16:27:15 -07:00
737ac48473
HBASE-21083 Introduce a mechanism to bypass the execution of a stuck procedure
2018-08-28 20:19:55 -07:00
22254cf4ce
HBASE-20942 Fix ArrayIndexOutOfBoundsException for RpcServer TRACE logging
...
Also makes the trace log message length configurable.
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-28 11:02:18 -04:00
fc21dc854b
HBASE-21017 Revisit the expected states for open/close
2018-08-28 14:49:47 +08:00
cdf5cfa396
HBASE-20941 Created and implemented HbckService in master
...
Added API setTableStateInMeta() to update table state only in Meta. This will be used by hbck2 tool.
2018-08-27 12:10:33 -07:00
a0cbfb8ea6
HBASE-21071 HBaseTestingUtility::startMiniCluster() to use builder pattern
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-08-27 10:22:04 -07:00
f9790b01d9
HBASE-21088 HStoreFile should be closed in HStore#hasReferences
2018-08-27 09:38:03 -07:00
4f32883e95
HBASE-21072 Addendum do not write lock file when running TestHBaseFsckReplication
2018-08-27 21:04:48 +08:00
2369afd7e5
HBASE-21084 When cloning a snapshot including a split parent region, the split parent region of the cloned table will be online
2018-08-27 09:58:52 +09:00
404d4d56cc
HBASE-21095 The timeout retry logic for several procedures are broken after master restarts
2018-08-26 18:17:20 +08:00
f533f01a3a
HBASE-20881 Introduce a region transition procedure to handle all the state transition for a region
2018-08-26 18:08:06 +08:00
2b7ef2e623
HBASE-20772 Controlled shutdown fills Master log with the disturbing message 'No matching procedure found for rit=OPEN, location=ZZZZ, table=YYYYY, region=XXXX transition to CLOSED'
...
Look for the particular case where RS does the close of region w/o
involving Master and log special message in this case. Dodgy. But
until we have Master run shutdown of all regions, better than
the message we currently show.
2018-08-25 14:30:27 +08:00
a83073aff0
HBASE-21078 [amv2] CODE-BUG NPE in RTP doing Unassign
2018-08-24 13:22:45 -07:00
6047fb05f5
HBASE-21113 Apply the branch-2 version of HBASE-21095, The timeout retry logic for several procedures are broken after master restarts
...
I applied the patch HBASE-21095 and then reverted it so could apply the
patch as HBASE-21113 (by reverting the HBASE-21095 revert but pushing
with this message!).
Revert "Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts""
This reverts commit a220566b98
2018-08-24 12:32:33 -07:00
a220566b98
Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts"
...
This reverts commit 09be1d0d57
2018-08-24 12:24:17 -07:00
09be1d0d57
HBASE-21095 The timeout retry logic for several procedures are broken after master restarts
2018-08-24 12:21:01 -07:00
2ea45e67d0
HBASE-21072 Block out HBCK1 in hbase2
...
Write the hbase-1.x hbck1 lock file to block out hbck1 instances writing
state to an hbase-2.x cluster (could do damage).
Set hbase.write.hbck1.lock.file to false to disable this writing.
2018-08-24 09:23:54 -07:00
c33af1e854
HBASE-21101 Remove the waitUntilAllRegionsAssigned call after split in TestTruncateTableProcedure
2018-08-24 10:35:05 +08:00
b318311dfd
HBASE-20193 Move TestCreateTableProcedure.testMRegions to a separated file
2018-08-24 10:09:23 +08:00
87f9b4accd
HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning
2018-08-23 11:49:47 -07:00
a7a281a644
HBASE-19008 Add missing equals or hashCode method(s) to stock Filter implementations
...
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-08-24 00:39:35 +08:00
cc6fc1f04f
HBASE-20194 Remove the explicit timeout config for TestTruncateTableProcedure
2018-08-23 06:27:37 +08:00
a7da458c5b
HBASE-21041 Memstore's heap size will be decreased to minus zero after flush
2018-08-22 23:03:20 +08:00
77e1a66f1e
HBASE-21031 Memory leak if replay edits failed during region opening
2018-08-22 22:22:35 +08:00
b5ae4e5f9e
HBASE-20666 Unsuccessful table creation leaves entry in hbase:rsgroup table
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-08-21 18:55:46 -07:00
d1b5113511
HBASE-21032 ScanResponses contain only one cell each
...
Amending-Author: Duo Zhang <zhangduo@apache.org>
2018-08-21 13:31:49 -07:00
3787c72953
HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region - addendum
2018-08-19 14:25:12 -07:00
52ba33aa73
HBASE-20940 HStore.cansplit should not allow split to happen if it has references (Vishal Khandelwal)
2018-08-17 15:02:22 -07:00
1fa67725c5
HBASE-21047 Object creation of StoreFileScanner thru constructor and close may leave refCount to -1 (Vishal Khandelwal)
2018-08-16 11:42:49 -07:00
75939775af
HBASE-20705 Having RPC quota on a table now no longer prevents Space Quota to be recreate/removed
...
Just added 2 test cases as the subtasks of this jira solves the issue
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-16 11:16:56 -04:00
cbe2fc113d
HBASE-21062 Correctly use the defaultProvider value on the Providers enum when constructing a WALProvider
2018-08-16 10:18:39 -04:00
d5abb92088
HBASE-20469 Directory used for sidelining old recovered edits files should be made configurable
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-08-15 18:01:24 -07:00
bee8566f2c
HBASE-20985 add two attributes when we do normalization
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 16:49:37 +08:00
d979f70142
HBASE-21025 Addendum missed a 'succ = true'
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 10:54:49 +08:00
51df2887c4
HBASE-21012 Revert the change of serializing TimeRangeTracker
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-08-11 22:25:10 +08:00
2311fa86d4
HBASE-21025 Add cache for TableStateManager
2018-08-10 21:11:45 +08:00
fb9aa8a95a
HBASE-18201 add UT and docs for DataBlockEncodingTool
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-08-10 11:16:36 +08:00
28c3336bfe
HBASE-20965 Separate region server report requests to new handlers
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-09 18:15:22 +08:00
6a8bd0fe84
HBASE-20813 Removed RPC quotas when the associated table/Namespace is dropped off
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-07 13:43:27 -04:00
1aacde4839
HBASE-20986 Separate the config of block size when we do log splitting and write Hlog
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-07 14:00:31 +08:00
b2cefab0e1
HBASE-20885 Removed entry for RPC quota from hbase:quota when RPC quota is removed
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-08-03 11:06:56 -04:00
06a92a3d20
HBASE-19036 Add action in Chaos Monkey to restart Active Namenode
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-08-02 05:00:16 -07:00
690d29bae7
HBASE-20856 PITA having to set WAL provider in two places
...
With this change if hbase.wal.meta_provider is not explicitly set,
it uses whatever set with hbase.wal.provider. this change avoids a use
case of unexpectedly using two different providers when only
hbase.wal.provider is set to non-default but not hbase.wal.meta_provider.
This change also include document (architecture.adoc) update
Also, this is a port from master to branch-2
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <Apache9@apache.org>
2018-08-01 14:36:29 -07:00
fc0c4660fa
HBASE-20989 Minor, miscellaneous logging fixes
...
Signed-off-by: Zach York <zyork@amazon.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-08-01 11:20:26 -07:00
bfb9c71e3b
HBASE-20794 add INFO level log to createTable operation
2018-08-01 11:04:18 -07:00
477d567fd7
HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing ADDENDUM: Rather than rollback, just do region reopens.
...
In split, reopen the parent if recovered.edits and in merge, reopen the
parent region or regions that happened to have recovered.edits on close.
2018-08-01 00:33:33 -07:00
777f3e88e2
HBASE-20935 HStore.removeCompactedFiles should log in case it is unable to delete a file (Vishal Khandelwal)
...
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
2018-07-31 15:59:53 -07:00
99714da30f
HBASE-20950 Helper method to configure secure DFS cluster for tests
...
Create a helper method HBaseKerberosUtils#setSecuredConfiguration().
TestSecureExport, TestSaslFanOutOneBlockAsyncDFSOutput,
SecureTestCluster and TestThriftSpnegoHttpServer uses this new helper
method.
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-31 13:37:10 -07:00
e14b60a539
HBASE-20886 [Auth] Support keytab login in hbase client
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-07-31 11:10:34 +08:00
584093c23f
HBASE-20538 Upgrade our hadoop versions to 2.7.7 and 3.0.3
2018-07-29 20:20:01 +08:00
5957fceda4
Revert "HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3"
...
This reverts commit 71a4c17d99
2018-07-29 20:19:16 +08:00
71a4c17d99
HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3
2018-07-29 20:04:53 +08:00
ab18440107
HBASE-19369 Switch to Builder Pattern In WAL
...
This patch switches to the builder pattern by adding a helper method.
It also checks to ensure that the pattern is available (i.e. that
HBase is running on a hadoop version that supports it).
Amending-Author: Mike Drob <mdrob@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-27 23:42:53 -05:00
51b0d0edd0
HBASE-20939 There will be race when we call suspendIfNotReady and then throw ProcedureSuspendedException
2018-07-27 21:25:22 +08:00
611ca11c9f
HBASE-20921 Possible NPE in ReopenTableRegionsProcedure
2018-07-27 09:34:25 +08:00
bb6f24cd59
HBASE-20932 Effective MemStoreSize::hashCode()
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-25 14:42:47 -07:00
3b84c9c8d2
HBASE-20649 Validate HFiles do not have PREFIX_TREE DataBlockEncoding
...
Amending-Author: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-07-25 09:12:17 -05:00
ac4b789f5f
HBASE-20867 RS may get killed while master restarts
2018-07-25 18:13:50 +08:00
0b283dae0d
HBASE-20846 Restore procedure locks when master restarts
2018-07-25 14:37:31 +08:00
097ae55708
HBASE-20928 Rewrite calculation of midpoint in binarySearch functions to prevent overflow
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-24 08:17:16 -07:00
Michael Stack
zhangduo
Allan Yang
Zach York
Mingliang Liu
Andrew Purtell
Guangxu Cheng
openinx
tianjingyun
Umesh Agashe
tedyu
Vasudevan
Francis Liu
Mike Drob
Tyler Mi
Nihal Jain
Toshihiro Suzuki
Huaxiang Sun
Reid Chan
David Manning
Xu Cang
krish.dey
liubangchen
Andrey Elenskiy
Sakthi
Josh Elser
brandboat
meiyi
Monani Mihir
TAK LON WU
Wei-Chiu Chuang
Alex Leblang
Peter Somogyi
Saurabh Singh