Commit Graph

2514 Commits

Author SHA1 Message Date
Ben Lau 13b68abb58
HBASE-21439 RegionLoads aren't being used in RegionLoad cost functions
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
2018-11-08 14:08:52 -08:00
tianjingyun 857abfb082
HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-29 20:56:11 -07:00
Guanghao Zhang f71768308c HBASE-21385 HTable.delete request use rpc call directly instead of AsyncProcess
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-25 16:22:18 -07:00
jingyuntian 4a609db30c
HBASE-21269 Forward-port HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign 2018-10-18 06:22:22 -07:00
Sahil Aggarwal ad6ca7c95a
HBASE-20716: Changes the bytes[] conversion done in Bytes and ByteBufferUtils. Instead of doing check unsafe_aligned available everytime, choose the best converter at startup. 2018-10-17 21:04:09 -07:00
Michael Stack 6125872f48 HBASE-21303 [shell] clear_deadservers with no args fails 2018-10-12 11:19:40 -07:00
Nihal Jain 3d1a2dbe68
HBASE-21196 HTableMultiplexer clears the meta cache after every put operation
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 16:35:53 -07:00
Michael Stack 2836f75625
HBASE-21223 [amv2] Remove abort_procedure from shell
Signed-off-by: Balazs Meszaros <balazs.meszaros@cloudera.com>
2018-09-25 11:28:54 -07:00
Guangxu Cheng fd68e7593e
HBASE-20636 Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
2018-09-21 16:06:34 -07:00
Umesh Agashe 899fddb4e7
HBASE-21023 Added bypassProcedure() API to HbckService 2018-09-19 15:17:52 -07:00
Michael Stack 8876f12c0c
HBASE-21156 [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
Adds 'raw' assigns and unassigns methods to Hbck Service.

Fixes HbckService so it works when cluster is Kerberized.
2018-09-19 09:47:32 -07:00
Guangxu Cheng e1548d3349 HBASE-21179 Fix the number of actions in responseTooSlow log 2018-09-12 10:45:55 +08:00
Guangxu Cheng 1a7dba44ce HBASE-21158 Empty qualifier cell is always returned when using QualifierFilter 2018-09-10 21:51:48 +08:00
Toshihiro Suzuki 49501cd5b5 HBASE-21052 After restoring a snapshot, table.jsp page for the table gets stuck 2018-09-09 10:25:10 +09:00
Reid Chan 0409c54ba7 HBASE-21129 Clean up duplicate codes in #equals and #hashCode methods of Filter
Signed-off-by Ted Yu <yuzhihong@gmail.com>
2018-09-04 11:15:57 +08:00
Toshihiro Suzuki 9b7c530b6c HBASE-21084 When cloning a snapshot including a split parent region, the split parent region of the cloned table will be online - addendum 2018-08-28 15:34:52 +09:00
Umesh Agashe cdf5cfa396 HBASE-20941 Created and implemented HbckService in master
Added API setTableStateInMeta() to update table state only in Meta. This will be used by hbck2 tool.
2018-08-27 12:10:33 -07:00
subrat.mishra 25e988dd43 HBASE-21030: Changed javadoc for append operation.
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2018-08-26 23:45:24 +09:00
zhangduo f533f01a3a HBASE-20881 Introduce a region transition procedure to handle all the state transition for a region 2018-08-26 18:08:06 +08:00
liubangchen a7a281a644 HBASE-19008 Add missing equals or hashCode method(s) to stock Filter implementations
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-08-24 00:39:35 +08:00
jingyuntian bee8566f2c HBASE-20985 add two attributes when we do normalization
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 16:49:37 +08:00
Sakthi b2cefab0e1 HBASE-20885 Removed entry for RPC quota from hbase:quota when RPC quota is removed
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-08-03 11:06:56 -04:00
Michael Stack fc0c4660fa HBASE-20989 Minor, miscellaneous logging fixes
Signed-off-by: Zach York <zyork@amazon.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-08-01 11:20:26 -07:00
Reid Chan e14b60a539 HBASE-20886 [Auth] Support keytab login in hbase client
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-07-31 11:10:34 +08:00
Allan Yang ac4b789f5f HBASE-20867 RS may get killed while master restarts 2018-07-25 18:13:50 +08:00
huzheng 99701968c8 HBASE-20565 ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result 2018-07-24 10:33:33 +08:00
Ankit Singhal cf1f63bf91 HBASE-20901 Reducing region replica has no effect
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-19 13:39:44 -07:00
Mohit Goel 35a3c605f2 HBASE-6028 Start/Stop compactions at region server level
Add switching on/off of compactions.

Switching off compactions will also interrupt any currently ongoing compactions.
Adds a "compaction_switch" to hbase shell. Switching off compactions will
interrupt any currently ongoing compactions. State set from shell will be
lost on restart. To persist the changes across region servers modify
hbase.regionserver.compaction.enabled in hbase-site.xml and restart.

Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-07-19 06:20:12 -07:00
Balazs Meszaros 29f1352edd HBASE-20853 Polish "Add defaults to Table Interface so Implementors don't have to"
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-07-19 19:57:09 +08:00
Mike Drob 2a9162a503
HBASE-20884 Replace Base64 with j.u.Base64
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-13 13:00:23 -05:00
zhaoyuan 7990283d50 HBASE-20697 Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations()
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-07-11 11:07:04 +08:00
zhangduo 8decae9b92 HBASE-20784 Will lose the SNAPSHOT suffix if we get the version of RS from ServerManager 2018-07-10 10:00:11 +08:00
zhangduo 8a394ade4c HBASE-20822 TestAsyncNonMetaRegionLocator is flakey 2018-07-09 14:56:42 +08:00
Michael Stack e2d0cdbc09 HBASE-20812 Add defaults to Table Interface so implementors don't have to
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-29 11:05:09 -07:00
Pankaj a16e5523fc HBASE-20357 AccessControlClient API Enhancement
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-06-29 07:54:14 -07:00
zhangduo 5ba314e3f3 HBASE-20810 Include the procedure id in the exception message in HBaseAdmin for better debugging 2018-06-29 11:40:48 +08:00
Michael Stack 4e16ecd7f4
HBASE-20770 WAL cleaner logs way too much; gets clogged when lots of work to do
General log cleanup; setting stuff that can flood the log to TRACE.
2018-06-25 12:12:43 -07:00
Michael Stack daa3e007ec HBASE-20710 extra cloneFamily() in Mutation.add(Cell); ADDENDUM checkstyle 2018-06-23 23:32:45 -07:00
Huaxiang Sun 6f4ff8dcf0 HBASE-20710 extra cloneFamily() in Mutation.add(Cell)
Signed-off-by: Michael Stack <stack@apache.org>
2018-06-23 23:32:37 -07:00
zhangduo 2f4b0a4a42 HBASE-20775 TestMultiParallel is flakey 2018-06-24 08:26:26 +08:00
zhangduo dde042cc93 HBASE-20776 Update branch-2 version to 2.2.0-SNAPSHOT 2018-06-22 22:15:18 +08:00
zhangduo 108e636247 HBASE-18569 Add prefetch support for async region locator 2018-06-22 18:20:08 +08:00
zhangduo a86141b625 HBASE-20752 Make sure the regions are truly reopened after ReopenTableRegionsProcedure 2018-06-22 14:06:29 +08:00
Ankit Singhal 28d0d8c5cc HBASE-20642 Clients should re-use the same nonce across DDL operations
Also changes modify table operations to help the case where a MTP spans
two master, avoiding the sanity-checks propagating back to the client
unnecessarily.

Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-06-20 15:10:52 -07:00
taiynlee 8aabe36a49 HBASE-20737 (addendum) put collection into ArrayList instead of addAll function -- RetriesExhaustedWithDetailsException.java
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-20 14:46:04 +08:00
zhangduo 3e33aecea2 HBASE-20708 Remove the usage of RecoverMetaProcedure in master startup 2018-06-19 15:09:11 +08:00
Sean Busbey ee84a8f243 HBASE-20332 shaded mapreduce module shouldn't include hadoop
* modify the jar checking script to take args; make hadoop stuff optional
* separate out checking the artifacts that have hadoop vs those that don't.
* * Unfortunately means we need two modules for checking things
* * put in a safety check that the support script for checking jar contents is maintained in both modules
* * have to carve out an exception for o.a.hadoop.metrics2. :(
* fix duplicated class warning
* clean up dependencies in hbase-server and some modules that depend on it.
* allow Hadoop to have its own htrace where it needs it
* add a precommit check to make sure we're not using old htrace imports

 Conflicts:
	hbase-backup/pom.xml
	hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml

Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-18 14:02:48 -07:00
Mike Drob b04c976fe6 HBASE-20478 Update checkstyle to v8.2
Cannot go to latest (8.9) yet due to
  https://github.com/checkstyle/checkstyle/issues/5279

* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors

Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-18 14:02:40 -07:00
Rajeshbabu Chintaguntla c0d67209df HBASE-20635 Add missing shaded-pb conversion method for UserPermission
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-06-11 15:13:17 -04:00
Guanghao Zhang 9d15e16946 HBASE-20698 Master don't record right server version until new started region server call regionServerReport method 2018-06-09 14:47:07 +08:00
EugeneNik 323eccc65b HBASE-20684 org.apache.hadoop.hbase.client.Scan#setStopRow javadoc uses incorrect method
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-05 21:58:06 +08:00
Josh Elser 26e7065cb4 HBASE-20640 Add missing test category and class rule
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-30 10:34:14 -04:00
Apekshit Sharma a2d5e3183c HBASE-20594 Provide utility to get table desc delta
Ammending Author: Mike Drob <mdrob@apache.org>

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-05-29 14:28:46 -05:00
huzheng fbda502435 HBASE-20612 TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn 2018-05-23 12:07:54 +08:00
zhangduo 60b8344cf1 HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream 2018-05-15 21:09:04 +08:00
Sean Busbey 61f96b6ffa HBASE-20544 Make HBTU default to random ports.
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Josh Elser <elserj@apache.org>

 Conflicts:
	hbase-backup/src/test/resources/hbase-site.xml
	hbase-spark-it/src/test/resources/hbase-site.xml
	hbase-spark/src/test/resources/hbase-site.xml
2018-05-09 23:45:39 -07:00
Nihal Jain 55badefa38 HBASE-20485 Copy constructor of Scan doesn't copy the readType and replicaId
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-05-09 16:45:39 +08:00
Mingdao Yang 2373451f94 HBASE-20527 Remove unused code in MetaTableAccessor
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-05-05 22:20:24 +08:00
Guangxu Cheng 3fc2c3bf34 HBASE-20484 Remove the unnecessary autoboxing in FilterListBase
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-04-26 16:32:14 +08:00
Sean Busbey 24eb141bac HBASE-20442 clean up incorrect use of commons-collections 3
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>

 Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/BackupLogCleaner.java
2018-04-20 08:17:22 -05:00
Huaxiang Sun 36680da549 HBASE-19994 Create a new class for RPC throttling exception, make it retryable 2018-04-17 10:45:44 -07:00
zhangduo ae8a21204d HBASE-20377 Deal with table in enabling and disabling state when modifying serial replication peer 2018-04-13 20:33:25 +08:00
Guangxu Cheng b0701434e3 HBASE-20243 [Shell] Add shell command to create a new table by cloning the existent table
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-11 19:34:30 -07:00
Balazs Meszaros b52bf4052f HBASE-20382 If RSGroups not enabled, rsgroup.jsp prints stack trace 2018-04-11 09:49:50 -07:00
zhangduo 3eee7d37f4 HBASE-20367 Write a replication barrier for regions when disabling a table 2018-04-11 20:36:59 +08:00
zhangduo 28d09cedcf HBASE-20182 Addendum throw IOException instead of NoServerForRegionException because it is a DoNotRetryRegionException 2018-04-11 14:47:41 +08:00
zhangduo 74ab10c353 HBASE-20296 Remove last pushed sequence ids when removing tables from a peer 2018-04-09 15:18:44 +08:00
zhangduo 9369cf26eb HBASE-20147 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves 2018-04-09 15:18:44 +08:00
zhangduo 644bfe36b2 HBASE-20117 Cleanup the unused replication barriers in meta table 2018-04-09 15:18:44 +08:00
zhangduo cea5199ea1 HBASE-20148 Make serial replication as a option for a peer instead of a table 2018-04-09 15:18:44 +08:00
zhangduo 8b61a061d3 HBASE-20129 Add UT for serial replication checker 2018-04-09 15:18:44 +08:00
zhangduo f29bf1d778 HBASE-20115 Reimplement serial replication based on the new replication storage layer 2018-04-09 15:18:44 +08:00
zhangduo 39c1ddc6e3 HBASE-20362 TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky 2018-04-09 14:55:42 +08:00
zhangduo dcc1d9e36a HBASE-20182 Can not locate region after split and merge 2018-04-07 13:37:59 +08:00
BELUGA BEHR 039bc73571 HBASE-19488 Move to using Apache commons CollectionUtils
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-04-04 14:16:33 -07:00
Michael Stack d4e115bf34 HBASE-20259 Doc configs for in-memory-compaction and add detail to
in-memory-compaction logging

Adds logging of CompactingMemStore configuration on construction.

Add logging of detail about Store on creation including memstore type.

Add chapter to refguide on new in-memory compaction feature.
2018-04-02 09:56:32 -07:00
Yu Li b72e19e3b9 HBASE-20159 Support using separate ZK quorums for client 2018-03-29 03:20:52 +08:00
Toshihiro Suzuki e1afc5932a HBASE-20229 ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on 2018-03-26 15:13:13 -07:00
Jan Hentschel 52014b9423 HBASE-13300 Fixed casing in set/getTimeStamp for Mutations 2018-03-23 17:23:55 -07:00
Chia-Ping Tsai 6aba045aae HBASE-19504 Add TimeRange support into checkAndMutate
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-24 00:05:22 +08:00
Umesh Agashe 96d63fee11 HBASE-20224 Web UI is broken in standalone mode
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 20:28:08 -07:00
Michael Stack 79e4c9d925
Revert "HBASE-20224 Web UI is broken in standalone mode"
Broke shell tests.

This reverts commit dd9fe813ec.
2018-03-22 10:47:47 -07:00
Umesh Agashe dd9fe813ec HBASE-20224 Web UI is broken in standalone mode
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 06:52:51 -07:00
Chia-Ping Tsai dd9e46bbf5 HBASE-20212 Make all Public classes have InterfaceAudience category
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:09:54 +08:00
Andrew Purtell 4fcc1ac7f7 HBASE-19024 Provide a configurable option to hsync WAL edits to the disk for better durability (Harshal Jain) 2018-03-20 17:30:39 -07:00
Chia-Ping Tsai 7ba86d056c HBASE-20119 (addendum) revert the removed methods in TableDescriptorBuilder and TableDescriptor
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-19 09:08:29 +08:00
Michael Stack 3f1c86786c HBASE-20213 [LOGGING] Aligning formatting and logging less (compactions,
in-memory compactions)

Log less. Log using same format as used elsewhere in log.

Align logs in HFileArchiver with how we format elsewhere. Removed
redundant 'region' qualifiers, tried to tighten up the emissions so
easier to read the long lines.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
 Add a label for each of the chunkcreators we make (I was confused by
two chunk creater stats emissions in log file -- didn't know that one
was for data and the other index).

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
 Formatting. Log less.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactionStrategy.java
 Make the emissions in here trace-level. When more than a few regions,
log is filled with this stuff.
2018-03-16 13:07:34 -07:00
Michael Stack 79d47dd57a HBASE-20202 [AMv2] Don't move region if its a split parent or offlined
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/DoNotRetryRegionException.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/MergeRegionException.java
 Allow passing cause to Constructor.

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
 Add prepare step to move procedure.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
 Add check that regions to merge are actually online to the Constructor
so we can fail fast if they are offline

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
 Add prepare step. Check regions and context and skip move if not right.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
 Add check parent region is online to constructor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
 Add generic check region is online utility function for use by subclasses.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
 Add test that we fail if we try to move an offlined region.
2018-03-16 09:34:15 -07:00
Chia-Ping Tsai 95596e8ba7 HBASE-20119 Introduce a pojo class to carry coprocessor information in order to make TableDescriptorBuilder accept multiple cp at once
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-16 01:26:08 +08:00
Michael Stack 63ce0c0733 HBASE-20178 [AMv2] Throw exception if hostile environment
Add Fail-Fast to Procedures by throwing exception out of Procedure
constructor so if move but table is disabled or if master is going
down, etc., we can give notice before the procedure is scheduled.
Will help guard against scheduling Procedures that will have a hard
time succeeding; e.g. a move when table is offline.

Also fixed bug around table state where we presumed ENABLED though no
entry in hbase:meta (we were using this mechanism for whether a table
existed or not).

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
 Test stolen from HBASE-20131

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableState.java
 Add convenience isEnabled/isDisabled

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Promote assert state to throw exception.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
 Add isClusterUp

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Move constructor now throws exception
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RestoreSnapshotProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java
 Do environment check at construction and fail-fast if hostile.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
 Add preflightCheck utility method.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Removed setting time setting table state; broke when using other than
 default environment edge masked by presumption that no state meant
 active.
2018-03-14 14:43:34 -07:00
maoling fac3b8aad6 HBASE-20047 AuthenticationTokenIdentifier should provide a toString
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-12 22:16:22 +08:00
Guanghao Zhang d36aacdf9e HBASE-19636 All rs should already start work with the new peer change when replication peer procedure is finished
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
zhangduo 525fef572e HBASE-19633 Clean up the replication queues in the postPeerModification stage when removing a peer 2018-03-09 20:55:48 +08:00
huzheng 1e36a84afc HBASE-19622 Reimplement ReplicationPeers with the new replication storage interface 2018-03-09 20:55:48 +08:00
huzheng 3fd417600e HBASE-19525 RS side changes for moving peer modification from zk watcher to procedure 2018-03-09 20:55:48 +08:00
Guanghao Zhang 750b7d8dbe HBASE-19536 Client side changes for moving peer modification from zk watcher to procedure
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
Sean Busbey 71cc7869db HBASE-20155 update branch-2 version to 2.1.0-SNAPSHOT
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2018-03-08 08:44:30 -08:00
Mike Drob af9a108a93 HBASE-19987 error-prone 2.2.0 2018-03-07 21:21:05 -06:00
Sean Busbey 9927c2e14a HBASE-20070 refactor website generation
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>

 Conflicts:
	hbase-backup/pom.xml
	hbase-spark-it/pom.xml
2018-03-02 09:51:43 -06:00
Michael Stack a2de29560f HBASE-20113 Move branch-2 version from 2.0.0-beta-2-SNAPSHOT to 2.0.0-beta-2 2018-03-01 15:46:38 -08:00
zhangduo 10270e36c9 HBASE-20065 Addendum remove wrong comment 2018-02-26 09:48:55 +08:00
zhangduo 2d5065bc7e HBASE-20065 Revisit the timestamp usage in MetaTableAccessor 2018-02-25 09:14:09 +08:00
Sean Busbey a66fa85a16 Revert "HBASE-19835 Use explicit casting to avoid side effects"
This reverts commit bc080e7500.

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Segment.java

patch reverted changes that happened in parallel without explanation. see jira.
2018-02-23 23:04:34 -06:00
zhangduo 4ddfdaffdc HBASE-20048 Revert serial replication feature 2018-02-23 13:54:10 +08:00
Chia-Ping Tsai bc1ac49de2 HBASE-20017 BufferedMutatorImpl submit the same mutation repeatedly
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-20 16:59:48 -08:00
zhangduo 34d3e847cc HBASE-20020 Make sure we throw DoNotRetryIOException when ConnectionImplementation is closed 2018-02-20 06:10:45 -08:00
Reid Chan 4ef6319af0 HBASE-19950 Introduce a ColumnValueFilter
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-20 05:05:19 +08:00
eshcar a458d7c400 HBASE-18294 Reduce global heap pressure: flush based on heap occupancy 2018-02-18 12:58:52 +02:00
Chia-Ping Tsai 34d5f2b70e HBASE-19680 BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations 2018-02-17 07:27:23 +08:00
Mike Drob 138f82c8c5 HBASE-19920 Lazy init for ProtobufUtil classloader 2018-02-16 09:11:45 -06:00
Chia-Ping Tsai 6923472f75 HBASE-19876 The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner 2018-02-13 21:20:21 +08:00
Michael Stack c7473df2c3 HBASE-16060 1.x clients cannot access table state talking to 2.0 cluster
This patch adds mirroring of table state out to zookeeper. HBase-1.x
clients look for table state in zookeeper, not in hbase:meta where
hbase-2.x maintains table state.

The patch also moves and refactors the 'migration' code that was put in
place by HBASE-13032.

D hbase-client/src/main/java/org/apache/hadoop/hbase/CoordinatedStateException.java
 Unused.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Move table state migration code from Master startup out to
TableStateManager where it belongs. Also start
MirroringTableStateManager dependent on config.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MirroringTableStateManager.java

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
 Move migration from zookeeper of table state in here. Also plumb in
mechanism so subclass can get a chance to look at table state as we do
the startup fixup full-table scan of meta.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Bug-fix. Now we create regions in CLOSED state but we fail to check
table state; were presuming table always enabled. Meant on startup
there'd be an unassigned region that never got assigned.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMirroringTableStateManager.java
 Test migration and mirroring.
2018-02-12 08:22:14 -08:00
Aman Poonia bc080e7500 HBASE-19835 Use explicit casting to avoid side effects
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-08 12:33:08 -08:00
Michael Stack 8f1e01b6e5 HBASE-19951 Cleanup the explicit timeout value for test method 2018-02-07 16:39:54 -08:00
Chia-Ping Tsai e2636c81f7 HBASE-19900 Region-level exception destroy the result of batch 2018-02-07 08:38:04 +08:00
Umesh Agashe 5905415f9c HBASE-19915 (addendum): Fixed a typo because of which only daughterA was getting stored with a CLOSED state and not daughterB 2018-02-05 15:41:28 -08:00
Mike Drob e69c1fd7a0 HBASE-19922 remove ProtobufUtil::PRIMITIVES 2018-02-05 16:55:45 -06:00
Michael Stack 0ca7a2e916 HBASE-19726 Failed to start HMaster due to infinite retrying on meta assign; ADDENDUM Fix failing TestMetaWithReplicas#testShutdownHandling; it was reading meta TableState""
Retry applying this addendum. Previous commit added it w/ wrong log
message so had to revert and then reapply here.
2018-02-03 21:49:42 -08:00
Michael Stack cd610607e7 Revert "HBASE-19928 TestVisibilityLabelsOnNewVersionBehaviorTable fails; ADDENDUM Fix failing TestMetaWithReplicas#testShutdownHandling; it was reading meta TableState"
Wrong log message!

This reverts commit fbcb453ce2.
2018-02-03 21:48:39 -08:00
Michael Stack fbcb453ce2 HBASE-19928 TestVisibilityLabelsOnNewVersionBehaviorTable fails; ADDENDUM Fix failing TestMetaWithReplicas#testShutdownHandling; it was reading meta TableState 2018-02-03 21:48:18 -08:00
zhangduo cb138c2da8 HBASE-19914 Refactor TestVisibilityLabelsOnNewVersionBehaviorTable 2018-02-04 13:37:09 +08:00
Michael Stack bac4687345 HBASE-19919 Tidying up logging 2018-02-02 22:42:30 -08:00
Umesh Agashe fdbc843516
HBASE-19915 Create merged/ daughter region/s with initial state CLOSED from MetaTableAccessor#splitRegion()/ mergeRegions()
If the initial state is not set to CLOSED while creating merged/ children region/s, merged/
children region/s get stored in meta with state OFFLINE(0). If the master gets restarted after
Merge/ Split procedure stores merged/ children region/s, master startup sequence tries to assign
offline region/s. This is followed by re-assignment of merged/ children region from resumed Merge/
Split procedure.

This bug is related to HBASE-19530.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-02 18:24:42 -08:00
Chia-Ping Tsai 6a1ed25581 HBASE-19897 RowMutations should follow the fluent pattern 2018-02-02 05:24:08 +08:00
zhangduo 28d13c54a9 HBASE-19887 Do not overwrite the surefire junit listener property in the pom of sub modules 2018-01-31 20:49:25 +08:00
Michael Stack 90a75fb052 HBASE-19888 Move branch-2 version from 2.0.0-beta-1 to 2.0.0-beta-2-SNAPSHOT 2018-01-29 14:17:54 -08:00
Duo Zhang bbf3bae72a
HBASE-19873 Add a CategoryBasedTimeout ClassRule for all UTs 2018-01-29 12:41:14 -08:00
zhangduo 0dea3656ab HBASE-19870 Fix the NPE in ReadOnlyZKClient#run
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-29 16:22:40 +08:00
Chia-Ping Tsai f952779ba2 HBASE-4224 Need a flush by regionserver rather than by table option 2018-01-23 09:47:15 +08:00
Sergey Soldatov 6f8c31242a HBASE-19774 incorrect behavior of locateRegionInMeta 2018-01-22 11:49:19 -08:00
Guangxu Cheng c01dc69123 HBASE-19799 Add web UI to rsgroup
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-18 19:30:06 -08:00
zhangduo 8dd4bf8e54 HBASE-19795 Move the tests which only need zookeeper in TestZooKeeper to hbase-zookeeper module 2018-01-15 10:58:03 +08:00
zhangduo 842f794a62 HBASE-19772 Do not close connection to zk when there are still pending request in ReadOnlyZKClient 2018-01-14 17:08:30 +08:00
Michael Stack a917a4e796 HBASE-19753 Miscellany of fixes for hbase-zookeeper tests to make them more robust
First, we add test resources to CLASSPATH when tests run. W/o it, there
was no logging of hbase-zookeeper test output (not sure why I have to
add this here and not over in hbase-server; research turns up nothing
so far).

M hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKMainServer.java
 Improve fail log message.

M hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestReadOnlyZKClient.java
M hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKNodeTracker.java
 Wait until ZK is connected before progressing. On my slow zk, it could
 be a while post construction before zk connected. Using an unconnected
 zk caused test to fail.

M hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKMainServer.java
 Change session timeout to default 30s from 1s which was way too short.

M hbase-zookeeper/src/test/resources/log4j.properties
 Set zk logs to DEBUG level in this module at least.

Adds a ZooKeeperHelper class that has utility to help interacting w/ ZK.
2018-01-11 11:22:56 -08:00
tedyu 47326729fe HBASE-19740 Repeated error message for NamespaceExistException 2018-01-10 18:26:23 -08:00
Josh Elser efa911f56f HBASE-19734 Fix IntegrationTestReplication and related impl changes
Adds (client-side) validation to ReplicationPeerConfigBuilder and javadoc
to builder methods in addition to the test fix.

Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-10 13:51:22 -05:00
Guanghao Zhang 895267d09c HBASE-19139 Create Async Admin methods for Clear Block Cache 2018-01-09 14:55:21 +08:00
Mike Drob b0a92bd4e1
HBASE-19721 Unnecessary stubbings in TestReversedScannerCallable
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-06 08:57:20 -06:00
Beluga Behr 88c7493948
HBASE-19651 Remove LimitInputStream 2018-01-05 17:01:50 -06:00
Michael Stack e7da907c3c
Revert "HBASE-19651 Remove LimitInputStream"
Revert to fix author.

This reverts commit 4c19e1f4e3.
2018-01-05 16:59:59 -06:00
Michael Stack 4c19e1f4e3
HBASE-19651 Remove LimitInputStream
Signed-off-by: Beluga Behr <dam6923@gmail.com>
2018-01-05 08:38:12 -06:00
Chia-Ping Tsai 654edc5fa5 HBASE-19596 RegionMetrics/ServerMetrics/ClusterMetrics should apply to all public classes 2018-01-04 13:05:21 +08:00
BELUGA BEHR 72631a08c1 HBASE-19473 Miscellaneous changes to ClientScanner
- Remove superfluous logging code guard
- Simplify some of the code
- Use ArrayDeque instead of LinkedList for queue implementation
2018-01-03 20:26:18 -08:00
zhangduo 4bf601a92d HBASE-19641 AsyncHBaseAdmin should use exponential backoff when polling the procedure result 2018-01-03 18:32:58 +08:00
BELUGA BEHR b6bff5f8e2 HBASE-19683 Remove Superfluous Methods From String Class (BELUGA BEHR).
* Remove isEmpty method
* Remove repeat
Use the Apache Commons implementations instead.
2018-01-02 11:34:52 -08:00
Niels Basjes 0dcbba1563 HBASE-19486: Ensure threadsafe WriteBufferPeriodicFlush operations
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-02 17:38:45 +08:00
Niels Basjes e2f07aafb6 HBASE-19486 Periodically ensure records are not buffered too long by BufferedMutator
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-29 22:24:30 +08:00
Chia-Ping Tsai 5612f2f692 HBASE-19428 Deprecate the compareTo(Row) 2017-12-29 20:07:21 +08:00
Michael Stack a8ef734ddd HBASE-19651 Remove LimitInputStream
Signed-off-by: Beluga Behr <dam6923@gmail.com>
2017-12-28 13:08:46 -08:00
Mike Drob 64cb777a8a HBASE-19552 find-and-replace thirdparty offset 2017-12-28 12:01:25 -06:00
Guanghao Zhang 4dc38aeb2f HBASE-19653 Reduce the default hbase.client.start.log.errors.counter
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-28 07:56:22 -08:00
Guanghao Zhang 33ce14cb29 HBASE-19643 Need to update cache location when get error in AsyncBatchRpcRetryingCaller 2017-12-28 14:44:17 +08:00
Chia-Ping Tsai 97f0aad66e HBASE-19626 Rename Cell.DataType to Cell.Type 2017-12-28 12:59:28 +08:00
Michael Stack 50348cc1a4 HBASE-19652 Turn down CleanerChore logging; too chatty 2017-12-27 20:50:46 -08:00
Michael Stack d6d8369655
HBASE-19648 Move branch-2 version from 2.0.0-beta-1-SNAPSHOT to 2.0.0-beta-1 2017-12-27 14:41:19 -08:00
Michael Stack e4ab29ada2 HBASE-19647 Logging cleanups; emit regionname when RegionTooBusyException inside RetriesExhausted... make netty connect/disconnect TRACE-level; ADDENDUM 2017-12-27 14:14:37 -08:00
Chia-Ping Tsai 7dee1bcd31 HBASE-19644 add the checkstyle rule to reject the illegal imports 2017-12-28 04:17:45 +08:00
Michael Stack cc8b37f847 HBASE-19647 Logging cleanups; emit regionname when RegionTooBusyException inside RetriesExhausted... make netty connect/disconnect TRACE-level 2017-12-27 11:56:40 -08:00
Chia-Ping Tsai 2468f30029 HBASE-19628 ByteBufferCell should extend ExtendedCell 2017-12-27 23:02:18 +08:00
Vasudevan 0454a389a3 HBASE-19629 RawCell#getTags should return the Iterator<Tag> in order to avoid iterating through whole tag array at once
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-27 21:30:08 +08:00
Peter Somogyi 15ed748286 HBASE-19545 Replace getBytes(StandardCharsets.UTF_8) with Bytes.toBytes
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-27 20:10:50 +08:00
Chia-Ping Tsai b6781c9bcc HBASE-19496 (addendum) don't store the duplicate cp names in ServerMetrics 2017-12-27 07:08:31 +08:00
Chia-Ping Tsai 8362b0dba5 HBASE-19550 Wrap the cell passed via Mutation#add(Cell) to be of ExtendedCell 2017-12-26 16:43:01 +08:00
Guanghao Zhang 2abf7b508c HBASE-19621 Revisit the methods in ReplicationPeerConfigBuilder 2017-12-26 14:44:07 +08:00
Guanghao Zhang b552eb2ff9 HBASE-19576 Introduce builder for ReplicationPeerConfig and make it immutable 2017-12-23 19:04:58 +08:00
Chia-Ping Tsai d4af099e9e HBASE-19496 Reusing the ByteBuffer in rpc layer corrupt the ServerLoad and RegionLoad 2017-12-22 18:58:08 +08:00
Guanghao Zhang 3a210d514c HBASE-19590 Remove the duplicate code in deprecated ReplicationAdmin 2017-12-22 18:36:28 +08:00
Apekshit Sharma 11ea19a101 HBASE-19589 New regions should always be added with state CLOSED (followup of HBASE-19530) 2017-12-21 22:58:28 -08:00
Chia-Ping Tsai 4aec7a491a HBASE-19575 add copy constructor to Mutation 2017-12-22 03:19:47 +08:00
Balazs Meszaros 992b5d8630 HBASE-10092 Move up on to log4j2
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test

This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.

Signed-off-by: Michael Stack <stack@apache.org>
2017-12-20 22:58:12 -08:00
Jan Hentschel d111b17c36 HBASE-19538 Removed unnecessary semicolons in hbase-client 2017-12-19 20:56:35 +01:00
Guanghao Zhang 60cd494d1c HBASE-19492 Add EXCLUDE_NAMESPACE and EXCLUDE_TABLECFS support to replication peer config 2017-12-19 16:57:38 +08:00
Michael Stack b7268cafac HBASE-19122 Suspect methods on Cell to be deprecated; ADDENDUM to fix
compile error
2017-12-18 16:03:42 -08:00
Michael Stack 495bedea35
HBASE-19122 Suspect methods on Cell to be deprecated 2017-12-18 15:23:03 -08:00
Yun Chi-Shih 5895719420 HBASE-19531 Remove needless volatile declaration
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-19 00:40:37 +08:00
Guanghao Zhang 8fa08b56f3 HBASE-19522 The complete order may be wrong in AsyncBufferedMutatorImpl 2017-12-16 14:09:35 +08:00
Apekshit Sharma 15a4aa0ce1 HBASE-19530 New regions should always be added with state CLOSED 2017-12-15 21:21:37 -08:00
Peter Somogyi 2043c337d8
HBASE-19498 Fix findbugs and error-prone warnings in hbase-client (branch-2)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-12-15 15:57:01 -08:00
Mike Drob 23a9059cb2 HBASE-18838 Fix hadoop3 check-shaded-invariants 2017-12-15 13:20:54 -06:00
Chia-Ping Tsai 33d48754f5
HBASE-19462 Deprecate all addImmutable methods in Put
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-14 14:18:04 -08:00
zhangduo c2bb63fbf9 HBASE-19512 Move EventType and ExecutorType from hbase-client to hbase-server 2017-12-14 20:15:35 +08:00
tedyu b682ea7c8a HBASE-19434 create_namespace command for existing namespace does not throw useful error message 2017-12-13 16:01:38 -08:00
Guangxu Cheng 015b66103a HBASE-19000 Group multiple block cache clear requests per server
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-13 07:47:33 -08:00
BELUGA BEHR 7c871849de HBASE-19472 Remove ArrayUtil Class
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-13 19:08:54 +08:00
Chia-Ping Tsai 55eb59e654 HBASE-19427 Add TimeRange support into Append to optimize for counters 2017-12-13 15:40:56 +08:00
Peter Somogyi 941acc5c05 HBASE-19213 Align check and mutate operations in Table and AsyncTable
- Deprecates old checkAnd*() operations in Table
- Adds Table#CheckAndMutateBuilder and implements it in HTable

Commiter note: When committing the patch, noticed redundant {@inheritDoc} being added in HTable.
Removed new and olds ones.

Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-12-12 20:46:39 -08:00
tedyu 0e47ded1a5 HBASE-19394 Support multi-homing env for the publication of RS status with multicast (hbase.status.published) (Toshihiro Suzuki) 2017-12-12 07:48:40 -08:00
Michael Stack 3366ebdc56
HBASE-19461 TestRSGroups is broke 2017-12-08 15:10:23 -08:00
Chia-Ping Tsai 2245dfc558 HBASE-12444 Total number of requests overflow because it's int 2017-12-07 18:42:32 +08:00
anoopsamjohn f8e2323323 HBASE-19357 Bucket cache no longer L2 for LRU cache. 2017-12-07 15:06:30 +05:30
anoopsamjohn e88b3c65b9 HBASE-19439 Mark ShortCircuitMasterConnection with InterfaceAudience Private. 2017-12-07 11:24:00 +05:30
Michael Stack d29ffd4e29 HBASE-19295 The Configuration returned by CPEnv should be read-only.
Adds a ReadOnlyConfiguration that delegates gets but throws exception
on sets/adds, etc.
2017-12-06 15:18:53 -08:00
anoopsamjohn 78a6e0532e Revert "HBASE-19357 Bucket cache no longer L2 for LRU cache."
This reverts commit d952c5df9f.
2017-12-06 11:36:19 +05:30
anoopsamjohn d952c5df9f HBASE-19357 Bucket cache no longer L2 for LRU cache. 2017-12-06 11:26:10 +05:30
Michael Stack c77778dd13
HBASE-19426 Move has() and setTimestamp() to Mutation (Chia-Ping Tsai)
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-05 09:42:13 -08:00
zhangduo c61ed5dc7e HBASE-19399 Purge curator dependency from hbase-client 2017-12-03 08:30:43 +08:00
Enis Soztutar 4f642f659f HBASE-16239 Better logging for RPC related exceptions
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-03 00:45:08 +08:00
Chia-Ping Tsai 2bda22a64e HBASE-19350 TestMetaWithReplicas is flaky 2017-12-01 03:40:17 +08:00
Apekshit Sharma 4f4aac77e1 HBASE-19367 Refactoring in RegionStates, and RSProcedureDispatcher
- Adding javadoc comments
- Bug: ServerStateNode#regions is HashSet but there's no synchronization to prevent concurrent addRegion/removeRegion. Let's use concurrent set instead.
- Use getRegionsInTransitionCount() directly to avoid instead of getRegionsInTransition().size() because the latter copies everything into a new array - what a waste for just the size.
- There's mixed use of getRegionNode and getRegionStateNode for same return type - RegionStateNode. Changing everything to getRegionStateNode. Similarly rename other *RegionNode() fns to *RegionStateNode().
- RegionStateNode#transitionState() return value is useless since it always returns it's first param.
- Other minor improvements
2017-11-29 22:42:39 -08:00
Guanghao Zhang 9e7ab9ffe2
HBASE-19359 Revisit the default config of hbase client retries number 2017-11-29 10:33:20 -08:00
Jerry He 0c4c395538 HBASE-19096 Add RowMutions batch support in AsyncTable
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-11-28 18:49:08 -08:00
Balazs Meszaros e946d9d841
HBASE-19242 Add MOB compact support for AsyncAdmin
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghaobac@gmail.com>
2017-11-28 15:04:56 -08:00
Josh Elser 4fef4cfc30 HBASE-19267 Remove compiler-plugin mapping executions as it breaks Java8 detection
It seems like the original reason this execution filter was added is no
longer an issue for 2.0. Actually, these entries actually preclude
Eclipse from correctly using the Java8 source/target version that we
have specified (which creates numerous compilation errors in Eclipse)

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-11-28 14:54:32 -05:00
Peter Somogyi 12efae4829 HBASE-19351 Deprecated is missing in Table implementations
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-28 11:18:07 -08:00
Guanghao Zhang 0406d06533 HBASE-19337 AsyncMetaTableAccessor may hang when call ScanController.terminate many times 2017-11-25 19:39:37 +08:00
Vasudevan 6ac6ae3fa2 HBASE-19092 Make Tag IA.LimitedPrivate and expose for CPs (Ram)
Signed-off-by: Chia-ping Tsai, Anoop Sam John, Stack
2017-11-24 12:38:42 +05:30
Guanghao Zhang ca6e7e68f4 HBASE-16868 Add a replicate_all flag to avoid misuse the namespaces and table-cfs config of replication peer
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-11-23 15:08:03 +08:00
wuguoquan 8c2a962d1c HBASE-19321 ZKAsyncRegistry ctor would hang when zookeeper cluster is not available 2017-11-22 15:51:02 +08:00
Peter Somogyi bcd367e293
HBASE-19315 Incorrect snapshot version is used for 2.0.0-beta-1
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-21 10:41:50 -08:00
anoopsamjohn f7331f9ceb HBASE-19301 Provide way for CPs to create short circuited connection with custom configurations. 2017-11-21 19:06:39 +05:30
zhangduo 4df2dffbab HBASE-19313 Call blockUntilConnected when constructing ZKAsyncRegistry(temporary workaround) 2017-11-21 21:29:18 +08:00
Sergey Soldatov 13209bce06 HBASE-19304 KEEP_DELETED_CELLS should ignore case
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-11-21 20:45:50 +08:00
Guanghao Zhang c978f8ab23 HBASE-19293 Support add a disabled state replication peer directly 2017-11-21 15:37:33 +08:00
Michael Stack 50c3bf9feb
HBASE-19260 Add lock back to avoid parallel accessing meta to locate region 2017-11-17 14:04:41 -08:00
Apekshit Sharma e0c4f374b5 HBASE-19114 Split out o.a.h.h.zookeeper from hbase-server and hbase-client
- Moved DrainingServerTracker and RegionServerTracker to hbase-server:o.a.h.h.master.
- Moved SplitOrMergeTracker to oahh.master (because it depends on a PB)
- Moving hbase-client:oahh.zookeeper.*  to hbase-zookeeper module.  After HBASE-19200, hbase-client doesn't need them anymore (except 3 classes).
- Renamed some classes to use a consistent naming for classes - ZK instead of mix of ZK, Zk , ZooKeeper. Couldn't rename following public classes: MiniZooKeeperCluster, ZooKeeperConnectionException. Left RecoverableZooKeeper for lack of better name. (suggestions?)
- Sadly, can't move tests out because they depend on HBaseTestingUtility (which defeats part of the purpose - trimming down hbase-server tests. We need to promote more use of mocks in our tests)
2017-11-17 13:23:28 -08:00
huzheng 57291108ed HBASE-19252 Move the transform logic of FilterList into transformCell() method to avoid extra ref to question cell 2017-11-17 10:42:58 +08:00
Guanghao Zhang 8b30adb834 HBASE-18911 Unify Admin and AsyncAdmin's methods name 2017-11-17 09:50:41 +08:00
zhangduo bc3542c0fb HBASE-19251 Merge RawAsyncTable and AsyncTable 2017-11-16 14:37:51 +08:00
Guanghao Zhang d885e2232d HBASE-19009 implement modifyTable and enable/disableTableReplication for AsyncAdmin 2017-11-16 07:19:34 +08:00
Sean Busbey 9c85d0017f HBASE-19248 Move tests that need to look at Connection internals to test of said internals.
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-11-15 08:40:03 -06:00
anoopsamjohn a1d86d90ba HBASE-19235 CoprocessorEnvironment should be exposed to CPs. 2017-11-15 14:51:04 +05:30
Lars Hofhansl 3d4fd90947 HBASE-12091 Optionally ignore edits for dropped tables for replication. 2017-11-14 17:16:56 -08:00
Abhishek Singh Chouhan d0fb4bd783 HBASE-19215 Incorrect exception handling on the client causes incorrect call timeouts and byte buffer allocations on the server
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-11-13 18:02:21 -08:00
Tamas Penzes 7a69ebc73e HBASE-18601: Update Htrace to 4.2
Updated HTrace version to 4.2
Created TraceUtil class to wrap htrace methods. Uses try with resources.

Signed-off-by: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-13 10:38:36 -08:00
zhangduo 8a5273f38c HBASE-19241 Improve javadoc for AsyncAdmin and cleanup warnings for the implementation classes 2017-11-13 17:00:14 +08:00
zhangduo 30f55f2316 HBASE-19200 Make hbase-client only depend on ZKAsyncRegistry and ZNodePaths
- Removes zookeeper connection from ClusterConnection
- Deletes class ZooKeeperKeepAliveConnection
- Removes Registry, ZooKeeperRegistry, and RegistryFactory
2017-11-10 10:09:04 -08:00
Chia-Ping Tsai 3511f678fb HBASE-18624 (addendum) correct the annotations 2017-11-10 09:04:51 +08:00
Michael Stack afdafb4bad HBASE-19220 Async tests time out talking to zk; 'clusterid came back null' 2017-11-09 10:45:54 -08:00
Michael Stack 0aafbe7aa9 HBASE-19165 TODO Handle stuck in transition: rit=OPENING, location=ve0538.... 2017-11-09 10:32:42 -08:00
Zach York 77e7c5ff27 HBASE-18624 Added support for clearing BlockCache based on tablename
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-11-09 04:03:35 +08:00
Mike Drob 0d4f33ca27 HBASE-19195 error-prone fixes for client, mr, and server 2017-11-08 12:04:46 -06:00
Apekshit Sharma a6d8023ff5 HBASE-19128 Purge Distributed Log Replay from codebase, configurations, text; mark the feature as unsupported, broken. 2017-11-07 17:48:52 -08:00
QilinCao 1110910b3a HBASE-19103 Add BigDecimalComparator for filter
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-11-07 03:30:43 -08:00
Michael Stack f13cf56f1c
HBASE-19197 Move version on branch-2 from 2.0.0-alpha4 to 2.0.0-beta-1.SNAPSHOT 2017-11-06 20:46:38 -08:00
Chia-Ping Tsai 43b4aab648 HBASE-19131 (Addendum) Use the emptyList() to replace EMPTY_LIST 2017-11-07 04:06:29 +08:00
Mike Drob cfddfcf23c HBASE-19160 expose CellComparator as IA.Public 2017-11-06 10:15:55 -06:00
Guanghao Zhang 47c614c706 HBASE-18950 Remove Optional parameters in AsyncAdmin interface 2017-11-06 20:41:20 +08:00
Mike Drob 33ae6dce42 HBASE-18983 fixes from update error-prone to 2.1.1 2017-11-04 21:29:48 -05:00
Chia-Ping Tsai d592b29619 HBASE-19131 Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook 2017-11-05 09:56:04 +08:00
zhangduo 547d46cfac HBASE-18972 Use Builder pattern to remove nullable parameters for coprocessor methods in RawAsyncTable interface 2017-11-02 13:58:33 +08:00
Apekshit Sharma d69570a485 HBASE-18925 Update mockito dependency from mockito-all:1.10.19 to mockito-core:2.1.0 for JDK8 support.
Last mockito-all release was in Dec'14. Mockito-core has had many releases since then.

From mockito's site:
- "Mockito does not produce the mockito-all artifact anymore ; this one was primarily
aimed at ant users, and contained other dependencies. We felt it was time to move on
and remove such artifacts as they cause problems in dependency management system like
maven or gradle."
- anyX() and any(SomeType.class) matchers now reject nulls and check type.
2017-11-01 14:38:50 -07:00
eshcar 526d2826f5 HBASE-16417: In-memory MemStore Policy for Flattening and Compactions 2017-11-01 16:48:40 +02:00
Michael Stack 16012f93a6 HBASE-18770 Remove bypass method in ObserverContext and implement the
'bypass' logic case by case

Changes Coprocessor ObserverContext 'bypass' semantic. We flip the
default so bypass is NOT supported on Observer invocations; only a
couple of preXXX methods in RegionObserver allow it: e.g.  preGet
and prePut but not preFlush, etc. Everywhere else, we throw
a DoesNotSupportBypassException if a Coprocessor Observer
tries to invoke bypass. Master Observers can no longer stop
or change move, split, assign, create table, etc.

Ditto on complete, the mechanism that allowed a Coprocessor
rule that all subsequent Coprocessors are skipped in an
invocation chain; now, complete is only available to
bypassable methods (and Coprocessors will get an exception if
they try to 'complete' when it is not allowed).

See javadoc for whether a Coprocessor Observer method supports
'bypass'. If no mention, 'bypass' is NOT supported.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 Added passing of 'bypassable' (and 'completable') and default 'result' argument to
 the Operation constructors rather than pass the excecution engine as parameters.
 Makes it so can clean up RegionObserverHost and make the calling
 clearer regards what is going on.
 Methods that support 'bypass' must set this flag on the Observer.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Refactoring in here is minor. A few methods that used support bypass
 no longer do so removed the check and the need of an if/else meant a
 left-shift in some code.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 Ditto

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
 In here label explicitly those methods that are bypassable.
 Some changes to make sure we call the corresponding execOperation.

TestMasterObserver had a bunch of test of bypass method. All removed or
disabled.

TODO: What to do w/ the Scanner methods.
2017-10-31 12:49:51 -07:00
Peter Somogyi 0ad8a6f51b HBASE-19031 Align exist method in Table and AsyncTable interfaces
Deprecate Table::existsAll method and add Table::exists.
RemoteHTable already had a deprecated exists method, remove that
and implement the new exists from Table interface.

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-10-29 07:17:41 +08:00
Michael Stack 250a8bc7ad
HBASE-18995 Move methods that are for internal usage from CellUtil to Private util class (Ramkrishna Vasudevan) 2017-10-27 17:51:32 -07:00
Tamas Penzes fc581270a8 HBASE-13346: Clean up Filter package for post 1.0 s/KeyValue/Cell/g
Added filterCell method to Filter, it calls filterKeyValue by default
Deprecated filterKeyValue in Filter, bud added default functionality to return Filter.ReturnCode.INCLUDE.
Added filterKeyValue (calling filterCell) to Filters extending FilterBase to be backward compatible.
renamed filterKeyValue to filterCell in all implementations
changed all internal calls to use filterCell instead of filterKeyValue
changed tests too

This way the change is simple and backward compatible.
Any existing custom filter should work since they override filterKeyValue
and the implementation is called by Filter.filterCell.

Moved FilterWrapper to hbase-server

Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
2017-10-27 11:17:53 +05:30
QilinCao 9a5ab58bc1 HBASE-19091 Code annotation wrote BinaryComparator instead of LongComparator
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2017-10-26 22:19:08 +02:00
huzheng a93e1aa8f6 HBASE-15410 (addendum) Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-10-26 18:46:49 +08:00
Peter Somogyi 5dadfdaa58
HBASE-19029 Align RPC timout methods in Table and AsyncTableBase
As part of HBASE-18978 the rpc timeout methods gets aligned
between Table and AsyncTable interfaces.
Deprecate the following methods in Table:
- int getRpcTimeout()
- int getReadRpcTimeout()
- int getWriteRpcTimeout()
- int getOperationTimeout()

Add the following methods to Table:
- long getRpcTimeout(TimeUnit)
- long getReadRpcTimeout(TimeUnit)
- long getWriteRpcTimeout(TimeUnit)
- long getOperationTimeout(TimeUnit)

Fix some javadoc issues.

Signed-off-by: Michael Stack <stack@apache.org>
2017-10-25 09:55:16 -07:00
huzheng c2dbef1465 HBASE-19057 Fix other code review comments about FilterList improvement 2017-10-25 20:41:25 +08:00
huzheng 7a2da02e6d HBASE-18368 FilterList with multiple FamilyFilters concatenated by OR does not work
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-10-25 20:41:25 +08:00
huzheng f6dd5e8b64 HBASE-18879 HBase FilterList cause KeyOnlyFilter not work 2017-10-25 20:41:24 +08:00
huzheng aa23cca58a HBASE-18411 Dividing FiterList into two separate sub-classes: FilterListWithOR , FilterListWithAND
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-10-25 20:41:24 +08:00
Biju Nair 1a5b3a3363 HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-25 20:41:24 +08:00
huzheng 15eae6ac35 HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-10-25 20:41:24 +08:00
tedyu 47d8549cec HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-10-25 20:41:24 +08:00
huzheng 50c9a412fe HBASE-17678 FilterList with MUST_PASS_ONE lead to redundancy cells returned - addendum
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-25 20:41:24 +08:00
huzheng 7237ecc3b3 HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-25 20:41:24 +08:00
Josh Elser 3e0b90b949 HBASE-18873 Move protobufs to private implementation on GlobalQuotaSettings
A hack to "hide" the protobufs, but it's not going to be a trivial
change to remove use of protobufs entirely as they're serialized
into the hbase:quota table.
2017-10-23 22:44:44 -04:00
Mike Drob 34df2e665e HBASE-16338 Remove Jackson1 deps
* Change imports from org.codehaus to com.fasterxml
* Exclude transitive jackson1 from hadoop and others
* Minor test cleanup to add assert messages, fix some parameter order
* Add anti-pattern check for using jackson 1 imports
* Add explicit non-null serialization directive to ScannerModel
2017-10-23 15:24:51 -05:00
Guanghao Zhang 00f2b18148 HBASE-19007 Align Services Interfaces in Master and RegionServer
Purges Server, MasterServices, and RegionServerServices from
CoprocessorEnvironments. Replaces removed functionality with
a set of carefully curated methods on the *CoprocessorEnvironment
implementations (Varies by CoprocessorEnvironment in that the
MasterCoprocessorEnvironment has Master-type facility exposed,
and so on).

A few core Coprocessors that should long ago have been converted
to be integral, violate their context; e.g. a RegionCoprocessor
wants free access to a hosting RegionServer (which may or may not
be present). Rather than let these violators make us corrupte the
CP API, instead, we've made up a hacky system that allows core
Coprocessors access to internals. A new CoreCoprocessor Annotation
has been introduced. When loading Coprocessors, if the instance is
annotated CoreCoprocessor, we pass it an Environment that has been
padded w/ extra-stuff. On invocation, CoreCoprocessors know how to
route their way to these extras in their environment.

See the *CoprocessoHost for how the do the check for CoreCoprocessor
and pass a fatter *Coprocessor, one that allows getting of either
a RegionServerService or MasterService out of the environment
via Marker Interfaces.

Removed org.apache.hadoop.hbase.regionserver.CoprocessorRegionServerServices

M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
 This Endpoint has been deprecated because its functionality has been
 moved to core. Marking it a CoreCoprocessor in the meantime to
 minimize change.

M hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java
 This should be integral to hbase. Meantime, marking it CoreCoprocessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/Server.java
 Added doc on where it is used and added back a few methods we'd
removed.

A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoreCoprocessor.java
 New annotation for core hbase coprocessors. They get richer environment
 on coprocessor loading.

A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/HasMasterServices.java
A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/HasRegionServerServices.java
 Marker Interface to access extras if present.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterCoprocessorEnvironment.java
  Purge MasterServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java
  Purge RegionServerServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerCoprocessorEnvironment.java
  Purge MasterServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterSpaceQuotaObserver.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java
  We no longer have access to MasterServices. Don't need it actually.
  Use short-circuiting Admin instead.

D hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CoprocessorRegionServerServices.java
  Removed. Not needed now we do CP Env differently.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  No need to go via RSS to getOnlineTables; just use HRS.

And so on. Adds tests to ensure we can only get at extra info
if the CP has been properly marked.
2017-10-21 11:07:27 -07:00
Guanghao Zhang 51ceeece25 HBASE-19010 Reimplement getMasterInfoPort for Admin 2017-10-21 18:33:12 +08:00
Michael Stack 9b4caf4b1b HBASE-19043 Purge TableWrapper and CoprocessorHConnnection
Also purge Coprocessor#getTable... Let Coprocessors manage their
Table Connections in hbase2.0.0.
2017-10-20 11:15:42 -07:00
Jerry He 75d2bba739 HBASE-10367 RegionServer graceful stop / decommissioning
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-10-19 22:10:52 -07:00
Ramkrishna 2cb64fb467 HBSE-18945 Make a IA.LimitedPrivate interface for CellComparator (Ram) 2017-10-17 23:19:21 +05:30
Reid Chan b96e2f055d HBASE-18990 ServerLoad doesn't override #equals which leads to #equals in ClusterStatus always false
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-10-18 00:45:34 +08:00
Guanghao Zhang 58b0585d66 HBASE-18914 Remove AsyncAdmin's methods which were already deprecated in Admin interface 2017-10-16 22:23:46 +08:00
Chia-Ping Tsai d2e4e0e1f3 HBASE-18997 Remove the redundant methods in RegionInfo 2017-10-14 23:32:48 +08:00
tedyu 77bfe05503 HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-10-13 15:15:12 -07:00
anoopsamjohn b212bf936e HBASE-18183 Region interface cleanup for CP expose. 2017-10-11 17:14:43 +05:30
Mike Drob 2e368cf461 HBASE-18981 Address issues found by error-prone in hbase-client 2017-10-10 14:09:08 -05:00
zhangduo d5b76547f0 HBASE-18951 Use Builder pattern to remove nullable parameters for checkAndXXX methods in RawAsyncTable/AsyncTable interface 2017-10-10 14:41:27 +08:00
Sean Busbey 38e52bb29e Revert "HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned"
This reverts commit 0d0c330401.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:54:26 -05:00
Sean Busbey 852b578384 Revert "HBASE-17678 FilterList with MUST_PASS_ONE lead to redundancy cells returned - addendum"
This reverts commit 347bef8d33.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:54:22 -05:00
Sean Busbey 1d07c8eec4 Revert "HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT"
This reverts commit 743f3ae221.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:54:20 -05:00
Sean Busbey 2dcdd13a01 Revert "HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue"
This reverts commit 7c2622baf7.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:54:15 -05:00
Sean Busbey 3dd66e6cda Revert "HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()"
This reverts commit d142f07129.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:53:50 -05:00
Maytee Chinavanichkit 9cd7619b5c HBASE-18921 Fix Result.current ArrayIndexOutOfBoundsException
Patch ArrayIndexOutOfBoundsException when current() is called after
advance() has already returned false

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-10-08 02:51:02 +08:00
Guanghao Zhang 8e5d88366d HBASE-18909 Deprecate Admin's methods which used String regex 2017-10-07 21:23:19 +08:00
Chia-Ping Tsai 2a5b6bc681 HBASE-18927 Add the DataType which is subset of KeyValue#Type to CellBuilder for building cell 2017-10-04 23:45:57 +08:00
Sean Busbey 35094bf4d5 HBASE-18933 set version number to 2.0.0-alpha4-SNAPSHOT following release of alpha3 2017-10-04 07:57:49 -05:00
Ramkrishna 3318e8724e HBASE-18490 Modifying a table descriptor to enable replicas does not
create replica regions (Ram)
2017-10-03 13:16:00 +05:30
Chia-Ping Tsai e047f518ef HBASE-18897 Substitute MemStore for Memstore 2017-10-02 21:10:11 +08:00
Apekshit Sharma 4579bba486 HBASE-18884 (addendum) Add more javadoc comment. Coprocessor Design Improvements follow up of HBASE-17732.
Change-Id: Id1a9e508cc04612b1e79e6cfa7e39b7755598be3
2017-09-29 17:32:41 -07:00
Robert Yokota 759110c225 HBASE-18559 Add histogram to MetricsConnection to track concurrent calls per server
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-29 17:12:26 -07:00
Biju Nair d142f07129 HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-29 16:19:39 -07:00
Andrew Purtell 5917ad4c0c HBASE-18436 Add client-side hedged read metrics (Yun Zhao) 2017-09-29 15:36:42 -07:00
Andy Yang 0658252ed6 HBASE-13844 Move static helper methods from KeyValue into CellUtils
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-29 13:20:24 +08:00
Michael Stack ac933f7af2 HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732; MINOR AMENDMENT adding README to design-doc dir 2017-09-28 13:25:07 -07:00
Apekshit Sharma a6a303816c HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732
- Change Service Coprocessor#getService() to List<Service> Coprocessor#getServices()
- Checkin the finalized design doc into repo
- Added example to javadoc of Coprocessor base interface on how to implement one in the new design
2017-09-28 10:33:30 -07:00
Chia-Ping Tsai 6693f45faf HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:41 +08:00
zhangduo d26b8f8ddd HBASE-18826 Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface 2017-09-28 15:41:56 +08:00
Apekshit Sharma 0c883a23c5 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:45:51 -07:00
anoopsamjohn 0fcc84cadd HBASE-18298 RegionServerServices Interface cleanup for CP expose. 2017-09-27 11:02:57 +05:30
huzheng 7c2622baf7 HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-09-26 11:23:38 +08:00
Josh Elser 40a73c9be5 HBASE-18807 Remove protobuf references from CP quota API calls 2017-09-23 21:28:58 -04:00
Abhishek Singh Chouhan 8df523bdd0 HBASE-18796 Addendum: Scanner returning null incorrectly during locateRegionInMeta retry, resulting in incorrect TableNotFoundException
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-22 10:21:06 -07:00
Sean Busbey c1f5122fab HBASE-18731 [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-21 14:01:02 -05:00
Abhishek Singh Chouhan 518674bce6 HBASE-18796 Admin#isTableAvailable returns incorrect result before daughter regions are opened
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-19 15:06:15 -07:00
Yu Li 52c809c35e HBASE-15931 Add log for long-running tasks in AsyncProcess (addendum) 2017-09-19 09:55:11 +08:00
Chia-Ping Tsai 3f112ee320 HBASE-17980 (Addendum) make UnmodifyableHRegionInfo deprecated 2017-09-16 17:08:34 +08:00
brandboat 42bdd665ae HBASE-17980 Any HRegionInfo we give out should be immutable
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-14 14:25:58 -07:00
Michael Stack 7660f9e86a HBASE-18819 Set version number to 2.0.0-alpha3 from 2.0.0-alpha3-SNAPSHOT 2017-09-14 12:38:46 -07:00
Michael Stack 274bbb64c6 HBASE-14998 Unify synchronous and asynchronous methods in Admin and cleanup
This is based on patch sent me by Balazs Meszaros. The good stuff in
here is from him. This patch does less than his ambition. It changes
Admin class only. Can work on making AsyncAdmin cohere in a follow-on.

 * Deprecates getAlterStatus. Everywhere else we talk of 'modify' rather
   'alter' and should use Future returned from async instead.
 * isTableAvailable(TableName, byte [][]) has been deprecated to be
   removed; use the overrie instead. This is a weird method.
 * Changed listTableDescriptor to getDescriptor.
 * Renamed other like methods to have same pattern (deprecating the old):
    balancer => balance
    setBalancerRunning => balancerSwitch
    setNormalizerRunning => normalizerSwitch
    enableCatalogJanitor => catalogJanitorSwitch
    setCleanerChoreRunning => cleanerChoreSwitch
    setSplitOrMergeEnabled => splitOrMergeEnabledSwitch

 * Renamed (with deprecation of old) runCatalogScan => runCatalogJanitor.
 * Reviewed generated javadoc and made some edits; purged reference to
   hbase issues from our API, fixed param names, etc.
 * Made all the enable services methods have same pattern.
 * Renamed takeSnapshotAsync as snapshotAsync (with deprecation of old)
 * Renamed execProcedureWithRet as execProcedureWithReturn (with
   deprecation)

Signed-off-by: Michael Stack <stack@apache.org>
2017-09-13 17:19:56 -07:00
Reid Chan dc1db8c5b3 HBASE-18609 Apply ClusterStatus#getClusterStatus(EnumSet<Option>) in code base
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-14 01:00:19 +08:00
Guangxu Cheng 5370aed410 HBASE-18789 Displays the reporting interval of each RS on the Master page
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-13 05:53:16 -07:00
Sean Busbey d576e5a32d HBASE-17823 Migrate to Apache Yetus Audience Annotations
Includes partial backport of hbase-build-configuration module

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2017-09-12 23:15:50 -05:00
Guangxu Cheng c75bb7424e HBASE-18131 Add an hbase shell command to clear deadserver list in ServerManager
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-12 09:25:57 -07:00
Umesh Agashe a11a17825a HBASE-13271 Added test for batch operations with validation errors. Updated Javadoc for batch methods.
Javadoc for following methods are updated:
* Table.put(List<Put> puts)
* Table.delete(List<Delete> deletes)

Added @apiNote for delete regarding input list will not be modied in version 3.0.0

Signed-off-by: Michael Stack <stack@apache.org>
2017-09-11 20:06:10 -07:00
Yun Zhao 7af0c44af3 HBASE-18662 The default values for many configuration items in the code are not consistent with hbase-default.xml
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-11 21:12:11 +08:00
Chia-Ping Tsai 1ee4923730 Revert "HBASE-18662 The default values for many configuration items in the code are not consistent with hbase-default.xml"
need more checks
This reverts commit 89f978beb8.
2017-09-09 22:55:33 +08:00
Yun Zhao 89f978beb8 HBASE-18662 The default values for many configuration items in the code are not consistent with hbase-default.xml
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-09 18:11:14 +08:00
Chia-Ping Tsai 26b009f1d1 HBASE-18783 Declare the builder of ClusterStatus as IA.Private, and remove the Writables from ClusterStatus 2017-09-09 15:55:21 +08:00
Reid Chan 6970c1e9eb HBASE-18621 Refactor ClusterOptions before applying to code base
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-09 03:38:42 +08:00
Balazs Meszaros c48dc02b76 HBASE-18106 Redo ProcedureInfo and LockInfo
Main changes:
- ProcedureInfo and LockInfo were removed, we use JSON instead of them
- Procedure and LockedResource are their server side equivalent
- Procedure protobuf state_data became obsolate, it is only kept for
reading previously written WAL
- Procedure protobuf contains a state_message field, which stores the internal
state messages (Any type instead of bytes)
- Procedure.serializeStateData and deserializeStateData were changed slightly
- Procedures internal states are available on client side
- Procedures are displayed on web UI and in shell in the following jruby format:
  { ID => '1', PARENT_ID = '-1', PARAMETERS => [ ..extra state information.. ] }

Signed-off-by: Michael Stack <stack@apache.org>
2017-09-08 11:56:28 -07:00
Michael Stack f08dd5d342 HBASE-18779 Move CompareOperator to hbase-client module 2017-09-08 10:18:32 -07:00
Michael Stack 7f70b6c8e3 HBASE-18769 Make CompareFilter use generic CompareOperator instead of
internal enum
2017-09-08 10:05:51 -07:00
Apekshit Sharma 56cba5e450 HBASE-14997 Move compareOp and Comparators out of filter to client package
Add a new generic CompareOperator enum and use it instead of the filter
CompareOp enum everywhere BUT inside CompareFilter.
2017-09-07 11:03:22 -07:00
tedyu 743f3ae221 HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-09-07 08:15:29 -07:00
Umesh Agashe bd219c0fb8 HBASE-18674 upgrading to commons-lang3
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-05 09:46:49 -07:00
Chia-Ping Tsai 0e01544838 HBASE-18736 Cleanup the HTD/HCD for Admin 2017-09-04 17:02:28 +08:00
cuijianwei a1efda29e9 HBASE-15497 Incorrect javadoc for atomicity guarantee of Increment and Append
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-04 16:23:02 +08:00
zhangduo a37417c254 HBASE-18699 Copy LoadIncrementalHFiles to another package and mark the old one as deprecated 2017-09-03 19:49:47 +08:00
Chia-Ping Tsai a32642e6f6 HBASE-18721 Cleanup unused configs and private declaration 2017-08-31 23:40:18 +08:00
Chia-Ping Tsai 2f7dcc919d HBASE-18306 Get rid of TableDescriptor#getConfiguration 2017-08-30 11:26:31 +08:00
Sean Busbey 72bb96076e HBASE-18577 test well-formed-ness of our client jars.
* Removes transitive hadoop deps from hbase-client that we don't use
* removes curator-recipes dep from hbase-client that we don't use
* removes jetty-util transitive from hbase-client that we don't use
* moves junit from compile to test scope for hbase-client
* adds an IT that checks the contents of jars
* excludes our server side web apps and related deps (jaxb, jersey, jetty) from shaded jars
* excludes proto source files from shaded jars
* relocates codahale metrics, commons-crypto, and curator
* finishes relocating jets3t
* moves logging backends to optional in shaded artifacts
* updates maven dependency plugin

Signed-off-by: Michael Stack <stack@apache.org>
2017-08-29 12:09:25 -05:00
Peter Somogyi 09b036a519 HBASE-18665 ReversedScannerCallable invokes getRegionLocations incorrectly
The way how ReversedScannerCallable#prepare called getRegionLocations was faulty.
Calling prepare with force reload used cache and vica versa.

Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-29 07:54:37 -07:00
Andy Yang d26e22d75b HBASE-3935 HServerLoad.storefileIndexSizeMB should be changed to storefileIndexSizeKB
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-29 13:43:27 +08:00
Chia-Ping Tsai 0142c09217 HBASE-18519 Use builder pattern to create cell 2017-08-28 14:19:45 +08:00
Chia-Ping Tsai 13769ab7c1 HBASE-18691 [compat 1-2] HCD remove and removeConfiguration change return type 2017-08-27 19:02:07 +08:00
Michael Stack 34ca89e83e HBASE-18687 Add @since 2.0.0 to new classes; AMENDMENT 2017-08-25 14:15:51 -07:00
Michael Stack 3e1c598d8e HBASE-18687 Add @since 2.0.0 to new classes 2017-08-25 13:12:52 -07:00
Huaxiang Sun f74cf679ec Revert "HBASE-17249 Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange reference as the parameter instead of creating a new setColumnFamilyTimeRange instance. (huaxiang sun)"
This reverts commit 61220e4d7c.
2017-08-25 11:19:46 -07:00
zhangduo 1ae9a39011 HBASE-18347 Implement a BufferedMutator for async client 2017-08-24 16:01:30 +08:00
Chia-Ping Tsai 45b20da23b HBASE-18546 Always overwrite the TS for Append/Increment unless no existing cells are found 2017-08-24 14:40:44 +08:00
Apekshit Sharma e2ce252b59 HBASE-17442 Move most of the replication related classes from hbase-client to new hbase-replication package. (Guanghao Zhang).
Change-Id: Ie0e24cc617ab4bf56de8b1747062d1b78a5d4669
2017-08-23 22:18:26 -07:00
Chia-Ping Tsai 205016ca79 HBASE-18503 Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor 2017-08-24 13:01:09 +08:00
Michael Stack b24e33312a HBASE-18594 Release hbase-2.0.0-alpha2; ADDENDUM update version from 2.0.0-alpha2 to 2.0.0-alpha3-SNAPSHOT 2017-08-23 11:07:41 -07:00
Michael Stack 9c36c0c52f HBASE-18660 Remove duplicate code from the checkAndPut method in HTable (Yun Zhao) 2017-08-22 21:19:53 -07:00
Michael Stack 1653b54de9 HBASE-18645 Loads of tests timing out.... Revert "HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable"
This reverts commit 1ab6882f62.
2017-08-22 01:42:27 -07:00
tedyu 22ecb7a7b5 HBASE-18617 FuzzyRowKeyFilter should not modify the filter pairs (vinisha) 2017-08-18 09:12:16 -07:00
Pankaj Kumar 12a7d2bace HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-17 19:03:11 -07:00
Guanghao Zhang c606a565c1 HBASE-18608 AsyncConnection should return AsyncAdmin interface instead of the implemenation 2017-08-17 09:56:06 +08:00
Xiang Li 4057552ed6 HBASE-18573 Update Append and Delete to use Mutation#getCellList(family)
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-08-16 14:45:33 -07:00
zhangduo 770312a8c2 HBASE-18553 Expose scan cursor for asynchronous scanner 2017-08-16 21:05:02 +08:00
Guanghao Zhang b2afd6c24e HBASE-18598 AsyncNonMetaRegionLocator use FIFO algorithm to get a candidate locate request 2017-08-16 13:41:32 +08:00
Ashish Singhi 1f7873d305 HBASE-18437 Revoke access permissions of a user from a table does not work as expected
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-08-15 22:29:15 -07:00
Lars Francke 9de5fd5bb5 HBASE-18599 Add missing @Deprecated annotations
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-15 10:45:40 -07:00
Deon Huang 5073bd6e04 HBASE-2631 Decide between InMB and MB as suffix for field names in ClusterStatus objects
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-15 21:43:43 +08:00
brandboat 8775f3027f HBASE-18582 Correct the docs for Mutation#setCellVisibility
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-15 21:38:08 +08:00
Mike Drob 0ded122b1e HBASE-18303 Clean up @Parameter boilerplate 2017-08-14 14:24:37 -05:00
Jerry He cf050de917 HBASE-18522 Add RowMutations support to Batch 2017-08-14 10:43:10 -07:00
Michael Stack add9974515 HBASE-18595 Set version in branch-2 from 2.0.0-alpha2-SNAPSHOT to 2.0.0-alpha2 2017-08-14 10:28:44 -07:00
Chia-Ping Tsai fffe4e278d HBASE-18528 DON'T allow user to modify the passed table/column descriptor 2017-08-14 14:03:19 +08:00
Michael Stack b4d44467fb HBASE-18271 Shade netty Purge mention of netty-all.
Add in hbase-thirdparty hbase-shaded-netty instead.
s/io.netty/org.apache.hadoop.hbase.shaded.io.netty/ everywhere in hbase.

Also set a system property when running tests and when starting
hbase; required by netty so can find the relocation files in the
bundled .so.
2017-08-13 12:50:20 -07:00
Reid Chan 802f602e11 HBASE-15511 ClusterStatus should be able to return responses by scope
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-14 01:08:31 +08:00
Xiang Li 8cebf7f1a8 HBASE-18555: Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and Query
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-08-11 22:53:37 -07:00
Guanghao Zhang 0c16bb591b HBASE-18500 Performance issue: Don't use BufferedMutator for HTable's put method 2017-08-11 16:38:08 +08:00
Guanghao Zhang 8197a31bbc HBASE-17125 Inconsistent result when use filter to read data 2017-08-11 11:04:42 +08:00
zhangduo 64345a5b5a HBASE-18489 Expose scan cursor in RawScanResultConsumer 2017-08-10 10:11:49 +08:00
Josh Elser f262548b1f HBASE-18514 Squash of space quota "phase 2" work
Contains the following commits:

  HBASE-17748 Include HBase snapshots in space quotas

  Introduces a new Chore in the Master which computes the size
  of the snapshots included in a cluster. The size of these
  snapshots are included in the table's which the snapshot was created
  from HDFS usage.

  Includes some test stabilization, trying to make the tests more
  deterministic by ensuring we observe stable values as we know
  that those values are mutable. This should help avoid problems
  where size reports are delayed and we see an incomplete value.

  HBASE-17752 Shell command to list snapshot sizes WRT quotas

  HBASE-17840 Update hbase book to space quotas on snapshots
2017-08-07 16:32:16 -04:00
no_apologies 2c067695dd HBASE-18515 Introduce Delete.add as a replacement for Delete#addDeleteMarker
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-07 11:13:12 +08:00
Guanghao Zhang be034c26b4 HBASE-18485 Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner 2017-08-07 10:43:03 +08:00
Benedict Jin ee6ff31c0d HBASE-18470 Remove the redundant comma from RetriesExhaustedWithDetailsException#getDesc
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-03 15:43:20 +08:00
Robert Yokota da2cb02422 HBASE-16893 Use Collection.removeIf instead of Iterator.remove in DependentColumnFilter
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-03 13:43:51 +08:00
Apekshit Sharma 41da64a144 HBASE-18231 Deprecate Admin#closeRegion*() commands in favor of Admin#unassign().
Other changes:
- Update corresponding tests in TestAdmin2. Removed tests centered around serverName part of old functions.
- Remove dead functions from ProtobufUtil and ServerManager
- Rename closeRegion* functions in HBTU to unassignRegion*

Change-Id: Ib9bdeb185e10750daf652be0bb328306accb73ab
2017-08-02 15:22:17 -07:00
Michael Stack ee70b1d2e0 HBASE-17056 Remove checked in PB generated files
Selective add of dependency on hbase-thirdparty jars.
Update to READMEs on how protobuf is done (and update to refguide).
Removed all checked in generated protobuf files. They are generated
on the fly now as part of mainline build.
2017-08-02 09:42:38 -07:00
Andrew Purtell 5e21e30682 HBASE-18431 Mitigate compatibility concerns between branch-1.3 and branch-1.4
We needed to move TableName out of HBase.proto to resolve a cycle after
repairing incompatible changes introduced into branch-1. Carry this forward.
2017-08-01 17:05:14 -07:00
Yi Liang 928137c1ce HBASE-18465: [AMv2] remove old split region code that is no longer needed
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-30 15:25:40 -05:00
anoopsamjohn d776a3caae HBASE-18473 VC.listLabels() erroneously closes any connection. 2017-07-30 15:38:18 +05:30
Yu Li 3be51a85ba HBASE-18441 ZookeeperWatcher#interruptedException should throw exception 2017-07-26 15:21:25 +08:00
Malcolm Taylor 7517f93265 HBASE-18434 Address some alerts raised by lgtm.com
Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2017-07-26 10:03:07 +05:30
Andrew Purtell e24f7f4675 HBASE-18054 log when we add/remove failed servers in client (Ali) 2017-07-25 18:53:08 -07:00
张世彬10204932 8ad76bbc88 HBASE-18323 Remove multiple ACLs for the same user in kerberos
Signed-off-by: Josh Elser <elserj@apache.org>
2017-07-25 18:17:02 -04:00
Phil Yang 86ce7fcbbf HBASE-15968 New behavior of versions considering mvcc and ts rather than ts only 2017-07-25 15:17:47 +08:00
Yi Liang 9e230dc321 HBASE-18107: [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-23 10:45:38 +01:00
Mike Drob 254b78c116 HBASE-18433 Convenience method for creating simple ColumnFamilyDescriptor
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-07-22 23:44:02 +08:00
rgidwani d461bec6c2 HBASE-15816 Provide client with ability to set priority on Operations
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-07-21 17:12:21 -07:00
Michael Stack d5c6e11016 HBASE-17908 Upgrade guava
Pull in guava 22.0 by using the shaded version up in new hbase-thirdparty project.

In poms, exclude guava everywhere except on hadoop-common. Do this so
we minimize transitive includes. hadoop-common is needed because hadoop
Configuration uses guava doing preconditions.

Everywhere we used guava, instead use shaded so fix a load of imports.

Stopwatch API changed as did hashing and toStringHelper which is now
in MoreObjects class. Otherwise, minimal changes to come up on 22.0
2017-07-21 15:41:52 +01:00
Chia-Ping Tsai f8fa04c717 HBASE-18420 Some methods of Admin don't use ColumnFamilyDescriptor 2017-07-21 19:15:09 +08:00
Andrew Purtell cbf390422b HBASE-18330 NPE in ReplicationZKLockCleanerChore 2017-07-19 15:46:22 -07:00
Phil Yang fc362f69cb HBASE-18390 Sleep too long when finding region location failed 2017-07-19 11:49:58 +08:00
Yi Liang a4575704bc HBASE-18229: create new Async Split API to embrace AM v2
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-14 22:26:17 +01:00
Sean Busbey 9cc9404261 HBASE-18187 update version to 2.0.0-alpha-2-SNAPSHOT 2017-07-14 11:20:32 -05:00
Guanghao Zhang 06a0bfc3ba HBASE-18342 Add coprocessor service support for async admin 2017-07-14 11:55:32 +08:00
Jan Hentschel 97b649eb93 HBASE-18344 Introduce Append.addColumn as a replacement for Append.add
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-07-13 20:05:22 +08:00
Chia-Ping Tsai ca5d8c40b7 HBASE-18268 Eliminate the findbugs warnings for hbase-client 2017-07-12 03:08:05 +08:00
Guanghao Zhang 8a8e299eee HBASE-18343 Track the remaining unimplemented methods for async admin 2017-07-11 14:05:02 +08:00
zhangduo 7acb226eb9 HBASE-18348 The implementation of AsyncTableRegionLocator does not follow the javadoc 2017-07-11 11:44:40 +08:00
tedyu 5d7dfa4d1d HBASE-17705 Procedure execution must fail fast if procedure is not registered (Vladimir Rodionov) 2017-07-10 09:11:58 -07:00
Guanghao Zhang 4368f09057 HBASE-18318 Implement updateConfiguration/stopMaster/stopRegionServer/shutdown methods 2017-07-10 13:49:03 +08:00
Guanghao Zhang da80839ecc HBASE-18316 Implement async admin operations for draining region servers 2017-07-09 20:22:06 +08:00
Chia-Ping Tsai 53ec9c5bd3 HBASE-18241 Change client.Table, client.Admin, Region, Store, and HBaseTestingUtility to not use HTableDescriptor or HColumnDescriptor 2017-07-08 17:00:15 +08:00
Guanghao Zhang 5d4e4f2007 HBASE-18317 Implement async admin operations for Normalizer/CleanerChore/CatalogJanitor 2017-07-08 10:59:01 +08:00
Guanghao Zhang 359f97711f HBASE-18319 Implement getClusterStatus/getRegionLoad/getCompactionState/getLastMajorCompactionTimestamp methods 2017-07-07 16:47:19 +08:00
Michael Stack b2b5cd6de6 Revert "HBASE-17056 Remove checked in PB generated files Selective add of dependency on"
Build is unstable and has interesting issues around CLASSPATH. Revert
for now.

This reverts commit 1049025e1d.
2017-07-06 21:48:41 -07:00
Michael Stack 1049025e1d HBASE-17056 Remove checked in PB generated files Selective add of dependency on
hbase-thirdparty jars. Update to READMEs on how protobuf is done (and update to
refguide) Removed all checked in generated protobuf files. They are generatedon
the fly now as part of mainline build.
2017-07-05 21:38:23 -07:00
Guanghao Zhang 507f5fb844 HBASE-18297 Provide a AsyncAdminBuilder to create new AsyncAdmin instance 2017-07-05 10:10:05 +08:00
Michael Stack 84a5d29391 HBASE-18286 Create static empty byte array to save memory
Revert "Revert "HBASE-18266 Create static empty byte array to save memory (Ben Epstein)""
... i.e. a reapply but with correct JIRA this time.

This reverts commit 3a0db474a2.
2017-07-03 19:51:16 -07:00
Michael Stack 3a0db474a2 Revert "HBASE-18266 Create static empty byte array to save memory (Ben Epstein)"
Wrong JIRA number

This reverts commit 549fb1677b.
2017-07-03 19:50:55 -07:00
Peter Somogyi eacbdb061e HBASE-18264 Update pom plugins
Update plugins in main and subprojects
Unified versions to use variable instead of direct values

Affected plugins:
- apache-rat-plugin 0.11 -> 0.12
- asciidoctor-maven-plugin 1.5.2.1 -> 1.5.5
- asciidoctorj-pdf 1.5.0-alpha.6 -> 1.5.0-alpha.15
- build-helper-maven-plugin 1.9.1 -> 3.0.0
- buildnumber-maven-plugin 1.3 -> 1.4
- exec-maven-plugin 1.2.1/1.4.0 -> 1.6.0
- extra-enforcer-rules 1.0-beta-3 -> 1.0-beta-6
- findbugs-maven-plugin 3.0.0 -> 3.0.4
- jamon-maven-plugin 2.4.1 -> 2.4.2
- maven-bundle-plugin 2.5.3 -> 3.3.0
- maven-compiler-plugin 3.2/3.5.1 -> 3.6.1
- maven-eclipse-plugin 2.9 -> 2.10
- maven-shade-plugin 2.4.1 -> 3.0.0
- maven-surefire-plugin 2.18.1 -> 2.20
- maven-surefire-report-plugin 2.7.2 -> 2.20
- scala-maven-plugin 3.2.0 -> 3.2.2
- spotbugs 3.1.0-RC1 -> 3.1.0-RC3
- wagon-ssh 2.2 -> 2.12
- xml-maven-plugin 1.0 -> 1.0.1

- maven-assembly-plugin 2.4 -> 2.6(inherited)
- maven-dependency-plugin 2.4 -> 2.10 (inherited)
- maven-enforcer-plugin 1.3.1 -> 1.4.1 (inherited)
- maven-javadoc-plugin 2.10.3 -> 2.10.4 (inherited)
- maven-resources-plugin 2.7 (inherited)
- maven-site-plugin 3.4 -> 3.5.1 (inherited)

Change-Id: I84539f555be498dff18caed1e3eea1e1aeb2143a

Signed-off-by: Michael Stack <stack@apache.org>
2017-07-03 19:43:42 -07:00
Guanghao Zhang af36bfb2f7 HBASE-18283 Provide a construct method which accept a thread pool for AsyncAdmin 2017-07-04 10:18:49 +08:00
Michael Stack f4c99a6a89 HBASE-11707 Using Map instead of list in FailedServers of RpcClient (Liu Shaohui)
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-01 12:25:21 -07:00
Michael Stack 549fb1677b HBASE-18266 Create static empty byte array to save memory (Ben Epstein) 2017-06-30 15:31:16 -07:00
Sean Busbey d00850f66d HBASE-15391 ADDENDUM Avoid too large "deleted from META" info log. 2017-06-30 09:52:19 -05:00
Michael Stack 09cafb1163 HBASE-15391 Avoid too large "deleted from META" info log (Liu Shaohui) 2017-06-30 03:53:58 -07:00
Michael Stack 619dcf24f4 Revert "HBASE-18229: create new Async Split API to embrace AM v2"
TestShell is failing.

This reverts commit 290fb8965d.
2017-06-30 03:30:50 -07:00
Yi Liang 290fb8965d HBASE-18229: create new Async Split API to embrace AM v2
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-29 16:21:08 -07:00
张世彬10204932 f6dd43be70 HBASE-17982 correct spelling error of 'occured'
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-29 15:10:08 -07:00
Xiang Li 2db99eb74a HBASE-18022: Refine the error message issued with TableNotFoundException when expected table is not the same as the one fetched from meta
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-29 15:07:01 -07:00
Guanghao Zhang 69e3234226 HBASE-18234 Revisit the async admin api (addendum) 2017-06-27 10:53:52 +08:00
Guanghao Zhang 289938337b HBASE-18234 Revisit the async admin api 2017-06-26 17:30:47 +08:00
Michael Stack 4184ae7563 HBASE-18004 getRegionLocations needs to be called once in
ScannerCallableWithReplicas#call() (Huaxiang Sun)
2017-06-15 13:41:49 -07:00
Andrew Purtell 6af3ee44c0 HBASE-18219 Fix typo in constant HConstants.HBASE_CLIENT_MEAT_REPLICA_SCAN_TIMEOUT 2017-06-14 16:03:50 -07:00
zhangduo eca1ec3356 HBASE-18199 Race in NettyRpcConnection may cause call stuck in BufferCallBeforeInitHandler forever 2017-06-10 19:12:29 +08:00
Yi Liang ea7d51e129 HBASE-18109: Assign system tables first
This issue adds comments and a sort so system tables are queued first
(which will ensure they go out first). This should be good enough
along w/ existing scheduling mechanisms to ensure system/meta get
assigned first.

Signed-off-by: Michael Stack <stack@apache.org>
2017-06-08 13:26:42 -07:00
Chia-Ping Tsai 1e7804634c HBASE-18008 Any HColumnDescriptor we give out should be immutable 2017-06-08 23:28:33 +08:00
Michael Stack adfb48eeb8 HBASE-18190 Set version in branch-2 to 2.0.0-alpha-1 2017-06-07 21:10:01 -07:00
huzheng 347bef8d33 HBASE-17678 FilterList with MUST_PASS_ONE lead to redundancy cells returned - addendum
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-07 07:35:03 -07:00
huzheng 0d0c330401 HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-06 21:08:12 -07:00
Phil Yang 2f1923a823 HBASE-15576 Scanning cursor to prevent blocking long time on ResultScanner.next() 2017-06-07 11:32:04 +08:00
tedyu ef46debde8 HBASE-18005 read replica: handle the case that region server hosting both primary replica and meta region is down (huaxiang sun) 2017-06-02 09:29:51 -07:00
Michael Stack 3975bbd008 HBASE-14614 Procedure v2 - Core Assignment Manager (Matteo Bertozzi) Move to a new AssignmentManager, one that describes Assignment using a State Machine built on top of ProcedureV2 facility.
This doc. keeps state on where we are at w/ the new AM:
https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.vfdoxqut9lqn
Includes list of tests disabled by this patch with reasons why.

Based on patches from Matteos' repository and then fix up to get it all to pass cluster
tests, filling in some missing functionality, fix of findbugs, fixing bugs, etc..
including:

    1. HBASE-14616 Procedure v2 - Replace the old AM with the new AM.
    The basis comes from Matteo's repo here:
    689227fcbf

    Patch replaces old AM with the new under subpackage master.assignment.
    Mostly just updating classes to use new AM -- import changes -- rather
    than the old. It also removes old AM and supporting classes.
    See below for more detail.

    2. HBASE-14614 Procedure v2 - Core Assignment Manager (Matteo Bertozzi)
    3622cba4e3

    Adds running of remote procedure. Adds batching of remote calls.
    Adds support for assign/unassign in procedures. Adds version info
    reporting in rpc. Adds start of an AMv2.

    3. Reporting of remote RS version is from here:
    ddb4df3964.patch

    4. And remote dispatch of procedures is from:
    186b9e7c4d

    5. The split merge patches from here are also melded in:
    9a3a95a2c2
    and d6289307a0

We add testing util for new AM and new sets of tests.

Does a bunch of fixup on logging so its possible to follow a procedures' narrative by grepping
procedure id. We spewed loads of log too on big transitions such as master fail; fixed.

Fix CatalogTracker. Make it use Procedures doing clean up of Region data on split/merge.
Without these changes, ITBLL was failing at larger scale (3-4hours 5B rows) because we were
splitting split Regions among other things (CJ would run but wasn't
taking lock on Regions so havoc).

    Added a bunch of doc. on Procedure primitives.

    Added new region-based state machine base class. Moved region-based
    state machines on to it.

    Found bugs in the way procedure locking was doing in a few of the
    region-based Procedures. Having them all have same subclass helps here.

    Added isSplittable and isMergeable to the Region Interface.

    Master would split/merge even though the Regions still had
    references. Fixed it so Master asks RegionServer if Region
    is splittable.

    Messing more w/ logging. Made all procedures log the same and report
    the state the same; helps when logging is regular.

    Rewrote TestCatalogTracker. Enabled TestMergeTableRegionProcedure.

    Added more functionality to MockMasterServices so can use it doing
    standalone testing of Procedures (made TestCatalogTracker use it
    instead of its own version).

    Add to MasterServices ability to wait on Master being up -- makes
    it so can Mock Master and start to implement standalone split testing.
    Start in on a Split region standalone test in TestAM.

    Fix bug where a Split can fail because it comes in in the middle of
    a Move (by holding lock for duration of a Move).

    Breaks CPs that were watching merge/split. These are run by Master now
    so you need to observe on Master, not on RegionServer.

    Details:

    M hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
    Takes List of regionstates on construction rather than a Set.
    NOTE!!!!! This is a change in a public class.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
    Add utility getShortNameToLog

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ShortCircuitMasterConnection.java
    Add support for dispatching assign, split and merge processes.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/master/RegionState.java
    Purge old overlapping states: PENDING_OPEN, PENDING_CLOSE, etc.

    M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
    Lots of doc on its inner workings. Bug fixes.

    M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
    Log and doc on workings. Bug fixes.

    A hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
    Dispatch remote procedures every 150ms or 32 items -- which ever
    happens first (configurable). Runs a timeout thread. This facility is
    not on yet; will come in as part of a later fix. Currently works a
    region at a time. This class carries notion of a remote procedure and of a buffer full of these.
    "hbase.procedure.remote.dispatcher.threadpool.size" with default = 128
    "hbase.procedure.remote.dispatcher.delay.msec" with default = 150ms
    "hbase.procedure.remote.dispatcher.max.queue.size" with default = 32

    M hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
    Add in support for merge. Remove no-longer used methods.

    M hbase-protocol-shaded/src/main/protobuf/Admin.proto b/hbase-protocol-shaded/src/main/protobuf/Admin.proto
    Add execute procedures call ExecuteProcedures.

    M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
    Add assign and unassign state support for procedures.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/client/VersionInfoUtil.java
    Adds getting RS version out of RPC
    Examples: (1.3.4 is 0x0103004, 2.1.0 is 0x0201000)

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
    Remove periodic metrics chore. This is done over in new AM now.
    Replace AM with the new. Host the procedures executor.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
    Have AMv2 handle assigning meta.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
    Extract version number of the server making rpc.

    A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
    Add new assign procedure. Runs assign via Procedure Dispatch.
    There can only be one RegionTransitionProcedure per region running at the time,
    since each procedure takes a lock on the region.

    D hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignCallable.java
    D hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
    D hbase-server/src/main/java/org/apache/hadoop/hbase/master/BulkAssigner.java
    D hbase-server/src/main/java/org/apache/hadoop/hbase/master/GeneralBulkAssigner.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/GeneralBulkAssigner.java
    Remove these hacky classes that were never supposed to live longer than
    a month or so to be replaced with real assigners.

    D hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStateStore.java
    D hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java
    D hbase-server/src/main/java/org/apache/hadoop/hbase/master/UnAssignCallable.java

    A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
    A procedure-based AM (AMv2).

    TODO
     - handle region migration
     - handle meta assignment first
     - handle sys table assignment first (e.g. acl, namespace)
     - handle table priorities
      "hbase.assignment.bootstrap.thread.pool.size"; default size is 16.
      "hbase.assignment.dispatch.wait.msec"; default wait is 150
      "hbase.assignment.dispatch.wait.queue.max.size"; wait max default is 100
      "hbase.assignment.rit.chore.interval.msec"; default is 5 * 1000;
      "hbase.assignment.maximum.attempts"; default is 10;

     A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
     Procedure that runs subprocedure to unassign and then assign to new location

     A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Manage store of region state (in hbase:meta by default).

     A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
     In-memory state of all regions. Used by AMv2.

     A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
     Base RIT procedure for Assign and Unassign.

     A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
     Unassign procedure.

     A hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
     Run region assignement in a manner that pays attention to target server version.
     Adds "hbase.regionserver.rpc.startup.waittime"; defaults 60 seconds.
2017-05-31 17:49:11 -07:00
Phil Yang c945d2b2d9 HBASE-18122 addendum for fixing async client scanner 2017-05-31 15:30:24 +08:00
zhangduo efc7edc81a HBASE-18115 Move SaslServer creation to HBaseSaslRpcServer 2017-05-27 11:38:41 +08:00
Phil Yang b74474dfc9 HBASE-18113 Handle old client without include_stop_row flag when startRow equals endRow 2017-05-26 13:38:56 +08:00
Michael Stack a3c5a74487 Revert "HBASE-14614 Procedure v2 - Core Assignment Manager (Matteo Bertozzi)"
Revert a mistaken commit!!!

This reverts commit dc1065a85d.
2017-05-24 23:31:36 -07:00
Michael Stack dc1065a85d HBASE-14614 Procedure v2 - Core Assignment Manager (Matteo Bertozzi)
Move to a new AssignmentManager, one that describes Assignment using
a State Machine built on top of ProcedureV2 facility.

This doc. keeps state on where we are at w/ the new AM:
https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.vfdoxqut9lqn
Includes list of tests disabled by this patch with reasons why.

Based on patches from Matteos' repository and then fix up to get it all to pass cluster
tests, filling in some missing functionality, fix of findbugs, fixing bugs, etc..
including:

1. HBASE-14616 Procedure v2 - Replace the old AM with the new AM.
The basis comes from Matteo's repo here:
689227fcbf

Patch replaces old AM with the new under subpackage master.assignment.
Mostly just updating classes to use new AM -- import changes -- rather
than the old. It also removes old AM and supporting classes.
See below for more detail.

2. HBASE-14614 Procedure v2 - Core Assignment Manager (Matteo Bertozzi)
3622cba4e3

Adds running of remote procedure. Adds batching of remote calls.
Adds support for assign/unassign in procedures. Adds version info
reporting in rpc. Adds start of an AMv2.

3. Reporting of remote RS version is from here:
ddb4df3964.patch

4. And remote dispatch of procedures is from:
186b9e7c4d

5. The split merge patches from here are also melded in:
9a3a95a2c2
and d6289307a0

We add testing util for new AM and new sets of tests.

Does a bunch of fixup on logging so its possible to follow a procedures' narrative by grepping
procedure id. We spewed loads of log too on big transitions such as master fail; fixed.

Fix CatalogTracker. Make it use Procedures doing clean up of Region data on split/merge.
Without these changes, ITBLL was failing at larger scale (3-4hours 5B rows) because we were
splitting split Regions among other things (CJ would run but wasn't
taking lock on Regions so havoc).

Added a bunch of doc. on Procedure primitives.

Added new region-based state machine base class. Moved region-based
state machines on to it.

Found bugs in the way procedure locking was doing in a few of the
region-based Procedures. Having them all have same subclass helps here.

Added isSplittable and isMergeable to the Region Interface.

Master would split/merge even though the Regions still had
references. Fixed it so Master asks RegionServer if Region
is splittable.

Messing more w/ logging. Made all procedures log the same and report
the state the same; helps when logging is regular.

Rewrote TestCatalogTracker. Enabled TestMergeTableRegionProcedure.

Added more functionality to MockMasterServices so can use it doing
standalone testing of Procedures (made TestCatalogTracker use it
instead of its own version).

Add to MasterServices ability to wait on Master being up -- makes
it so can Mock Master and start to implement standalone split testing.
Start in on a Split region standalone test in TestAM.

Fix bug where a Split can fail because it comes in in the middle of
a Move (by holding lock for duration of a Move).

Breaks CPs that were watching merge/split. These are run by Master now
so you need to observe on Master, not on RegionServer.

Details:

M hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
Takes List of regionstates on construction rather than a Set.
NOTE!!!!! This is a change in a public class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
Add utility getShortNameToLog

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ShortCircuitMasterConnection.java
Add support for dispatching assign, split and merge processes.

M hbase-client/src/main/java/org/apache/hadoop/hbase/master/RegionState.java
Purge old overlapping states: PENDING_OPEN, PENDING_CLOSE, etc.

M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
Lots of doc on its inner workings. Bug fixes.

M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
Log and doc on workings. Bug fixes.

A hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
Dispatch remote procedures every 150ms or 32 items -- which ever
happens first (configurable). Runs a timeout thread. This facility is
not on yet; will come in as part of a later fix. Currently works a
region at a time. This class carries notion of a remote procedure and of a buffer full of these.
"hbase.procedure.remote.dispatcher.threadpool.size" with default = 128
"hbase.procedure.remote.dispatcher.delay.msec" with default = 150ms
"hbase.procedure.remote.dispatcher.max.queue.size" with default = 32

M hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
Add in support for merge. Remove no-longer used methods.

M hbase-protocol-shaded/src/main/protobuf/Admin.proto b/hbase-protocol-shaded/src/main/protobuf/Admin.proto
Add execute procedures call ExecuteProcedures.

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Add assign and unassign state support for procedures.

M hbase-server/src/main/java/org/apache/hadoop/hbase/client/VersionInfoUtil.java
Adds getting RS version out of RPC
Examples: (1.3.4 is 0x0103004, 2.1.0 is 0x0201000)

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Remove periodic metrics chore. This is done over in new AM now.
Replace AM with the new. Host the procedures executor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
Have AMv2 handle assigning meta.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
Extract version number of the server making rpc.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
Add new assign procedure. Runs assign via Procedure Dispatch.
There can only be one RegionTransitionProcedure per region running at the time,
since each procedure takes a lock on the region.

D hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignCallable.java
D hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
D hbase-server/src/main/java/org/apache/hadoop/hbase/master/BulkAssigner.java
D hbase-server/src/main/java/org/apache/hadoop/hbase/master/GeneralBulkAssigner.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/GeneralBulkAssigner.java
Remove these hacky classes that were never supposed to live longer than
a month or so to be replaced with real assigners.

D hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStateStore.java
D hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java
D hbase-server/src/main/java/org/apache/hadoop/hbase/master/UnAssignCallable.java

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
A procedure-based AM (AMv2).

TODO
 - handle region migration
 - handle meta assignment first
 - handle sys table assignment first (e.g. acl, namespace)
 - handle table priorities
  "hbase.assignment.bootstrap.thread.pool.size"; default size is 16.
  "hbase.assignment.dispatch.wait.msec"; default wait is 150
  "hbase.assignment.dispatch.wait.queue.max.size"; wait max default is 100
  "hbase.assignment.rit.chore.interval.msec"; default is 5 * 1000;
  "hbase.assignment.maximum.attempts"; default is 10;

 A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
 Procedure that runs subprocedure to unassign and then assign to new location

 A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
 Manage store of region state (in hbase:meta by default).

 A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
 In-memory state of all regions. Used by AMv2.

 A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
 Base RIT procedure for Assign and Unassign.

 A hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
 Unassign procedure.

 A hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
 Run region assignement in a manner that pays attention to target server version.
 Adds "hbase.regionserver.rpc.startup.waittime"; defaults 60 seconds.
2017-05-24 20:47:25 -07:00
Josh Elser ed618da906 HBASE-17981 Consolidate the space quota shell commands 2017-05-22 13:41:36 -04:00
Josh Elser 13b9a1ed01 HBASE-17976 Remove stability annotation from public audience class 2017-05-22 13:41:36 -04:00
Josh Elser d671a1dbc6 HBASE-17955 Various reviewboard improvements to space quota work
Most notable change is to cache SpaceViolationPolicyEnforcement objects
in the write path. When a table has no quota or there is not SpaceQuotaSnapshot
for that table (yet), we want to avoid creating lots of
SpaceViolationPolicyEnforcement instances, caching one instance
instead. This will help reduce GC pressure.
2017-05-22 13:41:36 -04:00
Josh Elser a8460b8bad HBASE-17794 Swap "violation" for "snapshot" where appropriate
A couple of variables and comments in which violation is incorrectly
used to describe what the code is doing. This was a hold over from early
implementation -- need to scrub these out for clarity.
2017-05-22 13:41:35 -04:00
Josh Elser 13af7f8ac6 HBASE-17002 JMX metrics and some UI additions for space quotas 2017-05-22 13:41:35 -04:00
Josh Elser 80a1f8fa2a HBASE-17428 Implement informational RPCs for space quotas
Create some RPCs that can expose the in-memory state that the
RegionServers and Master hold to drive the space quota "state machine".
Then, create some hbase shell commands to interact with those.
2017-05-22 13:41:35 -04:00
Josh Elser 6c9082fe16 HBASE-17259 API to remove space quotas on a table/namespace 2017-05-22 13:41:35 -04:00
Josh Elser 34ba143fc8 HBASE-17001 Enforce quota violation policies in the RegionServer
The nuts-and-bolts of filesystem quotas. The Master must inform
RegionServers of the violation of a quota by a table. The RegionServer
must apply the violation policy as configured. Need to ensure
that the proper interfaces exist to satisfy all necessary policies.

This required a massive rewrite of the internal tracking by
the general space quota feature. Instead of tracking "violations",
we need to start tracking "usage". This allows us to make the decision
at the RegionServer level as to when the files in a bulk load request
should be accept or rejected which ultimately lets us avoid bulk loads
dramatically exceeding a configured space quota.
2017-05-22 13:41:35 -04:00
Josh Elser 98b4181f43 HBASE-16999 Implement master and regionserver synchronization of quota state
* Implement the RegionServer reading violation from the quota table
* Implement the Master reporting violations to the quota table
* RegionServers need to track its enforced policies
2017-05-22 13:41:35 -04:00
Josh Elser 533470f8c8 HBASE-16998 Implement Master-side analysis of region space reports
Adds a new Chore to the Master that analyzes the reports that are
sent by RegionServers. The Master must then, for all tables with
quotas, determine the tables that are violating quotas and move
those tables into violation. Similarly, tables no longer violating
the quota can be moved out of violation.

The Chore is the "stateful" bit, managing which tables are and
are not in violation. Everything else is just performing
computation and informing the Chore on the updated state.

Added InterfaceAudience annotations and clean up the QuotaObserverChore
constructor. Cleaned up some javadoc and QuotaObserverChore. Reuse
the QuotaViolationStore impl objects.
2017-05-22 13:41:35 -04:00
tedyu f74e051bce HBASE-16996 Implement storage/retrieval of filesystem-use quotas into quota table (Josh Elser) 2017-05-22 13:41:35 -04:00
tedyu c5172169f2 HBASE-16995 Build client Java API and client protobuf messages - addendum fixes line lengths (Josh Elser) 2017-05-22 13:41:35 -04:00
tedyu 140413c11b HBASE-16995 Build client Java API and client protobuf messages - addendum fixes white spaces (Josh Elser) 2017-05-22 13:41:35 -04:00
tedyu 4dfafd6e50 HBASE-16995 Build client Java API and client protobuf messages (Josh Elser) 2017-05-22 13:41:35 -04:00
Apekshit Sharma 23ea2c36f5 HBASE-18068 Fix flaky test TestAsyncSnapshotAdminApi
- internalRestoreSnapshot() returns future which completes by just getting proc_id from master. Changed it to wait for the procedure to complete.
- Refactor TestAsyncSnapshotAdminApi: Add cleanup() which deletes all tables and snapshots after every test run. Simplifies individual tests.

Change-Id: Idc30fb699db32d58fd0f60da220953a430f1d3cc
2017-05-22 09:20:37 -07:00
Josh Elser 709f5a1980 HBASE-18075 Support non-latin table names and namespaces 2017-05-21 22:24:12 -04:00
Guanghao Zhang 553d5db355 HBASE-15616 (addendum) Allow null qualifier for all table operations 2017-05-22 09:42:02 +08:00
Guanghao Zhang 3fe4b28bb0 HBASE-15616 Allow null qualifier for all table operations 2017-05-19 17:47:08 +08:00
tedyu 958cd2d1b7 HBASE-18035 Meta replica does not give any primaryOperationTimeout to primary meta region (huaxiang sun) 2017-05-18 15:56:41 -07:00
tedyu d137991ccc HBASE-18058 Zookeeper retry sleep time should have an upper limit (Allan Yang) 2017-05-18 15:50:59 -07:00
huzheng 37dd8ff722 HBASE-11013: Clone Snapshots on Secure Cluster Should provide option to apply Retained User Permissions
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-05-18 17:39:50 +08:00
Guanghao Zhang 62d7323023 HBASE-18053 AsyncTableResultScanner will hang when scan wrong column family 2017-05-17 12:16:51 +08:00
anoopsamjohn 67d1358311 HBASE-18043 Institute a hard limit for individual cell size that cannot be overridden by clients. - Addendum to fix test issue in TestMobStoreScanner. 2017-05-16 18:53:27 +05:30
Andrew Purtell 37650775a5 Revert "HBASE-18026 ProtobufUtil seems to do extra array copying"
This reverts commit 5895631634.
2017-05-15 18:06:33 -07:00
tedyu 5e046151d6 HBASE-11013: Clone Snapshots on Secure Cluster Should provide option to apply Retained User Permissions - revert, pending work in snapshot descriptor 2017-05-11 18:53:14 -07:00
tedyu b3dcfb659e HBASE-17928 Shell tool to clear compaction queues (Guangxu Cheng) 2017-05-11 18:47:12 -07:00
tedyu d8d4ba7c59 HBASE-17928 Shell tool to clear compaction queues - revert pending work in snapshot descriptor 2017-05-11 18:43:59 -07:00
Michael Stack d64acfd304 HBASE-18021 Add more info in timed out RetriesExhaustedException for read replica client get processing (Huaxiang Sun) 2017-05-11 17:23:43 -07:00
Vincent 5895631634 HBASE-18026 ProtobufUtil seems to do extra array copying
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-05-11 13:11:28 -07:00
tedyu 815b0f853b HBASE-17928 Shell tool to clear compaction queues (Guangxu Cheng) 2017-05-09 18:32:38 -07:00
huzheng 951b23a44c HBASE-11013: Clone Snapshots on Secure Cluster Should provide option to apply Retained User Permissions
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-05-09 09:32:48 -07:00
tedyu ac1024af21 HBASE-16356 REST API scanner: row prefix filter and custom filter parameters are mutually exclusive (Ben Watson) 2017-05-09 07:58:20 -07:00
huzheng 2026540ea3 HBASE-17667: Implement async flush/compact region methods
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-05-05 21:12:49 +08:00
huzheng ff998ef74f HBASE-17867: Implement async procedure RPC API(list/exec/abort/isFinished)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-05-03 21:46:37 +08:00
jc@lifove.net c0f265384f HBASE-17862 Fix a condition that always returns true
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-05-03 00:29:00 +08:00
Chia-Ping Tsai 923508c9de HBASE-8486 remove references to -ROOT- table from table descriptors where allowed by compatibility rules.
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-04-28 12:35:47 -05:00
QilinCao 03e8f6b56e HBASE-17835 Spelling mistakes in the Java source
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-04-28 11:25:57 -05:00
huzheng 4bc0eb31c3 HBASE-17865: Implement async listSnapshot/deleteSnapshot methods.
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-04-28 09:09:13 +08:00
ckulkarni eb6ded4849 HBASE-17448 Export metrics from RecoverableZooKeeper
Added metrics for RecoverableZooKeeper related to specific exceptions,
total failed ZooKeeper API calls and latency histograms for read,
write and sync operations. Also added unit tests for the same. Added
service provider for the ZooKeeper metrics implementation inside the
hadoop compatibility module.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-04-26 18:30:13 -07:00
zhangduo 177344cdbf HBASE-17873 Change the IA.Public annotation to IA.Private for unstable API
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-04-26 16:39:11 -05:00
Chia-Ping Tsai 053e61541e HBASE-15583 Any HTableDescriptor we give out should be immutable 2017-04-27 03:22:29 +08:00
Balazs Meszaros 2557506415 HBASE-15143 Procedure v2 - Web UI displaying queues
Signed-off-by: Michael Stack <stack@apache.org>
2017-04-25 09:39:28 -07:00
huzheng d39f40e787 HBASE-17864: Implement async snapshot/cloneSnapshot/restoreSnapshot methods
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-04-21 18:57:43 +08:00
Ashish Singhi 87f2bb5796 HBASE-13288 Fix naming of parameter in Delete constructor
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-04-21 11:09:44 +08:00
Guanghao Zhang 40cc666ac9 HBASE-17915 Implement async replication admin methods 2017-04-20 18:13:03 +08:00
huzheng 8db9760363 HBASE-17866: Implement async setQuota/getQuota methods
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-04-17 09:49:30 +08:00
Michael Stack 7678855fac HBASE-17904 Get runs into NoSuchElementException when using Read Replica, with hbase. ipc.client.specificThreadForWriting
to be true and hbase.rpc.client.impl to be org.apache.hadoop.hbase.ipc.RpcClientImpl (Huaxiang Sun)
2017-04-16 11:01:06 -07:00
Umesh Agashe c8461456d0 HBASE-17888: Added generic methods for updating metrics on submit and finish of a procedure execution
Signed-off-by: Michael Stack <stack@apache.org>
2017-04-14 11:51:08 -07:00
AShiou 23249eb0f5 HBASE-17895 TestAsyncProcess#testAction fails if unsafe support is false
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-04-12 11:42:13 +08:00
Nemo Chen 0b5bd78d6e HBASE-16469 Several log refactoring/improvement suggestions
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-04-11 14:16:12 -05:00
Chia-Ping Tsai 18c5ecf6ed HBASE-17881 Remove the ByteBufferCellImpl 2017-04-07 21:14:19 +08:00
Yu Li d7e3116a17 HBASE-17886 Fix compatibility of ServerSideScanMetrics 2017-04-06 17:29:22 +08:00
zhangduo a66d491892 HBASE-17857 Remove IS annotations from IA.Public classes 2017-04-05 15:34:06 +08:00
huzheng 5f98ad2053 HBASE-17668: Implement async assgin/offline/move/unassign methods
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-04-01 08:19:14 +08:00
zhangduo 0ec1459467 HBASE-17855 Fix typo in async client implementation 2017-03-31 13:00:27 +08:00
Michael Stack d033cbb715 HBASE-17844 Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical changes)
Minor changes related to HBASE-14614. Added comments. Changed logging.
Added toString formatting. Removed imports. Removed unused code.
2017-03-30 10:31:04 -07:00
Guanghao Zhang 752b258b7c HBASE-17520 Implement isTableEnabled/Disabled/Available methods 2017-03-30 13:37:17 +08:00
Jan Hentschel 50e9825139 HBASE-13395 Removed HTableInterface
Signed-off-by: CHIA-PING TSAI <chia7712@gmail.com>
2017-03-26 00:04:21 +08:00
huzheng faf81d5133 HBASE-17669: Implement async mergeRegion/splitRegion methods
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-03-24 21:25:24 +08:00
zhangduo f1c1f258e5 HBASE-17595 addendum fix the problem for mayHaveMoreCellsInRow 2017-03-23 20:34:10 +08:00
CHIA-PING TSAI fe3c32ebd5 HBASE-17809 cleanup unused class 2017-03-23 14:15:28 +08:00
zhangduo cc59fe4e91 HBASE-17812 Remove RpcConnection from pool in AbstractRpcClient.cancelConnections 2017-03-21 10:07:16 +08:00
brandboat 9c8f02e4ef HBASE-16014 Get and Put constructor argument lists are divergent
Signed-off-by: CHIA-PING TSAI <chia7712@gmail.com>
2017-03-21 09:33:24 +08:00
Jan Hentschel 55d6dcaf87 HBASE-16084 Cleaned up the stale references in Javadoc
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-03-20 10:55:36 -07:00
zhangduo 5b4bb8217d HBASE-17691 Add ScanMetrics support for async scan 2017-03-20 20:54:04 +08:00
tedyu 75d0f49dcd HBASE-14123 HBase Backup/Restore Phase 2 (Vladimir Rodionov) 2017-03-18 03:04:19 -07:00
CHIA-PING TSAI e2a070cae0 HBASE-17778 Remove the testing code in the AsyncRequestFutureImpl 2017-03-17 07:49:13 +08:00
Guanghao Zhang 6a6fff103e HBASE-17790 Mark ReplicationAdmin's peerAdded and listReplicationPeers as Deprecated 2017-03-16 16:55:18 +08:00
zhangduo 1849e8a5a7 HBASE-17740 Correct the semantic of batch and partial for async client 2017-03-16 09:44:23 +08:00
Chia-Ping Tsai 0ecb678259 HBASE-17723 ClientAsyncPrefetchScanner may end prematurely when the size of the cache is one 2017-03-16 03:07:20 +08:00
zhangduo a49bc58a54 HBASE-17584 Expose ScanMetrics with ResultScanner rather than Scan 2017-03-15 17:48:58 +08:00
tedyu 777fea552e HBASE-17779 disable_table_replication returns misleading message and does not turn off replication (Janos Gub) 2017-03-14 12:13:34 -07:00
zhangduo ed6e5d6999 HBASE-17763 IPCUtil.wrapException will wrap DoNotRetryIOException with IOException 2017-03-10 15:37:59 +08:00
Yu Li c63a871176 HBASE-17745 Support short circuit connection for master services 2017-03-10 09:15:04 +08:00
Guanghao Zhang 4ff838df13 HBASE-17600 Implement get/create/modify/delete/list namespace admin operations 2017-03-09 10:32:46 +08:00
tedyu c42a0665b9 HBASE-17460 enable_table_replication can not perform cyclic replication of a table - addendum removes replication check (Janos Gub) 2017-03-07 19:42:26 -08:00
Jan Hentschel b53f354763 HBASE-17532 Replaced explicit type with diamond operator
Signed-off-by: Michael Stack <stack@apache.org>
2017-03-07 11:22:51 -08:00
Karan Mehta fee86ebd7b HBASE-17716 Formalize Scan Metric names
Signed-off-by: Michael Stack <stack@apache.org>
2017-03-07 07:41:54 -08:00
Phil Yang b635414e83 HBASE-15484 Correct the semantic of batch and partial - amend to fix bug and revise the JavaDoc for related APIs. 2017-03-07 22:23:56 +08:00
Phil Yang 0d3e986f7e HBASE-15484 Correct the semantic of batch and partial 2017-03-07 11:52:04 +08:00
Michael Stack 4beae9a56e Javadoc on what BLOCKSIZE means (related to HBASE-15248) 2017-03-06 15:53:13 -08:00
Andrew Purtell 404a2883f2 HBASE-17722 Metrics subsystem stop/start messages add a lot of useless bulk to operational logging 2017-03-03 12:40:06 -08:00
Josh Elser 5645684ec9 HBASE-17717 Explicitly use "sasl" ACL scheme for hbase superuser
The special "auth" ZK ACL scheme will always set the ACL's id (the
user who is allowed) to be the authenticated user of the ZK connection.
This results in the HBase superuser not actually receiving the
permissions as the ZKUtil intends to do. Since we know we have security
enabled, we can instead explicitly list "sasl" as the ACL scheme
instead.
2017-03-02 15:46:33 -05:00
huzheng 697a55a878 HBASE-17646: Implement Async getRegion method
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-03-02 11:52:05 +08:00
Jingcheng Du f4e0ea24e6 HBASE-17688 MultiRowRangeFilter not working correctly if given same start and stop RowKey 2017-02-28 11:38:35 +08:00
Jingcheng Du 5a8f1e8aaa HBASE-17690 Clean up MOB code 2017-02-28 09:57:34 +08:00
Jan Hentschel 84a9eb3f5c HBASE-17634 Cleaned up the usage of Result.isPartial
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-02-27 20:44:35 +08:00
Apekshit Sharma f444b3b542 HBASE-17312 Use 'default' keyword in coprocessor Observer interfaces to get rid of 'Base...Observer' implementations. Some javadoc improvements too.
Reason for refactor:
In cases where one might need to use multiple observers, say region, master and regionserver; and the fact that only one class can be extended, it gives rise to following pattern:

public class BaseMasterAndRegionObserver
  extends BaseRegionObserver
  implements MasterObserver

class AccessController
  extends BaseMasterAndRegionObserver
  implements RegionServerObserver

were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.

There is an example of simple case too where the current design fails.
Say only one observer is needed by the coprocessor, but the design doesn't permit extending even that single observer (see RSGroupAdminEndpoint), that leads to copy of full Bas
e...Observer class into coprocessor class leading to 1000s of lines of code and this ugly mix of 5 main functions with 100 useless functions.

Javadocs changes:
- Adds class comments on 'default' methods and expectations.
- Adds explanaiton of Exception handling in Observers' class comment. Removes redundant @throws before each function.
- Improves javadocs for a bunch of functions
- deletes empty @params in a bunch of places

Change-Id: I265738d47e8554e7b4678e88bb916a0cc7d00ab3
2017-02-25 02:49:01 -08:00
zhangduo 62de29e6f2 HBASE-16991 Make the initialization of AsyncConnection asynchronous 2017-02-24 15:36:54 +08:00
tedyu 371f2bd907 HBASE-17460 enable_table_replication can not perform cyclic replication of a table - addendum addresses Enis' comments 2017-02-23 21:10:44 -08:00
Abhishek Singh Chouhan 0d656b1394 Amend HBASE-17069 RegionServer writes invalid META entries in some circumstances
Fix Hregioninfo being null during merge

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-02-23 13:45:58 -08:00
zhangduo 8fb44fae35 HBASE-17595 Add partial result support for small/limited scan 2017-02-23 16:03:42 +08:00
zhangduo f037f230fd HBASE-17608 Add suspend support for RawScanResultConsumer 2017-02-22 20:32:03 +08:00
huzheng 22fa1cd3df HBASE-17472: Correct the semantic of permission grant
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-02-20 20:13:43 +08:00
tedyu 0e05537dec HBASE-17460 enable_table_replication can not perform cyclic replication of a table (NITIN VERMA) 2017-02-17 17:56:53 -08:00
tedyu 3fc2889f7a HBASE-17460 enable_table_replication can not perform cyclic replication of a table - revert due to reproducible test failure 2017-02-16 20:17:17 -08:00
huzheng e129e6b65d HBASE-17619: Add async admin Impl which connect to RegionServer and implement close region methods.
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-02-17 09:32:45 +08:00
zhangduo c8f3c8630b HBASE-17583 Add inclusive/exclusive support for startRow and endRow of scan for sync client 2017-02-15 10:18:41 +08:00
Michael Stack e019961150 HBASE-17624 Address late review of HBASE-6721, rsgroups feature
Addresses review comments by Sean Busbey and Appy that happened
to come in long after the commit of HBASE-6721, the original
rsgroup issue.

Also includes subsequent accommodation of Duo Zhang review.

Adds a new type to hold hostname and port. It is called
Address. It is a facade over Guava's HostAndPort. Replace
all instances of HostAndPort with Address. In particular,
those places where HostAndPort was part of the rsgroup
public API.

Fix licenses. Add audience annotations.

Cleanup and note concurrency expectation on a few core classes.
In particular, all access on RSGroupInfoManager is made
synchronized.

M hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
 Host the hostname and port in an instance of the new type Address.
 Add a bunch of deprecation of exotic string parses that should never
 have been public.

M hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdmin.java
 Make this an Interface rather than abstract class. Creation was a
 static internal method that only chose one type.... Let it be free
 as a true Interface instead.
2017-02-14 09:29:17 -08:00
Jan Hentschel 55c2e2d484 HBASE-9702 Changed unit tests to use method names for tables 2017-02-13 13:27:55 -08:00
tedyu d51e287877 HBASE-17460 enable_table_replication can not perform cyclic replication of a table (NITIN VERMA) 2017-02-10 14:54:05 -08:00
zhangduo 712fe69e4d HBASE-17599 Use mayHaveMoreCellsInRow instead of isPartial 2017-02-09 15:38:02 +08:00
Guanghao Zhang 9a78d00884 HBASE-17389 Convert all internal usages from ReplicationAdmin to Admin 2017-02-07 10:18:59 +08:00
zhangduo af9d359b8e HBASE-17402 TestAsyncTableScan sometimes hangs 2017-02-07 09:42:42 +08:00
Enis Soztutar 4d730244a9 HBASE-17187 DoNotRetryExceptions from coprocessors should bubble up to the application 2017-02-06 12:00:42 -08:00
Guanghao Zhang 5093a49e0b HBASE-17596 Implement add/delete/modify column family methods 2017-02-06 13:45:01 +08:00
Guanghao Zhang 26a94844f5 HBASE-17511 Implement enable/disable table methods 2017-02-05 14:18:40 +08:00
zhangduo ffa0cea2a3 HBASE-17508 Unify the implementation of small scan and regular scan for sync client 2017-02-05 08:49:39 +08:00
Josh Elser 4e77b18da2 HBASE-17592 Fix rpcTimeout typo (Attila Sasvari) 2017-02-03 16:50:19 -05:00
Huaxiang Sun 1159296541 HBASE-16981 Expand Mob Compaction Partition policy from daily to weekly, monthly
Support weekly and monthly mob compact partition policies in addition to the existing
daily partition policy.

Signed-off-by: Jingcheng Du <jingchengdu@apache.org>
2017-02-03 15:52:43 +08:00
Jerry He bc168b419d HBASE-17581 mvn clean test -PskipXXXTests does not work properly for some modules (Yi Liang) 2017-02-02 11:05:17 -08:00
Ajay Jadhav f8b1f57b05 HBASE-17280 Add mechanism to control hbase cleaner behavior
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-02-01 22:16:01 -06:00
Sean Busbey 2c799fb70a Revert "Add mechanism to control hbase cleaner behavior"
This reverts commit ef052521cd.

Bad commit message.
2017-02-01 22:11:48 -06:00
Ajay Jadhav ef052521cd Add mechanism to control hbase cleaner behavior
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-02-01 19:33:06 -08:00
zhangduo b7fc7bf246 HBASE-17346 Add coprocessor service support 2017-01-31 15:19:33 +08:00
Michael Stack fb2c89b1b3 HBASE-16785 We are not running all tests
M TestStressWALProcedureStore.java
Disable test that now runs that fails because of difference in pb3.1.0.

Signed-off-by: Michael Stack <stack@apache.org>
2017-01-26 21:49:18 -08:00
Michael Stack 1ee8776273 HBASE-16785 We are not running all tests Do nothing patch just to get baseline of how many tests we run 2017-01-26 12:21:00 -08:00
Guanghao Zhang 8aea84d723 HBASE-17500 Implement getTable/creatTable/deleteTable/truncateTable methods 2017-01-25 11:30:11 +08:00
zhangduo 85d701892e HBASE-17045 Unify the implementation of small scan and regular scan 2017-01-25 09:53:06 +08:00
Guanghao Zhang 843ba9e545 HBASE-17443 Move listReplicated/enableTableRep/disableTableRep methods from ReplicationAdmin to Admin 2017-01-24 13:40:29 +08:00
anastas aa5d9a9ad3 HBASE-17492 Fix the compacting memstore part in hbase shell ruby script
Signed-off-by: Michael Stack <stack@apache.org>
2017-01-23 21:27:44 -08:00
Yu Li ba4a926b62 HBASE-17367 Make HTable#getBufferedMutator thread safe 2017-01-23 18:23:24 +08:00
Yu Li 07e0a30efa HBASE-17491 Remove all setters from HTable interface and introduce a TableBuilder to build Table instance 2017-01-23 13:57:01 +08:00
zhangduo 3abd13dacb HBASE-17489 ClientScanner may send a next request to a RegionScanner which has been exhausted 2017-01-22 16:39:00 +08:00
Jan Hentschel 55a1aa1e73 HBASE-10699 Set capacity on ArrayList where possible and use isEmpty instead of size() == 0
Signed-off-by: Michael Stack <stack@apache.org>
2017-01-20 22:58:20 -08:00
Guanghao Zhang 79018056f5 HBASE-17498 Implement listTables and listTableNames methods
Signed-off-by: Michael Stack <stack@apache.org>
2017-01-20 14:12:13 -08:00
Guanghao Zhang 2ee3c73f76 HBASE-17497 Add first async MetaTableAccessor impl and Implement tableExists method
Signed-off-by: Michael Stack <stack@apache.org>
2017-01-20 09:24:06 -08:00
Guanghao Zhang cb9ce2ceaf HBASE-17396 Add first async admin impl and implement balance methods 2017-01-19 10:15:12 +08:00
Michael Stack 8f1d0a2b84 HBASE-17486 Tighten the contract for batch client methods (Michael Axiak) 2017-01-18 15:16:52 -08:00
Stephen Yuan Jiang 805d39fca6 HBASE-17470 Remove merge region code from region server (Stephen Yuan Jiang) 2017-01-17 15:39:51 -08:00
tedyu faa9f735ca HBASE-17469 Properly handle empty TableName in TablePermission#readFields and #write (Manjunath Anand) 2017-01-17 09:36:59 -08:00
Allan Yang 9b38c1a33c HBASE-17475 Stack overflow in AsyncProcess if retry too much
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-01-17 15:45:48 +08:00
zhangduo 4ab95ebbce HBASE-17372 Make AsyncTable thread safe 2017-01-17 14:33:28 +08:00
Michael Stack 4cb09a494c HBASE-16744 Procedure V2 - Lock procedures to allow clients to acquire
locks on tables/namespaces/regions (Matteo Bertozzi)

Incorporates review comments from
    https://reviews.apache.org/r/52589/
    https://reviews.apache.org/r/54388/

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTableBase.java
 Fix for eclipse complaint (from Duo Zhang)

M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java
 Log formatting

M hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java
 Added wait procedures utility.

A hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/LockServiceProtos.java
A hbase-protocol-shaded/src/main/protobuf/LockService.proto b/hbase-protocol-shaded/src/main/protobuf/LockService.proto
 Implement new locking CP overrides.

A hbase-server/src/main/java/org/apache/hadoop/hbase/client/locking/EntityLock.java
 New hbase entity lock (ns, table, or regions)

A hbase-server/src/main/java/org/apache/hadoop/hbase/client/locking/LockServiceClient.java
 Client that can use the new internal locking service.
2017-01-13 21:07:03 -08:00
tedyu cdee1a7034 HBASE-17450 TablePermission#equals throws NPE after namespace support was added (huzheng) 2017-01-13 09:37:08 -08:00
Guanghao Zhang ac3b1c9aa9 HBASE-17337 list replication peers request should be routed through master 2017-01-10 08:57:26 +08:00
Yu Li f92a14ade6 HBASE-14061 Support CF-level Storage Policy 2017-01-09 09:52:58 +08:00
tedyu b2a9be02ac HBASE-17408 Introduce per request limit by number of mutations (ChiaPing Tsai) 2017-01-06 13:07:15 -08:00
tedyu 4c98f97c31 HBASE-17431 Incorrect precheck condition in RoundRobinPool#get() - revert due to test failure 2017-01-06 12:12:49 -08:00
tedyu 6d48eb06c9 HBASE-17431 Incorrect precheck condition in RoundRobinPool#get() (Jan Hentschel) 2017-01-06 09:07:02 -08:00
Ashish Singhi 5f631b9653 HBASE-17290 Potential loss of data for replication of bulk loaded hfiles 2017-01-06 16:18:20 +05:30
Guanghao Zhang e02ae7724d HBASE-17388 Move ReplicationPeer and other replication related PB messages to the replication.proto 2017-01-06 10:01:22 +08:00
Jan Hentschel df98d8dcd7 HBASE-17410 Changed size() == 0 to isEmpty in hbase-client
Signed-off-by: Josh Elser <elserj@apache.org>
2017-01-05 11:45:18 -05:00
Jan Hentschel dba103e1b6 HBASE-17390 Added master and backup masters to online update of configuration
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-01-04 22:08:05 -08:00
tedyu 63bd8be576 HBASE-17403 ClientAsyncPrefetchScanner doesn’t load any data if the MaxResultSize is too small (ChiaPing Tsai) 2017-01-04 08:21:25 -08:00
Guanghao Zhang 0e48665641 HBASE-17336 get/update replication peer config requests should be routed through master 2016-12-30 10:12:47 +08:00
Jan Hentschel 7572e96e3a HBASE-17385 Change usage documentation from bin/hbase to hbase in various tools
Signed-off-by: Enis Soztutar <enis@apache.org>
2016-12-29 15:20:43 -08:00
zhangduo 05b1d918b0 HBASE-17320 Add inclusive/exclusive support for startRow and endRow of scan 2016-12-29 09:43:31 +08:00
ChiaPing Tsai 5ffbd4a87d HBASE-17174 Addendum fix typo
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-12-27 13:57:51 +08:00
tedyu e18e9a22da HBASE-17376 ClientAsyncPrefetchScanner may fail due to too many rows (ChiaPing Tsai) 2016-12-26 15:55:22 -08:00
zhangduo 8fa5b0b946 HBASE-17345 Implement batch 2016-12-25 20:36:52 +08:00
Guanghao Zhang 8da7366fc2 HBASE-17348 Remove the unused hbase.replication from javadoc/comment/book completely 2016-12-25 08:46:29 +08:00
ChiaPing Tsai 8cb55c4080 HBASE-17174 Refactor the AsyncProcess, BufferedMutatorImpl, and HTable
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-12-24 12:02:05 +08:00
Jerry He 992e5717d4 HBASE-16010 Put draining function through Admin API (Matt Warhaftig) 2016-12-23 13:41:36 -08:00
Guanghao Zhang b3f2bec099 HBASE-17335 enable/disable replication peer requests should be routed through master 2016-12-23 09:27:12 +08:00
zhangduo 09bb428763 HBASE-17334 Add locate row before/after support for AsyncRegionLocator 2016-12-22 20:39:01 +08:00
Guanghao Zhang e1f4aaeacd HBASE-11392 add/remove peer requests should be routed through master 2016-12-21 13:27:13 +08:00
zhangduo a2e967d92f HBASE-17142 Implement multi get 2016-12-20 20:41:55 +08:00
zhangduo f041306cda HBASE-17282 Reduce the redundant requests to meta table 2016-12-18 08:52:06 +08:00
Michael Stack a5ee36d937 HBASE-17326 Fix findbugs warning in BufferedMutatorParams Fix above and
the other hbase-server findbugs complaint about null-check
Addendum to fix complaint out of BMParams#clone. Added CN_IDIOM_NO_SUPER_CALL
handling.
2016-12-16 21:37:54 -08:00
Michael Stack c3ce02d592 HBASE-17326 Fix findbugs warning in BufferedMutatorParams Fix above and the other hbase-server findbugs complaint about null-check 2016-12-16 13:47:58 -08:00
Michael Stack a5a48dbd23 HBASE-17313 Add BufferedMutatorParams#clone method (Joep Rottinghuis); AMENDMENT -- ADD MISSING FIELD 2016-12-14 08:15:47 -08:00
Michael Stack a73b0b3e6d HBASE-17313 Add BufferedMutatorParams#clone method (Joep Rottinghuis) 2016-12-13 22:47:41 -08:00
Michael Stack 68ce3f1e3b HBASE-17277 Allow alternate BufferedMutator implemenation Specify the name of an alternate BufferedMutator implementation by either:
+ Setting "hbase.client.bufferedmutator.classname" to the name of the
alternate implementation class
+ Or, by setting implementationClassName on BufferedMutatorParams and
passing the amended BufferedMutatorParams when calling Connection#getBufferedMutator.

Add a test to exercise both means.
2016-12-13 22:38:58 -08:00
eshcar a9310436d5 HBASE-17294: External configuration for memory compaction
Signed-off-by: Michael Stack <stack@apache.org>
2016-12-13 17:16:00 -08:00
tedyu 2333596279 HBASE-17296 Provide per peer throttling for replication (Guanghao Zhang) 2016-12-13 04:20:20 -08:00
tedyu adb319f5c2 HBASE-17297 Single Filter in parenthesis cannot be parsed correctly (Xuesen Liang) 2016-12-12 12:07:34 -08:00
zhangduo b3ae87bd7d HBASE-17251 Add a timeout parameter when locating region 2016-12-08 09:55:29 +08:00
anoopsamjohn 61220e4d7c HBASE-17249 Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange reference as the parameter instead of creating a new setColumnFamilyTimeRange instance. (huaxiang sun) 2016-12-07 12:22:41 +05:30
anoopsamjohn f112427425 HBASE-17207 Arrays.asList() with too few arguments. (John Leach) 2016-12-06 14:42:14 +05:30
tedyu 1c8822ddff HBASE-17170 HBase is also retrying DoNotRetryIOException because of class loader differences (Ankit Singhal) 2016-12-05 10:15:09 -08:00
anoopsamjohn e10baacd3e HBASE-17245 Replace HTableDescriptor#htd.getColumnFamilies().length with a low-cost implementation. (huaxiang sun) 2016-12-05 13:28:57 +05:30