Commit Graph

13975 Commits

Author SHA1 Message Date
Michael Stack 6f44b24860 HBASE-18551 [AMv2] UnassignProcedure and crashed regionservers
If an unassign is unable to communicate with its target server,
expire the server and then wait on a signal from ServerCrashProcedure
before proceeding. The unassign has lock on the region so no one else
can proceed till we complete. We prevent any subsequent assign from
running until logs have been split for crashed server.

In AssignProcedure, do not assign if table is DISABLING or DISABLED.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
 Change remoteCallFailed so it returns boolean on whether implementor
wants to stay suspended or not.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
  Doc. Also, if we are unable to talk to remote server, expire it and
then wait on SCP to wake us up after it has processed logs for failed
server.
2017-08-11 07:16:33 -07:00
Guanghao Zhang cabdbf181a HBASE-18500 Performance issue: Don't use BufferedMutator for HTable's put method 2017-08-11 16:33:35 +08:00
Yu Li 679f34e881 HBASE-18469 Correct RegionServer metric of totalRequestCount 2017-08-11 14:13:18 +08:00
Tamas Penzes 8da77b4146 HBASE-18387: [Thrift] Make principal configurable in DemoClient.java
Added optional (fourth) parameter "server-principal"
The solution is backward compatible, in case not given, uses "hbase" as default value
If the third parameter is skipped the fourth cannot be set.

Signed-off-by: Josh Elser <elserj@apache.org>
2017-08-10 23:47:07 -04:00
Guanghao Zhang 4dd24c52b8 HBASE-17125 Inconsistent result when use filter to read data 2017-08-11 10:58:00 +08:00
Esteban Gutierrez c37432fefb HBASE-18563 Fix RAT License complaint about website jenkins scripts 2017-08-10 20:08:03 -05:00
Esteban Gutierrez efd211debd HBASE-18024 HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists 2017-08-10 17:56:17 -05:00
dongtao.zhang 5507150a16 HBASE-18262 name of parameter quote need update
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-10 15:01:37 -07:00
Michael Stack e4ba404a5a Revert "HBASE-18551 [AMv2] UnassignProcedure and crashed regionservers"
This reverts commit 2dd75d10f8.
2017-08-10 14:59:52 -07:00
Michael Stack b65f119c78 Revert "name of parameter quote need update"
This reverts commit 14e3471f34.
2017-08-10 14:59:45 -07:00
Umesh Agashe e98b38bf6c HBASE-18560 Fixed master.assignment.TestAssignmentManager hangs on master and it shows up in flaky list 2017-08-10 14:58:52 -07:00
dongtao.zhang 14e3471f34 name of parameter quote need update
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-10 14:58:11 -07:00
Michael Stack 2dd75d10f8 HBASE-18551 [AMv2] UnassignProcedure and crashed regionservers
If an unassign is unable to communicate with its target server,
expire the server and then wait on a signal from ServerCrashProcedure
before proceeding. The unassign has lock on the region so no one else
can proceed till we complete. We prevent any subsequent assign from
running until logs have been split for crashed server.

In AssignProcedure, do not assign if table is DISABLING or DISABLED.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
 Change remoteCallFailed so it returns boolean on whether implementor
wants to stay suspended or not.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
  Doc. Also, if we are unable to talk to remote server, expire it and
then wait on SCP to wake us up after it has processed logs for failed
server.
2017-08-10 14:53:35 -07:00
Misty Stanley-Jones 6114824b53 HBASE-18548 Move sources of website gen and check jobs into source control 2017-08-10 14:48:14 -07:00
Ashu Pachauri ded0842caf HBASE-18398: Snapshot operation fails with FileNotFoundException 2017-08-10 14:20:08 -07:00
Umesh Agashe d5f34adcdb HBASE-18543 Disabled test TestMasterFailover
This test as it is written currently will not work with AMv2. This needs to be re-written after HBASE-18511 is committed. Disabled the test and update JIRA to re-enable it with dependency on HBASE-18511.

Signed-off-by: Michael Stack <stack@apache.org>
2017-08-10 11:01:27 -07:00
zhangduo 624652373e HBASE-18489 Expose scan cursor in RawScanResultConsumer 2017-08-10 10:11:40 +08:00
Andrew Purtell d0941127d4 HBASE-18248 Warn if monitored RPC task has been tied up beyond a configurable threshold 2017-08-09 18:16:38 -07:00
Michael Stack 794a3b104b HBASE-18545 Fix broke site build that fails in rsgroups 2017-08-09 11:24:51 -07:00
Umesh Agashe 67eddf5874 HBASE-18525 [AMv2] Fixed test TestAssignmentManager#testSocketTimeout on master branch 2017-08-09 10:15:37 -07:00
Umesh Agashe f314b5911b HBASE-18492 [AMv2] Embed code for selecting highest versioned region server for system table regions in AssignmentManager.processAssignQueue()
* Modified AssignmentManager.processAssignQueue() method to consider only highest versioned region servers for system table regions when
  destination server is not specified for them. Destination server is retained, if specified.
* Modified MoveRegionProcedure to allow null value for destination server i.e. moving a region from specific source server to non-specific/ unknown
  destination server (picked by load-balancer) is supported now.
* Removed destination server selection from HMaster.checkIfShouldMoveSystemRegionAsync(), as destination server will be picked by load balancer

Signed-off-by: Michael Stack <stack@apache.org>
2017-08-08 14:02:11 -07:00
Michael Stack 03390684cc Revert "HBASE-18511 Default no regions on master"
This reverts commit a8e0267c00.
2017-08-08 13:37:56 +08:00
Vikas Vishwakarma b653d1bf05 HBASE-18517 limit max log message width in log4j
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-08 12:11:13 +08:00
Michael Stack a8e0267c00 HBASE-18511 Default no regions on master 2017-08-08 12:11:02 +08:00
Chia-Ping Tsai 4376233e2a HBASE-18315 Eliminate the findbugs warnings for hbase-rest 2017-08-08 10:47:36 +08:00
Sean Busbey 7e7461e42a HBASE-18527 Update nightly job to account for changes in jenkins plugins.
- Stop presuming job will be configured with checkout into a 'component' dir
 - Disable autocheckout
 - Specify checkout within 'component' dir
 - leave the workspace around on pipeline failure so we can browse it.
2017-08-07 11:24:00 -05:00
Chia-Ping Tsai fd76eb39d7 HBASE-18502 Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor 2017-08-07 11:26:15 +08:00
no_apologies a7014ce46c HBASE-18515 Introduce Delete.add as a replacement for Delete#addDeleteMarker
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-07 11:05:23 +08:00
Guanghao Zhang 5915d73a70 HBASE-18485 Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner 2017-08-07 10:35:19 +08:00
Sean Busbey 2a717459bd HBASE-14220 nightly check that we can build a source tarball.
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-06 12:01:47 -05:00
Sean Busbey 04424ad65f HBASE-18426 nightly job should use independent stages to check supported jdks
* Jenkinsfile that works for all current branches.
* adds dev-support script for setting parameters for our yetus nightly invocation
* quiets the "zip test results" step

Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-06 11:59:45 -05:00
Zach York 637f7abf0b HBASE-18520 Add jmx value to determine true Master Start time
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-05 22:32:33 -07:00
Jan Hentschel 7da47509d2 HBASE-15042 Moved site materials to standard Maven place
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-05 15:22:46 +08:00
Umesh Agashe 62deb8172e HBASE-18516 Removed dead code in ServerManager resulted mostly from AMv2 refactoring
* Call to methods sendRegionOpen(), isServerReachable(), removeRequeuedDeadServers(), getRequeuedDeadServers() got removed in HBASE-14614
* Call to method ServerManager.sendFavoredNodes() got removed in HBASE-17198
2017-08-04 13:47:59 -07:00
Alex Leblang 2bde7a1077 HBASE-18020 Update API Compliance Checker to Incorporate Improvements Done in Hadoop
- Converted to python
- Updated to most recent release of Java ACC
- Added ability to include known number of problems

Author: Andrew Wang
Amending-Author: Alex Leblang
Amending-Author: Sean Busbey

Signed-off-by: Dima Spivak <dimaspivak@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-08-04 12:05:10 -05:00
Mike Drob 6266bb352b Add Mike Drob to pom.xml 2017-08-03 15:51:34 -05:00
Benedict Jin fe890b70ac HBASE-18470 Remove the redundant comma from RetriesExhaustedWithDetailsException#getDesc
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-03 15:42:53 +08:00
Robert Yokota 855dd48f0a HBASE-16893 Use Collection.removeIf instead of Iterator.remove in DependentColumnFilter
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-03 13:38:52 +08:00
Apekshit Sharma 504a1f14e3 HBASE-18102 (addendum fixing shell tests) - Purge close_region command that allows by-pass of Master
Change-Id: I753f20d50954969ee473f6a016d5c5f9ff4fd631
2017-08-02 18:11:48 -07:00
Apekshit Sharma 71151eb0e9 HBASE-18102 Purge close_region command that allows by-pass of Master
Change-Id: I67e3f74e16706043056bac73bc1ff3a713d0e977
2017-08-02 17:09:32 -07:00
Apekshit Sharma de696cf6b6 HBASE-18231 Deprecate Admin#closeRegion*() commands in favor of Admin#unassign().
Other changes:
- Update corresponding tests in TestAdmin2. Removed tests centered around serverName part of old functions.
- Remove dead functions from ProtobufUtil and ServerManager
- Rename closeRegion* functions in HBTU to unassignRegion*

Change-Id: Ib9bdeb185e10750daf652be0bb328306accb73ab
2017-08-02 15:19:06 -07:00
Michael Stack 7a6de1bd42 HBASE-17056 Remove checked in PB generated files
Selective add of dependency on hbase-thirdparty jars.
Update to READMEs on how protobuf is done (and update to refguide).
Removed all checked in generated protobuf files. They are generated
on the fly now as part of mainline build.
2017-08-02 09:33:20 -07:00
Konstantin Ryakhovskiy d4bd99312d HBASE-16116 Remove redundant pattern *.iml
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-03 00:22:45 +08:00
Chia-Ping Tsai f260f09865 HBASE-18480 The cost of BaseLoadBalancer.cluster is changed even if the rollback is done 2017-08-02 08:48:01 +08:00
Umesh Agashe ba5e8706de HBASE-18491 [AMv2] Fail UnassignProcedure if source Region Server is not online.
The patch also enables TestServerCrashProcedure.testRecoveryAndDoubleExecutionOnRsWithMeta()

Signed-off-by: Michael Stack <stack@apache.org>
2017-08-01 17:05:00 -07:00
James Taylor 422a57223a HBASE-18487 Minor fixes in row lock implementation
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-08-01 11:28:02 -07:00
Umesh Agashe a5db120e60 HBASE-18261 Created RecoverMetaProcedure and used it from ServerCrashProcedure and HMaster.finishActiveMasterInitialization().
This procedure can be used from any code before accessing meta, to initialize/ recover meta

Signed-off-by: Michael Stack <stack@apache.org>
2017-07-31 14:25:03 -07:00
Sean Busbey 331a6cface HBASE-18475 ensure only non-null procedures are sent to requireTableExclusiveLock
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-31 11:05:16 -05:00
Peter Somogyi 4d9982f33d HBASE-18259 HBase book link to "beginner" issues includes resolved issues - ADDENDUM
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-07-31 23:36:19 +08:00
Abhishek Singh Chouhan 95808b4672 HBASE-18374 RegionServer Metrics improvements 2017-07-31 12:42:41 +05:30