Commit Graph

13872 Commits

Author SHA1 Message Date
Umesh Agashe 7a00571795 HBASE-18492 [AMv2] Embed code for selecting highest versioned region server for system table regions in AssignmentManager.processAssignQueue()
* Modified AssignmentManager.processAssignQueue() method to consider only highest versioned region servers for system table regions when
  destination server is not specified for them. Destination server is retained, if specified.
* Modified MoveRegionProcedure to allow null value for destination server i.e. moving a region from specific source server to non-specific/ unknown
  destination server (picked by load-balancer) is supported now.
* Removed destination server selection from HMaster.checkIfShouldMoveSystemRegionAsync(), as destination server will be picked by load balancer

Signed-off-by: Michael Stack <stack@apache.org>
2017-08-08 14:02:36 -07:00
Vikas Vishwakarma 40cfb2aa89 HBASE-18517 limit max log message width in log4j
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-08 12:10:13 +08:00
Chia-Ping Tsai 18d29e4794 HBASE-18315 Eliminate the findbugs warnings for hbase-rest 2017-08-08 10:47:15 +08:00
Josh Elser 501176d40d HBASE-18244 Temporarily disable TestShellRSGroups
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-07 16:57:10 -04:00
Josh Elser f262548b1f HBASE-18514 Squash of space quota "phase 2" work
Contains the following commits:

  HBASE-17748 Include HBase snapshots in space quotas

  Introduces a new Chore in the Master which computes the size
  of the snapshots included in a cluster. The size of these
  snapshots are included in the table's which the snapshot was created
  from HDFS usage.

  Includes some test stabilization, trying to make the tests more
  deterministic by ensuring we observe stable values as we know
  that those values are mutable. This should help avoid problems
  where size reports are delayed and we see an incomplete value.

  HBASE-17752 Shell command to list snapshot sizes WRT quotas

  HBASE-17840 Update hbase book to space quotas on snapshots
2017-08-07 16:32:16 -04:00
Sean Busbey 7029c23b00 HBASE-18527 Update nightly job to account for changes in jenkins plugins.
- Stop presuming job will be configured with checkout into a 'component' dir
 - Disable autocheckout
 - Specify checkout within 'component' dir
 - leave the workspace around on pipeline failure so we can browse it.
2017-08-07 11:25:08 -05:00
Chia-Ping Tsai afd5866de0 HBASE-18502 Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor 2017-08-07 11:25:45 +08:00
no_apologies 2c067695dd HBASE-18515 Introduce Delete.add as a replacement for Delete#addDeleteMarker
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-07 11:13:12 +08:00
Guanghao Zhang be034c26b4 HBASE-18485 Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner 2017-08-07 10:43:03 +08:00
Sean Busbey 69fddb58bd HBASE-14220 nightly check that we can build a source tarball.
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-06 12:08:51 -05:00
Sean Busbey db18a00b76 HBASE-18426 nightly job should use independent stages to check supported jdks
* Jenkinsfile that works for all current branches.
* adds dev-support script for setting parameters for our yetus nightly invocation
* quiets the "zip test results" step

Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-06 12:08:48 -05:00
Sean Busbey 3c8b364379 HBASE-18147 POC jenkinsfile for nightly checks.
* adds ruby tools to dockerfile
* adds rubocop to dockerfile
* adds ruby-lint to dockerfile
* adds perlcritic to dockerfile

Signed-off-by: Alex Leblang <awleblang@gmail.com>
Signed-off-by: Josh Elser <elserj@apache.org>
2017-08-06 12:08:34 -05:00
Zach York 0f20ec395a HBASE-18520 Add jmx value to determine true Master Start time
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-05 22:33:11 -07:00
Umesh Agashe 55a754e6cb HBASE-18516 Removed dead code in ServerManager resulted mostly from AMv2 refactoring
* Call to methods sendRegionOpen(), isServerReachable(), removeRequeuedDeadServers(), getRequeuedDeadServers() got removed in HBASE-14614
* Call to method ServerManager.sendFavoredNodes() got removed in HBASE-17198
2017-08-04 13:57:17 -07:00
Alex Leblang 2b1b01fce3 HBASE-18020 Update API Compliance Checker to Incorporate Improvements Done in Hadoop
- Converted to python
- Updated to most recent release of Java ACC
- Added ability to include known number of problems

Author: Andrew Wang
Amending-Author: Alex Leblang
Amending-Author: Sean Busbey

Signed-off-by: Dima Spivak <dimaspivak@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-08-04 12:06:45 -05:00
Benedict Jin ee6ff31c0d HBASE-18470 Remove the redundant comma from RetriesExhaustedWithDetailsException#getDesc
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-03 15:43:20 +08:00
Robert Yokota da2cb02422 HBASE-16893 Use Collection.removeIf instead of Iterator.remove in DependentColumnFilter
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-03 13:43:51 +08:00
Apekshit Sharma bbe0e7684a HBASE-18102 (addendum fixing shell tests) - Purge close_region command that allows by-pass of Master
Change-Id: I753f20d50954969ee473f6a016d5c5f9ff4fd631
2017-08-02 18:12:23 -07:00
Apekshit Sharma 3a8d6b9131 HBASE-18102 Purge close_region command that allows by-pass of Master
Change-Id: I67e3f74e16706043056bac73bc1ff3a713d0e977
2017-08-02 17:14:44 -07:00
Apekshit Sharma 41da64a144 HBASE-18231 Deprecate Admin#closeRegion*() commands in favor of Admin#unassign().
Other changes:
- Update corresponding tests in TestAdmin2. Removed tests centered around serverName part of old functions.
- Remove dead functions from ProtobufUtil and ServerManager
- Rename closeRegion* functions in HBTU to unassignRegion*

Change-Id: Ib9bdeb185e10750daf652be0bb328306accb73ab
2017-08-02 15:22:17 -07:00
Michael Stack 6440509d7d HBASE-17056 Remove checked in PB generated files; ADDENDUM 2017-08-02 09:54:03 -07:00
Michael Stack ee70b1d2e0 HBASE-17056 Remove checked in PB generated files
Selective add of dependency on hbase-thirdparty jars.
Update to READMEs on how protobuf is done (and update to refguide).
Removed all checked in generated protobuf files. They are generated
on the fly now as part of mainline build.
2017-08-02 09:42:38 -07:00
Konstantin Ryakhovskiy f923342998 HBASE-16116 Remove redundant pattern *.iml
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-03 00:23:37 +08:00
Chia-Ping Tsai b4f8377aa0 HBASE-18480 The cost of BaseLoadBalancer.cluster is changed even if the rollback is done 2017-08-02 08:43:04 +08:00
Umesh Agashe 3f540f3801 HBASE-18491 [AMv2] Fail UnassignProcedure if source Region Server is not online.
The patch also enables TestServerCrashProcedure.testRecoveryAndDoubleExecutionOnRsWithMeta()

Signed-off-by: Michael Stack <stack@apache.org>
2017-08-01 17:05:22 -07:00
Andrew Purtell 5e21e30682 HBASE-18431 Mitigate compatibility concerns between branch-1.3 and branch-1.4
We needed to move TableName out of HBase.proto to resolve a cycle after
repairing incompatible changes introduced into branch-1. Carry this forward.
2017-08-01 17:05:14 -07:00
James Taylor f67dae90f9 HBASE-18487 Minor fixes in row lock implementation
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-08-01 12:16:43 -07:00
Umesh Agashe 7bdabed275 HBASE-18261 Created RecoverMetaProcedure and used it from ServerCrashProcedure and HMaster.finishActiveMasterInitialization().
This procedure can be used from any code before accessing meta, to initialize/ recover meta

Signed-off-by: Michael Stack <stack@apache.org>
2017-07-31 14:25:38 -07:00
Sean Busbey 5490c558bc HBASE-18475 ensure only non-null procedures are sent to requireTableExclusiveLock
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-31 11:06:57 -05:00
Abhishek Singh Chouhan 08b5a4b2c6 HBASE-18374 RegionServer Metrics improvements 2017-07-31 12:44:27 +05:30
Guanghao Zhang 8fc010636a HBASE-18481 The autoFlush flag was not used in PE tool 2017-07-31 11:04:09 +08:00
Yi Liang 928137c1ce HBASE-18465: [AMv2] remove old split region code that is no longer needed
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-30 15:25:40 -05:00
Alex Leblang dbc8ea6486 HBASE-18406 Remove NO-OP Method
This patch removes start(MasterProcedureEnv) from
ServerCrashProcedure.java which was a misnomer as a no-op. It
did not start anything.

Change-Id: I4e91864ace912e137471bfce03516746c4aff83e

Signed-off-by: Michael Stack <stack@apache.org>
2017-07-30 14:01:13 +01:00
anoopsamjohn d776a3caae HBASE-18473 VC.listLabels() erroneously closes any connection. 2017-07-30 15:38:18 +05:30
Abhishek Singh Chouhan 12b9a151e6 HBASE-15134 Add visibility into Flush and Compaction queues 2017-07-28 13:04:52 +05:30
Balazs Meszaros 34a1ae875c HBASE-18185 IntegrationTestTimeBoundedRequestsWithRegionReplicas unbalanced tests fails with AssertionError
unbalance.kill.meta.rs property was added which controls the monkey to
kill that region server which holds hbase:meta.

Change-Id: I049d05cc13a78ba868c917879f67ef4f34fffbf9
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-07-27 15:01:53 -07:00
Esteban Gutierrez 75789048c1 HBASE-18362 hbck should not report split replica parent region from meta as errors (Huaxiang Sun)
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
2017-07-27 16:04:45 -05:00
Mike Drob 10573c6c99 HBASE-18445 exclude guava from hadoop-auth
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-07-26 15:49:17 -07:00
huzheng 311ca34aec HBASE-18402 Thrift2 should support DeleteFamilyVersion type
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-07-26 15:49:17 -07:00
tedyu b7045d4fcf HBASE-18402 Thrift2 should support DeleteFamilyVersion - revert due to missing JIRA number 2017-07-26 15:49:16 -07:00
Chia-Ping Tsai 164df6bdd9 HBASE-18449 Fix client.locking.TestEntityLocks 2017-07-26 20:29:14 +08:00
Peter Somogyi fb80a8e886 HBASE-18447 MetricRegistryInfo#hashCode uses hashCode instead of toHashCode
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-07-26 16:18:35 +08:00
Yu Li 3be51a85ba HBASE-18441 ZookeeperWatcher#interruptedException should throw exception 2017-07-26 15:21:25 +08:00
Malcolm Taylor 7517f93265 HBASE-18434 Address some alerts raised by lgtm.com
Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2017-07-26 10:03:07 +05:30
Andrew Purtell e24f7f4675 HBASE-18054 log when we add/remove failed servers in client (Ali) 2017-07-25 18:53:08 -07:00
Umesh Agashe a5dbaaa56c HBASE-18427 minor cleanup around AssignmentManager
- unused imports
- superfluous exception in method definitions

Change-Id: I156383b9895fa718fe9d5227003c23bd945cf999
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-07-25 17:52:08 -07:00
huzheng 6abc8bd58e HBASE-18402 Thrift2 should support DeleteFamilyVersion type
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-07-25 17:40:19 -07:00
tedyu edc36b3f90 HBASE-18402 Thrift2 should support DeleteFamilyVersion - revert due to missing JIRA number 2017-07-25 17:40:06 -07:00
Josh Elser 50d9301136 HBASE-18023 Update row threshold warning from 1k to 5k (addendum) 2017-07-25 18:35:12 -04:00
张世彬10204932 8ad76bbc88 HBASE-18323 Remove multiple ACLs for the same user in kerberos
Signed-off-by: Josh Elser <elserj@apache.org>
2017-07-25 18:17:02 -04:00