Commit Graph

329 Commits

Author SHA1 Message Date
Michael Stack c5d6e166de HBASE-17569 HBase-Procedure module need to support mvn clean test -PskipProcedureTests to skip unit test (Yi Liang)
ADDENDUM Fix other places that are missing the skip command
2017-01-31 22:48:34 -08:00
zhangduo 85d701892e HBASE-17045 Unify the implementation of small scan and regular scan 2017-01-25 09:53:06 +08:00
Michael Stack 4fdd6ff9ae HBASE-16831 Procedure V2 - Remove org.apache.hadoop.hbase.zookeeper.lock
(Appy)
2017-01-19 16:51:44 -08:00
zhangduo 05b1d918b0 HBASE-17320 Add inclusive/exclusive support for startRow and endRow of scan 2016-12-29 09:43:31 +08:00
Guanghao Zhang 8da7366fc2 HBASE-17348 Remove the unused hbase.replication from javadoc/comment/book completely 2016-12-25 08:46:29 +08:00
tedyu 2333596279 HBASE-17296 Provide per peer throttling for replication (Guanghao Zhang) 2016-12-13 04:20:20 -08:00
zhangduo 890fcbd0e6 HBASE-17167 Pass mvcc to client when scan 2016-11-30 10:11:04 +08:00
Michael Stack ad0e862f78 "HBASE-16952 Replace hadoop-maven-plugins with protobuf-maven-plugin for building protos""
Rely on the new plugin to do all proto generation. No need of an
external protoc setup anymore. Mvn will do it all for you.

Updated all READMEs appropriately.

Signed-off-by: Michael Stack <stack@apache.org>
2016-10-28 16:49:35 -07:00
Michael Stack 738ff821dd Revert "HBASE-16952 Replace hadoop-maven-plugins with protobuf-maven-plugin for building protos"
Revert premature commit

This reverts commit d0e61b0e9a.
2016-10-27 13:17:07 -07:00
Michael Stack d0e61b0e9a HBASE-16952 Replace hadoop-maven-plugins with protobuf-maven-plugin for building protos
This patch changes poms to use protobuf-maven-plugin instaed of
hadoop-maven-plugins generating protos. Adds a few missing READMEs too
as well as purge of unused protos turned up by the new plugin.
2016-10-27 10:11:58 -07:00
Michael Stack c776b3144d HBASE-16949 Fix RAT License complaint about the hbase-protocol-shaded/src/main/patches content 2016-10-26 14:52:47 -07:00
Michael Stack 4533bb63cf Revert "Revert "HBASE-15789 PB related changes to work with offheap""
Restore this change but with a clean of the generated dirs first so
that we avoid trying to apply a patch on top of an already patched src.

This reverts commit 0f384158fc.
2016-10-24 16:51:50 -07:00
Michael Stack a46134bffc HBASE-16731 Inconsistent results from the Get/Scan if we use the empty FilterList (ChiaPing Tsai) 2016-10-12 12:13:41 -07:00
stack 95c1dc93fb HBASE-15638 Shade protobuf
Which includes

    HBASE-16742 Add chapter for devs on how we do protobufs going forward

    HBASE-16741 Amend the generate protobufs out-of-band build step
    to include shade, pulling in protobuf source and a hook for patching protobuf

    Removed ByteStringer from hbase-protocol-shaded. Use the protobuf-3.1.0
    trick directly instead. Makes stuff cleaner. All under 'shaded' dir is
    now generated.

    HBASE-16567 Upgrade to protobuf-3.1.x
    Regenerate all protos in this module with protoc3.
    Redo ByteStringer to use new pb3.1.0 unsafebytesutil
    instead of HBaseZeroCopyByteString

    HBASE-16264 Figure how to deal with endpoints and shaded pb Shade our protobufs.
    Do it in a manner that makes it so we can still have in our API references to
    com.google.protobuf (and in REST). The c.g.p in API is for Coprocessor Endpoints (CPEP)

            This patch is Tactic #4 from Shading Doc attached to the referenced issue.
            Figuring an appoach took a while because we have Coprocessor Endpoints
            mixed in with the core of HBase that are tough to untangle (FIX).

            Tactic #4 (the fourth attempt at addressing this issue) is COPY all but
            the CPEP .proto files currently in hbase-protocol to a new module named
            hbase-protocol-shaded. Generate .protos again in the new location and
            then relocate/shade the generated files. Let CPEPs keep on with the
            old references at com.google.protobuf.* and
            org.apache.hadoop.hbase.protobuf.* but change the hbase core so all
            instead refer to the relocated files in their new location at
            org.apache.hadoop.hbase.shaded.com.google.protobuf.*.

            Let the new module also shade protobufs themselves and change hbase
            core to pick up this shaded protobuf rather than directly reference
            com.google.protobuf.

            This approach allows us to explicitly refer to either the shaded or
            non-shaded version of a protobuf class in any particular context (though
            usually context dictates one or the other). Core runs on shaded protobuf.
            CPEPs continue to use whatever is on the classpath with
            com.google.protobuf.* which is pb2.5.0 for the near future at least.

            See above cited doc for follow-ons and downsides. In short, IDEs will complain
            about not being able to find the shaded protobufs since shading happens at package
            time; will fix by checking in all generated classes and relocated protobuf in
            a follow-on. Also, CPEPs currently suffer an extra-copy as marshalled from
            non-shaded to shaded. To fix. Finally, our .protos are duplicated; once
            shaded, and once not. Pain, but how else to reveal our protos to CPEPs or
            C++ client that wants to talk with HBase AND shade protobuf.

            Details:

            Add a new hbase-protocol-shaded module. It is a copy of hbase-protocol
    i       with all relocated offset from o.a.h.h. to o.a.h.h.shaded. The new module
            also includes the relocated pb. It does not include CPEPs. They stay in
            their old location.

            Add another module hbase-endpoint which has in it all the endpoints
            that ship as part of hbase -- at least the ones that are not
            entangled with core such as AccessControl and Auth. Move all protos
            for these CPEPs here as well as their unit tests (mostly moving a
            bunch of stuff out of hbase-server module)

            Much of the change looks like this:

                 -import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
                 -import org.apache.hadoop.hbase.protobuf.generated.ClusterIdProtos;
                 +import org.apache.hadoop.hbase.protobuf.shaded.ProtobufUtil;
                 +import org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterIdProtos;

            In HTable and in HBaseAdmin, regularize the way Callables are used and also hide
            protobuf usage as much as possible moving it up into Callable super classes or out
            to utility classes. Still TODO is adding in of retries, etc., but can wait on
            procedure which will redo all this.

            Also in HTable and HBaseAdmin as well as in HRegionServer and Server, be explicit
            when using non-shaded protobuf. Do the full-path so it is clear. This is around
            endpoint coprocessors registration of services and execution of CPEP methods.

            Shrunk ProtobufUtil by moving methods used by one CPEP only back to the CPEP either
            into Client class or as new Util class; e.g. AccessControlUtil.

            There are actually two versions of ProtobufUtil now; a shaded one and a subset
            that is used by CPEPs doing non-shaded work.

            Made it so hbase-common no longer depends on hbase-protocol (with Matteo's help)

            R*Converter classes got moved down under shaded package -- they are for internal
            use only. There are no non-shaded versions of these classes.

            D hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable
            D RetryingCallableBase
             Not used anymore and we have too many tiers of Callables so removed/cleaned-up.

            A ClientServicecallable
             Had to add this one. RegionServerCallable was made generic so it could be used
             for a few Interfaces (Client and Admin). Then added ClientServiceCallable to
             implement RegionServerCallable with the Client Interface.
2016-10-03 21:37:32 -07:00
tedyu 219c786457 HBASE-16672 Add option for bulk load to copy hfile(s) instead of renaming 2016-09-26 18:56:38 -07:00
Jerry He edc0ef3fe4 HBASE-16598 Enable zookeeper useMulti always and clean up in HBase code 2016-09-17 16:51:26 -07:00
Enis Soztutar 1a1003a482 HBASE-16447 Replication by namespaces config in peer (Guanghao Zhang) 2016-09-16 11:47:42 -07:00
zhangduo 45af3831fe HBASE-16526 Add more ipc tests 2016-08-31 20:44:56 +08:00
Phil Yang 5cadcd59aa HBASE-9465 Push entries to peer clusters serially
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-08-09 15:25:50 +08:00
stack 26c0426689 HBASE-15656 Fix unused protobuf warning in Admin.proto (Yi Liang) 2016-07-29 14:59:22 -07:00
Stephen Yuan Jiang bdd7782f05 HBASE-16008 A robust way deal with early termination of HBCK (Stephen Yuan Jiang) 2016-07-23 14:47:11 -07:00
Stephen Yuan Jiang cd0b85e0a8 Revert HBASE-15406: Split / merge switch left disabled after early termination of hbck (HBASE-16008 will address this issue) 2016-07-22 20:09:03 -07:00
stack 9d740f7b8b HBASE-16263 Move all to do w/ protobuf -- *.proto files and generated classes -- under hbase-protocol
Signed-off-by: stack <stack@apache.org>
2016-07-21 10:02:05 -07:00
Jerry He 70f330dc84 Consolidate SecureBulkLoadEndpoint into HBase core as default for bulk load 2016-07-19 19:53:10 -07:00
Stephen Yuan Jiang f04eeecffc HBASE-14552 Procedure V2: Reimplement DispatchMergingRegionHandler (Stephen Yuan Jiang) 2016-07-15 07:06:53 -07:00
Matteo Bertozzi 8cfaa0e721 HBASE-16092 Procedure v2 - complete child procedure support 2016-07-12 10:23:02 -07:00
Matteo Bertozzi 1d06850f40 Revert "HBASE-16092 Procedure v2 - complete child procedure support"
This reverts commit 96c4054e3b.
2016-06-24 14:35:29 -07:00
Matteo Bertozzi 96c4054e3b HBASE-16092 Procedure v2 - complete child procedure support 2016-06-24 04:38:50 -07:00
Enis Soztutar bdb0cc8808 HBASE-15467 Remove 1.x/2.0 TableDescriptor incompatibility 2016-06-17 17:25:04 -07:00
tedyu 39dc19236e HBASE-15806 revert due to discussion on security 2016-05-24 02:08:11 -07:00
tedyu c03ea895c4 HBASE-15806 An endpoint-based export tool (ChiaPing Tsai) 2016-05-23 10:51:25 -07:00
Phil Yang 3bd9220f0c HBASE-15593 Time limit of scanning should be offered by client
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-05-18 19:13:15 +08:00
zhangduo b2b3b1fa4d Revert "HBASE-15593 Time limit of scanning should be offered by client (Phil Yang)"
This reverts commit e47bfb9078.
2016-05-18 13:08:50 +08:00
stack e47bfb9078 HBASE-15593 Time limit of scanning should be offered by client (Phil Yang) 2016-05-17 12:49:31 -07:00
Elliott Clark 9269b8199e HBASE-15840 WAL.proto compilation broken for cpp 2016-05-17 12:04:14 -07:00
Ashish Singhi 34e9a6ff30 HBASE-15669 HFile size is not considered correctly in a replication request 2016-05-06 17:26:17 +05:30
Enis Soztutar 106fe99a9a HBASE-15670 Add missing Snapshot.proto to the maven profile for compiling protobuf 2016-04-22 11:34:49 -07:00
chenheng a9f1c4a871 HBASE-15406 Split / merge switch left disabled after early termination of hbck 2016-04-15 14:09:52 +08:00
Stephen Yuan Jiang e1d5c3d269 HBASE-15521 Procedure V2 - RestoreSnapshot and CloneSnapshot (Stephen Yuan Jiang) 2016-03-31 21:49:13 -07:00
chenheng 7f39baf0f4 HBASE-11393 Replication TableCfs should be a PB object rather than a string 2016-03-29 10:25:29 +08:00
Elliott Clark 925c185969 HBASE-15475 Allow TimestampsFilter to provide a seek hint
Summary:
Allow TestTimestampFilterSeekHint to provide a seek next hint.
This can be incorrect as it might skip deletes. However it can
make things much much faster.

Test Plan: Added a unit test.

Differential Revision: https://reviews.facebook.net/D55617
2016-03-23 18:56:52 -07:00
Enis Soztutar ca816f0780 HBASE-6721 RegionServer Group based Assignment (Francis Liu) 2016-03-14 18:28:50 -07:00
Jesse Yates ef712df944 HBASE-14703 HTable.mutateRow does not collect stats (Heng Chen) 2016-03-05 11:01:45 -08:00
chenheng 99955a3240 HBASE-15128 Disable region splits and merges switch in master 2016-02-27 08:36:59 +08:00
chenheng bf4fcc30c6 Revert "HBASE-15128 Disable region splits and merges switch in master"
This reverts commit 24d481c580.
2016-02-26 08:52:12 +08:00
chenheng 24d481c580 HBASE-15128 Disable region splits and merges switch in master 2016-02-26 08:11:16 +08:00
chenheng 8f20bc748d HBASE-15197 Expose filtered read requests metric to metrics framework and Web UI (Eungsop Yoo) 2016-02-05 10:57:14 +08:00
stack 46303dfd75 HBASE-14888 ClusterSchema: Add Namespace Operations 2016-01-05 14:35:27 -08:00
Lars Hofhansl 86a417eead HBASE-14822 Renewing leases of scanners doesn't work. 2015-12-19 09:55:03 -08:00
anoopsjohn 408666a06d HBASE-13158 When client supports CellBlock, return the result Cells as controller payload for get(Get) API also. 2015-12-18 09:50:53 +05:30
ramkrishna 26ac60b03f HBASE-13153 Bulk Loaded HFile Replication (Ashish Singhi) 2015-12-10 13:07:46 +05:30
stack 290ecbe829 HBASE-14355 Scan different TimeRange for each column family (Churro Morales) 2015-11-12 10:54:16 -10:00
chenheng 6871724104 HBASE-12986 Compaction pressure based client pushback
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2015-11-03 18:32:08 -08:00
tedyu f91546f2e6 HBASE-14696 Support setting allowPartialResults in mapreduce Mappers 2015-10-27 07:57:32 -07:00
Ashish Singhi 467bc098a9 HBASE-13538 Procedure v2 - client add/delete/modify column family sync (incompatible with branch-1.x)
Signed-off-by: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
2015-10-22 13:12:35 -07:00
tedyu dae1775a11 HBASE-14406 The dataframe datasource filter is wrong, and will result in data loss or unexpected behavior (Ted Malaska) 2015-10-16 11:22:07 -07:00
stack 445dbd8a0e HBASE-14517 Show regionserver's version in master status page (Liu Shaohui) 2015-10-09 15:06:39 -07:00
Mikhail Antonov 19045a5ea7 HBASE-14367 Add normalization support to shell 2015-10-04 00:29:39 -07:00
Stephen Yuan Jiang 5b7894f92b HBASE-14107 Administrative Task: Provide an API to List all procedures (Stephen Yuan Jiang) 2015-09-23 21:48:32 -07:00
Ted Malaska e95358a7fc HBASE-14181 Add Spark DataFrame DataSource to HBase-Spark Module
Signed-off-by: Sean Busbey <busbey@cloudera.com>
2015-09-07 09:44:10 -05:00
Stephen Yuan Jiang 3341f13e71 HBASE-14108 Procedure V2 Administrative Task: provide an API to abort a procedure (Stephen Yuan Jiang) 2015-09-02 19:18:46 -07:00
tedyu df341c4299 HBASE-14309 Allow load balancer to operate when there is region in transition by adding force flag 2015-08-29 13:15:53 -07:00
Stephen Yuan Jiang dc79b3c5c9 HBASE-13212: Procedure V2 - master Create/Modify/Delete namespace (Stephen Yuan Jiang) 2015-08-26 08:18:03 -07:00
Sean Busbey ba7ea0b524 HBASE-14260 skip javadoc generation for the hbase-protocol generated classes 2015-08-20 21:14:06 -05:00
Andrew Purtell 5e5bcceb53 HBASE-14122 Client API for determining if server side supports cell level security 2015-08-12 13:26:55 -07:00
Sean Busbey 1b0b67fb7c HBASE-14085 Update LICENSE and NOTICE files.
* corrects license/notice for source distribution
* adds inception year to correct copyright in generated NOTICE files for jars
* updates project names in poms to use "Apache HBase" instead of "HBase" so jar NOTICE files will be correct
* uses append-resources to include supplemental info on jars with 3rd party works in source
* adds an hbase specific resource bundle for jars that include 3rd party works for binaries
** uses supplemental-model to fill in license gaps
** uses the above and a shade plugin transformation to build proper files for shaded jars.
** uses the above and the assembly plugin to build the proper files for bin assembly
* adds a NOTICE item for things copied out of Hadoop (TODO legal-discuss)
2015-08-04 17:16:56 -05:00
anoopsjohn 3b6db26863 HBASE-13954 Remove HTableInterface#getRowOrBefore related server side code. (Ashish) 2015-07-21 09:08:26 +05:30
Elliott Clark 5315f0f11f HBASE-14094 Procedure.proto can't be compiled to C++ 2015-07-15 15:51:20 -07:00
Elliott Clark 62bce5f903 HBASE-14077 Add package to hbase-protocol protobuf files. 2015-07-15 11:15:58 -07:00
Matteo Bertozzi 951ec7a0b7 HBASE-13415 Procedure v2 - Use nonces for double submits from client (Stephen Yuan Jiang)
Signed-off-by: Sean Busbey <busbey@apache.org>
2015-07-13 11:02:30 -05:00
Sean Busbey 60d6435ad8 Revert "Procedure v2 - Use nonces for double submits from client (Stephen Yuan Jiang)"
This reverts commit 87ee37ceb8.
2015-07-13 11:01:45 -05:00
Matteo Bertozzi 87ee37ceb8 Procedure v2 - Use nonces for double submits from client (Stephen Yuan Jiang) 2015-07-09 07:44:36 -07:00
stack cff1a5f1f5 HBASE-14012 Double Assignment and Dataloss when ServerCrashProcedure
runs during Master failover
2015-07-06 07:57:40 -07:00
Gábor Lipták 5826bf2a7c HBASE-13963 Do not leak jdk.tools dependency from hbase-annotations
Signed-off-by: Sean Busbey <busbey@apache.org>
2015-06-26 08:48:02 -05:00
Nick Dimiduk 3e72dc9f08 HBASE-13938 Deletes done during the region merge transaction may get eclipsed (ddas, enis, ndimiduk) 2015-06-23 10:23:38 -07:00
Gábor Lipták 6a537eb854 HBASE-13900 Use ProtobufMagic methods in ProtobufUtil
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2015-06-23 00:32:23 -05:00
Sean Busbey abc2e61eb1 Revert "Use ProtobufMagic methods in ProtobufUtil"
This reverts commit 6b37f26280.
2015-06-23 00:30:25 -05:00
Gábor Lipták 6b37f26280 Use ProtobufMagic methods in ProtobufUtil
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2015-06-19 13:50:09 -07:00
Misty Stanley-Jones b16293b5e2 HBASE-13898 correct additional javadoc failures under java 8 <Gabor Liptak> 2015-06-16 14:18:09 +10:00
Misty Stanley-Jones 5e6373e8ec Revert "Correct Javadoc generation errors"
This reverts commit 0a227b79d6.
2015-06-16 14:18:09 +10:00
Gábor Lipták 0a227b79d6 Correct Javadoc generation errors 2015-06-16 13:27:45 +10:00
Gábor Lipták 682b8ab8a5 HBASE-13569 Correct Javadoc (for Java8)
Signed-off-by: Sean Busbey <busbey@apache.org>
2015-06-13 01:10:37 -05:00
stack 325614220f HBASE-13616 Move ServerShutdownHandler to Pv2 2015-05-28 20:45:38 -07:00
stack e1e8434340 HBASE-5980 Scanner responses from RS should include metrics on rows/KVs filtered 2015-05-21 11:06:27 -07:00
Enis Soztutar eddabdd353 HBASE-13709 Updates to meta table server columns may be eclipsed 2015-05-20 17:42:32 -07:00
stack e2e9ea5bd5 Revert "HBASE-5980 Scanner responses from RS should include metrics on rows/KVs filtered"
Committed by mistake. Backing out till sure it good.

This reverts commit dc72dad7cd.
2015-05-20 13:25:49 -07:00
stack dc72dad7cd HBASE-5980 Scanner responses from RS should include metrics on rows/KVs filtered 2015-05-20 11:55:06 -07:00
Srikanth Srungarapu 9e131dfa8e HBASE-13571 Procedure v2 - client modify table sync 2015-05-11 23:07:15 -07:00
Enis Soztutar 59a6e031fb HBASE-13661 Correct binary compatibility issues discovered in 1.1.0RC0 2015-05-11 18:28:29 -07:00
Jerry He 664b2e4f11 HBASE-13251 Correct HBase, MapReduce, and the CLASSPATH section in HBase Ref Guide (li xiang) 2015-05-05 21:25:06 -07:00
Ashish Singhi 84dd84fc58 HBASE-13572 Procedure v2 - client truncate table sync
Signed-off-by: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
2015-04-28 12:46:06 +01:00
Jonathan Lawlor abe3796a99 HBASE-13090 Progress heartbeats for long running scanners
Signed-off-by: stack <stack@apache.org>
2015-04-17 15:42:46 -07:00
Matteo Bertozzi 4788c6d1a8 HBASE-13455 Procedure V2 - master truncate table 2015-04-15 10:35:41 +01:00
Matteo Bertozzi f6512065c2 HBASE-13290 Procedure v2 - client enable/disable table sync (Stephen Yuan Jiang) 2015-04-10 18:53:43 +01:00
Matteo Bertozzi 57c70f0af8 HBASE-13211 Procedure V2 - master Enable/Disable table (Stephen Yuan Jiang) 2015-04-10 18:53:43 +01:00
Matteo Bertozzi 4ae8b8cc52 HBASE-13209 Procedure V2 - master Add/Modify/Delete Column Family (Stephen Yuan Jiang) 2015-04-10 18:53:43 +01:00
Matteo Bertozzi 7f5383368b HBASE-13210 Procedure V2 - master Modify table (Stephen Yuan Jiang) 2015-04-10 18:53:43 +01:00
Matteo Bertozzi 6a6e3f46fd HBASE-13204 Procedure v2 - client create/delete table sync 2015-04-10 18:53:42 +01:00
Matteo Bertozzi b5f1f98a25 HBASE-13203 Procedure v2 - master create/delete table 2015-04-10 18:53:42 +01:00
Matteo Bertozzi 04246c6c3d HBASE-13202 Procedure v2 - core framework 2015-04-10 18:53:37 +01:00