HBASE-23800 Add documentation about the CECPs changes (#1487)
Signed-off-by: stack <stack@apache.org>
This commit is contained in:
Duo Zhang
GitHub
parent
b02064f936
commit
2c8d104d71
|
|
@ -178,11 +178,28 @@ average or summation for an entire table which spans hundreds of regions.
|
|||
|
||||
In contrast to observer coprocessors, where your code is run transparently, endpoint
|
||||
coprocessors must be explicitly invoked using the
|
||||
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html#coprocessorService-java.lang.Class-byte:A-byte:A-org.apache.hadoop.hbase.client.coprocessor.Batch.Call-[CoprocessorService()]
|
||||
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/AsyncTable.html#coprocessorService-java.util.function.Function-org.apache.hadoop.hbase.client.ServiceCaller-byte:A-[CoprocessorService()]
|
||||
method available in
|
||||
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html[Table]
|
||||
or
|
||||
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/HTable.html[HTable].
|
||||
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/AsyncTable.html[AsyncTable].
|
||||
|
||||
[WARNING]
|
||||
.On using coprocessorService method with sync client
|
||||
====
|
||||
The coprocessorService method in link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html[Table]
|
||||
has been deprecated.
|
||||
|
||||
In link:https://issues.apache.org/jira/browse/HBASE-21512[HBASE-21512]
|
||||
we reimplement the sync client based on the async client. The coprocessorService
|
||||
method defined in `Table` interface directly references a method from protobuf's
|
||||
`BlockingInterface`, which means we need to use a separate thread pool to execute
|
||||
the method so we avoid blocking the async client(We want to avoid blocking calls in
|
||||
our async implementation).
|
||||
|
||||
Since coprocessor is an advanced feature, we believe it is OK for coprocessor users to
|
||||
instead switch over to use `AsyncTable`. There is a lightweight
|
||||
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Connection.html#toAsyncConnection--[toAsyncConnection]
|
||||
method to get an `AsyncConnection` from `Connection` if needed.
|
||||
====
|
||||
|
||||
Starting with HBase 0.96, endpoint coprocessors are implemented using Google Protocol
|
||||
Buffers (protobuf). For more details on protobuf, see Google's
|
||||
|
|
@ -193,6 +210,12 @@ link:https://issues.apache.org/jira/browse/HBASE-5448[HBASE-5448]). To upgrade y
|
|||
HBase cluster from 0.94 or earlier to 0.96 or later, you need to reimplement your
|
||||
coprocessor.
|
||||
|
||||
In HBase 2.x, we made use of a shaded version of protobuf 3.x, but kept the
|
||||
protobuf for coprocessors on 2.5.0. In HBase 3.0.0, we removed all dependencies on
|
||||
non-shaded protobuf so you need to reimplement your coprocessor to make use of the
|
||||
shaded protobuf version provided in hbase-thirdparty. Please see
|
||||
the <<protobuf,protobuf>> section for more details.
|
||||
|
||||
Coprocessor Endpoints should make no use of HBase internals and
|
||||
only avail of public APIs; ideally a CPEP should depend on Interfaces
|
||||
and data structures only. This is not always possible but beware
|
||||
|
|
|
|||
|
|
@ -148,3 +148,75 @@ consider extending it also in
|
|||
Going forward, we will provide a new module of common types for use
|
||||
by CPEPs that will have the same guarantees against change as does our
|
||||
public API. TODO.
|
||||
|
||||
=== protobuf changes for hbase-3.0.0 (HBASE-23797)
|
||||
Since hadoop(start from 3.3.x) also shades protobuf and bumps the version to
|
||||
3.x, there is no reason for us to stay on protobuf 2.5.0 any more.
|
||||
|
||||
In HBase 3.0.0, the hbase-protocol module has been purged, the CPEP
|
||||
implementation should use the protos in hbase-protocol-shaded module, and also
|
||||
make use of the shaded protobuf in hbase-thirdparty. In general, we will keep
|
||||
the protobuf version compatible for a whole major release, unless there are
|
||||
critical problems, for example, a critical CVE on protobuf.
|
||||
|
||||
Add this dependency to your pom:
|
||||
[source,xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.apache.hbase.thirdparty</groupId>
|
||||
<artifactId>hbase-shaded-protobuf</artifactId>
|
||||
<!-- use the version that your target hbase cluster uses -->
|
||||
<version>${hbase-thirdparty.version}</version>
|
||||
<scope>provided</scope>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
And typically you also need to add this plugin to your pom to make your
|
||||
generated protobuf code also use the shaded and relocated protobuf version
|
||||
in hbase-thirdparty.
|
||||
[source,xml]
|
||||
----
|
||||
<plugin>
|
||||
<groupId>com.google.code.maven-replacer-plugin</groupId>
|
||||
<artifactId>replacer</artifactId>
|
||||
<version>1.5.3</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<phase>process-sources</phase>
|
||||
<goals>
|
||||
<goal>replace</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
<configuration>
|
||||
<basedir>${basedir}/target/generated-sources/</basedir>
|
||||
<includes>
|
||||
<include>**/*.java</include>
|
||||
</includes>
|
||||
<!-- Ignore errors when missing files, because it means this build
|
||||
was run with -Dprotoc.skip and there is no -Dreplacer.skip -->
|
||||
<ignoreErrors>true</ignoreErrors>
|
||||
<replacements>
|
||||
<replacement>
|
||||
<token>([^\.])com.google.protobuf</token>
|
||||
<value>$1org.apache.hbase.thirdparty.com.google.protobuf</value>
|
||||
</replacement>
|
||||
<replacement>
|
||||
<token>(public)(\W+static)?(\W+final)?(\W+class)</token>
|
||||
<value>@javax.annotation.Generated("proto") $1$2$3$4</value>
|
||||
</replacement>
|
||||
<!-- replacer doesn't support anchoring or negative lookbehind -->
|
||||
<replacement>
|
||||
<token>(@javax.annotation.Generated\("proto"\) ){2}</token>
|
||||
<value>$1</value>
|
||||
</replacement>
|
||||
</replacements>
|
||||
</configuration>
|
||||
</plugin>
|
||||
----
|
||||
|
||||
In hbase-examples module, we have some examples under the
|
||||
`org.apache.hadoop.hbase.coprocessor.example` package. You can see
|
||||
`BulkDeleteEndpoint` and `BulkDelete.proto` for more details, and you can also
|
||||
check the `pom.xml` of hbase-examples module to see how to make use of the above
|
||||
plugin.
|
||||
|
|
|
|||
Loading…
Reference in New Issue