HBASE-23800 Add documentation about the CECPs changes (#1487)
Signed-off-by: stack <stack@apache.org>
This commit is contained in:
parent
b02064f936
commit
2c8d104d71
|
@ -178,11 +178,28 @@ average or summation for an entire table which spans hundreds of regions.
|
||||||
|
|
||||||
In contrast to observer coprocessors, where your code is run transparently, endpoint
|
In contrast to observer coprocessors, where your code is run transparently, endpoint
|
||||||
coprocessors must be explicitly invoked using the
|
coprocessors must be explicitly invoked using the
|
||||||
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html#coprocessorService-java.lang.Class-byte:A-byte:A-org.apache.hadoop.hbase.client.coprocessor.Batch.Call-[CoprocessorService()]
|
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/AsyncTable.html#coprocessorService-java.util.function.Function-org.apache.hadoop.hbase.client.ServiceCaller-byte:A-[CoprocessorService()]
|
||||||
method available in
|
method available in
|
||||||
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html[Table]
|
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/AsyncTable.html[AsyncTable].
|
||||||
or
|
|
||||||
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/HTable.html[HTable].
|
[WARNING]
|
||||||
|
.On using coprocessorService method with sync client
|
||||||
|
====
|
||||||
|
The coprocessorService method in link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html[Table]
|
||||||
|
has been deprecated.
|
||||||
|
|
||||||
|
In link:https://issues.apache.org/jira/browse/HBASE-21512[HBASE-21512]
|
||||||
|
we reimplement the sync client based on the async client. The coprocessorService
|
||||||
|
method defined in `Table` interface directly references a method from protobuf's
|
||||||
|
`BlockingInterface`, which means we need to use a separate thread pool to execute
|
||||||
|
the method so we avoid blocking the async client(We want to avoid blocking calls in
|
||||||
|
our async implementation).
|
||||||
|
|
||||||
|
Since coprocessor is an advanced feature, we believe it is OK for coprocessor users to
|
||||||
|
instead switch over to use `AsyncTable`. There is a lightweight
|
||||||
|
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Connection.html#toAsyncConnection--[toAsyncConnection]
|
||||||
|
method to get an `AsyncConnection` from `Connection` if needed.
|
||||||
|
====
|
||||||
|
|
||||||
Starting with HBase 0.96, endpoint coprocessors are implemented using Google Protocol
|
Starting with HBase 0.96, endpoint coprocessors are implemented using Google Protocol
|
||||||
Buffers (protobuf). For more details on protobuf, see Google's
|
Buffers (protobuf). For more details on protobuf, see Google's
|
||||||
|
@ -193,6 +210,12 @@ link:https://issues.apache.org/jira/browse/HBASE-5448[HBASE-5448]). To upgrade y
|
||||||
HBase cluster from 0.94 or earlier to 0.96 or later, you need to reimplement your
|
HBase cluster from 0.94 or earlier to 0.96 or later, you need to reimplement your
|
||||||
coprocessor.
|
coprocessor.
|
||||||
|
|
||||||
|
In HBase 2.x, we made use of a shaded version of protobuf 3.x, but kept the
|
||||||
|
protobuf for coprocessors on 2.5.0. In HBase 3.0.0, we removed all dependencies on
|
||||||
|
non-shaded protobuf so you need to reimplement your coprocessor to make use of the
|
||||||
|
shaded protobuf version provided in hbase-thirdparty. Please see
|
||||||
|
the <<protobuf,protobuf>> section for more details.
|
||||||
|
|
||||||
Coprocessor Endpoints should make no use of HBase internals and
|
Coprocessor Endpoints should make no use of HBase internals and
|
||||||
only avail of public APIs; ideally a CPEP should depend on Interfaces
|
only avail of public APIs; ideally a CPEP should depend on Interfaces
|
||||||
and data structures only. This is not always possible but beware
|
and data structures only. This is not always possible but beware
|
||||||
|
|
|
@ -148,3 +148,75 @@ consider extending it also in
|
||||||
Going forward, we will provide a new module of common types for use
|
Going forward, we will provide a new module of common types for use
|
||||||
by CPEPs that will have the same guarantees against change as does our
|
by CPEPs that will have the same guarantees against change as does our
|
||||||
public API. TODO.
|
public API. TODO.
|
||||||
|
|
||||||
|
=== protobuf changes for hbase-3.0.0 (HBASE-23797)
|
||||||
|
Since hadoop(start from 3.3.x) also shades protobuf and bumps the version to
|
||||||
|
3.x, there is no reason for us to stay on protobuf 2.5.0 any more.
|
||||||
|
|
||||||
|
In HBase 3.0.0, the hbase-protocol module has been purged, the CPEP
|
||||||
|
implementation should use the protos in hbase-protocol-shaded module, and also
|
||||||
|
make use of the shaded protobuf in hbase-thirdparty. In general, we will keep
|
||||||
|
the protobuf version compatible for a whole major release, unless there are
|
||||||
|
critical problems, for example, a critical CVE on protobuf.
|
||||||
|
|
||||||
|
Add this dependency to your pom:
|
||||||
|
[source,xml]
|
||||||
|
----
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.apache.hbase.thirdparty</groupId>
|
||||||
|
<artifactId>hbase-shaded-protobuf</artifactId>
|
||||||
|
<!-- use the version that your target hbase cluster uses -->
|
||||||
|
<version>${hbase-thirdparty.version}</version>
|
||||||
|
<scope>provided</scope>
|
||||||
|
</dependency>
|
||||||
|
----
|
||||||
|
|
||||||
|
And typically you also need to add this plugin to your pom to make your
|
||||||
|
generated protobuf code also use the shaded and relocated protobuf version
|
||||||
|
in hbase-thirdparty.
|
||||||
|
[source,xml]
|
||||||
|
----
|
||||||
|
<plugin>
|
||||||
|
<groupId>com.google.code.maven-replacer-plugin</groupId>
|
||||||
|
<artifactId>replacer</artifactId>
|
||||||
|
<version>1.5.3</version>
|
||||||
|
<executions>
|
||||||
|
<execution>
|
||||||
|
<phase>process-sources</phase>
|
||||||
|
<goals>
|
||||||
|
<goal>replace</goal>
|
||||||
|
</goals>
|
||||||
|
</execution>
|
||||||
|
</executions>
|
||||||
|
<configuration>
|
||||||
|
<basedir>${basedir}/target/generated-sources/</basedir>
|
||||||
|
<includes>
|
||||||
|
<include>**/*.java</include>
|
||||||
|
</includes>
|
||||||
|
<!-- Ignore errors when missing files, because it means this build
|
||||||
|
was run with -Dprotoc.skip and there is no -Dreplacer.skip -->
|
||||||
|
<ignoreErrors>true</ignoreErrors>
|
||||||
|
<replacements>
|
||||||
|
<replacement>
|
||||||
|
<token>([^\.])com.google.protobuf</token>
|
||||||
|
<value>$1org.apache.hbase.thirdparty.com.google.protobuf</value>
|
||||||
|
</replacement>
|
||||||
|
<replacement>
|
||||||
|
<token>(public)(\W+static)?(\W+final)?(\W+class)</token>
|
||||||
|
<value>@javax.annotation.Generated("proto") $1$2$3$4</value>
|
||||||
|
</replacement>
|
||||||
|
<!-- replacer doesn't support anchoring or negative lookbehind -->
|
||||||
|
<replacement>
|
||||||
|
<token>(@javax.annotation.Generated\("proto"\) ){2}</token>
|
||||||
|
<value>$1</value>
|
||||||
|
</replacement>
|
||||||
|
</replacements>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
|
----
|
||||||
|
|
||||||
|
In hbase-examples module, we have some examples under the
|
||||||
|
`org.apache.hadoop.hbase.coprocessor.example` package. You can see
|
||||||
|
`BulkDeleteEndpoint` and `BulkDelete.proto` for more details, and you can also
|
||||||
|
check the `pom.xml` of hbase-examples module to see how to make use of the above
|
||||||
|
plugin.
|
||||||
|
|
Loading…
Reference in New Issue