Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-18592 [hbase-thirdparty] Doc on new hbase-thirdparty dependency for the refguide

This commit is contained in:
Michael Stack 2017-08-18 16:18:42 -07:00
parent 56f9e1a60a
commit b932d38b2a
1 changed files with 54 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
src/main/asciidoc/_chapters/developer.adoc
View File

@ -2116,6 +2116,60 @@ However any substantive discussion (as with any off-list project-related discuss
Misspellings and/or bad grammar is preferable to the disruption a JIRA comment edit causes: See the discussion at link:http://search-hadoop.com/?q=%5BReopened%5D+%28HBASE-451%29+Remove+HTableDescriptor+from+HRegionInfo&fc_project=HBase[Re:(HBASE-451) Remove HTableDescriptor from HRegionInfo]
[[thirdparty]]
=== The hbase-thirdparty dependency and shading/relocation
A new project was created for the release of hbase-2.0.0. It was called
`hbase-thirdparty`. This project exists only to provide the main hbase
project with relocated -- or shaded -- versions of popular thirdparty
libraries such as guava, netty, and protobuf. The mainline HBase project
relies on the relocated versions of these libraries gotten from hbase-thirdparty
rather than on finding these classes in their usual locations. We do this so
we can specify whatever the version we wish. If we don't relocate, we must
harmonize our version to match that which hadoop and/or spark uses.
For developers, this means you need to be careful referring to classes from
netty, guava, protobuf, gson, etc. (see the hbase-thirdparty pom.xml for what
it provides). Devs must refer to the hbase-thirdparty provided classes. In
practice, this is usually not an issue (though it can be a bit of a pain). You
will have to hunt for the relocated version of your particular class. You'll
find it by prepending the general relocation prefix of `org.apache.hadoop.hbase.shaded.`.
For example if you are looking for `com.google.protobuf.Message`, the relocated
version used by HBase internals can be found at
`org.apache.hadoop.hbase.shaded.com.google.protobuf.Message`.
For a few thirdparty libs, like protobuf (see the protobuf chapter in this book
for the why), your IDE may give you both options -- the `com.google.protobuf.*`
and the `org.apache.hadoop.hbase.shaded.com.google.protobuf.*` -- because both
classes are on your CLASSPATH. Unless you are doing the particular juggling
required in Coprocessor Endpoint development (again see above cited protobuf
chapter), you'll want to use the shaded version, always.
Of note, the relocation of netty is particular. The netty folks have put in
place facility to aid relocation; it seems like shading netty is a popular project.
One case of this requires the setting of a peculiar system property on the JVM
so that classes out in the bundld shared library (.so) can be found in their
relocated location. Here is the property that needs to be set:
`-Dorg.apache.hadoop.hbase.shaded.io.netty.packagePrefix=org.apache.hadoop.hbase.shaded.`
(Note that the trailing '.' is required). Starting hbase normally or when running
test suites, the setting of this property is done for you. If you are doing something
out of the ordinary, starting hbase from your own context, you'll need to provide
this property on platforms that favor the bundled .so. See release notes on HBASE-18271
for more. The complaint you see is something like the following:
`Cause: java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.`
The `hbase-thirdparty` project has groupid of `org.apache.hbase.thirdparty`.
As of this writing, it provides three jars; one for netty with an artifactid of
`hbase-thirdparty-netty`, one for protobuf at `hbase-thirdparty-protobuf` and then
a jar for all else -- gson, guava -- at `hbase-thirdpaty-miscellaneous`.
The hbase-thirdparty artifacts are a product produced by the Apache HBase
project under the aegis of the HBase Project Management Committee. Releases
are done via the usual voting project on the hbase dev mailing list. If issue
in the hbase-thirdparty, use the hbase JIRA and mailing lists to post notice.
[[hbase.archetypes.development]]
=== Development of HBase-related Maven archetypes