diff --git a/hbase-archetypes/README.md b/hbase-archetypes/README.md new file mode 100644 index 00000000000..3af1f8b1fab --- /dev/null +++ b/hbase-archetypes/README.md @@ -0,0 +1,142 @@ + + +#hbase-archetypes + +##Overview +The hbase-archetypes subproject of hbase provides an infrastructure for +creation and maintenance of Maven archetypes[1](#f1) +pertinent to HBase. Upon deployment to the archetype +catalog[2](#f2) of the central Maven +repository[3](#f3), these archetypes may be used by +end-user developers to autogenerate completely configured Maven projects +(including fully-functioning sample code) through invocation of the +`archetype:generate` goal of the +maven-archetype-plugin[4](#f4). + +##Notes for contributors and committers to the HBase project + +####The structure of hbase-archetypes +The hbase-archetypes project contains a separate subproject for each archetype. +The top level components of such a subproject comprise a complete, standalone +exemplar Maven project containing: + +- a `src` directory with sample, fully-functioning code in the `./main` and +`./test` subdirectories, +- a `pom.xml` file defining all required dependencies, and +- any additional resources required by the exemplar project. + +For example, the components of the hbase-client-project consist of (a) sample +code `./src/main/.../HelloHBase.java` and `./src/test/.../TestHelloHBase.java`, +(b) a `pom.xml` file establishing dependency upon hbase-client and test-scope +dependency upon hbase-testing-util, and (c) a `log4j.properties` resource file. + +####How archetypes are created during the hbase install process +During the `mvn install` process, all standalone exemplar projects in the +`hbase-archetypes` subdirectory are first packaged/tested/installed, and then +the following steps are executed in `hbase-archetypes/hbase-archetype-builder` +(via the `pom.xml`, bash scripts, and xsl templates in that subdirectory): + +1. For each exemplar project, resources are copied (via +maven-resources-plugin) and transformed (via xml-maven-plugin xslt +functionality) to the exemplar project's `./target/build-archetype` +subdirectory[5](#f5). +2. The script `createArchetypes.sh` is executed to invoke the +maven-archetype-plugin's `create-from-project` goal within each exemplar +project's `./target/build-archetype` subdirectory. For each exemplar +project, this creates a corresponding Maven archetype in the +`./target/build-archetype/target/generate-sources/archetype` subdirectory. +(Note that this step always issues two platform-encoding warnings per +archetype, due to hard-wired behavior of the +maven-archetype-plugin[6](#f6).) +3. The `pom.xml` file of each newly-created archetype is copied (via +maven-resources-plugin) and transformed (via xml-maven-plugin xslt +functionality)[7](#f7). +4. The script `installArchetypes.sh` is executed to install each archetype +into the local Maven repository, ready for deployment to the central Maven +repository. (Note that installation of an archetype automatically includes +invocation of integration-testing prior to install, which performs a test +generation of a project from the archetype.) + +####How to add a new archetype to the hbase-archetypes collection +1. Create a new subdirectory in `hbase-archetypes`, populated with a +completely configured Maven project, which will serve as the exemplar project +of the new archetype. (It may be most straightforward to simply copy the `src` +and `pom.xml` components from one of the existing exemplar projects, replace +the `src/main` and `src/test` code, and modify the `pom.xml` file's +``, ``,` `, and `` elements.) +2. Modify the `hbase-archetype-builder/pom.xml` file: (a) add the new exemplar +project to the `` element, and (b) add appropriate `` +elements and `` elements within the `` elements +(using the existing entries from already-existing exemplar projects as a guide). +3. Add appropriate entries for the new exemplar project to the +`createArchetypes.sh` and `installArchetypes.sh` scripts in the +`hbase-archetype-builder` subdirectory (using the existing entries as a guide). + +####How to do additional testing/inspection of an archetype in this collection +Although integration-testing (which is automatically performed for each +archetype during the install process) already performs test generation of a +project from an archetype, it may often be advisable to do further manual +testing of a newly built and installed archetype, particularly to examine and +test a project generated from the archetype (emulating the end-user experience +of utilizing the archetype). Upon completion of the install process outlined +above, all archetypes will have been installed in the local Maven repository +and can be tested locally by executing the following: + `mvn archetype:generate -DarchetypeCatalog=local` +This displays a numbered list of all locally-installed archetypes for the user +to choose from for generation of a new Maven project. + +##Footnotes: +1 -- [Maven Archetype +](http://maven.apache.org/archetype/index.html) ("About" page). +-- [↩](#a1) + +2 -- [Maven Archetype Catalog +](http://repo1.maven.org/maven2/archetype-catalog.xml) (4MB+ xml file). +-- [↩](#a2) + +3 -- [Maven Central Repository](http://search.maven.org/) +(search engine). +-- [↩](#a3) + +4 -- [Maven Archetype Plugin - archetype:generate +](http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html). +-- [↩](#a4) + +5 -- Prior to archetype creation, each exemplar project's + `pom.xml` is transformed as follows to make it into a standalone project: + RESOURCE FILTERING (a) replaces `${project.version}` with the literal value + of the current project.version and (b) replaces `${compileSource}` with the + literal value of the version of Java that is being used for compilation; + XSLT TRANSFORMATION (a) copies `` and `` subelements of + `` to make them child elements of the root element, and (b) removes + the `` and `` elements. + -- [↩](#a5) + +6 -- For an explanation of the platform-encoding warning issued + during maven-archetype-plugin processing, see the first answer to [this + stackoverflow posting](http://stackoverflow.com/a/24161287/4112172). + -- [↩](#a6) + +7 -- Prior to archetype installation, each archetype's `pom.xml` + is transformed as follows: a `` subelement + with value 'UTF-8' is added to the `` element. This prevents + platform-encoding warnings from being issued when an end-user generates + a project from the archetype. + -- [↩](#a7) diff --git a/hbase-archetypes/hbase-archetype-builder/createArchetypes.sh b/hbase-archetypes/hbase-archetype-builder/createArchetypes.sh new file mode 100755 index 00000000000..3aeb1c38134 --- /dev/null +++ b/hbase-archetypes/hbase-archetype-builder/createArchetypes.sh @@ -0,0 +1,30 @@ +#!/bin/bash + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +set -e # exit if any step of this script fails +workingDir=$(pwd) +buildArchetypeSubdir=target/build-archetype + +# CREATE hbase-client archetype +cd /"$workingDir"/../hbase-client-project/$buildArchetypeSubdir +mvn archetype:create-from-project + +# add entries for additional archetypes above this comment (modeled on entries above) + +cd "$workingDir" diff --git a/hbase-archetypes/hbase-archetype-builder/installArchetypes.sh b/hbase-archetypes/hbase-archetype-builder/installArchetypes.sh new file mode 100755 index 00000000000..74f118eaea8 --- /dev/null +++ b/hbase-archetypes/hbase-archetype-builder/installArchetypes.sh @@ -0,0 +1,31 @@ +#!/bin/bash + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +set -e # exit if any step of this script fails +workingDir=$(pwd) +buildArchetypeSubdir=target/build-archetype +archetypeSourceSubdir=target/generated-sources/archetype + +# INSTALL hbase-client archetype +cd /"$workingDir"/../hbase-client-project/$buildArchetypeSubdir/$archetypeSourceSubdir +mvn install + +# add entries for additional archetypes above this comment (modeled on entries above) + +cd "$workingDir" diff --git a/hbase-archetypes/hbase-archetype-builder/modify_archetype_pom.xsl b/hbase-archetypes/hbase-archetype-builder/modify_archetype_pom.xsl new file mode 100644 index 00000000000..e27e51d3c72 --- /dev/null +++ b/hbase-archetypes/hbase-archetype-builder/modify_archetype_pom.xsl @@ -0,0 +1,53 @@ + + + + + + + + + + + + + + + + + + UTF-8 + + + + + + + + + + UTF-8 + + + + + diff --git a/hbase-archetypes/hbase-archetype-builder/modify_exemplar_pom.xsl b/hbase-archetypes/hbase-archetype-builder/modify_exemplar_pom.xsl new file mode 100644 index 00000000000..0d7414fa1e4 --- /dev/null +++ b/hbase-archetypes/hbase-archetype-builder/modify_exemplar_pom.xsl @@ -0,0 +1,48 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/hbase-archetypes/hbase-archetype-builder/pom.xml b/hbase-archetypes/hbase-archetype-builder/pom.xml new file mode 100644 index 00000000000..ae84b1ae49f --- /dev/null +++ b/hbase-archetypes/hbase-archetype-builder/pom.xml @@ -0,0 +1,226 @@ + + + + 4.0.0 + + + hbase-archetypes + org.apache.hbase + 1.4.0-SNAPSHOT + .. + + + hbase-archetype-builder + pom + + Apache HBase - Archetype builder + Manager of plugins for building Maven archetypes from exemplars + + + target/build-archetype + target/generated-sources/archetype + target/temp + target/temp-arch + hbase-client-project + + + + + + + maven-resources-plugin + 2.7 + + + + hbase-client__copy-src-to-build-archetype-subdir + generate-resources + + copy-resources + + + /${project.basedir}/../${hbase-client.dir}/${build.archetype.subdir} + + + /${project.basedir}/../${hbase-client.dir} + + src/** + + + + + + + hbase-client__copy-pom-to-temp-for-xslt-processing + generate-resources + + copy-resources + + + /${project.basedir}/../${hbase-client.dir}/${temp.exemplar.subdir} + + + /${project.basedir}/../${hbase-client.dir} + true + + pom.xml + + + + + + + + + + hbase-client-ARCHETYPE__copy-pom-to-temp-for-xslt-processing + prepare-package + + copy-resources + + + /${project.basedir}/../${hbase-client.dir}/${temp.archetype.subdir} + + + /${project.basedir}/../${hbase-client.dir}/${build.archetype.subdir}/${archetype.source.subdir} + + pom.xml + + + + + + + + + + + org.codehaus.mojo + xml-maven-plugin + 1.0.1 + + + + modify-exemplar-pom-files-via-xslt + process-resources + + transform + + + + + /${project.basedir}/../${hbase-client.dir}/${temp.exemplar.subdir} + + pom.xml + + /${project.basedir}/../${hbase-client.dir}/${build.archetype.subdir} + modify_exemplar_pom.xsl + + + + + + + + modify-archetype-pom-files-via-xslt + package + + transform + + + + + /${project.basedir}/../${hbase-client.dir}/${temp.archetype.subdir} + + pom.xml + + /${project.basedir}/../${hbase-client.dir}/${build.archetype.subdir}/${archetype.source.subdir} + modify_archetype_pom.xsl + + + + + + + + + + org.codehaus.mojo + exec-maven-plugin + 1.4.0 + + + + make-scripts-executable + process-resources + + exec + + + chmod + + +x + /${project.basedir}/createArchetypes.sh + /${project.basedir}/installArchetypes.sh + + + + + + run-createArchetypes-script + compile + + exec + + + /${project.basedir}/createArchetypes.sh + + + + + run-installArchetypes-script + install + + exec + + + /${project.basedir}/installArchetypes.sh + + + + + + + diff --git a/hbase-archetypes/hbase-client-project/pom.xml b/hbase-archetypes/hbase-client-project/pom.xml new file mode 100644 index 00000000000..1e4e50786cf --- /dev/null +++ b/hbase-archetypes/hbase-client-project/pom.xml @@ -0,0 +1,76 @@ + + + + 4.0.0 + + hbase-archetypes + org.apache.hbase + 1.4.0-SNAPSHOT + .. + + hbase-client-project + jar + Apache HBase - Exemplar for hbase-client archetype + Exemplar project for archetype with hbase-client dependency + + UTF-8 + ${compileSource} + ${compileSource} + 2.19 + 4.12 + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + ${surefire.version} + + + + + + + + + org.apache.hbase + hbase-testing-util + ${project.version} + test + + + org.apache.hbase + hbase-client + ${project.version} + + + junit + junit + ${junit.version} + test + + + diff --git a/hbase-archetypes/hbase-client-project/src/main/java/org/apache/hbase/archetypes/exemplars/client/HelloHBase.java b/hbase-archetypes/hbase-client-project/src/main/java/org/apache/hbase/archetypes/exemplars/client/HelloHBase.java new file mode 100644 index 00000000000..c5b1b6a9a88 --- /dev/null +++ b/hbase-archetypes/hbase-client-project/src/main/java/org/apache/hbase/archetypes/exemplars/client/HelloHBase.java @@ -0,0 +1,226 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hbase.archetypes.exemplars.client; + +import java.io.IOException; +import java.util.Map.Entry; +import java.util.NavigableMap; +import org.apache.hadoop.hbase.HColumnDescriptor; +import org.apache.hadoop.hbase.HTableDescriptor; +import org.apache.hadoop.hbase.NamespaceDescriptor; +import org.apache.hadoop.hbase.NamespaceNotFoundException; +import org.apache.hadoop.hbase.TableName; +import org.apache.hadoop.hbase.client.Admin; +import org.apache.hadoop.hbase.client.Connection; +import org.apache.hadoop.hbase.client.ConnectionFactory; +import org.apache.hadoop.hbase.client.Delete; +import org.apache.hadoop.hbase.client.Get; +import org.apache.hadoop.hbase.client.Put; +import org.apache.hadoop.hbase.client.Result; +import org.apache.hadoop.hbase.client.Table; +import org.apache.hadoop.hbase.util.Bytes; + +/** + * Successful running of this application requires access to an active instance + * of HBase. For install instructions for a standalone instance of HBase, please + * refer to https://hbase.apache.org/book.html#quickstart + */ +public final class HelloHBase { + + protected static final String MY_NAMESPACE_NAME = "myTestNamespace"; + static final TableName MY_TABLE_NAME = TableName.valueOf("myTestTable"); + static final byte[] MY_COLUMN_FAMILY_NAME = Bytes.toBytes("cf"); + static final byte[] MY_FIRST_COLUMN_QUALIFIER + = Bytes.toBytes("myFirstColumn"); + static final byte[] MY_SECOND_COLUMN_QUALIFIER + = Bytes.toBytes("mySecondColumn"); + static final byte[] MY_ROW_ID = Bytes.toBytes("rowId01"); + + // Private constructor included here to avoid checkstyle warnings + private HelloHBase() { + } + + public static void main(final String[] args) throws IOException { + final boolean deleteAllAtEOJ = true; + + /** + * ConnectionFactory#createConnection() automatically looks for + * hbase-site.xml (HBase configuration parameters) on the system's + * CLASSPATH, to enable creation of Connection to HBase via Zookeeper. + */ + try (Connection connection = ConnectionFactory.createConnection(); + Admin admin = connection.getAdmin()) { + + admin.getClusterStatus(); // assure connection successfully established + System.out.println("\n*** Hello HBase! -- Connection has been " + + "established via Zookeeper!!\n"); + + createNamespaceAndTable(admin); + + System.out.println("Getting a Table object for [" + MY_TABLE_NAME + + "] with which to perform CRUD operations in HBase."); + try (Table table = connection.getTable(MY_TABLE_NAME)) { + + putRowToTable(table); + getAndPrintRowContents(table); + + if (deleteAllAtEOJ) { + deleteRow(table); + } + } + + if (deleteAllAtEOJ) { + deleteNamespaceAndTable(admin); + } + } + } + + /** + * Invokes Admin#createNamespace and Admin#createTable to create a namespace + * with a table that has one column-family. + * + * @param admin Standard Admin object + * @throws IOException If IO problem encountered + */ + static void createNamespaceAndTable(final Admin admin) throws IOException { + + if (!namespaceExists(admin, MY_NAMESPACE_NAME)) { + System.out.println("Creating Namespace [" + MY_NAMESPACE_NAME + "]."); + + admin.createNamespace(NamespaceDescriptor + .create(MY_NAMESPACE_NAME).build()); + } + if (!admin.tableExists(MY_TABLE_NAME)) { + System.out.println("Creating Table [" + MY_TABLE_NAME.getNameAsString() + + "], with one Column Family [" + + Bytes.toString(MY_COLUMN_FAMILY_NAME) + "]."); + + admin.createTable(new HTableDescriptor(MY_TABLE_NAME) + .addFamily(new HColumnDescriptor(MY_COLUMN_FAMILY_NAME))); + } + } + + /** + * Invokes Table#put to store a row (with two new columns created 'on the + * fly') into the table. + * + * @param table Standard Table object (used for CRUD operations). + * @throws IOException If IO problem encountered + */ + static void putRowToTable(final Table table) throws IOException { + + table.put(new Put(MY_ROW_ID).addColumn(MY_COLUMN_FAMILY_NAME, + MY_FIRST_COLUMN_QUALIFIER, + Bytes.toBytes("Hello")).addColumn(MY_COLUMN_FAMILY_NAME, + MY_SECOND_COLUMN_QUALIFIER, + Bytes.toBytes("World!"))); + + System.out.println("Row [" + Bytes.toString(MY_ROW_ID) + + "] was put into Table [" + + table.getName().getNameAsString() + "] in HBase;\n" + + " the row's two columns (created 'on the fly') are: [" + + Bytes.toString(MY_COLUMN_FAMILY_NAME) + ":" + + Bytes.toString(MY_FIRST_COLUMN_QUALIFIER) + + "] and [" + Bytes.toString(MY_COLUMN_FAMILY_NAME) + ":" + + Bytes.toString(MY_SECOND_COLUMN_QUALIFIER) + "]"); + } + + /** + * Invokes Table#get and prints out the contents of the retrieved row. + * + * @param table Standard Table object + * @throws IOException If IO problem encountered + */ + static void getAndPrintRowContents(final Table table) throws IOException { + + Result row = table.get(new Get(MY_ROW_ID)); + + System.out.println("Row [" + Bytes.toString(row.getRow()) + + "] was retrieved from Table [" + + table.getName().getNameAsString() + + "] in HBase, with the following content:"); + + for (Entry> colFamilyEntry + : row.getNoVersionMap().entrySet()) { + String columnFamilyName = Bytes.toString(colFamilyEntry.getKey()); + + System.out.println(" Columns in Column Family [" + columnFamilyName + + "]:"); + + for (Entry columnNameAndValueMap + : colFamilyEntry.getValue().entrySet()) { + + System.out.println(" Value of Column [" + columnFamilyName + ":" + + Bytes.toString(columnNameAndValueMap.getKey()) + "] == " + + Bytes.toString(columnNameAndValueMap.getValue())); + } + } + } + + /** + * Checks to see whether a namespace exists. + * + * @param admin Standard Admin object + * @param namespaceName Name of namespace + * @return true If namespace exists + * @throws IOException If IO problem encountered + */ + static boolean namespaceExists(final Admin admin, final String namespaceName) + throws IOException { + try { + admin.getNamespaceDescriptor(namespaceName); + } catch (NamespaceNotFoundException e) { + return false; + } + return true; + } + + /** + * Invokes Table#delete to delete test data (i.e. the row) + * + * @param table Standard Table object + * @throws IOException If IO problem is encountered + */ + static void deleteRow(final Table table) throws IOException { + System.out.println("Deleting row [" + Bytes.toString(MY_ROW_ID) + + "] from Table [" + + table.getName().getNameAsString() + "]."); + table.delete(new Delete(MY_ROW_ID)); + } + + /** + * Invokes Admin#disableTable, Admin#deleteTable, and Admin#deleteNamespace to + * disable/delete Table and delete Namespace. + * + * @param admin Standard Admin object + * @throws IOException If IO problem is encountered + */ + static void deleteNamespaceAndTable(final Admin admin) throws IOException { + if (admin.tableExists(MY_TABLE_NAME)) { + System.out.println("Disabling/deleting Table [" + + MY_TABLE_NAME.getNameAsString() + "]."); + admin.disableTable(MY_TABLE_NAME); // Disable a table before deleting it. + admin.deleteTable(MY_TABLE_NAME); + } + if (namespaceExists(admin, MY_NAMESPACE_NAME)) { + System.out.println("Deleting Namespace [" + MY_NAMESPACE_NAME + "]."); + admin.deleteNamespace(MY_NAMESPACE_NAME); + } + } +} diff --git a/hbase-archetypes/hbase-client-project/src/main/java/org/apache/hbase/archetypes/exemplars/client/package-info.java b/hbase-archetypes/hbase-client-project/src/main/java/org/apache/hbase/archetypes/exemplars/client/package-info.java new file mode 100644 index 00000000000..554014e33f3 --- /dev/null +++ b/hbase-archetypes/hbase-client-project/src/main/java/org/apache/hbase/archetypes/exemplars/client/package-info.java @@ -0,0 +1,25 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * This package provides fully-functional exemplar Java code demonstrating + * simple usage of the hbase-client API, for incorporation into a Maven + * archetype with hbase-client dependency. + */ +package org.apache.hbase.archetypes.exemplars.client; diff --git a/hbase-archetypes/hbase-client-project/src/main/resources/log4j.properties b/hbase-archetypes/hbase-client-project/src/main/resources/log4j.properties new file mode 100644 index 00000000000..d7c4552f92d --- /dev/null +++ b/hbase-archetypes/hbase-client-project/src/main/resources/log4j.properties @@ -0,0 +1,111 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Define some default values that can be overridden by system properties +hbase.root.logger=INFO,console +hbase.security.logger=INFO,console +hbase.log.dir=. +hbase.log.file=hbase.log + +# Define the root logger to the system property "hbase.root.logger". +log4j.rootLogger=${hbase.root.logger} + +# Logging Threshold +log4j.threshold=ALL + +# +# Daily Rolling File Appender +# +log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender +log4j.appender.DRFA.File=${hbase.log.dir}/${hbase.log.file} + +# Rollver at midnight +log4j.appender.DRFA.DatePattern=.yyyy-MM-dd + +# 30-day backup +#log4j.appender.DRFA.MaxBackupIndex=30 +log4j.appender.DRFA.layout=org.apache.log4j.PatternLayout + +# Pattern format: Date LogLevel LoggerName LogMessage +log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %-5p [%t] %c{2}: %m%n + +# Rolling File Appender properties +hbase.log.maxfilesize=256MB +hbase.log.maxbackupindex=20 + +# Rolling File Appender +log4j.appender.RFA=org.apache.log4j.RollingFileAppender +log4j.appender.RFA.File=${hbase.log.dir}/${hbase.log.file} + +log4j.appender.RFA.MaxFileSize=${hbase.log.maxfilesize} +log4j.appender.RFA.MaxBackupIndex=${hbase.log.maxbackupindex} + +log4j.appender.RFA.layout=org.apache.log4j.PatternLayout +log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p [%t] %c{2}: %m%n + +# +# Security audit appender +# +hbase.security.log.file=SecurityAuth.audit +hbase.security.log.maxfilesize=256MB +hbase.security.log.maxbackupindex=20 +log4j.appender.RFAS=org.apache.log4j.RollingFileAppender +log4j.appender.RFAS.File=${hbase.log.dir}/${hbase.security.log.file} +log4j.appender.RFAS.MaxFileSize=${hbase.security.log.maxfilesize} +log4j.appender.RFAS.MaxBackupIndex=${hbase.security.log.maxbackupindex} +log4j.appender.RFAS.layout=org.apache.log4j.PatternLayout +log4j.appender.RFAS.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n +log4j.category.SecurityLogger=${hbase.security.logger} +log4j.additivity.SecurityLogger=false +#log4j.logger.SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController=TRACE +#log4j.logger.SecurityLogger.org.apache.hadoop.hbase.security.visibility.VisibilityController=TRACE + +# +# Null Appender +# +log4j.appender.NullAppender=org.apache.log4j.varia.NullAppender + +# +# console +# Add "console" to rootlogger above if you want to use this +# +log4j.appender.console=org.apache.log4j.ConsoleAppender +log4j.appender.console.target=System.err +log4j.appender.console.layout=org.apache.log4j.PatternLayout +log4j.appender.console.layout.ConversionPattern=%d{ISO8601} %-5p [%t] %c{2}: %m%n + +# Custom Logging levels + +log4j.logger.org.apache.zookeeper=INFO +#log4j.logger.org.apache.hadoop.fs.FSNamesystem=DEBUG +log4j.logger.org.apache.hadoop.hbase=INFO +# Make these two classes INFO-level. Make them DEBUG to see more zk debug. +log4j.logger.org.apache.hadoop.hbase.zookeeper.ZKUtil=INFO +log4j.logger.org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher=INFO +#log4j.logger.org.apache.hadoop.dfs=DEBUG +# Set this class to log INFO only otherwise its OTT +# Enable this to get detailed connection error/retry logging. +# log4j.logger.org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation=TRACE + + +# Uncomment this line to enable tracing on _every_ RPC call (this can be a lot of output) +#log4j.logger.org.apache.hadoop.ipc.HBaseServer.trace=DEBUG + +# Uncomment the below if you want to remove logging of client region caching' +# and scan of hbase:meta messages +# log4j.logger.org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation=INFO +# log4j.logger.org.apache.hadoop.hbase.client.MetaScanner=INFO diff --git a/hbase-archetypes/hbase-client-project/src/test/java/org/apache/hbase/archetypes/exemplars/client/TestHelloHBase.java b/hbase-archetypes/hbase-client-project/src/test/java/org/apache/hbase/archetypes/exemplars/client/TestHelloHBase.java new file mode 100644 index 00000000000..3d9dabda210 --- /dev/null +++ b/hbase-archetypes/hbase-client-project/src/test/java/org/apache/hbase/archetypes/exemplars/client/TestHelloHBase.java @@ -0,0 +1,131 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hbase.archetypes.exemplars.client; + +import java.io.IOException; +import org.apache.hadoop.hbase.HBaseTestingUtility; +import org.apache.hadoop.hbase.NamespaceDescriptor; +import org.apache.hadoop.hbase.client.Admin; +import org.apache.hadoop.hbase.client.Get; +import org.apache.hadoop.hbase.client.Put; +import org.apache.hadoop.hbase.client.Result; +import org.apache.hadoop.hbase.client.Table; +import org.apache.hadoop.hbase.testclassification.MediumTests; +import org.apache.hadoop.hbase.util.Bytes; +import org.junit.AfterClass; +import static org.junit.Assert.assertEquals; +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.experimental.categories.Category; + +/** + * Unit testing for HelloHBase. + */ +@Category(MediumTests.class) +public class TestHelloHBase { + + private static final HBaseTestingUtility TEST_UTIL + = new HBaseTestingUtility(); + + @BeforeClass + public static void beforeClass() throws Exception { + TEST_UTIL.startMiniCluster(1); + } + + @AfterClass + public static void afterClass() throws Exception { + TEST_UTIL.shutdownMiniCluster(); + } + + @Test + public void testNamespaceExists() throws Exception { + final String NONEXISTENT_NAMESPACE = "xyzpdq_nonexistent"; + final String EXISTING_NAMESPACE = "pdqxyz_myExistingNamespace"; + boolean exists; + Admin admin = TEST_UTIL.getHBaseAdmin(); + + exists = HelloHBase.namespaceExists(admin, NONEXISTENT_NAMESPACE); + assertEquals("#namespaceExists failed: found nonexistent namespace.", + false, exists); + + admin.createNamespace + (NamespaceDescriptor.create(EXISTING_NAMESPACE).build()); + exists = HelloHBase.namespaceExists(admin, EXISTING_NAMESPACE); + assertEquals("#namespaceExists failed: did NOT find existing namespace.", + true, exists); + admin.deleteNamespace(EXISTING_NAMESPACE); + } + + @Test + public void testCreateNamespaceAndTable() throws Exception { + Admin admin = TEST_UTIL.getHBaseAdmin(); + HelloHBase.createNamespaceAndTable(admin); + + boolean namespaceExists + = HelloHBase.namespaceExists(admin, HelloHBase.MY_NAMESPACE_NAME); + assertEquals("#createNamespaceAndTable failed to create namespace.", + true, namespaceExists); + + boolean tableExists = admin.tableExists(HelloHBase.MY_TABLE_NAME); + assertEquals("#createNamespaceAndTable failed to create table.", + true, tableExists); + + admin.disableTable(HelloHBase.MY_TABLE_NAME); + admin.deleteTable(HelloHBase.MY_TABLE_NAME); + admin.deleteNamespace(HelloHBase.MY_NAMESPACE_NAME); + } + + @Test + public void testPutRowToTable() throws IOException { + Admin admin = TEST_UTIL.getHBaseAdmin(); + admin.createNamespace + (NamespaceDescriptor.create(HelloHBase.MY_NAMESPACE_NAME).build()); + Table table + = TEST_UTIL.createTable + (HelloHBase.MY_TABLE_NAME, HelloHBase.MY_COLUMN_FAMILY_NAME); + + HelloHBase.putRowToTable(table); + Result row = table.get(new Get(HelloHBase.MY_ROW_ID)); + assertEquals("#putRowToTable failed to store row.", false, row.isEmpty()); + + TEST_UTIL.deleteTable(HelloHBase.MY_TABLE_NAME); + admin.deleteNamespace(HelloHBase.MY_NAMESPACE_NAME); + } + + @Test + public void testDeleteRow() throws IOException { + Admin admin = TEST_UTIL.getHBaseAdmin(); + admin.createNamespace + (NamespaceDescriptor.create(HelloHBase.MY_NAMESPACE_NAME).build()); + Table table + = TEST_UTIL.createTable + (HelloHBase.MY_TABLE_NAME, HelloHBase.MY_COLUMN_FAMILY_NAME); + + table.put(new Put(HelloHBase.MY_ROW_ID). + addColumn(HelloHBase.MY_COLUMN_FAMILY_NAME, + HelloHBase.MY_FIRST_COLUMN_QUALIFIER, + Bytes.toBytes("xyz"))); + HelloHBase.deleteRow(table); + Result row = table.get(new Get(HelloHBase.MY_ROW_ID)); + assertEquals("#deleteRow failed to delete row.", true, row.isEmpty()); + + TEST_UTIL.deleteTable(HelloHBase.MY_TABLE_NAME); + admin.deleteNamespace(HelloHBase.MY_NAMESPACE_NAME); + } +} diff --git a/hbase-archetypes/pom.xml b/hbase-archetypes/pom.xml new file mode 100644 index 00000000000..07542d07537 --- /dev/null +++ b/hbase-archetypes/pom.xml @@ -0,0 +1,82 @@ + + + + 4.0.0 + + hbase + org.apache.hbase + 1.4.0-SNAPSHOT + .. + + + hbase-archetypes + pom + + Apache HBase - Archetypes + Maven archetypes for generation of fully-configured HBase client projects + + + hbase-client-project + + hbase-archetype-builder + + + + + + + + org.codehaus.mojo + findbugs-maven-plugin + + ${project.basedir}/../../dev-support/findbugs-exclude.xml + true + true + Max + + + + + + + + org.codehaus.mojo + findbugs-maven-plugin + + + false + + findbugs + + + ${project.basedir}/../dev-support/findbugs-exclude.xml + + + + + + + diff --git a/pom.xml b/pom.xml index 88db3a13e99..10659a7e00c 100644 --- a/pom.xml +++ b/pom.xml @@ -69,6 +69,7 @@ hbase-checkstyle hbase-external-blockcache hbase-shaded + hbase-archetypes diff --git a/src/main/asciidoc/_chapters/developer.adoc b/src/main/asciidoc/_chapters/developer.adoc index 16bf69bfccb..c3ba0a2c180 100644 --- a/src/main/asciidoc/_chapters/developer.adoc +++ b/src/main/asciidoc/_chapters/developer.adoc @@ -1960,6 +1960,15 @@ However any substantive discussion (as with any off-list project-related discuss Misspellings and/or bad grammar is preferable to the disruption a JIRA comment edit causes: See the discussion at link:http://search-hadoop.com/?q=%5BReopened%5D+%28HBASE-451%29+Remove+HTableDescriptor+from+HRegionInfo&fc_project=HBase[Re:(HBASE-451) Remove HTableDescriptor from HRegionInfo] +[[hbase.archetypes.development]] +=== Development of HBase-related Maven archetypes + +The development of HBase-related Maven archetypes was begun with +link:https://issues.apache.org/jira/browse/HBASE-14876[HBASE-14876]. +For an overview of the hbase-archetypes infrastructure and instructions +for developing new HBase-related Maven archetypes, please see +`hbase/hbase-archetypes/README.md`. + ifdef::backend-docbook[] [index] == Index