481662ab39
IntegrationTestImportTsv is generating HFiles under the working directory of the current hdfs user executing the tool, before bulkloading it into HBase. Assuming you encrypt the HBase root directory within HDFS (using HDFS Transparent Encryption), you can bulkload HFiles only if they sit in the same encryption zone in HDFS as the HBase root directory itself. When IntegrationTestImportTsv is executed against a real distributed cluster and the working directory of the current user (e.g. /user/hbase) is not in the same encryption zone as the HBase root directory (e.g. /hbase/data) then you will get an exception: ``` ERROR org.apache.hadoop.hbase.regionserver.HRegion: There was a partial failure due to IO when attempting to load d : hdfs://mycluster/user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/ IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc org.apache.hadoop.ipc.RemoteException(java.io.IOException): /user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/ IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc can't be moved into an encryption zone. ``` In this commit I make it configurable where the IntegrationTestImportTsv generates the HFiles. Co-authored-by: Mate Szalay-Beko <symat@apache.com> Signed-off-by: Peter Somogyi <psomogyi@apache.org> |
||
|---|---|---|
| .idea | ||
| bin | ||
| conf | ||
| dev-support | ||
| hbase-annotations | ||
| hbase-archetypes | ||
| hbase-assembly | ||
| hbase-asyncfs | ||
| hbase-backup | ||
| hbase-balancer | ||
| hbase-build-configuration | ||
| hbase-checkstyle | ||
| hbase-client | ||
| hbase-common | ||
| hbase-endpoint | ||
| hbase-examples | ||
| hbase-external-blockcache | ||
| hbase-hadoop-compat | ||
| hbase-hbtop | ||
| hbase-http | ||
| hbase-it | ||
| hbase-logging | ||
| hbase-mapreduce | ||
| hbase-metrics | ||
| hbase-metrics-api | ||
| hbase-procedure | ||
| hbase-protocol-shaded | ||
| hbase-replication | ||
| hbase-resource-bundle | ||
| hbase-rest | ||
| hbase-server | ||
| hbase-shaded | ||
| hbase-shell | ||
| hbase-testing-util | ||
| hbase-thrift | ||
| hbase-zookeeper | ||
| src | ||
| .asf.yaml | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .pylintrc | ||
| .rubocop.yml | ||
| CHANGES.txt | ||
| LICENSE.txt | ||
| NOTICE.txt | ||
| README.txt | ||
| pom.xml | ||
README.txt
Apache HBase [1] is an open-source, distributed, versioned, column-oriented store modeled after Google' Bigtable: A Distributed Storage System for Structured Data by Chang et al.[2] Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Apache Hadoop [3]. To get started using HBase, the full documentation for this release can be found under the doc/ directory that accompanies this README. Using a browser, open the docs/index.html to view the project home page (or browse to [1]). The hbase 'book' at http://hbase.apache.org/book.html has a 'quick start' section and is where you should being your exploration of the hbase project. The latest HBase can be downloaded from an Apache Mirror [4]. The source code can be found at [5] The HBase issue tracker is at [6] Apache HBase is made available under the Apache License, version 2.0 [7] The HBase mailing lists and archives are listed here [8]. The HBase distribution includes cryptographic software. See the export control notice here [9]. 1. http://hbase.apache.org 2. http://research.google.com/archive/bigtable.html 3. http://hadoop.apache.org 4. http://www.apache.org/dyn/closer.lua/hbase/ 5. https://hbase.apache.org/source-repository.html 6. https://hbase.apache.org/issue-tracking.html 7. http://hbase.apache.org/license.html 8. http://hbase.apache.org/mail-lists.html 9. https://hbase.apache.org/export_control.html