Summary:
Intgration tests don't currently have a lot of splits going on while there is IO.
This changes that by changing the split aglorithm and the max region hfile size. That should make things split more.
Additionally this allows ITBLL to start with just one region if hbase.test.pre-split-table is False.
Test Plan: Test on a cluster with ITBLL slow determinitic monkey and stress am monkey.
Differential Revision: https://reviews.facebook.net/D44181
* fix hbase-it's incorrect use of hbase-server's unfiltered NOTICE
* fix printed comment in resource-bundle NOTICE
* add dependency url for non-aggregated works so that we meet Category B requirements
* corrects license/notice for source distribution
* adds inception year to correct copyright in generated NOTICE files for jars
* updates project names in poms to use "Apache HBase" instead of "HBase" so jar NOTICE files will be correct
* uses append-resources to include supplemental info on jars with 3rd party works in source
* adds an hbase specific resource bundle for jars that include 3rd party works for binaries
** uses supplemental-model to fill in license gaps
** uses the above and a shade plugin transformation to build proper files for shaded jars.
** uses the above and the assembly plugin to build the proper files for bin assembly
* adds a NOTICE item for things copied out of Hadoop (TODO legal-discuss)
Conflicts:
NOTICE.txt
hbase-checkstyle/pom.xml
hbase-resource-bundle/pom.xml
hbase-server/pom.xml
hbase-thrift/pom.xml
pom.xml
* IA.Public accessible logger instances deprecated
* logger instances modified by tests left in place
* all others made private static final
Signed-off-by: Sean Busbey <busbey@apache.org>
Conflicts:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientIdGenerator.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerImpl.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestIngestWithEncryption.java
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestStatusResource.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestTableDescriptor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestRowCounter.java
hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java
Summary: This test subclasses the BigLinkedList test. It takes two hbase clusters as arguments, sets up tables, sets up replication, and runs the BigLinkedList generator. The verification portion of the loop checks that the sink of the replication has the data and it is correct.
Test Plan: ran the test on my laptop and a small live cluster
Reviewers: dimaspivak, eclark
Reviewed By: eclark
Subscribers: srikanth235, asameet
Differential Revision: https://reviews.facebook.net/D36423
Signed-off-by: Elliott Clark <eclark@apache.org>
Adds a number of lifecycle-mapping entries which
prevent errors from showing up in Eclipse on a fresh
import of HBase. For plugins defined in the top-level
pom, the mapping is added there; otherwise, the mapping
is pushed down to the child pom.
Signed-off-by: Sean Busbey <busbey@apache.org>
In our pre-1.0 API, HTable is considered a light-weight object that consumed by
a single thread at a time. The HTablePool class provided a means of sharing
multiple HTable instances across a number of threads. As an optimization,
HTable managed a "write buffer", accumulating edits and sending a "batch" all
at once. By default the batch was sent as the last step in invocations of
put(Put) and put(List<Put>). The user could disable the automatic flushing of
the write buffer, retaining edits locally and only sending the whole "batch"
once the write buffer has filled or when the flushCommits() method in invoked
explicitly. Explicit or implicit batch writing was controlled by the
setAutoFlushTo(boolean) method. A value of true (the default) had the write
buffer flushed at the completion of a call to put(Put) or put(List<Put>). A
value of false allowed for explicit buffer management. HTable also exposed the
buffer to consumers via getWriteBuffer().
The combination of HTable with setAutoFlushTo(false) and the HTablePool
provided a convenient mechanism by which multiple "Put-producing" threads could
share a common write buffer. Both HTablePool and HTable are deprecated, and
they are officially replaced in The new 1.0 API by Table and BufferedMutator.
Table, which replaces HTable, no longer exposes explicit write-buffer
management. Instead, explicit buffer management is exposed via BufferedMutator.
BufferedMutator is made safe for concurrent use. Where code would previously
retrieve and return HTables from an HTablePool, now that code creates and
shares a single BufferedMutator instance across all threads.
Conflicts:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedListWithVisibility.java
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestLoadAndVerify.java
hbase-it/src/test/java/org/apache/hadoop/hbase/trace/IntegrationTestSendTraceRequests.java
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/PerformanceEvaluation.java
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/client/TestRemoteTable.java
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/MultiTableOutputFormat.java
hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMultiVersions.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientPushback.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCloneSnapshotFromClient.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRestoreSnapshotFromClient.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRpcControllerFactory.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestScannerWithBulkload.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationChangingPeerRegionservers.java
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWithTags.java
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsReplication.java
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreFlushSnapshotFromClient.java
tedyu
Elliott Clark
Stephen Yuan Jiang
Srikanth Srungarapu
Sean Busbey
Sean Busbey
Enis Soztutar
stack
Dima Spivak
Matteo Bertozzi
Rajeshbabu Chintaguntla
Andrew Purtell
Nick Dimiduk
Matt Warhaftig
Usha Kuchibhotla
Rajesh Nishtala
zhangduo
Josh Elser
Jonathan M Hsieh
Devaraj Das
Ramkrishna