Commit Graph

43 Commits

Author SHA1 Message Date
stack 45bb6180a3 REVERT of revert of "HBASE-16308 Contain protobuf references Gather up the pb references into a few locations only rather than have pb references distributed all about the code base."
This is a revert of a revert; i.e. we are adding back the change only adding
back with fixes for the broken unit test; was a real issue on a test that
went in just at same time as this commit; I was getting a new nonce on each
retry rather than getting one for the mutation.

Other changes since revert are more hiding of RpcController. Use
accessor method rather than always pass in a RpcController

Walked back retrying operations that used to be single-shot (though
code comment said need a retry) because it opens a can of worms where
we retry stuff like bad column family when we shouldn't (needs
work adding in DoNotRetryIOEs)

Changed name of class from PayloadCarryingServerCallable to
CancellableRegionServerCallable.

Fix javadoc and findbugs warnings.

Fix case of not initializing the ScannerCallable RpcController.

Below is original commit message:

 Remove mention of ServiceException and other protobuf classes from all over the codebase.
 Purge TimeLimitedRpcController. Lets just have one override of RpcController.
        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable.java
         Cleanup. Make it clear this is an odd class for async hbase intro.
        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
         Refactor of RegionServerCallable allows me clean up a bunch of
         boilerplate in here and remove protobuf references.
        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          Purge protobuf references everywhere except a reference to a throw of a
          ServiceException in method checkHBaseAvailable. I deprecated it in favor
          of new available method (the SE is not actually needed)
        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/PayloadCarryingServerCallable.java
         Move the RetryingTimeTracker instance in here from HTable.
         Allows me to contain tracker and remove a repeated code in HTable.
        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java
         Clean up move set up of rpc in here rather than have it repeat in HTable.
         Allows me to remove protobuf references from a bunch of places.
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/FlushRegionCallable.java
     Make use of the push of boilerplate up into RegionServerCallable
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/PayloadCarryingServerCallable.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/SecureBulkLoadClient.java
    M hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
     Move boilerplate up into superclass.
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetryingTimeTracker.java
     Cleanup
    M hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/PayloadCarryingRpcController.java
    M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
    M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java
    M hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
     Factor in TimeLimitedRpcController. Just have one RpcController override.
    D hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/TimeLimitedRpcController.java
     Removed. Lets have one override of pb rpccontroller only.
    M hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
     (handleRemoteException) added
     (toText) added
2016-08-10 10:12:06 -07:00
stack 0206dc67d6 Revert "HBASE-16308 Contain protobuf references Gather up the pb references into a few locations only rather than have pb references distributed all about the code base."
This reverts commit ed87a81b4b.
2016-08-05 15:18:48 -07:00
stack ed87a81b4b HBASE-16308 Contain protobuf references Gather up the pb references into a few locations only rather than have pb references distributed all about the code base.
Purge ServiceException from Callable subclasses by pushing SE handling
up into the parent Callable class (varies by context but this is basic
patten). Allows us remove a bunch of boilerplate.
Do this in the public facing classes in particular (though if
an API has SE in it -- which a few do, this patch leaves these
untouched -- for now.) Make it so HBaseAdmin and HTable have no
direct pb imports (except for endpoint processor API).

Change a few of the HBaseAdmin calls to be retrying where comments
ask that we do retry rather than one time.

Purge TimeLimitedRpcController. Lets just have one override of RpcController.

        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable.java
         Cleanup. Make it clear this is an odd class for async hbase intro.

        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
         Refactor of RegionServerCallable allows me clean up a bunch of
         boilerplate in here and remove protobuf references.

        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          Purge protobuf references everywhere except a reference to a throw of a
          ServiceException in method checkHBaseAvailable. I deprecated it in favor
          of new available method (the SE is not actually needed)

        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/PayloadCarryingServerCallable.java
         Move the RetryingTimeTracker instance in here from HTable.
         Allows me to contain tracker and remove a repeated code in HTable.

        M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java
         Clean up move set up of rpc in here rather than have it repeat in HTable.
         Allows me to remove protobuf references from a bunch of places.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/FlushRegionCallable.java
     Make use of the push of boilerplate up into RegionServerCallable

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/PayloadCarryingServerCallable.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/SecureBulkLoadClient.java
    M hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
     Move boilerplate up into superclass.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetryingTimeTracker.java
     Cleanup

    M hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/PayloadCarryingRpcController.java
    M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
    M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java
    M hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
     Factor in TimeLimitedRpcController. Just have one RpcController override.

    D hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/TimeLimitedRpcController.java
     Removed. Lets have one override of pb rpccontroller only.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
     (handleRemoteException) added
     (toText) added

Signed-off-by: stack <stack@apache.org>
2016-08-05 10:13:58 -07:00
Sean Busbey c59f76485e HBASE-16340 exclude Xerces iplementation jars from coming in transitively.
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
2016-08-05 10:16:29 -05:00
Sean Busbey 60b79e2daa HBASE-16321 ensure no findbugs-jsr305
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
2016-08-05 10:16:29 -05:00
stack ff7d0082b8 HBASE-16306 Add specific imports to avoid namespace clash in defaultSource.scala (Sai Teja Ranuva) 2016-07-29 13:43:16 -07:00
stack 9d740f7b8b HBASE-16263 Move all to do w/ protobuf -- *.proto files and generated classes -- under hbase-protocol
Signed-off-by: stack <stack@apache.org>
2016-07-21 10:02:05 -07:00
Apekshit 4ffea7711a HBASE-15944 Spark test flooding mvn output. Redirect test logs to file. This doesn't fix the problem fully as I still see few logs being dumped in stdout. But it cleans up majority of the earlier dump. (Apekshit)
Change-Id: I6893301d154078a7cfb6b9af2eedc744deafb8d7

Signed-off-by: stack <stack@apache.org>
2016-06-02 08:44:08 -07:00
Jurriaan Mous cdd532da8a HBASE-15610 Remove deprecated HConnection for 2.0 thus removing all PB references for 2.0
Signed-off-by: stack <stack@apache.org>
2016-05-29 07:50:55 -07:00
tedyu 084b036cb2 HBASE-15825 Fix the null pointer in DynamicLogicExpressionSuite (Zhan Zhang) 2016-05-16 09:04:50 -07:00
Jonathan M Hsieh b353e388bb HBASE-15333 [hbase-spark] Enhance the dataframe filters to handle naively encoded short, integer, long, float and double (Zhan Zhang) 2016-05-13 07:00:41 -07:00
tedyu ebb5d421f9 HBASE-15707 ImportTSV bulk output does not support tags with hfile.format.version=3 (huaxiang sun) 2016-04-26 11:21:29 -07:00
Apekshit 7efb9edecb HBASE-15296 Break out writer and reader from StoreFile. Done using Intellij15 Refactor > Move. (Apekshit)
Change-Id: Ie719569cc3393e0b5361e9d462c3cf125ad5144e

Signed-off-by: stack <stack@apache.org>
2016-04-13 22:43:03 -07:00
Weiqing Yang 58177c103f HBASE-15572 Adding optional timestamp semantics to HBase-Spark
4 parameters, "timestamp", "minTimestamp", "maxiTimestamp" and
"maxVersions" are added to HBaseSparkConf. Users can select a
timestamp, they can also select a time range with minimum timestamp and
maximum timestamp.

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Ted Yu <tedyu@apache.org>
Signed-off-by: Jerry He <jerryjch@apache.org>
2016-04-13 22:39:14 -05:00
Sean Busbey 6905d272d3 Revert "HBASE-15572 Adding optional timestamp semantics to HBase-Spark (Weiqing Yang)"
This reverts commit eec27ad7ef.
2016-03-31 21:40:50 -05:00
tedyu eec27ad7ef HBASE-15572 Adding optional timestamp semantics to HBase-Spark (Weiqing Yang) 2016-03-31 19:08:33 -07:00
tedyu 2b8a7f8d7b HBASE-15334 Add avro support for spark hbase connector (Zhan Zhang) 2016-03-17 09:11:38 -07:00
tedyu f6945c4631 HBASE-15336 Support Dataframe writer to the spark connector (Zhan Zhang) 2016-03-10 06:44:29 -08:00
Jonathan M Hsieh 97cce850fe HBASE-14801 Enhance the Spark-HBase connector catalog with json format (Zhan Zhang) 2016-03-09 10:41:56 -08:00
Ted Malaska b29ce7f114 HBASE-15271 Spark bulk load should write to temporary location and then rename on success.
Signed-off-by: Ted Yu <tedyu@apache.org>
Signed-off-by: Jonathan Hsieh <jon@cloudera.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2016-03-08 17:28:38 -08:00
Jonathan M Hsieh 20e14f449a HBASE-15310 hbase-spark module has compilation failures with clover profile 2016-02-24 11:54:43 -08:00
tedyu 00248656ee HBASE-15184 SparkSQL Scan operation doesn't work on kerberos cluster (Ted Malaska) 2016-02-23 16:52:13 -08:00
Jonathan M Hsieh f352f3c371 HBASE-15282 Bump hbase-spark to use Spark 1.6.0 2016-02-18 17:31:42 -08:00
Jerry He 3aff98c75b HBASE-15201 Add hbase-spark to hbase assembly 2016-02-08 14:13:46 -08:00
Apekshit 5e89ebcc2f HBASE-14159 Add dependency on hadoop-mapreduce-client-jobclient from hbase-spark.
Adds a test-scope dependency to get rid of following error:

[WARNING] warning: Class org.apache.hadoop.mapred.MiniMRCluster not found - continuing with a stub.
[WARNING] one warning found

Signed-off-by: Sean Busbey <busbey@cloudera.com>
2016-01-12 07:48:42 -06:00
tedyu 6868c63660 HBASE-14796 Enhance the Gets in the connector (Zhan Zhang) 2015-12-28 15:48:10 -08:00
tedyu e75e26e3c6 HBASE-14849 Add option to set block cache to false on SparkSQL executions (Zhan Zhang) 2015-12-19 15:14:58 -08:00
Jonathan M Hsieh bbfff0d072 HBASE-14991 Fix the '-feature' warning in scala build (Zhan Zhang) 2015-12-17 14:19:36 -08:00
tedyu 676ce01c82 HBASE-14795 Enhance the spark-hbase scan operations (Zhan Zhang) 2015-12-13 18:26:54 -08:00
tedyu 2317ec583d HBASE-14894 Fix misspellings of threshold in log4j.properties files for tests (Youngjoon Kim) 2015-11-28 10:08:38 -08:00
Daniel Vimont 6a493ddff7 Commit for HBASE-14825 -- corrections of typos, misspellings, and mangled links
Commit for HBASE-14825 -- corrections of typos, misspellings, and mangled links - addition commit for line lengths
2015-11-25 07:12:36 +10:00
Matteo Bertozzi 55087ce888 HBASE-14863 Add missing test/resources/log4j files in hbase modules 2015-11-22 07:31:16 -08:00
Andrew Purtell ca1048415b HBASE-14340 Add second bulk load option to Spark Bulk Load to send puts as the value (Ted Malaska) 2015-11-17 13:48:47 -08:00
stack 950050e9d6 Revert "HBASE-14728 TestRowCounter is broken in master (Abhishek Singh Chouhan)"
Revert because I overcommitted, included files from another patch.

This reverts commit 47659200ac.
2015-11-02 08:06:34 -08:00
stack 47659200ac HBASE-14728 TestRowCounter is broken in master (Abhishek Singh Chouhan) 2015-11-01 22:22:59 -08:00
tedyu dae1775a11 HBASE-14406 The dataframe datasource filter is wrong, and will result in data loss or unexpected behavior (Ted Malaska) 2015-10-16 11:22:07 -07:00
Sean Busbey e3383fd146 HBASE-14515 make sure spark tests are skippable like the other modules. 2015-09-30 08:35:11 -05:00
stack a47ff1d998 HBASE-14447 Spark tests failing: bind exception when putting up info server 2015-09-17 09:12:48 -07:00
Sean Busbey fda317cebb HBASE-14377 JavaHBaseContextSuite isn't being run. 2015-09-11 00:34:55 -05:00
Ted Malaska e95358a7fc HBASE-14181 Add Spark DataFrame DataSource to HBase-Spark Module
Signed-off-by: Sean Busbey <busbey@cloudera.com>
2015-09-07 09:44:10 -05:00
tedyu 72a48a1333 HBASE-14150 Add BulkLoad functionality to HBase-Spark Module (Ted Malaska) 2015-08-12 08:29:03 -07:00
tedyu e53d2481ee HBASE-14184 Fix indention and type-o in JavaHBaseContext (Ted Malaska) 2015-08-07 11:02:24 -07:00
Ted Malaska 30f7d127c3 HBASE-13992 Integrate SparkOnHBase into HBase
Signed-off-by: Sean Busbey <busbey@cloudera.com>
2015-07-28 11:45:23 -05:00