Commit Graph

4953 Commits

Author SHA1 Message Date
Joseph Hwang f292048ffd HBASE-16208 Check that the row exists before attempting to remove a queue from TableBasedReplicationQueuesImpl
Signed-off-by: Elliott Clark <eclark@apache.org>
2016-07-12 14:50:07 -07:00
Joseph Hwang 90ba723dc6 HBASE-16214 Add in a UI description for the Replication Table
Signed-off-by: Elliott Clark <eclark@apache.org>
2016-07-12 12:21:55 -07:00
Matteo Bertozzi 2650711e94 HBASE-16207 can't restore snapshot without "Admin" permission 2016-07-12 05:50:06 -07:00
Joseph Hwang ccf293d7fb HBASE-16081 Wait for Replication Tasks to complete before killing the ThreadPoolExecutor inside of HBaseInterClusterReplicationEndpoint
Signed-off-by: Mikhail Antonov <antonov@apache.org>
2016-07-11 17:34:12 -07:00
Gary Helmling 3b5fbf8d73 HBASE-16160 Support RPC encryption with direct ByteBuffers (Colin Ma via garyh) 2016-07-11 10:33:22 -07:00
Yu Li 356c130702 HBASE-16194 Should count in MSLAB chunk allocation into heap size change when adding duplicate cells, addendum 2016-07-11 22:44:32 +08:00
tedyu 0875c35781 HBASE-16185 TestReplicationSmallTests fails in master branch (Phil Yang) 2016-07-10 23:28:32 -07:00
Yu Li 9cf012cd00 HBASE-16194 Should count in MSLAB chunk allocation into heap size change when adding duplicate cells 2016-07-10 13:45:17 +08:00
stack 496fd9837a HBASE-16176 Bug fixes/improvements on HBASE-15650 Remove
TimeRangeTracker as point of contention when many threads reading a StoreFile
 Fixes HBASE-16074 ITBLL fails, reports lost big or tiny families broken
 scanning because of a side effect of a clean up in  HBASE-15650 to make
 TimeRange construction consistent exposed a latent issue in
 TimeRange#compare. See HBASE-16074 for more detail.

Also change HFile Writer constructor so we pass in the TimeRangeTracker, if one,
on construction rather than set later (the flag and reference were not volatile
so could have made for issues in concurrent case). And make sure the construction
of a TimeRange from a TimeRangeTracer on open of an HFile Reader never makes a
bad minimum value, one that would preclude us reading any values from a file
(set min to 0)

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
 Call through to next constructor (if minStamp was 0, we'd skip setting
 allTime=true). Add asserts that timestamps are not < 0 cos it messes
 us up if they are (we already were checking for < 0 on construction but
 assert passed in timestamps are not < 0).

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
 Add constructor override that takes a TimeRangeTracker (set when flushing
 but not when compacting)

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
 Add override creating an HFile in tmp that takes a TimeRangeTracker

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
 Add override for HFile Writer that takes a TimeRangeTracker Take it on
 construction instead of having it passed by a setter later (flags and
 reference set by the setter were not volatile... could have been prob
 in concurrent case)

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
 Log WARN if bad initial TimeRange value (and then 'fix' it)

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
 A few tests to prove serialization works as expected and that we'll get a bad min if not constructed properly.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
 Handle OLDEST_TIMESTAMP explictly. Don't expect TimeRange to do it.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
 Refactor from junit3 to junit4 and add test for this weird case.
2016-07-08 18:20:31 -07:00
Yu Li 3c39cbd92c HBASE-16201 NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication 2016-07-09 02:02:00 +08:00
Yu Li 1578a045b6 Revert "NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication"
This reverts commit a33097e067.
2016-07-09 02:01:43 +08:00
stack 5e0d97e0e9 HBASE-16171 Fix the potential problems in TestHCM.testConnectionCloseAllowsInterrupt (Colin Ma) 2016-07-08 10:18:36 -07:00
Yu Li a33097e067 NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication 2016-07-09 01:05:10 +08:00
anoopsjohn 581d2b7de5 HBASE-16162 Compacting Memstore : unnecessary push of active segments to pipeline. 2016-07-06 18:54:35 +05:30
Elliott Clark ae92668dd6 HBASE-16087 Replication shouldn't start on a master if if only hosts system tables 2016-07-05 13:50:37 -07:00
tedyu 20a99b4c06 HBASE-16144 Revert - test failure in TestMultiSlaveReplication#testMultiSlaveReplication seems to be related 2016-07-05 13:19:33 -07:00
Andrew Purtell 318751cfd6 HBASE-16091 Canary takes lot more time when there are delete markers in the table (Vishal Khandelwal) 2016-07-05 11:13:48 -07:00
tedyu d016338e45 HBASE-16157 The incorrect block cache count and size are caused by removing duplicate block key in the LruBlockCache (ChiaPing Tsai) 2016-07-05 10:21:27 -07:00
tedyu b5cdb38dc7 HBASE-16144 Replication queue's lock will live forever if RS acquiring the lock has died prematurely (Phil Yang) 2016-07-05 07:16:12 -07:00
Yu Li 7f44dfd85f HBASE-16132 Scan does not return all the result when regionserver is busy (binlijin) 2016-07-04 21:45:04 +08:00
zhangduo 6944a17ad4 HBASE-16135 PeerClusterZnode under rs of removed peer may never be deleted 2016-07-04 19:26:27 +08:00
anoopsjohn d22c23c396 HBASE-16163 Compacting Memstore : CompactionPipeline#addFirst can use LinkedList#addFirst. 2016-07-03 21:35:29 +05:30
anoopsjohn 97f2294bfd HBASE-16164 Missing close of new compacted segments in few occasions which might leak MSLAB chunks from pool. 2016-07-02 11:42:08 +05:30
chenheng 561eb82968 HBASE-15844 We should respect hfile.block.index.cacheonwrite when write intermediate index Block 2016-07-02 09:04:15 +08:00
tedyu fe75edb556 HBASE-16157 Revert - ChiaPing has new fix 2016-07-01 13:00:05 -07:00
tedyu d92a99da0e HBASE-16125 RegionMover uses hardcoded, Unix-style tmp folder - breaks Windows (Vladimir Rodionov) 2016-07-01 10:02:25 -07:00
tedyu 5a7c9939cb HBASE-16157 The incorrect block cache count and size are caused by removing duplicate block key in the LruBlockCache (ChiaPing Tsai) 2016-07-01 09:43:39 -07:00
tedyu cc73c03118 HBASE-16108 RowCounter should support multiple key ranges (Konstantin Ryakhovskiy) 2016-07-01 03:41:17 -07:00
anoopsjohn 139f0ed53c HBASE-16155 Compacting Memstore : Few log improvements. 2016-07-01 10:14:07 +05:30
stack d1d8cc71c9 HBASE-16150 Remove ConcurrentIndex (Hiroshi Ikeda) 2016-06-30 15:03:32 -07:00
Ramkrishna e2566eb0ee HBASE-16153 Correct the config name
'hbase.memestore.inmemoryflush.threshold.factor' (Ram)
2016-06-30 17:44:08 +05:30
Jonathan M Hsieh e1d130946b HBASE-16117 Fix Connection leak in mapred.TableOutputFormat 2016-06-29 17:18:43 -07:00
tedyu 42106b89b1 HBASE-16114 Get regionLocation of required regions only for MR jobs (Thiruvel Thirumoolan) 2016-06-29 16:34:44 -07:00
Ramkrishna 9b1ecb31f0 HBASE-16143 Change MemstoreScanner constructor to accept
List<KeyValueScanner> (Ram)
2016-06-29 23:10:55 +05:30
Jonathan M Hsieh 727394c2e9 HBASE-16140 bump owasp.esapi from 2.1.0 to 2.1.0.1 2016-06-29 08:51:00 -07:00
tedyu 6fb0dd07a8 HBASE-16139 Use CellUtil instead of KeyValueUtil in Import (NIDHI GAMBHIR) 2016-06-29 02:45:00 -07:00
Jingcheng Du 432868b038 HBASE-15976 RegionServerMetricsWrapperRunnable will be failure when disable blockcache. - Addendum 2016-06-29 16:31:23 +08:00
tedyu 394e7224a9 HBASE-16122 PerformanceEvaluation should provide user friendly hint when client threads argument is missing (Konstantin Ryakhovskiy) 2016-06-28 17:42:51 -07:00
Matteo Bertozzi 8bc4d4131c HBASE-16121 Require only MasterServices to the ServerManager constructor 2016-06-28 07:42:40 -07:00
Jingcheng Du e768c4afac HBASE-15976 RegionServerMetricsWrapperRunnable will be failure when disable blockcache. 2016-06-28 13:45:41 +08:00
tedyu 13cf4fb3fc HBASE-16052 Improve HBaseFsck Scalability (Ben Lau) 2016-06-27 13:24:35 -07:00
tedyu 424b789e03 HBASE-14007 Writing to table through MR should fail upfront if table does not exist/is disabled (Thiruvel Thirumoolan) 2016-06-27 06:41:00 -07:00
tedyu ba9286911f HBASE-16109 Add Interface audience annotation to a few classes 2016-06-27 06:29:04 -07:00
Elliott Clark fc4b8aa89d HBASE-16089 Add on FastPath for CoDel 2016-06-25 12:57:15 -07:00
Matteo Bertozzi e8599a2921 HBASE-16103 Procedure v2 - TestCloneSnapshotProcedure relies on execution order 2016-06-24 09:09:59 -07:00
Ramkrishna 99dc300d37 HBASE-16055 PutSortReducer loses any Visibility/acl attribute set on the
Puts (Ram)
2016-06-24 15:09:53 +05:30
Jingcheng Du 518faa735b HBASE-15353 Add metric for number of CallQueueTooBigException's 2016-06-24 13:00:06 +08:00
chenheng 68c1b34dbc HBASE-16040 Remove configuration "hbase.replication" 2016-06-24 10:41:35 +08:00
Gary Helmling f4cec2e202 HBASE-16085 Add a metric for failed compactions 2016-06-23 15:38:58 -07:00
tedyu 9cd097cfac HBASE-16090 ResultScanner is not closed in SyncTable#finishRemainingHashRanges() 2016-06-23 07:28:35 -07:00
Matteo Bertozzi d66316fd80 HBASE-16068 Procedure v2 - use consts for conf properties in tests 2016-06-22 22:48:07 -07:00
Ramkrishna c1a559a666 HBASE-16070 Mapreduce Serialization classes do not have Interface audience
(Ram)
2016-06-23 10:22:24 +05:30
tedyu d297ecdf3c HBASE-16083 Addendum fixes config property names in tests (Joseph Hwang) 2016-06-22 17:31:57 -07:00
tedyu 0ea4d2f461 HBASE-16083 revert Addendum - Joseph discovered the mistake first 2016-06-22 17:31:24 -07:00
tedyu da09fd0781 HBASE-16083 Addendum fixes config property names in tests 2016-06-22 16:35:37 -07:00
Joseph Hwang f3b3a7c9b5 HBASE-16080 Fix flakey test TestTableBasedReplicationSourceManager.cleanupFailoverQueues().
Instead of running the primary test in a separate thread and hoping it finishes in time, just run the test in the primary thread.

Signed-off-by: Elliott Clark <eclark@apache.org>
2016-06-22 12:26:35 -07:00
tedyu 6fc71c5d31 HBASE-16049 TestRowProcessorEndpoint is failing on Apache Builds (Guanghao Zhang) 2016-06-22 12:24:07 -07:00
tedyu 2846113b59 HBASE-16012 Major compaction can't work due to obsolete scanner read point in RegionServer (Guanghao Zhang) 2016-06-22 02:08:45 -07:00
Joseph Hwang 152594560e HBASE-16036 Made Replication Table creation non-blocking.
All ReplicationTableBase method's that need to access the Replication Table will block until it is created though.
Also refactored ReplicationSourceManager so that abandoned queue adoption is run in the background too so that it does not block HRegionServer initialization.

Signed-off-by: Elliott Clark <eclark@apache.org>
2016-06-21 13:05:50 -07:00
tedyu b006e41a37 HBASE-16051 TestScannerHeartbeatMessages fails on some machines (Phil Yang) 2016-06-21 07:32:30 -07:00
Yu Li 471f942ec8 HBASE-16032 Possible memory leak in StoreScanner 2016-06-21 20:06:33 +08:00
tedyu 3de0c63105 HBASE-16035 Nested AutoCloseables might not all get closed (Sean Mackrory) 2016-06-20 16:25:50 -07:00
Matteo Bertozzi 313a3d23a8 HBASE-14644 Region in transition metric is broken -- addendum (Huaxiang Sun) 2016-06-20 10:46:25 -07:00
tedyu be56b5ae71 HBASE-16059 Region normalizer fails to trigger merge action where one of the regions is empty 2016-06-20 10:40:50 -07:00
tedyu 095a82584e HBASE-16062 Improper error handling in WAL Reader/Writer creation (Vladimir Rodionov) 2016-06-20 07:34:26 -07:00
Ramkrishna 71c8cd5b1f HBASE-16042 Add support in PE tool for InMemory Compaction (Ram) 2016-06-20 16:29:24 +05:30
stack 01684c3417 HBASE-16023 Fastpath for the FIFO rpcscheduler AMENDMENT 2016-06-20 08:34:32 +01:00
Yu Li 4421fdf981 HBASE-16066 Resolve RpC_REPEATED_CONDITIONAL_TEST findbugs warnings in HMaster, addendum to HBASE-15467 2016-06-19 17:25:26 +08:00
Enis Soztutar 6717f0e388 HBASE-16058 TestHRegion fails on 1.4 builds 2016-06-17 18:41:51 -07:00
Enis Soztutar bdb0cc8808 HBASE-15467 Remove 1.x/2.0 TableDescriptor incompatibility 2016-06-17 17:25:04 -07:00
Enis Soztutar 81a9c1ac31 HBASE-16053 Master code is not setting the table in ENABLING state in create table 2016-06-17 13:29:33 -07:00
Joseph Hwang 61ff6ced5b HBASE-16018 Refactored the ReplicationPeers interface to clear up what some methods do and move away from a ZooKeeper-specific implementation.
Also added some documentation for undocumented methods.

Signed-off-by: Elliott Clark <eclark@apache.org>
2016-06-17 13:04:21 -07:00
tedyu 5147fb12a5 HBASE-16048 Tag InternalScanner with LimitedPrivate(HBaseInterfaceAudience.COPROC) 2016-06-17 07:27:52 -07:00
Ashu Pachauri 3abd52bdc6 HBASE-15429 Add split policy for busy regions
Signed-off-by: Elliott Clark <eclark@apache.org>
2016-06-16 17:36:54 -07:00
tedyu 6d0e0e3721 HBASE-5291 Addendum 2 passes correct path to deleteRecursively 2016-06-16 17:00:57 -07:00
tedyu d8902ba0e6 HBASE-16045 endtime argument for VerifyReplication was incorrectly specified in usage 2016-06-16 15:50:29 -07:00
Mikhail Antonov 62a4a2c029 HBASE-16047 TestFastFail is broken again 2016-06-16 14:09:48 -07:00
Yu Li 6d02f36ac7 HBASE-16033 Add more details in logging of responseTooSlow/TooLarge 2016-06-16 16:42:10 +08:00
Ramkrishna f19f1d9e99 HBASE-15999 NPE in MemstoreCompactor (Ram) 2016-06-16 10:06:40 +05:30
tedyu 1bad166f67 HBASE-5291 Addendum removes duplicate spnego (Josh Elser) 2016-06-15 14:59:17 -07:00
Enis Soztutar bff35d6387 HBASE-15950 Fix memstore size estimates to be more tighter 2016-06-15 13:34:39 -07:00
Mikhail Antonov 5f9d1a71c8 HBASE-16026 Master UI should display status of additional ZK switches 2016-06-15 12:59:24 -07:00
Joseph Hwang 2093aadec1 HBASE-15974 Create a ReplicationQueuesClientHBaseImpl
Building on HBase-15958.
Provided a ReplicationQueuesClientHBaseImpl that relies on the HBase Replication Table to track WAL queues.
Refactored out a large section of ReplicationQueuesHBaseImpl into a ReplicationTableClient class that handles Replication Table operations.

Signed-off-by: Elliott Clark <eclark@apache.org>
2016-06-15 10:43:14 -07:00
tedyu ae5fe1e616 HBASE-5291 Add Kerberos HTTP SPNEGO authentication support to HBase web consoles (Josh Elser) 2016-06-15 09:26:44 -07:00
tedyu 6c60bc9f6c HBASE-16017 HBase TableOutputFormat has connection leak in getRecordWriter (Zhan Zhang) 2016-06-15 06:32:23 -07:00
chenheng db234bf15d HBASE-16023 Fastpath for the FIFO rpcscheduler. addendum 2016-06-15 16:20:55 +08:00
stack ee86e91e7e HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor that does balanced queue and fast path handing off requests directly to waiting handlers if any present. Idea taken from Apace Kudu (incubating). See https://gerrit.cloudera.org/#/c/2938/7/src/kudu/rpc/service_queue.h
M hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java
 Refactor which makes a Handler type. Put all 'handler' stuff inside this
 new type. Also make it so subclass can provide its own Handler type.

M hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
 Name the handler threads for their type so can tell if configs are
 having an effect.

Signed-off-by: stack <stack@apache.org>
2016-06-14 17:42:01 -07:00
stack fa50d456a8 HBASE-15971 Regression: Random Read/WorkloadC slower in 1.x than 0.98 Restore FIFO as the default in place of 'deadline' that sorted on request priority. The sort costs us throughput. Enable it if you want it rather than have it as default.
Name threads to include the scheduler type.
2016-06-14 16:51:24 -07:00
Stephen Yuan Jiang df24004353 HBASE-15746 Remove extra RegionCoprocessor preClose() in RSRpcServices#closeRegion (Stephen Yuan Jiang) 2016-06-14 13:30:41 -07:00
anoopsjohn 17bcf14fea HBASE-15525 OutOfMemory could occur when using BoundedByteBufferPool during RPC bursts. 2016-06-14 19:03:54 +05:30
Stephen Yuan Jiang e486d274c5 HBASE-16016 Fix overflow deadline value in AssignmentManager#waitForAssignment (Stephen Yuan Jiang) 2016-06-13 18:21:41 -07:00
Ramkrishna 56c209c8aa HBASE-15991 CompactingMemstore#InMemoryFlushRunnable should implement
Comparable/Comparator (Ram)
2016-06-13 17:42:35 +05:30
Ramkrishna 2d0448fa84 Revert HBASE-15991 commit 2016-06-13 15:05:50 +05:30
Ramkrishna 2738518d43 HBASE-15991 CompactingMemstore#InMemoryFlushRunnable should implement
Comparable/Comparator (Ram)
2016-06-13 12:43:52 +05:30
tedyu 6360cc0307 HBASE-16007 Job's Configuration should be passed to TableMapReduceUtil#addDependencyJars() in WALPlayer 2016-06-11 07:05:27 -07:00
Matteo Bertozzi 4c885f4fdb HBASE-14644 Region in transition metric is broken (Huaxiang Sun) 2016-06-10 21:52:24 -07:00
Sean Mackrory 6da6babe4f HBASE-15946. Eliminate possible security concerns in Store File metrics.
Invoking 'hbase hfile' inside a servlet raises several concerns. This
patch avoids invoking a separate process, and also adds validation that
the file being read is at least inside the HBase root directory.

Signed-off-by: Mikhail Antonov <antonov@apache.org>
2016-06-09 16:08:19 -07:00
Joseph Hwang babdedc1b0 HBASE-15958 Implement ClaimQueues on top of HBase
Building on HBase-15883.
Now implementing the claim queues procedure within an HBase table.
Also added UnitTests to test claimQueue.
Peer tracking will still be performed by ZooKeeper though.
Also modified the queueId tracking procedure so we no longer have to perform scans over the Replication Table.
This does make our queue naming schema slightly different from ReplicationQueuesZKImpl though.

Signed-off-by: Elliott Clark <eclark@apache.org>
2016-06-09 15:05:54 -07:00
tedyu 55a04b7810 HBASE-15990 The priority value of subsequent coprocessors in the Coprocessor.Priority.SYSTEM list are not incremented by one (ChiaPing Tsai) 2016-06-09 10:26:10 -07:00
Ashish Singhi 9012a0b123 HBASE-15952 Bulk load data replication is not working when RS user does not have permission on hfile-refs node 2016-06-09 18:44:29 +05:30
stack 031b745001 HBASE-15994 Allow selection of RpcSchedulers
Adds logging by the RpcExecutors of their run configs
Adds a FifoRpcSchedulerFactory so you can try Fifo scheduler.
2016-06-08 20:23:11 -07:00
stack 407aa4d496 Revert "HBASE-15994 Allow selection of RpcSchedulers Adds logging by the RpcExecutors of their run configs Changes the default RpcSchedulerFactory from SimpleRpcSchedulerFactory.class to RpcSchedulerFactoryImpl.class. RpcSchedulerFactoryImpl.class. defaults to using SimpleRpcSchedulerFactory.class and the SimpleRpcScheduler, as has been default up to this, unless you set "hbase.region.server.rpc.scheduler.class" to org.apache.hadoop.hbase.ipc.FifoRpcScheduler"
This reverts commit 3ac4a57fd2.
2016-06-08 20:22:50 -07:00
stack 3ac4a57fd2 HBASE-15994 Allow selection of RpcSchedulers Adds logging by the RpcExecutors of their run configs Changes the default RpcSchedulerFactory from SimpleRpcSchedulerFactory.class to RpcSchedulerFactoryImpl.class. RpcSchedulerFactoryImpl.class. defaults to using SimpleRpcSchedulerFactory.class and the SimpleRpcScheduler, as has been default up to this, unless you set "hbase.region.server.rpc.scheduler.class" to org.apache.hadoop.hbase.ipc.FifoRpcScheduler
Signed-off-by: stack <stack@apache.org>
2016-06-08 20:20:50 -07:00
Jonathan M Hsieh a7172d5611 HBASE-15959 Fix flaky test TestRegionServerMetrics.testMobMetrics (Huaxiang Sun and Jingcheng Du) 2016-06-08 19:07:13 -07:00
Matteo Bertozzi d9463bcce0 HBASE-15989 Remove hbase.online.schema.update.enable 2016-06-08 13:09:31 -07:00
Matteo Bertozzi d5d9b7d500 HBASE-15107 Procedure v2 - Procedure Queue with Region locks 2016-06-08 12:52:58 -07:00
Enis Soztutar d05a3722c8 HBASE-15600 Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks (Rajeshbabu and Enis) 2016-06-07 20:12:51 -07:00
stack 3a95552cfe HBASE-15948 Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" Adds HADOOP-9955 RPC idle connection closing is extremely inefficient
Changes how we do accounting of Connections to match how it is done in Hadoop.
Adds a ConnectionManager class. Adds new configurations for this new class.

"hbase.ipc.client.idlethreshold" 4000
"hbase.ipc.client.connection.idle-scan-interval.ms" 10000
"hbase.ipc.client.connection.maxidletime" 10000
"hbase.ipc.client.kill.max", 10
"hbase.ipc.server.handler.queue.size", 100

The new scheme does away with synchronization that purportedly would freeze out
reads while we were cleaning up stale connections (according to HADOOP-9955)

Also adds in new mechanism for accepting Connections by pulling in as many
as we can at a time adding them to a Queue instead of doing one at a time.
Can help when bursty traffic according to HADOOP-9956. Removes a blocking
while Reader is busy parsing a request. Adds configuration
"hbase.ipc.server.read.connection-queue.size" with default of 100 for
queue size.

Signed-off-by: stack <stack@apache.org>
2016-06-07 16:42:21 -07:00
stack e66ecd7db6 Revert "HBASE-15948 Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers""
Revert mistaken commit...
This reverts commit e0b70c00e7.
2016-06-07 16:41:30 -07:00
stack 6d5a25935e Revert "HBASE-15967 Metric for active ipc Readers and make default fraction of cpu count"
Revert mistaken commit
This reverts commit 1125215aad.
2016-06-07 16:41:01 -07:00
stack 1125215aad HBASE-15967 Metric for active ipc Readers and make default fraction of cpu count
Add new metric hbase.regionserver.ipc.runningReaders
Also make it so Reader count is a factor of processor count
2016-06-07 13:10:14 -07:00
stack e0b70c00e7 HBASE-15948 Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
Adds HADOOP-9955 RPC idle connection closing is extremely inefficient
Then removes queue added by HADOOP-9956 at Enis suggestion

    Changes how we do accounting of Connections to match how it is done in Hadoop.
    Adds a ConnectionManager class. Adds new configurations for this new class.

    "hbase.ipc.client.idlethreshold" 4000
    "hbase.ipc.client.connection.idle-scan-interval.ms" 10000
    "hbase.ipc.client.connection.maxidletime" 10000
    "hbase.ipc.client.kill.max", 10
    "hbase.ipc.server.handler.queue.size", 100

    The new scheme does away with synchronization that purportedly would freeze out
    reads while we were cleaning up stale connections (according to HADOOP-9955)

    Also adds in new mechanism for accepting Connections by pulling in as many
    as we can at a time adding them to a Queue instead of doing one at a time.
    Can help when bursty traffic according to HADOOP-9956. Removes a blocking
    while Reader is busy parsing a request. Adds configuration
    "hbase.ipc.server.read.connection-queue.size" with default of 100 for
    queue size.
2016-06-07 13:10:14 -07:00
Andrew Purtell 376ad0d986 HBASE-15698 Increment TimeRange not serialized to server (Ted Yu) 2016-06-06 22:17:00 -07:00
Enis Soztutar 3d7840a173 HBASE-15954 REST server should log requests with TRACE instead of DEBUG 2016-06-06 10:58:37 -07:00
Stephen Yuan Jiang de1b5ff776 HBASE-15955 Disable action in CatalogJanitor#setEnabled should wait for active cleanup scan to finish (Stephen Yuan Jiang) 2016-06-03 15:49:23 -07:00
Joseph Hwang 21e98271c3 HBASE-15883 Adding WAL files and tracking offsets in HBase.
Implemented ReplicationQueuesHBaseImpl that tracks WAL offsets and replication queues in an HBase table.
Only wrote the basic tracking methods, have not implemented claimQueue() or HFileRef methods yet.
Wrote a basic unit test for ReplicationQueueHBaseImpl that tests the implemented functions on a single Region Server

Signed-off-by: Elliott Clark <elliott@fb.com>
Signed-off-by: Elliott Clark <eclark@apache.org>
2016-06-03 15:23:10 -07:00
Apekshit 9a53d8b385 HBASE-15929 There are two tests named TestRegionServerMetrics. This causes slight issues when parsing console output for hanging tests and flaky tests analysis. Moving .../hbase/TestRegionServerMetrics.java to .../hbase/regionserver/TestRegionServerReadRequestMetrics.java. (Apekshit)
Change-Id: I379c15fe2c2c01bed53bddf7619d5f2a07c5640e
2016-06-03 15:13:49 -07:00
Apekshit b557f0bec6 HBASE-15949 Cleanup TestRegionServerMetrics.
@Before and @After to setup/teardown tables using @Rule to set table name based on testname.
Refactor out copy-pasted code fragments to single function.
(Apekshit)

Change-Id: Ic22e5027cc3952bab5ec30070ed20e98017db65a
2016-06-03 14:27:24 -07:00
Matteo Bertozzi f0c159b5fe HBASE-15927 Remove HMaster.assignRegion() 2016-06-03 12:36:25 -07:00
Ashish Singhi 0cbce07626 HBASE-15888 Extend HBASE-12769 for bulk load data replication 2016-06-03 18:42:00 +05:30
Lars Hofhansl fc890a2ecb HBASE-15881 Allow BZIP2 compression. 2016-06-02 12:19:02 -07:00
tedyu 7e5d530870 HBASE-15727 Canary Tool for Zookeeper (churro morales) 2016-06-02 10:15:08 -07:00
tedyu cd2588001c HBASE-15933 Addendum - make merge decision when sizes of both regions are known 2016-06-02 09:30:36 -07:00
tedyu cfe868d56e HBASE-15933 NullPointerException may be thrown from SimpleRegionNormalizer#getRegionSize() 2016-06-02 01:55:28 -07:00
Apekshit 1a27278624 HBASE-15915 Set timeouts on hanging tests. Tests run: 8, Time elapsed: 94.191 sec - in org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures Tests run: 103, Time elapsed: 69.057 sec - in org.apache.hadoop.hbase.regionserver.TestHRegion Tests run: 103, Time elapsed: 67.957 sec - in org.apache.hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush Tests run: 5, Time elapsed: 34.629 sec - in org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster Tests run: 9, Time elapsed: 82.913 sec - in org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient Tests run: 9, Time elapsed: 80.458 sec - in org.apache.hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient (Apekshit)
Change-Id: Ia7b986ca6276ff5498f588c0b1b9c570e2a9d798

Signed-off-by: stack <stack@apache.org>
2016-05-30 21:13:55 -07:00
Daniel Vimont e6d613de70 HBASE-15835 - Prevent HBaseTestingUtility#startMiniCluster from throwing _HMasterAddress already in use_ RuntimeException when another HBase thread is running
Signed-off-by: stack <stack@apache.org>
2016-05-30 20:05:19 -07:00
Jurriaan Mous a1f0c1cbb3 HBASE-15875 Remove HTable references and HTableInterface
Signed-off-by: stack <stack@apache.org>
2016-05-30 19:56:54 -07:00
Jurriaan Mous cdd532da8a HBASE-15610 Remove deprecated HConnection for 2.0 thus removing all PB references for 2.0
Signed-off-by: stack <stack@apache.org>
2016-05-29 07:50:55 -07:00
Sean Mackrory f4470af95d HBASE-15891. Closeable resources potentially not getting closed if exception is thrown.
Signed-off-by: stack <stack@apache.org>
2016-05-28 21:32:22 -07:00
Sean Mackrory 3b6e6e6c25 HBASE-15889. String case conversions are locale-sensitive, used without locale
Signed-off-by: Sean Busbey <busbey@apache.org>
2016-05-28 10:41:31 -07:00
Mikhail Antonov 60c8f76a9d HBASE-15908 Checksum verification is broken due to incorrect passing of ByteBuffers in DataChecksum (Mikhail Antonov and Appy) 2016-05-28 03:14:52 -07:00
Gary Helmling da0d74cd27 HBASE-15830 SASL encryption doesn't work with AsyncRpcChannelImpl (Colin Ma) 2016-05-26 21:58:27 -07:00
Enis Soztutar 43d1a999fe HBASE-15837 Memstore size accounting is wrong if postBatchMutate() throws exception 2016-05-26 11:12:17 -07:00
Apekshit 94696d4067 HBASE-14635 Fix flaky test TestSnapshotCloneIndependence
Signed-off-by: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
2016-05-25 21:14:06 -07:00
Joseph Hwang b3362ccb0c HBASE-15471 Added in Priority, General, and Replication queue sizes for RegionServers to the Web UI
Changed UI labels so that queue "size" refers to size in bytes and queue "length" refers to number of items in queue.

Signed-off-by: Elliott Clark <elliott@fb.com>
2016-05-25 15:10:48 -07:00
tedyu fa74baeb40 HBASE-15884 NPE in StoreFileScanner#skipKVsNewerThanReadpoint during reverse scan (Sergey Soldatov) 2016-05-25 08:58:28 -07:00
tedyu 39dc19236e HBASE-15806 revert due to discussion on security 2016-05-24 02:08:11 -07:00
tedyu c03ea895c4 HBASE-15806 An endpoint-based export tool (ChiaPing Tsai) 2016-05-23 10:51:25 -07:00
Jurriaan Mous 7130a222ce HBASE-15876 Remove doBulkLoad(Path hfofDir, final HTable table) though it has not been through a full deprecation cycle
Signed-off-by: stack <stack@apache.org>
2016-05-23 10:25:22 -07:00
tedyu ae42c65cfd HBASE-15802 ConnectionUtils should use ThreadLocalRandom instead of Random (Matt Warhaftig) 2016-05-21 21:15:10 -07:00
Jerry He 56e4b85d06 HBASE-14818 user_permission does not list namespace permissions (li xiang) 2016-05-21 20:06:05 -07:00
tedyu 0671cba65f HBASE-15529 Override needBalance in StochasticLoadBalancer (Guanghao Zhang) 2016-05-21 04:03:00 -07:00
anoopsjohn 0e52ac2464 HBASE-15228 Add the methods to RegionObserver to trigger start/complete restoring WALs. (Toshihiro Suzuki) 2016-05-21 09:56:28 +05:30
tedyu 2bb07e209e HBASE-15529 Revert due to TestRegionRebalancing failing 2016-05-20 12:56:17 -07:00
Gary Helmling 4793988aef HBASE-15856 Don't cache unresolved addresses for connections 2016-05-20 09:36:11 -07:00
tedyu 58798fc091 HBASE-15529 Override needBalance in StochasticLoadBalancer (Guanghao Zhang) 2016-05-20 07:44:05 -07:00
Matteo Bertozzi d34f5d4b96 HBASE-15865 Move TestTableDeleteFamilyHandler and TestTableDescriptorModification handler tests to procedure 2016-05-20 07:03:11 -07:00
Matteo Bertozzi 7a84ee262f HBASE-15864 Reuse the testing helper to wait regions in transition 2016-05-20 06:48:38 -07:00
eshcar a27504c701 HBASE-14920: Compacting memstore
Signed-off-by: stack <stack@apache.org>
2016-05-20 03:41:43 -07:00
Jerry He af5146ee04 HBASE-15465 userPermission returned by getUserPermission() for the selected namespace does not have namespace set (li xiang) 2016-05-19 20:35:58 -07:00