Commit Graph

95 Commits

Author SHA1 Message Date
Lars Hofhansl 254af5a321 Merge branch 'branch-1' of https://git-wip-us.apache.org/repos/asf/hbase into branch-1 2016-01-05 15:54:49 -08:00
Elliott Clark 8508dd07ff HBASE-14946 Don't allow multi's to over run the max result size.
Summary:
* Add VersionInfoUtil to determine if a client has a specified version or better
* Add an exception type to say that the response should be chunked
* Add on client knowledge of retry exceptions
* Add on metrics for how often this happens

Test Plan: Added a unit test

Differential Revision: https://reviews.facebook.net/D51771
2015-12-10 18:31:05 -08:00
ramkrishna 65117d3d04 HBASE-13153 Bulk Loaded HFile Replication (Ashish Singhi) 2015-12-10 13:10:41 +05:30
Lars Hofhansl fba9e49dcf HBASE-14869 Better request latency and size histograms. (Vikas Vishwakarma and Lars Hofhansl)
Conflicts:

	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java
2015-12-08 17:07:45 -08:00
Vrishal Kulkarni 2e5499ed6c HBASE-14719 Add metrics for master WAL count (numMasterWALs). Metric numMasterWALs appears as follows in metrics dump
{
    "name" : "Hadoop:service=HBase,name=Master,sub=Procedure",
    "modelerType" : "Master,sub=Procedure",
    "tag.Context" : "master",
    "tag.Hostname" : "vrishal-mbp",
    "numMasterWALs" : 1
},

Signed-off-by: Elliott Clark <eclark@apache.org>
2015-12-07 11:15:25 -08:00
Sanjeev Lakshmanan 2ce27951b0 HBASE-14862 Add support for reporting p90 for histogram metrics
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2015-11-23 15:56:07 -08:00
Elliott Clark 604c9b2cca HBASE-14793 Allow limiting size of block into L1 block cache. 2015-11-17 10:38:02 -08:00
Elliott Clark e8e0e86b49 HBASE-14778 Make block cache hit percentages not integer in the metrics system 2015-11-10 12:28:24 -08:00
stack 51f5412a92 Revert "HBASE-14725 Vet categorization of tests so they for sure go into the right small/medium/large buckets"
Reverting. It seems to have destabilized the build.

This reverts commit aaaa813225.
2015-11-02 08:16:59 -08:00
stack aaaa813225 HBASE-14725 Vet categorization of tests so they for sure go into the right small/medium/large buckets 2015-11-01 22:55:59 -08:00
Gary Helmling 85f2aee070 HBASE-14700 Support a permissive mode for secure clusters to allow SIMPLE auth clients 2015-10-30 20:06:23 -07:00
Sean Busbey 436bd5e823 HBASE-14516 categorize hadoop-compat tests
* make sure the test classifications are in test scope for their use in the hadoop-compat modules
* added a test category for 'metrics related' since that's what all these tests are for
* categorized tests as small,metrics

Conflicts:
	hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/master/TestMetricsMasterSourceImpl.java
2015-10-03 01:15:52 -05:00
Sanjeev Srivatsa d047c37871 HBASE-14459 Add response and request size metrics
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2015-09-30 18:23:17 -07:00
Andrew Purtell fc79ba338a HBASE-14205 RegionCoprocessorHost System.nanoTime() performance bottleneck 2015-09-24 11:16:47 -07:00
Enis Soztutar bb4a690b79 HBASE-14082 Add replica id to JMX metrics names (Lei Chen) 2015-09-16 17:43:32 -07:00
tedyu 00bdf14f96 HBASE-14314 Metrics for block cache should take region replicas into account 2015-09-09 13:32:02 -07:00
Elliott Clark d6c2beb4bf HBASE-14166 Per-Region metrics can be stale 2015-08-17 11:22:21 -07:00
tedyu 6a2b618d97 HBASE-13965 Stochastic Load Balancer JMX Metrics (Lei Chen) 2015-08-05 19:22:44 -07:00
Sean Busbey a45651c2cf HBASE-14085 Update LICENSE and NOTICE files.
* corrects license/notice for source distribution
* adds inception year to correct copyright in generated NOTICE files for jars
* updates project names in poms to use "Apache HBase" instead of "HBase" so jar NOTICE files will be correct
* uses append-resources to include supplemental info on jars with 3rd party works in source
* adds an hbase specific resource bundle for jars that include 3rd party works for binaries
** uses supplemental-model to fill in license gaps
** uses the above and a shade plugin transformation to build proper files for shaded jars.
** uses the above and the assembly plugin to build the proper files for bin assembly
* adds a NOTICE item for things copied out of Hadoop (TODO legal-discuss)

Conflicts:
	NOTICE.txt
	hbase-checkstyle/pom.xml
	hbase-resource-bundle/pom.xml
	hbase-server/pom.xml
	hbase-thrift/pom.xml
	pom.xml
2015-08-05 14:52:33 -07:00
tedyu 24dbe25e95 HBASE-13965 Revert due to test failure in TestAssignmentManager 2015-08-03 15:32:43 -07:00
tedyu c215b900f4 HBASE-13965 Stochastic Load Balancer JMX Metrics (Lei Chen) 2015-08-03 12:50:44 -07:00
Andrew Purtell 9227c2b99d HBASE-14176 Add missing headers to META-INF files 2015-07-31 11:22:50 -07:00
Sean Busbey f2f19ddb5c HBASE-13909 update branch-1 poms to 1.3.0-SNAPSHOT 2015-06-15 22:30:56 -05:00
Gábor Lipták 0a0ff3354a HBASE-13569 Correct Javadoc (for Java8)
Signed-off-by: Sean Busbey <busbey@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterBase.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileScanner.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/util/BloomFilterWriter.java
2015-06-13 22:23:53 -05:00
Enis Soztutar 77b155858f HBASE-13661 Correct binary compatibility issues discovered in 1.1.0RC0 2015-05-11 18:33:44 -07:00
Elliott Clark 91e09bc0ab HBASE-13477 Create metrics on failed requests
Summary: Add metrics on how many requests are exceptions and what type.

Test Plan: behold unit tests.

Differential Revision: https://reviews.facebook.net/D37167
2015-04-17 15:38:38 -07:00
Nick Dimiduk cf45c8d30a Bump pom version to 1.2.0-SNAPSHOT 2015-04-16 17:30:48 -07:00
Jonathan M Hsieh 8dbe956eb4 HBASE-13289 typo in splitSucessCount metric
In hbase 1.0.0, 0.98.10, 0.98.10.1, 0.98.11, and 0.98.12 'splitSuccessCount' was misspelled as 'splitSuccessCounnt'
2015-04-07 14:41:29 -07:00
tedyu bb65a8664d HBASE-13369 Expose scanNext stats to region server level 2015-03-31 15:27:39 -07:00
tedyu f1601c0d12 HBASE-13213 Split out locality metrics among primary and secondary region 2015-03-31 15:00:56 -07:00
nke 797eee33ab HBASE-8725 Add total time RPC call metrics
Signed-off-by: Elliott Clark <eclark@apache.org>
2015-03-26 11:12:29 -07:00
Josh Elser c3b47f2251 HBASE-13236 Add addt'l lifecycle-mapping executions.
Adds a number of lifecycle-mapping entries which
prevent errors from showing up in Eclipse on a fresh
import of HBase. For plugins defined in the top-level
pom, the mapping is added there; otherwise, the mapping
is pushed down to the child pom.

Signed-off-by: Sean Busbey <busbey@apache.org>
2015-03-14 00:19:44 -05:00
Ashish Singhi f5b40200db HBASE-9531 a command line (hbase shell) interface to retreive the replication metrics and show replication lag
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSmallTests.java
2015-02-12 14:16:28 -08:00
Lars Hofhansl e491f337f1 HBASE-12779 SplitTransaction: Add metrics. 2015-01-05 17:14:20 -08:00
Enis Soztutar d66ae28c07 Update pom.xml version for 1.1.0-SNAPSHOT for branch-1 2014-12-17 23:16:41 -08:00
stack f0afcd2cc2 HBASE-12689 Move version on from 0.99.2 to HBASE-1.0.0-SNAPSHOT 2014-12-14 20:37:59 -08:00
Sean Busbey 2372739e46 HBASE-12661 add missing licenses. 2014-12-09 09:52:55 -06:00
Elliott Clark cc1a0d7c36 HBASE-12574 Update replication metrics to not do so many map look ups. 2014-12-08 10:58:21 -08:00
Enis Soztutar 5f150e49f9 Update pom.xml version for 0.99.2 2014-12-02 20:28:29 -08:00
Elliott Clark 091142f47a HBASE-12576 Add metrics for rolling the HLog if there are too few DN's in the write pipeline 2014-12-02 09:30:29 -08:00
Sean Busbey 8959828fb5 HBASE-12522 Backport of write-ahead-log refactoring and follow-ons.
Issues incorporated and incompatible changes called out on the jira.

* Cleaned up references to HLog
* Deprecates HLogKey but maintains it for compatibility
  - Moves all Writeable from WALKey to HLogKey
* Adds utility code to CoprocessorHost to help with evolving Coprocessor APIs
* RSRpcServices roll WAL call now requests the non-meta LogRoller roll all logs
  - rolls actually happen asynchronously
  - deprecated old api (and noted incompatible behavior change)
  - modified api in new Admin interface to reflect lack of return values.
* Moved WAL user facing API to "WAL"
  - only 1 sync offered
  - WALTrailer removed from API
* make provider used by the WALFactory configurable.
* Move all WAL requests to use opaque ids instead of paths
* WALProvider provides API details for implementers and handles creation of WALs.
* Refactor WALActionsListener to have a basic implementation.
* turn MetricsWAL into a WALActionsListener.
* tests that needs FSHLog implementation details use them directly, others just reference provider + factory
  - Some tests moved from Large to Medium based on run time.
* pull out wal disabling into its own no-op class
* update region open to delegate to WALFactory
* update performance test tool to allow for multiple regions
* Removed references to meta-specific wals within wal code
  - replaced with generic suffixes
  - WALFactory maintains a dedicated WALProvider for meta (and so knows about the distinction)
* maintain backwards compat on HLogPrettyPrinter and mark it deprecated.
  - made WALPrettyPrinter IA.Private in favor of `bin/hbase wal`
* move WALUtil stuff that's implementation specific to said implementation
  - WALUtil now acts as an integration point between the RegionServer and hte WAL code.

Incorporates contributions from v.himanshu and stack.

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/migration/NamespaceUpgrade.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/util/HMerge.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/util/MetaUtils.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestIOFencing.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverScannerOpenHook.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestInvocationRecordFilter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/fs/TestBlockReorder.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransaction.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollingNoCluster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestMultiSlaveReplication.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationEndpoint.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSource.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpoint.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationWALReaderManager.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestFSVisitor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSecureWAL.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALFiltering.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALMethods.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALReaderOnSecureWAL.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALSplit.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALSplitCompressed.java
	src/main/docbkx/ops_mgt.xml
2014-12-02 11:05:02 -06:00
Andrew Purtell eb34c956a1 HBASE-12496 A blockedRequestsCount metric (Yi Deng) 2014-11-18 14:24:18 -08:00
Andrew Purtell 3eed03268f HBASE-12424 Finer grained logging and metrics for split transactions
Conflicts:
	hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitRequest.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
2014-11-07 11:44:49 -08:00
stack c99d89fd63 HBASE-12427 Change branch-1 version from 0.99.2-SNAPSHOT to 0.99.3-SNAPSHOT --REVERT 2014-11-05 19:15:24 -08:00
stack c66a6b02a5 HBASE-12427 Change branch-1 version from 0.99.2-SNAPSHOT to 0.99.3-SNAPSHOT 2014-11-04 12:41:38 -08:00
stack bca92603e0 HBASE-12328 Need to separate JvmMetrics for Master and RegionServer (Sanghyun Yun) 2014-10-23 21:50:09 -07:00
Enis Soztutar 10ebc49130 Update pom.xml version to 0.99.2-SNAPSHOT 2014-10-17 11:32:59 -07:00
stack 8828e0a680 HBASE-12246 Compilation with hadoop-2.3.x and 2.2.x is broken -> HBASE-12220 Add hedgedReads and hedgedReadWins metrics -- REVERT 2014-10-13 15:04:34 -07:00
Enis Soztutar 7d0be32b1d Update pom.xml version for 0.99.1 2014-10-11 21:56:30 -07:00
stack 71ed703367 Add hedgedReads and hedgedReadWins count metrics
Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java
2014-10-09 15:02:50 -07:00