Commit Graph

1043 Commits

Author SHA1 Message Date
tedyu 050c13e83f HBASE-15479 No more garbage or beware of autoboxing (Vladimir Rodionov) 2016-03-19 17:23:29 -07:00
Jerry He 8151503fa0 HBASE-14963 Remove use of Guava Stopwatch from HBase client code (Devaraj Das) 2016-03-19 13:23:05 -07:00
Mikhail Antonov a317a0f89c HBASE-15390 Unnecessary MetaCache evictions cause elevated number of requests to meta (ADDENDUM) 2016-03-17 01:09:10 -07:00
Elliott Clark aaaae83423 HBASE-15441 Fix WAL splitting when region has moved multiple times
Summary:
Currently WAL splitting is broken when a region has been opened multiple times in recent minutes.

Region open and region close write event markers to the wal. These markers should have the sequence id in them. However it is currently getting 1. That means that if a region has moved multiple times in the last few mins then multiple split log workers will try and create the recovered edits file for sequence id 1. One of the workers will fail and on failing they will delete the recovered edits. Causing all split wal attempts to fail.

We need to:

It appears that the close event with a sequence id of one is coming from region warm up.

This patch fixes that by making sure the close on warm up doesn't happen. Also splitting will ignore any of the events that are already in the logs.

Test Plan: Unit tests pass

Differential Revision: https://reviews.facebook.net/D55557
2016-03-16 16:35:18 -07:00
Mikhail Antonov 94d576025f HBASE-15390 Unnecessary MetaCache evictions cause elevated number of requests to meta 2016-03-16 16:32:11 -07:00
anoopsjohn 75547a42b9 HBASE-15322 Operations using Unsafe path broken for platforms not having sun.misc.Unsafe. 2016-03-11 09:38:24 +05:30
tedyu e103f75ae7 HBASE-15378 Scanner cannot handle heartbeat message with no results (Phil Yang) 2016-03-09 10:05:06 -08:00
Mikhail Antonov 3339bac563 HBASE-15137 CallTimeoutException and CallQueueTooBigException should trigger PFFE 2016-03-08 02:59:20 -08:00
tedyu 4ec6ad12c9 HBASE-15243 Utilize the lowest seek value when all Filters in MUST_PASS_ONE FilterList return SEEK_NEXT_USING_HINT 2016-03-07 19:28:21 -08:00
Mikhail Antonov 43698b3fcb HBASE-15354 Use same criteria for clearing meta cache for all operations (Ashu Pachauri) 2016-03-07 17:34:11 -08:00
tedyu c1524485c5 HBASE-15393 Enable table replication command will fail when parent znode is not default in peer cluster (Ashish Singhi) 2016-03-07 04:57:55 -08:00
tedyu 42c3fcabc0 HBASE-15397 Create bulk load replication znode(hfile-refs) in ZK replication queue by default (Ashish Singhi) 2016-03-04 13:49:00 -08:00
chenheng 18194f8a56 HBASE-15128 Disable region splits and merges switch in master 2016-02-29 20:41:27 +08:00
Mikhail Antonov 998e339d6e HBASE-15342 create branch-1.3 and update branch-1 poms to 1.4.0-SNAPSHOT 2016-02-28 16:23:29 -08:00
Josh Elser d747188f2c HBASE-15232 Handle region location cache mgmt in AsyncProcess for multi()'s
Further investigation after HBASE-15221 lead to some findings that
AsyncProcess should have been managing the contents of the region
location cache, appropriately clearing it when necessary (e.g. an
RPC to a server fails because the server doesn't host that region)

For multi() RPCs, the tableName argument is null since there is no
single table that the updates are destined to. This inadvertently
caused the existing region location cache updates to fail on 1.x
branches. AsyncProcess needs to handle when tableName is null
and perform the necessary cache evictions.

As such, much of the new retry logic in HTableMultiplexer is
unnecessary and is removed with this commit. Getters which were
added as a part of testing were left since that are mostly
harmless and should contain no negative impact.

Signed-off-by: stack <stack@apache.org>
2016-02-23 10:28:33 -08:00
anoopsjohn cc45b4ab38 HBASE-15247 InclusiveStopFilter does not respect reverse Filter property. (Amal Joshy) 2016-02-23 09:59:35 +05:30
Jonathan M Hsieh 7b4646fe3d HBASE-15285 ADDENDUM make RETURN_RESULTS attribute name protected to match branch-1.1 and deprecated to warn of 2.0 (Sean Busbey) 2016-02-18 07:17:10 -08:00
stack 23917519e8 HBASE-15285 Forward-port respect for isReturnResult from HBASE-15095 2016-02-17 23:34:42 -06:00
Jonathan M Hsieh 2e1a3ef644 HBASE-15283 Revert to IOException in TimeRange constructor to maintain API compat in 1.x line
[branch-1 and branch-1.2 only] HBASE-14355 changed the type of exception thrown if an invalid TimeRange is specified.  This reverts to 1.1.x semantics.
2016-02-17 15:57:54 -08:00
anoopsjohn 7e5ce6b0a3 HBASE-15198 RPC client not using Codec and CellBlock for puts by default-addendum. 2016-02-12 07:35:42 +05:30
anoopsjohn 19f8faeb88 HBASE-15198 RPC client not using Codec and CellBlock for puts by default. 2016-02-11 16:37:20 +05:30
Josh Elser 01b73e9877 HBASE-15221 Reload the cache on re-tried puts in HTableMultiplexer and adds a close() method to HTableMultiplexer
When a Put fails due to a NotServingRegionException, the cached location
of that Region is never cleared. Thus, subsequent calls to resubmit
the Put will fail in the same way as the original, never determining
the new location of the Region.

If the Connection is not closed by the user before the Multiplexer
is discarded, it will leak resources and could cause resource
issues.

Signed-off-by: Sean Busbey <busbey@cloudera.com>
2016-02-08 00:07:08 -06:00
anoopsjohn 64c87f2476 HBASE-15214 Valid mutate Ops fail with RPC Codec in use and region moves across. 2016-02-06 02:41:29 +05:30
Andrew Purtell 1b420be56a Amend HBASE-15200 ZooKeeper znode ACL checks should only compare the shortname
Fixes for newly introduced FindBugs warnings
2016-02-04 17:33:29 -08:00
Enis Soztutar 73d6778820 HBASE-15177 Reduce garbage created under high load
Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
2016-02-04 13:26:22 -08:00
Andrew Purtell 360bb62469 HBASE-15200 ZooKeeper znode ACL checks should only compare the shortname 2016-02-03 10:43:29 -08:00
Enis Soztutar f1e7c06539 HBASE-15129 Set default value for hbase.fs.tmp.dir rather than fully depend on hbase-default.xml (Yu Li)
Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/security/SecureBulkLoadUtil.java
2016-02-02 17:06:20 -08:00
tedyu 486f7612be HBASE-15173 Execute mergeRegions RPC call as the request user 2016-01-28 10:02:49 -08:00
Elliott Clark 421fe24e9b HBASE-15146 Don't block on Reader threads queueing to a scheduler queue 2016-01-28 08:11:11 -05:00
ramkrishna bc0e5fc048 HBASE-14970 Backport HBASE-13082 and its sub-jira to branch-1 - recommit
(Ram)
2016-01-25 08:04:22 +05:30
stack a5228a0b4d Revert "HBASE-14970 Backport HBASE-13082 and its sub-jira to branch-1 (Ram)"
Causes TestHFileOutputFormat to fail every time. Messes up our 1.3 CI.

This reverts commit 58521869b0.
2016-01-22 10:45:51 -08:00
stack a9c008344f HBASE-15091 Forward-port to 1.2 HBASE-15031 "Fix merge of MVCC and SequenceID performance regression in branch-1.0" 2016-01-21 10:26:46 -08:00
stack d965d14a63 Revert "HBASE-14865 Support passing multiple QOPs to SaslClient/Server via hbase.rpc.protection (Apply)"
Needs newer Hadoop with n HADOOP-10786/HADOOP-11287 (courtesy of Matteo)

This reverts commit e76a2e4e6d.
2016-01-21 08:27:06 -08:00
ramkrishna 58521869b0 HBASE-14970 Backport HBASE-13082 and its sub-jira to branch-1 (Ram) 2016-01-21 21:22:40 +05:30
Mikhail Antonov cea43788fe HBASE-15139 Connection manager doesn't pass client metrics to RpcClient 2016-01-20 14:20:26 -08:00
tedyu e76a2e4e6d HBASE-14865 Support passing multiple QOPs to SaslClient/Server via hbase.rpc.protection (Apply) 2016-01-19 13:08:24 -08:00
tedyu 7a096ac6cb HBASE-15073 Revert due to different opinion on usefulness 2016-01-18 17:35:18 -08:00
stack d9fd87d5a8 HBASE-15115 Fix findbugs complaints in hbase-client 2016-01-16 16:14:13 -08:00
tedyu 53983f5b2e HBASE-15073 Finer grained control over normalization actions for RegionNormalizer 2016-01-06 15:38:46 -08:00
tedyu 2797adf45f HBASE-14987 Compaction marker whose region name doesn't match current region's needs to be handled 2016-01-04 07:19:13 -08:00
Ashish Singhi 64553236a9 HBASE-15018 Inconsistent way of handling TimeoutException in the rpc client implementations
Signed-off-by: Sean Busbey <busbey@apache.org>
2015-12-30 09:18:48 -06:00
tedyu afaa7f843a HBASE-14717 enable_table_replication command should only create specified table for a peer cluster (Ashish) 2015-12-24 11:18:32 -08:00
anoopsjohn 4a7565af9c HBASE-14940 Make our unsafe based ops more safe. 2015-12-24 07:56:27 +05:30
stack fdeca854ec Revert "HBASE-15018 Inconsistent way of handling TimeoutException in the rpc client implemenations (Ashish Singhi)"
This reverts commit 59cca6297f.
2015-12-23 15:31:35 -08:00
stack 59cca6297f HBASE-15018 Inconsistent way of handling TimeoutException in the rpc client implemenations (Ashish Singhi) 2015-12-23 07:29:44 -08:00
anoopsjohn b7100c934b HBASE-13158 When client supports CellBlock, return the result Cells as controller payload for get(Get) API also - Addendum 2015-12-22 09:58:12 +05:30
Lars Hofhansl c27d2c9c3e HBASE-14822 Renewing leases of scanners doesn't work. 2015-12-19 10:15:26 -08:00
anoopsjohn 574774d8b1 HBASE-13158 When client supports CellBlock, return the result Cells as controller payload for get(Get) API also. 2015-12-19 14:58:52 +05:30
Enis Soztutar f3ebeeb3fc HBASE-14989 Implementation of Mutation.getWriteToWAL() is backwards 2015-12-17 11:48:14 -08:00
anoopsjohn a93c0e822e HBASE-14999 Remove ref to org.mortbay.log.Log. 2015-12-17 23:33:39 +05:30
stack 2f7d5e6354 HBASE-14946 Don't allow multi's to over run the max result size. 2015-12-10 21:25:46 -08:00
Enis Soztutar aeb6ae97d3 HBASE-14960 Fallback to using default RPCControllerFactory if class cannot be loaded
Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRpcControllerFactory.java
2015-12-10 19:13:14 -08:00
Elliott Clark 8508dd07ff HBASE-14946 Don't allow multi's to over run the max result size.
Summary:
* Add VersionInfoUtil to determine if a client has a specified version or better
* Add an exception type to say that the response should be chunked
* Add on client knowledge of retry exceptions
* Add on metrics for how often this happens

Test Plan: Added a unit test

Differential Revision: https://reviews.facebook.net/D51771
2015-12-10 18:31:05 -08:00
ramkrishna 65117d3d04 HBASE-13153 Bulk Loaded HFile Replication (Ashish Singhi) 2015-12-10 13:10:41 +05:30
Gary Helmling 5dec5ad250 HBASE-14866 VerifyReplication and ReplicationAdmin should use full peer configuration for peer connection 2015-12-09 16:47:25 -08:00
tedyu 66adeb3048 HBASE-14894 Fix misspellings of threshold in log4j.properties files for tests (Youngjoon Kim) 2015-11-28 10:10:27 -08:00
tedyu 447a0e7b7e HBASE-14737 Clear cachedMaxVersions when HColumnDescriptor#setValue(VERSIONS, value) is called (Pankaj Kumar) 2015-11-23 20:26:18 -08:00
chenheng 40e01512cf HBASE-14782 FuzzyRowFilter skips valid rows (Vladimir Rodionov) 2015-11-19 11:41:05 +08:00
tedyu a6e0ad636a HBASE-14824 HBaseAdmin.mergeRegions should recognize both full region names and encoded region names (Eungsop Yoo) 2015-11-18 14:26:31 -08:00
Elliott Clark e40ed0e836 HBASE-14812 Fix ResultBoundedCompletionService deadlock 2015-11-17 13:48:50 -08:00
Elliott Clark 4d5ac316b6 HBASE-14708 Use copy on write Map for region location cache
Summary:
Create and use a copy on write map for region location.

  - Create a copy on write map backed by a sorted array.
  - Create a test for both comparing each with a jdk provided map.
  - Change MetaCache to use the new map.

Test Plan:
  - org.apache.hadoop.hbase.client.TestFromClientSide
  - TestHCM

Differential Revision: https://reviews.facebook.net/D49545
2015-11-17 11:01:52 -08:00
stack 336eb92016 HBASE-14355 Scan different TimeRange for each column family 2015-11-13 22:44:05 -08:00
tedyu 3e551ea538 HBASE-14498 Revert for on-going review 2015-11-12 12:55:44 -08:00
tedyu c36b9489aa HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable (Pankaj Kumar) 2015-11-12 07:22:42 -08:00
chenheng 7330e11aa4 HBASE-14693 Add client-side metrics for received pushback signals
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientPushback.java
2015-11-09 15:32:28 -08:00
tedyu f3af7b642c HBASE-14758 Add UT case for unchecked error/exception thrown in AsyncProcess#sendMultiAction (Yu Li) 2015-11-04 07:29:01 -08:00
chenheng fa2d0859f7 HBASE-12986 Compaction pressure based client pushback
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
2015-11-03 18:40:59 -08:00
stack 759252027d HBASE-14754 TestFastFailWithoutTestUtil failing on trunk now in #testPreemptiveFastFailException50Times 2015-11-03 16:21:23 -08:00
Jesse Yates ed3b6bda4b HBASE-14715 Add javadocs to DelegatingRetryingCallable 2015-11-03 13:45:05 -08:00
stack 51f5412a92 Revert "HBASE-14725 Vet categorization of tests so they for sure go into the right small/medium/large buckets"
Reverting. It seems to have destabilized the build.

This reverts commit aaaa813225.
2015-11-02 08:16:59 -08:00
stack aaaa813225 HBASE-14725 Vet categorization of tests so they for sure go into the right small/medium/large buckets 2015-11-01 22:55:59 -08:00
Elliott Clark 3bdfa782f3 HBASE-14687 Un-synchronize BufferedMutator
Summary:
Use concurrent collections and atomic longs to keep track of edits in buffered mutator.
This keeps buffered mutator as thread safe but it means that shared buffered mutators are not contending on thread locking.

Test Plan: Unit Tests.

Differential Revision: https://reviews.facebook.net/D49467
2015-10-30 13:39:38 -07:00
Matteo Bertozzi 569fe82acf HBASE-14714 some cleanup to snapshot code 2015-10-29 14:04:48 -07:00
tedyu 3c8e92019c HBASE-12769 Replication fails to delete all corresponding zk nodes when peer is removed (Jianwei Cui) 2015-10-28 15:43:26 -07:00
Enis Soztutar c174a54d87 HBASE-14425 In Secure Zookeeper cluster superuser will not have sufficient permission if multiple values are configured in hbase.superuser (Pankaj Kumar)
Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
2015-10-27 17:00:46 -07:00
tedyu 8fc9c2803f HBASE-14696 Support setting allowPartialResults in mapreduce Mappers 2015-10-27 11:13:58 -07:00
tedyu 842593c439 HBASE-14694 Scan copy constructor doesn't handle allowPartialResults 2015-10-27 07:55:16 -07:00
Elliott Clark a268d151e2 HBASE-14683 Fix Batching in buffered mutator is awful when adding lists of mutations.
Summary: Send the list of mutations to AsyncProcess only after done adding the list otherwise there's a lot of contention

Test Plan: UnitTests.

Differential Revision: https://reviews.facebook.net/D49251
2015-10-24 10:03:03 -07:00
Enis Soztutar 6dc33cad75 HBASE-14474 Addendum closes connection in writeRequest() outside synchronized block 2015-10-16 12:00:03 -07:00
tedyu be04ede5b9 HBASE-14594 Use new DNS API introduced in HADOOP-12437 (Josh Elser) 2015-10-14 14:24:24 -07:00
stack 11066d0459 HBASE-14598 ByteBufferOutputStream grows its HeapByteBuffer beyond JVM limitations (Ian Friedman) 2015-10-14 13:13:35 -07:00
stack cab158173d Revert "HBASE-14583 Enabled client-side metrics by default"
Caused TestShell to hang. Reverting for now.

This reverts commit dced02ad9f.
2015-10-10 13:19:52 -07:00
anoopsjohn 90f0644d9a HBASE-14366 NPE in case visibility expression is not present in labels table during importtsv run. (Bhupendra) 2015-10-10 12:08:23 +05:30
stack cd298eae4f HBASE-14517 Show regionserver's version in master status page (Liu Shaohui) 2015-10-09 15:11:14 -07:00
Nick Dimiduk dced02ad9f HBASE-14583 Enabled client-side metrics by default 2015-10-09 10:30:50 -07:00
Nick Dimiduk a048c32a99 HBASE-12911 Client-side metrics 2015-10-07 17:05:51 -07:00
Elliott Clark ef8cc7a575 HBASE-14544 Allow HConnectionImpl to not refresh the dns on errors 2015-10-05 09:44:52 -07:00
Mikhail Antonov b9bfd6f7e3 HBASE-14367 Add normalization support to shell 2015-10-04 01:24:15 -07:00
smaddineni 373c75dde6 HBASE-13770 Programmatic JAAS configuration option for secure zookeeper may be broken
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/HQuorumPeer.java
2015-10-02 18:00:47 -07:00
Matteo Bertozzi afb3b19a15 HBASE-14485 ConnectionImplementation leaks on construction failure 2015-10-01 12:50:26 -07:00
stack 16ad5e61c8 HBASE-14510 Fix coprocessor handling of duplicate classes (Yerui Sun) 2015-09-29 15:35:44 -07:00
Enis Soztutar bc1a03d9b4 HBASE-14474 DeadLock in RpcClientImpl.Connection.close() 2015-09-25 11:05:26 -07:00
Andrew Purtell 12e2341f3d HBASE-13324 o.a.h.h.Coprocessor should be LimitedPrivate("Coprocessor") 2015-09-24 10:57:39 -07:00
Stephen Yuan Jiang e38fd84d3f HBASE-14107 Administrative Task: Provide an API to List all procedures (Stephen Yuan Jiang) 2015-09-24 07:17:23 -07:00
tedyu 992856c11d HBASE-14397 Revert pending review comments 2015-09-22 10:00:11 -07:00
tedyu ca6c7f0a68 HBASE-14431 Addendum checks for null connectionInPool (Yu Li) 2015-09-21 07:23:47 -07:00
tedyu 88adccd553 HBASE-14431 AsyncRpcClient#removeConnection() never removes connection from connections pool if server fails (Samir Ahmic) 2015-09-19 07:33:26 -07:00
tedyu 00f467b225 HBASE-14449 Rewrite deadlock prevention for concurrent connection close 2015-09-18 14:47:58 -07:00
stack 82554e2750 HBASE-14433 Set down the client executor core thread count from 256 in tests: REAPPLY AGAIN (WAS MISSING JIRA) 2015-09-16 10:15:00 -07:00
stack b6549007b3 Revert "Set down the client executor core thread count from 256 in tests: REAPPLY"
Missing JIRA. Will reapply in a sec with commit log fixed.

This reverts commit c100fb835a.
2015-09-16 10:14:32 -07:00