Commit Graph

11808 Commits

Author SHA1 Message Date
Enis Soztutar afdfd1bd9c HBASE-15295 MutateTableAccess.multiMutate() does not get high priority causing a deadlock - ADDENDUM for failing test 2016-03-29 15:02:18 -07:00
chenheng 7f39baf0f4 HBASE-11393 Replication TableCfs should be a PB object rather than a string 2016-03-29 10:25:29 +08:00
Enis Soztutar 0520097611 HBASE-15295 MutateTableAccess.multiMutate() does not get high priority causing a deadlock 2016-03-28 17:56:32 -07:00
tedyu cbf9c1e116 HBASE-15191 CopyTable and VerifyReplication - Option to specify batch size, versions (Parth Shah) 2016-03-28 17:25:00 -07:00
tedyu 7793bc54c2 HBASE-15548 SyncTable: sourceHashDir is supposed to be optional but won't work without (Dave Latham) 2016-03-28 16:52:44 -07:00
zhangduo c96b642f15 HBASE-15265 Implement an asynchronous FSHLog 2016-03-26 15:41:27 +08:00
Sean Busbey 1a9837ab74 HBASE-15494 Close obviated PRs on the GitHub mirror.
- closes #1 HBASE-1015 obviated by HBASE-14850
  - closes #3 obviated by HBASE-15059
  - closes #17 obviated by HBASE-15223

Signed-off-by: Elliott Clark <eclark@apache.org>
2016-03-25 22:21:22 -05:00
tedyu b865fb9434 HBASE-15496 Throw RowTooBigException only for user scan/get (Guanghao Zhang) 2016-03-25 18:18:35 -07:00
zhangduo 11d11d3fcc HBASE-15389 Write out multiple files when compaction 2016-03-25 15:07:54 +08:00
chenheng e9c4f12856 HBASE-15384 Avoid using '/tmp' directory in TestBulkLoad 2016-03-25 12:06:55 +08:00
tedyu 07c597c8a2 HBASE-15526 Make SnapshotManager accessible through MasterServices 2016-03-24 15:59:46 -07:00
tedyu ad3feaa448 HBASE-15515 Improve LocalityBasedCandidateGenerator in Balancer (Guanghao Zhang) 2016-03-24 11:10:58 -07:00
Jerry He 7c93098210 HBASE-14256 Flush task message may be confusing when region is recovered (Gabor Liptak) 2016-03-24 10:06:38 -07:00
Elliott Clark e71bc71c84 HBASE-15475 Addendum 2016-03-24 09:11:19 -07:00
tedyu 52fd70500e HBASE-15486 Avoid multiple disable/enable balancer calls while running rolling-restart.sh --graceful (Samir Ahmic) 2016-03-24 06:55:45 -07:00
Mikhail Antonov db3ba652f8 HBASE-15524 Fix NPE in client-side metrics 2016-03-23 21:23:54 -07:00
zhangduo fd5c0934b6 HBASE-15520 Fix broken TestAsyncIPC 2016-03-24 10:46:44 +08:00
Elliott Clark 925c185969 HBASE-15475 Allow TimestampsFilter to provide a seek hint
Summary:
Allow TestTimestampFilterSeekHint to provide a seek next hint.
This can be incorrect as it might skip deletes. However it can
make things much much faster.

Test Plan: Added a unit test.

Differential Revision: https://reviews.facebook.net/D55617
2016-03-23 18:56:52 -07:00
stack d7a4499dfc HBASE-15392 Single Cell Get reads two HFileBlocks
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
moreRowsMayExistAfterCell Exploit the fact a Scan is a Get Scan. Also save compares
if no non-default stopRow.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
optimize Add doc on what is being optimized. Also, if a Get Scan, do not
optimize else we'll keep going after our row is DONE.
Another place to make use of the Get Scan fact is when we are DONE.. if
Get Scan, we can close out the scan.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java
Add tests for Get Scans and optimize around block loading.
2016-03-23 14:37:14 -07:00
tedyu 95e6d2276b HBASE-15300 Upgrade to zookeeper 3.4.8 2016-03-23 13:05:17 -07:00
Matteo Bertozzi 7177e4be64 HBASE-15508 Add command for exporting snapshot in hbase command script (Yufeng Jiang) 2016-03-23 10:26:32 -07:00
anoopsjohn cadfb21f4b HBASE-15064 BufferUnderflowException after last Cell fetched from an HFile Block served from L2 offheap cache - Addendum. 2016-03-23 15:03:50 +05:30
zhangduo 5fcadb86ab HBASE-15495 Connection leak in FanOutOneBlockAsyncDFSOutputHelper 2016-03-23 09:56:40 +08:00
stack 12f66e3060 HBASE-15477 Purge 'next block header' from cached blocks
When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body.  We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting: our block plus the next blocks'
header (33 bytes).

This patch undoes this over-persisting.

Removes support for version 1 blocks (0.2 was added in hbase-0.92.0).
Not needed any more.

There is an open question on whether checksums should be persisted
when caching. The code seems to say no but if cache is SSD backed or
backed by anything that does not do error correction, we'll want
checksums.

Adds loads of documentation.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
  (write) Add writing from a ByteBuff.

M hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java
  (toString) Add one so ByteBuff looks like ByteBuffer when you click on
  it in IDE

M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
  Remove support for version 1 blocks.

  Cleaned up handling of metadata added when we serialize a block to
  caches. Metadata is smaller now.

  When we serialize (used when caching), do not persist the next blocks
  header if present.

  Removed a bunch of methods, a few of which had overlapping
  functionality and others that exposed too much of our internals.
  Also removed a bunch of constructors and unified the constructors we
  had left over making them share a common init method.
  Shutdown access to defines that should only be used internally here.

  Renamed all to do w/ 'EXTRA' and 'extraSerialization' to instead talk
  about metadata saved to caches; was unclear previously what EXTRA was
  about.

  Renamed static final declarations as all uppercase.

  (readBlockDataInternal): Redid. Couldn't make sense of it previously.
  Undid heavy-duty parse of header by constructing HFileBlock. Other
  cleanups. Its 1/3rd the length it used to be. More to do in here.
2016-03-22 18:45:17 -07:00
stack a13d6e000d Revert "HBASE-15392 Single Cell Get reads two HFileBlocks M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java moreRowsMayExistAfterCell Exploit the fact a Scan is a Get Scan. Also save compares if no non-default stopRow."
Revert mistaken commit

This reverts commit 7073f69993.
2016-03-22 18:39:22 -07:00
stack c49d0caf53 Revert "HBASE-15477 Purge 'next block header' from cached blocks"
Overcommit. Revert.

This reverts commit ef94b55239.
2016-03-22 18:38:29 -07:00
stack 54a543de22 Revert "HBASE-15477 Purge 'next block header' from cached blocks"
Overcommit. Revert to fix.

This reverts commit 000117ad9f.
2016-03-22 18:37:25 -07:00
Enis Soztutar 3f3613a234 HBASE-15212 RRCServer should enforce max request size 2016-03-22 16:23:15 -07:00
stack 8af9ed7433 HBASE-15416 TestHFileBackedByBucketCache is flakey since it went in 2016-03-22 15:44:25 -07:00
Enis Soztutar b3fe4ed16c HBASE-15412 Add average region size metric (Alicia Ying Shu) 2016-03-22 14:46:27 -07:00
zhangduo 2348478506 HBASE-15360 addendum fix testCoDelScheduling
Signed-off-by: stack <stack@apache.org>
2016-03-22 13:52:26 -07:00
stack 000117ad9f HBASE-15477 Purge 'next block header' from cached blocks
When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body.  We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting: our block plus the next blocks'
header (33 bytes).

This patch undoes this over-persisting.

Removes support for version 1 blocks (0.2 was added in hbase-0.92.0).
Not needed any more.

There is an open question on whether checksums should be persisted
when caching. The code seems to say no but if cache is SSD backed or
backed by anything that does not do error correction, we'll want
checksums.

Adds loads of documentation.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
  (write) Add writing from a ByteBuff.

M hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java
  (toString) Add one so ByteBuff looks like ByteBuffer when you click on
  it in IDE

M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
  Remove support for version 1 blocks.

  Cleaned up handling of metadata added when we serialize a block to
  caches. Metadata is smaller now.

  When we serialize (used when caching), do not persist the next blocks
  header if present.

  Removed a bunch of methods, a few of which had overlapping
  functionality and others that exposed too much of our internals.
  Also removed a bunch of constructors and unified the constructors we
  had left over making them share a common init method.
  Shutdown access to defines that should only be used internally here.

  Renamed all to do w/ 'EXTRA' and 'extraSerialization' to instead talk
  about metadata saved to caches; was unclear previously what EXTRA was
  about.

  Renamed static final declarations as all uppercase.

  (readBlockDataInternal): Redid. Couldn't make sense of it previously.
  Undid heavy-duty parse of header by constructing HFileBlock. Other
  cleanups. Its 1/3rd the length it used to be. More to do in here.
2016-03-22 13:42:38 -07:00
stack ef94b55239 HBASE-15477 Purge 'next block header' from cached blocks
When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body.  We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting: our block plus the next blocks'
header (33 bytes).

This patch undoes this over-persisting.

Removes support for version 1 blocks (0.2 was added in hbase-0.92.0).
Not needed any more.

There is an open question on whether checksums should be persisted
when caching. The code seems to say no but if cache is SSD backed or
backed by anything that does not do error correction, we'll want
checksums.

Adds loads of documentation.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
  (write) Add writing from a ByteBuff.

M hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java
  (toString) Add one so ByteBuff looks like ByteBuffer when you click on
  it in IDE

M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
  Remove support for version 1 blocks.

  Cleaned up handling of metadata added when we serialize a block to
  caches. Metadata is smaller now.

  When we serialize (used when caching), do not persist the next blocks
  header if present.

  Removed a bunch of methods, a few of which had overlapping
  functionality and others that exposed too much of our internals.
  Also removed a bunch of constructors and unified the constructors we
  had left over making them share a common init method.
  Shutdown access to defines that should only be used internally here.

  Renamed all to do w/ 'EXTRA' and 'extraSerialization' to instead talk
  about metadata saved to caches; was unclear previously what EXTRA was
  about.

  Renamed static final declarations as all uppercase.

  (readBlockDataInternal): Redid. Couldn't make sense of it previously.
  Undid heavy-duty parse of header by constructing HFileBlock. Other
  cleanups. Its 1/3rd the length it used to be. More to do in here.
2016-03-22 13:34:51 -07:00
stack 891569bbd9 Add javadoc on how BBIOEngine works (by Anoop Sam John) 2016-03-22 13:01:13 -07:00
stack 7073f69993 HBASE-15392 Single Cell Get reads two HFileBlocks M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java moreRowsMayExistAfterCell Exploit the fact a Scan is a Get Scan. Also save compares if no non-default stopRow.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
optimize Add doc on what is being optimized. Also, if a Get Scan, do not
optimize else we'll keep going after our row is DONE.
Another place to make use of the Get Scan fact is when we are DONE.. if
Get Scan, we can close out the scan.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java
Add tests for Get Scans and optimize around block loading.
2016-03-22 13:00:41 -07:00
Misty Stanley-Jones b6e1f63074 HBASE-15447 Improve javadocs description for Delete methods <Wellington Chevreuil> 2016-03-22 12:56:30 -07:00
tedyu 47471c35e3 HBASE-15488 Add ACL for setting split merge switch 2016-03-22 10:38:13 -07:00
Enis Soztutar 797562e6c3 HBASE-15464 Flush / Compaction metrics revisited 2016-03-21 17:50:02 -07:00
Elliott Clark 75252af3a9 HBASE-15460 Fix infer issues in hbase-common
Summary: Fix issues found by static analysis

Test Plan: Unit test pass

Differential Revision: https://reviews.facebook.net/D55551
2016-03-21 11:51:50 -07:00
stack 1123be9181 HBASE-15502 Skeleton unit test to copy/paste 2016-03-21 11:27:18 -07:00
tedyu f1d453599a HBASE-15433 SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception (Jianwei Cui) 2016-03-21 03:46:23 -07:00
Sean Busbey f48c92d143 HBASE-15478 add comments to syncRunnerIndex handling explaining constraints on possible values.
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-03-21 00:28:15 -05:00
stack 64204b96c1 HBASE-15349 Update surefire version to 2.19.1; Trying a REVERT to see if this responsible for plethora of test timeouts on hadoopqa 2016-03-20 11:22:22 -07:00
tedyu dc6cd0bb74 HBASE-15479 No more garbage or beware of autoboxing (Vladimir Rodionov) 2016-03-19 17:22:53 -07:00
Yu Li d2d8058ab3 HBASE-15451 Remove unnecessary wait in MVCC 2016-03-19 21:24:07 +08:00
tedyu 3a6d683d63 HBASE-15456 CreateTableProcedure/ModifyTableProcedure needs to fail when there is no family in table descriptor (huaxiang sun) 2016-03-18 10:21:43 -07:00
tedyu 8c031e9d6b HBASE-15456 Revert due to unit test failure in thrift module 2016-03-17 17:36:44 -07:00
tedyu fbf58f330b HBASE-15325 ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests (Phil Yang) 2016-03-17 11:03:29 -07:00
tedyu 2b8a7f8d7b HBASE-15334 Add avro support for spark hbase connector (Zhan Zhang) 2016-03-17 09:11:38 -07:00
Mikhail Antonov a2c99b133f HBASE-15390 Unnecessary MetaCache evictions cause elevated number of requests to meta (ADDENDUM) 2016-03-17 01:08:36 -07:00