d05a3722c8
HBASE-15600 Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks (Rajeshbabu and Enis)
2016-06-07 20:12:51 -07:00
3a95552cfe
HBASE-15948 Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" Adds HADOOP-9955 RPC idle connection closing is extremely inefficient
...
Changes how we do accounting of Connections to match how it is done in Hadoop.
Adds a ConnectionManager class. Adds new configurations for this new class.
"hbase.ipc.client.idlethreshold" 4000
"hbase.ipc.client.connection.idle-scan-interval.ms" 10000
"hbase.ipc.client.connection.maxidletime" 10000
"hbase.ipc.client.kill.max", 10
"hbase.ipc.server.handler.queue.size", 100
The new scheme does away with synchronization that purportedly would freeze out
reads while we were cleaning up stale connections (according to HADOOP-9955)
Also adds in new mechanism for accepting Connections by pulling in as many
as we can at a time adding them to a Queue instead of doing one at a time.
Can help when bursty traffic according to HADOOP-9956. Removes a blocking
while Reader is busy parsing a request. Adds configuration
"hbase.ipc.server.read.connection-queue.size" with default of 100 for
queue size.
Signed-off-by: stack <stack@apache.org>
2016-06-07 16:42:21 -07:00
e66ecd7db6
Revert "HBASE-15948 Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers""
...
Revert mistaken commit...
This reverts commit e0b70c00e7
2016-06-07 16:41:30 -07:00
6d5a25935e
Revert "HBASE-15967 Metric for active ipc Readers and make default fraction of cpu count"
...
Revert mistaken commit
This reverts commit 1125215aad
2016-06-07 16:41:01 -07:00
1125215aad
HBASE-15967 Metric for active ipc Readers and make default fraction of cpu count
...
Add new metric hbase.regionserver.ipc.runningReaders
Also make it so Reader count is a factor of processor count
2016-06-07 13:10:14 -07:00
e0b70c00e7
HBASE-15948 Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
...
Adds HADOOP-9955 RPC idle connection closing is extremely inefficient
Then removes queue added by HADOOP-9956 at Enis suggestion
Changes how we do accounting of Connections to match how it is done in Hadoop.
Adds a ConnectionManager class. Adds new configurations for this new class.
"hbase.ipc.client.idlethreshold" 4000
"hbase.ipc.client.connection.idle-scan-interval.ms" 10000
"hbase.ipc.client.connection.maxidletime" 10000
"hbase.ipc.client.kill.max", 10
"hbase.ipc.server.handler.queue.size", 100
The new scheme does away with synchronization that purportedly would freeze out
reads while we were cleaning up stale connections (according to HADOOP-9955)
Also adds in new mechanism for accepting Connections by pulling in as many
as we can at a time adding them to a Queue instead of doing one at a time.
Can help when bursty traffic according to HADOOP-9956. Removes a blocking
while Reader is busy parsing a request. Adds configuration
"hbase.ipc.server.read.connection-queue.size" with default of 100 for
queue size.
2016-06-07 13:10:14 -07:00
376ad0d986
HBASE-15698 Increment TimeRange not serialized to server (Ted Yu)
2016-06-06 22:17:00 -07:00
3d7840a173
HBASE-15954 REST server should log requests with TRACE instead of DEBUG
2016-06-06 10:58:37 -07:00
de1b5ff776
HBASE-15955 Disable action in CatalogJanitor#setEnabled should wait for active cleanup scan to finish (Stephen Yuan Jiang)
2016-06-03 15:49:23 -07:00
21e98271c3
HBASE-15883 Adding WAL files and tracking offsets in HBase.
...
Implemented ReplicationQueuesHBaseImpl that tracks WAL offsets and replication queues in an HBase table.
Only wrote the basic tracking methods, have not implemented claimQueue() or HFileRef methods yet.
Wrote a basic unit test for ReplicationQueueHBaseImpl that tests the implemented functions on a single Region Server
Signed-off-by: Elliott Clark <elliott@fb.com>
Signed-off-by: Elliott Clark <eclark@apache.org>
2016-06-03 15:23:10 -07:00
9a53d8b385
HBASE-15929 There are two tests named TestRegionServerMetrics. This causes slight issues when parsing console output for hanging tests and flaky tests analysis. Moving .../hbase/TestRegionServerMetrics.java to .../hbase/regionserver/TestRegionServerReadRequestMetrics.java. (Apekshit)
...
Change-Id: I379c15fe2c2c01bed53bddf7619d5f2a07c5640e
2016-06-03 15:13:49 -07:00
b557f0bec6
HBASE-15949 Cleanup TestRegionServerMetrics.
...
@Before and @After to setup/teardown tables using @Rule to set table name based on testname.
Refactor out copy-pasted code fragments to single function.
(Apekshit)
Change-Id: Ic22e5027cc3952bab5ec30070ed20e98017db65a
2016-06-03 14:27:24 -07:00
f0c159b5fe
HBASE-15927 Remove HMaster.assignRegion()
2016-06-03 12:36:25 -07:00
0cbce07626
HBASE-15888 Extend HBASE-12769 for bulk load data replication
2016-06-03 18:42:00 +05:30
fc890a2ecb
HBASE-15881 Allow BZIP2 compression.
2016-06-02 12:19:02 -07:00
7e5d530870
HBASE-15727 Canary Tool for Zookeeper (churro morales)
2016-06-02 10:15:08 -07:00
cd2588001c
HBASE-15933 Addendum - make merge decision when sizes of both regions are known
2016-06-02 09:30:36 -07:00
cfe868d56e
HBASE-15933 NullPointerException may be thrown from SimpleRegionNormalizer#getRegionSize()
2016-06-02 01:55:28 -07:00
1a27278624
HBASE-15915 Set timeouts on hanging tests. Tests run: 8, Time elapsed: 94.191 sec - in org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures Tests run: 103, Time elapsed: 69.057 sec - in org.apache.hadoop.hbase.regionserver.TestHRegion Tests run: 103, Time elapsed: 67.957 sec - in org.apache.hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush Tests run: 5, Time elapsed: 34.629 sec - in org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster Tests run: 9, Time elapsed: 82.913 sec - in org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient Tests run: 9, Time elapsed: 80.458 sec - in org.apache.hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient (Apekshit)
...
Change-Id: Ia7b986ca6276ff5498f588c0b1b9c570e2a9d798
Signed-off-by: stack <stack@apache.org>
2016-05-30 21:13:55 -07:00
e6d613de70
HBASE-15835 - Prevent HBaseTestingUtility#startMiniCluster from throwing _HMasterAddress already in use_ RuntimeException when another HBase thread is running
...
Signed-off-by: stack <stack@apache.org>
2016-05-30 20:05:19 -07:00
a1f0c1cbb3
HBASE-15875 Remove HTable references and HTableInterface
...
Signed-off-by: stack <stack@apache.org>
2016-05-30 19:56:54 -07:00
cdd532da8a
HBASE-15610 Remove deprecated HConnection for 2.0 thus removing all PB references for 2.0
...
Signed-off-by: stack <stack@apache.org>
2016-05-29 07:50:55 -07:00
f4470af95d
HBASE-15891. Closeable resources potentially not getting closed if exception is thrown.
...
Signed-off-by: stack <stack@apache.org>
2016-05-28 21:32:22 -07:00
3b6e6e6c25
HBASE-15889. String case conversions are locale-sensitive, used without locale
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2016-05-28 10:41:31 -07:00
60c8f76a9d
HBASE-15908 Checksum verification is broken due to incorrect passing of ByteBuffers in DataChecksum (Mikhail Antonov and Appy)
2016-05-28 03:14:52 -07:00
da0d74cd27
HBASE-15830 SASL encryption doesn't work with AsyncRpcChannelImpl (Colin Ma)
2016-05-26 21:58:27 -07:00
43d1a999fe
HBASE-15837 Memstore size accounting is wrong if postBatchMutate() throws exception
2016-05-26 11:12:17 -07:00
94696d4067
HBASE-14635 Fix flaky test TestSnapshotCloneIndependence
...
Signed-off-by: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
2016-05-25 21:14:06 -07:00
b3362ccb0c
HBASE-15471 Added in Priority, General, and Replication queue sizes for RegionServers to the Web UI
...
Changed UI labels so that queue "size" refers to size in bytes and queue "length" refers to number of items in queue.
Signed-off-by: Elliott Clark <elliott@fb.com>
2016-05-25 15:10:48 -07:00
fa74baeb40
HBASE-15884 NPE in StoreFileScanner#skipKVsNewerThanReadpoint during reverse scan (Sergey Soldatov)
2016-05-25 08:58:28 -07:00
39dc19236e
HBASE-15806 revert due to discussion on security
2016-05-24 02:08:11 -07:00
c03ea895c4
HBASE-15806 An endpoint-based export tool (ChiaPing Tsai)
2016-05-23 10:51:25 -07:00
7130a222ce
HBASE-15876 Remove doBulkLoad(Path hfofDir, final HTable table) though it has not been through a full deprecation cycle
...
Signed-off-by: stack <stack@apache.org>
2016-05-23 10:25:22 -07:00
ae42c65cfd
HBASE-15802 ConnectionUtils should use ThreadLocalRandom instead of Random (Matt Warhaftig)
2016-05-21 21:15:10 -07:00
56e4b85d06
HBASE-14818 user_permission does not list namespace permissions (li xiang)
2016-05-21 20:06:05 -07:00
0671cba65f
HBASE-15529 Override needBalance in StochasticLoadBalancer (Guanghao Zhang)
2016-05-21 04:03:00 -07:00
0e52ac2464
HBASE-15228 Add the methods to RegionObserver to trigger start/complete restoring WALs. (Toshihiro Suzuki)
2016-05-21 09:56:28 +05:30
2bb07e209e
HBASE-15529 Revert due to TestRegionRebalancing failing
2016-05-20 12:56:17 -07:00
4793988aef
HBASE-15856 Don't cache unresolved addresses for connections
2016-05-20 09:36:11 -07:00
58798fc091
HBASE-15529 Override needBalance in StochasticLoadBalancer (Guanghao Zhang)
2016-05-20 07:44:05 -07:00
d34f5d4b96
HBASE-15865 Move TestTableDeleteFamilyHandler and TestTableDescriptorModification handler tests to procedure
2016-05-20 07:03:11 -07:00
7a84ee262f
HBASE-15864 Reuse the testing helper to wait regions in transition
2016-05-20 06:48:38 -07:00
a27504c701
HBASE-14920: Compacting memstore
...
Signed-off-by: stack <stack@apache.org>
2016-05-20 03:41:43 -07:00
af5146ee04
HBASE-15465 userPermission returned by getUserPermission() for the selected namespace does not have namespace set (li xiang)
2016-05-19 20:35:58 -07:00
a050e1d9f8
HBASE-15847 VerifyReplication prefix filtering (Geoffrey Jacoby)
2016-05-19 06:38:27 -07:00
460b41c800
HBASE-15617 Canary in regionserver mode might not enumerate all regionservers
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2016-05-18 16:25:29 -07:00
8aa8abfcb6
HBASE-15850 Localize the configuration change in testCheckTableLocks to reduce flakiness of TestHBaseFsck test suite (Stephen Yuan Jiang)
2016-05-18 13:22:41 -07:00
5ac54e6abb
HBASE-15843 Replace RegionState.getRegionInTransition() Map with a Set
2016-05-18 12:48:26 -07:00
393bcd69f9
HBASE-15841 Performance Evaluation tool total rows may not be set correctly
2016-05-18 12:09:13 -07:00
c1ada0a373
HBASE-15824 LocalHBaseCluster gets bind exception in master info port - ADDENDUM
2016-05-18 11:36:10 -07:00
3bd9220f0c
HBASE-15593 Time limit of scanning should be offered by client
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-05-18 19:13:15 +08:00
7b5d5394c0
HBASE-15784 Misuse core/maxPoolSize of LinkedBlockingQueue in
...
ThreadPoolExecutor (Jingcheng Du)
2016-05-18 12:40:43 +05:30
b2b3b1fa4d
Revert "HBASE-15593 Time limit of scanning should be offered by client (Phil Yang)"
...
This reverts commit e47bfb9078
2016-05-18 13:08:50 +08:00
e47bfb9078
HBASE-15593 Time limit of scanning should be offered by client (Phil Yang)
2016-05-17 12:49:31 -07:00
9269b8199e
HBASE-15840 WAL.proto compilation broken for cpp
2016-05-17 12:04:14 -07:00
838c306c0a
HBASE-15769 Perform validation on cluster key for add_peer (Matt Warhaftig)
2016-05-17 11:01:26 -07:00
00e61d2cfd
HBASE-15842 SnapshotInfo should display ownership information
2016-05-16 20:34:04 -07:00
c3223a59ff
HBASE-15824 LocalHBaseCluster gets bind exception in master info port
2016-05-16 10:32:43 -07:00
e0aff10901
HBASE-15780 Make AuthUtil public and rely on it for talking to secure HBase.
...
Signed-off-by: Gary Helmling <garyh@apache.org>
Signed-off-by: Mikhail Antonov <antonov@apache.org>
Signed-off-by: stack <stack@apache.org>
2016-05-16 00:50:14 -05:00
2482062d34
HBASE-15615 Wrong sleep time when RegionServerCallable need retry (Guanghao Zhang)
2016-05-15 20:37:26 -07:00
60e19f60a9
HBASE-13532 Make UnknownScannerException less scary by giving more information in the exception string. (Apekshit)
...
Signed-off-by: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
2016-05-13 15:00:23 -07:00
acca95fb50
HBASE-15808 Reduce potential bulk load intermediate space usage and waste
2016-05-12 15:43:48 -07:00
1267f76e9a
HBASE-15813 Rename DefaultWALProvider to a more specific name and clean up unnecessary reference to it
2016-05-12 11:27:00 +08:00
c867858c44
HBASE-15812 Revert pending investigation on whether metrics dump can be re-implemented
2016-05-11 09:39:20 -07:00
f3fee82ac4
HBASE-15812 HttpServer fails to come up against hadoop trunk
2016-05-11 09:00:27 -07:00
a11091c49c
HBASE-15798 Add Async RpcChannels to all RpcClients
...
Signed-off-by: stack <stack@apache.org>
2016-05-10 07:44:51 -07:00
8604f9eebb
HBASE-15415 Improve Master WebUI snapshot information (huaxiang sun)
2016-05-09 20:20:49 -07:00
11740570c1
HBASE-15236 Inconsistent cell reads over multiple bulk-loaded HFiles. In KeyValueHeap, if two cells are same i.e. have same key and timestamp, then instead of directly using seq id to determine newer one, we should use StoreFile.Comparater.SEQ_ID because that's what is used to determine order of hfiles. In this patch, we assign each scanner an order based on it's index in storefiles list, which is then used in KeyValueHeap to disambiguate between same cells. Changes the getSequenceId() in KeyValueScanner class to getScannerOrder(). Testing: Adds unit test to TestKeyValueHeap. Manual testing: Three cases (Tables t, t2, t3 in the jira description), single region, 2 hfiles with same seq id, timestamps and duplicate KVs. Made sure that returned kv was same for get and scan. (Apekshit)
...
Change-Id: I22600c91c0a51fb63eb17db73472839d2f13957c
Signed-off-by: stack <stack@apache.org>
2016-05-09 16:57:06 -07:00
a6e29676db
HBASE-15612: Minor improvements to CellCounter and RowCounter documentation
...
Signed-off-by: stack <stack@apache.org>
2016-05-09 14:12:43 -07:00
b75b226804
HBASE-15740 Replication source.shippedKBs metric is undercounting because it is in KB
2016-05-09 10:25:49 -07:00
05378cbf69
HBASE-15609 Addendum fix compilation error
2016-05-09 20:45:04 +08:00
97ad33c691
HBASE-15609 Remove PB references from Result, DoubleColumnInterpreter and
...
any such public facing class for 2.0 (Ram)
2016-05-09 14:56:00 +05:30
48b77dbcbc
HBASE-11625 - Verifies data before building HFileBlock. - Adds HFileBlock.Header class which contains information about location of fields. Testing: Adds CorruptedFSReaderImpl to TestChecksum.
...
Change-Id: I107e9636b28abb6b15ec329e885f1e31b1b1b988
Signed-off-by: stack <stack@apache.org>
2016-05-08 17:33:04 -07:00
99e2deb86f
HBASE-15778 replace master.am and master.sm direct access with getter calls
2016-05-07 05:19:15 -07:00
513ca3483f
HBASE-15776 Replace master.am.getTableStateManager() with the direct master.getTableStateManager()
2016-05-06 19:30:45 -07:00
86ca09e0e5
HBASE-15773 Improvements to CellCounter job
2016-05-06 11:08:18 -07:00
d90f0571e6
HBASE-15781 Remove unused TableEventHandler and TotesHRegionInfo
2016-05-06 10:05:42 -07:00
34e9a6ff30
HBASE-15669 HFile size is not considered correctly in a replication request
2016-05-06 17:26:17 +05:30
bec81b1977
HBASE-15608 Remove PB references from SnapShot related Exceptions (ram)
2016-05-06 17:06:25 +05:30
0d252918f6
HBASE-15743 Add Transparent Data Encryption support for FanOutOneBlockAsyncDFSOutput
2016-05-06 08:55:58 +08:00
532b914f08
HBASE-15759 RegionObserver.preStoreScannerOpen() doesn't have acces to current readpoint
2016-05-05 15:20:37 -07:00
c716a75416
HBASE-15767 upgrade httpclient to 4.3.6
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2016-05-05 15:41:26 -05:00
0bf065a5d5
HBASE-15768 fix capitalization of ZooKeeper usage
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2016-05-05 15:35:44 -05:00
b7ce55f001
HBASE-15575 Rename table DDL *Handler methods in MasterObserver to more meaningful names (Stephen Yuan Jiang)
2016-05-05 11:44:22 -07:00
efcf94def7
HBASE-15563 'counter' may overflow in BoundedGroupingStrategy (Matt Warhaftig)
2016-05-05 10:28:33 -07:00
c0ce47d1cb
HBASE-15755 SnapshotDescriptionUtils and SnapshotTestingUtils do not have any Interface audience marked
2016-05-05 07:24:29 -07:00
d23d600d76
HBASE-15741 Provide backward compatibility for HBase coprocessor service names
2016-05-04 12:09:57 -07:00
29eb3e8bcc
HBASE-15613 TestNamespaceCommand times out
2016-05-04 11:37:03 -07:00
809ed326f0
HBASE-15760 TestBlockEvictionFromClient#testParallelGetsAndScanWithWrappedRegionScanner
...
fails in master branch (Ram)
2016-05-04 22:07:11 +05:30
a8a2c516a0
HBASE-15763 Isolate Wal related stuff from MasterFileSystem
2016-05-04 07:59:44 -07:00
6abe1879dd
HBASE-15754 Add testcase for AES encryption
2016-05-04 09:40:57 +08:00
082d193455
HBASE-15675 Add more details about region on table.jsp
2016-05-03 20:18:48 +08:00
c77a153001
HBASE-15752 ClassNotFoundException is encountered when custom WAL codec is not found in WALPlayer job
2016-05-03 01:57:25 -07:00
c06a976a98
HBASE-15607 Remove PB references from Admin for 2.0 (Ram)
2016-05-03 10:51:46 +05:30
d77972ff16
HBASE-15714 We are calling checkRow() twice in doMiniBatchMutation()
2016-05-03 09:42:41 +10:00
58c4c3d174
HBASE-15703 Deadline scheduler needs to return to the client info about skipped calls, not just drop them
2016-05-02 15:23:07 -07:00
bbc7b90335
HBASE-15281 Allow the FileSystem inside HFileSystem to be wrapped (Rajesh Nishtala)
2016-05-02 13:24:51 -07:00
d1130582d5
HBASE-15735 Tightening of the CP contract.
2016-05-02 09:48:51 +05:30
15631a76f5
HBASE-15744 Port over small format/text improvements from HBASE-13784
...
(Jurriaan Mous)
Signed-off-by: stack <stack@apache.org>
2016-05-01 19:48:43 -07:00
ccd8888b4b
Revert "HBASE-15278 AsyncRPCClient hangs if Connection closes before RPC call response"
...
This reverts commit 01c0448ccd
2016-04-30 11:37:05 +08:00
01c0448ccd
HBASE-15278 AsyncRPCClient hangs if Connection closes before RPC call response
2016-04-30 09:27:32 +08:00
c236409c3b
HBASE-15357 TableInputFormatBase getSplitKey does not handle signed bytes correctly (Nathan Schile)
2016-04-29 17:08:25 -07:00
730b077666
HBASE-15551 Make call queue too big exception use servername
2016-04-29 10:25:29 -07:00
4706303186
HBASE-15706 HFilePrettyPrinter should print out nicely formatted tags. (huaxiang sun)
2016-04-29 09:23:57 +05:30
8a28c23431
HBASE-15686 Add override mechanism for the exempt classes when dynamically loading table coprocessor
2016-04-28 09:19:12 -07:00
889d89a74c
HBASE-15697 Excessive TestHRegion running time on branch-1 (Ram)
2016-04-28 14:24:39 +05:30
9343ef438f
HBASE-15676 FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s (Matt Warhaftig)
2016-04-27 21:00:11 -07:00
e8b37422c3
HBASE-15720 Print row locks at the debug dump page
2016-04-28 11:40:29 +08:00
ce318a2906
HBASE-15658 Avoid excess meta cache clearing for RegionServerCallable
2016-04-26 22:54:13 -07:00
ebb5d421f9
HBASE-15707 ImportTSV bulk output does not support tags with hfile.format.version=3 (huaxiang sun)
2016-04-26 11:21:29 -07:00
53d7316075
HBASE-15645 hbase.rpc.timeout is not used in operations of HTable
...
Signed-off-by: stack <stack@apache.org>
2016-04-26 10:51:29 -07:00
dce59294ff
HBASE-15634 TestDateTieredCompactionPolicy#negativeForMajor is flaky (Duo Zhang and Ted Yu)
2016-04-26 09:23:54 -07:00
793d976d87
HBASE-14252 RegionServers fail to start when setting hbase.ipc.server.callqueue.scan.ratio to 0 (Yubao Liu and Ted Yu)
2016-04-25 13:26:47 -07:00
1eac103ead
HBASE-15628 Implement an AsyncOutputStream which can work with any FileSystem implementation
2016-04-25 22:20:27 +08:00
3cfe363f13
HBASE-15699 Can not sync AsyncFSWAL if no edit is appended
2016-04-25 13:17:26 +08:00
106fe99a9a
HBASE-15670 Add missing Snapshot.proto to the maven profile for compiling protobuf
2016-04-22 11:34:49 -07:00
1ecb10ce07
HBASE-15688 Use MasterServices directly instead of casting to HMaster when possible
2016-04-22 10:15:58 -07:00
4c0587134a
HBASE-15671 Add per-table metrics on memstore, storefile and regionsize (Alicia Ying Shu)
2016-04-21 13:33:26 -07:00
03f3c392a3
HBASE-15674 HRegionLocator#getAllRegionLocations should put the results in cache
2016-04-21 14:19:39 +08:00
58f175f0ea
HBASE-15368 Add pluggable window support
2016-04-21 12:02:16 +08:00
2c26fe37ac
HBASE-15614 Report metrics from JvmPauseMonitor
2016-04-20 17:37:34 -07:00
18d70bc680
HBASE-15518 Add Per-Table metrics back (Alicia Ying Shu)
2016-04-20 14:35:45 -07:00
6dd938c20b
HBASE-15640 L1 cache doesn't give fair warning that it is showing partial stats only when it hits limit
2016-04-20 14:14:37 -07:00
fa215a67e2
HBASE-15385 PREFETCH_BLOCKS_ON_OPEN in HColumnDescriptor is ignored
2016-04-20 09:38:30 -07:00
3e1bdccc53
HBASE-15679 Assertion on wrong variable in TestReplicationThrottler#testThrottling
2016-04-20 07:11:17 -07:00
954a41790c
HBASE-15672 hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes fails (Anoop Sam John)
2016-04-19 17:10:48 -07:00
501030d204
HBASE-15673 Fix latency metrics for multiGet. - Also fixes some stuff in help text. (Apekshit)
...
Change-Id: I49d76896d5cc070b9f60dadf6746865e98fbbbd0
Signed-off-by: stack <stack@apache.org>
2016-04-19 16:12:43 -07:00
bd3b9753a9
HBASE-15665 Support using different StoreFileComparators for different CompactionPolicies
2016-04-19 19:35:17 +08:00
6d7a7fa3aa
HBASE-13372 Add unit tests for SplitTransaction and RegionMergeTransaction listeners
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2016-04-18 17:04:56 -07:00
70687c18bb
HBASE-15668 HFileReplicator fails to replicate other hfiles in the request when a hfile in not found in FS anywhere
2016-04-18 22:17:02 +05:30
f2e0aca2b6
HBASE-15664 Use Long.MAX_VALUE instead of HConstants.FOREVER in CompactionPolicy
2016-04-18 20:10:31 +08:00
e9211e415a
HBASE-15287 mapreduce.RowCounter returns incorrect result with binary row key inputs (Matt Warhaftig)
2016-04-16 12:46:21 -07:00
d815211f5a
HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
...
Refactor so we use the immutable, unsynchronized TimeRange when doing
time-based checks at read time rather than use heavily synchronized
TimeRangeTracker; let TimeRangeTracker be for write-time only.
While in here, changed the Segment stuff so that when an immutable
segment, it uses TimeRange rather than TimeRangeTracker too.
M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
Make allTime final.
Add a includesTimeRange method copied from TimeRangeTracker.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
Change name of a few methods so they match TimeRange methods that do
same thing.
(getTimeRangeTracker, getTimeRange, toTimeRange) add utility methods
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ImmutableSegment.java
Change ImmutableSegment so it uses a TimeRange rather than
TimeRangeTracker.. it is read-only. Redo shouldSeek, getMinTimestamp,
updateMetaInfo, and getTimeRangeTracker so we use TimeRange-based
implementations instead of TimeRangeTracker implementations.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MutableSegment.java
Implement shouldSeek, getMinTimestamp, updateMetaInfo, and
getTimeRangeTracker using TimeRangeTracker.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Segment.java
Make methods that were using TimeRangeTracker abstract and instead
have the implementations do these methods how they want either using
TimeRangeTracker when a mutable segment or TimeRange when an immutable
segment.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
Change Reader to use TimeRange-based checks instead of
TimeRangeTracker.
Signed-off-by: stack <stack@apache.org>
2016-04-15 10:13:42 -07:00
3e42f82600
HBASE-15622 Superusers does not consider the keytab credentials
2016-04-15 08:28:39 -07:00
a9f1c4a871
HBASE-15406 Split / merge switch left disabled after early termination of hbck
2016-04-15 14:09:52 +08:00
7efb9edecb
HBASE-15296 Break out writer and reader from StoreFile. Done using Intellij15 Refactor > Move. (Apekshit)
...
Change-Id: Ie719569cc3393e0b5361e9d462c3cf125ad5144e
Signed-off-by: stack <stack@apache.org>
2016-04-13 22:43:03 -07:00
5a7c8dcb68
HBASE-15504 Fix Balancer in 1.3 not moving regions off overloaded regionserver
2016-04-13 18:07:00 -07:00
8f9e46a640
HBASE-15405 Fix PE logging and wrong defaults in help message. (Apekshit)
...
Change-Id: Ib3007fae8d05e651b62bf4c15352ed43ea79e739
2016-04-13 12:10:33 -07:00
ae7e5e29f9
HBASE-15481 Add pre/post roll to WALObserver
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2016-04-13 00:43:51 -05:00
543e7081f5
HBASE-15636 hard coded wait time out value in HBaseTestingUtility#waitUntilAllRegionsAssigned might cause test failure (Stephen Yuan Jiang)
2016-04-12 16:46:39 -07:00
d03cdc795c
HBASE-15621 Suppress Hbase SnapshotHFile cleaner error messages when a snaphot is going on (Huaxiang Sun)
2016-04-12 12:17:55 -07:00
0f23e61ec5
HBASE-15632 Undo the checking of lastStoreFlushTimeMap.isEmpty() introduced in HBASE-13145 (Huaxiang Sun)
2016-04-12 11:35:18 -07:00
ff9c92e168
HBASE-14985 TimeRange constructors should set allTime when appropriate (Geoffrey Jacoby)
2016-04-11 16:59:45 -07:00
8964573394
HBASE-15627 Miss space and closing quote in AccessController#checkSystemOrSuperUser (Huaxiang Sun)
2016-04-11 08:28:26 -07:00
8541fe4ad1
HBASE-15093 Replication can report incorrect size of log queue for the global source when multiwal is enabled (Ashu Pachauri)
2016-04-11 08:17:20 -07:00
a395922ad5
HBASE-15605 Remove PB references from HCD and HTD for 2.0 (Ram)
2016-04-11 14:12:07 +05:30
80df1cb7b6
HBASE-15591 ServerCrashProcedure not yielding
2016-04-10 17:02:39 -07:00
f7d44e929f
HBASE-15527 Refactor Compactor related classes
2016-04-10 09:26:28 +08:00
e0f31ba6e6
HBASE-15507 Online modification of enabled ReplicationPeerConfig (Geoffrey Jacoby)
2016-04-08 21:26:31 -07:00
6ea4994569
HBASE-15407 Add SASL support for fan out OutputStream
2016-04-08 21:46:47 +08:00
e450d94a2c
HBASE-15380 Purge rollback support in Store etc.
2016-04-07 22:35:23 -07:00
394b89d153
HBASE-15537 Make multi WAL work with WALs other than FSHLog
2016-04-08 10:36:16 +08:00
2dcd08bc3d
HBASE-15586 Unify human readable numbers in the web UI
2016-04-07 17:07:33 -07:00
a146a71a33
HBASE-15400 Use DateTieredCompactor for Date Tiered Compaction - drop TestDateTieredCompaction.java
2016-04-07 15:01:00 -07:00
f60fc9d1a0
HBASE-15400 Use DateTieredCompactor for Date Tiered Compaction (Clara Xiong)
2016-04-07 14:58:59 -07:00
3826894f89
HBASE-15369 Handle NPE in region.jsp (Samir Ahmic)
2016-04-06 06:52:51 -07:00
c93cffb95c
HBASE-15587 FSTableDescriptors.getDescriptor() logs stack trace erronously
2016-04-05 18:13:40 -07:00
a93a8878fe
HBASE-15485 Filter.reset() should not be called between batches (Phil Yang)
2016-04-04 12:52:24 -07:00
33396c3629
HBASE-15578 Handle HBASE-15234 for ReplicationHFileCleaner
2016-04-04 15:02:19 +05:30
79868bd394
HBASE-15582 SnapshotManifestV1 too verbose when there are no regions
2016-04-01 20:55:21 -07:00
25419d8b18
HBASE-15424 Add bulk load hfile-refs for replication in ZK after the event is appended in the WAL
2016-04-01 15:40:36 +05:30
5d79790c55
HBASE-15293 Handle TableNotFound and IllegalArgument exceptions in table.jsp (Samir Ahmic)
2016-04-01 02:13:18 -07:00
89d750166d
HBASE-15568 Procedure V2 - Remove CreateTableHandler in HBase Apache 2.0 release (Stephen Yuan Jiang)
2016-04-01 01:31:09 -07:00
2d8e0a0477
HBASE-15234 Don't abort ReplicationLogCleaner on ZooKeeper errors
2016-03-31 22:28:18 -07:00
e1d5c3d269
HBASE-15521 Procedure V2 - RestoreSnapshot and CloneSnapshot (Stephen Yuan Jiang)
2016-03-31 21:49:13 -07:00
7d3a89ce8e
HBASE-15396 Enhance mapreduce.TableSplit to add encoded region name
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2016-03-31 22:59:43 -05:00
8c7f044efb
HBASE-15571 Make MasterProcedureManagerHost accessible through MasterServices
2016-03-31 15:45:46 -07:00
d6fd859451
HBASE-15538 Implement secure async protobuf wal writer
2016-03-31 10:33:35 +08:00
9d56105eec
HBASE-15324 Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger unexpected split (Yu Li)
2016-03-30 13:31:09 -07:00
21301a8a95
HBASE-15566 Add timeouts on TestMobFlushSnapshotFromClient and TestRegionMergeTransactionOnCluster
2016-03-30 13:23:20 -07:00
a71ce6e738
HBASE-14983 Create metrics for per block type hit/miss ratios
...
Summary: Missing a root index block is worse than missing a data block. We should know the difference
Test Plan: Tested on a local instance. All numbers looked reasonable.
Differential Revision: https://reviews.facebook.net/D55563
2016-03-30 11:41:11 -07:00
b18de5ef45
HBASE-15559 Fix BaseMasterAndRegionObserver doesn't implement all the methods
2016-03-30 11:17:08 -07:00
31aee19f28
HBASE-15327 Canary will always invoke admin.balancer() in each sniffing period when writeSniffing is enabled (Jianwei Cui)
2016-03-30 08:58:42 -07:00
afdfd1bd9c
HBASE-15295 MutateTableAccess.multiMutate() does not get high priority causing a deadlock - ADDENDUM for failing test
2016-03-29 15:02:18 -07:00
7f39baf0f4
HBASE-11393 Replication TableCfs should be a PB object rather than a string
2016-03-29 10:25:29 +08:00
0520097611
HBASE-15295 MutateTableAccess.multiMutate() does not get high priority causing a deadlock
2016-03-28 17:56:32 -07:00
cbf9c1e116
HBASE-15191 CopyTable and VerifyReplication - Option to specify batch size, versions (Parth Shah)
2016-03-28 17:25:00 -07:00
7793bc54c2
HBASE-15548 SyncTable: sourceHashDir is supposed to be optional but won't work without (Dave Latham)
2016-03-28 16:52:44 -07:00
c96b642f15
HBASE-15265 Implement an asynchronous FSHLog
2016-03-26 15:41:27 +08:00
b865fb9434
HBASE-15496 Throw RowTooBigException only for user scan/get (Guanghao Zhang)
2016-03-25 18:18:35 -07:00
11d11d3fcc
HBASE-15389 Write out multiple files when compaction
2016-03-25 15:07:54 +08:00
e9c4f12856
HBASE-15384 Avoid using '/tmp' directory in TestBulkLoad
2016-03-25 12:06:55 +08:00
07c597c8a2
HBASE-15526 Make SnapshotManager accessible through MasterServices
2016-03-24 15:59:46 -07:00
ad3feaa448
HBASE-15515 Improve LocalityBasedCandidateGenerator in Balancer (Guanghao Zhang)
2016-03-24 11:10:58 -07:00
7c93098210
HBASE-14256 Flush task message may be confusing when region is recovered (Gabor Liptak)
2016-03-24 10:06:38 -07:00
e71bc71c84
HBASE-15475 Addendum
2016-03-24 09:11:19 -07:00
fd5c0934b6
HBASE-15520 Fix broken TestAsyncIPC
2016-03-24 10:46:44 +08:00
925c185969
HBASE-15475 Allow TimestampsFilter to provide a seek hint
...
Summary:
Allow TestTimestampFilterSeekHint to provide a seek next hint.
This can be incorrect as it might skip deletes. However it can
make things much much faster.
Test Plan: Added a unit test.
Differential Revision: https://reviews.facebook.net/D55617
2016-03-23 18:56:52 -07:00
d7a4499dfc
HBASE-15392 Single Cell Get reads two HFileBlocks
...
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
moreRowsMayExistAfterCell Exploit the fact a Scan is a Get Scan. Also save compares
if no non-default stopRow.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
optimize Add doc on what is being optimized. Also, if a Get Scan, do not
optimize else we'll keep going after our row is DONE.
Another place to make use of the Get Scan fact is when we are DONE.. if
Get Scan, we can close out the scan.
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java
Add tests for Get Scans and optimize around block loading.
2016-03-23 14:37:14 -07:00
7177e4be64
HBASE-15508 Add command for exporting snapshot in hbase command script (Yufeng Jiang)
2016-03-23 10:26:32 -07:00
5fcadb86ab
HBASE-15495 Connection leak in FanOutOneBlockAsyncDFSOutputHelper
2016-03-23 09:56:40 +08:00
12f66e3060
HBASE-15477 Purge 'next block header' from cached blocks
...
When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body. We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting: our block plus the next blocks'
header (33 bytes).
This patch undoes this over-persisting.
Removes support for version 1 blocks (0.2 was added in hbase-0.92.0).
Not needed any more.
There is an open question on whether checksums should be persisted
when caching. The code seems to say no but if cache is SSD backed or
backed by anything that does not do error correction, we'll want
checksums.
Adds loads of documentation.
M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
(write) Add writing from a ByteBuff.
M hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java
(toString) Add one so ByteBuff looks like ByteBuffer when you click on
it in IDE
M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
Remove support for version 1 blocks.
Cleaned up handling of metadata added when we serialize a block to
caches. Metadata is smaller now.
When we serialize (used when caching), do not persist the next blocks
header if present.
Removed a bunch of methods, a few of which had overlapping
functionality and others that exposed too much of our internals.
Also removed a bunch of constructors and unified the constructors we
had left over making them share a common init method.
Shutdown access to defines that should only be used internally here.
Renamed all to do w/ 'EXTRA' and 'extraSerialization' to instead talk
about metadata saved to caches; was unclear previously what EXTRA was
about.
Renamed static final declarations as all uppercase.
(readBlockDataInternal): Redid. Couldn't make sense of it previously.
Undid heavy-duty parse of header by constructing HFileBlock. Other
cleanups. Its 1/3rd the length it used to be. More to do in here.
2016-03-22 18:45:17 -07:00
a13d6e000d
Revert "HBASE-15392 Single Cell Get reads two HFileBlocks M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java moreRowsMayExistAfterCell Exploit the fact a Scan is a Get Scan. Also save compares if no non-default stopRow."
...
Revert mistaken commit
This reverts commit 7073f69993
2016-03-22 18:39:22 -07:00
c49d0caf53
Revert "HBASE-15477 Purge 'next block header' from cached blocks"
...
Overcommit. Revert.
This reverts commit ef94b55239
2016-03-22 18:38:29 -07:00
54a543de22
Revert "HBASE-15477 Purge 'next block header' from cached blocks"
...
Overcommit. Revert to fix.
This reverts commit 000117ad9f
2016-03-22 18:37:25 -07:00
3f3613a234
HBASE-15212 RRCServer should enforce max request size
2016-03-22 16:23:15 -07:00
8af9ed7433
HBASE-15416 TestHFileBackedByBucketCache is flakey since it went in
2016-03-22 15:44:25 -07:00
b3fe4ed16c
HBASE-15412 Add average region size metric (Alicia Ying Shu)
2016-03-22 14:46:27 -07:00
2348478506
HBASE-15360 addendum fix testCoDelScheduling
...
Signed-off-by: stack <stack@apache.org>
2016-03-22 13:52:26 -07:00
000117ad9f
HBASE-15477 Purge 'next block header' from cached blocks
...
When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body. We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting: our block plus the next blocks'
header (33 bytes).
This patch undoes this over-persisting.
Removes support for version 1 blocks (0.2 was added in hbase-0.92.0).
Not needed any more.
There is an open question on whether checksums should be persisted
when caching. The code seems to say no but if cache is SSD backed or
backed by anything that does not do error correction, we'll want
checksums.
Adds loads of documentation.
M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
(write) Add writing from a ByteBuff.
M hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java
(toString) Add one so ByteBuff looks like ByteBuffer when you click on
it in IDE
M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
Remove support for version 1 blocks.
Cleaned up handling of metadata added when we serialize a block to
caches. Metadata is smaller now.
When we serialize (used when caching), do not persist the next blocks
header if present.
Removed a bunch of methods, a few of which had overlapping
functionality and others that exposed too much of our internals.
Also removed a bunch of constructors and unified the constructors we
had left over making them share a common init method.
Shutdown access to defines that should only be used internally here.
Renamed all to do w/ 'EXTRA' and 'extraSerialization' to instead talk
about metadata saved to caches; was unclear previously what EXTRA was
about.
Renamed static final declarations as all uppercase.
(readBlockDataInternal): Redid. Couldn't make sense of it previously.
Undid heavy-duty parse of header by constructing HFileBlock. Other
cleanups. Its 1/3rd the length it used to be. More to do in here.
2016-03-22 13:42:38 -07:00
Enis Soztutar
stack
Andrew Purtell
Stephen Yuan Jiang
Joseph Hwang
Apekshit
Matteo Bertozzi
Ashish Singhi
Lars Hofhansl
tedyu
Daniel Vimont
Jurriaan Mous
Sean Mackrory
Mikhail Antonov
Gary Helmling
Jerry He
anoopsjohn
eshcar
Vincent
Phil Yang
Ramkrishna
zhangduo
Elliott Clark
Sean Busbey
Esteban Gutierrez
Alex Moundalexis
Yu Li
chenheng
Gábor Lipták
Ashish Singhi
Harsh J