Commit Graph

4764 Commits

Author SHA1 Message Date
tedyu abc8c5edd3 HBASE-17494 Guard against cloning family of all cells if no data need be replicated (ChiaPing Tsai) 2017-01-19 16:21:17 -08:00
tedyu 0e06ade694 HBASE-17469 Properly handle empty TableName in TablePermission#readFields and #write (Manjunath Anand) 2017-01-17 09:38:50 -08:00
Michael Stack 354e213852 HBASE-17464 Fix HBaseTestingUtility.getNewDataTestDirOnTestFS to always return a unique path 2017-01-13 17:53:48 -08:00
tedyu ee66ab8e29 HBASE-17450 TablePermission#equals throws NPE after namespace support was added (huzheng) 2017-01-13 09:39:06 -08:00
tedyu 49b5a6e803 HBASE-17452 Failed taking snapshot - region Manifest proto-message too large (huaxiang sun) 2017-01-13 09:00:15 -08:00
tedyu 56f963f4a6 HBASE-17445 Count size of serialized exceptions in checking max result size quota 2017-01-11 10:06:40 -08:00
tedyu 2f7ce65b81 HBASE-17429 HBase bulkload cannot support HDFS viewFs (shenxianqiang) 2017-01-09 18:53:39 -08:00
Michael Stack 6130ea4d54 HBASE-12148 Remove TimeRangeTracker as point of contention when many threads writing a Store (Huaxiang Sun) 2017-01-09 13:35:20 -08:00
tedyu 9b26c9ff37 HBASE-17435 Call to preCommitStoreFile() hook encounters SaslException in secure deployment 2017-01-09 08:53:17 -08:00
Ashish Singhi e8e40d8625 HBASE-17290 Potential loss of data for replication of bulk loaded hfiles 2017-01-06 16:57:52 +05:30
rahul gidwani 667c5eb3a0 HBASE-16710 Add ZStandard Codec to Compression.java
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-01-05 10:47:12 -08:00
Jan Hentschel a9bac6a496 HBASE-17390 Added master and backup masters to online update of configuration
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-01-04 22:25:22 -08:00
Josh Elser 4dcb07f996 HBASE-17409 Limit jsonp callback name to prevent xss 2017-01-04 19:36:24 -05:00
tedyu 36f16bbe10 HBASE-17387 Reduce the overhead of exception report in RegionActionResult for multi() 2016-12-29 19:23:44 -08:00
tedyu cec40f941f HBASE-17374 ZKPermissionWatcher crashed when grant after region close (Liu Junhong) 2016-12-29 05:46:40 -08:00
Stephen Yuan Jiang ce33cf2d3d HBASE-17149 Procedure V2 - Fix nonce submission to avoid unnecessary calling coprocessor multiple times (Matteo Bertozzi) 2016-12-28 15:43:50 -08:00
Stephen Yuan Jiang d4b2627916 HBASE-17238 Wrong in-memory hbase:meta location causing SSH failure (Stephen Yuan jiang) 2016-12-28 13:53:22 -08:00
Phil Yang 29e390c808 HBASE-17314 Limit total buffered size for all replication sources 2016-12-23 12:17:39 +08:00
Phil Yang dd0206ff31 Revert "HBASE-17314 Limit total buffered size for all replication sources"
This reverts commit e9444ed077.
2016-12-22 19:08:17 +08:00
tedyu 819d32f951 HBASE-17330 SnapshotFileCache will always refresh the file cache (Jianwei Cui) 2016-12-22 02:30:41 -08:00
tedyu f94180a3e9 HBASE-17341 Add a timeout during replication endpoint termination (Vincent Poon) 2016-12-21 08:27:45 -08:00
Phil Yang e9444ed077 HBASE-17314 Limit total buffered size for all replication sources 2016-12-21 13:10:29 +08:00
Vincent e79afbf0cb HBASE-17328 Properly dispose of looped replication peers
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestMasterReplication.java
2016-12-20 18:01:33 -08:00
tedyu fa975fa382 HBASE-17347 ExportSnapshot may write snapshot info file to wrong directory when specifying target name (Jianwei Cui) 2016-12-20 15:58:28 -08:00
tedyu 0b69f59133 HBASE-17292 Add observer notification before bulk loaded hfile is moved to region directory 2016-12-20 13:23:27 -08:00
tedyu b9689808eb HBASE-17344 The regionserver web UIs miss the coprocessors of RegionServerCoprocessorHost (ChiaPing Tsai) 2016-12-20 08:55:07 -08:00
Guanghao Zhang 97d1ba4015 HBASE-17317 [branch-1] The updatePeerConfig method in ReplicationPeersZKImpl didn't update the table-cfs map 2016-12-20 18:51:26 +08:00
binlijin 4c7f0f2436 HBASE-17332 Replace HashMap to Array for DataBlockEncoding.idToEncoding 2016-12-20 12:28:04 +08:00
binlijin ed39396497 HBASE-16398 optimize HRegion computeHDFSBlocksDistribution 2016-12-16 13:57:43 +08:00
tedyu f3a3069796 HBASE-17319 Truncate table with preserve after split may cause truncation to fail (Allan Yang) 2016-12-15 13:45:11 -08:00
tedyu ffe70158cc HBASE-17318 Increment does not add new column if the increment amount is zero at first time writing (Guangxu Cheng) 2016-12-15 08:45:05 -08:00
Guanghao Zhang 4b3df0f926 HBASE-17296 Provide per peer throttling for replication
Signed-off-by: Phil Yang <yangzhe1991@apache.org>
2016-12-15 16:13:04 +08:00
tedyu e029c554bb HBASE-17302 The region flush request disappeared from flushQueue (Guangxu Cheng) 2016-12-13 12:46:42 -08:00
tedyu 1f9214bee7 HBASE-17297 Single Filter in parenthesis cannot be parsed correctly (Xuesen Liang) 2016-12-12 12:09:22 -08:00
tedyu 30576991bc HBASE-17289 Avoid adding a replication peer named "lock" 2016-12-12 09:38:55 -08:00
Josh Elser f13283bc05 HBASE-17276 Only log stacktraces for exceptions once for updates in a batch
For large batches of updates, repeatedly logging WrongRegionExceptions,
FailedSanityCheckExceptions, and/or NoSuchColumnFamilyExceptions can
easily dominate the contents of a RegionServer log. After the first
occurence of logging the full exception, switch to logging only the
message on the exception.
2016-12-11 15:40:22 -05:00
tedyu d87b05f040 HBASE-16336 Removing peers seems to be leaving spare queues (Guanghao Zhang) 2016-12-11 07:43:10 -08:00
Stephen Yuan Jiang e51584381a HBASE-17023 Region left unassigned due to AM and SSH each thinking others would do the assignment work (Stephen Yuan Jiang) 2016-12-10 15:05:00 -08:00
tedyu c2801a2ea8 HBASE-17256 Rpc handler monitoring will be removed when the task queue is full (Guangxu Cheng) 2016-12-05 12:14:20 -08:00
Ashu Pachauri cbdc9fcb8a Addendum HBASE-16209: Add an ExponentialBackOffPolicy so that we spread out the timing of open region retries in AssignmentManager.
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-12-02 10:30:28 +08:00
Guanghao Zhang 682dd57cd6 HBASE-17205 Add a metric for the duration of region in transition 2016-12-01 10:32:24 -08:00
Jonathan M Hsieh 9da0d5d00e HBASE-17224 Fix lots of spelling errors in HBase logging and exception messages (Grant Sohn) 2016-12-01 09:10:03 -08:00
zhangduo c8ea82299c HBASE-17206 FSHLog may roll a new writer successfully with unflushed entries 2016-12-01 14:20:38 +08:00
Yu Li b3627ef51e HBASE-17212 Should add null checker on table name in HTable and RegionServerCallable constructor (addendum) 2016-12-01 13:32:33 +08:00
Guanghao Zhang f26b3bf5ba HBASE-17178 Add region balance throttling
Signed-off-by: Phil Yang <yangzhe1991@apache.org>
2016-11-30 18:07:02 +08:00
zhangduo af69783128 HBASE-17167 Pass mvcc to client when scan 2016-11-30 10:02:17 +08:00
Ashu Pachauri b8da9f83cb HBASE-16302 age of last shipped op and age of last applied op should be histograms
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2016-11-29 13:54:28 +05:30
Guanghao Zhang 7b2673db12 HBASE-16561 Add metrics about read/write/scan queue length and active read/write/scan handler count
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-11-29 16:09:56 +08:00
zhangduo be042652aa Revert "HBASE-16561 Add metrics about read/write/scan queue length and active read/write/scan handler count"
Forget to add signoff

This reverts commit 5ec218dbc2.
2016-11-29 16:09:22 +08:00
Guanghao Zhang 5ec218dbc2 HBASE-16561 Add metrics about read/write/scan queue length and active read/write/scan handler count 2016-11-29 16:00:37 +08:00
Stephen Yuan Jiang cdf539a8e0 HBASE-17189 TestMasterObserver#wasModifyTableActionCalled uses wrong variables (Stephen Yuan Jiang) 2016-11-28 23:18:42 -08:00
Michael Stack 987205caf9 HBASE-17072 CPU usage starts to climb up to 90-100% when using G1GC
Removes ThreadLocal. Uses AtomicReference instead (based on patch
posted up in HBASE-10676 "Removing ThreadLocal of PrefetchedHeader in
HFileBlock.FSReaderV2 make higher perforamce of scan")

Signed-off-by: Michael Stack <stack@apache.org>
2016-11-28 13:41:17 -08:00
Jerry He 10c070825f HBASE-17116 [PerformanceEvaluation] Add option to configure block size (Yi Liang) 2016-11-27 19:13:08 -08:00
Jingcheng Du fb1ace490c HBASE-17095 The ClientSimpleScanner keeps retrying if the hfile is corrupt or cannot found 2016-11-22 11:26:57 +08:00
binlijin 2da5ed098e HBASE-17118 StoreScanner leaked in KeyValueHeap (addendum) 2016-11-21 20:55:52 +08:00
tedyu e8f056935b HBASE-17131 Avoid livelock caused by HRegion#processRowsWithLocks (ChiaPing Tsai) 2016-11-20 18:17:21 -08:00
Guanghao Zhang 3c45ff08d9 HBASE-17088 Refactor RWQueueRpcExecutor/BalancedQueueRpcExecutor/RpcExecutor
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-11-19 14:18:06 +08:00
Esteban Gutierrez 19441937ea HBASE-17058 Lower epsilon used for jitter verification from HBASE-15324 2016-11-17 12:29:08 -08:00
tedyu c6f1b6e624 HBASE-17112 Prevent setting timestamp of delta operations the same as previous value's (Phil Yang) 2016-11-17 09:04:39 -08:00
tedyu d722b2aab7 HBASE-17118 StoreScanner leaked in KeyValueHeap (binlijin) 2016-11-17 08:46:11 -08:00
Andrew Purtell e54826434e HBASE-17044 Fix merge failed before creating merged region leaves meta inconsistent 2016-11-14 17:14:48 -08:00
tedyu b77bfe9d3b HBASE-17077 Don't copy the replication queue belonging to the peer which has been deleted (Guanghao Zhang) 2016-11-13 07:00:25 -08:00
tedyu 469462c850 HBASE-17062 RegionSplitter throws ClassCastException (Jeongdae Kim) 2016-11-11 07:18:22 -08:00
thiruvel 44ab659b93 HBASE-16962: Add readPoint to preCompactScannerOpen() and preFlushScannerOpen() API
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
2016-11-11 11:09:55 +05:30
Yu Sun 18b31fdd32 HBASE-17020 keylen in midkey() dont computed correctly
Signed-off-by: Yu Li <liyu@apache.org>
2016-11-11 08:01:00 +08:00
Guanghao Zhang a6397e3b0c HBASE-16938 TableCFsUpdater maybe failed due to no write permission on peerNode
Signed-off-by: Enis Soztutar <enis@apache.org>
2016-11-10 14:07:20 -08:00
binlijin dac73eceb0 HBASE-16570 Compute region locality in parallel at startup (addendum)
Addendum mainly for:
1. Avoid interfering with block location cache in RegionLocationFinder
2. Avoid refreshing block lcoations during HMaster startup (or else the startup could be really slow)

Signed-off-by: Yu Li <liyu@apache.org>
2016-11-10 16:53:39 +08:00
Guanghao Zhang e929156f96 HBASE-16985 TestClusterId failed due to wrong hbase rootdir
Signed-off-by: Michael Stack <stack@apache.org>
2016-11-09 17:19:31 -08:00
tedyu 1e322e68a5 HBASE-17054 Compactor#preCreateCoprocScanner should be passed user 2016-11-09 07:00:19 -08:00
Charlie Qiangeng Xu d248d6b0b3 HBASE-17039 SimpleLoadBalancer schedules large amount of invalid region moves
Signed-off-by: Yu Li <liyu@apache.org>
2016-11-09 17:07:28 +08:00
Enis Soztutar 123d26ed90 HBASE-17017 Remove the current per-region latency histogram metrics 2016-11-08 18:31:12 -08:00
Michael Stack a70f73c1e1 HBASE-16983 TestMultiTableSnapshotInputFormat failing with Unable to create region directory: /tmp/... (Addendum by Guanghao Zhang) 2016-11-08 11:59:49 -08:00
tedyu 97276da9a7 HBASE-17010 Serial replication should handle daughter regions being assigned to another RS (Phil Yang) 2016-11-08 06:35:12 -08:00
Stephen Yuan Jiang 874cf5128f HBASE-17042 Remove 'public' keyword from MasterObserver interface (Stephen Yuan Jiang) 2016-11-07 17:51:44 -08:00
Enis Soztutar b99690ac6b HBASE-17033 LogRoller makes a lot of allocations unnecessarily 2016-11-07 14:31:30 -08:00
anoopsamjohn 2fdba6521d HBASE-16992 The usage of mutation from CP is weird. (ChiaPing Tsai) 2016-11-07 23:26:27 +05:30
tedyu ecd4de938b HBASE-17026 VerifyReplication log should distinguish whether good row key is result of revalidation 2016-11-07 07:41:55 -08:00
binlijin c4132bbf30 HBASE-16840 Reuse cell's timestamp and type in ScanQueryMatcher
Signed-off-by: Yu Li <liyu@apache.org>
2016-11-07 22:42:46 +08:00
Yu Li a67481209f Revert "HBASE-16840 Reuse cell's timestamp and type in ScanQueryMatcher" to update author
This reverts commit 4d8611fa0f.
2016-11-07 22:42:38 +08:00
Yu Li 4d8611fa0f HBASE-16840 Reuse cell's timestamp and type in ScanQueryMatcher
Signed-off-by: Yu Li <liyu@apache.org>
2016-11-07 21:38:03 +08:00
Mikhail Antonov 3063943c78 HBASE-17032 CallQueueTooBigException and CallDroppedException should not be triggering PFFE 2016-11-06 21:34:11 -08:00
tedyu 8ade73c0cb HBASE-16946 Provide Raw scan as an option in VerifyReplication (Sreekar Pallapothu) 2016-11-04 03:36:17 -07:00
binlijin 5bfec397d1 HBASE-16960 RegionServer hang when aborting
Signed-off-by: Yu Li <liyu@apache.org>
2016-11-04 14:08:42 +08:00
Apekshit Sharma 51ba7cfde3 HBASE-17006 Give name to existing threads.
Having thread names in logs and thread dumps greatly improve debugability. This patch is simply adding the names to the threads we spawn.

Change-Id: I6ff22cc3804bb81147dde3a8e9ab671633c6f6ce
2016-11-03 18:31:03 -07:00
Eddie Elizondo b4676d51ad HBASE-14329 Report region in transition should check all regions in request
Signed-off-by: Gary Helmling <garyh@apache.org>
2016-11-03 11:32:19 -07:00
Gary Helmling 4fe51ebaa7 HBASE-16964 Always clear successfully archived files from compacted files list 2016-11-03 09:28:22 -07:00
Yu Li 514add0086 HBASE-16972 Log more details for Scan#next request when responseTooSlow 2016-11-02 14:33:44 +08:00
Andrew Purtell 961b952171 HBASE-16980 TestRowProcessorEndpoint failing consistently (Yu Li) 2016-11-01 19:09:17 -07:00
Lars Hofhansl 6113f9a34f HBASE-16765 New SteppingRegionSplitPolicy, avoid too aggressive spread of regions for small tables. 2016-11-01 13:07:12 -07:00
Michael Stack b3ddea9fc4 HBASE-16983 TestMultiTableSnapshotInputFormat failing with Unable to
create region directory: /tmp/...
2016-10-31 21:48:49 -07:00
Guanghao Zhang 722be3a3d8 HBASE-16947 Some improvements for DumpReplicationQueues tool
Signed-off-by: Michael Stack <stack@apache.org>
2016-10-31 21:00:45 -07:00
Michael Stack a969f8d790 HBASE-16969 RegionCoprocessorServiceExec should override the toString() for debugging (ChiaPing Tsai) 2016-10-31 14:37:51 -07:00
Mikhail Antonov ea5b0a05d6 HBASE-16743 TestSimpleRpcScheduler#testCoDelScheduling is broke 2016-10-28 16:32:58 -07:00
Esteban Gutierrez 0a41493ac4 HBASE-16450 Shell tool to dump replication queues
New tool to dump existing replication peers, configurations and
queues when using HBase Replication. The tool provides two flags:

 --distributed  This flag will poll each RS for information about
                the replication queues being processed on this RS.
                By default this is not enabled and the information
                about the replication queues and configuration will
                be obtained from ZooKeeper.
 --hdfs         When --distributed is used, this flag will attempt
                to calculate the total size of the WAL files used
                by the replication queues. Since its possible that
                multiple peers can be configured this value can be
                overestimated.

Signed-off-by: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
2016-10-27 21:38:42 -07:00
tedyu d4826e1665 HBASE-16886 hbase-client: scanner with reversed=true and small=true gets no result (huzheng) 2016-10-27 07:47:49 -07:00
Michael Stack 24a92ed63a HBASE-16948 Fix inconsistency between HRegion and Region javadoc on getRowLock 2016-10-26 16:11:26 -07:00
Guanghao Zhang 4edd8a63d2 HBASE-16939 ExportSnapshot: set owner and permission on right directory
Signed-off-by: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
2016-10-25 14:11:59 -07:00
Guanghao Zhang ae502a9d5c HBASE-16910 Avoid NPE when starting StochasticLoadBalancer
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2016-10-25 11:58:41 +05:30
binlijin 16823ff55e HBASE-16931 Setting cell's seqId to zero in compaction flow might cause RS down.
Signed-off-by: Yu Li <liyu@apache.org>
2016-10-24 23:33:34 +08:00
Guanghao Zhang d76cc4c1f0 HBASE-16870 Add the metrics of replication sources which were transformed from other dead rs to ReplicationLoad
Signed-off-by: zhangduo <zhangduo@apache.org>
2016-10-22 14:27:01 +08:00