Commit Graph

11614 Commits

Author SHA1 Message Date
tedyu a4426a7c2d HBASE-16739 Timed out exception message should include encoded region name - Revert due to misspelled word in subject 2016-10-03 06:44:11 -07:00
tedyu 69a3fd2ebf HBASE-16739) imed out exception message should include encoded region name 2016-10-02 21:10:20 -07:00
Enis Soztutar 06cc123849 HBASE-16721 Concurrency issue in WAL unflushed seqId tracking - ADDENDUM to change WAL to extend Closeable 2016-10-01 14:50:54 -07:00
Enis Soztutar 911f9b9eb7 HBASE-16678 MapReduce jobs do not update counters from ScanMetrics
Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
2016-09-29 17:14:55 -07:00
Enis Soztutar bf5a7aba5c HBASE-16721 Concurrency issue in WAL unflushed seqId tracking 2016-09-29 14:51:14 -07:00
Jerry He 5ac2776d23 HBASE-16732 Avoid possible NPE in MetaTableLocator 2016-09-29 14:14:01 -07:00
Sean Busbey df25ebf84f HBASE-15984 Handle premature EOF treatment of WALs in replication.
In some particular deployments, the Replication code believes it has
reached EOF for a WAL prior to succesfully parsing all bytes known to
exist in a cleanly closed file.

Consistently this failure happens due to an InvalidProtobufException
after some number of seeks during our attempts to tail the in-progress
RegionServer WAL. As a work-around, this patch treats cleanly closed
files differently than other execution paths. If an EOF is detected due
to parsing or other errors while there are still unparsed bytes before
the end-of-file trailer, we now reset the WAL to the very beginning and
attempt a clean read-through.

In current testing, a single such reset is sufficient to work around
observed dataloss. However, the above change will retry a given WAL file
indefinitely. On each such attempt, a log message like the below will
be emitted at the WARN level:

  Processing end of WAL file '{}'. At position {}, which is too far away
  from reported file length {}. Restarting WAL reading (see HBASE-15983
  for details).

Additionally, this patch adds some additional log detail at the TRACE
level about file offsets seen while handling recoverable errors. It also
add metrics that measure the use of this recovery mechanism.
2016-09-29 10:47:57 -05:00
tedyu df57859258 HBASE-16725 Don't let flushThread hang in TestHRegion 2016-09-29 06:57:02 -07:00
tedyu d4b5645a4e HBASE-16723 RMI registry is not destroyed after stopping JMX Connector Server (Pankaj Kumar) 2016-09-29 01:58:22 -07:00
Abhishek Singh Chouhan 96a8e8dce4 HBASE-16660 ArrayIndexOutOfBounds during the majorCompactionCheck in DateTieredCompaction
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2016-09-27 11:08:00 -07:00
anoopsamjohn a3485cc5ab HBASE-16705 Eliminate long to Long auto boxing in LongComparator. (binlijin)
Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/filter/LongComparator.java
2016-09-26 15:41:45 -07:00
Andrew Purtell 67a43c3059 HBASE-16694 Reduce garbage for onDiskChecksum in HFileBlock (binlijin) 2016-09-26 13:55:28 -07:00
Matteo Bertozzi 4566e4df58 HBASE-16649 Truncate table with splits preserved can cause both data loss and truncated data appeared again 2016-09-26 13:09:06 -07:00
chenheng d6f9eab4be HBASE-16677 Add table size (total store file size) to table page (Guang Yang) 2016-09-25 10:50:03 +08:00
Jerry He 92b1b5ac80 HBASE-16667 Building with JDK 8: ignoring option MaxPermSize=256m (Niels Basjes) 2016-09-24 16:29:41 -07:00
Apekshit Sharma 9a510e9ba2 HBASE-16679 Flush throughput controller: Minor perf change and fix flaky TestFlushWithThroughputController.
Change-Id: I79f4a65a67bfdc46868d68ff8a0c32f3a02b8cb1
2016-09-23 11:47:12 -07:00
tedyu b503843bae HBASE-16423 Add re-compare option to VerifyReplication to avoid occasional inconsistent rows (Jianwei Cui) 2016-09-23 05:08:19 -07:00
Enis Soztutar 8a797e81b8 HBASE-16604 Scanner retries on IOException can cause the scans to miss data 2016-09-22 18:48:06 -07:00
Ben Lau 6973304026 HBASE-16662 Fix open POODLE vulnerabilities
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2016-09-22 19:59:08 +00:00
tedyu ba650a8a10 HBASE-16675 Average region size may be incorrect when there is region whose RegionLoad cannot be retrieved 2016-09-22 07:34:29 -07:00
Yu Li 76da21fc59 HBASE-16670 Make RpcServer#processRequest logic more robust 2016-09-22 15:44:45 +08:00
Jonathan M Hsieh ecc1c294f5 HBASE-12088 Addendum - fix spacing 2016-09-21 20:58:34 -07:00
Jonathan M Hsieh 13d6acbc7f HBASE-12088 Remove unused hadoop-1.0, hadoop-1.1 profiles from non-root poms 2016-09-21 20:52:19 -07:00
Apekshit Sharma e7e660d5b2 HBASE-14734 Prevent BindException when setting up MiniKdc.
Port for kdc service gets selected in the constructor, but we bind to it later in MiniKdc.start()-->MiniKdc.initKDCServer() --> KdcServer.start(). In meantime, some other service can capture the port which results in BindException. The solution here is to catch the exception and retry.

Testing methodology:
- Used python and intellij.
- breakpoint on kdc.start(1), in catch block(2) and just after catch block(3).
- used python to bind to the selected port on breakpoint 1 --> run the program --> stops at breakpoint 2 (catch block)
- On breakpoint 1 and after 2 failures, close the port --> run the program --> skips catch block and goes to breakpoint 3.

Change-Id: I4e06e69819d1ec9a0a7fa471bf017f3a72c75cb3
2016-09-21 16:44:14 -07:00
Umesh Agashe 23c5ea39bd HBASE-16294 hbck reporting "No HDFS region dir found" for replicas
Signed-off-by: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
2016-09-21 14:43:15 -07:00
Jerry He c80d671a06 HBASE-12949 Scanner can be stuck in infinite loop if the HFile is corrupted 2016-09-21 13:31:18 -07:00
Jerry He abfb20206d HBASE-16647 hbck should do offline reference repair before online repair 2016-09-20 11:53:01 -07:00
zhangduo 91d03697d9 HBASE-16165 Decrease RpcServer.callQueueSize before writeResponse causes OOM (Guanghao Zhang) 2016-09-18 10:05:48 +08:00
tedyu 591cc4cfb8 HBASE-16349 TestClusterId may hang during cluster shutdown 2016-09-16 06:57:42 -07:00
Matteo Bertozzi 08e44919cc HBASE-16639 TestProcedureInMemoryChore#testChoreAddAndRemove occasionally fails 2016-09-15 18:32:53 -07:00
tedyu 76a0760269 HBASE-16640 TimeoutBlockingQueue#remove() should return whether the entry is removed 2016-09-15 17:35:59 -07:00
Phil Yang 069d1f73fa HBASE-16388 Prevent client threads being blocked by only one slow region server
Signed-off-by: stack <stack@apache.org>
2016-09-14 09:28:20 -07:00
tedyu ac1ee77f40 HBASE-16612 Use array to cache Types for KeyValue.Type.codeToType (Phil Yang) 2016-09-13 12:44:36 -07:00
tedyu 8e25ea536a HBASE-16613 Return the unused ByteBuffer to BoundedByteBufferPool when no cell is retrieved from the CellScanner (ChiaPing Tsai) 2016-09-13 09:49:13 -07:00
Umesh Agashe 059a169d3a HBASE-15297 Correct handling of namespace existence checks in shell.
Changes namespace_exists? method in SecurityAdmin ruby code to catch NamespaceNotFoundException
and modified Admin.java file to document the exception.

Signed-off-by: Sean Busbey <busbey@apache.org>

 Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
2016-09-12 23:58:21 -07:00
Dustin Pho 7028a0d889 HBASE-16540 Adding checks in Scanner's setStartRow and setStopRow for invalid row key sizes.
Signed-off-by: Gary Helmling <garyh@apache.org>
2016-09-12 17:38:15 -07:00
tedyu 8ad14bac67 HBASE-16616 Rpc handlers stuck on ThreadLocalMap.expungeStaleEntry (Tomu Tsuruhara) 2016-09-12 16:29:24 -07:00
chenheng 94026d0d09 HBASE-16086 TableCfWALEntryFilter and ScopeWALEntryFilter should not redundantly iterate over cells (Vincent Poon)
Conflicts:
    hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ScopeWALEntryFilter.java
    hbase-server/src/main/java/org/apache/hadoop/hbase/replication/TableCfWALEntryFilter.java
    hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java
2016-09-11 10:48:00 +08:00
Andrew Purtell fe57fa4daa HBASE-16576 Shell add_peer doesn't allow setting cluster_key for custom endpoints
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:

	hbase-shell/src/main/ruby/shell/commands/add_peer.rb
	hbase-shell/src/test/ruby/hbase/replication_admin_test.rb
2016-09-09 14:54:15 -07:00
binlijin ad67fd0816 HBASE-16530 Reduce DBE code duplication
Signed-off-by: Yu Li <liyu@apache.org>
2016-09-09 16:25:56 +08:00
zhangduo bc4ab47147 HBASE-16309 TestDefaultCompactSelection.testCompactionRatio is flaky 2016-09-09 14:21:31 +08:00
chenheng b4086795f2 HBASE-16570 Compute region locality in parallel at startup (binlijin) 2016-09-09 14:16:37 +08:00
Andrew Purtell 52963b3428 HBASE-16589 Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal 2016-09-08 13:03:49 -07:00
tedyu 354706a7d8 HBASE-16572 Sync method in RecoverableZooKeeper failed to pass callback function in (Allan Yang) 2016-09-07 11:36:18 -07:00
chenheng 6f73ef2dff HBASE-16562 ITBLL should fail to start if misconfigured, addendum 2016-09-07 15:45:09 +08:00
chenheng 38b946c276 HBASE-16562 ITBLL should fail to start if misconfigured 2016-09-07 13:50:36 +08:00
tedyu b694b63ed7 HBASE-16460 Can't rebuild the BucketAllocator's data structures when BucketCache uses FileIOEngine (Guanghao Zhang) 2016-09-05 06:52:03 -07:00
tedyu e1aab356b3 HBASE-16556 The read/write timeout are not used in HTable.delete(List), HTable.get(List), and HTable.existsAll(List) (ChiaPing Tsai) 2016-09-04 21:15:10 -07:00
tedyu a0e52a2dab HBASE-16541 Avoid unnecessary cell copy in Result#compareResults (ChiaPing Tsai) 2016-09-04 07:08:38 -07:00
Andrew Purtell abbb36f625 Amend HBASE-16538 Changes the way version information is stored during build
Generated Version class needs an interface audience annotation
2016-09-02 19:54:14 -07:00