hbase/CHANGES.txt

88 lines
5.0 KiB
Plaintext
Raw Normal View History

HBase Change Log
Trunk (unreleased changes)
1. HADOOP-1384. HBase omnibus patch. (jimk, Vuk Ercegovac, and Michael Stack)
2. HADOOP-1402. Fix javadoc warnings in hbase contrib. (Michael Stack)
3. HADOOP-1404. HBase command-line shutdown failing (Michael Stack)
4. HADOOP-1397. Replace custom hbase locking with
java.util.concurrent.locks.ReentrantLock (Michael Stack)
5. HADOOP-1403. HBase reliability - make master and region server more fault
tolerant.
6. HADOOP-1418. HBase miscellaneous: unit test for HClient, client to do
'Performance Evaluation', etc.
7. HADOOP-1420, HADOOP-1423. Findbugs changes, remove reference to removed
class HLocking.
8. HADOOP-1424. TestHBaseCluster fails with IllegalMonitorStateException. Fix
regression introduced by HADOOP-1397.
9. HADOOP-1426. Make hbase scripts executable + add test classes to CLASSPATH.
10. HADOOP-1430. HBase shutdown leaves regionservers up.
11. HADOOP-1392. Part1: includes create/delete table; enable/disable table;
add/remove column.
12. HADOOP-1392. Part2: includes table compaction by merging adjacent regions
that have shrunk in size.
13. HADOOP-1445 Support updates across region splits and compactions
14. HADOOP-1460 On shutdown IOException with complaint 'Cannot cancel lease
that is not held'
15. HADOOP-1421 Failover detection, split log files.
For the files modified, also clean up javadoc, class, field and method
visibility (HADOOP-1466)
16. HADOOP-1479 Fix NPE in HStore#get if store file only has keys < passed key.
17. HADOOP-1476 Distributed version of 'Performance Evaluation' script
18. HADOOP-1469 Asychronous table creation
19. HADOOP-1415 Integrate BSD licensed bloom filter implementation.
20. HADOOP-1465 Add cluster stop/start scripts for hbase
21. HADOOP-1415 Provide configurable per-column bloom filters - part 2.
22. HADOOP-1498. Replace boxed types with primitives in many places.
23. HADOOP-1509. Made methods/inner classes in HRegionServer and HClient protected
instead of private for easier extension. Also made HRegion and HRegionInfo public too.
Added an hbase-default.xml property for specifying what HRegionInterface extension to use
2007-07-05 15:50:04 -04:00
for proxy server connection. (James Kennedy via Jim Kellerman)
24. HADOOP-1534. [hbase] Memcache scanner fails if start key not present
25. HADOOP-1537. Catch exceptions in testCleanRegionServerExit so we can see
what is failing.
26. HADOOP-1543 [hbase] Add HClient.tableExists
2007-07-05 15:50:04 -04:00
27. HADOOP-1519 [hbase] map/reduce interface for HBase. (Vuk Ercegovac and
Jim Kellerman)
HADOOP-1523 'Hung region servers waiting on write locks' On shutdown, region servers and masters were just cancelling leases without letting 'lease expired' code run -- code to clean up outstanding locks in region server. Outstanding read locks were getting in the way of region server getting necessary write locks needed for the shutdown process. Also, cleaned up messaging around shutdown so its clean -- no timeout messages as region servers try to talk to a master that has already shutdown -- even when region servers take their time going down. M src/contrib/hbase/conf/hbase-default.xml Make region server timeout 30 seconds instead of 3 minutes. Clients retry anyways. Make so its likely region servers report in their shutdown message before their lease expires on master. M src/contrib/hbase/src/java/org/apache/hadoop/hbase/Leases.java (closeAfterLeasesExpire): Added. * src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionServer.java Added comments. (stop): Converted from public to default access (master shuts down regionservers). (run): Use leases.closeAfterLeasesExpire instead of leases.close. Changed log of main thread exit from DEBUG to INFO. * src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMaster.java (letRegionsServersShutdown): Add better explaination of shutdown process to method doc. Changed timeout waits from hbase.regionserver.msginterval to threadWakeFrequency. (regionServerReport): If closing, we used to immediately respond to region server with a MSG_REGIONSERVER_STOP. This meant that we avoided handling of the region servers MSG_REPORT_EXITING sent on shutdown so region servers had no chance to cancel their lease in the master. Reordered. Moved sending of MSG_REGIONSERVER_STOP to after handling of MSG_REPORT_EXITING. Also, in handling of MSG_REGIONSERER_STOP removed cancelling of leases. Let leases expire normally (or get cancelled when the region server comes in with MSG_RPORT_EXITING). * src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMsg.java (MSG_REGIONSERVER_STOP_IN_ARRAY): Added. git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@552376 13f79535-47bb-0310-9956-ffa450edef68
2007-07-01 20:47:13 -04:00
28. HADOOP-1523 Hung region server waiting on write locks
29. HADOOP-1560 NPE in MiniHBaseCluster on Windows
2007-07-05 15:50:04 -04:00
30. HADOOP-1531 Add RowFilter to HRegion.HScanner
Adds a row filtering interface and two implemenentations: A page scanner,
and a regex row/column-data matcher. (James Kennedy via Stack)
31. HADOOP-1566 Key-making utility
32. HADOOP-1415 Provide configurable per-column bloom filters.
HADOOP-1466 Clean up visibility and javadoc issues in HBase.
33. HADOOP-1538 Provide capability for client specified time stamps in HBase
HADOOP-1466 Clean up visibility and javadoc issues in HBase.
34. HADOOP-1589 Exception handling in HBase is broken over client server connections
HADOOP-1375 a simple parser for hbase M src/contrib/hbase/NOTICE.txt Add notice of udanax contributions. Msrc/contrib/hbase/conf/hbase-default.xml (hbaseshell.jline.bell.enabled): Added. M src/contrib/hbase/CHANGES.txt (hadoop-1375) Added. M src/contrib/hbase/src/java/org/apache/hadoop/hbase/package.html Add note on how to start up hbase shell M src/contrib/hbase/bin/hbase Add 'shell'. Remove 'client' (shell does what it used do and more). Removed all reader and logreader until better developed. Starting up a reader or logreader on a running hbase system could do damage). M src/contrib/hbase/build.xml Add a javacc target to generate content of shell/generated subpackage. A src/contrib/hbase/src/test/org/apache/hadoop/hbase/shell/TestHBaseShell.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/Shell.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/DeleteCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/CreateCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/DropCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/InsertCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/CommandFactory.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/HelpContents.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/ExitCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/ConsoleTable.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/DescCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/SelectCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/Command.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/ShowCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/BasicCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/HelpManager.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/ReturnMsg.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/HelpCommand.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/HBaseShell.jj Added. A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/generated/Token.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/generated/TokenMgrError.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/generated/SimpleCharStream.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/generated/ParserTokenManager.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/generated/ParseException.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/generated/ParserConstants.java A src/contrib/hbase/src/java/org/apache/hadoop/hbase/shell/generated/Parser.java Added javacc generated files. git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@555415 13f79535-47bb-0310-9956-ffa450edef68
2007-07-11 17:54:15 -04:00
35. HADOOP-1375 a simple parser for hbase (Edward Yoon via Stack)
36. HADOOP-1600 Update license in HBase code
37. HADOOP-1589 Exception handling in HBase is broken over client server
38. HADOOP-1574 Concurrent creates of a table named 'X' all succeed
39. HADOOP-1581 Un-openable tablename bug
40. HADOOP-1607 [shell] Clear screen command (Edward Yoon via Stack)
41. HADOOP-1614 [hbase] HClient does not protect itself from simultaneous updates
42. HADOOP-1468 Add HBase batch update to reduce RPC overhead
43. HADOOP-1616 Sporadic TestTable failures
44. HADOOP-1615 Replacing thread notification-based queue with
java.util.concurrent.BlockingQueue in HMaster, HRegionServer
45. HADOOP-1606 Updated implementation of RowFilterSet, RowFilterInterface
(Izaak Rubin via Stack)
46. HADOOP-1579 Add new WhileMatchRowFilter and StopRowFilter filters
(Izaak Rubin via Stack)
47. HADOOP-1637 Fix to HScanner to Support Filters, Add Filter Tests to
TestScanner2 (Izaak Rubin via Stack)
HADOOP-1516 HClient fails to readjust when ROOT or META redeployed on new region server Detailed changes: MiniHBaseCluster - rewrite abortRegionServer, stopRegionServer - they now remove the server from the map of servers. - rewrite waitOnRegionServer - now removes thread from map of threads TestCleanRegionServerExit - reduce Hadoop ipc client timeout and number of retries - use rewritten stopRegionServer and waitOnRegionServer from MiniHBaseCluster - add code to verify that failover worked - moved testRegionServerAbort to separate test file TestRegionServerAbort - new test. Uses much the same code as TestCleanRegionServerExit but aborts the region server instead of shutting it down cleanly. Includes code to verify that failover worked. hbase-site.xml (in src/contrib/hbase/src/test) - reduce master lease timeout and time between lease timeout checks so that tests will run quicker. HClient - Major restructing of code that determines what region server to contact for a specific region. The main method findServersForTable is now recursive so that it will find the meta and root regions if they have not already been located or will re-find them if they have been reassigned and the old server can no longer be contacted. - re-ordered administrative and general purpose methods so they are no longer located in seemingly random order. - re-ordered code in ClientScanner.loadRegions so that if the location of the region changes, it will actually try to connect to the new server rather than continually trying to use the connection to the old server. HLog - use HashMap<Text, SequenceFile.Writer> instead of TreeMap<Text, SequenceFile.Writer> because the TreeMap would return a value for a key it did not have (it was the value of another key). I have observed this before when the key is Text, but could not create a simple test case that reproduced the problem. - added some new DEBUG level logging - removed call to rollWriter() from closeAndDelete(). We don't need to start a new writer if we are closing the log. HLogKey - cleaned up per HADOOP-1466 (I initially modified it to add some debug logging which was later removed, but when I was making the modifications I took the opportunity to clean up the file) - changed toString() format HMaster - better handling of RemoteException - modified BaseScanner - now knows if it is scanning the root or a meta region - scanRegion no longer returns a value - if scanning the root region, it counts the number of meta regions it finds and sets a new AtomicInteger, numberOfMetaRegions when the scan is complete. - added abstract methods initialScan and maintenanceScan this allowed run method to be implemented in the base class. - boolean rootScanned is now volatile - modified RootScanner - moved actual scan into private method for readability (scanRoot) - implementation of abstract methods just call scanRoot - add constructor for inner static class MetaRegion - use a BlockingQueue to queue up work for the MetaScanner - clean up handling of an unexpected region server exit - PendingOperation.process now returns a boolean so that HMaster.run can determine if the operation completed or needs to be retried later - PendingOperation processing no longer does a wait inside the process method since this might cause a deadlock if the current operation is waiting for another operation that has yet to be processed HMsg - removed MSG_REGIONSERVER_STOP_IN_ARRAY, MSG_NEW_REGION - added MSG_REPORT_SPLIT HRegionServer - changed reportSplit to contain old region and new regions - use IP from default interface rather than host name - abort calls HLog.close() instead of HLog.rollWriter() git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@559819 13f79535-47bb-0310-9956-ffa450edef68
2007-07-26 10:15:17 -04:00
48. HADOOP-1516 HClient fails to readjust when ROOT or META redeployed on new
region server
49. HADOOP-1646 RegionServer OOME's under sustained, substantial loading by
10 concurrent clients
50. HADOOP-1468 Add HBase batch update to reduce RPC overhead (restrict batches
to a single row at a time)
HADOOP-1528 HClient for multiple tables (phase 1) Modified: HConstants static final Text[] COL_REGIONINFO_ARRAY = new Text [] {COL_REGIONINFO}; static final Text EMPTY_START_ROW = new Text(); HMaster - don't process a region server exit message if the lease has timed out. Otherwise we end up with two pending server shutdown messages to process and chaos ensues. - don't reassign the root region when the server's lease expires. The lease expiration handler will queue a PendingServerShutdown operation that must run before the root region is reassigned because the HLog of the dead server must be split before any regions served by the dead server are reassigned. - added some additional debug level logging HBaseClusterTestCase - call HConnectionManager.deleteConnection(conf) in tearDown() so that multiple tests can be run from the same test class. TestScanner2 - changes to make test compatible with the change from inner class HClient.RegionLocation to public class HRegionLocation Leases - cancelLease just returns if the lease is not found instead of throwing an IOException New: HConnection - an interface that describes the operations performed by a connection implementation HConnectionManager - manages connections for multiple HBase instances and returns an object that implements HConnection from its static method getConnection HBaseAdmin - the HBase administrative methods refactored out of HClient. Each HBaseAdmin object can control a single HBase instance. To manipulate multiple instances, create multiple HBaseAdmin objects. HTable - The data manipulation methods refactored out of HClient. Each HTable object talks to a single table in a single HBase instance. Create multiple HTable objects to use more than one table. HRegionLocation - an inner class refactored out of HClient. Each HRegionLocation has an HRegionInfo object and an HServerAddress object. HClient - totally re-implemented in terms of the new classes above. HClient is now deprecated. git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@561935 13f79535-47bb-0310-9956-ffa450edef68
2007-08-01 16:10:11 -04:00
51. HADOOP-1528 HClient for multiple tables (phase 1)
52. HADOOP-1528 HClient for multiple tables (phase 2) all HBase client side code
(except TestHClient and HBaseShell) have been converted to use the new client
side objects (HTable/HBaseAdmin/HConnection) instead of HClient.
53. HADOOP-1528 HClient for multiple tables - expose close table function
54. HADOOP-1466 Clean up warnings, visibility and javadoc issues in HBase.