HBase Change Log Trunk (unreleased changes) INCOMPATIBLE CHANGES NEW FEATURES HADOOP-2061 Add new Base64 dialects HADOOP-2084 Add a LocalHBaseCluster OPTIMIZATIONS BUG FIXES HADOOP-2059 In tests, exceptions in min dfs shutdown should not fail test (e.g. nightly #272) HADOOP-2064 TestSplit assertion and NPE failures (Patch build #952 and #953) HADOOP-2056 A table with row keys containing colon fails to split regions HADOOP-2079 HADOOP-2056 Fix generated HLog, HRegion names HADOOP-2124 Use of `hostname` does not work on Cygwin in some cases HADOOP-2083 TestTableIndex failed in #970 and #956 HADOOP-2109 Fixed race condition in processing server lease timeout. HADOOP-2137 hql.jsp : The character 0x19 is not valid HADOOP-2109 Fix another race condition in processing dead servers, Fix error online meta regions: was using region name and not startKey as key for map.put. Change TestRegionServerExit to always kill the region server for the META region. This makes the test more deterministic and getting META reassigned was problematic. HADOOP-2155 Method expecting HBaseConfiguration throws NPE when given Configuration HADOOP-2156 BufferUnderflowException for un-named HTableDescriptors HADOOP-2161 getRow() is orders of magnitudes slower than get(), even on rows with one column (Clint Morgan and Stack) HADOOP-2040 Hudson hangs AFTER test has finished HADOOP-2274 Excess synchronization introduced by HADOOP-2139 negatively impacts performance HADOOP-2196 Fix how hbase sits in hadoop 'package' product HADOOP-2276 Address regression caused by HADOOP-2274, fix HADOOP-2173 (When the master times out a region servers lease, the region server may not restart) IMPROVEMENTS HADOOP-2401 Add convenience put method that takes writable (Johan Oskarsson via Stack) HADOOP-2074 Simple switch to enable DEBUG level-logging in hbase HADOOP-2088 Make hbase runnable in $HADOOP_HOME/build(/contrib/hbase) HADOOP-2126 Use Bob Jenkins' hash for bloom filters HADOOP-2157 Make Scanners implement Iterable HADOOP-2176 Htable.deleteAll documentation is ambiguous HADOOP-2139 (phase 1) Increase parallelism in region servers. HADOOP-2267 [Hbase Shell] Change the prompt's title from 'hbase' to 'hql'. HADOOP-2139 (phase 2) Make region server more event driven HADOOP-2289 Useless efforts of looking for the non-existant table in select command. HADOOP-2262 HADOOP-2261 fail fast on non-existing table, change abort to function after commit even if commit was successful Release 0.15.1 Branch 0.15 INCOMPATIBLE CHANGES HADOOP-1931 Hbase scripts take --ARG=ARG_VALUE when should be like hadoop and do ---ARG ARG_VALUE NEW FEATURES HADOOP-1768 FS command using Hadoop FsShell operations (Edward Yoon via Stack) HADOOP-1784 Delete: Fix scanners and gets so they work properly in presence of deletes. Added a deleteAll to remove all cells equal to or older than passed timestamp. Fixed compaction so deleted cells do not make it out into compacted output. Ensure also that versions > column max are dropped compacting. HADOOP-1720 Addition of HQL (Hbase Query Language) support in Hbase Shell. The old shell syntax has been replaced by HQL, a small SQL-like set of operators, for creating, altering, dropping, inserting, deleting, and selecting, etc., data in hbase. (Inchul Song and Edward Yoon via Stack) HADOOP-1913 Build a Lucene index on an HBase table (Ning Li via Stack) HADOOP-1957 Web UI with report on cluster state and basic browsing of tables OPTIMIZATIONS BUG FIXES HADOOP-1527 Region server won't start because logdir exists HADOOP-1723 If master asks region server to shut down, by-pass return of shutdown message HADOOP-1729 Recent renaming or META tables breaks hbase shell HADOOP-1730 unexpected null value causes META scanner to exit (silently) HADOOP-1747 On a cluster, on restart, regions multiply assigned HADOOP-1776 Fix for sporadic compaction failures closing and moving compaction result HADOOP-1780 Regions are still being doubly assigned HADOOP-1797 Fix NPEs in MetaScanner constructor HADOOP-1799 Incorrect classpath in binary version of Hadoop HADOOP-1805 Region server hang on exit HADOOP-1785 TableInputFormat.TableRecordReader.next has a bug (Ning Li via Stack) HADOOP-1800 output should default utf8 encoding HADOOP-1801 When hdfs is yanked out from under hbase, hbase should go down gracefully HADOOP-1813 OOME makes zombie of region server HADOOP-1814 TestCleanRegionServerExit fails too often on Hudson HADOOP-1820 Regionserver creates hlogs without bound (reverted 2007/09/25) (Fixed 2007/09/30) HADOOP-1821 Replace all String.getBytes() with String.getBytes("UTF-8") HADOOP-1832 listTables() returns duplicate tables HADOOP-1834 Scanners ignore timestamp passed on creation HADOOP-1847 Many HBase tests do not fail well. HADOOP-1847 Many HBase tests do not fail well. (phase 2) HADOOP-1870 Once file system failure has been detected, don't check it again and get on with shutting down the hbase cluster. HADOOP-1888 NullPointerException in HMemcacheScanner (reprise) HADOOP-1903 Possible data loss if Exception happens between snapshot and flush to disk. HADOOP-1920 Wrapper scripts broken when hadoop in one location and hbase in another HADOOP-1923, HADOOP-1924 a) tests fail sporadically because set up and tear down is inconsistent b) TestDFSAbort failed in nightly #242 HADOOP-1929 Add hbase-default.xml to hbase jar HADOOP-1941 StopRowFilter throws NPE when passed null row HADOOP-1966 Make HBase unit tests more reliable in the Hudson environment. HADOOP-1975 HBase tests failing with java.lang.NumberFormatException HADOOP-1990 Regression test instability affects nightly and patch builds HADOOP-1996 TestHStoreFile fails on windows if run multiple times HADOOP-1937 When the master times out a region server's lease, it is too aggressive in reclaiming the server's log. HADOOP-2004 webapp hql formatting bugs HADOOP_2011 Make hbase daemon scripts take args in same order as hadoop daemon scripts HADOOP-2017 TestRegionServerAbort failure in patch build #903 and nightly #266 HADOOP-2029 TestLogRolling fails too often in patch and nightlies HADOOP-2038 TestCleanRegionExit failed in patch build #927 IMPROVEMENTS HADOOP-1737 Make HColumnDescriptor data publically members settable HADOOP-1746 Clean up findbugs warnings HADOOP-1757 Bloomfilters: single argument constructor, use enum for bloom filter types HADOOP-1760 Use new MapWritable and SortedMapWritable classes from org.apache.hadoop.io HADOOP-1793 (Phase 1) Remove TestHClient (Phase2) remove HClient. HADOOP-1794 Remove deprecated APIs HADOOP-1802 Startup scripts should wait until hdfs as cleared 'safe mode' HADOOP-1833 bin/stop_hbase.sh returns before it completes (Izaak Rubin via Stack) HADOOP-1835 Updated Documentation for HBase setup/installation (Izaak Rubin via Stack) HADOOP-1868 Make default configuration more responsive HADOOP-1884 Remove useless debugging log messages from hbase.mapred HADOOP-1856 Add Jar command to hbase shell using Hadoop RunJar util (Edward Yoon via Stack) HADOOP-1928 Have master pass the regionserver the filesystem to use HADOOP-1789 Output formatting HADOOP-1960 If a region server cannot talk to the master before its lease times out, it should shut itself down HADOOP-2035 Add logo to webapps Below are the list of changes before 2007-08-18 1. HADOOP-1384. HBase omnibus patch. (jimk, Vuk Ercegovac, and Michael Stack) 2. HADOOP-1402. Fix javadoc warnings in hbase contrib. (Michael Stack) 3. HADOOP-1404. HBase command-line shutdown failing (Michael Stack) 4. HADOOP-1397. Replace custom hbase locking with java.util.concurrent.locks.ReentrantLock (Michael Stack) 5. HADOOP-1403. HBase reliability - make master and region server more fault tolerant. 6. HADOOP-1418. HBase miscellaneous: unit test for HClient, client to do 'Performance Evaluation', etc. 7. HADOOP-1420, HADOOP-1423. Findbugs changes, remove reference to removed class HLocking. 8. HADOOP-1424. TestHBaseCluster fails with IllegalMonitorStateException. Fix regression introduced by HADOOP-1397. 9. HADOOP-1426. Make hbase scripts executable + add test classes to CLASSPATH. 10. HADOOP-1430. HBase shutdown leaves regionservers up. 11. HADOOP-1392. Part1: includes create/delete table; enable/disable table; add/remove column. 12. HADOOP-1392. Part2: includes table compaction by merging adjacent regions that have shrunk in size. 13. HADOOP-1445 Support updates across region splits and compactions 14. HADOOP-1460 On shutdown IOException with complaint 'Cannot cancel lease that is not held' 15. HADOOP-1421 Failover detection, split log files. For the files modified, also clean up javadoc, class, field and method visibility (HADOOP-1466) 16. HADOOP-1479 Fix NPE in HStore#get if store file only has keys < passed key. 17. HADOOP-1476 Distributed version of 'Performance Evaluation' script 18. HADOOP-1469 Asychronous table creation 19. HADOOP-1415 Integrate BSD licensed bloom filter implementation. 20. HADOOP-1465 Add cluster stop/start scripts for hbase 21. HADOOP-1415 Provide configurable per-column bloom filters - part 2. 22. HADOOP-1498. Replace boxed types with primitives in many places. 23. HADOOP-1509. Made methods/inner classes in HRegionServer and HClient protected instead of private for easier extension. Also made HRegion and HRegionInfo public too. Added an hbase-default.xml property for specifying what HRegionInterface extension to use for proxy server connection. (James Kennedy via Jim Kellerman) 24. HADOOP-1534. [hbase] Memcache scanner fails if start key not present 25. HADOOP-1537. Catch exceptions in testCleanRegionServerExit so we can see what is failing. 26. HADOOP-1543 [hbase] Add HClient.tableExists 27. HADOOP-1519 [hbase] map/reduce interface for HBase. (Vuk Ercegovac and Jim Kellerman) 28. HADOOP-1523 Hung region server waiting on write locks 29. HADOOP-1560 NPE in MiniHBaseCluster on Windows 30. HADOOP-1531 Add RowFilter to HRegion.HScanner Adds a row filtering interface and two implemenentations: A page scanner, and a regex row/column-data matcher. (James Kennedy via Stack) 31. HADOOP-1566 Key-making utility 32. HADOOP-1415 Provide configurable per-column bloom filters. HADOOP-1466 Clean up visibility and javadoc issues in HBase. 33. HADOOP-1538 Provide capability for client specified time stamps in HBase HADOOP-1466 Clean up visibility and javadoc issues in HBase. 34. HADOOP-1589 Exception handling in HBase is broken over client server connections 35. HADOOP-1375 a simple parser for hbase (Edward Yoon via Stack) 36. HADOOP-1600 Update license in HBase code 37. HADOOP-1589 Exception handling in HBase is broken over client server 38. HADOOP-1574 Concurrent creates of a table named 'X' all succeed 39. HADOOP-1581 Un-openable tablename bug 40. HADOOP-1607 [shell] Clear screen command (Edward Yoon via Stack) 41. HADOOP-1614 [hbase] HClient does not protect itself from simultaneous updates 42. HADOOP-1468 Add HBase batch update to reduce RPC overhead 43. HADOOP-1616 Sporadic TestTable failures 44. HADOOP-1615 Replacing thread notification-based queue with java.util.concurrent.BlockingQueue in HMaster, HRegionServer 45. HADOOP-1606 Updated implementation of RowFilterSet, RowFilterInterface (Izaak Rubin via Stack) 46. HADOOP-1579 Add new WhileMatchRowFilter and StopRowFilter filters (Izaak Rubin via Stack) 47. HADOOP-1637 Fix to HScanner to Support Filters, Add Filter Tests to TestScanner2 (Izaak Rubin via Stack) 48. HADOOP-1516 HClient fails to readjust when ROOT or META redeployed on new region server 49. HADOOP-1646 RegionServer OOME's under sustained, substantial loading by 10 concurrent clients 50. HADOOP-1468 Add HBase batch update to reduce RPC overhead (restrict batches to a single row at a time) 51. HADOOP-1528 HClient for multiple tables (phase 1) (James Kennedy & JimK) 52. HADOOP-1528 HClient for multiple tables (phase 2) all HBase client side code (except TestHClient and HBaseShell) have been converted to use the new client side objects (HTable/HBaseAdmin/HConnection) instead of HClient. 53. HADOOP-1528 HClient for multiple tables - expose close table function 54. HADOOP-1466 Clean up warnings, visibility and javadoc issues in HBase. 55. HADOOP-1662 Make region splits faster 56. HADOOP-1678 On region split, master should designate which host should serve daughter splits. Phase 1: Master balances load for new regions and when a region server fails. 57. HADOOP-1678 On region split, master should designate which host should serve daughter splits. Phase 2: Master assigns children of split region instead of HRegionServer serving both children. 58. HADOOP-1710 All updates should be batch updates 59. HADOOP-1711 HTable API should use interfaces instead of concrete classes as method parameters and return values 60. HADOOP-1644 Compactions should not block updates 60. HADOOP-1672 HBase Shell should use new client classes (Edward Yoon via Stack). 61. HADOOP-1709 Make HRegionInterface more like that of HTable HADOOP-1725 Client find of table regions should not include offlined, split parents