38 KiB
Executable File
38 KiB
Executable File
HBASE Changelog
Release 2.1.0 - Unreleased (as of 2018-07-07)
INCOMPATIBLE CHANGES:
JIRA | Summary | Priority | Component |
---|---|---|---|
HBASE-20691 | Storage policy should allow deferring to HDFS | Blocker | Filesystem Integration, wal |
HBASE-20270 | Turn off command help that follows all errors in shell | Major | shell |
HBASE-20501 | Change the Hadoop minimum version to 2.7.1 | Blocker | community, documentation |
HBASE-20406 | HBase Thrift HTTP - Shouldn't handle TRACE/OPTIONS methods | Major | security, Thrift |
HBASE-20159 | Support using separate ZK quorums for client | Major | Client, Operability, Zookeeper |
NEW FEATURES:
JIRA | Summary | Priority | Component |
---|---|---|---|
HBASE-15809 | Basic Replication WebUI | Critical | Replication, UI |
HBASE-19735 | Create a minimal "client" tarball installation | Major | build, Client |
HBASE-20656 | Validate pre-2.0 coprocessors against HBase 2.0+ | Major | tooling |
HBASE-20592 | Create a tool to verify tables do not have prefix tree encoding | Minor | Operability, tooling |
HBASE-20046 | Reconsider the implementation for serial replication | Major | Replication |
HBASE-19397 | Design procedures for ReplicationManager to notify peer change event from master | Major | proc-v2, Replication |
IMPROVEMENTS:
JIRA | Summary | Priority | Component |
---|---|---|---|
HBASE-20474 | Show non-RPC tasks on master/regionserver Web UI by default | Major | UI |
HBASE-20826 | Truncate responseInfo attributes on RpcServer WARN messages | Major | rpc |
HBASE-20450 | Provide metrics for number of total active, priority and replication rpc handlers | Major | metrics |
HBASE-20810 | Include the procedure id in the exception message in HBaseAdmin for better debugging | Major | Admin, proc-v2 |
HBASE-20040 | Master UI should include "Cluster Key" needed to use the cluster as a replication sink | Minor | Replication, Usability |
HBASE-20095 | Redesign single instance pool in CleanerChore | Critical | . |
HBASE-19164 | Avoid UUID.randomUUID in tests | Major | test |
HBASE-20739 | Add priority for SCP | Major | Recovery |
HBASE-20737 | put collection into ArrayList instead of addAll function | Trivial | . |
HBASE-20695 | Implement table level RegionServer replication metrics | Minor | metrics |
HBASE-20733 | QABot should run checkstyle tests if the checkstyle configs change | Minor | build, community |
HBASE-20625 | refactor some WALCellCodec related code | Minor | wal |
HBASE-19852 | HBase Thrift 1 server SPNEGO Improvements | Major | Thrift |
HBASE-20579 | Improve snapshot manifest copy in ExportSnapshot | Minor | mapreduce |
HBASE-20444 | Improve version comparison logic for HBase specific version string and add unit tests | Major | util |
HBASE-20594 | provide utility to compare old and new descriptors | Major | . |
HBASE-20640 | TestQuotaGlobalsSettingsBypass missing test category and ClassRule | Critical | test |
HBASE-20478 | move import checks from hbaseanti to checkstyle | Minor | test |
HBASE-20548 | Master fails to startup on large clusters, refreshing block distribution | Major | . |
HBASE-20488 | PE tool prints full name in help message | Minor | shell |
HBASE-20567 | Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace | Major | . |
HBASE-20545 | Improve performance of BaseLoadBalancer.retainAssignment | Major | Balancer |
HBASE-16191 | Add stop_regionserver and stop_master to shell | Major | . |
HBASE-20536 | Make TestRegionServerAccounting stable and it should not use absolute number | Minor | . |
HBASE-20523 | PE tool should support configuring client side buffering sizes | Minor | . |
HBASE-20527 | Remove unused code in MetaTableAccessor | Trivial | . |
HBASE-20507 | Do not need to call recoverLease on the broken file when we fail to create a wal writer | Major | wal |
HBASE-20484 | Remove the unnecessary autoboxing in FilterListBase | Trivial | . |
HBASE-20327 | When qualifier is not specified, append and incr operation do not work (shell) | Minor | shell |
HBASE-20389 | Move website building flags into a profile | Minor | build, website |
HBASE-20379 | shadedjars yetus plugin should add a footer link | Major | test |
HBASE-20243 | [Shell] Add shell command to create a new table by cloning the existent table | Minor | shell |
HBASE-20286 | Improving shell command compaction_state | Minor | shell |
HBASE-19488 | Move to using Apache commons CollectionUtils | Trivial | . |
HBASE-20197 | Review of ByteBufferWriterOutputStream.java | Minor | . |
HBASE-20047 | AuthenticationTokenIdentifier should provide a toString | Minor | Usability |
HBASE-19024 | Configurable default durability for synchronous WAL | Critical | wal |
HBASE-19389 | Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted | Critical | Performance |
HBASE-20186 | Improve RSGroupBasedLoadBalancer#balanceCluster() to be more efficient when calculating cluster state for each rsgroup | Minor | rsgroup |
HBASE-19449 | Minor logging change in HFileArchiver | Trivial | . |
HBASE-20120 | Remove some unused classes/ java files from hbase-server | Minor | . |
BUG FIXES:
JIRA | Summary | Priority | Component |
---|---|---|---|
HBASE-20808 | Wrong shutdown order between Chores and ChoreService | Minor | . |
HBASE-20789 | TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky | Major | . |
HBASE-20829 | Remove the addFront assertion in MasterProcedureScheduler.doAdd | Major | Replication |
HBASE-20825 | Fix pre and post hooks of CloneSnapshot and RestoreSnapshot for Access checks | Major | security |
HBASE-20812 | Add defaults to Table Interface so implementors don't have to | Major | . |
HBASE-20817 | Infinite loop when executing ReopenTableRegionsProcedure | Blocker | Region Assignment |
HBASE-20792 | info:servername and info:sn inconsistent for OPEN region | Blocker | Region Assignment |
HBASE-20769 | getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl | Major | . |
HBASE-20732 | Shutdown scan pool when master is stopped. | Minor | . |
HBASE-20785 | NPE getting metrics in PE testing scans | Major | Performance |
HBASE-20795 | Allow option in BBKVComparator.compare to do comparison without sequence id | Major | . |
HBASE-20777 | RpcConnection could still remain opened after we shutdown the NettyRpcServer | Major | rpc |
HBASE-20403 | Prefetch sometimes doesn't work with encrypted file system | Major | . |
HBASE-20635 | Support to convert the shaded user permission proto to client user permission object | Major | . |
HBASE-20778 | Make it so WALPE runs on DFS | Major | test |
HBASE-20775 | TestMultiParallel is flakey | Major | Region Assignment |
HBASE-20752 | Make sure the regions are truly reopened after ReopenTableRegionsProcedure | Major | proc-v2 |
HBASE-18622 | Mitigate API compatibility concerns between branch-1 and branch-2 | Blocker | API |
HBASE-20767 | Always close hbaseAdmin along with connection in HBTU | Major | test |
HBASE-20642 | IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException | Major | . |
HBASE-20742 | Always create WAL directory for region server | Major | wal |
HBASE-20708 | Remove the usage of RecoverMetaProcedure in master startup | Blocker | proc-v2, Region Assignment |
HBASE-20723 | Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them | Critical | Recovery, wal |
HBASE-20681 | IntegrationTestDriver fails after HADOOP-15406 due to missing hamcrest-core | Major | integration tests |
HBASE-20561 | The way we stop a ReplicationSource may cause the RS down | Major | Replication |
HBASE-19377 | Compatibility checker complaining about hash collisions | Major | community |
HBASE-20689 | Docker fails to install rubocop for precommit | Blocker | build |
HBASE-20707 | Move MissingSwitchDefault check from checkstyle to error-prone | Major | build |
HBASE-20699 | QuotaCache should cancel the QuotaRefresherChore service inside its stop() | Major | . |
HBASE-20590 | REST Java client is not able to negotiate with the server in the secure mode | Critical | REST, security |
HBASE-20683 | Incorrect return value for PreUpgradeValidator | Critical | . |
HBASE-20684 | org.apache.hadoop.hbase.client.Scan#setStopRow javadoc uses incorrect method | Trivial | Client, documentation |
HBASE-20678 | NPE in ReplicationSourceManager#NodeFailoverWorker | Minor | . |
HBASE-20670 | NPE in HMaster#isInMaintenanceMode | Minor | . |
HBASE-20634 | Reopen region while server crash can cause the procedure to be stuck | Critical | . |
HBASE-12882 | Log level for org.apache.hadoop.hbase package should be configurable | Major | . |
HBASE-20668 | Avoid permission change if ExportSnapshot's copy fails | Major | . |
HBASE-18116 | Replication source in-memory accounting should not include bulk transfer hfiles | Major | Replication |
HBASE-20602 | hbase.master.quota.observer.ignore property seems to be not taking effect | Minor | documentation |
HBASE-20664 | Variable shared across multiple threads | Major | . |
HBASE-20659 | Implement a reopen table regions procedure | Major | . |
HBASE-20582 | Bump up JRuby version because of some reported vulnerabilities | Major | dependencies, shell |
HBASE-20533 | Fix the flaky TestAssignmentManagerMetrics | Major | . |
HBASE-20597 | Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint | Minor | Replication |
HBASE-20633 | Dropping a table containing a disable violation policy fails to remove the quota upon table delete | Major | . |
HBASE-20645 | Fix security_available method in security.rb | Major | . |
HBASE-20612 | TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn | Major | . |
HBASE-20648 | HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch | Major | . |
HBASE-20588 | Space quota change after quota violation doesn't seem to take in effect | Major | regionserver |
HBASE-20616 | TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state | Major | amv2 |
HBASE-20638 | nightly source artifact testing should fail the stage if it's going to report an error on jira | Major | test |
HBASE-20624 | Race in ReplicationSource which causes walEntryFilter being null when creating new shipper | Major | Replication |
HBASE-20601 | Add multiPut support and other miscellaneous to PE | Minor | tooling |
HBASE-20627 | Relocate RS Group pre/post hooks from RSGroupAdminServer to RSGroupAdminEndpoint | Major | . |
HBASE-20591 | nightly job doesn't respect maven options | Critical | test |
HBASE-20560 | Revisit the TestReplicationDroppedTables ut | Major | . |
HBASE-20571 | JMXJsonServlet generates invalid JSON if it has NaN in metrics | Major | UI |
HBASE-20585 | Need to clear peer map when clearing MasterProcedureScheduler | Major | proc-v2 |
HBASE-20457 | Return immediately for a scan rpc call when we want to switch from pread to stream | Major | scan |
HBASE-20447 | Only fail cacheBlock if block collisions aren't related to next block metadata | Major | BlockCache, BucketCache |
HBASE-20544 | downstream HBaseTestingUtility fails with invalid port | Blocker | test |
HBASE-20004 | Client is not able to execute REST queries in a secure cluster | Minor | REST, security |
HBASE-20475 | Fix the flaky TestReplicationDroppedTables unit test. | Major | . |
HBASE-20554 | "WALs outstanding" message from CleanerChore is noisy | Trivial | . |
HBASE-20204 | Add locking to RefreshFileConnections in BucketCache | Major | BucketCache |
HBASE-20485 | Copy constructor of Scan doesn't copy the readType and replicaId | Minor | . |
HBASE-20543 | Fix the flaky TestThriftHttpServer | Major | . |
HBASE-20521 | TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail | Major | mapreduce |
HBASE-20500 | [rsgroup] should keep at least one server in default group | Major | rsgroup |
HBASE-20517 | Fix PerformanceEvaluation 'column' parameter | Major | test |
HBASE-20524 | Need to clear metrics when ReplicationSourceManager refresh replication sources | Minor | . |
HBASE-20476 | Open sequence number could go backwards in AssignProcedure | Major | Region Assignment |
HBASE-20506 | Add doc and test for unused RetryCounter, useful-looking utility | Minor | . |
HBASE-20492 | UnassignProcedure is stuck in retry loop on region stuck in OPENING state | Critical | amv2 |
HBASE-20497 | The getRecoveredQueueStartPos always return 0 in RecoveredReplicationSourceShipper | Major | Replication |
HBASE-18842 | The hbase shell clone_snaphost command returns bad error message | Minor | shell |
HBASE-20466 | Consistently use override mechanism for exempt classes in CoprocessClassloader | Major | Coprocessors |
HBASE-20006 | TestRestoreSnapshotFromClientWithRegionReplicas is flakey | Critical | read replicas |
HBASE-18059 | The scanner order for memstore scanners are wrong | Critical | regionserver, scan, Scanners |
HBASE-20404 | Ugly cleanerchore complaint that dir is not empty | Major | master |
HBASE-20419 | Fix potential NPE in ZKUtil#listChildrenAndWatchForNewChildren callers | Major | . |
HBASE-20364 | nightly job gives old results or no results for stages that timeout on SCM | Critical | test |
HBASE-20335 | nightly jobs no longer contain machine information | Critical | test |
HBASE-20338 | WALProcedureStore#recoverLease() should have fixed sleeps for retrying rollWriter() | Major | . |
HBASE-20356 | make skipping protoc possible | Critical | dependencies, thirdparty |
HBASE-15291 | FileSystem not closed in secure bulkLoad | Major | . |
HBASE-20068 | Hadoopcheck project health check uses default maven repo instead of yetus managed ones | Major | community, test |
HBASE-20361 | Non-successive TableInputSplits may wrongly be merged by auto balancing feature | Major | mapreduce |
HBASE-20260 | Purge old content from the book for branch-2/master | Critical | documentation |
HBASE-20058 | improper quoting in presplitting command docs | Minor | documentation |
HBASE-19923 | Reset peer state and config when refresh replication source failed | Major | Replication |
HBASE-19748 | TestRegionReplicaFailover and TestRegionReplicaReplicationEndpoint UT hangs | Major | . |
TESTS:
JIRA | Summary | Priority | Component |
---|---|---|---|
HBASE-20667 | Rename TestGlobalThrottler to TestReplicationGlobalThrottler | Trivial | . |
HBASE-20646 | TestWALProcedureStoreOnHDFS failing on branch-1 | Trivial | . |
HBASE-20505 | PE should support multi column family read and write cases | Minor | . |
HBASE-20513 | Collect and emit ScanMetrics in PerformanceEvaluation | Minor | test |
HBASE-20414 | TestLockProcedure#testMultipleLocks may fail on slow machine | Major | . |
SUB-TASKS:
JIRA | Summary | Priority | Component |
---|---|---|---|
HBASE-20831 | Copy master doc into branch-2.1 and edit to make it suit 2.1.0 | Blocker | documentation |
HBASE-20839 | Fallback to FSHLog if we can not instantiated AsyncFSWAL when user does not specify AsyncFSWAL explicitly | Blocker | wal |
HBASE-20244 | NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient | Blocker | wal |
HBASE-20193 | Basic Replication Web UI - Regionserver | Critical | Replication, Usability |
HBASE-20489 | Update Reference Guide that CLUSTER_KEY value is present on the Master UI info page. | Minor | documentation |
HBASE-19722 | Meta query statistics metrics source | Major | . |
HBASE-20781 | Save recalculating families in a WALEdit batch of Cells | Major | Performance |
HBASE-20194 | Basic Replication WebUI - Master | Critical | Replication, Usability |
HBASE-20780 | ServerRpcConnection logging cleanup | Major | logging, Performance |
HBASE-19764 | Fix Checkstyle errors in hbase-endpoint | Minor | . |
HBASE-20710 | extra cloneFamily() in Mutation.add(Cell) | Minor | regionserver |
HBASE-18569 | Add prefetch support for async region locator | Major | asyncclient, Client |
HBASE-20706 | [hack] Don't add known not-OPEN regions in reopen phase of MTP | Critical | amv2 |
HBASE-20334 | add a test that expressly uses both our shaded client and the one from hadoop 3 | Major | hadoop3, shading |
HBASE-20615 | emphasize use of shaded client jars when they're present in an install | Major | build, Client, Usability |
HBASE-20333 | break up shaded client into one with no Hadoop and one that's standalone | Critical | shading |
HBASE-20332 | shaded mapreduce module shouldn't include hadoop | Critical | mapreduce, shading |
HBASE-20722 | Make RegionServerTracker only depend on children changed event | Major | . |
HBASE-20700 | Move meta region when server crash can cause the procedure to be stuck | Critical | master, proc-v2, Region Assignment |
HBASE-20628 | SegmentScanner does over-comparing when one flushing | Critical | Performance |
HBASE-19761 | Fix Checkstyle errors in hbase-zookeeper | Minor | . |
HBASE-19724 | Fix Checkstyle errors in hbase-hadoop2-compat | Minor | . |
HBASE-20518 | Need to serialize the enabled field for UpdatePeerConfigProcedure | Major | Replication |
HBASE-20481 | Replicate entries from same region serially in ReplicationEndpoint for serial replication | Major | . |
HBASE-20378 | Provide a hbck option to cleanup replication barrier for a table | Major | . |
HBASE-20128 | Add new UTs which extends the old replication UTs but set replication scope to SERIAL | Major | . |
HBASE-20417 | Do not read wal entries when peer is disabled | Major | Replication |
HBASE-20294 | Also cleanup last pushed sequence id in ReplicationBarrierCleaner | Major | Replication |
HBASE-20377 | Deal with table in enabling and disabling state when modifying serial replication peer | Major | Replication |
HBASE-20367 | Write a replication barrier for regions when disabling a table | Major | Replication |
HBASE-20296 | Remove last pushed sequence ids when removing tables from a peer | Major | Replication |
HBASE-20285 | Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer | Major | Replication |
HBASE-20138 | Find a way to deal with the conflicts when updating replication position | Major | Replication |
HBASE-20127 | Add UT for serial replication after failover | Major | Replication, test |
HBASE-20271 | ReplicationSourceWALReader.switched should use the file name instead of the path object directly | Major | Replication |
HBASE-20227 | Add UT for ReplicationUtils.contains method | Major | Replication, test |
HBASE-20147 | Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves | Major | . |
HBASE-20116 | Optimize the region last pushed sequence id layout on zk | Major | Replication |
HBASE-20242 | The open sequence number will grow if we fail to open a region after writing the max sequence id file | Major | . |
HBASE-20155 | update branch-2 version to 2.1.0-SNAPSHOT | Major | build, community |
HBASE-20206 | WALEntryStream should not switch WAL file silently | Major | Replication |
HBASE-20117 | Cleanup the unused replication barriers in meta table | Major | master, Replication |
HBASE-20165 | Shell command to make a normal peer to be a serial replication peer | Major | . |
HBASE-20167 | Optimize the implementation of ReplicationSourceWALReader | Major | Replication |
HBASE-20148 | Make serial replication as a option for a peer instead of a table | Major | Replication |
HBASE-20125 | Add UT for serial replication after region split and merge | Major | Replication |
HBASE-20129 | Add UT for serial replication checker | Major | Replication |
HBASE-20115 | Reimplement serial replication based on the new replication storage layer | Major | Replication |
HBASE-20050 | Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer | Major | . |
HBASE-20082 | Fix findbugs errors only on master which are introduced by HBASE-19397 | Major | findbugs |
HBASE-19936 | Introduce a new base class for replication peer procedure | Major | . |
HBASE-19719 | Fix checkstyle issues | Major | proc-v2, Replication |
HBASE-19711 | TestReplicationAdmin.testConcurrentPeerOperations hangs | Major | proc-v2 |
HBASE-19707 | Race in start and terminate of a replication source after we async start replicatione endpoint | Major | proc-v2, Replication |
HBASE-19636 | All rs should already start work with the new peer change when replication peer procedure is finished | Major | proc-v2, Replication |
HBASE-19634 | Add permission check for executeProcedures in AccessController | Major | proc-v2, Replication |
HBASE-19697 | Remove TestReplicationAdminUsingProcedure | Major | proc-v2, Replication |
HBASE-19661 | Replace ReplicationStateZKBase with ZKReplicationStorageBase | Major | proc-v2, Replication |
HBASE-19687 | Move the logic in ReplicationZKNodeCleaner to ReplicationChecker and remove ReplicationZKNodeCleanerChore | Major | proc-v2, Replication |
HBASE-19544 | Add UTs for testing concurrent modifications on replication peer | Major | proc-v2, Replication, test |
HBASE-19686 | Use KeyLocker instead of ReentrantLock in PeerProcedureHandlerImpl | Major | proc-v2, Replication |
HBASE-19623 | Create replication endpoint asynchronously when adding a replication source | Major | proc-v2, Replication |
HBASE-19633 | Clean up the replication queues in the postPeerModification stage when removing a peer | Major | proc-v2, Replication |
HBASE-19622 | Reimplement ReplicationPeers with the new replication storage interface | Major | proc-v2, Replication |
HBASE-19635 | Introduce a thread at RS side to call reportProcedureDone | Major | proc-v2 |
HBASE-19617 | Remove ReplicationQueues, use ReplicationQueueStorage directly | Major | Replication |
HBASE-19642 | Fix locking for peer modification procedure | Critical | proc-v2, Replication |
HBASE-19592 | Add UTs to test retry on update zk failure | Major | proc-v2, Replication |
HBASE-19630 | Add peer cluster key check when add new replication peer | Major | proc-v2, Replication |
HBASE-19573 | Rewrite ReplicationPeer with the new replication storage interface | Major | proc-v2, Replication |
HBASE-19579 | Add peer lock test for shell command list_locks | Major | proc-v2, Replication |
HBASE-19599 | Remove ReplicationQueuesClient, use ReplicationQueueStorage directly | Major | Replication |
HBASE-19543 | Abstract a replication storage interface to extract the zk specific code | Major | proc-v2, Replication |
HBASE-19525 | RS side changes for moving peer modification from zk watcher to procedure | Major | proc-v2, Replication |
HBASE-19580 | Use slf4j instead of commons-logging in new, just-added Peer Procedure classes | Major | proc-v2, Replication |
HBASE-19520 | Add UTs for the new lock type PEER | Major | proc-v2 |
HBASE-19564 | Procedure id is missing in the response of peer related operations | Major | proc-v2, Replication |
HBASE-19536 | Client side changes for moving peer modification from zk watcher to procedure | Major | Replication |
HBASE-19524 | Master side changes for moving peer modification from zk watcher to procedure | Major | proc-v2, Replication |
HBASE-19216 | Implement a general framework to execute remote procedure on RS | Major | proc-v2, Replication |
OTHER:
JIRA | Summary | Priority | Component |
---|---|---|---|
HBASE-19997 | [rolling upgrade] 1.x => 2.x | Blocker | . |
HBASE-20360 | Further optimization for serial replication | Major | Replication |
HBASE-20665 | "Already cached block XXX" message should be DEBUG | Minor | BlockCache |
HBASE-20677 | Backport test of HBASE-20566 'Creating a system table after enabling rsgroup feature puts region into RIT' to branch-2 | Major | . |
HBASE-19475 | Extend backporting strategy in documentation | Trivial | documentation |
HBASE-20595 | Remove the concept of 'special tables' from rsgroups | Major | Region Assignment, rsgroup |
HBASE-20415 | branches-2 don't need maven-scala-plugin | Major | build |
HBASE-20112 | Include test results from nightly hadoop3 tests in jenkins test results | Critical | test |
HBASE-17918 | document serial replication | Critical | documentation, Replication |
HBASE-19737 | Manage a HBASE-19397-branch-2 branch and merge it to branch-2 | Major | proc-v2, Replication |