70 KiB
70 KiB
HBASE Changelog
Release 2.4.3 - 2021-05-21
NEW FEATURES:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25751 | Add writable TimeToPurgeDeletes to ScanOptions | Major | . |
| HBASE-25587 | [hbck2] Schedule SCP for all unknown servers | Major | hbase-operator-tools, hbck2 |
IMPROVEMENTS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25860 | Add metric for successful wal roll requests. | Major | metrics, wal |
| HBASE-25754 | StripeCompactionPolicy should support compacting cold regions | Minor | Compaction |
| HBASE-25766 | Introduce RegionSplitRestriction that restricts the pattern of the split point | Major | . |
| HBASE-25798 | typo in MetricsAssertHelper | Minor | . |
| HBASE-25770 | Http InfoServers should honor gzip encoding when requested | Major | UI |
| HBASE-25776 | Use Class.asSubclass to fix the warning in StochasticLoadBalancer.loadCustomCostFunctions | Minor | Balancer |
| HBASE-25767 | CandidateGenerator.getRandomIterationOrder is too slow on large cluster | Major | Balancer, Performance |
| HBASE-25762 | Improvement for some debug-logging guards | Minor | logging, Performance |
| HBASE-25653 | Add units and round off region size to 2 digits after decimal | Major | master, Normalizer |
| HBASE-25759 | The master services field in LocalityBasedCostFunction is never used | Major | Balancer |
| HBASE-25747 | Remove unused getWriteAvailable method in OperationQuota | Minor | Quotas |
| HBASE-25558 | Adding audit log for execMasterService | Major | . |
| HBASE-25703 | Support conditional update in MultiRowMutationEndpoint | Major | Coprocessors |
| HBASE-25627 | HBase replication should have a metric to represent if the source is stuck getting initialized | Major | Replication |
| HBASE-25688 | Use CustomRequestLog instead of Slf4jRequestLog for jetty | Major | logging, UI |
| HBASE-25678 | Support nonce operations for Increment/Append in RowMutations and CheckAndMutate | Major | . |
| HBASE-25679 | Size of log queue metric is incorrect in branch-1/branch-2 | Major | . |
| HBASE-25518 | Support separate child regions to different region servers | Major | . |
| HBASE-25621 | Balancer should check region plan source to avoid misplace region groups | Major | Balancer |
| HBASE-25374 | Make REST Client connection and socket time out configurable | Minor | REST |
| HBASE-25597 | Add row info in Exception when cell size exceeds maxCellSize | Minor | . |
| HBASE-25660 | Print split policy in use on Region open (as well as split policy vitals) | Trivial | . |
| HBASE-25635 | CandidateGenerator may miss some region balance actions | Major | Balancer |
| HBASE-25636 | Expose HBCK report as metrics | Minor | metrics |
| HBASE-25548 | Optionally allow snapshots to preserve cluster's max filesize config by setting it into table descriptor | Major | . |
BUG FIXES:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25032 | Do not assign regions to region server which has not called regionServerReport yet | Major | . |
| HBASE-25875 | RegionServer failed to start due to IllegalThreadStateException in AuthenticationTokenSecretManager.start | Major | . |
| HBASE-25867 | Extra doc around ITBLL | Minor | documentation |
| HBASE-25859 | Reference class incorrectly parses the protobuf magic marker | Minor | regionserver |
| HBASE-25774 | ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations | Critical | Replication |
| HBASE-25825 | RSGroupBasedLoadBalancer.onConfigurationChange should chain the request to internal balancer | Major | Balancer |
| HBASE-25792 | Filter out o.a.hadoop.thirdparty building shaded jars | Major | shading |
| HBASE-25806 | Backport the region location finder initialization fix in HBASE-25802 | Major | Balancer |
| HBASE-25735 | Add target Region to connection exceptions | Major | rpc |
| HBASE-25717 | RegionServer aborted due to ClassCastException | Major | . |
| HBASE-25743 | Retry REQUESTTIMEOUT KeeperExceptions from ZK | Major | Zookeeper |
| HBASE-25726 | MoveCostFunction is not included in the list of cost functions for StochasticLoadBalancer | Major | Balancer |
| HBASE-25692 | Failure to instantiate WALCellCodec leaks socket in replication | Major | Replication |
| HBASE-25568 | Upgrade Thrift jar to fix CVE-2020-13949 | Critical | Thrift |
| HBASE-25562 | ReplicationSourceWALReader log and handle exception immediately without retrying | Major | Replication |
| HBASE-25693 | NPE getting metrics from standby masters (MetricsMasterWrapperImpl.getMergePlanCount) | Major | master |
| HBASE-25685 | asyncprofiler2.0 no longer supports svg; wants html | Major | . |
| HBASE-25594 | graceful_stop.sh fails to unload regions when ran at localhost | Minor | . |
| HBASE-25674 | RegionInfo.parseFrom(DataInputStream) sometimes fails to read the protobuf magic marker | Minor | Client |
| HBASE-25595 | TestLruBlockCache.testBackgroundEvictionThread is flaky | Major | . |
| HBASE-25662 | Fix spotbugs warning in RoundRobinTableInputFormat | Major | findbugs |
| HBASE-25657 | Fix spotbugs warnings after upgrading spotbugs to 4.x | Major | findbugs |
| HBASE-25646 | Possible Resource Leak in CatalogJanitor | Major | master |
TESTS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25691 | Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile | Major | test |
SUB-TASKS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25876 | Add retry if we fail to read all bytes of the protobuf magic marker | Trivial | io |
| HBASE-25790 | NamedQueue 'BalancerRejection' for recent history of balancer skipping | Major | Balancer, master |
| HBASE-25854 | Remove redundant AM in-memory state changes in CatalogJanitor | Major | . |
| HBASE-25847 | More DEBUG and TRACE level logging in CatalogJanitor and HbckChore | Minor | . |
| HBASE-25838 | Use double instead of Double in StochasticLoadBalancer | Major | Balancer, Performance |
| HBASE-25835 | Ignore duplicate split requests from regionserver reports | Major | . |
| HBASE-25836 | RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions | Major | . |
| HBASE-25840 | CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip | Minor | . |
| HBASE-25775 | Use a special balancer to deal with maintenance mode | Major | Balancer |
| HBASE-25199 | Remove HStore#getStoreHomedir | Minor | . |
| HBASE-25696 | Need to initialize SLF4JBridgeHandler in jul-to-slf4j for redirecting jul to slf4j | Major | logging |
| HBASE-25695 | Link to the filter on hbase:meta from user tables panel on master page | Major | UI |
| HBASE-25629 | Reimplement TestCurrentHourProvider to not depend on unstable TZs | Major | test |
| HBASE-25671 | Backport HBASE-25608 to branch-2 | Major | . |
| HBASE-25677 | Server+table counters on each scan #nextRaw invocation becomes a bottleneck when heavy load | Major | metrics |
| HBASE-25667 | Remove RSGroup test addition made in parent; depends on functionality not in old branches | Major | . |
OTHER:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25884 | NPE while getting Balancer decisions | Major | . |
| HBASE-25755 | Exclude tomcat-embed-core from libthrift | Critical | dependencies, Thrift |
| HBASE-25750 | Upgrade RpcControllerFactory and HBaseRpcController from Private to LimitedPrivate(COPROC,PHOENIX) | Major | Coprocessors, phoenix, rpc |
| HBASE-25734 | Backport HBASE-24305 to branch-2.4 | Minor | . |
| HBASE-25604 | Upgrade spotbugs to 4.x | Major | build, findbugs |
| HBASE-24305 | Handle deprecations in ServerName | Minor | . |
Release 2.4.2 - Unreleased (as of 2021-03-08)
NEW FEATURES:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25460 | Expose drainingServers as cluster metric | Major | metrics |
| HBASE-25496 | add get_namespace_rsgroup command | Major | . |
IMPROVEMENTS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-23578 | [UI] Master UI shows long stack traces when table is broken | Minor | master, UI |
| HBASE-25492 | Create table with rsgroup info in branch-2 | Major | rsgroup |
| HBASE-25539 | Add metric for age of oldest wal. | Major | metrics, regionserver |
| HBASE-25574 | Revisit put/delete/increment/append related RegionObserver methods | Major | Coprocessors |
| HBASE-25541 | In WALEntryStream, set the current path to null while dequeing the log | Major | . |
| HBASE-23887 | New L1 cache : AdaptiveLRU | Major | BlockCache, Performance |
| HBASE-25534 | Honor TableDescriptor settings earlier in normalization | Major | Normalizer |
| HBASE-25507 | Leak of ESTABLISHED sockets when compaction encountered "java.io.IOException: Invalid HFile block magic" | Major | Compaction |
| HBASE-25542 | Add client detail to scan name so when lease expires, we have clue on who was scanning | Major | scan |
| HBASE-25528 | Dedicated merge dispatch threadpool on master | Minor | master |
| HBASE-25536 | Remove 0 length wal file from logQueue if it belongs to old sources. | Major | Replication |
| HBASE-25482 | Improve SimpleRegionNormalizer#getAverageRegionSizeMb | Minor | Normalizer |
BUG FIXES:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25626 | Possible Resource Leak in HeterogeneousRegionCountCostFunction | Major | . |
| HBASE-25644 | Scan#setSmall blindly sets ReadType as PREAD | Critical | . |
| HBASE-25609 | There is a problem with the SPLITS_FILE in the HBase shell statement | Minor | . |
| HBASE-25385 | TestCurrentHourProvider fails if the latest timezone changes are not present | Blocker | . |
| HBASE-25596 | Fix NPE in ReplicationSourceManager as well as avoid permanently unreplicated data due to EOFException from WAL | Critical | . |
| HBASE-25367 | Sort broken after Change 'State time' in UI | Major | UI |
| HBASE-25421 | There is no limit on the column family length when creating a table | Major | Client |
| HBASE-25371 | When openRegion fails during initial verification(before initializing and setting seq num), exception is observed during region close. | Major | Region Assignment |
| HBASE-25611 | ExportSnapshot chmod flag uses value as decimal | Major | . |
| HBASE-25586 | Fix HBASE-22492 on branch-2 (SASL GapToken) | Major | rpc |
| HBASE-25598 | TestFromClientSide5.testScanMetrics is flaky | Major | . |
| HBASE-25556 | Frequent replication "Encountered a malformed edit" warnings | Minor | Operability, Replication |
| HBASE-25575 | Should validate Puts in RowMutations | Minor | Client |
| HBASE-25559 | Terminate threads of oldsources while RS is closing | Major | . |
| HBASE-25543 | When configuration "hadoop.security.authorization" is set to false, the system will still try to authorize an RPC and raise AccessDeniedException | Minor | IPC/RPC |
| HBASE-25554 | NPE when init RegionMover | Major | . |
| HBASE-25523 | Region normalizer chore thread is getting killed | Major | Normalizer |
| HBASE-25533 | The metadata of the table and family should not be an empty string | Major | . |
| HBASE-25478 | Implement retries when enabling tables in TestRegionReplicaReplicationEndpoint | Minor | . |
| HBASE-25513 | When the table is turned on normalize, the first region may not be merged even the size is 0 | Major | Normalizer |
| HBASE-25497 | move_namespaces_rsgroup should change hbase.rsgroup.name config in NamespaceDescriptor | Major | . |
SUB-TASKS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-24900 | Make retain assignment configurable during SCP | Major | amv2 |
| HBASE-25509 | ChoreService.cancelChore will not call ScheduledChore.cleanup which may lead to resource leak | Major | util |
OTHER:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25553 | It is better for ReplicationTracker.getListOfRegionServers to return ServerName instead of String | Major | . |
| HBASE-25620 | Increase timeout value for pre commit | Major | build, test |
| HBASE-25615 | Upgrade java version in pre commit docker file | Major | build |
| HBASE-25601 | Remove search hadoop references in book | Major | documentation |
Release 2.4.1 - 2021-01-18
NEW FEATURES:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-24620 | Add a ClusterManager which submits command to ZooKeeper and its Agent which picks and execute those Commands. | Major | integration tests |
IMPROVEMENTS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25329 | Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time | Minor | . |
| HBASE-25475 | Improve unit test for HBASE-25445 : SplitWALRemoteProcedure failed to archive split WAL | Minor | wal |
| HBASE-25249 | Adding StoreContext | Major | . |
| HBASE-25449 | 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml | Major | conf |
| HBASE-25476 | Enable error prone check in pre commit | Major | build |
| HBASE-25211 | Rack awareness in region_mover | Major | . |
| HBASE-25483 | set the loadMeta log level to debug. | Major | MTTR, Region Assignment |
| HBASE-25435 | Slow metric value can be configured | Minor | metrics |
| HBASE-25318 | Configure where IntegrationTestImportTsv generates HFiles | Minor | integration tests |
| HBASE-24850 | CellComparator perf improvement | Critical | Performance, scan |
| HBASE-25425 | Some notes on RawCell | Trivial | . |
| HBASE-25420 | Some minor improvements in rpc implementation | Minor | rpc |
| HBASE-25246 | Backup/Restore hbase cell tags. | Major | backup&restore |
| HBASE-25328 | Add builder method to create Tags. | Minor | . |
BUG FIXES:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25356 | HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName | Major | shell |
| HBASE-25279 | Non-daemon thread in ZKWatcher | Critical | Zookeeper |
| HBASE-25504 | [branch-2.4] Restore method removed by HBASE-25277 to LP(CONFIG) coprocessors | Major | compatibility, Coprocessors, security |
| HBASE-25503 | HBase code download is failing on windows with invalid path error | Major | . |
| HBASE-24813 | ReplicationSource should clear buffer usage on ReplicationSourceManager upon termination | Major | Replication |
| HBASE-25459 | WAL can't be cleaned in some scenes | Major | . |
| HBASE-25434 | SlowDelete & SlowPut metric value should use updateDelete & updatePut | Major | regionserver |
| HBASE-25441 | add security check for some APIs in RSRpcServices | Critical | . |
| HBASE-25432 | we should add security checks for setTableStateInMeta and fixMeta | Blocker | . |
| HBASE-25445 | Old WALs archive fails in procedure based WAL split | Critical | wal |
| HBASE-25287 | Forgetting to unbuffer streams results in many CLOSE_WAIT sockets when loading files | Major | . |
| HBASE-25447 | remoteProc is suspended due to OOM ERROR | Major | proc-v2 |
| HBASE-24755 | [LOG][RSGroup]Error message is confusing while adding a offline RS to rsgroup | Major | rsgroup |
| HBASE-25463 | Fix comment error | Minor | shell |
| HBASE-25457 | Possible race in AsyncConnectionImpl between getChoreService and close | Major | Client |
| HBASE-25456 | setRegionStateInMeta need security check | Critical | . |
| HBASE-25383 | HBase doesn't update and remove the peer config from hbase.replication.source.custom.walentryfilters if the config is already set on the peer. | Major | . |
| HBASE-25404 | Procedures table Id under master web UI gets word break to single character | Minor | UI |
| HBASE-25277 | postScannerFilterRow impacts Scan performance a lot in HBase 2.x | Critical | Coprocessors, scan |
| HBASE-25365 | The log in move_servers_rsgroup is incorrect | Minor | . |
| HBASE-25372 | Fix typo in ban-jersey section of the enforcer plugin in pom.xml | Major | build |
| HBASE-25361 | [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations | Major | flakies |
TESTS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25502 | IntegrationTestMTTR fails with TableNotFoundException | Major | integration tests |
| HBASE-25334 | TestRSGroupsFallback.testFallback is flaky | Major | . |
| HBASE-25370 | Fix flaky test TestClassFinder#testClassFinderDefaultsToOwnPackage | Major | test |
SUB-TASKS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25293 | Followup jira to address the client handling issue when chaning from meta replica to non-meta-replica at the server side. | Minor | . |
| HBASE-25353 | [Flakey Tests] branch-2 TestShutdownBackupMaster | Major | flakies |
OTHER:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25333 | Add maven enforcer rule to ban VisibleForTesting imports | Major | build, pom |
| HBASE-25452 | Use MatcherAssert.assertThat instead of org.junit.Assert.assertThat | Major | test |
| HBASE-25400 | [Flakey Tests] branch-2 TestRegionMoveAndAbandon | Major | . |
| HBASE-25389 | [Flakey Tests] branch-2 TestMetaShutdownHandler | Major | flakies |
Release 2.4.0 - Unreleased (as of 2020-12-03)
NEW FEATURES:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25242 | Add Increment/Append support to RowMutations | Critical | Client, regionserver |
| HBASE-25278 | Add option to toggle CACHE_BLOCKS in count.rb | Minor | shell |
| HBASE-18070 | Enable memstore replication for meta replica | Critical | . |
| HBASE-24528 | Improve balancer decision observability | Major | Admin, Balancer, Operability, shell, UI |
| HBASE-24776 | [hbtop] Support Batch mode | Major | hbtop |
| HBASE-24602 | Add Increment and Append support to CheckAndMutate | Major | . |
| HBASE-24760 | Add a config hbase.rsgroup.fallback.enable for RSGroup fallback feature | Major | rsgroup |
| HBASE-24694 | Support flush a single column family of table | Major | . |
| HBASE-24289 | Heterogeneous Storage for Date Tiered Compaction | Major | Compaction |
| HBASE-24038 | Add a metric to show the locality of ssd in table.jsp | Major | metrics |
| HBASE-8458 | Support for batch version of checkAndMutate() | Major | Client, regionserver |
IMPROVEMENTS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25339 | Method parameter and member variable are duplicated in checkSplittable() of SplitTableRegionProcedure | Minor | . |
| HBASE-25237 | 'hbase master stop' shuts down the cluster, not the master only | Major | . |
| HBASE-25324 | Remove unnecessary array to list conversion in SplitLogManager | Minor | . |
| HBASE-25263 | Change encryption key generation algorithm used in the HBase shell | Major | encryption, shell |
| HBASE-25323 | Fix potential NPE when the zookeeper path of RegionServerTracker does not exist when start | Minor | . |
| HBASE-25281 | Bulkload split hfile too many times due to unreasonable split point | Minor | tooling |
| HBASE-25325 | Remove unused class ClusterSchemaException | Minor | . |
| HBASE-25213 | Should request Compaction when bulkLoadHFiles is done | Minor | . |
| HBASE-24877 | Add option to avoid aborting RS process upon uncaught exceptions happen on replication source | Major | Replication |
| HBASE-24664 | Some changing of split region by overall region size rather than only one store size | Major | regionserver |
| HBASE-25026 | Create a metric to track full region scans RPCs | Minor | . |
| HBASE-25289 | [testing] Clean up resources after tests in rsgroup_shell_test.rb | Major | rsgroup, test |
| HBASE-25261 | Upgrade Bootstrap to 3.4.1 | Major | security, UI |
| HBASE-25272 | Support scan on a specific replica | Critical | Client, scan |
| HBASE-25267 | Add SSL keystore type and truststore related configs for HBase RESTServer | Major | REST |
| HBASE-25181 | Add options for disabling column family encryption and choosing hash algorithm for wrapped encryption keys. | Major | encryption |
| HBASE-25254 | Rewrite TestMultiLogThreshold to remove the LogDelegate in RSRpcServices | Major | logging, test |
| HBASE-25252 | Move HMaster inner classes out | Minor | master |
| HBASE-25240 | gson format of RpcServer.logResponse is abnormal | Minor | . |
| HBASE-25210 | RegionInfo.isOffline is now a duplication with RegionInfo.isSplit | Major | meta |
| HBASE-25212 | Optionally abort requests in progress after deciding a region should close | Major | regionserver |
| HBASE-24859 | Optimize in-memory representation of mapreduce TableSplit objects | Major | mapreduce |
| HBASE-24967 | The table.jsp cost long time to load if the table include closed regions | Major | UI |
| HBASE-25167 | Normalizer support for hot config reloading | Major | master, Normalizer |
| HBASE-24419 | Normalizer merge plans should consider more than 2 regions when possible | Major | master, Normalizer |
| HBASE-25224 | Maximize sleep for checking meta and namespace regions availability | Major | master |
| HBASE-25223 | Use try-with-resources statement in snapshot package | Minor | . |
| HBASE-25201 | YouAreDeadException should be moved to hbase-server module | Major | Client |
| HBASE-25193 | Add support for row prefix and type in the WAL Pretty Printer and some minor fixes | Minor | wal |
| HBASE-25128 | RSGroupInfo's toString() and hashCode() does not take into account configuration map. | Minor | rsgroup |
| HBASE-24628 | Region normalizer now respects a rate limit | Major | Normalizer |
| HBASE-25179 | Assert format is incorrect in HFilePerformanceEvaluation class. | Minor | Performance, test |
| HBASE-25065 | WAL archival to be done by a separate thread | Major | wal |
| HBASE-14067 | bundle ruby files for hbase shell into a jar. | Major | shell |
| HBASE-24875 | Remove the force param for unassign since it dose not take effect any more | Major | Client |
| HBASE-24025 | Improve performance of move_servers_rsgroup and move_tables_rsgroup by using async region move API | Major | rsgroup |
| HBASE-25160 | Refactor AccessController and VisibilityController | Major | . |
| HBASE-25146 | Add extra logging at info level to HFileCorruptionChecker in order to report progress | Major | hbck, hbck2 |
| HBASE-24054 | The Jetty's version number leak occurred while using the thrift service | Minor | . |
| HBASE-25091 | Move LogComparator from ReplicationSource to AbstractFSWALProvider#.WALsStartTimeComparator | Minor | . |
| HBASE-24981 | Enable table replication fails from 1.x to 2.x if table already exist at peer. | Major | Replication |
| HBASE-25109 | Add MR Counters to WALPlayer; currently hard to tell if it is doing anything | Major | . |
| HBASE-25082 | Per table WAL metrics: appendCount and appendSize | Major | metrics |
| HBASE-25079 | Upgrade Bootstrap to 3.3.7 | Major | security, UI |
| HBASE-24976 | REST Server failes to start without any error message | Major | REST |
| HBASE-25066 | Use FutureUtils.rethrow in AsyncTableResultScanner to better catch the stack trace | Major | Client, Scanners |
| HBASE-25069 | Display region name instead of encoded region name in HBCK report page. | Minor | hbck |
| HBASE-24991 | Replace MovedRegionsCleaner with guava cache | Minor | . |
| HBASE-25057 | Fix typo "memeber" | Trivial | documentation |
| HBASE-24764 | Add support of adding base peer configs via hbase-site.xml for all replication peers. | Minor | Replication |
| HBASE-25037 | Lots of thread pool are changed to non daemon after HBASE-24750 which causes trouble when shutting down | Major | . |
| HBASE-24831 | Avoid invoke Counter using reflection in SnapshotInputFormat | Major | . |
| HBASE-25002 | Create simple pattern matching query for retrieving metrics matching the pattern | Major | . |
| HBASE-25022 | Remove 'hbase.testing.nocluster' config | Major | test |
| HBASE-25006 | Make the cost functions optional for StochastoicBalancer | Major | . |
| HBASE-24974 | Provide a flexibility to print only row key and filter for multiple tables in the WALPrettyPrinter | Minor | wal |
| HBASE-24994 | Add hedgedReadOpsInCurThread metric | Minor | metrics |
| HBASE-25005 | Refactor CatalogJanitor | Major | master, meta |
| HBASE-24992 | log after Generator success when running ITBLL | Trivial | . |
| HBASE-24937 | table.rb use LocalDateTime to replace Instant | Minor | shell |
| HBASE-24940 | runCatalogJanitor() API should return -1 to indicate already running status | Major | . |
| HBASE-24973 | Remove read point parameter in method StoreFlush#performFlush and StoreFlush#createScanner | Minor | . |
| HBASE-24569 | Get hostAndWeights in addition using localhost if it is null in local mode | Minor | regionserver |
| HBASE-24949 | Optimize FSTableDescriptors.get to not always go to fs when cache miss | Major | master |
| HBASE-24898 | Use EnvironmentEdge.currentTime() instead of System.currentTimeMillis() in CurrentHourProvider | Major | tooling |
| HBASE-24942 | MergeTableRegionsProcedure should not call clean merge region | Major | proc-v2, Region Assignment |
| HBASE-24811 | Use class access static field or method | Minor | . |
| HBASE-24686 | [LOG] Log improvement in Connection#close | Major | Client, logging |
| HBASE-24912 | Enlarge MemstoreFlusherChore/CompactionChecker period for unit test | Major | . |
| HBASE-24627 | Normalize one table at a time | Major | Normalizer |
| HBASE-24872 | refactor valueOf PoolType | Minor | Client |
| HBASE-24854 | Correct the help content of assign and unassign commands in hbase shell | Minor | shell |
| HBASE-24750 | All executor service should start using guava ThreadFactory | Major | . |
| HBASE-24709 | Support MoveCostFunction use a lower multiplier in offpeak hours | Major | Balancer |
| HBASE-24824 | Add more stats in PE for read replica | Minor | PE, read replicas |
| HBASE-21721 | FSHLog : reduce write#syncs() times | Major | . |
| HBASE-24404 | Support flush a single column family of region | Major | shell |
| HBASE-24826 | Add some comments for processlist in hbase shell | Minor | shell |
| HBASE-24659 | Calculate FIXED_OVERHEAD automatically | Major | . |
| HBASE-24827 | BackPort HBASE-11554 Remove Reusable poolmap Rpc client type. | Major | Client |
| HBASE-24823 | Port HBASE-22762 Print the delta between phases in the split/merge/compact/flush transaction journals to master branch | Minor | . |
| HBASE-24795 | RegionMover should deal with unknown (split/merged) regions | Major | . |
| HBASE-24821 | Simplify the logic of getRegionInfo in TestFlushFromClient to reduce redundancy code | Minor | test |
| HBASE-24704 | Make the Table Schema easier to view even there are multiple families | Major | UI |
| HBASE-24695 | FSHLog - close the current WAL file in a background thread | Major | . |
| HBASE-24803 | Unify hbase-shell ::Shell::Commands::Command#help behavior | Minor | shell |
| HBASE-11686 | Shell code should create a binding / irb workspace instead of polluting the root namespace | Minor | shell |
| HBASE-20226 | Performance Improvement Taking Large Snapshots In Remote Filesystems | Minor | snapshots |
| HBASE-24669 | Logging of ppid should be consistent across all occurrences | Minor | Operability, proc-v2 |
| HBASE-24757 | ReplicationSink should limit the batch rowcount for batch mutations based on hbase.rpc.rows.warning.threshold | Major | . |
| HBASE-24777 | InfoServer support ipv6 host and port | Minor | UI |
| HBASE-24758 | Avoid flooding replication source RSes logs when no sinks are available | Major | Replication |
| HBASE-24743 | Reject to add a peer which replicate to itself earlier | Major | . |
| HBASE-24696 | Include JVM information on Web UI under "Software Attributes" | Minor | UI |
| HBASE-24747 | Log an ERROR if HBaseSaslRpcServer initialisation fails with an uncaught exception | Major | . |
| HBASE-24586 | Add table level locality in table.jsp | Major | UI |
| HBASE-24663 | Add procedure process time statistics UI | Major | . |
| HBASE-24653 | Show snapshot owner on Master WebUI | Major | . |
| HBASE-24431 | RSGroupInfo add configuration map to store something extra | Major | rsgroup |
| HBASE-24671 | Add excludefile and designatedfile options to graceful_stop.sh | Major | . |
| HBASE-24560 | Add a new option of designatedfile in RegionMover | Major | . |
| HBASE-24382 | Flush partial stores of region filtered by seqId when archive wal due to too many wals | Major | wal |
| HBASE-24208 | Remove RS entry from zk draining servers node after RS been stopped | Major | . |
| HBASE-24456 | Immutable Scan as unmodifiable subclass or wrapper of Scan | Major | . |
| HBASE-24471 | The way we bootstrap meta table is confusing | Major | master, meta, proc-v2 |
| HBASE-24350 | HBase table level replication metrics for shippedBytes are always 0 | Major | Replication |
| HBASE-24311 | Add more details in MultiVersionConcurrencyControl STUCK log message | Major | . |
BUG FIXES:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25355 | [Documentation] Fix spelling error | Trivial | documentation |
| HBASE-25230 | Embedded zookeeper server not clean up the old data | Minor | Zookeeper |
| HBASE-25349 | [Flakey Tests] branch-2 TestRefreshRecoveredReplication.testReplicationRefreshSource:141 Waiting timed out after [60,000] msec | Major | flakies |
| HBASE-25332 | one NPE | Major | Zookeeper |
| HBASE-25345 | [Flakey Tests] branch-2 TestReadReplicas#testVerifySecondaryAbilityToReadWithOnFiles | Major | test |
| HBASE-25307 | ThreadLocal pooling leads to NullPointerException | Major | Client |
| HBASE-25341 | Fix ErrorProne error which causes nightly to fail | Major | test |
| HBASE-25330 | RSGroupInfoManagerImpl#moveServers return is not set of servers moved | Major | rsgroup |
| HBASE-25321 | The sort icons not shown after Upgrade JQuery to 3.5.1 | Major | UI |
| HBASE-24268 | REST and Thrift server do not handle the "doAs" parameter case insensitively | Minor | REST, Thrift |
| HBASE-25050 | We initialize Filesystems more than once. | Minor | . |
| HBASE-25311 | ui throws NPE | Major | . |
| HBASE-25306 | The log in SimpleLoadBalancer#onConfigurationChange is wrong | Major | . |
| HBASE-25300 | 'Unknown table hbase:quota' happens when desc table in shell if quota disabled | Major | shell |
| HBASE-25255 | Master fails to initialize when creating rs group table | Critical | master, rsgroup |
| HBASE-25275 | Upgrade asciidoctor | Blocker | website |
| HBASE-25276 | Need to throw the original exception in HRegion#openHRegion | Major | . |
| HBASE-20598 | Upgrade to JRuby 9.2 | Major | dependencies, shell |
| HBASE-25216 | The client zk syncer should deal with meta replica count change | Major | master, Zookeeper |
| HBASE-25238 | Upgrading HBase from 2.2.0 to 2.3.x fails because of “Message missing required fields: state” | Critical | . |
| HBASE-25234 | [Upgrade]Incompatibility in reading RS report from 2.1 RS when Master is upgraded to a version containing HBASE-21406 | Major | . |
| HBASE-25053 | WAL replay should ignore 0-length files | Major | master, regionserver |
| HBASE-25090 | CompactionConfiguration logs unrealistic store file sizes | Minor | Compaction |
| HBASE-24977 | Meta table shouldn't be modified as read only | Major | meta |
| HBASE-25176 | MasterStoppedException should be moved to hbase-client module | Major | Client |
| HBASE-25206 | Data loss can happen if a cloned table loses original split region(delete table) | Major | proc-v2, Region Assignment, snapshots |
| HBASE-25207 | Revisit the implementation and usage of RegionStates.include | Major | Region Assignment |
| HBASE-25186 | TestMasterRegionOnTwoFileSystems is failing after HBASE-25065 | Blocker | master |
| HBASE-25204 | Nightly job failed as the name of jdk and maven changed | Major | . |
| HBASE-25093 | the RSGroupBasedLoadBalancer#retainAssignment throws NPE | Major | rsgroup |
| HBASE-25117 | ReplicationSourceShipper thread can not be finished | Major | . |
| HBASE-25168 | Unify WAL name timestamp parsers | Major | . |
| HBASE-23834 | HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch | Major | dependencies |
| HBASE-25165 | Change 'State time' in UI so sorts | Minor | UI |
| HBASE-25048 | [HBCK2] Bypassed parent procedures are not updated in store | Major | hbck2, proc-v2 |
| HBASE-25147 | Should store the regionNames field in state data for ReopenTableRegionsProcedure | Major | proc-v2 |
| HBASE-25115 | HFilePrettyPrinter can't seek to the row which is the first row of a hfile | Major | HFile, tooling |
| HBASE-25135 | Convert the internal seperator while emitting the memstore read metrics to # | Minor | . |
| HBASE-24665 | MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll | Major | wal |
| HBASE-25096 | WAL size in RegionServer UI is wrong | Major | . |
| HBASE-25077 | hbck.jsp page loading fails, logs NPE in master log. | Major | hbck |
| HBASE-25088 | CatalogFamilyFormat/MetaTableAccessor.parseRegionInfoFromRegionName incorrectly setEndKey to regionId | Critical | meta |
| HBASE-25097 | Wrong RIT page number in Master UI | Minor | UI |
| HBASE-24896 | 'Stuck' in static initialization creating RegionInfo instance | Major | . |
| HBASE-24956 | ConnectionManager#locateRegionInMeta waits for user region lock indefinitely. | Major | Client |
| HBASE-24481 | HBase Rest: Request for region detail of a table which doesn't exits is success(200 success code) instead of 404 | Minor | . |
| HBASE-25047 | WAL split edits number is negative in RegionServerUI | Minor | UI, wal |
| HBASE-25021 | Nightly job should skip hadoop-2 integration test for master | Major | build, scripts |
| HBASE-25012 | HBASE-24359 causes replication missed log of some RemoteException | Major | Replication |
| HBASE-25009 | Hbck chore logs wrong message when loading regions from RS report | Minor | . |
| HBASE-25014 | ScheduledChore is never triggered when initalDelay > 1.5*period | Major | . |
| HBASE-25016 | Should close ResultScanner in MetaTableAccessor.scanByRegionEncodedName | Critical | master, meta |
| HBASE-24958 | CompactingMemStore.timeOfOldestEdit error update | Critical | regionserver |
| HBASE-24995 | MetaFixer fails to fix overlaps when multiple tables have overlaps | Major | hbck2 |
| HBASE-24719 | Renaming invalid rsgroup throws NPE instead of proper error message | Major | . |
| HBASE-19352 | Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags | Major | . |
| HBASE-24971 | Upgrade JQuery to 3.5.1 | Major | security, UI |
| HBASE-24968 | One of static initializers of CellComparatorImpl referring to subclass MetaCellComparator | Major | . |
| HBASE-24916 | Region hole contains wrong regions pair when hole is created by first region deletion | Major | hbck2 |
| HBASE-24885 | STUCK RIT by hbck2 assigns | Major | hbck2, Region Assignment |
| HBASE-24926 | Should call setFailure in MergeTableRegionsProcedure when isMergeable returns false | Major | master, proc-v2 |
| HBASE-24884 | BulkLoadHFilesTool/LoadIncrementalHFiles should accept -D options from command line parameters | Minor | . |
| HBASE-24583 | Normalizer can't actually merge empty regions when neighbor is larger than average size | Major | master, Normalizer |
| HBASE-24844 | Exception on standalone (master) shutdown | Minor | Zookeeper |
| HBASE-24856 | Fix error prone error in FlushTableSubprocedure | Major | . |
| HBASE-24838 | The pre commit job fails to archive surefire reports | Critical | build, scripts |
| HBASE-23157 | WAL unflushed seqId tracking may wrong when Durability.ASYNC_WAL is used | Major | regionserver, wal |
| HBASE-24625 | AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. | Critical | Replication, wal |
| HBASE-24830 | Some tests involving RS crash fail with NullPointerException after HBASE-24632 in branch-2 | Major | . |
| HBASE-24788 | Fix the connection leaks on getting hbase admin from unclosed connection | Major | mapreduce |
| HBASE-24805 | HBaseTestingUtility.getConnection should be threadsafe | Major | test |
| HBASE-24808 | skip empty log cleaner delegate class names (WAS => cleaner.CleanerChore: Can NOT create CleanerDelegate= ClassNotFoundException) | Trivial | . |
| HBASE-24767 | Change default to false for HBASE-15519 per-user metrics | Major | metrics |
| HBASE-24713 | RS startup with FSHLog throws NPE after HBASE-21751 | Minor | wal |
| HBASE-24794 | hbase.rowlock.wait.duration should not be <= 0 | Minor | regionserver |
| HBASE-24797 | Move log code out of loop | Minor | Normalizer |
| HBASE-24752 | NPE/500 accessing webui on master startup | Minor | master |
| HBASE-24766 | Document Remote Procedure Execution | Major | documentation |
| HBASE-11676 | Scan FORMATTER is not applied for columns using non-printable name in shell | Minor | shell |
| HBASE-24738 | [Shell] processlist command fails with ERROR: Unexpected end of file from server when SSL enabled | Major | shell |
| HBASE-24675 | On Master restart all servers are assigned to default rsgroup. | Major | rsgroup |
| HBASE-22146 | SpaceQuotaViolationPolicy Disable is not working in Namespace level | Major | . |
| HBASE-24742 | Improve performance of SKIP vs SEEK logic | Major | Performance, regionserver |
| HBASE-24710 | Incorrect checksum calculation in saveVersion.sh | Major | scripts |
| HBASE-24714 | Error message is displayed in the UI of table's compaction state if any region of that table is not open. | Major | Compaction, UI |
| HBASE-24748 | Add hbase.master.balancer.stochastic.moveCost.offpeak to doc as support dynamically change | Minor | documentation |
| HBASE-24746 | The sort icons overlap the col name in master UI | Major | UI |
| HBASE-24721 | rename_rsgroup overwriting the existing rsgroup. | Major | . |
| HBASE-24615 | MutableRangeHistogram#updateSnapshotRangeMetrics doesn't calculate the distribution for last bucket. | Major | metrics |
| HBASE-24705 | MetaFixer#fixHoles() does not include the case for read replicas (i.e, replica regions are not created) | Major | read replicas |
| HBASE-24720 | Meta replicas not cleaned when disabled | Minor | read replicas |
| HBASE-24693 | regioninfo#isLast() has a logic error | Minor | . |
| HBASE-23744 | FastPathBalancedQueueRpcExecutor should enforce queue length of 0 | Minor | . |
| HBASE-22738 | Fallback to default group to choose RS when there are no RS in current group | Major | rsgroup |
| HBASE-23126 | IntegrationTestRSGroup is useless now | Major | rsgroup |
| HBASE-24518 | waitForNamespaceOnline() should return false if any region is offline | Major | . |
| HBASE-24564 | Make RS abort call idempotent | Major | regionserver |
| HBASE-24340 | PerformanceEvaluation options should not mandate any specific order | Minor | . |
| HBASE-24130 | rat plugin complains about having an unlicensed file. | Minor | . |
| HBASE-24017 | Turn down flakey rerun rate on all but hot branches | Major | . |
TESTS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-24015 | Coverage for Assign and Unassign of Regions on RegionServer on failure | Major | amv2 |
| HBASE-25156 | TestMasterFailover.testSimpleMasterFailover is flaky | Major | test |
| HBASE-24979 | Include batch mutatations in client operation timeout tests | Major | . |
| HBASE-24894 | [Flakey Test] TestStochasticLoadBalancer.testMoveCostMultiplier | Major | Balancer, master, test |
SUB-TASKS:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25127 | Enhance PerformanceEvaluation to profile meta replica performance. | Major | . |
| HBASE-25284 | Check-in "Enable memstore replication..." design | Major | . |
| HBASE-25126 | Add load balance logic in hbase-client to distribute read load over meta replica regions. | Major | . |
| HBASE-25291 | Document how to enable the meta replica load balance mode for the client and clean up around hbase:meta read replicas | Major | . |
| HBASE-25253 | Deprecated master carrys regions related methods and configs | Major | Balancer, master |
| HBASE-25203 | Change the reference url to flaky list in our jenkins jobs | Major | flakies, jenkins |
| HBASE-25194 | Do not publish workspace in flaky find job | Major | jenkins |
| HBASE-25169 | Update documentation about meta region replica | Major | documentation |
| HBASE-25164 | Make ModifyTableProcedure support changing meta replica count | Major | meta, read replicas |
| HBASE-25162 | Make flaky tests run more aggressively | Major | jenkins, scripts, test |
| HBASE-25163 | Increase the timeout value for nightly jobs | Major | jenkins, scripts, test |
| HBASE-22976 | [HBCK2] Add RecoveredEditsPlayer | Major | hbck2, walplayer |
| HBASE-25124 | Support changing region replica count without disabling table | Major | meta, proc-v2 |
| HBASE-23959 | Fix javadoc for JDK11 | Major | . |
| HBASE-25151 | warmupRegion frustrates registering WALs on the catalog replicationsource | Major | read replicas |
| HBASE-25154 | Set java.io.tmpdir to project build directory to avoid writing std*deferred files to /tmp | Major | build, test |
| HBASE-25121 | Refactor MetaTableAccessor.addRegionsToMeta and its usage places | Major | meta |
| HBASE-25055 | Add ReplicationSource for meta WALs; add enable/disable when hbase:meta assigned to RS | Major | . |
| HBASE-25133 | Migrate HBase Nightly jenkins job from Hadoop to hbase | Major | jenkins, scripts |
| HBASE-25132 | Migrate flaky test jenkins job from Hadoop to hbase | Major | jenkins, scripts |
| HBASE-25103 | Remove ZNodePaths.metaReplicaZNodes | Major | . |
| HBASE-25107 | Migrate flaky reporting jenkins job from Hadoop to hbase | Major | jenkins, scripts |
| HBASE-25068 | Pass WALFactory to Replication so it knows of all WALProviders, not just default/user-space | Minor | . |
| HBASE-25067 | Edit of log messages around async WAL Replication; checkstyle fixes; and a bugfix | Major | . |
| HBASE-24857 | Fix several problems when starting webUI | Minor | canary, UI |
| HBASE-24964 | Remove MetaTableAccessor.tableExists | Major | meta |
| HBASE-24765 | Dynamic master discovery | Major | Client |
| HBASE-24945 | Remove MetaTableAccessor.getRegionCount | Major | mapreduce, meta |
| HBASE-24944 | Remove MetaTableAccessor.getTableRegionsAndLocations in hbase-rest module | Major | meta, REST |
| HBASE-24918 | Make RegionInfo#UNDEFINED IA.Private | Major | . |
| HBASE-24806 | Small Updates to Functionality of Shell IRB Workspace | Major | shell |
| HBASE-24876 | Fix the flaky job url in hbase-personality.sh | Major | . |
| HBASE-24841 | Change the jenkins job urls in our jenkinsfile | Major | build, scripts |
| HBASE-24680 | Refactor the checkAndMutate code on the server side | Major | . |
| HBASE-24817 | Allow configuring WALEntry filters on ReplicationSource | Major | Replication, wal |
| HBASE-24632 | Enable procedure-based log splitting as default in hbase3 | Major | wal |
| HBASE-24718 | Generic NamedQueue framework for recent in-memory history (refactor slowlog) | Major | . |
| HBASE-24698 | Turn OFF Canary WebUI as default | Major | canary |
| HBASE-24650 | Change the return types of the new checkAndMutate methods introduced in HBASE-8458 | Major | Client |
| HBASE-24013 | Bump branch-2 version to 2.4.0-SNAPSHOT | Major | . |
OTHER:
| JIRA | Summary | Priority | Component |
|---|---|---|---|
| HBASE-25099 | Change meta replica count by altering meta table descriptor | Major | meta, read replicas |
| HBASE-25320 | Upgrade hbase-thirdparty dependency to 3.4.1 | Blocker | dependencies |
| HBASE-24640 | Purge use of VisibleForTesting | Major | community |
| HBASE-24081 | Provide documentation for running Yetus with HBase | Major | documentation |
| HBASE-24667 | Rename configs that support atypical DNS set ups to put them in hbase.unsafe | Major | conf, Operability |
| HBASE-25228 | Delete dev-support/jenkins_precommit_jira_yetus.sh | Minor | build |
| HBASE-24200 | Upgrade to Yetus 0.12.0 | Minor | build |
| HBASE-25120 | Remove the deprecated annotation for MetaTableAccessor.getScanForTableName | Major | meta |
| HBASE-25073 | Should not use XXXService.Interface.class.getSimpleName as stub key prefix in AsyncConnectionImpl | Major | Client |
| HBASE-25072 | Remove the unnecessary System.out.println in MasterRegistry | Minor | Client |
| HBASE-25004 | Log RegionTooBusyException details | Major | . |
| HBASE-24993 | Remove OfflineMetaRebuildTestCore | Major | test |
| HBASE-24809 | Yetus IA Javadoc links are no longer available | Minor | . |
| HBASE-14847 | Add FIFO compaction section to HBase book | Major | documentation |
| HBASE-24843 | Sort the constants in `hbase_constants.rb` | Minor | shell |
| HBASE-24835 | Normalizer should log a successful run at INFO level | Minor | Normalizer |
| HBASE-24779 | Improve insight into replication WAL readers hung on checkQuota | Minor | Replication |
| HBASE-24662 | Update DumpClusterStatusAction to notice changes in region server count | Major | integration tests |
| HBASE-24658 | Update PolicyBasedChaosMonkey to handle uncaught exceptions | Minor | integration tests |
| HBASE-24648 | Remove the legacy 'forceSplit' related code at region server side | Major | regionserver |
| HBASE-24492 | ProtobufLogReader.readNext does not need looping | Minor | Replication, wal |
| HBASE-22033 | Update to maven-javadoc-plugin 3.2.0 and switch to non-forking aggregate goals | Major | build, website |
Release 2.3.0
CHANGES.md at rel/2.3.0 (e0e1382)
Release 2.2.0
CHANGES.md at rel/2.2.0 (3ec6932)
Release 2.1.0
CHANGES.md at rel/2.1.0 (e1673bb)
Release 2.0.0
CHANGES.md at rel/2.0.0 (7483b11)