hbase/hbase-it
Andrew Purtell 55fce40918 HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements (#4488)
* HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements

- Use an async client and work stealing executor for parallelism during loads.
- Remove the verification read retries, these are not that effective during
  replication lag anyway.
- Increase max task attempts because S3 might throttle.
- Implement a side task that exercises Increments by extracting urls from
  content and updating a cf that tracks referrer counts. These are not
  validated at this time. It could be possible to log the increments, sum
  them with a reducer, and then verify the total, but this is left as a
  future exercise.

Signed-off-by: Viraj Jasani <vjasani@apache.org>

* Sum RPC time for writes (loader) and reads (verifier) and mutation bytes submitted. Expose as job counters.

* Fix an issue with completion chaining

* Pause loading if too many operations are in flight
2022-07-13 09:04:28 -07:00
..
src HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements (#4488) 2022-07-13 09:04:28 -07:00
pom.xml Preparing development version 2.5.1-SNAPSHOT 2022-05-31 20:06:32 -07:00