hbase/hbase-it
Andrew Purtell ef1641d277
HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements (#4488)
* HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements

- Use an async client and work stealing executor for parallelism during loads.
- Remove the verification read retries, these are not that effective during
  replication lag anyway.
- Increase max task attempts because S3 might throttle.
- Implement a side task that exercises Increments by extracting urls from
  content and updating a cf that tracks referrer counts. These are not
  validated at this time. It could be possible to log the increments, sum
  them with a reducer, and then verify the total, but this is left as a
  future exercise.

Signed-off-by: Viraj Jasani <vjasani@apache.org>

* Sum RPC time for writes (loader) and reads (verifier) and mutation bytes submitted. Expose as job counters.

* Fix an issue with completion chaining

* Pause loading if too many operations are in flight
2022-07-13 09:01:21 -07:00
..
src HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements (#4488) 2022-07-13 09:01:21 -07:00
pom.xml HBASE-27165 Set version as 3.0.0-alpha-4-SNAPSHOT in master (#4582) 2022-06-29 11:04:21 +08:00