hbase/hbase-it
Andrew Purtell 4e19892a53 HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements (#4488)
* HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements

- Use an async client and work stealing executor for parallelism during loads.
- Remove the verification read retries, these are not that effective during
  replication lag anyway.
- Increase max task attempts because S3 might throttle.
- Implement a side task that exercises Increments by extracting urls from
  content and updating a cf that tracks referrer counts. These are not
  validated at this time. It could be possible to log the increments, sum
  them with a reducer, and then verify the total, but this is left as a
  future exercise.

Signed-off-by: Viraj Jasani <vjasani@apache.org>

* Sum RPC time for writes (loader) and reads (verifier) and mutation bytes submitted. Expose as job counters.

* Fix an issue with completion chaining

* Pause loading if too many operations are in flight
2022-07-13 09:02:00 -07:00
..
2022-05-01 22:41:49 +08:00