HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction (#4383)

There are two cases here:
  1. Chaos Monkey thread died and there is no chaos after that.
  2. Sometimes, regions are being moved back too quick that region server has not finished its initliazation yet.
     wait sometime to make sure that region server finishes its initialization.

Signed-off-by:  Wellington Chevreuil <wellington.chevreuil@gmail.com>
This commit is contained in:
huaxiangsun 2022-05-05 09:37:41 -05:00 committed by GitHub
parent 9e03253447
commit 597098ebba
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 4 additions and 0 deletions

View File

@ -63,6 +63,8 @@ public class GracefulRollingRestartRsAction extends RestartActionBaseAction {
rm.load();
} catch (Shell.ExitCodeException e) {
getLogger().info("Problem restarting but presume successful; code={}", e.getExitCode(), e);
} catch (Exception e) {
getLogger().info("Exception but continuing...", e);
}
sleep(rand.nextInt((int) sleepTime));
}

View File

@ -80,6 +80,8 @@ public abstract class RestartActionBaseAction extends Action {
sleep(sleepTime);
getLogger().info("Starting region server: {}", server);
startRs(server);
// Sleep some time to make sure RS is online.
sleep(sleepTime);
}
void restartZKNode(ServerName server, long sleepTime) throws IOException {