-Added new LocalityCostFunction and LocalityCandidateGenerator that
cache localities of every region/rack combination and mappings of every
region to its most local server and to its most local rack.
-Made LocalityCostFunction incremental so that it only computes locality
based on most recent region moves/swaps, rather than recomputing the
locality of every region in the cluster at every iteration of the
balancer
-Changed locality cost function to reflect the ratio of:
(Current locality) / (Best locality possible given current cluster)
Signed-off-by: tedyu <yuzhihong@gmail.com>
After enabling test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta, this bug is found in ServerCrashProcedure
Signed-off-by: Michael Stack <stack@apache.org>
In Standalone mode with local filesystem HBase logs Warning message:Failed to invoke 'unbuffer' method in class org.apache.hadoop.fs.FSDataInputStream
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
Modified jruby scripts to pick up drainingZNode and masterAddressZNode
correctly, from within znodePaths in ZooKeeperWatcher. Modified
get-active-master.rb to retrieve server name using MasterAddressTracker
instead of ZKUtil.
Tested on a standalone HBase deployment to check that the scripts work
as expected.
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
-Added new LocalityCostFunction and LocalityCandidateGenerator that
cache localities of every region/rack combination and mappings of every
region to its most local server and to its most local rack.
-Made LocalityCostFunction incremental so that it only computes locality
based on most recent region moves/swaps, rather than recomputing the
locality of every region in the cluster at every iteration of the
balancer
-Changed locality cost function to reflect the ratio of:
(Current locality) / (Best locality possible given current cluster)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Unit test (TestAssignmentManager) uses mock which always aggregates. So added trace level log message and verified manually on a single node cluster.
Signed-off-by: Michael Stack <stack@apache.org>
Calling closeRegion() directly on remote server is not supported post-AMv2. Calling unassign() on master
Signed-off-by: Michael Stack <stack@apache.org>
Introduces a new Chore in the Master which computes the size
of the snapshots included in a cluster. The size of these
snapshots are included in the table's which the snapshot was created
from HDFS usage.
Includes some test stabilization, trying to make the tests more
deterministic by ensuring we observe stable values as we know
that those values are mutable. This should help avoid problems
where size reports are delayed and we see an incomplete value.
Hadoop 3.x added some new dependencies that refer to the apache license
by it's older title (from apache parent pom 14 and earlier). Instead of
listing them all individually, we can group them under ASL for our
LICENSE file.
Signed-off-by: Sean Busbey <busbey@apache.org>
Guanghao Zhang
张世彬10204932
Kahlil Oppenheimer
Misty Stanley-Jones
David Harju
Ramkrishna
Umesh Agashe
Michael Stack
Andrew Purtell
Mike Drob
QilinCao
tedyu
Josh Elser
Ashish Singhi
Chinmay Kulkarni
Jan Hentschel
dskskv
Sean Busbey
Pankaj Kumar
Ashu Pachauri
zhangduo
Balazs Meszaros
Vincent
Ashu Pachauri