From f90420b3a556952f73d1e7cf76c33abfd06c5919 Mon Sep 17 00:00:00 2001 From: Michael Stack Date: Wed, 6 Dec 2017 17:15:01 -0800 Subject: [PATCH] HBASE-19301 Provide way for CPs to create short circuited connection with custom configurations; ADDENDUM -- adds warning to powerful new, Signed-off-by: anoopsamjohn --- .../MasterCoprocessorEnvironment.java | 56 ++++++++++++------- .../RegionCoprocessorEnvironment.java | 56 ++++++++++++------- .../RegionServerCoprocessorEnvironment.java | 56 ++++++++++++------- 3 files changed, 111 insertions(+), 57 deletions(-) diff --git a/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterCoprocessorEnvironment.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterCoprocessorEnvironment.java index 0331a0b373b..be6c4448052 100644 --- a/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterCoprocessorEnvironment.java +++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterCoprocessorEnvironment.java @@ -39,32 +39,50 @@ public interface MasterCoprocessorEnvironment extends CoprocessorEnvironmentUsing this Connection to get at a local resource -- say a Region that is on the local - * Server or using Admin Interface from a Coprocessor hosted on the Master -- will result in a - * short-circuit of the RPC framework to make a direct invocation avoiding RPC. - *

- * Note: If you want to create Connection with your own Configuration and NOT use the Master's - * Connection (though its cache of locations will be warm, and its life-cycle is not the concern - * of the CP), see {@link #createConnection(Configuration)}. + * Do not close! Doing so will buckle the hosting server as it depends on its + * Connection to function. For light-weight usage only. Heavy-duty usage will pull down + * the hosting RegionServer responsiveness as well as that of other Coprocessors making use of + * this Connection. Use to create table on start or to do administrative operations. Coprocessors + * should create their own Connections if heavy usage to avoid impinging on hosting Server + * operation. To create a Connection or if a Coprocessor requires a region with a particular + * Configuration, use {@link org.apache.hadoop.hbase.client.ConnectionFactory} or + * {@link #createConnection(Configuration)}}. + * + *

Be aware that operations that make use of this Connection are executed as the RegionServer + * User, the hbase super user that started this server process. Exercise caution running + * operations as this User (See {@link #createConnection(Configuration)}} to run as other than + * the RegionServer User). + * + *

Be careful RPC'ing from a Coprocessor context. RPC's will fail, stall, retry, and/or crawl + * because the remote side is not online, is struggling or it is on the other side of a network + * partition. Any use of Connection from inside a Coprocessor must be able to handle all such + * hiccups. + * + * @see #createConnection(Configuration) * @return The host's Connection to the Cluster. */ Connection getConnection(); /** - * Creates a cluster connection using the passed configuration. - *

Using this Connection to get at a local resource -- say a Region that is on the local - * Server or using Admin Interface from a Coprocessor hosted on the Master -- will result in a - * short-circuit of the RPC framework to make a direct invocation avoiding RPC. - *

- * Note: HBase will NOT cache/maintain this Connection. If Coprocessors need to cache and reuse - * this connection, it has to be done by Coprocessors. Also make sure to close it after use. + * Creates a cluster connection using the passed Configuration. * - * @param conf configuration + * Creating a Connection is a heavy-weight operation. The resultant Connection's cache of + * region locations will be empty. Therefore you should cache and reuse Connections rather than + * create a Connection on demand. Create on start of your Coprocessor. You will have to cast + * the CoprocessorEnvironment appropriately to get at this API at start time because + * Coprocessor start method is passed a subclass of this CoprocessorEnvironment or fetch + * Connection using a synchronized accessor initializing the Connection on first access. Close + * the returned Connection when done to free resources. Using this API rather + * than {@link org.apache.hadoop.hbase.client.ConnectionFactory#createConnection(Configuration)} + * returns a Connection that will short-circuit RPC if the target is a local resource. Use + * ConnectionFactory if you don't need this ability. + * + *

Be careful RPC'ing from a Coprocessor context. RPC's will fail, stall, retry, and/or crawl + * because the remote side is not online, is struggling or it is on the other side of a network + * partition. Any use of Connection from inside a Coprocessor must be able to handle all such + * hiccups. * @return Connection created using the passed conf. */ Connection createConnection(Configuration conf) throws IOException; diff --git a/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java index ca57fc899a2..65e1c4c1649 100644 --- a/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java +++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java @@ -58,32 +58,50 @@ public interface RegionCoprocessorEnvironment extends CoprocessorEnvironmentUsing a Connection to get at a local resource -- say a Region that is on the local - * Server or using Admin Interface from a Coprocessor hosted on the Master -- will result in a - * short-circuit of the RPC framework to make a direct invocation avoiding RPC. - *

- * Note: If you want to create Connection with your own Configuration and NOT use the RegionServer - * Connection (though its cache of locations will be warm, and its life-cycle is not the concern - * of the CP), see {@link #createConnection(Configuration)}. + * Do not close! Doing so will buckle the hosting server as it depends on its + * Connection to function. For light-weight usage only. Heavy-duty usage will pull down + * the hosting RegionServer responsiveness as well as that of other Coprocessors making use of + * this Connection. Use to create table on start or to do administrative operations. Coprocessors + * should create their own Connections if heavy usage to avoid impinging on hosting Server + * operation. To create a Connection or if a Coprocessor requires a region with a particular + * Configuration, use {@link org.apache.hadoop.hbase.client.ConnectionFactory} or + * {@link #createConnection(Configuration)}}. + * + *

Be aware that operations that make use of this Connection are executed as the RegionServer + * User, the hbase super user that started this server process. Exercise caution running + * operations as this User (See {@link #createConnection(Configuration)}} to run as other than + * the RegionServer User). + * + *

Be careful RPC'ing from a Coprocessor context. RPC's will fail, stall, retry, and/or crawl + * because the remote side is not online, is struggling or it is on the other side of a network + * partition. Any use of Connection from inside a Coprocessor must be able to handle all such + * hiccups. + * + * @see #createConnection(Configuration) * @return The host's Connection to the Cluster. */ Connection getConnection(); /** - * Creates a cluster connection using the passed configuration. - *

Using this Connection to get at a local resource -- say a Region that is on the local - * Server or using Admin Interface from a Coprocessor hosted on the Master -- will result in a - * short-circuit of the RPC framework to make a direct invocation avoiding RPC. - *

- * Note: HBase will NOT cache/maintain this Connection. If Coprocessors need to cache and reuse - * this connection, it has to be done by Coprocessors. Also make sure to close it after use. + * Creates a cluster connection using the passed Configuration. * - * @param conf configuration + * Creating a Connection is a heavy-weight operation. The resultant Connection's cache of + * region locations will be empty. Therefore you should cache and reuse Connections rather than + * create a Connection on demand. Create on start of your Coprocessor. You will have to cast + * the CoprocessorEnvironment appropriately to get at this API at start time because + * Coprocessor start method is passed a subclass of this CoprocessorEnvironment or fetch + * Connection using a synchronized accessor initializing the Connection on first access. Close + * the returned Connection when done to free resources. Using this API rather + * than {@link org.apache.hadoop.hbase.client.ConnectionFactory#createConnection(Configuration)} + * returns a Connection that will short-circuit RPC if the target is a local resource. Use + * ConnectionFactory if you don't need this ability. + * + *

Be careful RPC'ing from a Coprocessor context. RPC's will fail, stall, retry, and/or crawl + * because the remote side is not online, is struggling or it is on the other side of a network + * partition. Any use of Connection from inside a Coprocessor must be able to handle all such + * hiccups. * @return Connection created using the passed conf. */ Connection createConnection(Configuration conf) throws IOException; diff --git a/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerCoprocessorEnvironment.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerCoprocessorEnvironment.java index c1c0c25d5ad..c6f64849eb8 100644 --- a/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerCoprocessorEnvironment.java +++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerCoprocessorEnvironment.java @@ -45,32 +45,50 @@ public interface RegionServerCoprocessorEnvironment OnlineRegions getOnlineRegions(); /** - * Be careful RPC'ing from a Coprocessor context. - * RPC's will fail, stall, retry, and/or crawl because the remote side is not online, is - * struggling or it is on the other side of a network partition. Any use of Connection from - * inside a Coprocessor must be able to handle all such hiccups. + * Returns the hosts' Connection to the Cluster. * - *

Using a Connection to get at a local resource -- say a Region that is on the local - * Server or using Admin Interface from a Coprocessor hosted on the Master -- will result in a - * short-circuit of the RPC framework to make a direct invocation avoiding RPC. - *

- * Note: If you want to create Connection with your own Configuration and NOT use the RegionServer - * Connection (though its cache of locations will be warm, and its life-cycle is not the concern - * of the CP), see {@link #createConnection(Configuration)}. + * Do not close! Doing so will buckle the hosting server as it depends on its + * Connection to function. For light-weight usage only. Heavy-duty usage will pull down + * the hosting RegionServer responsiveness as well as that of other Coprocessors making use of + * this Connection. Use to create table on start or to do administrative operations. Coprocessors + * should create their own Connections if heavy usage to avoid impinging on hosting Server + * operation. To create a Connection or if a Coprocessor requires a region with a particular + * Configuration, use {@link org.apache.hadoop.hbase.client.ConnectionFactory} or + * {@link #createConnection(Configuration)}}. + * + *

Be aware that operations that make use of this Connection are executed as the RegionServer + * User, the hbase super user that started this server process. Exercise caution running + * operations as this User (See {@link #createConnection(Configuration)}} to run as other than + * the RegionServer User). + * + *

Be careful RPC'ing from a Coprocessor context. RPC's will fail, stall, retry, and/or crawl + * because the remote side is not online, is struggling or it is on the other side of a network + * partition. Any use of Connection from inside a Coprocessor must be able to handle all such + * hiccups. + * + * @see #createConnection(Configuration) * @return The host's Connection to the Cluster. */ Connection getConnection(); /** - * Creates a cluster connection using the passed configuration. - *

Using this Connection to get at a local resource -- say a Region that is on the local - * Server or using Admin Interface from a Coprocessor hosted on the Master -- will result in a - * short-circuit of the RPC framework to make a direct invocation avoiding RPC. - *

- * Note: HBase will NOT cache/maintain this Connection. If Coprocessors need to cache and reuse - * this connection, it has to be done by Coprocessors. Also make sure to close it after use. + * Creates a cluster connection using the passed Configuration. * - * @param conf configuration + * Creating a Connection is a heavy-weight operation. The resultant Connection's cache of + * region locations will be empty. Therefore you should cache and reuse Connections rather than + * create a Connection on demand. Create on start of your Coprocessor. You will have to cast + * the CoprocessorEnvironment appropriately to get at this API at start time because + * Coprocessor start method is passed a subclass of this CoprocessorEnvironment or fetch + * Connection using a synchronized accessor initializing the Connection on first access. Close + * the returned Connection when done to free resources. Using this API rather + * than {@link org.apache.hadoop.hbase.client.ConnectionFactory#createConnection(Configuration)} + * returns a Connection that will short-circuit RPC if the target is a local resource. Use + * ConnectionFactory if you don't need this ability. + * + *

Be careful RPC'ing from a Coprocessor context. RPC's will fail, stall, retry, and/or crawl + * because the remote side is not online, is struggling or it is on the other side of a network + * partition. Any use of Connection from inside a Coprocessor must be able to handle all such + * hiccups. * @return Connection created using the passed conf. */ Connection createConnection(Configuration conf) throws IOException;