mirror of https://github.com/apache/lucene.git
SOLR-15022: Add RefGuide documentation for cluster plugins.
This commit is contained in:
Andrzej Bialecki
parent
510f96c69c
commit
8dcaa6c6d3
|
|
@ -76,5 +76,4 @@ public class PlacementPluginFactoryLoader {
|
|||
};
|
||||
plugins.registerListener(pluginListener);
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,143 +0,0 @@
|
|||
= Container Plugins
|
||||
:toc: macro
|
||||
:toclevels: 3
|
||||
|
||||
|===
|
||||
| Initial version released| 2020, November 2|Andrzej Białecki
|
||||
|===
|
||||
|
||||
toc::[]
|
||||
|
||||
== Container plugins subsystem
|
||||
Container plugins are pluggable components that are defined and instantiated at the
|
||||
`CoreContainer` (node) level. The components usually provide an admin-level API for
|
||||
additional functionality at the Solr node level.
|
||||
|
||||
=== Plugin configurations
|
||||
Plugin configurations are maintained in ZooKeeper in the `/clusterprops.json` file, under
|
||||
the `plugin` entry. The configuration is a JSON map where keys are the unique plugin names, and
|
||||
values are serialized `org.apache.solr.client.solrj.request.beans.PluginMeta` beans.
|
||||
|
||||
=== Types of container plugins
|
||||
Typically classes loaded from plugins support two types of functionality (not mutually exclusive):
|
||||
|
||||
* plugins that expose REST API endpoints (the implementing class is annotated with
|
||||
`@EndPoint` and optionally `@Command` annotations). The APIs of these plugins are automatically
|
||||
registered as REST endpoints under the paths defined in the `@EndPoint` annotations.
|
||||
|
||||
* plugins that implement a specific interface. Upon loading they are automatically
|
||||
discovered and registered with sub-systems that use this type of plugin. Examples here
|
||||
include the `ClusterSingleton`, ClusterEventProducer`, `ClusterEventListener`.
|
||||
// and `PlacementPluginConfig`
|
||||
|
||||
=== Plugin life-cycle
|
||||
Plugin instances are loaded and initialized when Solr's `CoreContainer` is first created.
|
||||
|
||||
Then on each update of the plugin configurations the existing plugin configs are compared
|
||||
with the new configs, and plugin instances are respectively created, removed, or
|
||||
replaced (i.e. removed and added using the new configuration).
|
||||
|
||||
If a plugin implementation class has a constructor that accepts an instance of
|
||||
`CoreContainer` then it is instantiated using this constructor, and the current instance
|
||||
of `CoreContainer` is passed as argument.
|
||||
|
||||
=== PluginRegistryListener
|
||||
Components that need to be aware of changes in plugin configurations or registration can
|
||||
implement `org.apache.solr.api.ContainerPluginsRegistry.PluginRegistryListener` and register
|
||||
it with the instance of registry available from `coreContainer.getContainerPluginsRegistry()`.
|
||||
|
||||
== Plugin types
|
||||
|
||||
=== Predefined plugin names
|
||||
|
||||
Plugins with these names are used in specific parts of Solr. Their names are reserved
|
||||
and cannot be used for other plugin types:
|
||||
|
||||
// XXX uncomment when we move the config to plugins
|
||||
//`placement-plugin`::
|
||||
//plugin that implements `PlacementPlugin` interface. This type of plugin
|
||||
//determines the replica placement strategy in the cluster.
|
||||
|
||||
`cluster-event-producer`::
|
||||
plugin that implements `ClusterEventProducer` interface. This type of plugin
|
||||
is used for generating cluster-level events.
|
||||
|
||||
=== ClusterSingleton plugins
|
||||
Plugins that implement `ClusterSingleton` interface are instantiated on each
|
||||
Solr node. However, their start/stop life-cycle, as defined in the interface,
|
||||
is controlled in such a way that only a single running instance of the plugin
|
||||
is present in the cluster at any time.
|
||||
|
||||
(Currently this is implemented by re-using the Overseer leader election, so all
|
||||
`ClusterSingleton`-s that are in the RUNNING state execute on the Overseer leader).
|
||||
|
||||
Any plugin type can implement this interface to indicate to Solr that
|
||||
it requires this cluster singleton behavior.
|
||||
|
||||
=== ClusterEventProducer plugin
|
||||
In order to support the generation of cluster-level events an implementation of
|
||||
`ClusterEventProducer` is created on each Solr node. This component is also a
|
||||
`ClusterSingleton`, which means that only one active instance is present in the
|
||||
cluster at any time.
|
||||
|
||||
If no plugin configuration is specified then the default implementation
|
||||
`org.apache.solr.cluster.events.impl.NoOpProducer` is used, which doesn't generate
|
||||
any events - this means that by default event generation is turned off. An implementation
|
||||
that supports node and collection event generation is also available in
|
||||
`org.apache.solr.cluster.events.impl.DefaultClusterEventProducer`.
|
||||
|
||||
Event producer configuration can be changed dynamically by changing the predefined
|
||||
plugin configuration, for example:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"add":{
|
||||
"name": "cluster-event-producer",
|
||||
"class": "org.apache.solr.cluster.events.impl.DefaultClusterEventProducer"
|
||||
}}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
It can be restored to the default no-op configuration by simply removing the plugin:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"remove": "cluster-event-producer"
|
||||
}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
|
||||
=== ClusterEventListener plugins
|
||||
Plugins that implement the `ClusterEventListener` interface will be automatically
|
||||
registered with the instance of `ClusterEventProducer`.
|
||||
|
||||
// XXX edit this once SOLR-14977 is done
|
||||
Implementations will be notified of all events that are generated by the
|
||||
`ClusterEventProducer` and need to select only events that they are interested in.
|
||||
|
||||
==== org.apache.solr.cluster.events.impl.CollectionsRepairEventListener
|
||||
An implementation of listener that reacts to NODE_LOST events and checks what replicas
|
||||
need to be re-added to other nodes to keep the replication counts the same as before.
|
||||
|
||||
This implementation waits for a certain period (default is 30s) to make sure the node
|
||||
is really down, and for the replicas located on nodes that were down sufficiently long
|
||||
it generates appropriate ADDREPLICA commands to counter-balance the lost replicas on
|
||||
these nodes.
|
||||
|
||||
Example plugin configuration:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"add":{
|
||||
"name": "collections-repair-listener",
|
||||
"class": "org.apache.solr.cluster.events.impl.CollectionsRepairEventListener"
|
||||
}}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
== Plugin management API
|
||||
|
||||
|
|
@ -0,0 +1,242 @@
|
|||
= Cluster Plugins
|
||||
:toc: macro
|
||||
:toclevels: 3
|
||||
|
||||
toc::[]
|
||||
|
||||
== Cluster (CoreContainer-level) plugins subsystem
|
||||
Cluster plugins are pluggable components that are defined and instantiated at the
|
||||
`CoreContainer` (node) level. These components usually provide admin-level functionality
|
||||
and APIs for additional functionality at the Solr node level.
|
||||
|
||||
=== Plugin configurations
|
||||
Plugin configurations are maintained using `/cluster/plugin` API.
|
||||
|
||||
This API endpoint allows adding, removing and updating plugin configurations.
|
||||
|
||||
Each plugin MUST have a unique name under which it's registered. Attempting to
|
||||
add a plugin with a duplicate name is an error. Some types of plugins use
|
||||
pre-defined names, and they MUST be registered under these names in order to
|
||||
properly function.
|
||||
|
||||
Internally, as of Solr 9.0 plugin configurations are maintained in ZooKeeper in the
|
||||
`/clusterprops.json` file, under the `plugin` entry. The configuration is a JSON map
|
||||
where keys are the unique plugin names, and values are serialized
|
||||
`org.apache.solr.client.solrj.request.beans.PluginMeta` beans.
|
||||
|
||||
The following common plugin properties are supported:
|
||||
|
||||
`name`::
|
||||
(required) unique plugin name. Some plugin types require using one of the
|
||||
pre-defined names to properly function. By convention such predefined names use
|
||||
a leading-dot prefix (e.g. `.placement-plugin`)
|
||||
|
||||
`class`::
|
||||
(required) implementation class. This can be specified as a fully-qualified
|
||||
class name if the class is available as a part of Solr, or it can be also
|
||||
specified using the `<package>:<className>` syntax to refer to a class inside
|
||||
one of the Solr packages.
|
||||
|
||||
`version`::
|
||||
(required when class is loaded from a package). Solr package version.
|
||||
|
||||
`path-prefix`::
|
||||
(optional, default is none). Path prefix to be added to the REST API endpoints defined in the plugin.
|
||||
|
||||
`config`::
|
||||
(optional, default is none). A JSON map of additional plugin configuration parameters.
|
||||
Plugins that implement `ConfigurablePlugin` interface will be initialized with a
|
||||
plugin-specific configuration object deserialized from this map.
|
||||
|
||||
Example plugin configuration:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"add":{
|
||||
"name": ".placement-plugin",
|
||||
"class": "org.apache.solr.cluster.placement.impl.CollectionsRepairEventListener"
|
||||
}}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
=== Types of cluster plugins
|
||||
Classes loaded from plugins in general support two types of functionality (not mutually exclusive):
|
||||
|
||||
* request handler plugins that expose REST API endpoints (the implementing class is annotated with
|
||||
`@EndPoint` and optionally `@Command` annotations). The APIs of these plugins are automatically
|
||||
registered as REST endpoints under the paths defined in the `@EndPoint` annotations.
|
||||
|
||||
* plugins that implement a specific interface, for use as an internal component. Upon loading they are
|
||||
automatically discovered and registered with sub-systems that use this type of plugin. Examples here
|
||||
include the `ClusterSingleton`, ClusterEventProducer`, `ClusterEventListener`
|
||||
and `PlacementPluginFactory`.
|
||||
|
||||
=== Plugin life-cycle
|
||||
Plugin instances are loaded and initialized when Solr's `CoreContainer` is first created during
|
||||
Solr node start-up.
|
||||
|
||||
Then on each update of the configurations each node is notified about the change,
|
||||
and then the existing plugins are compared with the new configs, and plugin instances
|
||||
present on the node are respectively created, removed, or
|
||||
replaced (i.e. removed and added using the new configuration).
|
||||
|
||||
In practice this means that cluster-level plugins managed by this API can be
|
||||
dynamically changed and reconfigured without restarting the Solr nodes, and the changes
|
||||
apply to all nodes nearly simultaneously.
|
||||
|
||||
== Plugin types
|
||||
|
||||
=== Predefined plugin names
|
||||
|
||||
Plugins with these names are used in specific parts of Solr. Their names are reserved
|
||||
and cannot be used for other plugin types:
|
||||
|
||||
`.placement-plugin`::
|
||||
plugin that implements `PlacementPluginFactory` interface. This type of plugin
|
||||
determines the replica placement strategy in the cluster.
|
||||
|
||||
`.cluster-event-producer`::
|
||||
plugin that implements `ClusterEventProducer` interface. This type of plugin
|
||||
is used for generating cluster-level events.
|
||||
|
||||
=== PlacementPluginFactory plugins
|
||||
This type of plugin supports configurable placement strategies for collection
|
||||
replicas.
|
||||
|
||||
=== ClusterSingleton plugins
|
||||
Plugins that implement `ClusterSingleton` interface are instantiated on each
|
||||
Solr node. However, their start/stop life-cycle, as defined in the interface,
|
||||
is controlled in such a way that only a single running instance of the plugin
|
||||
is present in the cluster at any time.
|
||||
|
||||
(Currently this is implemented by re-using the Overseer leader election, so all
|
||||
`ClusterSingleton`-s that are in the RUNNING state execute on the Overseer leader node).
|
||||
|
||||
Any plugin can implement this interface to indicate to Solr that
|
||||
it requires this cluster singleton behavior.
|
||||
|
||||
=== ClusterEventProducer plugin
|
||||
In order to support the generation of cluster-level events an implementation of
|
||||
`ClusterEventProducer` is created on each Solr node. This component is also a
|
||||
`ClusterSingleton`, which means that only one active instance is present in the
|
||||
cluster at any time.
|
||||
|
||||
If no plugin configuration is specified then the default implementation
|
||||
`org.apache.solr.cluster.events.impl.NoOpProducer` is used, which doesn't generate
|
||||
any events - this means that by default event generation is turned off. An implementation
|
||||
that supports node and collection event generation is also available in
|
||||
`org.apache.solr.cluster.events.impl.DefaultClusterEventProducer`.
|
||||
|
||||
Event producer configuration can be changed dynamically by changing the predefined
|
||||
plugin configuration, for example:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"add":{
|
||||
"name": ".cluster-event-producer",
|
||||
"class": "org.apache.solr.cluster.events.impl.DefaultClusterEventProducer"
|
||||
}}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
It can be restored to the default no-op configuration by simply removing the plugin:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"remove": ".cluster-event-producer"
|
||||
}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
|
||||
=== ClusterEventListener plugins
|
||||
Plugins that implement the `ClusterEventListener` interface will be automatically
|
||||
registered with the instance of `ClusterEventProducer`.
|
||||
|
||||
// XXX edit this once SOLR-14977 is done
|
||||
Implementations will be notified of all events that are generated by the
|
||||
`ClusterEventProducer` and need to select only events that they are interested in.
|
||||
|
||||
==== org.apache.solr.cluster.events.impl.CollectionsRepairEventListener
|
||||
An implementation of listener that reacts to NODE_LOST events and checks what replicas
|
||||
need to be re-added to other nodes to keep the replication counts the same as before.
|
||||
|
||||
This implementation waits for a certain period (default is 30s) to make sure the node
|
||||
is really down, and for the replicas located on nodes that were down sufficiently long
|
||||
it generates appropriate ADDREPLICA commands to counter-balance the lost replicas on
|
||||
these nodes.
|
||||
|
||||
Example plugin configuration:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"add":{
|
||||
"name": "collections-repair-listener",
|
||||
"class": "org.apache.solr.cluster.events.impl.CollectionsRepairEventListener"
|
||||
}}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
== Plugin management API
|
||||
|
||||
=== List plugins
|
||||
This command uses HTTP GET and returns a list of loaded plugins and their configurations:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
=== Add plugin
|
||||
This command uses HTTP POST to add a new plugin configuration. If a plugin with the
|
||||
same name already exists this results in an error.
|
||||
|
||||
Example command, which adds a plugin contained in a Solr package:
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"add":{
|
||||
"name": "my-plugin1",
|
||||
"class": "my-package:com.example.MyPlugin",
|
||||
"version": "1.0"
|
||||
}}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
=== Update plugin
|
||||
This command uses HTTP POST to update an existing plugin configuration. If a plugin
|
||||
with this name doesn't exist this results in an error.
|
||||
|
||||
This example updates an existing plugin, possibly changing its configuration paramers.
|
||||
The old instance of the plugin is removed and a new instance is created using the supplied
|
||||
configuration.
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"update":{
|
||||
"name": "collections-repair-listener",
|
||||
"class": "org.apache.solr.cluster.events.impl.CollectionsRepairEventListener",
|
||||
"config":{
|
||||
"waitForSecond": 30
|
||||
}}}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
||||
=== Remove plugin
|
||||
This command uses HTTP POST to delete an existing plugin configuration. If a plugin
|
||||
with this name doesn't exist this results in an error.
|
||||
|
||||
Unlike other commands the command payload here consists just of
|
||||
the name of the plugin to remove, as a string.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-type: application/json' -d '{
|
||||
"remove": "my-plugin1"
|
||||
}'
|
||||
http://localhost:8983/api/cluster/plugin
|
||||
----
|
||||
|
|
@ -55,3 +55,7 @@ Contrib modules ship with Solr so there's no effort for them but not so for othe
|
|||
Describes a new and experimental system to manage packages of plugins in SolrCloud.
|
||||
It includes CLI commands, cluster-wide installation, use of plugin registries that host plugins, cryptographically signed plugins for security, and more.
|
||||
Only some plugins support this as of now (support for more types of plugins coming soon).
|
||||
|
||||
* <<cluster-plugins.adoc#cluster-plugins,Cluster Plugins>>:
|
||||
Describes the API used for managing cluster-level plugins such as request handlers,
|
||||
cluster-level event producer and replica placement plugins.
|
||||
|
|
|
|||
Loading…
Reference in New Issue