mirror of https://github.com/apache/lucene.git
Ref Guide: standardize parameter formatting
This commit is contained in:
Cassandra Targett
parent
2a7826f7a4
commit
8ba6a95e7c
|
|
@ -487,17 +487,17 @@ distributing documents among the referenced collections so all updates will go t
|
|||
`/admin/collections?action=CREATEALIAS&name=_name_&collections=_collectionlist_`
|
||||
|
||||
Routed aliases are more complex to configure, but include logic for automatically inspecting a field on the document
|
||||
and using the value in that field to select a destination collection. Additionally the routed alias automates the
|
||||
and using the value in that field to select a destination collection. Additionally, the routed alias automates the
|
||||
partitioning of data across a series of collections by creating new collections periodically. This feature allows for
|
||||
indefinite indexing of data without degradation of performance otherwise experienced due to the continuous growth of
|
||||
an index. As new data arrives, a field on the document is inspected and the document is then potentially re-routed to
|
||||
another collection. The underlying collections can can be queried independently but usually the alias will be used.
|
||||
These collections are created automatically on the fly as new data arrives based on the parameters supplied in this
|
||||
command. Also for very high volume use cases or for use cases requiring only a limited span of data to be retained,
|
||||
command. For very high volume use cases or for use cases requiring only a limited span of data to be retained,
|
||||
collections older than a given age can be deleted. This delete of old collections only occurs if new documents are
|
||||
sent to the alias.
|
||||
|
||||
*NOTE:* Presently this is only supported for temporal fields stored as a
|
||||
NOTE: Presently this is only supported for temporal fields stored as a
|
||||
<<field-types-included-with-solr.adoc#field-types-included-with-solr,DatePointField or TrieDateField>> type. Other
|
||||
well ordered field types may be added in future versions.
|
||||
|
||||
|
|
@ -506,24 +506,24 @@ well ordered field types may be added in future versions.
|
|||
localhost:8983/solr/admin/collections?action=CREATEALIAS&name=timedata&router.start=NOW/DAY&router.field=evt_dt&router.name=time&router.interval=%2B1DAY&router.maxFutureMs=3600000&create-collection.collection.configName=myConfig&create-collection.numShards=2
|
||||
----
|
||||
|
||||
If run on Jan 15, 2018 The above will create an alias named timedata, that contains collections with names prefixed with
|
||||
`timedata` and an initial collection named `timedata_2018_01_15` will be created immediately. Updates sent to this
|
||||
If run on Jan 15, 2018, the above will create an alias named timedata, that contains collections with names prefixed
|
||||
with `timedata` and an initial collection named `timedata_2018_01_15` will be created immediately. Updates sent to this
|
||||
alias with a (required) value in `evt_dt` that is before or after 2018-01-15 will be rejected, until the last 60
|
||||
minutes of 2018-01-15. After 2018-01-15T23:00:00 documents for either 2018-01-15 or 2018-01-16 will be accepted.
|
||||
As soon as the system receives a document for an allowable time window for which there is no collection it will
|
||||
automatically create the next required collection (and potentially any intervening collections if router.interval is
|
||||
smaller than router.maxFutureMs). Both the initial collection and any subsequent collections will be created using
|
||||
the specified configset. All Collection creation parameters other than `name` are allowed, prefixed
|
||||
automatically create the next required collection (and potentially any intervening collections if `router.interval` is
|
||||
smaller than `router.maxFutureMs`). Both the initial collection and any subsequent collections will be created using
|
||||
the specified configset. All collection creation parameters other than `name` are allowed, prefixed
|
||||
by `create-collection.`
|
||||
|
||||
This means that one could, for example, partition their collections by day, and within each daily collection route
|
||||
the data to shards based on customer id. Such shards can be of any type (NRT, PULL or TLOG), and rule based replica
|
||||
the data to shards based on customer id. Such shards can be of any type (NRT, PULL or TLOG), and rule-based replica
|
||||
placement strategies may also be used.
|
||||
|
||||
The values supplied in this command for collection creation will be retained
|
||||
in alias properties, and can be verified by inspecting `aliases.json` in ZooKeeper.
|
||||
|
||||
*NOTE:* Presently only updates are routed and queries are distributed to all collections in the alias, but future
|
||||
NOTE: Presently only updates are routed and queries are distributed to all collections in the alias, but future
|
||||
features may enable routing of the query to the single appropriate collection based on a special parameter or perhaps
|
||||
a filter on the routed field.
|
||||
|
||||
|
|
|
|||
|
|
@ -40,22 +40,34 @@ Listener configuration can also specify what particular actions are of interest,
|
|||
== Listener Configuration
|
||||
Currently the following listener configuration properties are supported:
|
||||
|
||||
* `name` - (string, required) A unique listener configuration name.
|
||||
* `trigger` - (string, required) The name of an existing trigger configuration.
|
||||
* `class` - (string, required) A listener implementation class name.
|
||||
* `stage` - (list of strings, optional, ignored case) A list of processing stages that
|
||||
`name`::
|
||||
(string, required) A unique listener configuration name.
|
||||
|
||||
`trigger`::
|
||||
(string, required) The name of an existing trigger configuration.
|
||||
|
||||
`class`::
|
||||
(string, required) A listener implementation class name.
|
||||
|
||||
`stage`::
|
||||
(list of strings, optional, ignored case) A list of processing stages that
|
||||
this listener should be notified. Default is empty list.
|
||||
* `beforeAction` - (list of strings, optional) A list of action names (as defined in trigger configuration) before
|
||||
|
||||
`beforeAction`::
|
||||
(list of strings, optional) A list of action names (as defined in trigger configuration) before
|
||||
which the listener will be notified. Default is empty list.
|
||||
* `afterAction` - (list of strings, optional) A list of action names after which the listener will be notified.
|
||||
|
||||
`afterAction`::
|
||||
(list of strings, optional) A list of action names after which the listener will be notified.
|
||||
Default is empty list.
|
||||
* Additional implementation-specific properties may be provided.
|
||||
|
||||
TIP: Additional implementation-specific properties may be provided, depending on the listener implementation.
|
||||
|
||||
Note: when both `stage` and `beforeAction` / `afterAction` lists are non-empty then the listener will be notified both
|
||||
when a specified stage is entered and before / after specified actions.
|
||||
|
||||
=== Managing Listener Configurations
|
||||
Listener configurations can be managed using the Autoscaling Write API, and using `set-listener` and `remove-listener`
|
||||
Listener configurations are managed using the Autoscaling Write API, and using `set-listener` and `remove-listener`
|
||||
commands.
|
||||
|
||||
For example:
|
||||
|
|
@ -95,9 +107,12 @@ actions related to the autoscaling framework are logged to the `.system` collect
|
|||
|
||||
Supported configuration properties:
|
||||
|
||||
* `collection` - (string, optional) specifies the target collection where documents are sent.
|
||||
Default value is `.system`
|
||||
* `enabled` - (boolean, optional) enables the listener when true. Default value is true.
|
||||
`collection`::
|
||||
(string, optional) Specifies the target collection where documents are sent.
|
||||
Default value is `.system`.
|
||||
|
||||
`enabled`::
|
||||
(boolean, optional) Enables the listener when true. Default value is true.
|
||||
|
||||
Documents created by this listener have several predefined fields:
|
||||
|
||||
|
|
@ -148,12 +163,23 @@ if the value is absent.
|
|||
|
||||
Supported configuration properties:
|
||||
|
||||
* `url` - (string, required) A URL template.
|
||||
* `payload` - (string, optional) A payload template. If absent, a JSON map of all properties listed above will be used.
|
||||
* `contentType` - (string, optional) A payload content type. If absent then `application/json` will be used.
|
||||
* `header.*` - (string, optional) A header template(s). The name of the property without "header." prefix defines the literal header name.
|
||||
* `timeout` - (int, optional) Connection and socket timeout in milliseconds. Default is `60000` milliseconds (60 seconds).
|
||||
* `followRedirects` - (boolean, optional) Allows following redirects. Default is `false`.
|
||||
`url`::
|
||||
(string, required) A URL template.
|
||||
|
||||
`payload`::
|
||||
(string, optional) A payload template. If absent, a JSON map of all properties listed above will be used.
|
||||
|
||||
`contentType`::
|
||||
(string, optional) A payload content type. If absent then `application/json` will be used.
|
||||
|
||||
`header.*`::
|
||||
(string, optional) A header template(s). The name of the property without "header." prefix defines the literal header name.
|
||||
|
||||
`timeout`::
|
||||
(int, optional) Connection and socket timeout in milliseconds. Default is `60000` milliseconds (60 seconds).
|
||||
|
||||
`followRedirects`::
|
||||
(boolean, optional) Allows following redirects. Default is `false`.
|
||||
|
||||
The following properties are available in context and can be referenced from templates:
|
||||
|
||||
|
|
|
|||
|
|
@ -26,8 +26,7 @@ change that merits attention they generate events, which are then queued and pro
|
|||
`TriggerAction` implementations. This usually involves computing and executing a plan to manage the new cluster
|
||||
resources (e.g., move replicas). Solr provides predefined implementations of triggers for specific event types.
|
||||
|
||||
Triggers execute on the node that runs `Overseer`. They are scheduled to run periodically,
|
||||
currently at default interval of 1 second between each execution (not every execution produces events).
|
||||
Triggers execute on the node that runs `Overseer`. They are scheduled to run periodically, at a default interval of 1 second between each execution (not every execution produces events).
|
||||
|
||||
== Event Types
|
||||
Currently the following event types (and corresponding trigger implementations) are defined:
|
||||
|
|
@ -36,7 +35,7 @@ Currently the following event types (and corresponding trigger implementations)
|
|||
* `nodeLost`: generated when a node leaves the cluster
|
||||
* `metric`: generated when the configured metric crosses a configured lower or upper threshold value
|
||||
* `searchRate`: generated when the 1-minute average search rate exceeds configured upper threshold
|
||||
* `scheduled` - generated according to a scheduled time period such as every 24 hours etc
|
||||
* `scheduled`: generated according to a scheduled time period such as every 24 hours etc
|
||||
|
||||
Events are not necessarily generated immediately after the corresponding state change occurred - the
|
||||
maximum rate of events is controlled by the `waitFor` configuration parameter (see below).
|
||||
|
|
@ -153,25 +152,35 @@ the threshold rate and the current request rate.
|
|||
}
|
||||
----
|
||||
|
||||
== Scheduled trigger
|
||||
== Scheduled Trigger
|
||||
|
||||
The Scheduled trigger generates events according to a fixed rate schedule.
|
||||
|
||||
The trigger supports the following configuration:
|
||||
|
||||
* `startTime` - (string, required) the start date/time of the schedule. This should either be a DateMath string e.g. 'NOW' or be an ISO-8601 date time string (the same standard used during search and indexing in Solr, thus defaulting to UTC) or be specified without the trailing 'Z' accompanied with the `timeZone` parameter. For example, each of the following values are acceptable:
|
||||
** `2018-01-31T15:30:00Z` - ISO-8601 date time string. The trailing `Z` signals that the time is in UTC
|
||||
** `NOW+5MINUTES` - Solr's date math string
|
||||
** `2018-01-31T15:30:00` - No trailing 'Z' signals that the `timeZone` parameter must be specified to avoid ambiguity
|
||||
* `every` - (string, required) a positive Solr date math string which is added to the `startTime` or the last run time to arrive at the next scheduled time
|
||||
* `graceTime` - (string, optional) a positive Solr date math string. This is the additional grace time over the scheduled time within which the trigger is allowed to generate an event.
|
||||
* `timeZone` - (string, optional) a time zone string which is used for calculating the scheduled times
|
||||
* `preferredOp` - (string, optional, defaults to `MOVEREPLICA`) the preferred operation to perform in response to an event generated by this trigger. The only supported values are `MOVEREPLICA` and `ADDREPLICA`.
|
||||
`startTime`::
|
||||
(string, required) The start date/time of the schedule. This should either be a DateMath string e.g., 'NOW', or be an ISO-8601 date time string (the same standard used during search and indexing in Solr, which defaults to UTC), or be specified without the trailing 'Z' accompanied with the `timeZone` parameter. For example, each of the following values are acceptable:
|
||||
* `2018-01-31T15:30:00Z`: ISO-8601 date time string. The trailing `Z` signals that the time is in UTC
|
||||
* `NOW+5MINUTES`: Solr's date math string
|
||||
* `2018-01-31T15:30:00`: No trailing 'Z' signals that the `timeZone` parameter must be specified to avoid ambiguity
|
||||
|
||||
`every`::
|
||||
(string, required) A positive Solr date math string which is added to the `startTime` or the last run time to arrive at the next scheduled time.
|
||||
|
||||
`graceTime`::
|
||||
(string, optional) A positive Solr date math string. This is the additional grace time over the scheduled time within which the trigger is allowed to generate an event.
|
||||
|
||||
`timeZone`::
|
||||
(string, optional) A time zone string which is used for calculating the scheduled times.
|
||||
|
||||
`preferredOp`::
|
||||
(string, optional, defaults to `MOVEREPLICA`) The preferred operation to perform in response to an event generated by this trigger. The only supported values are `MOVEREPLICA` or `ADDREPLICA`.
|
||||
|
||||
This trigger applies the `every` date math expression on the `startTime` or the last event time to derive the next scheduled time and if current time is greater than next scheduled time but within `graceTime` then an event is generated.
|
||||
|
||||
Apart from the common event properties described in the Event Types section, the trigger adds an additional `actualEventTime` event property which has the actual event time as opposed to the scheduled time.
|
||||
For example, if the scheduled time was `2018-01-31T15:30:00Z` and grace time was `+15MINUTES` then an event may be fired at `2018-01-31T15:45:00Z`. Such an event will have `eventTime` as `2018-01-31T15:30:00Z` i.e. the scheduled time but the `actualEventTime` property will have a value of `2018-01-31T15:45:00Z`.
|
||||
|
||||
For example, if the scheduled time was `2018-01-31T15:30:00Z` and grace time was `+15MINUTES` then an event may be fired at `2018-01-31T15:45:00Z`. Such an event will have `eventTime` as `2018-01-31T15:30:00Z`, the scheduled time, but the `actualEventTime` property will have a value of `2018-01-31T15:45:00Z`, the actual time.
|
||||
|
||||
== Trigger Configuration
|
||||
Trigger configurations are managed using the Autoscaling Write API and the commands `set-trigger`, `remove-trigger`,
|
||||
|
|
|
|||
|
|
@ -29,6 +29,6 @@ The following sections describe the autoscaling features of SolrCloud:
|
|||
* <<solrcloud-autoscaling-triggers.adoc#solrcloud-autoscaling-triggers,Autoscaling Triggers>>
|
||||
* <<solrcloud-autoscaling-trigger-actions.adoc#solrcloud-autoscaling-trigger-actions,Autoscaling Trigger Actions>>
|
||||
* <<solrcloud-autoscaling-listeners.adoc#solrcloud-autoscaling-listeners,Autoscaling Listeners>>
|
||||
* <<solrcloud-autoscaling-auto-add-replicas.adoc#solrcloud-autoscaling-auto-add-replicas,Autoscaling - Automatically Adding Replicas>>
|
||||
* <<solrcloud-autoscaling-auto-add-replicas.adoc#solrcloud-autoscaling-auto-add-replicas,Automatically Adding Replicas>>
|
||||
* <<solrcloud-autoscaling-fault-tolerance.adoc#solrcloud-autoscaling-fault-tolerance,Autoscaling Fault Tolerance>>
|
||||
* <<solrcloud-autoscaling-api.adoc#solrcloud-autoscaling-api,Autoscaling API>>
|
||||
|
|
|
|||
Loading…
Reference in New Issue