2017-01-05 10:10:34 -05:00
|
|
|
[[modules-cross-cluster-search]]
|
2017-04-04 07:34:16 -04:00
|
|
|
== Cross Cluster Search
|
2017-01-05 10:10:34 -05:00
|
|
|
|
|
|
|
|
The _cross cluster search_ feature allows any node to act as a federated client across
|
2018-01-30 14:40:19 -05:00
|
|
|
multiple clusters. A cross cluster search node won't join the remote cluster, instead
|
|
|
|
|
it connects to a remote cluster in a light fashion in order to execute
|
2017-01-05 10:10:34 -05:00
|
|
|
federated search requests.
|
|
|
|
|
|
2017-04-04 07:34:16 -04:00
|
|
|
[float]
|
2018-10-20 08:33:59 -04:00
|
|
|
=== Using cross cluster search
|
2017-01-05 10:10:34 -05:00
|
|
|
|
2018-10-20 08:33:59 -04:00
|
|
|
Cross-cluster search requires <<modules-remote-clusters,configuring remote clusters>>.
|
2017-04-04 07:34:16 -04:00
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
--------------------------------
|
|
|
|
|
PUT _cluster/settings
|
|
|
|
|
{
|
|
|
|
|
"persistent": {
|
2018-09-05 20:43:44 -04:00
|
|
|
"cluster": {
|
2017-04-04 07:34:16 -04:00
|
|
|
"remote": {
|
|
|
|
|
"cluster_one": {
|
|
|
|
|
"seeds": [
|
|
|
|
|
"127.0.0.1:9300"
|
|
|
|
|
]
|
|
|
|
|
},
|
|
|
|
|
"cluster_two": {
|
|
|
|
|
"seeds": [
|
|
|
|
|
"127.0.0.1:9301"
|
|
|
|
|
]
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
},
|
|
|
|
|
"cluster_three": {
|
|
|
|
|
"seeds": [
|
|
|
|
|
"127.0.0.1:9302"
|
|
|
|
|
]
|
2017-04-04 07:34:16 -04:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
--------------------------------
|
|
|
|
|
// CONSOLE
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
// TEST[setup:host]
|
|
|
|
|
// TEST[s/127.0.0.1:9300/\${transport_host}/]
|
|
|
|
|
|
|
|
|
|
To search the `twitter` index on remote cluster `cluster_one` the index name
|
|
|
|
|
must be prefixed with the cluster alias separated by a `:` character:
|
2017-01-05 10:10:34 -05:00
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
--------------------------------------------------
|
2017-12-14 11:47:53 -05:00
|
|
|
GET /cluster_one:twitter/_search
|
2017-01-05 10:10:34 -05:00
|
|
|
{
|
2017-06-16 21:14:34 -04:00
|
|
|
"query": {
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"match": {
|
|
|
|
|
"user": "kimchy"
|
|
|
|
|
}
|
2017-06-16 21:14:34 -04:00
|
|
|
}
|
2017-01-05 10:10:34 -05:00
|
|
|
}
|
|
|
|
|
--------------------------------------------------
|
2017-05-04 21:01:14 -04:00
|
|
|
// CONSOLE
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
// TEST[continued]
|
|
|
|
|
// TEST[setup:twitter]
|
|
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
--------------------------------------------------
|
|
|
|
|
{
|
|
|
|
|
"took": 150,
|
|
|
|
|
"timed_out": false,
|
|
|
|
|
"_shards": {
|
|
|
|
|
"total": 1,
|
|
|
|
|
"successful": 1,
|
|
|
|
|
"failed": 0,
|
|
|
|
|
"skipped": 0
|
|
|
|
|
},
|
|
|
|
|
"_clusters": {
|
|
|
|
|
"total": 1,
|
|
|
|
|
"successful": 1,
|
|
|
|
|
"skipped": 0
|
|
|
|
|
},
|
|
|
|
|
"hits": {
|
2018-12-05 13:49:06 -05:00
|
|
|
"total" : {
|
|
|
|
|
"value": 1,
|
|
|
|
|
"relation": "eq"
|
|
|
|
|
},
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"max_score": 1,
|
|
|
|
|
"hits": [
|
|
|
|
|
{
|
|
|
|
|
"_index": "cluster_one:twitter",
|
2017-12-14 11:47:53 -05:00
|
|
|
"_type": "_doc",
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"_id": "0",
|
|
|
|
|
"_score": 1,
|
|
|
|
|
"_source": {
|
|
|
|
|
"user": "kimchy",
|
|
|
|
|
"date": "2009-11-15T14:12:12",
|
|
|
|
|
"message": "trying out Elasticsearch",
|
|
|
|
|
"likes": 0
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
--------------------------------------------------
|
|
|
|
|
// TESTRESPONSE[s/"took": 150/"took": "$body.took"/]
|
|
|
|
|
// TESTRESPONSE[s/"max_score": 1/"max_score": "$body.hits.max_score"/]
|
|
|
|
|
// TESTRESPONSE[s/"_score": 1/"_score": "$body.hits.hits.0._score"/]
|
|
|
|
|
|
2017-01-05 10:10:34 -05:00
|
|
|
|
2018-04-19 21:02:01 -04:00
|
|
|
Indices can also be searched with the same name on different clusters:
|
2017-01-05 10:10:34 -05:00
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
--------------------------------------------------
|
2017-12-14 11:47:53 -05:00
|
|
|
GET /cluster_one:twitter,twitter/_search
|
2017-01-05 10:10:34 -05:00
|
|
|
{
|
2017-06-16 21:14:34 -04:00
|
|
|
"query": {
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"match": {
|
|
|
|
|
"user": "kimchy"
|
|
|
|
|
}
|
2017-06-16 21:14:34 -04:00
|
|
|
}
|
2017-01-05 10:10:34 -05:00
|
|
|
}
|
|
|
|
|
--------------------------------------------------
|
2017-05-04 21:01:14 -04:00
|
|
|
// CONSOLE
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
// TEST[continued]
|
2017-01-05 10:10:34 -05:00
|
|
|
|
|
|
|
|
Search results are disambiguated the same way as the indices are disambiguated in the request. Even if index names are
|
2017-01-11 08:52:41 -05:00
|
|
|
identical these indices will be treated as different indices when results are merged. All results retrieved from a
|
|
|
|
|
remote index
|
2017-01-09 12:22:11 -05:00
|
|
|
will be prefixed with their remote cluster name:
|
2017-01-05 10:10:34 -05:00
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
--------------------------------------------------
|
2017-05-04 21:01:14 -04:00
|
|
|
{
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"took": 150,
|
|
|
|
|
"timed_out": false,
|
|
|
|
|
"_shards": {
|
|
|
|
|
"total": 2,
|
|
|
|
|
"successful": 2,
|
|
|
|
|
"failed": 0,
|
|
|
|
|
"skipped": 0
|
2017-01-05 10:10:34 -05:00
|
|
|
},
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"_clusters": {
|
|
|
|
|
"total": 2,
|
|
|
|
|
"successful": 2,
|
|
|
|
|
"skipped": 0
|
|
|
|
|
},
|
|
|
|
|
"hits": {
|
2018-12-05 13:49:06 -05:00
|
|
|
"total" : {
|
|
|
|
|
"value": 2,
|
|
|
|
|
"relation": "eq"
|
|
|
|
|
},
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"max_score": 1,
|
|
|
|
|
"hits": [
|
2017-01-05 10:10:34 -05:00
|
|
|
{
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"_index": "cluster_one:twitter",
|
2017-12-14 11:47:53 -05:00
|
|
|
"_type": "_doc",
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"_id": "0",
|
|
|
|
|
"_score": 1,
|
|
|
|
|
"_source": {
|
|
|
|
|
"user": "kimchy",
|
|
|
|
|
"date": "2009-11-15T14:12:12",
|
|
|
|
|
"message": "trying out Elasticsearch",
|
|
|
|
|
"likes": 0
|
2017-01-05 10:10:34 -05:00
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
{
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"_index": "twitter",
|
2017-12-14 11:47:53 -05:00
|
|
|
"_type": "_doc",
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"_id": "0",
|
|
|
|
|
"_score": 2,
|
|
|
|
|
"_source": {
|
|
|
|
|
"user": "kimchy",
|
|
|
|
|
"date": "2009-11-15T14:12:12",
|
|
|
|
|
"message": "trying out Elasticsearch",
|
|
|
|
|
"likes": 0
|
2017-01-05 10:10:34 -05:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
--------------------------------------------------
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
// TESTRESPONSE[s/"took": 150/"took": "$body.took"/]
|
|
|
|
|
// TESTRESPONSE[s/"max_score": 1/"max_score": "$body.hits.max_score"/]
|
|
|
|
|
// TESTRESPONSE[s/"_score": 1/"_score": "$body.hits.hits.0._score"/]
|
|
|
|
|
// TESTRESPONSE[s/"_score": 2/"_score": "$body.hits.hits.1._score"/]
|
|
|
|
|
|
|
|
|
|
[float]
|
|
|
|
|
=== Skipping disconnected clusters
|
|
|
|
|
|
|
|
|
|
By default all remote clusters that are searched via Cross Cluster Search need to be available when
|
|
|
|
|
the search request is executed, otherwise the whole request fails and no search results are returned
|
|
|
|
|
despite some of the clusters are available. Remote clusters can be made optional through the
|
|
|
|
|
boolean `skip_unavailable` setting, set to `false` by default.
|
|
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
--------------------------------
|
|
|
|
|
PUT _cluster/settings
|
|
|
|
|
{
|
|
|
|
|
"persistent": {
|
2018-09-05 20:43:44 -04:00
|
|
|
"cluster.remote.cluster_two.skip_unavailable": true <1>
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
--------------------------------
|
|
|
|
|
// CONSOLE
|
|
|
|
|
// TEST[continued]
|
|
|
|
|
<1> `cluster_two` is made optional
|
|
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
--------------------------------------------------
|
2017-12-14 11:47:53 -05:00
|
|
|
GET /cluster_one:twitter,cluster_two:twitter,twitter/_search <1>
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
{
|
|
|
|
|
"query": {
|
|
|
|
|
"match": {
|
|
|
|
|
"user": "kimchy"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
--------------------------------------------------
|
|
|
|
|
// CONSOLE
|
|
|
|
|
// TEST[continued]
|
|
|
|
|
<1> Search against the `twitter` index in `cluster_one`, `cluster_two` and also locally
|
|
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
--------------------------------------------------
|
|
|
|
|
{
|
|
|
|
|
"took": 150,
|
|
|
|
|
"timed_out": false,
|
|
|
|
|
"_shards": {
|
|
|
|
|
"total": 2,
|
|
|
|
|
"successful": 2,
|
|
|
|
|
"failed": 0,
|
|
|
|
|
"skipped": 0
|
|
|
|
|
},
|
|
|
|
|
"_clusters": { <1>
|
|
|
|
|
"total": 3,
|
|
|
|
|
"successful": 2,
|
|
|
|
|
"skipped": 1
|
|
|
|
|
},
|
|
|
|
|
"hits": {
|
2018-12-05 13:49:06 -05:00
|
|
|
"total" : {
|
|
|
|
|
"value": 2,
|
|
|
|
|
"relation": "eq"
|
|
|
|
|
},
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"max_score": 1,
|
|
|
|
|
"hits": [
|
|
|
|
|
{
|
|
|
|
|
"_index": "cluster_one:twitter",
|
2017-12-14 11:47:53 -05:00
|
|
|
"_type": "_doc",
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"_id": "0",
|
|
|
|
|
"_score": 1,
|
|
|
|
|
"_source": {
|
|
|
|
|
"user": "kimchy",
|
|
|
|
|
"date": "2009-11-15T14:12:12",
|
|
|
|
|
"message": "trying out Elasticsearch",
|
|
|
|
|
"likes": 0
|
|
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"_index": "twitter",
|
2017-12-14 11:47:53 -05:00
|
|
|
"_type": "_doc",
|
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.
This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.
Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.
The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:
"_clusters" : {
"total" : 3,
"successful" : 2,
"skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.
The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 05:41:47 -05:00
|
|
|
"_id": "0",
|
|
|
|
|
"_score": 2,
|
|
|
|
|
"_source": {
|
|
|
|
|
"user": "kimchy",
|
|
|
|
|
"date": "2009-11-15T14:12:12",
|
|
|
|
|
"message": "trying out Elasticsearch",
|
|
|
|
|
"likes": 0
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
--------------------------------------------------
|
|
|
|
|
// TESTRESPONSE[s/"took": 150/"took": "$body.took"/]
|
|
|
|
|
// TESTRESPONSE[s/"max_score": 1/"max_score": "$body.hits.max_score"/]
|
|
|
|
|
// TESTRESPONSE[s/"_score": 1/"_score": "$body.hits.hits.0._score"/]
|
|
|
|
|
// TESTRESPONSE[s/"_score": 2/"_score": "$body.hits.hits.1._score"/]
|
|
|
|
|
<1> The `clusters` section indicates that one cluster was unavailable and got skipped
|
