mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-05 20:48:22 +00:00
4a4e3d70d5
This commit changes the default out-of-the-box configuration for the number of shards from five to one. We think this will help address a common problem of oversharding. For users with time-based indices that need a different default, this can be managed with index templates. For users with non-time-based indices that find they need to re-shard with the split API in place they no longer need to resort only to reindexing. Since this has the impact of changing the default number of shards used in REST tests, we want to ensure that we still have coverage for issues that could arise from multiple shards. As such, we randomize (rarely) the default number of shards in REST tests to two. This is managed via a global index template. However, some tests check the templates that are in the cluster state during the test. Since this template is randomly there, we need a way for tests to skip adding the template used to set the number of shards to two. For this we add the default_shards feature skip. To avoid having to write our docs in a complicated way because sometimes they might be behind one shard, and sometimes they might be behind two shards we apply the default_shards feature skip to all docs tests. That is, these tests will always run with the default number of shards (one).
224 lines
4.9 KiB
Plaintext
224 lines
4.9 KiB
Plaintext
[[mixing-exact-search-with-stemming]]
|
|
=== Mixing exact search with stemming
|
|
|
|
When building a search application, stemming is often a must as it is desirable
|
|
for a query on `skiing` to match documents that contain `ski` or `skis`. But
|
|
what if a user wants to search for `skiing` specifically? The typical way to do
|
|
this would be to use a <<multi-fields,multi-field>> in order to have the same
|
|
content indexed in two different ways:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT index
|
|
{
|
|
"settings": {
|
|
"analysis": {
|
|
"analyzer": {
|
|
"english_exact": {
|
|
"tokenizer": "standard",
|
|
"filter": [
|
|
"lowercase"
|
|
]
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"mappings": {
|
|
"_doc": {
|
|
"properties": {
|
|
"body": {
|
|
"type": "text",
|
|
"analyzer": "english",
|
|
"fields": {
|
|
"exact": {
|
|
"type": "text",
|
|
"analyzer": "english_exact"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
PUT index/_doc/1
|
|
{
|
|
"body": "Ski resort"
|
|
}
|
|
|
|
PUT index/_doc/2
|
|
{
|
|
"body": "A pair of skis"
|
|
}
|
|
|
|
POST index/_refresh
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
|
|
With such a setup, searching for `ski` on `body` would return both documents:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET index/_search
|
|
{
|
|
"query": {
|
|
"simple_query_string": {
|
|
"fields": [ "body" ],
|
|
"query": "ski"
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[continued]
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"took": 2,
|
|
"timed_out": false,
|
|
"_shards": {
|
|
"total": 1,
|
|
"successful": 1,
|
|
"skipped" : 0,
|
|
"failed": 0
|
|
},
|
|
"hits": {
|
|
"total": 2,
|
|
"max_score": 0.18232156,
|
|
"hits": [
|
|
{
|
|
"_index": "index",
|
|
"_type": "_doc",
|
|
"_id": "1",
|
|
"_score": 0.18232156,
|
|
"_source": {
|
|
"body": "Ski resort"
|
|
}
|
|
},
|
|
{
|
|
"_index": "index",
|
|
"_type": "_doc",
|
|
"_id": "2",
|
|
"_score": 0.18232156,
|
|
"_source": {
|
|
"body": "A pair of skis"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE[s/"took": 2,/"took": "$body.took",/]
|
|
|
|
On the other hand, searching for `ski` on `body.exact` would only return
|
|
document `1` since the analysis chain of `body.exact` does not perform
|
|
stemming.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET index/_search
|
|
{
|
|
"query": {
|
|
"simple_query_string": {
|
|
"fields": [ "body.exact" ],
|
|
"query": "ski"
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[continued]
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"took": 1,
|
|
"timed_out": false,
|
|
"_shards": {
|
|
"total": 1,
|
|
"successful": 1,
|
|
"skipped" : 0,
|
|
"failed": 0
|
|
},
|
|
"hits": {
|
|
"total": 1,
|
|
"max_score": 0.80259144,
|
|
"hits": [
|
|
{
|
|
"_index": "index",
|
|
"_type": "_doc",
|
|
"_id": "1",
|
|
"_score": 0.80259144,
|
|
"_source": {
|
|
"body": "Ski resort"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE[s/"took": 1,/"took": "$body.took",/]
|
|
|
|
This is not something that is easy to expose to end users, as we would need to
|
|
have a way to figure out whether they are looking for an exact match or not and
|
|
redirect to the appropriate field accordingly. Also what to do if only parts of
|
|
the query need to be matched exactly while other parts should still take
|
|
stemming into account?
|
|
|
|
Fortunately, the `query_string` and `simple_query_string` queries have a feature
|
|
that solve this exact problem: `quote_field_suffix`. This tell Elasticsearch
|
|
that the words that appear in between quotes are to be redirected to a different
|
|
field, see below:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET index/_search
|
|
{
|
|
"query": {
|
|
"simple_query_string": {
|
|
"fields": [ "body" ],
|
|
"quote_field_suffix": ".exact",
|
|
"query": "\"ski\""
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[continued]
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"took": 2,
|
|
"timed_out": false,
|
|
"_shards": {
|
|
"total": 1,
|
|
"successful": 1,
|
|
"skipped" : 0,
|
|
"failed": 0
|
|
},
|
|
"hits": {
|
|
"total": 1,
|
|
"max_score": 0.80259144,
|
|
"hits": [
|
|
{
|
|
"_index": "index",
|
|
"_type": "_doc",
|
|
"_id": "1",
|
|
"_score": 0.80259144,
|
|
"_source": {
|
|
"body": "Ski resort"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE[s/"took": 2,/"took": "$body.took",/]
|
|
|
|
In the above case, since `ski` was in-between quotes, it was searched on the
|
|
`body.exact` field due to the `quote_field_suffix` parameter, so only document
|
|
`1` matched. This allows users to mix exact search with stemmed search as they
|
|
like.
|