Commit Graph

303 Commits

Author SHA1 Message Date
Jim Ferenczi 439b2a96e5 Add an index setting to limit the maximum number of slices allowed in a scroll request (default to 1024). 2016-06-10 09:43:32 +02:00
Britta Weber 053a615686 [TEST] wait for yellow before query execution
We can remove this once https://github.com/elastic/elasticsearch/pull/18759
is in.
2016-06-09 15:11:48 +02:00
Jim Ferenczi b9030bf6fe Add the ability to partition a scroll in multiple slices.
API:

```
curl -XGET 'localhost:9200/twitter/tweet/_search?scroll=1m' -d '{
    "slice": {
        "field": "_uid", <1>
        "id": 0, <2>
        "max": 10 <3>
    },
    "query": {
        "match" : {
            "title" : "elasticsearch"
        }
    }
}
```

<1> (optional) The field name used to do the slicing (_uid by default)
<2> The id of the slice

By default the splitting is done on the shards first and then locally on each shard using the _uid field
with the following formula:
`slice(doc) = floorMod(hashCode(doc._uid), max)`
For instance if the number of shards is equal to 2 and the user requested 4 slices then the slices 0 and 2 are assigned
to the first shard and the slices 1 and 3 are assigned to the second shard.

Each scroll is independent and can be processed in parallel like any scroll request.

Closes #13494
2016-06-07 16:21:53 +02:00
Isabel Drost-Fromm ea3320e171 Merge pull request #18424 from MaineC/docs/add_console_to_highlighting
Docs/add console to highlighting
2016-05-24 12:14:36 +02:00
Isabel Drost-Fromm 4057682d6f Add CONSOLE to inner hits examples. 2016-05-19 11:01:36 +02:00
Isabel Drost-Fromm 10874fbdf9 Add CONSOLE to scroll docs
Relates to #18160
2016-05-19 10:30:58 +02:00
Isabel Drost-Fromm 27e6908c8d Add indent 2016-05-19 09:33:29 +02:00
Isabel Drost-Fromm bf471c6950 Merge branch 'master' into docs/add_console_to_search_request_options 2016-05-19 09:32:46 +02:00
Isabel Drost-Fromm 9d2a3c0600 Merge pull request #18442 from MaineC/docs/add_console_to_fromsize
Add CONSOLE to from/size docs
2016-05-18 18:38:41 +02:00
Isabel Drost-Fromm 394a60f3fd Switch to more match query for better illustration 2016-05-18 15:52:08 +02:00
Isabel Drost-Fromm 1f0f6132be Merge branch 'master' into docs/add_console_to_highlighting 2016-05-18 15:45:20 +02:00
Isabel Drost-Fromm a5268cd40d Add CONSOLE to version docs 2016-05-18 15:32:48 +02:00
Isabel Drost-Fromm c20a669c2d Add CONSOLE to source filtering docs 2016-05-18 15:20:21 +02:00
Isabel Drost-Fromm a849cc97ea Add CONSOLE to script-fields docs 2016-05-18 14:38:54 +02:00
Isabel Drost-Fromm 0032d4760e Add CONSOLE to preference docs 2016-05-18 14:34:22 +02:00
Isabel Drost-Fromm eca53d909c Merge branch 'master' into docs/add_console_to_search_request_options 2016-05-18 14:31:48 +02:00
Isabel Drost-Fromm a3425b4bf8 Add CONSOLE to post-filter 2016-05-18 14:31:04 +02:00
Isabel Drost-Fromm f22f3c7df5 Add CONSOLE to several trivial search request docs.
Relates to #18160

Touches explain, fielddata-fields, fields, index-boost, min-score,
named-queries-and-filters, query
2016-05-18 13:15:19 +02:00
Nik Everett 148f9af585 Convert search-after tests to // CONSOLE
Relates to #18160
2016-05-18 06:37:13 -04:00
Isabel Drost-Fromm 947daf68d1 Add CONSOLE to from/size docs
Relates to #18160
2016-05-18 11:54:41 +02:00
Isabel Drost-Fromm 8016a2f951 Switch to keyword/text type, add refresh parameter 2016-05-18 11:08:27 +02:00
Isabel Drost-Fromm 5753bcca83 Add Console to highlighting docs
... in order to execute the snippets through rest tests.

Relates to #18160
2016-05-17 21:00:15 +02:00
Isabel Drost-Fromm 48ea9137da Add CONSOLE annotation to sort documentation
This adds CONSOLE to sort docs in order to automatically execute the doc
snippets. Fixes a few minor types along the way.

Relates to #18160
2016-05-17 15:14:37 +02:00
Jason Tedor 4ae5c0d93d Add link to field data docs from field data fields
This commit adds a link to the field data docs from the field data
fields docs for completeness.

Closes #17672
2016-05-07 11:09:17 -04:00
Britta Weber ddebbb9536 add string to documentation 2016-05-06 16:50:34 +02:00
Britta Weber d3c5f865be Exclude all but string fields from highlighting if wildcards are used in fieldname
We should prevent highlighting if a field is anything but a text or keyword field.
However, someone might implement a custom field type that has text and still want to
highlight on that. We cannot know in advance if the highlighter will be able to
highlight such a field and so we do the following:
If the field is only highlighted because the field matches a wildcard we assume
it was a mistake and do not process it.
If the field was explicitly given we assume that whoever issued the query knew
what they were doing and try to highlight anyway.

closes #17537
2016-05-06 13:41:16 +02:00
Martijn van Groningen 6c3beaa2eb Drop top level inner hits in favour of inner hits defined in the query dsl.
Fix a limitation that prevent from hierarchical inner hits be defined in query dsl.

Removed the nested_path, parent_child_type and query options from inner hits dsl. These options are only set by ES
upon parsing the has_child, has_parent and nested queries are using their respective query builders.

These options are still used internally, when these options are set a new private copy is created based on the
provided InnerHitBuilder and configuring either nested_path or parent_child_type and the inner query of the query builder
being used.

Closes #11118
2016-04-29 11:17:24 +02:00
Jeff ba34faa1ef Call out where we are making a setting change.
IMHO the original text here was incomplete. Adding the simple words 'in the index mapping' makes this sentence more clear. Perhaps a be more clear to make this a link.
2016-04-05 13:51:31 -06:00
Martijn van Groningen 7e2696c570 Refactored inner hits parsing and intoduced InnerHitBuilder
Both top level and inline inner hits are now covered by InnerHitBuilder.
Although there are differences between top level and inline inner hits,
they now make use of the same builder logic.

The parsing of top level inner hits slightly changed to be more readable.
Before the nested path or parent/child type had to be specified as encapsuting
json object, now these settings are simple fields. Before this was required
to allow streaming parsing of inner hits without missing contextual information.

Once some issues are fixed with inline inner hits (around multi level hierachy of inner hits),
top level inner hits will be deprecated and removed in the next major version.
2016-03-30 15:15:56 +02:00
Isabel Drost-Fromm 801d178ade Remove mention of reverse in docs and add to migration doc 2016-03-24 12:04:31 +01:00
Clinton Gormley 380ecd7604 Merge pull request #16777 from radar/patch-1
Add example for require_field_match to highlighting docs
2016-02-28 21:33:47 +01:00
Dongjoon Hyun 21ea552070 Fix typos in docs. 2016-02-09 02:07:32 -08:00
Jim Ferenczi aea7660e37 Add search_after parameter in the Search API.
The search_after parameter provides a way to efficiently paginate from one page to the next. This parameter accepts an array of sort values, those values are then used by the searcher to sort the top hits from the first document that is greater to the sort values.
This parameter must be used in conjunction with the sort parameter, it must contain exactly the same number of values than the number of fields to sort on.

NOTE: A field with one unique value per document should be used as the last element of the sort specification. Otherwise the sort order for documents that have the same sort values would be undefined. The recommended way is to use the field `_uuid` which is certain to contain one unique value for each document.

Fixes #8192
2016-01-27 09:42:58 +01:00
Clinton Gormley 1cf48b08fb Merge pull request #15534 from jmluy/patch-2
Fix minor typos in inner hits document
2016-01-10 21:52:31 +01:00
Simon Willnauer 2dbad1d65a Merge pull request #15841 from simonw/patch-1
Fixed broken link on from-size documentation page
2016-01-08 23:51:18 +01:00
Jason Bryan 9a1133ca50 Fix typo in scroll.asciidoc
Fix scroll request with sort.

Closes #15493
2015-12-16 20:31:46 -05:00
Clinton Gormley 4597a22ace Merge pull request #15473 from jmluy/patch-1
Update sample in sort for consistency
2015-12-16 12:53:09 +01:00
Clinton Gormley 1685126bb6 Merge pull request #15085 from kaneshin/docs/modify/post_filter
Remove a trailing comma from an example data of JSON
2015-11-30 08:05:10 +01:00
Shintaro Kaneko d7baeb1e7b Remove a trailing comma from an example data of JSON 2015-11-28 16:50:28 +00:00
Jamie McCarthy ce20337d03 Fix doc of nested_path sort option 2015-11-19 12:22:00 -05:00
Adrien Grand c9120c5c2a Docs: Add a warning about fields vs. source filtering.
Close #14470
2015-11-03 11:18:00 +01:00
javanna 49f5757ae2 Remove support for multiple highlighter names
The only way to refer to the plain highlighter is now `plain`, the only way to refer to the fast vector highlighter is `fvh` and the only way to refer to the postings highlighter is `postings`. The name variants like `highlighter`, `postings-highlighter` and `fast-vector-highlighter` have been removed.
2015-10-28 10:50:29 +01:00
Clinton Gormley dc018cf622 Updated docs for 3.0.0-beta 2015-10-07 13:27:46 +02:00
Thomas Cucchietti ecc2985b84 Update inner-hits.asciidoc
Fix a glitch in inner_hits feature documentation (though I'm not absolutely sure of the final version)
2015-09-30 11:07:51 +02:00
Adrien Grand 86f1b07df0 Docs: Remove docs for the `filtered`, `and`, `or` and `(f)query` queries. 2015-09-11 11:00:54 +02:00
Nik Everett e4981968ad [search] Limit the size of the result window
Requesting a million hits, or page 100,000 is always a bad idea, but users
may not be aware of this. This adds a per-index limit on the maximum size +
from that can be requested which defaults to 10,000.

This should not interfere with deep-scrolling.

Closes #9311
2015-09-10 15:38:29 -04:00
Martijn van Groningen 2eadc6d595 nested sorting: If sorting by nested field then the `nested_path` should always be specified.
Closes #13420
2015-09-10 12:21:12 +02:00
Martijn van Groningen 11c87106ce docs: inner hits is no longer experimental 2015-09-07 16:58:46 +02:00
Adrien Grand 0c26e7cd83 Remove the scan and count search types.
These search types have been deprecated in 2.1 and 2.0 respectively, and will
be removed in 3.0.
2015-09-07 15:18:45 +02:00
Britta Weber 2b27bc11b6 [doc] remove comment about function_score faster than script sort. It is not so. 2015-09-03 12:33:00 +02:00
Michael McCandless 1c85b68674 Don't document expert segment merge settings 2015-08-29 17:21:46 -04:00
Adrien Grand 7b878b5b5c Docs: Document the `_doc` sort order. 2015-08-24 15:39:50 +02:00
Adrien Grand 6fa258b8fa Deprecate the `scan` search type.
This commit deprecates the `scan` search type in favour of regular scroll
requests sorted by `_doc`.

Related to #12983
2015-08-20 12:47:23 +02:00
Adrien Grand 551e92ec71 Fix documentation: scrolls are not closed automatically.
The documentation states that scrolls are automatically closed when all
documents are consumed, but this is not the case. I first tried to fix
the code to close scrolls automatically but this made REST tests fail
because clearing a scroll that is already closed returned a 4xx error
instead of a 2xx code, so this has probably been this way for a very long
time.
2015-08-20 09:20:40 +02:00
Clinton Gormley c6c3a40cb6 Docs: Updated annotations for 2.0.0-beta1 2015-08-14 10:51:09 +02:00
Clinton Gormley ac2b8951c6 Docs: Mapping docs completely rewritten for 2.0 2015-08-06 17:24:51 +02:00
Lee Hinman a8391fcae9 Add _replica and _replica_first as search preference.
Just like specifying `?preference=_primary`, this adds the ability to
specify `?preference=_replica` or `?preference=_replica_first` on
requests that support it.

Resolves #12222
2015-07-16 09:25:23 -06:00
markharwood 52fb3c3a09 Docs fix- added performance note about plain highlighter
Closes #11442
2015-07-15 14:28:28 +01:00
Clinton Gormley 2b512f1f29 Docs: Use "js" instead of "json" and "sh" instead of "shell" for source highlighting 2015-07-14 18:14:09 +02:00
Clinton Gormley d9dfa9a24c Merge pull request #12183 from erichard/patch-1
Fix documentation typo
2015-07-10 19:15:56 +02:00
Adrien Grand d7af88631f Merge pull request #11538 from Collaborne/docs-sort-sr-typo
Fix a typo in the documentation: six_hun -> "narrower"
2015-07-08 19:22:03 +02:00
Ruslan Boyarskiy e5e422b880 Docs: Update post-filter.asciidoc
Removing useless comma

Closes #11912
2015-07-01 09:32:39 +02:00
Clinton Gormley f19a748d3c Docs: Move field highlight order to the highlight page 2015-06-26 17:36:48 +02:00
Nirmal Chidambaram 72a9d34eb8 5925 - Allow node specification in preference
-Allow node selector api's with new preference
ONLY_NODES ( selector apis like https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster.html)

-Update documentation
2015-06-16 11:49:12 -05:00
Andreas Kohn 1c0ad8c724 Fix a typo in the documentation: six_hun -> "narrower"
This was introduced in https://github.com/elastic/elasticsearch.github.com/commit/defaf4f0, probably
as a search-and-replace mistake.
2015-06-08 18:07:52 +02:00
Colin Goodheart-Smithe 35a58d874e Scripting: Unify script and template requests across codebase
This change unifies the way scripts and templates are specified for all instances in the codebase. It builds on the Script class added previously and adds request building and parsing support as well as the ability to transfer script objects between nodes. It also adds a Template class which aims to provide the same functionality for template APIs

Closes #11091
2015-05-29 16:52:04 +01:00
javanna a843008b17 Highlighting: require_field_match set to true by default
The default `false` for `require_field_match` is a bit odd and confusing for users, given that field names get ignored by default and every field gets highlighted if it contains terms extracted out of the query, regardless of which fields were queries. Changed the default to `true`, it can always be changed per request.

Closes #10627
Closes #11067
2015-05-15 21:38:45 +02:00
javanna 46c521f7ec Highlighting: nuke XPostingsHighlighter
Our own fork of the lucene PostingsHighlighter is not easy to maintain and doesn't give us any added value at this point. In particular, it was introduced to support the require_field_match option and discrete per value highlighting, used in case one wants to highlight the whole content of a field, but get back one snippet per value. These two features won't
 make it into lucene as they slow things down and shouldn't have been supported from day one on our end probably.

One other customization we had was support for a wider range of queries via custom rewrite etc. (yet another way to slow
 things down), which got added to lucene and works much much better than what we used to do (instead of or rewrite, term
s are pulled out of the automata for multi term queries).

Removing our fork means the following in terms of features:
- dropped support for require_field_match: the postings highlighter will only highlight fields that were queried
- some custom es queries won't be supported anymore, meaning they won't be highlighted. The only one I found up until now is the phrase_prefix. Postings highlighter rewrites against an empty reader to avoid slow operations (like the ones that we were performing with the fork that we are removing here), thus the prefix will not be expanded to any term. What the postings highlighter does instead is pulling the automata out of multi term queries, but this is not supported at the moment with our MultiPhrasePrefixQuery.

Closes #10625
Closes #11077
2015-05-15 20:41:33 +02:00
Jack Conradson a5c0ac0d67 Scripting: Add Multi-Valued Field Methods to Expressions
Add methods to operate on multi-valued fields in the expressions language.
Note that users will still not be able to access individual values
within a multi-valued field.

The following methods will be included:

* min
* max
* avg
* median
* count
* sum

Additionally, changes have been made to MultiValueMode to support the
new median method.

closes #11105
2015-05-14 08:27:24 -07:00
Andrew Selden c953e99324 Merge pull request #10864 from aleph-zero/issues/9606
Remove (dfs_)query_and_fetch from the REST API
2015-05-07 12:51:28 -07:00
Pascal Borreli af6d890ad5 Docs: Fixed typos
Closes #10973
2015-05-05 10:38:05 +02:00
aleph-zero 89542facb3 Remove (dfs_)query_and_fetch from the REST API
Remove the ability to specify search type ‘query_and_fetch’ and
‘df_query_and_fetch’ from the REST API.

- Adds REST tests
- Updates REST API spec to remove ‘query_and_fetch’ and
‘df_query_and_fetch’ as options
- Removes documentation for these options

Closes #9606
2015-04-28 15:27:59 -07:00
Mehdi Mollaverdi dce920b75f Docs: The name of scroll ID attribute in the response is "_scroll_id" rather than "scroll_id"
Closes #10691
2015-04-25 19:32:32 +02:00
Clinton Gormley a60571c597 Docs: Removed some unused callout from the scroll docs 2015-04-22 12:49:06 +02:00
Jun Ohtani 0955c127c0 Rest: Add json in request body to scroll, clear scroll, and analyze API
Change analyze.asciidoc and scroll.asciidoc
Add json support to Analyze and Scroll, and clear scrollAPI
Add rest-api-spec/test

Closes #5866
2015-04-22 17:53:20 +09:00
Honza Král e929c1560d [DOCS] Be explicit about scan doing no scoring 2015-04-20 18:05:45 +02:00
Adrien Grand a608db122d Search: Remove the `count` search type.
This commit brings the benefits of the `count` search type to search requests
that have a `size` of 0:
 - a single round-trip to shards (no fetch phase)
 - ability to use the query cache

Since `count` now provides no benefits over `query_then_fetch`, it has been
deprecated.

Close #7630
2015-03-31 11:31:49 +02:00
Boaz Leskes 4970e3e225 Revert "Rest: Add json in request body to scroll, clear scroll, and analyze API"
This reverts commit 16083d454c.
2015-03-23 12:57:19 +01:00
Jun Ohtani 16083d454c Rest: Add json in request body to scroll, clear scroll, and analyze API
Add json support to scroll, clear scroll, and analyze

Closes #5866
2015-03-23 15:35:38 +09:00
Martijn van Groningen 4393939f5e inner_hits: Nested parent field should be resolved based on the parent inner hit definition, instead of the nested parent field in the mapping.
The behaviour is better in the case someone has multiple levels of nested object fields defined in the mapping and like to define a single inner_hits definition that is two or more levels deep.

If someone wants inner hits on a nested field that is 2 levels deep the following would need to be defined:

```
{
  ...
  "inner_hits" : {
     "path" : {
        "level1" : {
            "inner_hits" : {
               "path" : {
                  "level2" : {
                     "query" : { .... }
                  }
               }
            }
        }
     }
  }
}
```

With this change the above can be defined as:

```
{
  ...
  "inner_hits" : {
     "path" : {
        "level1.level2" : {
            "query" : { .... }
        }
     }
  }
}
```

Closes #9251
2015-03-16 16:31:03 -07:00
Clinton Gormley c223ed0db4 Update search-type.asciidoc
Changed search_type docs to reflect that the `(dfs_)query_and_fetch` modes are an internal optimization and should not be specified explicitly by the user.

Relates to #9606
2015-03-02 10:55:22 +01:00
Geoff Bourne 0e09c02c56 Spelling out the sort order options
Closes #9768
2015-03-01 21:05:52 +01:00
Adrien Grand 95f46f1212 Docs: Use the new experimental annotation.
We now have a very useful annotation to mark features or parameters as
experimental. Let's use it! This commit replaces some custom text warnings with
this annotation and adds this annotation to some existing features/parameters:
 - inner_hits (unreleased yet)
 - terminate_after (released in 1.4)
 - per-bucket doc count errors in the terms agg (released in 1.4)

I also tagged with this annotation settings which should either be not needed
(like the ability to evict entries from the filter cache based on time) or that
are too deep into the way that Elasticsearch works like the Directory
implementation or merge settings.

Close #9563
2015-02-05 15:29:45 +01:00
Martijn van Groningen 8e0292b1aa docs: fix inner hits snippet 2015-01-19 18:56:45 +01:00
David Pilato fc7a0d3a4a [Docs] fix three to four 2015-01-12 12:13:23 +01:00
Martijn van Groningen d8054ec299 inner_hits: Added another more compact syntax for inner hits.
Closes #8770
2014-12-24 17:41:35 +01:00
Adam Menges 3a3030e217 Docs: Fix the wording for inner hits a bit
Closes #8747
2014-12-09 13:36:26 +01:00
Martijn van Groningen d7e224da04 Added `inner_hits` feature that allows to include nested hits.
Inner hits allows to embed nested inner objects, children documents or the parent document that contributed to the matching of the returned search hit as inner hits, which would otherwise be hidden.

Closes #8153
Closes #3022
Closes #3152
2014-12-02 12:01:01 +01:00
Michael McCandless 85065f9c8e Core: cutover to Lucene's query rescorer
This is functionally equivalent to before, so there should be no
user-visible impact, except I added a NOTE in the docs warning about
the interaction of pagination and rescoring.

Closes #6232

Closes #7707
2014-10-18 05:25:50 -04:00
Clinton Gormley 6a180d1803 Docs: Update highlighting.asciidoc
Added note about how to highlight on the `_all` field

Closes #7991
2014-10-15 13:45:56 +02:00
Clinton Gormley cb00d4a542 Docs: Removed all the added/deprecated tags from 1.x 2014-09-26 21:04:42 +02:00
Adrien Grand 4bfad644b3 Aggregations: Forbid usage of aggregations in conjunction with search_type=SCAN.
Aggregations are collection-wide statistics, which is incompatible with the
collection mode of search_type=SCAN since it doesn't collect all matches on
calls to the search API.

Close #7429
2014-09-03 09:03:01 +02:00
Adrien Grand 203e80e650 Aggregations: Only return aggregations on the first page when scrolling.
Aggregations are collection-wide statistics so they would always be the same.
In order to save CPU/bandwidth, we can just return them on the first page.

Same as #1642 but for aggregations.
2014-09-03 09:03:01 +02:00
Adrien Grand 8e1d3d56b3 Docs: Replace added[1.4.0] with coming[1.4.0] since 1.4 is not released yet. 2014-08-29 11:57:22 +02:00
Adrien Grand ea96359d82 Facets: Removal from master.
Close #7337
2014-08-21 10:34:39 +02:00
Britta Weber 639692943f Docs: Document distance type and sort mode for many to many geo_points
closes #7280
2014-08-18 16:15:55 +02:00
Britta Weber d49ed93488 Docs: md -> asciidoc 2014-08-08 11:25:14 +02:00
Adrien Grand d9d5b35be9 Sort: Make `ignore_unmapped` work for cross-index queries.
Close #2255
2014-08-01 15:30:17 +02:00
Britta Weber d6a18ab2ba Docs: add 1.4.0 label to many to many geo distance sort 2014-08-01 12:30:08 +02:00
Kurt Hurtado 66560acebb Update fielddata-fields.asciidoc 2014-08-01 09:20:19 +02:00
Britta Weber fe86c8bc88 _geo_distance sort: allow many to many geo point distance
Add computation of disyance to many geo points. Example request:

```
{
  "sort": [
    {
      "_geo_distance": {
        "location": [
          {
            "lat":1.2,
            "lon":3
          },
          {
             "lat":1.2,
            "lon":3
          }
        ],
        "order": "desc",
        "unit": "km",
        "sort_mode": "max"
      }
    }
  ]
}
```

closes #3926
2014-07-31 17:33:45 +02:00
Clinton Gormley 36e1c7928c Rewrote post-filter.asciidoc
Closes #5166
2014-07-31 12:56:11 +02:00
Clinton Gormley 10b4177def Docs: Fixed path to search-shards 2014-07-26 15:05:53 +02:00
Clinton Gormley 0f943850a0 Update named-queries-and-filters.asciidoc 2014-07-23 17:28:49 +02:00
Clinton Gormley 6a7a77eada Docs: Add links to client helper classes for bulk/scroll/reindexing 2014-07-18 13:55:47 +02:00
Clinton Gormley b6baa4be4a Update preference.asciidoc
Clarify that `preference` is a query string parameter only
and provide an example.
2014-07-09 11:13:17 +02:00
Clinton Gormley feb81e228b Docs: Rewrote the scroll/scan docs
Closes #6774
2014-07-08 11:54:53 +02:00
Clinton Gormley 64a4acc49b Docs: Added IDs to the highlighters for linking 2014-06-22 16:46:42 +02:00
Luke Fender f9da5259bc [DOCS] Fixed typo in post-filter.asciidoc
Remove 'be' where it is not needed
2014-06-12 12:09:19 +02:00
Nik Everett 3573822b7e Highlight fields in request order
Because json objects are unordered this also adds an explicit order syntax
that looks like
    "highlight": {
        "fields": [
            {"title":{ /*params*/ }},
            {"text":{ /*params*/ }}
        ]
    }

This is not useful for any of the builtin highlighters but will be useful
in plugins.

Closes #4649
2014-05-22 16:44:14 +02:00
Clinton Gormley ff12585fea Improved wording in search-type.asciidoc
Closes #5951
2014-05-14 12:15:48 +02:00
Lee Hinman 57bee03193 [DOCS] Add /_search_shards documentation 2014-04-22 08:54:32 -06:00
Andrew O'Brien 48031b6236 Fixes typo in "Scan" search type documention 2014-04-07 16:01:37 -06:00
Hannes Korte c11293ad78 Fix some typos in documentation. 2014-03-31 13:48:17 +02:00
bleskes 5d832374dd Update Documentation Feature Flags [1.1.0] 2014-03-25 17:51:30 +01:00
Randy Stauner 1486188a3b [DOCS] Reword clear-scroll sentence 2014-03-17 12:08:49 +01:00
Simon Willnauer fbb8c0fafa [DOCS] Add `coming` tag to multiple rescores
Closes #5365
2014-03-10 09:27:44 +01:00
Luca Cavanna 4e6610a798 Fixed multi term queries support in postings highlighter for non top-level queries
In #4052 we added support for highlighting multi term queries using the postings highlighter. That worked only for top-level queries though, and not for multi term queries that are nested for instance within a bool query, or filtered query, or a constant score query.

The way we make this work is by walking the query structure and temporarily overriding the query rewrite method with a method that allows for multi terms extraction.

Closes #5102
2014-02-21 21:43:40 +01:00
Simon Willnauer 990ce658a4 [Docs] Remove `custom_score` from documentation and add a migration
section.
2014-02-11 14:59:15 +01:00
Clinton Gormley 93930d6dc7 Removed 0.90.* deprecation and addition notifications
Closes #5052
2014-02-07 20:52:49 +01:00
Lars Francke 1bd9dc129b Fix confusing sentence
The original sentence didn't make much sense. I hope this is a bit better. Taken heavy inspiration from c63d8c4fb5
2014-02-03 17:20:40 +01:00
Nik Everett 93a8e80aff Support multiple rescores
Detects if rescores arrive as an array instead of a plain object.  If so
then parse each element of the array as a separate rescore to be executed
one after another.  It looks like this:
   "rescore" : [ {
      "window_size" : 100,
      "query" : {
         "rescore_query" : {
            "match" : {
               "field1" : {
                  "query" : "the quick brown",
                  "type" : "phrase",
                  "slop" : 2
               }
            }
         },
         "query_weight" : 0.7,
         "rescore_query_weight" : 1.2
      }
   }, {
      "window_size" : 10,
      "query" : {
         "score_mode": "multiply",
         "rescore_query" : {
            "function_score" : {
               "script_score": {
                  "script": "log10(doc['numeric'].value + 2)"
               }
            }
         }
      }
   } ]

Rescores as a single object are still supported.

Closes #4748
2014-01-23 16:29:07 +01:00
Nik Everett 37f80c8d80 Documentation for score_mode
Closes #4742
2014-01-23 16:24:48 +01:00
Lee Hinman 2c289fb538 Add the ability to retrieve fields from field data
Adds a new FetchSubPhase, FieldDataFieldsFetchSubPhase, which loads the
field data cache for a field and returns an array of values for the
field.

Also removes `doc['<field>']` and `_source.<field>` workaround no longer
needed in field name resolving.

Closes #4492
2014-01-21 09:13:32 -07:00
Clinton Gormley 2e4b70d40f [DOCS] Fixed duplicate ID in highlighting 2014-01-09 00:37:18 +01:00
Nik Everett 8bd9e34e39 Stop FVH from throwing away some query boosts
The FVH was throwing away some boosts on queries stopping a number of
ways to boost phrase matches to the top of the list of fragments from
working.

The plain highlighter also doesn't work for this but that is because it
doesn't support the concept of the same term having a different score at
different positions.

Also update documentation claiming that FHV is nicer for weighing terms
found by query combinations.

Closes #4351
2014-01-08 11:51:48 +01:00
Nik Everett 522d620eb6 Use FHV's phraseLimit
This prevents poisoning the FVH with documents that contain TONS of matches
which take tons of memory and time to highlight.

Closes #4645
2014-01-08 11:27:58 +01:00
Simon Willnauer fa16969360 Cleanup comments and class names s/ElasticSearch/Elasticsearch
* Clean up s/ElasticSearch/Elasticsearch on docs/*
 * Clean up s/ElasticSearch/Elasticsearch on src/* bin/* & pom.xml
 * Clean up s/ElasticSearch/Elasticsearch on NOTICE.txt and README.textile

Closes #4634
2014-01-07 11:21:51 +01:00
Martijn van Groningen f1bf585089 The `fields` option should always return an array for json document fields and single valued field for metadata fields.
Also the `fields` option can only be used to fetch leaf fields, trying to do fetch object fields will return in a client error.

Closes #4542
2014-01-03 17:29:12 +01:00
Clinton Gormley 34b9b16233 [DOCS] Fixed some bad link refs 2013-12-16 18:07:33 +01:00
Martijn van Groningen 23d2b1ea7b Renamed top level `filter` to `post_filter`.
Closes #4119
2013-12-16 17:10:14 +01:00
Martijn van Groningen 10e2528cce Added the `force_source` option to highlighting that enforces to use of the _source even if there are stored fields.
The percolator uses this option to deal with the fact that the MemoryIndex doesn't support stored fields,
this is possible b/c the _source of the document being percolated is always present.

Closes #4348
2013-12-13 13:39:53 +01:00
Nik Everett 8e34057bc0 Add support for combining fields to the FVH
The Fast Vector Highlighter can combine matches on multiple fields to
highlight a single field using `matched_fields`.  This is most
intuitive for multifields that analyze the same string in different
ways.  Example:
{
    "query": {
        "query_string": {
            "query": "content.plain:running scissors",
            "fields": ["content"]
        }
    },
    "highlight": {
        "order": "score",
        "fields": {
            "content": {
                "matched_fields": ["content", "content.plain"],
                "type" : "fvh"
            }
        }
    }
}

Closes #3750
2013-12-03 11:10:01 +01:00
Conrad Pankoff 87246af256 [DOCS] Fixed typos and corrected grammar 2013-12-02 10:08:26 +01:00
Clinton Gormley bc393b6d79 Changed the minScore comparator from > to >=
Closes #4303
2013-11-29 20:29:20 +01:00
Boaz Leskes c63d8c4fb5 [Docs] Added _source filtering to documentation
Relates to #3301
2013-11-26 19:16:24 +01:00
Clinton Gormley 3465e69e83 [DOCS] Changed all store:yes/no to store:true/false
which is how this setting is stored internally
2013-11-07 16:57:18 +01:00
Luca Cavanna 48ac9747a8 Added third highlighter type based on lucene postings highlighter
Requires field index_options set to "offsets" in order to store positions and offsets in the postings list.
Considerably faster than the plain highlighter since it doesn't require to reanalyze the text to be highlighted: the larger the documents the better the performance gain should be.
Requires less disk space than term_vectors, needed for the fast_vector_highlighter.
Breaks the text into sentences and highlights them. Uses a BreakIterator to find sentences in the text. Plays really well with natural text, not quite the same if the text contains html markup for instance.
Treats the document as the whole corpus, and scores individual sentences as if they were documents in this corpus, using the BM25 algorithm.

Uses forked version of lucene postings highlighter to support:
- per value discrete highlighting for fields that have multiple values, needed when number_of_fragments=0 since we want to return a snippet per value
- manually passing in query terms to avoid calling extract terms multiple times, since we use a different highlighter instance per doc/field, but the query is always the same

The lucene postings highlighter api is  quite different compared to the existing highlighters api, the main difference being that it allows to highlight multiple fields in multiple docs with a single call, ensuring sequential IO.
The way it is introduced in elasticsearch in this first round is a compromise trying not to change the current highlight api, which works per document, per field. The main disadvantage is that we lose the sequential IO, but we can always refactor the highlight api to work with multiple documents.

Supports pre_tag, post_tag, number_of_fragments (0 highlights the whole field), require_field_match, no_match_size, order by score and html encoding.

Closes #3704
2013-10-24 23:38:00 +02:00
Luca Cavanna e981e411d7 [DOCS] rephrased docs for highlight no_match_size parameter
(removed 0.90.6 coming tag as it's needed only in 0.90 branch)
2013-10-24 14:38:32 +02:00
Nik Everett 14a709f563 Highlighting can return excerpt with no highlights
You can configure the highlighting api to return an excerpt of a field
even if there wasn't a match on the field.

The FVH makes excerpts from the beginning of the string to the first
boundary character after the requested length or the boundary_max_scan,
whichever comes first.  The Plain highlighter makes excerpts from the
beginning of the string to the end of the last token before the requested
length.

Closes #1171
2013-10-24 14:38:32 +02:00
Clinton Gormley b2d82d7e75 [DOCS] Reorganised the highlight_query docs and added a version flag 2013-10-18 18:03:31 +02:00
Clinton Gormley ba1b4886e3 [DOCS] Moved "named filters/queries" up one level 2013-10-10 11:23:08 +02:00
Clinton Gormley 7a53d41446 [DOCS] Changed capitalization of operator in rescore query 2013-10-05 17:18:15 +02:00
Nik Everett 6b000d8c6d Support specifing score query on highlight.
This is useful if you want to highlight terms not in the search query or
you want sort highlighted snippets based on another query.

Closes #3630
2013-10-02 15:46:24 -04:00
Lee Hinman ba40aa374e Uniquify anchor links to fix asciidoc/docbook generation 2013-09-30 15:32:00 -06:00
Lee Hinman 0442b737be Add more anchor links to documentation
Related to #3679
2013-09-30 13:13:16 -06:00
Clinton Gormley 85bba668f7 [DOCS] Tidied up various doc formatting errors 2013-09-16 16:13:01 +02:00
Martijn van Groningen f6f4b5014f Added docs for named queries.
Relates to #3581
2013-09-16 11:17:01 +02:00
Martijn van Groningen 8ddb809f98 If all scroll ids should be removed then the `_all` value should be used instead of not specifying any scroll ids. 2013-09-12 10:41:38 +02:00
Martijn van Groningen 0efa78710b Added clear scroll api.
The clear scroll api allows clear all resources associated with a `scroll_id` by deleting the `scroll_id` and its associated SearchContext.

Closes #3657
2013-09-10 21:17:34 +02:00
Clinton Gormley 6d667e5d41 [DOCS] Missing sort values now works for all field types 2013-09-04 23:20:55 +02:00
Clinton Gormley 393c28bee4 [DOCS] Removed outdated new/deprecated version notices 2013-09-03 21:28:31 +02:00
Clinton Gormley 822043347e Migrated documentation into the main repo 2013-08-29 01:24:34 +02:00