Commit Graph

920 Commits

Author SHA1 Message Date
James Rodewig 2b59f8ac34 [DOCS] Correct `hits.total.relation` response parm def (#52847)
Fixes a partially completed definition for the `hits.total.relation`
response parameter in the search API docs.
2020-03-04 08:23:34 -05:00
Josh Devins 68ba571f70
Adds recall@k metric to rank eval API (#52889)
This change adds the recall@k metric and refactors precision@k to match
the new metric.

Recall@k is an important metric to use for learning to rank (LTR)
use-cases. Candidate generation or first ranking phase ranking functions
are often optimized for high recall, in order to generate as many
relevant candidates in the top-k as possible for a second phase of
ranking. Adding this metric allows tuning that base query for LTR.

See: https://github.com/elastic/elasticsearch/issues/51676
Backports: https://github.com/elastic/elasticsearch/pull/52577
2020-02-27 16:04:24 +01:00
James Rodewig 98bcf06bae [DOCS] Correct multi search API docs (#52523)
* Adds an example request to the top of the page.
* Relocates several parameters erroneously listed under "Request body"
to the appropriate "Query parameters" section.
* Updates the "Request body" section to better document the NDJSON
  structure of msearch requests.
2020-02-24 07:43:10 -05:00
Marios Trivyzas c03f51f68f
[Docs] Clarify default value for `allow_no_indices` (#52635) (#52697)
Add default value to each one of the usages of `allow_no_indices`
since it differs between different APIs.

Relates to: #52534

(cherry picked from commit 2eb986488ac326d6da6ab8ad0203a94e08684a36)
2020-02-24 11:57:32 +01:00
debadair 2588022b81 [DOCS] Fixed typo. (#52071) 2020-02-07 11:04:56 -08:00
Jess 4b31ad1c0c [Docs] Small edits to Ranking Evaluation API docs (#51116)
Small updates to grammar, syntax, and unclear wordings.
2020-01-20 10:30:23 +01:00
Adrien Grand 31158ab3d5
Add per-field metadata. (#50333)
This PR adds per-field metadata that can be set in the mappings and is later
returned by the field capabilities API. This metadata is completely opaque to
Elasticsearch but may be used by tools that index data in Elasticsearch to
communicate metadata about fields with tools that then search this data. A
typical example that has been requested in the past is the ability to attach
a unit to a numeric field.

In order to not bloat the cluster state, Elasticsearch requires that this
metadata be small:
 - keys can't be longer than 20 chars,
 - values can only be numbers or strings of no more than 50 chars - no inner
   arrays or objects,
 - the metadata can't have more than 5 keys in total.

Given that metadata is opaque to Elasticsearch, field capabilities don't try to
do anything smart when merging metadata about multiple indices, the union of
all field metadatas is returned.

Here is how the meta might look like in mappings:

```json
{
  "properties": {
    "latency": {
      "type": "long",
      "meta": {
        "unit": "ms"
      }
    }
  }
}
```

And then in the field capabilities response:

```json
{
  "latency": {
    "long": {
      "searchable": true,
      "aggreggatable": true,
      "meta": {
        "unit": [ "ms" ]
      }
    }
  }
}
```

When there are no conflicts, values are arrays of size 1, but when there are
conflicts, Elasticsearch includes all unique values in this array, without
giving ways to know which index has which metadata value:

```json
{
  "latency": {
    "long": {
      "searchable": true,
      "aggreggatable": true,
      "meta": {
        "unit": [ "ms", "ns" ]
      }
    }
  }
}
```

Closes #33267
2020-01-08 16:21:18 +01:00
James Rodewig 3f7f31b6b0 [DOCS] Fix search request body links (#50500)
PR #44238 changed several links related to the Elasticsearch search request body API. This updates several places still using outdated links or anchors.

This will ultimately let us remove some redirects related to those link changes.
2019-12-26 14:31:09 -05:00
Nik Everett 01293ebad5
Fix docs typos (#50365) (#50464)
Fixes a few typos in the docs.

Co-authored-by: Xiang Dai <764524258@qq.com>
2019-12-23 12:38:17 -05:00
James Rodewig 27ae9a1435 [DOCS] Remove outdated file scripts refererence (#50437)
File scripts were removed in 6.0 with #24627.

This removes an outdated file scripts reference from the conditional clauses section of the search templates docs.
2019-12-20 14:53:40 -05:00
Adrien Grand 87e72156ce
Upgrade to lucene 8.4.0-snapshot-662c455. (#50016) (#50039)
Lucene 8.4 is about to be released so we should check it doesn't cause problems
with Elasticsearch.
2019-12-10 18:04:58 +01:00
Mayya Sharipova 7cf170830c
Optimize sort on numeric long and date fields. (#49732)
This rewrites long sort as a `DistanceFeatureQuery`, which can
efficiently skip non-competitive blocks and segments of documents.
Depending on the dataset, the speedups can be 2 - 10 times.

The optimization can be disabled with setting the system property
`es.search.rewrite_sort` to `false`.

Optimization is skipped when an index has 50% or more data with
the same value.

Optimization is done through:
1. Rewriting sort as `DistanceFeatureQuery` which can
efficiently skip non-competitive blocks and segments of documents.

2. Sorting segments according to the primary numeric sort field(#44021)
This allows to skip non-competitive segments.

3. Using collector manager.
When we optimize sort, we sort segments by their min/max value.
As a collector expects to have segments in order,
we can not use a single collector for sorted segments.
We use collectorManager, where for every segment a dedicated collector
will be created.

4. Using Lucene's shared TopFieldCollector manager
This collector manager is able to exchange minimum competitive
score between collectors, which allows us to efficiently skip
the whole segments that don't contain competitive scores.

5. When index is force merged to a single segment, #48533 interleaving
old and new segments allows for this optimization as well,
as blocks with non-competitive docs can be skipped.

Backport for #48804


Co-authored-by: Jim Ferenczi <jim.ferenczi@elastic.co>
2019-11-29 15:37:40 -05:00
James Rodewig 03600e4e12 [DOCS] Document `script_score` float precision limit (#49402)
All document scores are positive 32-bit floating point numbers. However, this
wasn't previously documented.

This can result in surprising behavior, such as precision loss, for users when
customizing scores using the function score query.

This commit updates an existing admonition in the function score query docs to
document the 32-bits precision limit. It also updates the search API reference
docs to note that `_score` is a 32-bit float.
2019-11-21 08:54:49 -05:00
Orhan Toy 561351d2fc [Docs] Fix _count HTTP method (#48979) 2019-11-12 15:45:26 +01:00
Patrick Maynard 4b85498617 [DOCS] Fix typo in search type docs (#48868) 2019-11-11 09:38:48 -05:00
Christoph Büscher 1de49d8a70 Remove Ranking Evaluation API experimental status (#48603)
The API has been released long enough to remove the experimental status.
2019-10-29 20:57:39 +01:00
Ian Danforth 82e25c4ac7 [Docs] Fix typo in suggesters search API doc (#48477) 2019-10-29 09:58:05 +01:00
James Rodewig e9c8e4f6d1 [DOCS] Fix note format in index suggestion docs (#48536) 2019-10-25 11:31:47 -04:00
Christoph Büscher 055a0800eb [Docs] Mention reserved completion suggestion characters (#48445)
We currently don't mention the three reserved characters anywhere. This change
adds a short note mentioning them

Closes #48341
2019-10-25 16:58:23 +02:00
James Rodewig 852622d970 [DOCS] Remove binary gendered language (#48362) 2019-10-23 09:37:12 -05:00
Jim Ferenczi dc39196ea4 Fix tag in the search request timeout option docs (#47776)
and add missing parentheses `search_timeout` param
2019-10-10 10:35:44 +02:00
James Rodewig c03cdb4b15 [DOCS] Correct callouts in search template docs (#47655) 2019-10-07 09:25:32 -04:00
James Rodewig fd421bd12d
[7.x] [DOCS] Add response body parms to search API docs (#47042) (#47303) 2019-09-30 13:54:06 -04:00
István Zoltán Szabó 0ab7132c47 [DOCS] Reformats Profile API (#47168)
* [DOCS] Reformats Profile API.

* [DOCS] Fixes failing docs test.
2019-09-27 11:14:14 +02:00
István Zoltán Szabó 74fd21f0b0 [DOCS] Reformats ranking evaluation API (#46974)
* [DOCS] Reformats ranking evaluation API.
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-25 15:01:10 +02:00
István Zoltán Szabó 83365e94ba [DOCS] Reformat suggesters page. (#47010) 2019-09-25 14:42:16 +02:00
István Zoltán Szabó fcea154f2e [DOCS] Reformats Field capabilities API (#46866)
* [DOCS] Reformats Field capabilities API.
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-20 11:28:19 +02:00
István Zoltán Szabó 363075cf1d [DOCS] Reformats explain API (#46857)
* [DOCS] Reformats explain API.
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-20 11:00:33 +02:00
James Rodewig 251dbd8522 [DOCS] Remove `lowercase_terms` parm from term suggester docs (#46879) 2019-09-19 15:56:47 -04:00
Takumasa Ochi 7a3054c5dc Fix typos in `match` in profile API (#46723)
* Replace `matches` with correct `match`
* Use present tense consistently
* Replace `metric` with correct `match`
2019-09-19 16:07:52 +02:00
István Zoltán Szabó e59be0354a [DOCS] Reformats validate API (#46389)
* [DOCS] Reformats validate API.
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-18 14:31:17 +02:00
István Zoltán Szabó 595bf52927 [DOCS] Reformats count API (#46377)
* [DOCS] Reformats count API.
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-17 09:54:19 +02:00
James Rodewig e253ee6ba6
[DOCS] Change // CONSOLE comments to [source,console] (#46440) (#46494) 2019-09-09 12:35:50 -04:00
James Rodewig f04573f8e8
[DOCS] [5 of 5] Change // TESTRESPONSE comments to [source,console-results] (#46449) (#46459) 2019-09-06 16:09:09 -04:00
James Rodewig bb7bff5e30
[DOCS] Replace "// TESTRESPONSE" magic comments with "[source,console-result] (#46295) (#46418) 2019-09-06 09:22:08 -04:00
James Rodewig 1f36c4e50c
[DOCS] Replace "// CONSOLE" comments with [source,console] (#46159) (#46332) 2019-09-05 10:11:25 -04:00
István Zoltán Szabó 0f0b77b263 [DOCS] Reformats search template and multi search template APIs (#46236)
* [DOCS] Reformats search template and multi search template APIs.
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-04 15:14:06 +02:00
István Zoltán Szabó c71d959d61 [DOCS] Reformats search shards API (#46240)
* [DOCS] Reformats search shards API
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-04 11:36:08 +02:00
István Zoltán Szabó 5c5af77565 [DOCS] Reformats request body search API (#46254)
* [DOCS] Reformats request body search API.
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-04 10:53:16 +02:00
István Zoltán Szabó f2bdd392e7 [DOCS] Reformats multi search API (#46256)
* [DOCS] Reformats multi search API.

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-09-04 10:19:43 +02:00
István Zoltán Szabó 53f70ee996 [DOCS] Reformats URI search request (#45844)
* [DOCS] Reformats URI search request.

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

Co-Authored-By: debadair <debadair@elastic.co>
2019-08-30 13:45:29 +02:00
István Zoltán Szabó 4b086fbef2 Revert "[DOCS] Reformats URI search request (#45844)"
This reverts commit 7f11c32400.
2019-08-29 11:58:28 +02:00
István Zoltán Szabó 7f11c32400 [DOCS] Reformats URI search request (#45844)
* [DOCS] Reformats URI search request.

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

Co-Authored-By: debadair <debadair@elastic.co>
2019-08-29 10:06:16 +02:00
James Rodewig ceb8b9bbee Change `{var}` convention to `<var>` (#45904) 2019-08-23 10:57:48 -04:00
István Zoltán Szabó 6e696296fe [DOCS] Reformats search API (#45786)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-23 11:56:49 +02:00
Nathan Howard bdfd90560f Adding a warning to from-size.asciidoc
Customers occasionally discover a known behavior in Elasticsearch's pagination that does not appear to be documented. This warning is intended to educate customers of this behavior while still highlighting alternative solutions.
2019-08-22 19:03:13 -07:00
James Rodewig 33d4801213 Revert "[DOCS] Reformats search API (#45786)"
This reverts commit f6ffa00142.
2019-08-22 09:49:15 -04:00
István Zoltán Szabó f6ffa00142 [DOCS] Reformats search API (#45786)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-22 15:09:09 +02:00
James Rodewig 5e44e695fd [DOCS] Add template docs to scripts. Reorder template examples. (#45817)
* [DOCS] Add template docs to scripts. Reorder template examples.

* Adds a 'Search template' section to the 'How to use scripts' chapter.
  This links to the 'Search template' chapter for detailed info and
  examples.

* Reorders and retitles several examples in the 'Search template'
  chapter. This is primarily to make examples for storing, deleting, and
  using search templates more prominent.

* Change <templatename> to <templateid>
2019-08-22 08:40:32 -04:00
Jonathan Hult 041385559c [DOCS] Fix typo in highlighting doc (#45707) 2019-08-20 07:28:05 -04:00
James Rodewig c75fd40f2c [DOCS] Add diagrams to cross-cluster search documentation (#45569) 2019-08-15 11:00:25 -04:00
Emmanuel DEMEY f8c08c537b Add snippet for the search_type query parameter (#43540) 2019-08-11 18:36:52 -04:00
Jesse Wright f19f2adbe6 [Docs] Fix typo in rank-eval.asciidoc (#44978) 2019-07-31 12:37:49 +02:00
James Rodewig a63f60b776 [DOCS] Remove heading offsets for REST APIs (#44568)
Several files in the REST APIs nav section are included using
:leveloffset: tags. This increments headings (h2 -> h3, h3 -> h4, etc.)
in those files and removes the :leveloffset: tags.

Other supporting changes:
* Alphabetizes top-level REST API nav items.
* Change 'indices APIs' heading to 'index APIs.'
* Changes 'Snapshot lifecycle management' heading to sentence case.
2019-07-19 14:36:06 -04:00
James Rodewig d46545f729 [DOCS] Update anchors and links for Elasticsearch API relocation (#44500) 2019-07-19 09:18:23 -04:00
James Rodewig 34725e20fb [DOCS] Move Elasticsearch APIs to REST APIs section. (#44238) (#44372)
Moves the following API sections under the REST APIs navigations:
- API Conventions
- Document APIs
- Search APIs
- Index APIs (previously named Indices APIs)
- cat APIs
- Cluster APIs

Other supporting changes:
- Removes the previous index APIs page under REST APIs. Adds a redirect for the removed page.
- Removes several [partintro] macros so the docs build correctly.
- Changes anchors for pages that become sections of a parent page.
- Adds several redirects for existing pages that become sections of a parent page.

This commit re-applies changes from #44238. Changes from that PR were reverted due to broken links in several repos. This commit adds redirects for those broken links.
2019-07-17 09:18:31 -04:00
Julie Tibshirani 141d09ee15 Correct a formatting mistake in the _field_caps docs. (#44303)
The 'indices' block that was recently added should appear in the top-level of
the response, as opposed to being nested under 'fields'.
2019-07-15 09:46:02 -07:00
John Murphy 8030d8f6dc [DOCS] Add `lowercase` filter to phrase suggester example so searches are case insensitive (#44186) 2019-07-11 15:27:31 -04:00
David Kyle d1280339a8
specifies which index to search in docs for various queries (#43307) (#43428)
the geo-bounding-box and phrase-suggest docs were susceptible to
failing due to other indices in the cluster. This change restricts
the queries to the index that is set up for the test.

relates to #43271.
2019-06-21 10:15:51 +01:00
Luca Cavanna 4da0fadedc [DOCS] Clarify phrase suggester docs smoothing parameter (#42947)
Closes #28512
2019-06-12 11:25:03 +02:00
Luca Cavanna e538592652 Update max_concurrent_shard_request parameter docs (#42227)
Some of the docs were outdated as they did not mention that the limit is
not per node. Also, The default value changed.

Relates to #31206
2019-06-12 11:25:03 +02:00
Christoph Büscher d9c582e66b [Docs] Add to preference parameter docs (#42797)
Adding notes to the existing docs about how using `preference` might increase
request cache utilization but also add warning about the downsides.

Closes #24278
2019-06-04 14:38:18 +02:00
Julie Tibshirani 3a00d08c50 Clarify that inner_hits must be used to access nested fields. (#42724)
This PR updates the docs for `docvalue_fields` and `stored_fields` to clarify
that nested fields must be accessed through `inner_hits`. It also tweaks the
nested fields documentation to make this point more visible.

Addresses #23766.
2019-05-31 10:06:11 -07:00
bellengao 380f296631 Update script-fields.asciidoc (#42490) 2019-05-27 11:48:37 +02:00
James Rodewig 53702efddd [DOCS] Add anchors for Asciidoctor migration (#41648) 2019-04-30 10:20:17 -04:00
James Rodewig fc4f401214 [DOCS] Document 200 http code response for partial results (#40367) 2019-04-29 08:30:17 -04:00
James Rodewig 4adf7963c7 [DOCS] Escape commas in experimental[] for Asciidoctor migration (#41578) 2019-04-26 11:35:47 -04:00
Jim Ferenczi 6184efaff6
Handle unmapped fields in _field_caps API (#34071) (#41426)
Today the `_field_caps` API returns the list of indices where a field
is present only if this field has different types within the requested indices.
However if the request is an index pattern (or an alias, or both...) there
is no way to infer the indices if the response contains only fields that have
the same type in all indices. This commit changes the response to always return
the list of indices in the response. It also adds a way to retrieve unmapped field
in a specific section per field called `unmapped`. This section is created for each field
that is present in some indices but not all if the parameter `include_unmapped` is set to
true in the request (defaults to false).
2019-04-25 18:13:48 +02:00
David Turner 411994b489 Mention the cost of tracking live docs in scrolls (#41375)
Relates #41337, in which a heap dump shows hundreds of MBs allocated on the
heap for tracking the live docs for each scroll.
2019-04-23 15:37:41 +01:00
Adrien Grand f7e590ce0d
ProfileScorer should propagate `setMinCompetitiveScore`. (#40958) (#41302)
Currently enabling profiling disables top-hits optimizations, which is
unfortunate: it would be nice to be able to notice the difference in method
counts and timings depending on whether total hit counts are requested.
2019-04-17 16:11:14 +02:00
James Rodewig 9f3fae2c59 [DOCS] Fix code block length for Asciidoctor migration (#41152) 2019-04-12 12:27:29 -04:00
Alexander Reelsen 3b9ab5da04 Fix order of request body search parameter names in documentation (#40777)
The order was random, which made it super hard to find anything. This
changes the order to be alphabetically.
2019-04-09 16:35:45 +02:00
Christoph Büscher 40638d7b28 [Docs] Delete explanation for completion suggester default analyzer choice (#36720)
The explanation given in the completion suggester documentation why we use the
"simple" analyzer as the default is no longer valid. Since we still use "simple"
as the default, we should just delete the explanation that doesn't fit anymore.

Closes #36715
2019-04-09 13:50:29 +02:00
Mayya Sharipova 1e90b29c00 Add information about the default sort mode (#40657) 2019-03-30 10:58:47 -04:00
Andy Bristol 23395a9b9f
search as you type fieldmapper (#35600)
Adds the search_as_you_type field type that acts like a text field optimized
for as-you-type search completion. It creates a couple subfields that analyze
the indexed terms as shingles, against which full terms are queried, and a
prefix subfield that analyze terms as the largest shingle size used and
edge-ngrams, against which partial terms are queried

Adds a match_bool_prefix query type that creates a boolean clause of a term
query for each term except the last, for which a boolean clause with a prefix
query is created.

The match_bool_prefix query is the recommended way of querying a search as you
type field, which will boil down to term queries for each shingle of the input
text on the appropriate shingle field, and the final (possibly partial) term
as a term query on the prefix field. This field type also supports phrase and
phrase prefix queries however
2019-03-27 13:29:13 -07:00
Jim Ferenczi 3400483af4
Add date and date_nanos conversion to the numeric_type sort option (#40199) (#40224)
This change adds an option to convert a `date` field to nanoseconds resolution
 and a `date_nanos` field to millisecond resolution when sorting.
The resolution of the sort can be set using the `numeric_type` option of the
field sort builder. The conversion is done at the shard level and is restricted
to dates from 1970 to 2262 for the nanoseconds resolution in order to avoid
numeric overflow.
2019-03-20 16:50:28 +01:00
Jim Ferenczi 5b73a1bc7d
Add an option to force the numeric type of a field sort (#38095) (#40084)
This change adds an option to the `FieldSortBuilder` that allows to transform the type
of a numeric field into another. Possible values for this option are `long` that transforms
the source field into an integer and `double` that transforms the source field into a floating point.
This new option is useful for cross-index search when the sort field is mapped differently on some
indices. For instance if a field is mapped as a floating point in one index and as an integer in another
it is possible to align the type for both indices using the `numeric_type` option:

```
{
   "sort": {
    "field": "my_field",
    "numeric_type": "double" <1>
   }
}
```

<1> Ensure that values for this field are transformed to a floating point if needed.
2019-03-18 09:32:45 +01:00
Lisa Cawley c92476f591 [DOCS] Replaces CCS terms with attributes (#40076) 2019-03-15 07:57:51 -07:00
Darren Meiss eae2c9dd5c Edits to text in Phrase Suggester doc (#38966) 2019-02-20 12:35:21 +01:00
Clinton Gormley d59ec89726 Update track-total-hits.asciidoc
Added missing `
2019-02-18 13:33:23 +01:00
Darren Meiss dc0e657091 Edits to text & formatting in Term Suggester doc (#38963) 2019-02-15 16:00:28 -05:00
Darren Meiss 56997bf53d Edits to text in Completion Suggester doc (#38980) 2019-02-15 15:47:54 -05:00
Darren Meiss 34b6383c47 Edits to text of Profile API documentation (#38742)
Minor edits of text.
2019-02-13 10:08:50 +01:00
Mayya Sharipova 6eec065353
Describe what _source.includes/excludes do (#38319) (#38794) 2019-02-12 11:09:15 -05:00
Darren Meiss f8426d9b76 Fix typos in Field-Caps documentation (#38580)
Fix typo in Field-Caps documentation

Reworded because asciidoc was formatting the ellipse/space as a numbered list.
2019-02-11 20:58:31 +01:00
Nik Everett 5d949dddfb
Docs: Drop inline callout from scroll example (#38340)
Coalesces two calls into one in a scroll example so all callouts are at
the end of the line. This is the only sort of callouts that are
supported by asciidoctor and we'd like to start building our docs with
asciidoctor.

At present we don't have any mechanism to stop folks adding more inline
callouts but we ought to be able to have one in a few weeks. For now,
though, removing these inline callouts is a step in the right direction.

Relates to #38335
2019-02-04 14:57:38 -05:00
Luca Cavanna 622fb7883b
Introduce ability to minimize round-trips in CCS (#37828)
With #37566 we have introduced the ability to merge multiple search responses into one. That makes it possible to expose a new way of executing cross-cluster search requests, that makes CCS much faster whenever there is network latency between the CCS coordinating node and the remote clusters. The coordinating node can now send a single search request to each remote cluster, which gets reduced by each one of them. from + size results are requested to each cluster, and the reduce phase in each cluster is non final (meaning that buckets are not pruned and pipeline aggs are not executed). The CCS coordinating node performs an additional, final reduction, which produces one search response out of the multiple responses received from the different clusters.

This new execution path will be activated by default for any CCS request unless a scroll is provided or inner hits are requested as part of field collapsing. The search API accepts now a new parameter called ccs_minimize_roundtrips that allows to opt-out of the default behaviour.

Relates to #32125
2019-01-31 15:12:14 +01:00
Adrien Grand c8af0f4bfa
Use mappings to format doc-value fields by default. (#30831)
Doc-value fields now return a value that is based on the mappings rather than
the script implementation by default.

This deprecates the special `use_field_mapping` docvalue format which was added
in #29639 only to ease the transition to 7.x and it is not necessary anymore in
7.0.
2019-01-30 10:31:51 +01:00
Jim Ferenczi 787acb14b9
Track total hits up to 10,000 by default (#37466)
This commit changes the default for the `track_total_hits` option of the search request
to `10,000`. This means that by default search requests will accurately track the total hit count
up to `10,000` documents, requests that match more than this value will set the `"total.relation"`
to `"gte"` (e.g. greater than or equals) and the `"total.value"` to `10,000` in the search response.
Scroll queries are not impacted, they will continue to count the total hits accurately.
The default is set back to `true` (accurate hit count) if `rest_total_hits_as_int` is set in the search request.
I choose `10,000` as the default because that's also the number we use to limit pagination. This means that users will be able to know how far they can jump (up to 10,000) even if the total number of hits is not accurate.

Closes #33028
2019-01-25 13:45:39 +01:00
Christoph Büscher 95a6951f78
Use new bulk API endpoint in the docs (#37698)
This change switches to using the typeless bulk API endpoint in the
documentation snippets where possible
2019-01-23 09:46:28 +01:00
Boaz Leskes 52ba407931
Expose sequence number and primary terms in search responses (#37639)
Users may require the sequence number and primary terms to perform optimistic concurrency control operations. Currently, you can get the sequence number via the `docvalues_fields` API but the primary term is not accessible because it is maintained by the `SeqNoFieldMapper` and the infrastructure can't find it. 

This commit adds a dedicated sub fetch phase to return both numbers that is connected to a new `seq_no_primary_term` parameter.
2019-01-23 09:01:58 +01:00
Christoph Büscher 34f2d2ec91
Remove remaining occurances of "include_type_name=true" in docs (#37646) 2019-01-22 15:13:52 +01:00
Christoph Büscher 3a96608b3f
Remove more include_type_name and types from docs (#37601) 2019-01-18 14:11:18 +01:00
Christoph Büscher 25aac4f77f
Remove `include_type_name` in asciidoc where possible (#37568)
The "include_type_name" parameter was temporarily introduced in #37285 to facilitate
moving the default parameter setting to "false" in many places in the documentation
code snippets. Most of the places can simply be reverted without causing errors.
In this change I looked for asciidoc files that contained the
"include_type_name=true" addition when creating new indices but didn't look
likey they made use of the "_doc" type for mappings. This is mostly the case
e.g. in the analysis docs where index creating often only contains settings. I
manually corrected the use of types in some places where the docs still used an
explicit type name and not the dummy "_doc" type.
2019-01-18 09:34:11 +01:00
Julie Tibshirani 36a3b84fc9
Update the default for include_type_name to false. (#37285)
* Default include_type_name to false for get and put mappings.

* Default include_type_name to false for get field mappings.

* Add a constant for the default include_type_name value.

* Default include_type_name to false for get and put index templates.

* Default include_type_name to false for create index.

* Update create index calls in REST documentation to use include_type_name=true.

* Some minor clean-ups around the get index API.

* In REST tests, use include_type_name=true by default for index creation.

* Make sure to use 'expression == false'.

* Clarify the different IndexTemplateMetaData toXContent methods.

* Fix FullClusterRestartIT#testSnapshotRestore.

* Fix the ml_anomalies_default_mappings test.

* Fix GetFieldMappingsResponseTests and GetIndexTemplateResponseTests.

We make sure to specify include_type_name=true during xContent parsing,
so we continue to test the legacy typed responses. XContent generation
for the typeless responses is currently only covered by REST tests,
but we will be adding unit test coverage for these as we implement
each typeless API in the Java HLRC.

This commit also refactors GetMappingsResponse to follow the same appraoch
as the other mappings-related responses, where we read include_type_name
out of the xContent params, instead of creating a second toXContent method.
This gives better consistency in the response parsing code.

* Fix more REST tests.

* Improve some wording in the create index documentation.

* Add a note about types removal in the create index docs.

* Fix SmokeTestMonitoringWithSecurityIT#testHTTPExporterWithSSL.

* Make sure to mention include_type_name in the REST docs for affected APIs.

* Make sure to use 'expression == false' in FullClusterRestartIT.

* Mention include_type_name in the REST templates docs.
2019-01-14 13:08:01 -08:00
Josh Soref edb48321ba [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
Jim Ferenczi e38cf1d0dc
Add the ability to set the number of hits to track accurately (#36357)
In Lucene 8 searches can skip non-competitive hits if the total hit count is not requested.
It is also possible to track the number of hits up to a certain threshold. This is a trade off to speed up searches while still being able to know a lower bound of the total hit count. This change adds the ability to set this threshold directly in the track_total_hits search option. A boolean value (true, false) indicates whether the total hit count should be tracked in the response. When set as an integer this option allows to compute a lower bound of the total hits while preserving the ability to skip non-competitive hits when enough matches have been collected.

Relates #33028
2019-01-04 20:36:49 +01:00
Abdullah DURSUN 8a02bacf76 Fix typo in multi-search.asciidoc (#37060) 2019-01-02 10:32:42 +01:00
lcawl 504cfb2fb1 [DOCS] Adds missing anchors for profile API 2018-12-18 15:20:19 -08:00
Julie Tibshirani 87831051dc
Deprecate types in explain requests. (#35611)
The following updates were made:
- Add a new untyped endpoint `{index}/_explain/{id}`.
- Add deprecation warnings to Rest*Action, plus tests in Rest*ActionTests.
- For each REST yml test, make sure there is one version without types, and another legacy version that retains types (called *_with_types.yml).
- Deprecate relevant methods on the Java HLRC requests/ responses.
- Update documentation (for both the REST API and Java HLRC).
2018-12-10 19:45:13 -08:00