Commit Graph

5023 Commits

Author SHA1 Message Date
Jason Tedor faa7fe86c5
Introduce analyze thread pool (#29541)
We want to remove the index thread pool as it is no longer needed since
single-document indexing requests are executed as bulk requests
now. Analyze requests are also executed on the index thread pool though
and they need a thread pool to execute on. The bulk thread does not seem
like the right thread pool, let us keep that thread pool conceptually
for bulk requests and free for bulk requests. None of the existing
thread pools make sense for analyze requests either. The generic thread
pool would be a terrible choice since it has an unbounded queue and that
is a bad idea for user-facing APIs. This commit introduces a small by
default (size=1, queue_size=16) thread pool for analyze requests.
2018-04-17 06:46:15 -04:00
olcbean b3e3b80f1b REST high-level client: add support for Indices Update Settings API [take 2] (#29327)
Relates to #27205
2018-04-16 21:39:11 +02:00
Igor Motov e334baf6fc
Fix overflow error in parsing of long geohashes (#29418)
Fixes a possible overflow error that geohashes longer than 12 characters
can cause during parsing.

Fixes #24616
2018-04-16 12:37:38 -04:00
Ke Li 0bfb59dcf2 Using ObjectParser in UpdateRequest (#29293)
CRUD: Parsing changes for UpdateRequest (#29293)

Use `ObjectParser` to parse `UpdateRequest` so we reject unknown fields
and drop support for the `_fields` parameter because it was deprecated
in 5.x.
2018-04-16 08:39:35 -04:00
Bolarinwa Saheed Olayemi 7931cf87f0 [Docs] Add definitions to glossary (#29127)
Definitions for "filter" and "query" are added to the glossary of terms.

Closes #29127
2018-04-16 10:49:54 +02:00
Mayya Sharipova 5dcfdb09cb
Control max size and count of warning headers (#28427)
Control max size and count of warning headers

Add a static persistent cluster level setting
"http.max_warning_header_count" to control the maximum number of
warning headers in client HTTP responses.
Defaults to unbounded.

Add a static persistent cluster level setting
"http.max_warning_header_size" to control the maximum total size of
warning headers in client HTTP responses.
Defaults to unbounded.

With every warning header that exceeds these limits,
a message will be logged in the main ES log,
and any more warning headers for this response will be
ignored.
2018-04-13 05:55:33 -04:00
Adrien Grand ebd6b5b7ba
Deprecate filtering on `_type`. (#29468)
As indices are only allowed to have one type now, and types are going away in
the future, we should deprecate filtering by `_type`.

Relates #15613
2018-04-13 09:07:51 +02:00
Jason Tedor d44bbb7602
Fix auto-generated ID example format (#29461)
This commit fixes the format of an example auto-generated ID in the docs
where the format misleadingly looked like a UUID rather than a base64
ID.
2018-04-12 22:33:38 -04:00
Sachin Frayne 183ba9a7e5 Fix typo in max number of threads check docs (#29469)
Historically, the bootstrap checks used 2048 as the minimum limit for
the maximum number of threads. This limit was guided by the fact that
the number of processors was artificially capped at 32. This limit was
removed in 6.0.0 and the minimum limit was raised to 4096 to accommodate
this. However, the docs were not updated and this commit addresses that
miss.
2018-04-12 21:16:37 -04:00
Nik Everett 0a21533097 Docs: Fix callouts after _parent removed
Fix the callouts in the docs for the high level rest client after
_parent support was removed.
2018-04-11 12:56:13 -04:00
Adrien Grand 6a6c0ea5e6
Add an `include_type_name` option. (#29453)
This adds an `include_type_name` option to the `indices.create`,
`indices.get_mapping` and `indices.put_mapping` APIs, which defaults to `true`.
When set to `false`, then mappings will be returned directly in the body of
the `indices.get_mapping` API, without keying them by the type name, the
`indices.create` will expect mappings directly under the `mappings` key, and
the `indices.put_mapping` will use `_doc` as a type name and fail if a `type`
is provided explicitly.

Relates #15613
2018-04-11 15:54:16 +02:00
Adrien Grand 3a147b442a Fix docs build. 2018-04-11 13:48:53 +02:00
Jim Ferenczi 1b6d5e531b
Fail _search request with trailing tokens (#29428)
This change validates that the `_search` request does not have trailing
tokens after the main object and fails the request with a parsing exception otherwise.

Closes #28995
2018-04-11 13:10:22 +02:00
Adrien Grand 4918924fae
Remove legacy mapping code. (#29224)
Some features have been deprecated since `6.0` like the `_parent` field or the
ability to have multiple types per index. This allows to remove quite some
code, which in-turn will hopefully make it easier to proceed with the removal
of types.
2018-04-11 09:41:37 +02:00
Adrien Grand aeac682869
Make purely negative queries return scores of 0. (#26015)
It would make them consistent with queries that are only made of filters.

Closes #23449
2018-04-10 14:31:06 +02:00
Christoph Büscher 9f0c5ccf34 [Docs] Correct typos in rank-eval and multi-search 2018-04-10 12:48:16 +02:00
D Pinto 8d6a368402 [Docs] Correct typo in pipeline.asciidoc (#29431) 2018-04-10 10:42:07 +02:00
Lisa Cawley 28e9ef3c83
[DOCS] Updated installation pages with X-Pack indices (#29077) 2018-04-09 08:19:38 -07:00
yangyang.liu 73019ae887 [Docs] Update painless-lang-spec.asciidoc (#29425)
Remove redundant word.
2018-04-09 10:42:16 +02:00
Van0SS 4310ef0be6 [Docs] Update Copyright notices to 2018 (#29404) 2018-04-06 16:21:20 +02:00
Christoph Büscher 231fd4eb18
Remove `delimited_payload_filter` (#27705)
From 7.0 on, using `delimited_payload_filter` should throw an error. 
It was deprecated in 6.2 in favour of `delimited_payload` (#26625).

Relates to #27704
2018-04-05 18:41:04 +02:00
Jason Tedor 8fdca6a89a
Align cat thread pool info to thread pool config (#29195)
Today we report thread pool info using a common object. This means that
we use a shared set of terminology that is not consistent with the
terminology used to the configure thread pools. This holds in particular
for the minimum and maximum number of threads in the thread pool where
we use the following terminology:
 thread pool info | fixed | scaling
  min                core    size
  max                max     size

A previous change addressed this for the nodes info API. This commit
changes the display of thread pool info in the cat thread pool API too
to be dependent on the type of the thread pool so that we can align the
terminology in the output of thread pool info with the terminology used
to configure a thread pool.
2018-04-03 17:27:26 -04:00
Adrien Grand 569d0c0e89
Improve similarity integration. (#29187)
This improves the way similarities are plugged in in order to:
 - reject the classic similarity on 7.x indices and emit a deprecation
   warning otherwise
 - reject unkwown parameters on 7.x indices and emit a deprecation
   warning otherwise

Even though this breaks the plugin API, I'd like to backport to 7.x so
that users can get deprecation warnings when they are doing something
that will become unsupported in the future.

Closes #23208
Closes #29035
2018-04-03 16:45:25 +02:00
Christoph Büscher dc1c16964a [Docs] Correct experimental note formatting 2018-04-03 16:16:21 +02:00
Ragnar Nevries 5adfe2d8e3 [Docs] Update getting-started.asciidoc (#29294)
Description after first use of search API with body does not refer to verb POST.
2018-04-03 14:57:42 +02:00
Yannick Welsch 3be98edc69
[DOCS] Refer back to index API for full-document updates in _update API section (#28677)
This clarifies how full-document updates are performed in ES.
2018-04-03 13:53:15 +02:00
rzmf 080cefec73 Fix missing comma in ingest-node.asciidoc (#29343) 2018-04-03 11:33:44 +01:00
Adrien Grand 3bdfc8f3fb
Upgrade to lucene-7.3.0-snapshot-98a6b3d. (#29298)
Most notable changes include:
 - this release doesn't have the 7.2.1 version constant so I had to create one
 - spatial4j and jts were upgraded
2018-04-03 09:27:14 +02:00
Jason Tedor 1df43a09b7
Remove HTTP max content length leniency (#29337)
I am not sure why we have this leniency for HTTP max content length, it
has been there since the beginning
(5ac51ee93f) with no explanation of its
source. That said, our philosophy today is different than the philosophy
of the past where Elasticsearch would be quite lenient in its handling
of settings and today we aim for predictability for both users and
us. This commit removes leniency in the parsing of
http.max_content_length.
2018-04-02 20:20:01 -04:00
David Turner 40d19532bc
Clarify expectations of false positives/negatives (#27964)
Today this part of the documentation just says that Geo queries are not 100% 
accurate, but in fact we can be more precise about which kinds of queries see
which kinds of error. This commit clarifies this point.
2018-04-02 10:03:42 +01:00
David Turner 3ca9310aee
Update docs on vertex ordering (#27963)
At time of writing, GeoJSON did not enforce a specific ordering of vertices in
a polygon, but it now does. We occasionally get reports of Elasticsearch
rejecting apparently-valid GeoJSON because of badly oriented polygons, and it's
helpful to be able to point at this bit of the documentation when responding.
2018-04-02 09:59:12 +01:00
Mayya Sharipova e70cd35bda
Revert "REST high-level client: add support for Indices Update Settings API (#28892)" (#29323)
This reverts commit b67b5b1bbd.
2018-03-30 16:26:46 -07:00
Fabien Baligand 199d131385 Improve query string docs (#28882)
fix query string syntax doc when OR operator is missed
2018-03-30 16:36:40 +02:00
Fabien Baligand 437ad06e40 fix query string example for boolean query (#28881) 2018-03-30 15:10:14 +02:00
olcbean b67b5b1bbd REST high-level client: add support for Indices Update Settings API (#28892)
Relates to #27205
2018-03-30 10:53:29 +02:00
Sue Gallagher 5518640d46
[DOCS] Added info on WGS-84. Closes issue #3590 (#29305) 2018-03-29 15:50:05 -07:00
Boaz Leskes eb8b31746a Move trimming unsafe commits from engine ctor to store (#29260)
As follow up to #28245 , this PR removes the logic for selecting the 
right start commit from the Engine constructor in favor of explicitly
trimming them in the Store, before the engine is opened. This makes the
constructor in engine follow standard Lucene semantics and use the last
commit.

Relates #28245
Relates #29156
2018-03-29 13:35:57 -04:00
Bolarinwa Saheed Olayemi a3e5773522 Docs: Link to Ansible playbook for Elasticsearch (#29238)
Links to the official Ansible playbook for Elasticsearch.
2018-03-28 18:18:42 -04:00
Christoph Büscher c3fdf8fbfb [Docs] Fix small typo in ranking evaluation docs 2018-03-28 17:45:44 +02:00
Diwas Joshi 63203b228b [Docs] Update aggregations.asciidoc (#29265)
Add note about accuracy of some aggregation results.
2018-03-28 15:01:45 +02:00
Andrew Banchich d2baf4b191 [Docs] Spelling and grammar changes to reindex.asciidoc (#29232) 2018-03-27 12:17:46 +02:00
Boaz Leskes f5d4550e93
Fold EngineDiskUtils into Store, for better lock semantics (#29156)
#28245 has introduced the utility class`EngineDiskUtils` with a set of methods to prepare/change
translog and lucene commit points. That util class bundled everything that's needed to create and
empty shard, bootstrap a shard from a lucene index that was just restored etc. 

In order to safely do these manipulations, the util methods acquired the IndexWriter's lock. That
would sometime fail due to concurrent shard store fetching or other short activities that require the
files not to be changed while they read from them. 

Since there is no way to wait on the index writer lock, the `Store` class has other locks to make
sure that once we try to acquire the IW lock, it will succeed. To side step this waiting problem, this
PR folds `EngineDiskUtils` into `Store`. Sadly this comes with a price - the store class doesn't and
shouldn't know about the translog. As such the logic is slightly less tight and callers have to do the
translog manipulations on their own.
2018-03-26 14:08:03 +02:00
Jim Ferenczi 5288235ca3
Optimize the composite aggregation for match_all and range queries (#28745)
This change refactors the composite aggregation to add an execution mode that visits documents in the order of the values
present in the leading source of the composite definition. This mode does not need to visit all documents since it can early terminate
the collection when the leading source value is greater than the lowest value in the queue.
Instead of collecting the documents in the order of their doc_id, this mode uses the inverted lists (or the bkd tree for numerics) to collect documents
in the order of the values present in the leading source.
For instance the following aggregation:

```
"composite" : {
  "sources" : [
    { "value1": { "terms" : { "field": "timestamp", "order": "asc" } } }
  ],
  "size": 10
}
```
... can use the field `timestamp` to collect the documents with the 10 lowest values for the field instead of visiting all documents.
For composite aggregation with more than one source the execution can early terminate as soon as one of the 10 lowest values produces enough
composite buckets. For instance if visiting the first two lowest timestamp created 10 composite buckets we can early terminate the collection since it
is guaranteed that the third lowest timestamp cannot create a composite key that compares lower than the one already visited.

This mode can execute iff:
 * The leading source in the composite definition uses an indexed field of type `date` (works also with `date_histogram` source), `integer`, `long` or `keyword`.
 * The query is a match_all query or a range query over the field that is used as the leading source in the composite definition.
 * The sort order of the leading source is the natural order (ascending since postings and numerics are sorted in ascending order only).

If these conditions are not met this aggregation visits each document like any other agg.
2018-03-26 09:51:37 +02:00
Christoph Büscher afe95a7738
[Docs] Add rank_eval size parameter k (#29218)
The rank_eval documentation was missing an explanation of the parameter
`k` that controls the number of top hits that are used in the ranking evaluation.

Closes #29205
2018-03-23 18:04:32 +01:00
Nicholas Knize d400a08788 [DOCS] Remove ignore_z_value parameter link
Removes invalid ignore_z_value parameter link in geo-point.asciidoc.
2018-03-23 11:07:24 -05:00
Jean-Charles Legras 687fe860ac Docs: Update docs/index_.asciidoc (#29172)
Use `_doc` in the routing example instead of `tweet` to agree with the
text and line up with the other examples.
2018-03-23 11:35:10 -04:00
Petr Novák 16bffc7394 Docs: Link C++ client lib elasticlient (#28949)
elasticlient is simple library for simplified work with Elasticsearch in C++
2018-03-23 11:30:01 -04:00
Yannick Welsch 3b8a8867c4
[DOCS] Unregister repository instead of deleting it (#29206)
Relates to #15426
2018-03-23 15:53:36 +01:00
Nik Everett 8c59e43ac7
Docs: HighLevelRestClient#multiSearch (#29144)
Adds docs for `HighLevelRestClient#multiSearch`. Unlike the `multiGet`
docs these are much more sparse because multi-search doesn't support
setting many options on the `MultiSearchRequest` and instead just wraps
a list of `SearchRequest`s.

Closes #28389
2018-03-23 10:11:50 -04:00
Nicholas Knize fede633563 Add Z value support to geo_shape
This enhancement adds Z value support (source only) to geo_shape fields. If vertices are provided with a third dimension, the third dimension is ignored for indexing but returned as part of source. Like beofre, any values greater than the 3rd dimension are ignored.

closes #23747
2018-03-23 08:50:55 -05:00