Commit Graph

4291 Commits

Author SHA1 Message Date
Alan Woodward 424ecb3c7d
Add ability to index prefixes on text fields (#28290)
This adds the ability to index term prefixes into a hidden subfield, enabling prefix queries to be run without multitermquery rewrites. The subfield reuses the analysis chain of its parent text field, appending an EdgeNGramTokenFilter. It can be configured with minimum and maximum ngram lengths. Query terms with lengths outside this min-max range fall back to using prefix queries against the parent text field.

The mapping looks like this:

"my_text_field" : {
"type" : "text",
"analyzer" : "english",
"index_prefix" : { "min_chars" : 1, "max_chars" : 10 }
}

Relates to #27049
2018-01-30 08:26:56 +00:00
Nik Everett 66ff1b2a59
Tests: Wipe cluster settings after every test (#28410)
Cluster settings shouldn't leak into the next test.

I played with failing the test if it left over any settings but that
felt like it added more ceremony then it was worth. The advantage is
that any test that intentionally wants to leave settings in place after
the test would fail and require looking at but, so far as I can tell, we
don't have any such tests.
2018-01-29 11:47:04 -05:00
Nik Everett 3d19006cfa
Docs: Clear watermarks after setting them (#28402)
Clear the disk watermark after the snippet showing users how to set it.
Without this our tests will fail if the disks have less than 10GB free.

Closes #28325
2018-01-26 15:42:53 -05:00
Tanguy Leroux be74f11517
Replace jvm-example by two plugin examples (#28339)
This pull request replaces the jvm-example plugin (from the jvm/site plugins era) by two new plugins: a custom-settings that shows how to register and use custom settings (including secured settings) in a plugin, and rest-handler plugin that shows how to register a rest handler.

The two plugins now reside in the plugins/examples project. They can serve as sample plugins for users, a special attention has been put on documentation. The packaging tests have been adapted to use the custom-settings plugin.
2018-01-26 17:34:24 +01:00
Devin Young e8a78df555 Fix markdown formatting (#28392) 2018-01-26 08:15:16 -07:00
Jason Tedor 976a5090a0
Add note to docs regarding multi-get ordering
The implementation maintains the order of the original requests yet this
functionality is not documented. This commit adds a note to the docs
regarding the ordering of responses to an multi-get request.

Relates #28356
2018-01-26 09:07:12 -05:00
olcbean 0c83240b5f Java Api clean up: remove deprecated `isShardsAcked` (#28311)
This PR removes previously deprecated `isShardsAcked()` method in
favour of `isShardsAcknowledged()` on `CreateIndexResponse`, `CreateIndexClusterStateUpdateResponse` and `RolloverResponse` 

Related to #27784
Follow-up of #27819
2018-01-25 14:13:20 +01:00
Alex Moros Marco 261fb6a29e [Docs] Fix explanation for `from` and `size` example (#28320) 2018-01-25 11:40:39 +01:00
Jim Ferenczi c26d4ac6c1
Always return the after_key in composite aggregation response (#28358)
This change adds the `after_key` of a composite aggregation directly in the response.
It is redundant when all buckets are not filtered/removed by a pipeline aggregation since in this case the `after_key` is always the last bucket
in the response. Though when using a pipeline aggregation to filter composite buckets, the `after_key` can be lost if the last bucket is filtered.
This commit fixes this situation by always returning the `after_key` in a dedicated section.
2018-01-25 09:15:27 +01:00
Jim Ferenczi 65184d0b5b
Adds a note in the `terms` aggregation docs regarding pagination (#28360)
This change adds a note in the `terms` aggregation that explains how to retrieve **all**
terms (or all combinations of terms in a nested agg) using the `composite` aggregation.
2018-01-25 08:59:41 +01:00
Alex Moros Marco 090ac3c2a2 [Doc] Fixs typo in reverse-nested-aggregation.asciidoc (#28348) 2018-01-24 17:54:02 +01:00
Robin Stocker 64bbb3a235 [Docs] Clarify `html` encoder in highlighting.asciidoc (#27766)
The previous description was a bit confusing because the pre/post tags used for highlighting are not escaped, the rest of the content is.
2018-01-24 16:45:40 +01:00
Lukas Olson 7c5619a29a
Fix spelling error 2018-01-23 12:29:11 -07:00
Jim Ferenczi b2ce994be7 [Docs] Fix asciidoc style in composite agg docs 2018-01-23 16:41:32 +01:00
Jim Ferenczi 19cfc25873
Adds the ability to specify a format on composite date_histogram source (#28310)
This commit adds the ability to specify a date format on the `date_histogram` composite source.
If the format is defined, the key for the source is returned as a formatted date.

Closes #27923
2018-01-23 15:14:49 +01:00
Adrien Grand 700d9ecc95
Remove the `update_all_types` option. (#28288)
This option is not useful in 7.x since no indices may have more than one type
anymore.
2018-01-22 12:03:07 +01:00
Peter Dyson 1ae920cb90
Provide explanation of dangling indices, fixes #26008 (#26999)
* Provide explanation of dangling indices, fixes #26008

Adjusted from PR review comments

* updates to suggested wording and minor typo fix.
2018-01-22 18:39:21 +10:00
David Turner 0a4a4c8a0e
Minor improvements to translog docs (#28237)
The use of the phrase "translog" vs "transaction log" was inconsistent, and
it was apparently unclear that the translog was stored on every shard copy.
2018-01-19 10:17:22 +00:00
Andrew Kramarev ef468327e9 mistyping in one of the highlighting examples comment -> content (#28139) 2018-01-18 17:32:42 -05:00
David Kemp 531c58cf81 Documents applicability of term query to range type (#28166)
Closes #27030
2018-01-18 17:19:01 -05:00
Jin Liang 66c81e7f5e [Docs] Update tophits-aggregation.asciidoc (#28273) 2018-01-18 18:06:20 +01:00
David Shimon c92b42ef84 Docs: match between snippet to its description (#28296)
s/400/200/ in the text to match a snippet.
2018-01-18 09:31:02 -05:00
deepybee 48c8098e15 Fixed several typos in analyzers section (#28247) 2018-01-18 08:51:53 +00:00
Jim Ferenczi defb53a0bc
add a note regarding rescore and sort (#28251) 2018-01-18 09:23:19 +01:00
Christoph Büscher d4ac0026fc
[Docs] Clarify numeric datatype ranges (#28240)
Since #25826 we reject infinite values for float, double and half_float
datatypes. This change adds this restriction to the documentation for the
supported datatypes.

Closes #27653
2018-01-16 15:53:28 +01:00
fbsolo 71ba314c73 [Docs] Changes to ingest.asciidoc (#28212) 2018-01-16 09:36:19 +01:00
Nicholas Knize 5ed25f1e12 [GEO] Add WKT Support to GeoBoundingBoxQueryBuilder
Add WKT BBOX parsing support to GeoBoundingBoxQueryBuilder.
2018-01-15 13:30:51 -06:00
Igor Motov c75ac319a6
Add ability to associate an ID with tasks (#27764)
Adds support for capturing the X-Opaque-Id header from a REST request and storing it's value in the tasks that this request started. It works for all user-initiated tasks (not only search).

Closes #23250

Usage:
```
$ curl -H "X-Opaque-Id: imotov" -H "foo:bar" "localhost:9200/_tasks?pretty&group_by=parents"
{
  "tasks" : {
    "7qrTVbiDQKiZfubUP7DPkg:6998" : {
      "node" : "7qrTVbiDQKiZfubUP7DPkg",
      "id" : 6998,
      "type" : "transport",
      "action" : "cluster:monitor/tasks/lists",
      "start_time_in_millis" : 1513029940042,
      "running_time_in_nanos" : 266794,
      "cancellable" : false,
      "headers" : {
        "X-Opaque-Id" : "imotov"
      },
      "children" : [
        {
          "node" : "V-PuCjPhRp2ryuEsNw6V1g",
          "id" : 6088,
          "type" : "netty",
          "action" : "cluster:monitor/tasks/lists[n]",
          "start_time_in_millis" : 1513029940043,
          "running_time_in_nanos" : 67785,
          "cancellable" : false,
          "parent_task_id" : "7qrTVbiDQKiZfubUP7DPkg:6998",
          "headers" : {
            "X-Opaque-Id" : "imotov"
          }
        },
        {
          "node" : "7qrTVbiDQKiZfubUP7DPkg",
          "id" : 6999,
          "type" : "direct",
          "action" : "cluster:monitor/tasks/lists[n]",
          "start_time_in_millis" : 1513029940043,
          "running_time_in_nanos" : 98754,
          "cancellable" : false,
          "parent_task_id" : "7qrTVbiDQKiZfubUP7DPkg:6998",
          "headers" : {
            "X-Opaque-Id" : "imotov"
          }
        }
      ]
    }
  }
}
```
2018-01-12 15:34:17 -05:00
akadko 6a5807ad8f [DOCS] Removed differencies between text and code (#27993) 2018-01-12 10:36:48 -05:00
Andrew Banchich a10c406292 text fixes (#28136) 2018-01-12 10:21:39 -05:00
Boudewijn 3a96518b58 Update getting-started.asciidoc (#28145)
Replaced single quotation marks with double quotation marks surrounding localhost address
2018-01-12 10:21:16 -05:00
Christoph Büscher 39ff7b5a3f [Docs] Correct response json in rank-eval.asciidoc 2018-01-11 15:52:11 +01:00
Yu 228f7ffcdf Add scroll parameter to _reindex API (#28041)
Be able to change scroll timeout in _reindex API (by default: 5m)
2018-01-11 14:40:04 +01:00
Andrew Banchich e92acefba0 [Docs] Improvements in script-fields.asciidoc (#28174) 2018-01-11 10:59:27 +01:00
Ryan Ernst 3fb0cb91a9
Docs: Clarify password protection support with keystore (#28157)
closes #27932
2018-01-09 15:01:37 -08:00
Andrew Banchich a58dc8d82c [Docs] Fix Date Math example descriptions (#28125) 2018-01-08 16:58:12 +01:00
Christoph Büscher 556d77c9ad
[Docs] Add note on limitation for significant_text with nested objects (#28052)
Add section to `significant_text` documentation mentioning that it currently
does not support use on nested objects.

Relates to #28050
2018-01-03 16:28:23 +01:00
Sian Lerk Lau 5e3ba8a88d Enable convert processor to support Long and Double. (#27957)
Closes #23085
2018-01-03 11:27:55 +01:00
Jason Tedor a91da9a9af
Only bind loopback addresses when binding to local
* Only bind loopback addresses when binding to local

Today when binding to local (the default) we bind to any address that is
a loopback address, or any address on an interface that declares itself
as a loopback interface. Yet, not all addresses on loopback interfaces
are loopback addresses. This arises on macOS where there is a link-local
address assigned to the loopback interface (fe80::1%lo0) and in Docker
services where virtual IPs of the service are assigned to the loopback
interface (docker/libnetwork#1877). These situations cause problems:
 - because we do not handle the scope ID of a link-local address, we end
   up bound to an address for which publishing of that address does not
   allow that address to be reached (since we drop the scope)
 - the virtual IPs in the Docker situation are not loopback addresses,
   they are not link-local addresses, so we end up bound to interfaces
   that cause the bootstrap checks to be enforced even though the
   instance is only bound to local

We address this by only binding to actual loopback addresses, and skip
binding to any address on a loopback interface that is not a loopback
address. This lets us simplify some code where in the bootstrap checks
we were skipping link-local addresses, and in writing the ports file
where we had to skip link-local addresses because again the formatting
of them does not allow them to be connected to by another node (to be
clear, they could be connected to via the scope-qualified address, but
that information is not written out).

Relates #28029
2018-01-02 07:04:09 -05:00
Gytis Šk 86bffa870b Update fuzzy-query.asciidoc (#28032) 2018-01-01 08:44:04 +01:00
Mayya Sharipova dcde895f49
Introduce limit to the number of terms in Terms Query (#27968)
- Introduce index level settings to control the maximum number of terms
    that can be used in a Terms Query
- Throw an error if a request exceeds this max number

Closes #18829
2017-12-28 17:36:29 -05:00
Shaunak Kashyap da0ed578b2 Fixing typo in param name: values => sources (#28016) 2017-12-28 18:18:30 +01:00
Vlad Holubiev 31d4a4bf7c [DOCS] Fix link formatting (#27990) 2017-12-26 16:25:05 +00:00
Vlad Holubiev 7b14e4b8e0 [DOCS] Remove extra word (#27989) 2017-12-26 16:24:29 +00:00
Mayya Sharipova cbd271e497
Limit the analyzed text for highlighting (#27934)
* Limit the analyzed text for highlighting

- Introduce index level settings to control the max number of character
to be analyzed for highlighting
- Throw an error if analysis is required on a larger text

Closes #27517
2017-12-21 10:19:58 -05:00
Sian Lerk Lau 47eefbe889 Enable grok processor to support long, double and boolean (#27896) 2017-12-20 11:19:49 -08:00
David Pilato 3ca39186d1
Fix missing comma in examples (#27904) 2017-12-19 18:28:39 +01:00
Andrew Banchich 3b58582059 [Docs] Grammatical fix in important-settings.asciidoc (#27654) 2017-12-19 17:08:08 +01:00
sandstrom e542c03bea Clarify that number of threads is set by packages
This commit clarifies that on systemd the packages already set the number of
threads for the Elasticsearch process.

Relates #27840
2017-12-15 11:29:13 -05:00
Martijn van Groningen cef7bd2079
docs: add best practises for wildcard queries inside percolator queries 2017-12-15 10:49:59 +01:00
Christoph Büscher f3293879b5 [Docs] Improve rendering of ranking evaluation docs 2017-12-15 10:45:44 +01:00
Alex Benusovich 5f74e81f29 [Docs] Clarify version for restoring old indices. (#27800)
Sentence should say that an index created in 2.x cannot be restored in a 6.x cluster even if 5.x cluster created the snapshot.
2017-12-15 09:42:36 +01:00
Adrien Grand 1b660821a2
Allow `_doc` as a type. (#27816)
Allowing `_doc` as a type will enable users to make the transition to 7.0
smoother since the index APIs will be `PUT index/_doc/id` and `POST index/_doc`.
This also moves most of the documentation to `_doc` as a type name.

Closes #27750
Closes #27751
2017-12-14 17:47:53 +01:00
Christoph Büscher bb14b8f7c5 Merge branch 'rankeval'
This commit adds a new module that provides an endpoint that can be used to
evaluate search ranking results.

Closes #19195
2017-12-14 16:45:03 +01:00
Jason Tedor 10d2667498
Clarify using supported LTS versions of the Java
This commit clarifies that we recommended using supported LTS versions
of Java as opposed to supporting a minimum version and any version above
that.

Relates #27795
2017-12-14 07:47:50 -05:00
Sandeep Kanabar 7e0fc8a112 [Docs] Correct spelling in update-settings.asciidoc (#27808) 2017-12-14 10:16:50 +01:00
Glen Smith 94cfc2a0df [Docs] Fix explanation of "cluster.routing.allocation.exclude" (#27735) 2017-12-13 17:26:13 +01:00
Martijn van Groningen 442c3b8bcf
docs: fix link 2017-12-13 16:51:21 +01:00
Jason Tedor 2994366195 Reinsert paragraph about heap size
This paragraph was accidentally dropped when preparing to merge
setting the heap size.
2017-12-12 11:09:24 -05:00
Jason Tedor 008296e2b6
Reorganize configuring Elasticsearch docs
This commit reorganizes some of the content in the configuring
Elasticsearch section of the docs. The changes are:
 - move JVM options out of system configuration into configuring
   Elasticsearch
 - move JVM options to its own page of the docs
 - move configuring the heap to important Elasticsearch settings
 - move configuring the heap to its own page of the docs
 - move all important settings to individual pages in the docs
 - remove bootstrap.memory_lock from important settings, this is covered
   in the swap section of system configuration

Relates #27755
2017-12-12 10:24:37 -05:00
Andrew Banchich 1cd5575997 Update query-dsl.asciidoc (#27669) 2017-12-11 18:06:08 +01:00
Christoph Büscher 97b25f3b0c Merge branch 'master' into rankeval 2017-12-11 15:19:16 +01:00
olcbean 25c606cf09 Remove deprecated names for string distance algorithms (#27640)
#27409 deprecated the incorrectly-spelled `levenstein` in favour of `levenshtein`.
#27526 deprecated the inconsistent `jarowinkler` in favour of `jaro_winkler`.

These changes were merged into 6.2, and this change removes them entirely in 7.0.
2017-12-11 12:16:04 +00:00
Christoph Büscher 3d3a1d2a0d Adding short description for experimental status in docs 2017-12-08 15:12:15 +01:00
Christoph Büscher 52cb6c8ef2 Merge branch 'master' into rankeval 2017-12-07 14:22:46 +01:00
markwalkom ee21045697 [Docs] Add date math examples to api-conventions.asciidoc (#25217) 2017-12-07 14:10:51 +01:00
Jason Tedor 6c7374804f
Extend JVM options to support multiple versions
JDK 9 has removed JVM options that were valid in JDK 8 (e.g., GC logging
flags) and replaced them with new flags that are not available in JDK
8. This means that a single JVM options file can no longer apply to JDK
8 and JDK 9, complicating development, complicating our packaging story,
and complicating operations. This commit extends the JVM options syntax
to specify the range of versions the option applies to. If the running
JVM matches the range of versions, the flag will be used to start the
JVM otherwise the flag will be ignored.

We implement this parser in Java for simplicity, and with this we start
our first step towards a Java launcher.

Relates #27675
2017-12-06 18:03:13 -05:00
Jim Ferenczi caea6b70fa
Add a new cluster setting to limit the total number of buckets returned by a request (#27581)
This commit adds a new dynamic cluster setting named `search.max_buckets` that can be used to limit the number of buckets created per shard or by the reduce phase. Each multi bucket aggregator can consume buckets during the final build of the aggregation at the shard level or during the reduce phase (final or not) in the coordinating node. When an aggregator consumes a bucket, a global count for the request is incremented and if this number is greater than the limit an exception is thrown (TooManyBuckets exception).
This change adds the ability for multi bucket aggregator to "consume" buckets in the global limit, the default is 10,000. It's an opt-in consumer so each multi-bucket aggregator must explicitly call the consumer when a bucket is added in the response.

Closes #27452 #26012
2017-12-06 09:15:28 +01:00
Ryan Ernst c51e48bec0
Correct docs for binary fields and their default for doc values (#27680)
closes #27240
2017-12-05 15:10:18 -08:00
Deb Adair 2f9a882061 [DOCS] Fixed typos and broken attribute. 2017-12-05 11:46:40 -08:00
debadair 25ec068aef
[DOCS] Added link to upgrade guide and bumped the upgrade topic up to the top level (#27621)
* [DOCS] Added link to the upgrade guide & tweaked the intro.

* [DOCS] Bumped upgrade topic up to the top level of the TOC
2017-12-05 10:58:52 -08:00
Nicholas Knize 8bcf5393f2 [Geo] Add Well Known Text (WKT) Parsing Support to ShapeBuilders
This commit adds WKT support to Geo ShapeBuilders.

This supports the following format:

POINT (30 10)
LINESTRING (30 10, 10 30, 40 40)
BBOX (-10, 10, 10, -10)
POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))
POLYGON ((35 10, 45 45, 15 40, 10 20, 35 10), (20 30, 35 35, 30 20, 20 30))
MULTIPOINT ((10 40), (40 30), (20 20), (30 10))
MULTIPOINT (10 40, 40 30, 20 20, 30 10)
MULTILINESTRING ((10 10, 20 20, 10 40),(40 40, 30 30, 40 20, 30 10))
MULTIPOLYGON (((30 20, 45 40, 10 40, 30 20)), ((15 5, 40 10, 10 20, 5 10, 15 5)))
MULTIPOLYGON (((40 40, 20 45, 45 30, 40 40)), ((20 35, 10 30, 10 10, 30 5, 45 20, 20 35), (30 20, 20 15, 20 25, 30 20)))
GEOMETRYCOLLECTION (POINT (30 10), MULTIPOINT ((10 40), (40 30), (20 20), (30 10)))

closes #9120
2017-12-05 10:56:41 -06:00
Andrew Banchich 1be286c592 [Docs] Grammatical fix in bootstrap-checks.asciidoc (#27655) 2017-12-05 11:46:19 +01:00
Clinton Gormley 0bba2a8438 Update removal_of_types.asciidoc
Corrected  `include_in_type` to `include_type_name`
2017-12-05 10:44:48 +01:00
Christoph Büscher bbec33d35c Merge branch 'master' into rankeval 2017-12-04 12:57:19 +01:00
Christoph Büscher c4fe7d3f72 [Docs] add deprecation warning for `delimited_payload_filter` renaming 2017-12-04 10:22:05 +01:00
Jason Tedor cd67f6a8d7
Enable GC logs by default
For too long we have been groping around in the dark when faced with GC
issues because we rarely have GC logs at our disposal. This commit
enables GC logging by default out of the box.

Relates #27610
2017-12-03 08:33:21 -05:00
Clinton Gormley e1aa6e2cda Fix cluster usage docs test
#27611 broke the docs tests because $node_name in the URL doesn't (#27616)seem to be replaced.

Changing this to a * to match all nodes seems to fix the test
2017-12-01 16:55:10 +01:00
Lee Hinman 623d3700f0
Add accounting circuit breaker and track segment memory usage (#27116)
* Add accounting circuit breaker and track segment memory usage

This commit adds a new circuit breaker "accounting" that is used for tracking
the memory usage of non-request-tied memory users. It also adds tracking for the
amount of Lucene segment memory used by a shard as a user of the new circuit
breaker.

The Lucene segment memory is updated when the shard refreshes, and removed when
the shard relocates away from a node or is deleted. It should also be noted that
all tracking for segment memory uses `addWithoutBreaking` so as not to fail the
shard if a limit is reached.

The `accounting` breaker has a default limit of 100% and will contribute to the
parent breaker limit.

Resolves #27044
2017-12-01 07:59:45 -07:00
Martijn van Groningen 6cda5b292c
docs: add paragraph about using `percolate` query in a filter context 2017-12-01 10:55:01 +01:00
佛陀.RML 756e170674 [Docs] Fix order of nodes usage example (#27611) 2017-12-01 10:42:42 +01:00
Denis Gladkikh 95bcee56c4 Add note to keystore docks on requirement for restart to take effect (#27488) 2017-11-30 14:23:40 -08:00
Mayya Sharipova c6b73239ae
Limit the number of tokens produced by _analyze (#27529)
Add an index level setting `index.analyze.max_token_count` to control
the number of generated tokens in the  _analyze endpoint.
Defaults to 10000.

Throw an error if the number of generated tokens exceeds this limit.

Closes #27038
2017-11-30 11:54:39 -05:00
olcbean d25c9671de Deprecate `jarowinkler` in favor of `jaro_winkler` (#27526)
Jaro and Winkler are two people, so we should use the same naming convention as for Damerau–Levenshtein.
2017-11-30 12:49:34 +00:00
Philipp Krenn 64ca0fe9bb Update docs regarding SHA-512 checksums
This commit updates the docs for the new SHA-512 checksums that are
supported for official plugins.

Relates #27524
2017-11-29 21:29:06 -05:00
Jason Tedor 6655689b15 Move DNS cache docs to system configuration docs
When these docs were moved they should have been moved to the system
configuration docs. This commit does that, and also fixes a missing
heading that broke the docs build.
2017-11-29 19:57:26 -05:00
Jason Tedor ff3c19ed13
Move DNS cache settings to important configuration
This commit moves the DNS cache settings for the JVM to the important
settings section of the docs.

Relates #27592
2017-11-29 18:02:26 -05:00
Martijn van Groningen dbf17152d1
docs: use `doc_value_fields` fields as alternative for nested inner hits _source fetching
instead of stored fields as doc values are more likely to be enabled by default
2017-11-29 17:31:39 +01:00
Christoph Büscher 35688f6441 Merge branch 'master' into rankeval 2017-11-29 15:24:06 +01:00
Clinton Gormley 65e602c2be Update index-modules.asciidoc
Docs: Clarified `blocks.write` vs `blocks.read_only`
2017-11-29 13:05:12 +01:00
Christoph Büscher 0d11b9fe34
[Docs] Unify spelling of Elasticsearch (#27567)
Removes occurences of "elasticsearch" or "ElasticSearch" in favour of
"Elasticsearch" where appropriate.
2017-11-29 09:44:25 +01:00
Kanako Nakai 23f85fe6d4 Fix max number of threads bootstrap docs
Previously the bootstrap check for max number of threads was increased
from 2048 to 4096 yet the docs were never adjusted for this change. This
commit addresses this so the docs are in-line with the limit enforced in
the bootstrap check.

Relates #27511
2017-11-28 22:19:04 -05:00
Martijn van Groningen cb1204774b
Include the _index, _type and _id to nested search hits in the top_hits and inner_hits response.
Also include _type and _id for parent/child hits inside inner hits.

In the case of top_hits aggregation the nested search hits are
directly returned and are not grouped by a root or parent document, so
it is important to include the _id and _index attributes in order to know
to what documents these nested search hits belong to.

Closes #27053
2017-11-28 14:05:29 +01:00
David Turner a165d1df40
Minor improvements to docs for numeric types (#27553)
* Caps
* Fix awkward wording that took multiple passes to parse
* Floating point _number_
* Something more descriptive about the `scaled_float` scaling factor.
2017-11-28 11:36:07 +00:00
Jason Tedor d8c28044da
Forbid granting the all permission in production
Running with the all permission java.security.AllPermission granted is
equivalent to disabling the security manager. This commit adds a
bootstrap check that forbids running with this permission granted.

Relates #27548
2017-11-27 16:05:27 -05:00
Simon Willnauer f23ed6188d
Skip shard refreshes if shard is `search idle` (#27500)
Today we refresh automatically in the background by default very second.
This default behavior has a significant impact on indexing performance
if the refreshes are not needed.
This change introduces a notion of a shard being `search idle` which a
shard transitions to after (default) `30s` without any access to an
external searcher. Once a shard is search idle all scheduled refreshes
will be skipped unless there are any refresh listeners registered.
If a search happens on a `serach idle` shard the search request _park_
on a refresh listener and will be executed once the next scheduled refresh
occurs. This will also turn the shard into the `non-idle` state immediately.

This behavior is only applied if there is no explicit refresh interval set.
2017-11-27 18:16:10 +01:00
lcawley af971b3081 [DOCS] Fixed broken link in breaking changes 2017-11-24 09:16:14 -08:00
Christoph Büscher 5661b1c3df Merge branch 'master' into rankeval 2017-11-24 16:25:05 +01:00
kel 4885acb048 Replace `delimited_payload_filter` by `delimited_payload` (#26625)
The `delimited_payload_filter` is renamed to `delimited_payload`, the old name is 
deprecated and should be replaced by `delimited_payload`.

Closes #21978
2017-11-24 13:03:19 +01:00
Nhat Nguyen 46b508d6c9
Add wait_for_no_initializing_shards to cluster health API (#27489)
This adds a new option to the cluster health request allowing to wait
until there is no initializing shards.

Closes #25623
2017-11-23 15:09:58 -05:00
Clinton Gormley d1b1d711df Update composite-aggregation.asciidoc
Fixed asciidoc typo
2017-11-23 15:05:14 +01:00
olcbean fd564b10db Deprecate `levenstein` in favor of `levenshtein` (#27409)
Support both spellings thoughout 6.x, reporting the incorrect one as deprecated.
2017-11-23 12:53:47 +00:00
Christoph Büscher 5735477283 Fix some documentation typos 2017-11-23 12:31:25 +01:00
Simon Willnauer fadbe0de08
Automatically prepare indices for splitting (#27451)
Today we require users to prepare their indices for split operations.
Yet, we can do this automatically when an index is created which would
make the split feature a much more appealing option since it doesn't have
any 3rd party prerequisites anymore.

This change automatically sets the number of routinng shards such that
an index is guaranteed to be able to split once into twice as many shards.
The number of routing shards is scaled towards the default shard limit per index
such that indices with a smaller amount of shards can be split more often than
larger ones. For instance an index with 1 or 2 shards can be split 10x
(until it approaches 1024 shards) while an index created with 128 shards can only
be split 3x by a factor of 2. Please note this is just a default value and users
can still prepare their indices with `index.number_of_routing_shards` for custom
splitting.

NOTE: this change has an impact on the document distribution since we are changing
the hash space. Documents are still uniformly distributed across all shards but since
we are artificually changing the number of buckets in the consistent hashign space
document might be hashed into different shards compared to previous versions.

This is a 7.0 only change.
2017-11-23 09:48:54 +01:00
Mayya Sharipova 57e4d10007
Limit the number of nested documents (#27405)
Add an index level setting `index.mapping.nested_objects.limit` to control
the number of nested json objects that can be in a single document
across all fields. Defaults to 10000.

Throw an error if the number of created nested documents exceed this
limit during the parsing of a document.

Closes #26962
2017-11-22 10:16:28 -05:00
Takumasa Ochi eed8d1aee5 [DOC] Fix mathematical representation on interval (range) (#27450) 2017-11-21 17:06:26 +00:00
Christoph Büscher d979ccace9 Merge branch 'master' into rankeval 2017-11-21 14:11:02 +01:00
Christoph Büscher 3348d2317f Reworking javadocs, minor changes in some implementation classes 2017-11-21 14:09:04 +01:00
Christoph Büscher 5c65a59369 Extending rank_eval asciidocs 2017-11-21 14:08:42 +01:00
Christoph Büscher d9e67a2c95 Extending `_rank_eval` documentation 2017-11-21 14:08:28 +01:00
Luca Cavanna 29450de7b5
Cross Cluster Search: make remote clusters optional (#27182)
Today Cross Cluster Search requires at least one node in each remote cluster to be up once the cross cluster search is run. Otherwise the whole search request fails despite some of the data (either local and/or remote) is available. This happens when performing the _search/shards calls to find out which remote shards the query has to be executed on. This scenario is different from shard failures that may happen later on when the query is actually executed, in case e.g. remote shards are missing, which is not going to fail the whole request but rather yield partial results, and the _shards section in the response will indicate that.

This commit introduces a boolean setting per cluster called search.remote.$cluster_alias.skip_if_disconnected, set to false by default, which allows to skip certain clusters if they are down when trying to reach them through a cross cluster search requests. By default all clusters are mandatory.

Scroll requests support such setting too when they are first initiated (first search request with scroll parameter), but subsequent scroll rounds (_search/scroll endpoint) will fail if some of the remote clusters went down meanwhile.

The search API response contains now a new _clusters section, similar to the _shards section, that gets returned whenever one or more clusters were disconnected and got skipped:

"_clusters" : {
    "total" : 3,
    "successful" : 2,
    "skipped" : 1
}
Such section won't be part of the response if no clusters have been skipped.

The per cluster skip_unavailable setting value has also been added to the output of the remote/info API.
2017-11-21 11:41:47 +01:00
Ulrich Reffle dd0bb580b0 [Docs] Fix broken bulleted lists (#27470) 2017-11-21 11:10:35 +01:00
Jim Ferenczi d1093bd2fa #26800: Fix docs rendering 2017-11-20 08:41:02 +01:00
Michael Basnight 2949c53174
Remove config prompting for secrets and text (#27216)
This commit removes the ability to use ${prompt.secret} and
${prompt.text} as valid config settings. Secure settings has obsoleted
the need for this, and it cleans up some of the code in Bootstrap.
2017-11-19 22:33:17 -06:00
K. Daniel Newton 365dda8748 Correct usage of "an" to "a" in getting started docs
This commit corrects a word usage error in the getting started
docs. Since pronunciation is what determines when to use either "a" or
"an" and the word "ubiquitous" is pronounced /yo͞oˈbikwədəs/, it should
be preceded by "a."

Relates #27420
2017-11-18 07:36:43 -05:00
Mayya Sharipova 858b2c7cb8
Standardize underscore requirements in parameters (#27414)
Stardardize underscore requirements in parameters across different type of
requests:
_index, _type, _source, _id keep their underscores
params like version and retry_on_conflict will be without underscores
Throw an error if older versions of parameters are used

BulkRequest, MultiGetRequest, TermVectorcRequest, MoreLikeThisQuery
were changed

Closes #26886
2017-11-17 15:31:52 -05:00
Simon Willnauer a5df2ef538 peanut butter hamburgers 2017-11-17 20:51:39 +01:00
Jim Ferenczi 53462f6499
Make fields optional in multi_match query and rely on index.query.default_field by default (#27380)
* Make fields optional in multi_match query and rely on index.query.default_field by default

This commit adds the ability to send `multi_match` query without providing any `fields`.
When no fields are provided the `multi_match` query will use the fields defined in the index setting `index.query.default_field`
(which in turns defaults to `*`).
The same behavior is already implemented in `query_string` and `simple_query_string` so this change just applies
the heuristic to `multi_match` queries.
Relying on `index.query.default_field` rather than `*` is safer for big mappings that break the 1024 field expansion limit added in 7.0 for all
text queries. For these kind of mappings the admin can change the `index.query.default_field` in order to make sure that exploratory queries using
`multi_match`, `query_string` or `simple_query_string` do not throw an exception.
2017-11-17 10:25:21 +01:00
Jim Ferenczi 623367d793
Add composite aggregator (#26800)
* This change adds a module called `aggs-composite` that defines a new aggregation named `composite`.
The `composite` aggregation is a multi-buckets aggregation that creates composite buckets made of multiple sources.
The sources for each bucket can be defined as:
  * A `terms` source, values are extracted from a field or a script.
  * A `date_histogram` source, values are extracted from a date field and rounded to the provided interval.
This aggregation can be used to retrieve all buckets of a deeply nested aggregation by flattening the nested aggregation in composite buckets.
A composite buckets is composed of one value per source and is built for each document as the combinations of values in the provided sources.
For instance the following aggregation:

````
"test_agg": {
  "terms": {
    "field": "field1"
  },
  "aggs": {
    "nested_test_agg":
      "terms": {
        "field": "field2"
      }
  }
}
````
... which retrieves the top N terms for `field1` and for each top term in `field1` the top N terms for `field2`, can be replaced by a `composite` aggregation in order to retrieve **all** the combinations of `field1`, `field2` in the matching documents:

````
"composite_agg": {
  "composite": {
    "sources": [
      {
	"field1": {
          "terms": {
              "field": "field1"
            }
        }
      },
      {
	"field2": {
          "terms": {
            "field": "field2"
          }
        }
      },
    }
  }
````

The response of the aggregation looks like this:

````
"aggregations": {
  "composite_agg": {
    "buckets": [
      {
        "key": {
          "field1": "alabama",
          "field2": "almanach"
        },
        "doc_count": 100
      },
      {
        "key": {
          "field1": "alabama",
          "field2": "calendar"
        },
        "doc_count": 1
      },
      {
        "key": {
          "field1": "arizona",
          "field2": "calendar"
        },
        "doc_count": 1
      }
    ]
  }
}
````

By default this aggregation returns 10 buckets sorted in ascending order of the composite key.
Pagination can be achieved by providing `after` values, the values of the composite key to aggregate after.
For instance the following aggregation will aggregate all composite keys that sorts after `arizona, calendar`:

````
"composite_agg": {
  "composite": {
    "after": {"field1": "alabama", "field2": "calendar"},
    "size": 100,
    "sources": [
      {
	"field1": {
          "terms": {
            "field": "field1"
          }
        }
      },
      {
	"field2": {
          "terms": {
            "field": "field2"
          }
	}
      }
    }
  }
````

This aggregation is optimized for indices that set an index sorting that match the composite source definition.
For instance the aggregation above could run faster on indices that defines an index sorting like this:

````
"settings": {
  "index.sort.field": ["field1", "field2"]
}
````

In this case the `composite` aggregation can early terminate on each segment.
This aggregation also accepts multi-valued field but disables early termination for these fields even if index sorting matches the sources definition.
This is mandatory because index sorting picks only one value per document to perform the sort.
2017-11-16 15:13:36 +01:00
Jim Ferenczi bf72858ce8
[Docs] Restore section about multi-level parent/child relation in parent-join (#27392)
This section was removed to hide this ability to new users.
This change restores the section and adds a warning regarding the expected performance.

Closes #27336
2017-11-16 11:29:16 +01:00
Russ Cam c42899b27e
Docs/windows installer (#27369)
* Add additional command line parameters along with important note for INSTALLDIR when upgrading
* Update windows installer images
2017-11-15 21:35:54 +11:00
Alexander Reelsen 66b5a43d0e
Logging: Unify log rotation for index/search slow log (#27298)
The existing log rotation configuration allowed the index
and search slow log to grow unbounded. This commit removes the
date based rotation and adds the same size based rotation, that
the depreciation log already has.
2017-11-15 10:01:32 +01:00
Tal Levy 5c34533761
add json-processor support for non-map json types (#27335)
The Json Processor originally only supported parsing field values into Maps even
though the JSON spec specifies that strings, null-values, numbers, booleans, and arrays
are also valid JSON types. This commit enables parsing these values now.

response to #25972.
2017-11-13 10:28:19 -08:00
Alexander Reelsen 08037eebff
Tests: Improve size regex in documentation test (#26879)
The regex has been changed to not only be able to deal with something
like `260b`, but also support `1.1kb`.
2017-11-13 10:21:53 +01:00
lcawley 3ed558d718 [DOCS] Fixed link to docker content 2017-11-10 12:10:28 -08:00
Lisa Cawley 9f43d7329b
[DOCS] Move X-Pack-specific Docker content (#27333) 2017-11-10 09:38:32 -08:00
Jim Ferenczi 29331f1127
Fail queries with scroll that explicitely set request_cache (#27342)
Queries that create a scroll context cannot use the cache.
They modify the search context during their execution so using the cache
can lead to duplicate result for the next scroll query.

This change fails the entire request if the request_cache option is explictely set
on a query that creates a scroll context (`scroll=1m`) and make sure internally that we never
use the cache for these queries when the option is not explicitely used.
For 6.x a deprecation log will be printed instead of failing the entire request and the request_cache hint
will be ignored (forced to false).
2017-11-10 16:02:06 +01:00
Martijn van Groningen b4048b4e7f
Use CoveringQuery to select percolate candidate matches and
extract all clauses from a conjunction query.

When clauses from a conjunction are extracted the number of clauses is
also stored in an internal doc values field (minimum_should_match field).
This field is used by the CoveringQuery and allows the percolator to
reduce the number of false positives when selecting candidate matches and
in certain cases be absolutely sure that a conjunction candidate match
will match and then skip MemoryIndex validation. This can greatly improve
performance.

Before this change only a single clause was extracted from a conjunction
query. The percolator tried to extract the clauses that was rarest in order
(based on term length) to attempt less candidate queries to be selected
in the first place. However this still method there is still a very high
chance that candidate query matches are false positives.

This change also removes the influencing query extraction added via #26081
as this is no longer needed because now all conjunction clauses are extracted.

https://www.elastic.co/guide/en/elasticsearch/reference/6.x/percolator.html#_influencing_query_extraction

Closes #26307
2017-11-10 07:44:42 +01:00
Nicholas Knize 06ff92d237 Add ignore_malformed to geo_shape fields
This commit adds ignore_malformed support to geo_shape field types to skip malformed geoJson fields.

closes #23747
2017-11-09 17:59:05 -06:00
Dimitris Athanasiou 66bef26495
Aggregations: bucket_sort pipeline aggregation (#27152)
This commit adds a parent pipeline aggregation that allows
sorting the buckets of a parent multi-bucket aggregation.

The aggregation also offers [from] and [size] parameters
in order to truncate the result as desired.

Closes #14928
2017-11-09 17:59:57 +00:00
Tal Levy d22fd4ea58
Introduce templating support to timezone/locale in DateProcessor (#27089)
Sometimes systems like Beats would want to extract the date's timezone and/or locale
from a value in a field of the document. This PR adds support for mustache templating
to extract these values.

Closes #24024.
2017-11-09 09:45:32 -08:00
Mayya Sharipova abbe853f1e
Add limits for ngram and shingle settings (#27211) (#27318)
Relates to #25887
2017-11-08 10:12:57 -05:00
Mayya Sharipova 148376c2c5
Add limits for ngram and shingle settings (#27211)
* Add limits for ngram and shingle settings (#27211)

Create index-level settings:
max_ngram_diff - maximum allowed difference between max_gram and min_gram in
NGramTokenFilter/NGramTokenizer. Default is 1.
max_shingle_diff - maximum allowed difference between max_shingle_size and
 min_shingle_size in ShingleTokenFilter.  Default is 3.

Throw an IllegalArgumentException when
trying to create NGramTokenFilter, NGramTokenizer, ShingleTokenFilter
where difference between max_size and min_size exceeds the settings value.

Closes #25887
2017-11-07 08:14:55 -05:00
Zachary Tong 6e9e07d6f8
Fix profiling naming issues (#27133)
Some code-paths use anonymous classes (such as NonCollectingAggregator
in terms agg), which messes up the display name of the profiler.  If
we encounter an anonymous class, we need to grab the super's name.

Another naming issue was that ProfileAggs were not delegating to the
wrapped agg's name for toString(), leading to ugly display.

This PR also fixes up the profile documentation.  Some of the examples were
executing against empty indices, which shows different profile results
than a populated index (and made for confusing examples).

Finally, I switched the agg display names from the fully qualified name
to the simple name, so that it's similar to how the query profiles work.

Closes #26405
2017-11-06 16:37:33 -05:00
Shubham Aggarwal 5a925cd40c Fixed references to Multi Index Syntax (#27283) 2017-11-06 19:15:36 +01:00
Boris Tyukin 8e9b30417c Update to support bulk updates by query (#27172)
Getting started doc stated that bulk updates by query are not supported but they are now
2017-11-06 17:32:20 +01:00
Boaz Leskes a8ff4960f3 add split index reference in indices.asciidoc
Relates to #26931
2017-11-06 12:55:41 +01:00
Simon Willnauer bd7efa908a Add ability to split shards (#26931)
This change adds a new `_split` API that allows to split indices into a new
index with a power of two more shards that the source index.  This API works
alongside the `_shrink` API but doesn't require any shard relocation before
indices can be split.

The split operation is conceptually an inverse `_shrink` operation since we
initialize the index with a _syntetic_ number of routing shards that are used
for the consistent hashing at index time. Compared to indices created with
earlier versions this might produce slightly different shard distributions but
has no impact on the per-index backwards compatibility.  For now, the user is
required to prepare an index to be splittable by setting the
`index.number_of_routing_shards` at index creation time.  The setting allows the
user to prepare the index to be splittable in factors of
`index.number_of_routing_shards` ie. if the index is created with
`index.number_of_routing_shards: 16` and `index.number_of_shards: 2` it can be
split into `4, 8, 16` shards. This is an intermediate step until we can make
this the default. This also allows us to safely backport this change to 6.x.

The `_split` operation is implemented internally as a DeleteByQuery on the
lucene level that is executed while the primary shards execute their initial
recovery. Subsequent merges that are triggered due to this operation will not be
executed immediately. All merges will be deferred unti the shards are started
and will then be throttled accordingly.

This change is intended for the 6.1 feature release but will not support pre-6.1
indices to be split unless these indices have been shrunk before. In that case
these indices can be split backwards into their original number of shards.
2017-11-06 11:37:55 +01:00
Pablo Musa 7b03d68f9f [Docs] Fix minor paragraph indentation error for multiple Indices params (#25535) 2017-11-06 10:20:20 +01:00
Nhat c7ce5a07f2
Add size-based condition to the index rollover API (#27160)
This is to add a max_size condition to the index rollover API. We use
a totalSizeInBytes from DocsStats to evaluate this condition.

Closes #27004
2017-11-04 19:51:48 -04:00
Loek van Gool 67e677f443
Add an example of dynamic field names (#27255) 2017-11-03 23:20:58 +01:00
David Turner fbf8c3ee83
Reinstate recommendation for ≥ 3 master-eligible nodes. (#27204)
In the docs for 1.7 ([doc][doc-1.7], [src][src-1.7]) there was a recommendation
for at least 3 master-eligible nodes "in critical clusters" but this was lost
when that page was updated in 2.0 ([doc][doc-2.0], [src][src-2.0]). I'd like to
reinstate this.

[doc-1.7]: https://www.elastic.co/guide/en/elasticsearch/reference/1.7/modules-node.html
[src-1.7]: 2cbaccb2f2/docs/reference/modules/node.asciidoc
[doc-2.0]: https://www.elastic.co/guide/en/elasticsearch/reference/2.0/modules-node.html#split-brain
[src-2.0]: 4799009ad7/docs/reference/modules/node.asciidoc
2017-11-03 08:48:48 +00:00
Yannick Welsch 7791e72626
Add additional explanations around discovery.zen.ping_timeout (#27231)
Makes it clearer that this setting should only be changed with extra care.
2017-11-02 16:52:10 +01:00
Martijn van Groningen d805c41b28
Added new terms_set query
This query returns documents that match with at least one ore more
of the provided terms. The number of terms that must match varies
per document and is either controlled by a minimum should match
field or computed per document in a minimum should match script.

Closes #26915
2017-11-01 10:55:18 +01:00
Toby McLaughlin b71f7d3559
Update Docker docs for 6.0.0-rc2 (#27166)
* Update Docker docs for 6.0.0-rc2

* Update the docs to match the new Docker "image flavours" of "basic",
"platinum", and "oss".

* Clarifications for Openshift and bind-mounts

* Bump docker-compose 2.x format to 2.2

* Combine Docker Toolbox instructions for setting vm.max_map_count for
  both macOS + Windows

* devicemapper is not the default storage driver any more on RHEL
2017-11-01 14:24:30 +11:00
Igor Motov d14486bce6
Docs: restore now fails if it encounters incompatible settings (#26933)
This change was introduced in 5.0.0, but the documentation wasn't updated to reflect it.

Closes #26453
2017-10-31 20:04:00 -04:00
javanna 506a2c276d [DOCS] Link remote info API in Cross Cluster Search docs page
Closes #26327
2017-10-31 15:24:46 +01:00
Shai Erera bd0261916c Fix Laplace scorer to multiply by alpha (and not add) (#27125) 2017-10-31 13:08:44 +01:00
javanna 34666844b3 [DOCS] Clarify migrate guide and search request validation
Relates to  #26811
2017-10-31 12:36:00 +01:00
kel c3e2bdf20c Raise IllegalArgumentException if query validation failed (#26811)
Closes #26799
2017-10-31 12:17:27 +01:00
Jim Ferenczi 792641a6e3 [Docs] #26541: add warning regarding the limit on the number of fields that can be queried at once in the multi_match query. 2017-10-30 18:03:56 +01:00
Dimitrios Athanasiou 3796471ac4 [Docs] Fix note in bucket_selector 2017-10-30 15:20:46 +00:00
Clarkie b1ce5cf836 [Docs] Fix indentation of examples (#27168) 2017-10-30 11:56:38 +01:00
Jim Ferenczi a4105c6b4a
[Docs] Clarify `span_not` query behavior for non-overlapping matches (#27150)
Closes #27134
2017-10-30 11:29:40 +01:00
Christoph Büscher 8e62314ce4
[Docs] Remove first person "I" from getting started (#27155)
Avoid first person style and consistently switch to an unpersonal style in the getting started docs.
2017-10-30 10:45:50 +01:00
Holger Bartnick aa03fb72b7 [Docs] Correct link target for datatype murmur3 (#27143) 2017-10-30 09:31:55 +01:00
Clinton Gormley 0499dc0873 Removed the beta tag from cross-cluster search 2017-10-27 08:51:36 +02:00
Martijn van Groningen f1e944a675
docs: describe parent/child performances 2017-10-26 11:49:13 +02:00
markwalkom 2b864156ca [Docs] Clarify mapping `index` option default (#27104) 2017-10-25 12:42:29 +02:00
David Turner 559fc5a4de Update numbers to reflect 4-byte UTF-8-encoded characters (#27083)
You need 4 bytes for characters outside the BMP, which includes many emoji and
a bunch of less-common writing characters too.
2017-10-24 09:50:47 +01:00
Martijn van Groningen 87c9b79b10
Return the _source of inner hit nested as is without wrapping it into its full path context
Due to a change happened via #26102 to make the nested source consistent
with or without source filtering, the _source of a nested inner hit was
always wrapped in the parent path. This turned out to be not ideal for
users relying on the nested source, as it would require additional parsing
on the client side. This change fixes this, the _source of nested inner hits
is now no longer wrapped by parent json objects, irregardless of whether
the _source is included as is or source filtering is used.

Internally source filtering and highlighting relies on the fact that the
_source of nested inner hits are accessible by its full field path, so
in order to now break this, the conversion of the _source into its binary
form is performed in FetchSourceSubPhase, after any potential source filtering
is performed to make sure the structure of _source of the nested inner hit
is consistent irregardless if source filtering is performed.

PR for #26944

Closes #26944
2017-10-19 12:04:56 +02:00
İsmail Arılık 71f5e2ce6b Fix a typo. (#27043)
`=== Instalation with Homebrew` should be `=== Installation with Homebrew`.
2017-10-18 09:46:53 -04:00
Divyum Rastogi 984731f36b [DOCS] better formatting of ES cluster status (#26838)
* better formatting of ES cluster status

* change phrase missing data
2017-10-18 01:40:21 -06:00
Pius 400480e3b0 action.auto_create_index can be set as a dynamic cluster setting (#27026)
Per https://github.com/elastic/elasticsearch/pull/20274, action.auto_create_index can be set as a dynamic cluster setting.
2017-10-17 20:44:18 +00:00
Anton Pozhidaev 70668dddf3 Update docs about `script` parameter (#27010)
Added a description of short script form. Also removed references to the obsolete `script.default_lang`.
2017-10-16 05:04:43 -07:00
Simon Willnauer 8dda827ff4 Don't refresh on `_flush` `_force_merge` and `_upgrade` (#27000)
Today all these API calls have a sideeffect of making documents visible
to search requests. While this is sometimes desired it's an unnecessary sideeffect
and now that we have an internal (engine-private) index reader (#26972) we artificially
add a refresh call for bwc. This change removes this sideeffect in 7.0.
2017-10-16 10:16:35 +02:00
Jason Tedor 8eba1fa17c Add docs on full_id parameter in cat nodes API
This commit adds a note to the docs on the full_id parameter in the cat
nodes API. This is a useful parameter but was not previously documented
anywhere.

Relates #27009
2017-10-13 13:49:25 -04:00
Jason Tedor a7895839a0 Reformat paragraph in template docs to 80 columns
This commit reformats a paragraph in the template docs to fit in 80
columns as for the rest of the doc, and as-is a standard that we loosely
adhere to.
2017-10-12 17:52:43 -04:00
Pius 1125bc635c Clarify settings and template on create index
This commit clarifies the interaction between settings specified in a
create index request, and those that would come from any templates that
apply to the create index request.

Relates #26994
2017-10-12 17:48:57 -04:00
agent5566 93a47cf860 Fix a typo in the similarity docs (#26970) 2017-10-12 09:29:25 -07:00
Alexander Kazakov 592ab043dd Change default value to true for transpositions parameter of fuzzy query (#26901) 2017-10-11 15:31:48 +02:00
Nicolas Sierra d6fc4affae Clarify systemd overrides
This commit clarifies how to apply an override to the systemd unit file
for Elasticsearch.

Relates #26950
2017-10-10 13:06:34 -04:00
vurple b3e9aa89dc Add Homebrew instructions to getting started
This commit adds instructions for installing Elasticsearch via Homebrew
to the Getting Started guide.

Relates #26847
2017-10-10 06:21:33 -04:00
Nhat bf4c3642b2 remove _primary and _replica shard preferences (#26791)
The shard preference _primary, _replica and its variants were useful
for the asynchronous replication. However, with the current impl, they
are no longer useful and should be removed.

Closes #26335
2017-10-08 11:03:06 -04:00
shaulzorea 9db21cd23f fixing typo in datehistogram-aggregation.asciidoc (#26924) 2017-10-08 15:12:43 +02:00
Deb Adair b57cb83567 [DOCS] Added info about snapshotting your data before an upgrade. 2017-10-06 12:14:26 -07:00
Adrien Grand 4e1ff8d086 Add documentation about disabling `_field_names`. (#26813)
This field has significant index-time overhead.

Closes #26779
2017-10-06 16:49:15 +02:00
Clinton Gormley eb3ead6561 Update type-field.asciidoc
Fixed asciidoc syntax on deprecated annotation
2017-10-06 11:57:27 +02:00
Steve Kotsopoulos dd95849b62 Document JVM option MaxFDLimit for macOS ()
This commit documents a JVM option that is needed on macOS when raising
file descriptor limits there.

Relates #26900
2017-10-05 14:56:15 -04:00
Md. Abdulla-Al-Sun a40c474e10
Added Bengali Analyzer to Elasticsearch with respect to the lucene update(PR#238) 2017-10-05 13:25:05 +02:00
Alexander Kazakov 9c95e91471 Expose `fuzzy_transpositions` parameter in fuzzy queries (#26870)
Add fuzzy_transpositions parameter to multi_match and query_string queries.
Add fuzzy_transpositions, fuzzy_prefix_length and fuzzy_max_expansions
parameters to simple_query_string query.
2017-10-05 09:01:09 +02:00
Jim Ferenczi 17b9baf5fd Clarify pure wilcard matching with `query_string` (#26814)
In 5.x pure wildcard queries `*` in `query_string` are rewritten to `exists` query for efficiency.
Though this introduced a change in the document that match such queries because
`exists` query also return documents with an empty value for the field.
This change clarifies this behavior for 5.x and beyond.

Closes #26801

* review
2017-10-04 09:55:26 +02:00
Shane Connelly b33c444db5 Shows how to disable CCS from dedicated master/data (#26860)
This is really just the last bit of the OSS component of https://github.com/elastic/elasticsearch/issues/25210
2017-10-03 06:15:30 -07:00
David Roberts a292740b9e Add cgroup memory usage/limit to OS stats on Linux (#26166)
This change adds cgroup memory usage/limit to the OS stats section of
the node stats on Linux.  This information is useful because in Docker
containers the standard node stats report the host memory limit, not
taking account of extra restrictions that may have been applied to the
container.

The original idea was to store these values as Long, truncating any values
outside the range of long.  However, this meant that in the relatively common
case of no limit being applied, users would not see the same value in the OS
stats as they see by querying Linux directly.  So instead the values are stored
as String.  This change places a burden on consumers of the strings to
convert the strings to numbers and decide what to do about extremely large
values, but there will be very few consumers and they would need to have a
policy for dealing with "no limit" in any case.
2017-10-03 12:08:36 +01:00
markwalkom dbea83a1d0 [Docs] Update length-tokenfilter.asciidoc (#26849)
Made it clear what the numeric value of `Integer.MAX_VALUE`  is,
2017-10-02 11:01:43 +02:00
Amine Daï 3cb99aeec1 Fix references to vm.max_map_count in Docker docs
This commit fixes some references to vm.max_map_count in the Docker
docs.

Relates #26798
2017-09-29 15:56:18 -04:00
David Turner 8fe9a20982 Forbid negative values for index.unassigned.node_left.delayed_timeout (#26828)
Change delayed_timeout to be a positiveTimeSetting, and add note that this is a breaking change
2017-09-29 14:44:43 +01:00
Jason Tedor cfd6f35fc3 Add note to docs on /etc/elasticsearch ownership
This commit adds a note to the docs for the RPM and Debian installation
regarding the expected permissions for /etc/elasticsearch.

Relates #26795
2017-09-27 09:22:52 -04:00
olcbean 6952f7b560 Validate top-level keys for create index request (#23755) (#23869)
This commit ensures create index requests do not ignore unknown keys passed to the request.

closes #23755
2017-09-26 09:49:20 -07:00
Jim Ferenczi 74473c1c3d Early termination with index sorting should not set terminated_early in the response (#26597)
Early termination with index sorting always return the best top N in the response but set the flag `terminated_early`
in the response. This can be confusing because we use the same flag for `terminate_after` which on the contrary returns partial results.
This change removes the flag when results are not partial (early termination due to index sorting) and keeps it only when `terminate_after` is used.

Closes #26408
2017-09-26 11:37:11 +02:00
Christoph Büscher 6189c54c84 Reject the `index_options` parameter for numeric fields (#26668)
Numeric fields no longer support the index_options parameter. This changes the parameter
to be rejected in numeric field types after it was deprecated in 6.0.

Closes #21475
2017-09-25 23:43:14 +02:00
Christoph Büscher 3827918417 Add configurable `maxTokenLength` parameter to whitespace tokenizer (#26749)
Other tokenizers like the standard tokenizer allow overriding the default
maximum token length of 255 using the `"max_token_length` parameter. This change
enables using this parameter also with the whitespace tokenizer. The range that
is currently allowed is from 0 to StandardTokenizer.MAX_TOKEN_LENGTH_LIMIT,
which is 1024 * 1024 = 1048576 characters.

Closes #26643
2017-09-25 17:21:19 +02:00
javanna dee2ae1023 [DOCS] Replace mention of string field type with text and keyword
Closes #25713
2017-09-25 11:12:06 +02:00
Jason Tedor d8bb413b1b Configure heap dump path out of the box
The JVM defaults to dumping the heap to the working directory of
Elasticsearch. For the RPM and Debian packages, this location is
/usr/share/elasticsearch. This directory is not writable by the
elasticsearch user, so by default heap dumps in this situation are
lost. This commit modifies the packaging for the RPM and Debian packages
to set the heap dump path to /var/lib/elasticsearch as the default
location for dumping the heap. This location is writable by the
elasticsearch user by default. We add documentation of this important
setting if /var/lib/elasticsearch is not suitable for receiving heap
dumps.

Relates #26755
2017-09-22 14:22:03 -04:00
Yannick Welsch df5c450e89 Add v6.1 BWC layer for adding wait_for_active_shards to index open command
This commit disables BWC tests while adding a v6.1 BWC layer for the PR #26682
2017-09-22 16:30:07 +02:00
Alexander Kazakov ff737a880c Add wait_for_active_shards parameter to index open command (#26682)
Adds the wait_for_active_shards parameter to the index open command. Similar to the index creation command, the index open command will now, by default, wait until the primaries have been allocated.

Closes #20937
2017-09-22 11:15:03 +02:00
lcawley 06551a8549 [DOCS] Added index-shared4 and index-shared5.asciidoc 2017-09-20 10:54:26 -07:00
Tahmim Ahmed Shibli 34662c9e6d [Docs] Fix name of character filter in example. (#26724) 2017-09-20 17:08:43 +02:00
Christoph Büscher 86b00b84bc Remove parse field deprecations in query builders (#26711)
The `fielddata` field and the use of the `_name` field in the short syntax of the range 
query have been deprecated in 5.0 and can be removed.

The same goes for the deprecated `score_mode` field in HasParentQueryBuilder,
the deprecated `like_text`, `ids` and `docs` parameter in the `more_like_this` query,
the deprecated query name in the short version of the `regexp` query, and several
deprecated alternative field names in other query builders.
2017-09-20 16:22:21 +02:00
Tanguy Leroux c16c653c3e [Test] Fix reference/cat/allocation/line_8 test failure
In this test, 260b is replaced by the regexp \d+b
but the test sometimes produces results like 1.1kb
so this commit adapts the regexp to match values
with decimals
2017-09-18 10:46:19 +02:00