Commit Graph

4461 Commits

Author SHA1 Message Date
Jason Tedor 540413b24a Also skip JAVA_TOOL_OPTIONS on Windows
On non-Windows platforms, we ignore the environment variable
JAVA_TOOL_OPTIONS (this is an environment variable that the JVM respects
by default for picking up extra JVM options). The primary reason that we
ignore this because of the Jayatana agent on Ubuntu; a secondary reason
is that it produces an annoying "Picked up JAVA_TOOL_OPTIONS: ..."
output message. When the elasticsearch-env batch script was introduced
for Windows, ignoring this environment variable was deliberately not
carried over as the primary reason does not apply on Windows. However,
after additional thinking, it seems that we should simply be consistent
to the extent possible here (and also avoid that annoying "Picked up
JAVA_TOOL_OPTIONS: ..." on Windows too). This commit causes the Windows
version of elasticsearch-env to also ignore JAVA_TOOL_OPTIONS.

Relates #25968
2017-07-31 21:27:42 +09:00
Jason Tedor 2ef0f8af38 Add max file size bootstrap check
This commit adds a bootstrap check for the maximum file size, and
ensures the limit is set correctly when Elasticsearch is installed as a
service on systemd-based systems.

Relates #25974
2017-07-31 21:01:47 +09:00
Adrien Grand 57e8b9ee29 Remove another reference to 64-bit systems. 2017-07-31 09:56:06 +02:00
Adrien Grand 1b34f691e5 Remove reference to 32-bit systems. (#25971)
They are not supported anymore as of #25435.
2017-07-31 09:55:09 +02:00
propulkit a2fd1b9f6d No more using 'hybrid mmapfs / niofs' (#25944)
It looks a bit ambiguous here.

ElasticSearch no more using 'hybrid mmapfs / niofs' which chooses filesystem based on the file. It is any one of the mmapfs, niofs or simplefs depending on the operating system.
As quoted here https://www.elastic.co/guide/en/elasticsearch/reference/5.5/index-modules-store.html

Thanks,
Pulkit Agrawal
2017-07-31 09:53:55 +02:00
Martijn van Groningen ec7ac32772
docs: document work around for the percolator if query time text analysis is expensive. 2017-07-28 15:04:15 +02:00
Martijn van Groningen 7c3735bdc4
percolator: Store the QueryBuilder's Writable representation instead of its XContent representation.
The Writeble representation is less heavy to parse and that will benefit percolate performance and throughput.

The query builder's binary format has now the same bwc guarentees as the xcontent format.

Added a qa test that verifies that percolator queries written in older versions are still readable by the current version.
2017-07-28 12:24:10 +02:00
Daniel Mitterdorfer db90455afd Update plugin-related output in reference docs (#25897)
The example output for node info and cluster stats was outdated w.r.t.
to the information that is shown for plugins. With this commit we
updated the example output and update the explanation of the respective
fields.
2017-07-28 11:27:54 +02:00
Martijn van Groningen 5cf56a846a
docs: Remove incorrect warning
Closes #25935
2017-07-28 10:53:47 +02:00
Clinton Gormley 3e568f52c1 Fixed asciidoc formatting 2017-07-27 15:55:52 +02:00
javanna 19843b50e9 [DOCS] update low level client artifact name 2017-07-27 11:19:40 +02:00
Tanguy Leroux 91dc1c5da6 [Docs] Update Java Low-Level documentation to reflect shaded deps (#25882)
Since #25208 the Java Low-Level Rest Client has shaded dependencies. This commit updates the documentation to reflect that.
2017-07-26 12:17:21 +02:00
Tanguy Leroux 90ebaaa9a8 [Docs] Add profile section to the Search API documentation (#25880) 2017-07-26 10:31:46 +02:00
Lee Hinman faee825fea Fix elvis operator documentation 2017-07-25 12:50:09 -06:00
Clinton Gormley 3759aad737 Updated doc versions to 7.0.0-alpha1 unreleased 2017-07-25 13:09:27 +02:00
Jim Ferenczi 7868373069 [Docs] remove reference to the deprecated in the docs 2017-07-25 09:41:53 +02:00
Jim Ferenczi 4a9995145c [Docs]: Clarify query_string parser splits on operator 2017-07-24 18:36:16 +02:00
Jim Ferenczi c3784326eb Refactor field expansion for match, multi_match and query_string query (#25726)
This commit changes the way we handle field expansion in `match`, `multi_match` and `query_string` query.
 The main changes are:

- For exact field name, the new behavior is to rewrite to a matchnodocs query when the field name is not found in the mapping.

- For partial field names (with `*` suffix), the expansion is done only on `keyword`, `text`, `date`, `ip` and `number` field types. Other field types are simply ignored.

- For all fields (`*`), the expansion is done on accepted field types only (see above) and metadata fields are also filtered.

- The `*` notation can also be used to set `default_field` option on`query_string` query. This should replace the needs for the extra option `use_all_fields` which is deprecated in this change.

This commit also rewrites simple `*` query to matchalldocs query when all fields are requested (Fixes #25556). 

The same change should be done on `simple_query_string` for completeness.

`use_all_fields` option in `query_string` is also deprecated in this change, `default_field` should be set to `*` instead.

Relates #25551
2017-07-21 16:52:57 +02:00
Colin Goodheart-Smithe f1f1725fcf [DOCS] improve explanation of dynamic mapping setting (#25829)
Closes #25825
2017-07-21 12:24:38 +01:00
Clinton Gormley 618ff159eb Reorganised setup docs into better order 2017-07-21 11:24:46 +02:00
Ryan Ernst 8ab0d10387 Add compatibility versions to main action response (#25799)
This commit adds the min wire/index compat versions to the main action
output. Not only will this make the compatility expected more
transparent, but it also allows to test which version others think the
compat versions are, similar to how we test the lucene version.
2017-07-20 13:01:41 -07:00
Boaz Leskes 7488877d1a Validate a joining node's version with version of existing cluster nodes (#25808)
When a node tries to join a cluster, it goes through a validation step to make sure the node is compatible with the cluster. Currently we validation that the node can read the cluster state and that it is compatible with the indexes of the cluster. This PR adds validation that the joining node's version is compatible with the versions of existing nodes. Concretely we check that:

1) The node's min compatible version is higher or equal to any node in the cluster (this prevents a too-new node from joining)
2) The node's version is higher or equal to the min compat version of all cluster nodes (this prevents a too old join where, for example, the master is on 5.6, there's another 6.0 node in the cluster and a 5.4 node tries to join).
3) The node's major version is at least as higher as the lowest node in the cluster. This is important as we use the minimum version in the cluster to stop executing bwc code for operations that require multiple nodes. If the nodes are already operating in "new cluster mode", we should prevent nodes from the previous major to join (even if they are wire level compatible). This does mean that if you have a very unlucky partition during the upgrade which partitions all old nodes which are also a minority / data nodes only, the may not be able to re-join the cluster. We feel this edge case risk is well worth the simplification it brings to BWC layers only going one way. This restriction only holds if the cluster state has been recovered (i.e., the cluster has properly formed).

 Also, the node join validation can now selectively fail specific nodes (previously the entire batch was failed). This is an important preparation for a follow up PR where we plan to have a rejected joining node die with dignity.
2017-07-20 20:11:29 +02:00
Clinton Gormley febb4bf7bc Update removal_of_types.asciidoc
Fixed `include_in_type` -> `include_type_name`
2017-07-20 19:18:51 +02:00
Jack Conradson 9f7463e796 remove lang url parameter from stored script requests (#25779)
Also has updates to ScriptMetaData for allowing the old namespace format to be loaded all the way back through 5.0; however, it will throw an exception if two scripts share the same id but different languages.
2017-07-20 08:51:08 -07:00
Jay Modi 3e4bc027eb RestClient uses system properties and system default SSLContext (#25757)
This commit calls the `useSystemProperties` method on the HttpAsyncClientBuilder so that the jvm
system properties are used. The primary reason for doing this is to ensure the builder uses the
system default SSLContext rather than the default instance created by the http client library.

Closes #23231
2017-07-20 07:36:56 -06:00
Jason Tedor 67a4288c9a Remove support for ES_INCLUDE
Today we enable users to customize the environment through the use of
ES_INCLUDE. This made sense for legacy reasons when we did not have
nicities like jvm.options (so dumped JVM options in the default include
script) and somewhat duplicates some of the functionality that we will
need from a dedicated environment script. This commit removes support
for ES_INCLUDE as a first step towards a dedicated include script.

Relates #25804
2017-07-20 15:41:59 +09:00
Boaz Leskes 9989ac69a4 Revert "Validate a joining node's version with version of existing cluster nodes (#25770)"
This reverts commit 1e1f8e6376.
2017-07-19 17:34:53 +02:00
Adrien Grand d607c3be92 Fix list of unconverted snippets. 2017-07-19 14:57:55 +02:00
Adrien Grand f1ff7f2454 Require a field when a `seed` is provided to the `random_score` function. (#25594)
We currently use fielddata on the `_id` field which is trappy, especially as we
do it implicitly. This changes the `random_score` function to use doc ids when
no seed is provided and to suggest a field when a seed is provided.

For now the change only emits a deprecation warning when no field is supplied
but this should be replaced by a strict check on 7.0.

Closes #25240
2017-07-19 14:11:15 +02:00
Clinton Gormley f69decf509 NOCONSOLE -> NOTCONSOLE in removal-of-types 2017-07-19 14:06:04 +02:00
Boaz Leskes 1e1f8e6376 Validate a joining node's version with version of existing cluster nodes (#25770)
When a node tries to join a cluster, it goes through a validation step to make sure the node is compatible with the cluster. Currently we validation that the node can read the cluster state and that it is compatible with the indexes of the cluster. This PR adds validation that the joining node's version is compatible with the versions of existing nodes. Concretely we check that:

1) The node's min compatible version is higher or equal to any node in the cluster (this prevents a too-new node from joining)
2) The node's version is higher or equal to the min compat version of all cluster nodes (this prevents a too old join where, for example, the master is on 5.6, there's another 6.0 node in the cluster and a 5.4 node tries to join).
3) The node's major version is at least as higher as the lowest node in the cluster. This is important as we use the minimum version in the cluster to stop executing bwc code for operations that require multiple nodes. If the nodes are already operating in "new cluster mode", we should prevent nodes from the previous major to join (even if they are wire level compatible). This does mean that if you have a very unlucky partition during the upgrade which partitions all old nodes which are also a minority / data nodes only, the may not be able to re-join the cluster. We feel this edge case risk is well worth the simplification it brings to BWC layers only going one way.

 Also, the node join validation can now selectively fail specific nodes (previously the entire batch was failed). This is an important preparation for a follow up PR where we plan to have a rejected joining node die with dignity.
2017-07-19 12:57:29 +02:00
Jason Tedor 3d3d99557d Expand migration note regarding default paths
This commit expands on the migration note regarding the removal of
default.path.data and default.path.logs to include a note that users
that were relying on the defaults (the common case for path.logs), and
they carry over their previous elasticsearch.yml configruation file,
then they must add explicit values for path.data and path.logs.
2017-07-19 13:40:42 +09:00
Deb Adair 23c810b334 [DOCS] Changes xrefs to cross doc links to enable building GS "mini-docs" 2017-07-18 13:52:38 -07:00
Deb Adair d9e55179f1 [DOCS] Adding index file for GS "mini book". 2017-07-18 13:44:08 -07:00
Christoph Büscher 43bfe06759 [Docs] Add sorting and source filtering section to client docs (#25767) 2017-07-18 16:58:46 +02:00
Clinton Gormley ff4a2519f2 Update experimental labels in the docs (#25727)
Relates https://github.com/elastic/elasticsearch/issues/19798

Removed experimental label from:
* Painless
* Diversified Sampler Agg
* Sampler Agg
* Significant Terms Agg
* Terms Agg document count error and execution_hint
* Cardinality Agg precision_threshold
* Pipeline Aggregations
* index.shard.check_on_startup
* index.store.type (added warning)
* Preloading data into the file system cache
* foreach ingest processor
* Field caps API
* Profile API

Added experimental label to:
* Moving Average Agg Prediction


Changed experimental to beta for:
* Adjacency matrix agg
* Normalizers
* Tasks API
* Index sorting

Labelled experimental in Lucene:
* ICU plugin custom rules file
* Flatten graph token filter
* Synonym graph token filter
* Word delimiter graph token filter
* Simple pattern tokenizer
* Simple pattern split tokenizer

Replaced experimental label with warning that details may change in the future:
* Analysis explain output format
* Segments verbose output format
* Percentile Agg compression and HDR Histogram
* Percentile Rank Agg HDR Histogram
2017-07-18 14:06:22 +02:00
Luca Cavanna 0d8b753325 IndexClosedException to return 400 rather than 403 (#25752)
403 can be confused with security. If an API doesn't support working against closed indices and closed indices are referred to in a request, that is a bad request, hence 400 is more appropriate.
2017-07-18 10:26:32 +02:00
Christoph Büscher a6e3d356ed Change parsing of numeric `to` and `from` parameters in `date_range` aggregation (#25376)
Currently the `to` and `from` parameter in the `date_range` aggregation is not
parsed with the correct date field format from the mappings or the aggregation
if the argument is numeric, but always treated as a long value specifying
`epoch_millis`. This leads to problems e.g. when the format is `epoch_second`,
but the `to` and `from` are currently treated as millis.

With this change, we interpret these parameters according to the `format` of the target field.
If the `format` in the mappings is not compatible with numeric input values,
a compatible `format` (e.g. `epoch_millis`, `epoch_second`) must be specified in
the `date_range` aggregation itself, otherwise an error is thrown.

#Closes #17920
2017-07-18 09:45:28 +02:00
Christoph Büscher 56b1250a34 [Docs] Adding highlighting section to high level client docs (#25751)
Adding a section about how to use highlighting in the SearchSourceBuilder and
how to retrieve highlighted fragments from the SearchResponse.
2017-07-17 19:30:58 +02:00
Simon Willnauer cb4eebcd6a Make `index` in TermsLookup mandatory (#25753)
This change removes the leniency of having a `null` index to fetch
terms from in 6.0 onwards. This feature will be deprecated in the 5.x series
and 6.0 nodes will require the index to be set.

Closes #25750
2017-07-17 18:50:30 +02:00
Clinton Gormley 25a89e613a Broke recipes into separate pages 2017-07-17 18:21:39 +02:00
Glen Smith e9dfb2a215 Fix another simulate example in ingest docs
When simulating an ingest pipeline against an existing pipeline, the
_source field is required to wrap each doc. This commit fixes another
example in the docs that is missing this.
    
Relates #25743, relates e3a0c11239
2017-07-17 15:17:42 +09:00
Glen Smith e3a0c11239 Fix simulate example in ingest docs
When simulating an ingest pipeline against an existing pipeline, the
_source field is required to wrap each doc. This commit fixes an example
in the docs that is missing this.

Relates #25742
2017-07-17 14:17:41 +09:00
Ryan Ernst 072402463b Scripting: Remove search template actions (#25717)
The dedicated search template put/get/delete actions are deprecated in
5.6. This commit removes them from 6.0.
2017-07-14 23:12:05 -07:00
javanna 2c38e93e96 [DOCS] Added note to high level client docs on version
The alpha2 docs is built out of master which may make users think that the high level client was already released as part of alpha2 which it was not. This note should clarify that the client will be released with 6.0.0-beta1
2017-07-15 07:50:25 +02:00
Ryan Ernst b1762d69b5 Setup: Change default heap to 1G (#25695)
This commit changes the default heap size to 1 GB. Experimenting with
elasticsearch is often done on laptops, and 1 GB is much friendlier to
laptop memory. It does put more pressure on the gc, but the tradeoff is
a smaller default footprint. Users running in production can (and
should) adjust the heap size as necessary for their usecase.
2017-07-14 09:38:08 -07:00
Christoph Büscher 5387ed00d2 [Docs] Adding suggestion sections to high level client docs (#25724)
This adds a section about how to add suggestions to the SearchSourceBuilder and
how to retrieve them from a SearchResponse.
2017-07-14 18:33:28 +02:00
Christoph Büscher f809a12493 [Docs] Adding aggregation sections to high level client docs (#25707)
This adds a section about how to add aggregations to the SearchSourceBuilder and how
to retrieve them from a SearchRepsonse to the documentation for the high level rest client.
2017-07-14 12:47:47 +02:00
Bodecker DellaMaria 4f0dc5bf32 Mark filtered query example as not to be used (#25661)
The Filtered Query has been deprecated in favour of the Bool Query with a filter context. However, this deleted page for the Filtered Query is often ranked highly in search results when searching for documentation on "filtered queries". Often people just copy the first code snippet they see, which in this case is the INCORRECT syntax (the correct syntax follows). I think reordering the examples would help avoid a lot of confusion (I have seen people make this same mistake 3 times now)

Adding a comment to indicate that the first example shouldn't be used
2017-07-14 11:45:21 +02:00
Martijn van Groningen c8777c4c2e
docs: Updated reference docs that `document_type` is deprecated 2017-07-14 11:07:46 +02:00