Commit Graph

2992 Commits

Author SHA1 Message Date
Robert Muir fff82db681 Add tests/doc for boolean fields with expressions 2016-05-02 18:13:03 -04:00
Robert Muir 693c1f6671 Support geo_point fields in lucene expressions.
Closes #18096
2016-05-02 17:49:21 -04:00
Robert Muir 28409e4509 Add support for .empty to expressions, and some docs improvements
Closes #18077
2016-05-02 09:07:25 -04:00
Clinton Gormley 9fee8c76af Added release notes for 5.0.0-alpha2 2016-05-02 14:21:59 +02:00
Clinton Gormley 7c8397d99b Update keyword.asciidoc
`ignore_above` doesn't apply to analyzed `text` fields
2016-05-02 13:47:14 +02:00
Todd Dicken 9162fd27a0 Change qoutes to quotes (#18078)
Updated misspelling in documentation
2016-05-02 13:06:07 +02:00
Josef Salyer b2ce2f5afa Fixed the documentation formatting. (#17990)
The source code example for the initial example was missing the correct JSON object formatting and syntax.  That has been fixed with my change.
2016-04-29 18:07:48 +02:00
Robin Joseph e322903f2c Fix typo in include-in-all.asciidoc (#18055) 2016-04-29 18:03:22 +02:00
Martijn van Groningen 7aca1389e2 ingest: Add `date_index_name` processor.
Closes #17814
2016-04-29 17:20:48 +02:00
David Pilato 2232a7cdf3 Merge branch 'pr/cat-size-time-units' 2016-04-29 15:09:14 +02:00
Shane Connelly 713c0df3a3 Merge pull request #17994 from eskibars/master
Add new IPv6 types to docs where it's supported
2016-04-29 06:00:32 -07:00
Clinton Gormley 04bd55d61c Added perl migration script for indexed scripts to migration docs 2016-04-29 14:18:27 +02:00
Martijn van Groningen 6c3beaa2eb Drop top level inner hits in favour of inner hits defined in the query dsl.
Fix a limitation that prevent from hierarchical inner hits be defined in query dsl.

Removed the nested_path, parent_child_type and query options from inner hits dsl. These options are only set by ES
upon parsing the has_child, has_parent and nested queries are using their respective query builders.

These options are still used internally, when these options are set a new private copy is created based on the
provided InnerHitBuilder and configuring either nested_path or parent_child_type and the inner query of the query builder
being used.

Closes #11118
2016-04-29 11:17:24 +02:00
Tal Levy 6302fb65a3 add ability to disable ability to override values of existing fields in set processor 2016-04-28 13:50:19 -07:00
Isabel Drost-Fromm c1fa9cd18e Add note that coerce and ignore_malformed are deprecated for geo
distance sorting
2016-04-28 14:13:58 +02:00
Isabel Drost-Fromm a19c426e0f Deprecate coerce/ignore_malformed for GeoBoundingBoxQuery 2016-04-28 14:10:59 +02:00
Isabel Drost-Fromm 3f743a30cf Deprecate coerce/ignore_malformed in GeoDistanceQueryBuilder 2016-04-28 14:06:27 +02:00
Isabel Drost-Fromm 3160798084 Deprecate coerce/ignore_malformed for GeoDistanceRangeQuery 2016-04-28 14:01:54 +02:00
Isabel Drost-Fromm 5306de3ce3 Deprecate coerce/ignore_malformed for GeoPolygonQueryBuilder
Includes update to parsing code, tests, migration docs and reference
docs.
2016-04-28 13:56:50 +02:00
Clinton Gormley 84a2b4e17e Update id-field.asciidoc
Clarified which queries support the `_id` field
2016-04-28 13:36:14 +02:00
Shane Connelly 10b45852e9 Revert IPv6 here 2016-04-27 08:07:44 -07:00
Jim Ferenczi 573c4f3ed1 Extend field stats:
* Add isSearchable and isAggregatable (collapsed to true if any of the instances of that field are searchable or aggregatable).
* Accept wildcards in field names.
* Add a section named conflicts for fields with the same name but with incompatible types (instead of throwing an exception).
2016-04-27 16:51:53 +02:00
Alexander Kazakov a8a33a1a94 Row-centric output for _cat/fielddata 2016-04-27 13:29:02 +03:00
Christoph Büscher a2c3b5cae1 Update keyword.asciidoc 2016-04-27 12:10:19 +02:00
Shane Connelly aff148f532 Add new IPv6 types to docs where it's supported 2016-04-26 11:38:49 -07:00
Rafał Bigaj ead431f524 Duplicated colon was removed (#17988)
Hope this help :-)
2016-04-26 20:31:00 +02:00
Jihun No bb3c5ac355 removing duplicated parenthese open (#17975)
removing duplicated parenthese open
2016-04-26 20:13:08 +02:00
Areek Zillur afacc18dcc Merge branch 'master' into docs/completion_suggester 2016-04-26 10:16:38 -04:00
Areek Zillur cc99b24bf7 Document completion suggest breaking changes 2016-04-26 10:15:21 -04:00
Mike Buhot 206c4a2b0b Add kotlin query DSL to community clients (#17943)
Hi, 

I've created a query builder DSL for Kotlin language that mimics the JSON query DSL.
This makes it easier to translate the documentation targeting the JSON api onto kotlin code.
Please consider adding it to the list of community clients.

Thanks,

Mike Buhot
2016-04-26 16:03:50 +02:00
Jason Tedor efeec4d096 Merge pull request #17017 from jasontedor/generic-thread-pool
Actually bound the generic thread pool
2016-04-26 08:27:48 -04:00
Martijn Laarman 166cfcee8a Document missing shard version in routing table of cluster state (#17945)
as breaking change

removed as per: https://github.com/elastic/elasticsearch/pull/16243

because of: https://github.com/elastic/elasticsearch/issues/14739
2016-04-26 10:50:38 +02:00
Pius 66686040ca Update settings.asciidoc
Changed "must bet set" to "must be set" under Discovery Settings.
2016-04-26 00:15:39 -07:00
Pius f6656aa6ff Update settings.asciidoc
Added the 2 missing ` under Request Cache Settings section.
2016-04-25 23:57:42 -07:00
Pius 1364cc89f1 Update mapping.asciidoc
Changed "referrer to" to "refer to".
2016-04-25 23:35:42 -07:00
Sergii Golubev 2f6405ee27 serial-diff-aggregation.asciidoc: fix a mistake (#17950) 2016-04-25 07:45:54 -04:00
Clinton Gormley d56a8e5dd8 Update index-apis.asciidoc
Asciidoc typo
2016-04-25 13:06:57 +02:00
Jason Tedor 5608fa7ac1 Actually bound the generic thread pool
This commit actually bounds the size of the generic thread pool. The
generic thread pool was of type cached, a thread pool with an unbounded
number of workers and an unbounded work queue. With this commit, the
generic thread pool is now of type scaling. As such, the cached thread
pool type has been removed. By default, the generic thread pool is
constructed with a core pool size of four, a max pool size of 128 and
idle workers can be reaped after a keep-alive time of thirty seconds
expires. The work queue for this thread pool remains unbounded.
2016-04-25 06:47:26 -04:00
Lee Hinman 5fe1916be9 Merge pull request #17924 from elastic/russcam-patch-1
Update settings.asciidoc
2016-04-24 18:25:08 -06:00
Ryan Ernst 30eb72b9cb Merge pull request #17942 from rjernst/aggs_example_keyword
Docs: Update example aggs to use dynamic keyword field
2016-04-23 16:20:07 -07:00
Clinton Gormley b9978ace40 Update settings.asciidoc
Asciidoc typo
2016-04-23 13:44:42 +02:00
Russ Cam fb58ae3b4f Update settings.asciidoc
Add note for removal of index.translog.interval
2016-04-23 11:44:59 +10:00
Ryan Ernst 034adeb947 Docs: Update example aggs to use dynamic keyword field
The getting started docs use dynamic mappings. With the recent change to
string split into text and keyword, text lost the default ability to do
aggs. This was added back in #17188. This change updates the getting
started examples to use the keyword multi field added to dynamically
mapped text fields.

closes #17941
2016-04-22 16:34:15 -07:00
xuzha cd527c5b92 Add support for customizing the rule file in ICU tokenizer
Lucene allows to create a ICUTokenizer with a special config argument
enabling the customization of the rule based iterator by providing
custom rules files.

This commit enable this feature. Users could provide a list of RBBI rule
files to ICU tokenizer.

closes #13146
2016-04-22 12:39:20 -07:00
chenxiang a0aea5baf7 Update terms-query.asciidoc
user id of tweet hould exist in the `followers`, otherwise the search result is empty
2016-04-22 10:56:13 -06:00
ericamick 069eb72604 Update bucket.asciidoc 2016-04-22 10:54:25 -06:00
ericamick f081bf4e26 Update bulk.asciidoc 2016-04-22 10:51:33 -06:00
ericamick 3004c45f7b Update update.asciidoc 2016-04-22 10:50:42 -06:00
ericamick 276b89242c Update get.asciidoc 2016-04-22 10:48:58 -06:00
Nik Everett 61f0b665b8 Fix fallback setting for two get/2 2016-04-22 11:10:01 -04:00
Christoph Büscher a1c9025eaa Update completion-suggest.asciidoc
Removed trailing comma.
2016-04-22 14:00:37 +02:00
Martijn van Groningen c5ad2e2865 Changed indexed scripts to be stored in the cluster state instead of the `.scripts` index.
Also added max script size soft limit for stored scripts.

Closes #16651
2016-04-22 13:42:55 +02:00
Clinton Gormley e4df68b627 Added cautionary note to match_phrase_prefix explaining its shortcomings
Closes #17655
2016-04-22 12:45:12 +02:00
Christoph Büscher 0ec4ffcb3a Remove QueryFilterBuilder section from migration docs.
This query builder was deprecated in 2.0 and has been removed.
2016-04-21 18:11:01 +02:00
Martijn van Groningen dd2184ab25 ingest: Streamline option naming for several processors:
* `rename` processor, renamed `to` to `target_field`
* `date` processor, renamed `match_field` to `field` and renamed `match_formats` to `formats`
* `geoip` processor, renamed `source_field` to `field` and renamed `fields` to `properties`
* `attachment` processor, renamed `source_field` to `field` and renamed `fields` to `properties`

Closes #17835
2016-04-21 13:40:43 +02:00
Jun Ohtani 9eb242a5fe Analyze API : Rename filters/token_filters/char_filter to filter/token_filter/char_filter
Closes #15189
2016-04-21 18:05:11 +09:00
Zachary Tong 80288ad60c Add `fingerprint` token filter and `fingerprint` analyzer
Adds a `fingerprint` token filter which uses Lucene's FingerprintFilter,
and a `fingerprint` analyzer that combines the Fingerprint filter with
lowercasing, stop word removal and asciifolding.

Closes #13325
2016-04-20 16:10:56 -04:00
Martijn van Groningen 81449fc912 percolator: renamed `percolator` query to `percolate` query 2016-04-20 15:23:54 +02:00
Clinton Gormley ca8ea36b30 Updated decay-function image in function_score query
Closes #17479
2016-04-20 13:37:52 +02:00
Clinton Gormley b89e6cd5d8 Added link to breaking changes to release notes 2016-04-19 20:05:18 +02:00
Lee Hinman b8899cdb78 Merge remote-tracking branch 'dakrone/allow-bad-json' 2016-04-19 10:02:53 -06:00
Martijn van Groningen ba08313417 settings: Removed `action.get.realtime` setting
Closes #12543
2016-04-19 17:14:23 +02:00
Lee Hinman a1e8fb794c Allow JSON with unquoted field names by enabling system property
In Elasticsearch 5.0.0, by default unquoted field names in JSON will be
rejected. This can cause issues, however, for documents that were
already indexed with unquoted field names. To alleviate this, a system
property has been added that can be enabled so migration can occur.

This system property will be removed in Elasticsearch 6.0.0

Resolves #17674
2016-04-19 09:14:13 -06:00
Clinton Gormley 102a398d9f Fixed split processor example 2016-04-19 14:11:45 +02:00
Clinton Gormley 68f96868a6 Percolator docs missing a callout 2016-04-19 14:11:23 +02:00
Russ Cam e53131dd79 Update has-parent-query.asciidoc (#17841)
Change reference to `score_mode` to `score`
2016-04-19 11:56:05 +02:00
Clinton Gormley c024504842 Update search.asciidoc
Corrected breaking changes for `has_parent`.  Relates to https://github.com/elastic/elasticsearch/pull/17841
2016-04-19 11:54:48 +02:00
Martijn van Groningen 8e63ce00f0 docs: removed confusing statement. 2016-04-19 11:49:51 +02:00
Martijn van Groningen 40c22fc654 percolator: removed .percolator type instead a field of type `percolator` should be configured before indexing percolator queries
* Added an extra `field` parameter to the `percolator` query to indicate what percolator field should be used. This must be an existing field in the mapping of type `percolator`.
* The `.percolator` type is now forbidden. (just like any type that starts with a `.`)

This only applies for new indices created on 5.0 and later. Indices created on previous versions the .percolator type is still allowed to exist.
The new `percolator` field type isn't active in such indices and the `PercolatorQueryCache` knows how to load queries from these legacy indices.
The `PercolatorQueryBuilder` will not enforce that the `field` parameter is of type `percolator`.
2016-04-19 11:20:31 +02:00
Clinton Gormley a2ab13ddd1 Update ingest-node.asciidoc
Documented `separator` in the `split processor

Closes https://github.com/elastic/elasticsearch/issues/17831
2016-04-19 11:11:58 +02:00
Clinton Gormley 40b84d2ef6 Update mapping.asciidoc
Correct `fielddata.frequency.regex` to `fielddata.filter.regex` in breaking changes
2016-04-18 21:00:27 +02:00
Danilo Vaz 2e2d8c1442 Updated copyright years to include 2016 (#17808) 2016-04-18 12:39:23 +02:00
Sergii Golubev 5ce3eb96b0 tophits-aggregation.asciidoc: fix a typo 2016-04-18 09:23:39 +02:00
David Pilato 5e1f26c22a Add support for documented byte/size units and for micros as a time unit in _cat API
We advertise in our documentation that byte units are like `kb`, `mb`... But we actually only support the simple notation `k` or `m`.
This commit adds support for the documented form and keeps the non documented options to avoid any breaking change.

It also adds support for `micros`, `nanos` and `d` as a time unit in `_cat` API.

Remove the support for `b` as a SizeValue unit. Actually, for numbers, when using raw numbers without unit, there is no text to add/parse after the number. For example, you don't write `10` as `10b`. We support option like `size=` in `_cat` API which means that we want to display raw data without unit (singles).

Documentation updated accordingly.

Add test for the empty size option.

Fix missing TimeValues options for some cat APIs
2016-04-15 20:55:41 +02:00
LeonardGC 0b8be7f894 Update field-mapping.asciidoc (#17670) 2016-04-15 09:22:38 +02:00
bloublou 83944c5628 Typo correction heap_size.asciidoc (#17745)
* Typo correction Xms Xmx

Typo correction on "-Xms4000mb -Xmx4000mb"

* Change mb to m for Xms/Xmx
2016-04-14 20:37:37 +02:00
Adrien Grand d84c643f58 Use the new points API to index numeric fields. #17746
This makes all numeric fields including `date`, `ip` and `token_count` use
points instead of the inverted index as a lookup structure. This is expected
to perform worse for exact queries, but faster for range queries. It also
requires less storage.

Notes about how the change works:
 - Numeric mappers have been split into a legacy version that is essentially
   the current mapper, and a new version that uses points, eg.
   LegacyDateFieldMapper and DateFieldMapper.
 - Since new and old fields have the same names, the decision about which one
   to use is made based on the index creation version.
 - If you try to force using a legacy field on a new index or a field that uses
   points on an old index, you will get an exception.
 - IP addresses now support IPv6 via Lucene's InetAddressPoint and store them
   in SORTED_SET doc values using the same encoding (fixed length of 16 bytes
   and sortable).
 - The internal MappedFieldType that is stored by the new mappers does not have
   any of the points-related properties set. Instead, it keeps setting the index
   options when parsing the `index` property of mappings and does
   `if (fieldType.indexOptions() != IndexOptions.NONE) { // add point field }`
   when parsing documents.

Known issues that won't fix:
 - You can't use numeric fields in significant terms aggregations anymore since
   this requires document frequencies, which points do not record.
 - Term queries on numeric fields will now return constant scores instead of
   giving better scores to the rare values.

Known issues that we could work around (in follow-up PRs, this one is too large
already):
 - Range queries on `ip` addresses only work if both the lower and upper bounds
   are inclusive (exclusive bounds are not exposed in Lucene). We could either
   decide to implement it, or drop range support entirely and tell users to
   query subnets using the CIDR notation instead.
 - Since IP addresses now use a different representation for doc values,
   aggregations will fail when running a terms aggregation on an ip field on a
   list of indices that contains both pre-5.0 and 5.0 indices.
 - The ip range aggregation does not work on the new ip field. We need to either
   implement range aggs for SORTED_SET doc values or drop support for ip ranges
   and tell users to use filters instead. #17700

Closes #16751
Closes #17007
Closes #11513
2016-04-14 17:56:23 +02:00
Colin Goodheart-Smithe c595322d90 Adds ignore_unmapped option to geo queries
The change adds a new option to the geo_* queries: ignore_unmapped. If this option is set to false, the toQuery method on the QueryBuilder will throw an exception if the field specified in the query is unmapped. If the option is set to true, the toQuery method on the QueryBuilder will return a MatchNoDocsQuery. The default value is false so the queries work how they do today (throwing an exception on unmapped field)
2016-04-14 15:29:07 +01:00
Colin Goodheart-Smithe 686aff1545 Adds ignore_unmapped option to nested and P/C queries
The change adds a new option to the `nested`, `has_parent`, `has_children` and `parent_id` queries: `ignore_unmapped`. If this option is set to false, the `toQuery` method on the QueryBuilder will throw an exception if the type/path specified in the query is unmapped. If the option is set to true, the `toQuery` method on the QueryBuilder will return a MatchNoDocsQuery. The default value is `false`so the queries work how they do today (throwing an exception on unmapped paths/types)
2016-04-14 10:34:30 +01:00
Clinton Gormley acec464eb8 Docs: Clarified the purpose of the parent_id query 2016-04-14 11:25:26 +02:00
Sergii Golubev 434a563fe0 terms-aggregation.asciidoc tiny edit 2016-04-13 16:51:47 -06:00
Martijn van Groningen 16fa3e546e docs: remove mention of file based grok pattern 2016-04-13 22:51:12 +02:00
Clinton Gormley 447f099544 Improve glossary to not refer to types as "like a table" (#17704)
Closes #17673
2016-04-13 14:29:47 +02:00
Nik Everett 0f9804b0e2 reindex: gracefully handle when _source is disabled
Closes #17666
2016-04-13 08:19:58 -04:00
Sergii Golubev 39b914bd77 histogram-aggregation.asciidoc: tiny edit (#17706) 2016-04-13 14:19:05 +02:00
Martijn van Groningen ca5bd89581 docs: adjust grok processor docs to not mention pattern files as these no longer exist
Closes #17692
2016-04-13 12:37:50 +02:00
Daniel Mitterdorfer 0c7795f53d Merge remote-tracking branch 'danielmitterdorfer/bulk-size-limit'
Closes #17133
2016-04-13 10:43:00 +02:00
Clinton Gormley a62b9296c6 Docs: Fixed link to phonetic plugin 2016-04-13 10:17:46 +02:00
Clinton Gormley bdf62b5615 More asciidoc errors 2016-04-13 10:14:09 +02:00
Clinton Gormley 1a15e55f94 More asciidoc errors 2016-04-13 10:02:09 +02:00
Clinton Gormley b201605a81 Fix bad asciidoc 2016-04-13 09:57:00 +02:00
Daniel Mitterdorfer 52b2016447 Limit request size on transport level
With this commit we limit the size of all in-flight requests on
transport level. The size is guarded by a circuit breaker and is
based on the content size of each request.

By default we use 100% of available heap meaning that the parent
circuit breaker will limit the maximum available size. This value
can be changed by adjusting the setting

network.breaker.inflight_requests.limit

Relates #16011
2016-04-13 09:54:59 +02:00
Clinton Gormley d26b7457cf Docs: Added note about older versions of RPM not being supported, and mentioned CentOS 5 2016-04-13 09:43:38 +02:00
Clinton Gormley 29b75960df Docs: Added note about RPMs not being supported on SLES 11 2016-04-13 09:34:01 +02:00
Jason Tedor a581d7cca4 Merge pull request #17675 from jasontedor/java-opts
Add JVM options configuration file
2016-04-12 23:07:40 -04:00
Clinton Gormley 098b2e03b5 Removed all references to site plugins from plugin docs 2016-04-12 19:28:09 +02:00
Jason Tedor 3879aa2a98 Add JVM options configuration file
This commit adds a new configuration file jvm.options to centralize and
simplify management of JVM options. This separates the configuration of
the JVM from the packaging scripts (bin/elasticsearch*, bin/service.bat,
and init.d/elasticsearch) simplifying end-user operational management of
custom JVM options.
2016-04-12 11:19:16 -04:00
Adrien Grand 0eb1a816c8 Allow the query cache to be disabled. #16268
This replaces the internal `index.queries.cache.type` setting with
a new `index.queries.cache.enabled` setting, which is documented.

Closes #15802
2016-04-11 18:06:16 +02:00
Alexander Reelsen da19ddf3e6 Ingest Attachment: Allow to prevent base64 conversions by using raw bytes (#16601)
CBOR is natively supported in Elasticsearch and allows for byte arrays.
This means, that by using CBOR the user can prevent base64 conversions
for the data being sent back and forth.

This PR adds support to extract data from a byte array in addition to
a string. This also required to add a ByteArrayValueSource class.
2016-04-11 14:14:56 +02:00
Adrien Grand 4adc31fe11 Use `mmapfs` by default.
I case any problem was discovered, you can still enable the legacy `default`
directory instead. But the plan is to get rid of it in 6.0.

Closes #16983
2016-04-08 20:23:27 +02:00
Greg Marzouka 37cb00a0b5 Merge pull request #17576 from gmarz/docs/windows-install
[DOCS] Update instructions for running on Windows
2016-04-07 18:01:42 -04:00
gmarz 6d9ed8ebf6 [DOCS] Update instructions for running on Windows
Closes #16455
2016-04-07 17:59:58 -04:00
Igor Motov 81c59cae18 Add _cat/tasks
Adds new _cat endpoint that lists all tasks
2016-04-07 09:28:21 -06:00
Adrien Grand 1d0239c125 Add a warning about the impact of sorting terms aggregations on the accuracy of doc counts. 2016-04-07 16:57:44 +02:00
Clinton Gormley 88c5dfeca4 Docs: Removed references to deprecated functionality
* search_type=count
* DFS in term vectors
* Replaced string with text/keyword as appropriate
2016-04-07 13:33:35 +02:00
Boaz Leskes 8eee28e798 Update resiliency page (#17586)
#14252 , #7572 , #15900, #12573, #14671, #15281 and #9126 have all been closed/merged and will be part of 5.0.0.
2016-04-07 12:17:13 +02:00
Herbert Fischer c0ebce0ba0 Add some clarification regarding docs.count
* Add some clarification regarding docs.count

* Some improvements as suggested by @jasontedor
2016-04-07 11:15:32 +02:00
val e3199824ba Replaced SCAN search type with _doc sort in scroll query 2016-04-07 06:52:14 +02:00
Ibrahim Awwal 5121060e75 Fix typo in templates.asciidoc
The doc mentions match_path in one place but the correct syntax is path_match which is mentioned everywhere else. Using the wrong string leads to errors because the mapping becomes too greedy, and matches things it shouldn't.
2016-04-06 16:40:20 -06:00
Lee Hinman 6e44ddaf0e Merge branch 'pr/11338' 2016-04-06 15:06:30 -06:00
Dmitrii Izgurskii 272f3eb140 Add missing comma
Added missing comma
2016-04-06 15:03:37 -06:00
Jimmy Jones f157dae053 Disallow unquoted field names, fix testcases using unquoted JSON 2016-04-06 14:37:15 -06:00
Colin Goodheart-Smithe c20c49963f [DOCS] update match query documentation
Now the `match` query has been split out into `match`, `match_phrase` and `match_phrase_prefix` we need to update the docs to remove the deprecated syntax
2016-04-06 15:44:22 +01:00
Mahmoud Lababidi 20b398daea Add a more descriptive example to Index Template
Closes #17513
The current example in the documentation for Index Templates lacks any properties values. This is helpful to many devs that aren't sure how to take a regular Index Mapping and convert it to a template.
2016-04-06 12:18:10 +02:00
Igor Motov f599ac5d5a Expose whether a task is cancellable in the _tasks list API
Closes #17369
2016-04-05 19:16:08 -06:00
Sergii Golubev 8430b379d8 string.asciidoc: fix for `position_increment_gap`
Remove  outdated and duplicate description for the `position_increment_gap` parameter.
2016-04-05 16:23:42 -04:00
Jeff ba34faa1ef Call out where we are making a setting change.
IMHO the original text here was incomplete. Adding the simple words 'in the index mapping' makes this sentence more clear. Perhaps a be more clear to make this a link.
2016-04-05 13:51:31 -06:00
Dimitrios Liappis b86d0983bb Merge pull request #17505 from dliappis/docs_fixes
Fix locahost typo and distro clarifications
2016-04-05 19:24:33 +03:00
Clinton Gormley b5b00c5d66 Updated 5.0.0-alpha1 release notes 2016-04-05 16:08:22 +02:00
Luca Cavanna 67ccfc354e Switch to using ParseField to parse query names
* [TEST] check registered queries one by one in SearchModuleTests

* Switch to using ParseField to parse query names

If we have a deprecated query name, at the moment we don't have a way to log any deprecation warning nor fail when we are in strict mode. With this change we use ParseField, which will take care of the camel casing that we currently do manually (so that one day we can remove it more easily). This also means, that each query will have a unique preferred name, and all the other names are deprecated.

Terms query "in" synonym is now formally deprecated, as well as fuzzy_match, match_fuzzy, match_phrase and match_phrase_prefix for match query, mlt for more_like_this and geo_bbox for geo_bounding_box. All these will be removed in 6.0.

Every QueryParser holds now a ParseField constant called QUERY_NAME_FIELD that holds the name for it. The first name is the preferred one, all the others are deprecated. The first name is taken from the NAME constant already present in each query builder object, so that we somehow keep the serialization constant separated from ParseField. This change also allowed us to remove the names method from the QueryParser interface.
2016-04-05 15:38:53 +02:00
Dimitrios Liappis e6d6eaa1bf Reverting 127.0.0.1 to localhost 2016-04-04 19:03:13 +03:00
Dimitrios Liappis be602ee026 clarify that dnf is mostly for Fedora >=22
RHEL7 and CentOS 7 are still on yum
2016-04-04 16:10:27 +03:00
Dimitrios Liappis 22e6567995 Replacing typo with distro safe address
apart from locahost typo, the issue is that localhost is not 100% safe
for all distros with IPv6.

For example fedora23 defines localhost4 and localhost6 (among other
aliases) so `curl localhost:9200` doesn't work.

For this reason, I think it's safer to replace localhost with 127.0.0.1
2016-04-04 16:01:03 +03:00
Clinton Gormley 7d4ed5b19e Changed JAVA_OPTS to ES_JAVA_OPTS in plugin docs 2016-04-03 16:52:37 +02:00
Clinton Gormley 6ff947427d Fixed plugin docs links to dir layouts 2016-04-03 16:50:28 +02:00
Clinton Gormley 06604708d4 Docs: Complete rewrite of setup, installation, and configuration docs 2016-04-03 16:09:48 +02:00
Joe Hillenbrand f355bb2a9b Merge pull request #17431 from joehillen/issue/17430
Set MAX_OPEN_FILES to 65536
2016-03-31 10:33:40 -07:00
Nik Everett 3caa5a929a Documentation for scroll size in reindex 2016-03-31 12:21:32 -04:00
javanna 0ea6dba04b [DOCS] Update _cat/nodes docs after recent changes 2016-03-31 16:36:29 +02:00
Daniel Mitterdorfer 6f0581c67c Update version and major-version variable in reference docs
Checked with @clintongormley
2016-03-31 16:17:46 +02:00
Daniel Mitterdorfer b7b65b93d1 Merge remote-tracking branch 'danielmitterdorfer/update-cluster-stats-docs' 2016-03-31 14:56:07 +02:00
Daniel Mitterdorfer 930ce1bfec Add up-to-date example of cluster stats API output 2016-03-31 14:41:37 +02:00
Martijn van Groningen 91fc9daeda Merge pull request #17443 from martijnvg/reindex_small_doc_fixed
Fix small reindex doc issues
2016-03-31 14:40:16 +02:00
Clinton Gormley 637da86e5f Docs: Updated the upgrade API docs to explain that the reindex API should be used instead 2016-03-31 14:34:31 +02:00
Martijn van Groningen 6bcfc730d7 reindex: fix small documentation issues 2016-03-31 12:59:12 +02:00
Clinton Gormley 18c5ea8599 Update templates.asciidoc
Relates to #17375
2016-03-31 11:15:57 +02:00
Adrien Grand 51d9574916 Fix reference guide about the default value of min_should_match. 2016-03-31 10:49:02 +02:00
Adrien Grand 26a0fb37a4 Add examples of useful dynamic templates to the docs. #17413 2016-03-31 09:45:11 +02:00
Joe Hillenbrand 81b6326891 Set MAX_OPEN_FILES to 65536
Relates to #17430
2016-03-30 16:44:37 -07:00
Igor Motov e073b0c75d Add ability to group tasks by common parent
By default, tasks are grouped by node. However, task execution in elasticsearch can be quite complex and an individual task that runs on a coordinating node can have many subtasks running on other nodes in the cluster. This commit makes it possible to list task grouped by common parents instead of by node. When this option is enabled all subtask are grouped under the coordinating node task that started all subtasks in the group. To group tasks by common parents, use the following syntax:

 GET /tasks?group_by=parents
2016-03-30 17:50:27 -04:00
Clinton Gormley 48c566da73 Documented how tribe node clients inherit their settings
Closes #17309
2016-03-30 20:05:44 +02:00
Adrien Grand d7179cafcc Add a soft limit on the number of shards that can be queried in a single search request. #17396
This commit adds the new `action.search.shard_count.limit` setting which
configures the maximum number of shards that can be queried in a single search
request. It has a default value of 1000.
2016-03-30 16:55:01 +02:00
Martijn van Groningen 7e2696c570 Refactored inner hits parsing and intoduced InnerHitBuilder
Both top level and inline inner hits are now covered by InnerHitBuilder.
Although there are differences between top level and inline inner hits,
they now make use of the same builder logic.

The parsing of top level inner hits slightly changed to be more readable.
Before the nested path or parent/child type had to be specified as encapsuting
json object, now these settings are simple fields. Before this was required
to allow streaming parsing of inner hits without missing contextual information.

Once some issues are fixed with inline inner hits (around multi level hierachy of inner hits),
top level inner hits will be deprecated and removed in the next major version.
2016-03-30 15:15:56 +02:00
Adrien Grand fc47007e17 Add a soft limit on the mapping depth. #17400
This commit adds the new `index.mapping.depth.limit` setting which controls the
maximum mapping depth that is allowed. It has a default value of 20.
2016-03-30 14:37:00 +02:00
Simon Willnauer 2514681f66 updateing filtering.asciidoc to also use 'node.attr' namespace 2016-03-30 14:11:59 +02:00
Simon Willnauer ad24653948 update allocation_awareness.asciidoc to also use 'node.attr' namespace 2016-03-30 13:52:45 +02:00
Simon Willnauer 8b075dbb75 Remove ability to specify arbitrary node attributes with `node.` prefix
Today the basic node settings like `node.data` and `node.master` can't really be fully validated
since we allow to specify custom user attributes on the node level. We have to, in order to
support that, add a wildcard setting for `node.*` to let these setting pass validation.
Instead we should require a more contraint prefix like `node.attr.` that defines a namespace
that is reserved for user attributes.
This commit adds a new namespace for attributes in `node.attr`.

Closes #17280
2016-03-30 13:29:48 +02:00
Isabel Drost-Fromm f27399dc0e Merge pull request #17282 from MaineC/deprecation/sort-option-reverse-removal
Remove deprecated reverse option from sorting
2016-03-30 11:02:19 +02:00
javanna 19eeb68bc4 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 21:53:22 +02:00
Spencer f07b35b328 Merge pull request #17368 from spalger/docs/clarifyMinMasterCheck
[docs] clarify where discovery.zen.minimum_master_node is required
2016-03-29 11:44:50 -07:00
javanna ae34c20a62 add node.client breaking changes to migrate guide 2016-03-29 20:33:59 +02:00
javanna 061f09d9a4 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 20:19:33 +02:00
Yanjun Huang 361adcf387 Add limit to total number of fields in mapping. #17357
This is to prevent mapping explosion when dynamic keys such as UUID are used as field names. index.mapping.total_fields.limit specifies the total number of fields an index can have. An exception will be thrown when the limit is reached. The default limit is 1000. Value 0 means no limit. This setting is runtime adjustable

Closes #11443
2016-03-29 19:39:46 +02:00
Igor Motov c356b30cff Update task management docs to reflect the latest changes in the interface
Brings docs in line with new list task syntax and adds task cancellation API docs.
2016-03-29 12:26:37 -04:00
Tal Levy 9ac3887139 Merge pull request #17263 from talevy/auto-convert
add  type conversion support to ConvertProcessor
2016-03-29 07:57:57 -07:00
Tal Levy 2064fe3985 add type conversion support to ConvertProcessor 2016-03-29 07:56:53 -07:00
Clinton Gormley 798e4281fa Added experimental annotation to the update-by-query and reindex docs 2016-03-29 15:06:27 +02:00
javanna 8fc9dbbb99 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 14:27:04 +02:00
Clinton Gormley 978b24327e Docs: Included Nodes Task API and tidied reindex/update-by-query 2016-03-29 13:51:11 +02:00
Isabel Drost-Fromm 407e2cdcf9 Merge branch 'master' into deprecation/sort-option-reverse-removal
Conflicts:
	core/src/main/java/org/elasticsearch/search/sort/ScoreSortBuilder.java
	core/src/test/java/org/elasticsearch/search/sort/FieldSortBuilderTests.java
2016-03-29 11:04:02 +02:00
javanna de5cbda8e7 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 10:48:47 +02:00
Lee Hinman 80ab366de4 Add API to explain why a shard is or isn't assigned
This adds a new `/_cluster/allocation/explain` API that explains why a
shard can or cannot be allocated to nodes in the cluster. Additionally,
it will show where the master *desires* to put the shard, according to
the `ShardsAllocator`.

It looks like this:

```
GET /_cluster/allocation/explain?pretty
{
  "index": "only-foo",
  "shard": 0,
  "primary": false
}
```

Though, you can optionally send an empty body, which means "explain the
allocation for the first unassigned shard you find".

The output when a shard is unassigned looks like this:

```
{
  "shard" : {
    "index" : "only-foo",
    "index_uuid" : "KnW0-zELRs6PK84l0r38ZA",
    "id" : 0,
    "primary" : false
  },
  "assigned" : false,
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",
    "at" : "2016-03-22T20:04:23.620Z"
  },
  "nodes" : {
    "V-Spi0AyRZ6ZvKbaI3691w" : {
      "node_name" : "Susan Storm",
      "node_attributes" : {
        "bar" : "baz"
      },
      "final_decision" : "NO",
      "weight" : 0.06666675,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    },
    "Qc6VL8c5RWaw1qXZ0Rg57g" : {
      "node_name" : "Slipstream",
      "node_attributes" : {
        "bar" : "baz",
        "foo" : "bar"
      },
      "final_decision" : "NO",
      "weight" : -1.3833332,
      "decisions" : [ {
        "decider" : "same_shard",
        "decision" : "NO",
        "explanation" : "the shard cannot be allocated on the same node id [Qc6VL8c5RWaw1qXZ0Rg57g] on which it already exists"
      } ]
    },
    "PzdyMZGXQdGhqTJHF_hGgA" : {
      "node_name" : "The Symbiote",
      "node_attributes" : { },
      "final_decision" : "NO",
      "weight" : 2.3166666,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    }
  }
}
```

And when the shard *is* assigned, the output looks like:

```
{
  "shard" : {
    "index" : "only-foo",
    "index_uuid" : "KnW0-zELRs6PK84l0r38ZA",
    "id" : 0,
    "primary" : true
  },
  "assigned" : true,
  "assigned_node_id" : "Qc6VL8c5RWaw1qXZ0Rg57g",
  "nodes" : {
    "V-Spi0AyRZ6ZvKbaI3691w" : {
      "node_name" : "Susan Storm",
      "node_attributes" : {
        "bar" : "baz"
      },
      "final_decision" : "NO",
      "weight" : 1.4499999,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    },
    "Qc6VL8c5RWaw1qXZ0Rg57g" : {
      "node_name" : "Slipstream",
      "node_attributes" : {
        "bar" : "baz",
        "foo" : "bar"
      },
      "final_decision" : "CURRENTLY_ASSIGNED",
      "weight" : 0.0,
      "decisions" : [ {
        "decider" : "same_shard",
        "decision" : "NO",
        "explanation" : "the shard cannot be allocated on the same node id [Qc6VL8c5RWaw1qXZ0Rg57g] on which it already exists"
      } ]
    },
    "PzdyMZGXQdGhqTJHF_hGgA" : {
      "node_name" : "The Symbiote",
      "node_attributes" : { },
      "final_decision" : "NO",
      "weight" : 3.6999998,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    }
  }
}
```

Only "NO" decisions are returned by default, but all decisions can be
shown by specifying the `?include_yes_decisions=true` parameter in the
request.

Resolves #14593
2016-03-28 15:21:02 -06:00
spalger ce44bbfadf [docs] clarify where discovery.zen.minimum_master_node is required
https://github.com/elastic/elasticsearch/pull/17288 added a check to enforce that the `discovery.zen.minimum_master_nodes` configuration is set when nodes have the `host`, `port`, or `bind_host` set in either `transport` or general `network` configuration sections. This was documented incorrectly as "nodes that are bound to a non-loopback interface", which lead to confusion as I set `network.host: "localhost"` and the check was still failing.

This change updates the docs to detail the actual check. I think it also highlights how complex the check is and the need for a simpler solution.
2016-03-28 12:53:40 -07:00
javanna a9f4982c40 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-25 20:16:40 +01:00
Clinton Gormley 3da7393b00 Tidied up percolator doc annotations 2016-03-25 15:36:51 +01:00
Boaz Leskes b8227a7222 Enforce `discovery.zen.minimum_master_nodes` is set when bound to a public ip #17288
discovery.zen.minimum_master_nodes is the single most important setting to set on a production cluster. We have no way of supplying a good default so it must be set by the user. Binding a node to a public IP (as opposed to the default local host) is a good enough indication that a node will be part of a production cluster cluster and thus it's a good tradeoff to enforce the settings. Note that nothing prevent users from setting it to 1 in a single node cluster.

Closes #17288
2016-03-25 12:56:20 +01:00
Jason Tedor 7f0134e725 Revert "Merge pull request #16843 from xuzha/s3-encryption"
This reverts commit 37a183d9ed, reversing
changes made to 08903f1ed8.
2016-03-24 17:11:02 -04:00
Xu Zhang 7499e3aa4a Update and rebase the init implementation.
Also removes the MD5 checks from our side, AWS S3 SDK java is doing the
check.
2016-03-24 11:21:40 -07:00
Clinton Gormley 08903f1ed8 Tidied the Painless docs and added the experimental tag 2016-03-24 18:34:40 +01:00
javanna 27d4994aff Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-24 18:10:11 +01:00
Boaz Leskes 6dd164d0bd Include pings from client nodes in master election
We currently have a `discovery.zen.master_election.filter_client` setting that control whether their ping responses are ignored for master election (which is the current default). With the push to treat client nodes as normal nodes (and promote the transport/rest clients for client work), this should be changed. This commit remove this setting and it's companion `discovery.zen.master_election.filter_data` setting (currently defaulting to  false) in favor of singe `discovery.zen.master_election.ignore_non_master_pings` setting with more intuitive name (defaulting to false).

Resolves #17325
Closes #17329
2016-03-24 17:48:05 +01:00
Clinton Gormley 0e7054cf02 Merge pull request #17272 from pengqiuyuan/master
Update template-query.asciidoc
2016-03-24 16:49:49 +01:00
javanna ce86fc5647 Cluster Stats: remove mem section
The available memory metric was always set to `0` since 2.0.beta1 (bug).  was left behind but never set. Turns out the section wasn't that useful, as it would only output the total memory available throughout all nodes in the cluster. We decided to remove the section then.
2016-03-24 15:49:27 +01:00
Isabel Drost-Fromm 08d989d9b6 Merge branch 'master' into deprecation/sort-option-reverse-removal
Conflicts:
	core/src/main/java/org/elasticsearch/search/sort/FieldSortBuilder.java
	core/src/main/java/org/elasticsearch/search/sort/ScoreSortBuilder.java
2016-03-24 12:06:10 +01:00
Isabel Drost-Fromm 801d178ade Remove mention of reverse in docs and add to migration doc 2016-03-24 12:04:31 +01:00
Jim Ferenczi da42f199bd Enforce isolated mode for all plugins
This commit removes the isolated option, each plugin have its own classloader.
2016-03-24 09:17:33 +01:00
Areek Zillur e16e113691 Remove suggest threadpool
In #17198, we removed suggest transport action, which
used the `suggest` threadpool to execute requests. Now
`suggest` threadpool is unused and suggest requests are
executed on the `search` threadpool.
2016-03-23 18:01:45 -04:00
debadair ad28fb9ec0 Docs: Adding Painless to the Scripting documentation. 2016-03-23 13:52:40 -07:00
Areek Zillur 442a6e0009 document suggest stats being merged with search stats 2016-03-23 16:37:57 -04:00
Areek Zillur e7e93f98e3 add migration guide to use search api for suggest 2016-03-23 16:37:57 -04:00
David Pilato 6e80b5f2dd Merge branch 'fix/17244-s3-chunk-buffer-sizes' 2016-03-23 18:51:54 +01:00
Nik Everett bbe0f3af3b Merge pull request #17255 from simonw/patch-2
Link to named queries docs from bool query page
2016-03-23 11:51:40 -04:00
javanna 030453d320 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-23 11:25:34 +01:00
javanna 4bfef1fde1 [DOCS] clarify that tribe node connects to every node in every cluster
Closes #16756
2016-03-23 10:43:58 +01:00
David Pilato e907b7c11e Check that S3 setting `buffer_size` is always lower than `chunk_size`
We can be better at checking `buffer_size` and `chunk_size` for S3 repositories.
For example, we know that:

* `buffer_size` should be more than `5mb`
* `chunk_size` should be no more than `5tb`
* `buffer_size` should be lower than `chunk_size`

Otherwise, setting `buffer_size` is useless.

For the record:

`chunk_size` is a Snapshot setting whatever the implementation is.
`buffer_size` is an S3 implementation setting.

Let say that you are snapshotting a 500mb file. If you set `chunk_size` to `200mb`, then Snapshot service will call S3 repository to snapshot 3 files with the following sizes:

* `200mb`
* `200mb`
* `100mb`

If you set `buffer_size` to `100mb` (AWS maximum size recommendation), the first file of `200mb` will be uploaded on S3 using the multipart feature in 2 chunks and the workflow is basically the following:

* create the multipart request and get back an `id` from AWS S3 platform
* upload part1: `100mb`
* upload part2: `100mb`
* "commit" the full upload using the `id`.

Closes #17244.
2016-03-23 10:39:54 +01:00
pengqiuyuan 80ef18c3b2 Update template-query.asciidoc 2016-03-23 17:14:31 +08:00
Simon Willison fdac0c7c6c Link to named queries docs from bool query page
The named queries feature only makes sense with bool queries, but was not cross-referenced from the bool query documentation page.
2016-03-22 12:07:57 -07:00
Nik Everett da96b6e41d [reindex] Add thottling support
The throttle is applied when starting the next scroll request so that its
timeout can include the throttle time.
2016-03-22 12:34:14 -04:00
Colin Goodheart-Smithe d6fe7515fd Merge pull request #17243 from colings86/docs/searchRequestBreakingChanges
added breaking changes for the Java API to the breaking changes doc for 5.0
2016-03-22 15:58:40 +00:00
Colin Goodheart-Smithe 25c4446942 iter 2016-03-22 15:58:12 +00:00
Colin Goodheart-Smithe ee7e84acc3 review comments 2016-03-22 15:34:47 +00:00
Adrien Grand c52b1f3a7c An `exists` query on an object should query a single term.
Currently if you run an `exists` query on an object, it will resolve all sub
fields and create a disjunction for all those fields. However the `_field_names`
mapper indexes paths for objects so we could query object paths directly.

I also changed the query parser to reject `exists` queries if the `_field_names`
field is disabled since it would be a big performance trap.
2016-03-22 16:26:45 +01:00
Adrien Grand b42f66c8ac Document 5.0 mapping changes. 2016-03-22 16:22:58 +01:00
Colin Goodheart-Smithe b8a96d9a65 added breaking changes for the Java API to the breaking changes doc for 5.0 2016-03-22 14:39:16 +00:00
Luca Cavanna 3764b3ff80 Merge pull request #17145 from alexshadow007/fix-17101
Fix column aliases in _cat/indices, _cat/nodes and _cat/shards APIs
2016-03-22 15:37:21 +01:00
Jun Ohtani a9a0f262af Analysis Kuromoji: Add nbest option and NumberFilter
Add nbest_cost and nbest_examples parameter to KuromojiTokenizerFactory
Add KuromojiNumberFilterFactory
2016-03-22 20:09:56 +09:00
javanna eebd0cfccd Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-22 10:34:40 +01:00
Simon Willnauer 7f16a1d9a7 Improve upgrade experience of node level index settings
In 5.0 we don't allow index settings to be specified on the node level ie.
in yaml files or via commandline argument. This can cause problems during
upgrade if this was used extensively. For instance if analyzers where
specified on a node level this might cause the index to be closed when
imported (see #17187). In such a case all indices relying on this
must be updated via `PUT /${index}/_settings`. Yet, this API has slightly
different semantics since it overrides existing settings. To make this less
painful this change adds a `preserve_existing` parameter on that API to ensure
we have the same semantics as if the setting was applied on the node level.

This change also adds a better error message and a change to the migration guide
to ensure upgrades are smooth if index settings are specified on the node level.

If a index setting is detected this change fails the node startup and prints a message
like this:
```
*************************************************************************************
Found index level settings on node level configuration.

Since elasticsearch 5.x index level settings can NOT be set on the nodes
configuration like the elasticsearch.yaml, in system properties or command line
arguments.In order to upgrade all indices the settings must be updated via the
/${index}/_settings API. Unless all settings are dynamic all indices must be closed
in order to apply the upgradeIndices created in the future should use index templates
to set default values.

Please ensure all required values are updated on all indices by executing:

curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
  "index.number_of_shards" : "1",
  "index.query.default_field" : "main_field",
  "index.translog.durability" : "async",
  "index.ttl.disable_purge" : "true"
}'
*************************************************************************************
```
2016-03-21 20:12:18 +01:00
javanna bf390a935e Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-21 17:18:23 +01:00
Martijn van Groningen 000e419795 docs: fix link 2016-03-21 14:09:30 +01:00