Commit Graph

2780 Commits

Author SHA1 Message Date
Lee Hinman 864ba8dac1 Merge remote-tracking branch 'dakrone/there-can-be-only-one2' 2016-05-13 10:28:41 -06:00
Lee Hinman 9bcdafedda Allow only a single extension for a scripting engine
Previously multiple extensions could be provided, however, this can lead
to confusion with on-disk scripts (ie, "foo.js" and "foo.javascript")
having different content. Only a single extension is now supported.

The only language currently supporting multiple extensions was the
Javascript engine ("js" and "javascript"). It now only supports the
`.js` extension.

Relates to #10598
2016-05-13 09:54:31 -06:00
Lee Hinman d5b75491dc Merge remote-tracking branch 'dakrone/remove-script-sandbox' 2016-05-13 09:50:39 -06:00
Christoph Büscher a40c397c67 Don't allow `fuzziness` for `multi_match` types cross_fields, phrase and phrase_prefix
Currently `fuzziness` is not supported for the `cross_fields` type
of the `multi_match` query since it complicates the logic that
blends the term queries that cross_fields uses internally. At the
moment using this combination is silently ignored, which can lead to
confusions. Instead we should throw an exception in this case.
The same is true for phrase and phrase_prefix type.

Closes #7764
2016-05-13 17:32:14 +02:00
Lee Hinman efff3918d8 Remove support for mulitple languages per scripting engine 2016-05-13 09:24:31 -06:00
Lee Hinman a4060f7436 Remove vestiges of script engine sandboxing
This removes all the mentions of the sandbox from the script engine
services and permissions model. This means that the following settings
are no longer supported:

```yaml
script.inline: sandbox
script.stored: sandbox
```

Instead, only a `true` or `false` value can be specified.

Since this would otherwise break the default-allow parameter for
languages like expressions, painless, and mustache, all script engines
have been updated to have individual settings, for instance:

```yaml
script.engine.groovy.inline: true
```

Would enable all inline scripts for groovy. (they can still be
overridden on a per-operation basis).

Expressions, Painless, and Mustache all default to `true` for inline,
file, and stored scripts to preserve the old scripting behavior.

Resolves #17114
2016-05-13 09:24:31 -06:00
Adrien Grand 638da06c1d Add back support for `ip` range aggregations. #17859
This commit adds support for range aggregations on `ip` fields. However it will
only work on 5.x indices.

Closes #17700
2016-05-13 17:22:01 +02:00
Shane Connelly 6e566cdf6c Merge pull request #18313 from eskibars/docs_test
Add CONSOLE tests to aliases documentation
2016-05-12 20:50:59 -07:00
Shane Connelly dd825c8730 Add TESTRESPONSE code 2016-05-12 18:44:08 -07:00
Nik Everett 8155e1efda [docs] Add wait_for_status=yellow
Another unstable snippet....

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-os-compatibility/os=sles/402/console
2016-05-12 17:53:34 -04:00
Aaron Mildenstein 10381de815 Removing the superfluous 's'
Pretty sure we're not making a brand new `/var/logs` directory when everything else goes into `/var/log`

Relates #18305
2016-05-12 16:42:27 -04:00
Shane Connelly 73c9cfbe8d Adds CONSOLE tests to aliases documentation 2016-05-12 11:25:13 -07:00
Nik Everett 94317b1d33 Add wait_for_health=yellow to reindex snippets
This should help the tests pass more consistently. Should.

This also removes from `?pretty` from the docs. It isn't a thing
with `// CONSOLE`.

Relates to
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-os-compatibility/os=oraclelinux/399/console
2016-05-12 12:20:06 -04:00
Zachary Tong 5ee5cc25cc Move AsciiFolding earlier in FingerprintAnalyzer filter chain
Rearranges the FingerprintAnalyzer so that AsciiFolding comes earlier in the chain (after lowercasing, before stop removal, for maximum deduping power)

Closes #18266
2016-05-12 09:34:15 -04:00
Isabel Drost-Fromm 36cd69c6ac Fix build failure 2016-05-12 13:23:06 +02:00
Isabel Drost-Fromm 0ad87b25cf Something messed with auto-indent. Fixed now. 2016-05-12 12:58:22 +02:00
Isabel Drost-Fromm 126ff91bf6 Fix indent 2016-05-12 12:30:33 +02:00
Isabel Drost-Fromm 6d5e24726f Fix test failures. 2016-05-12 12:29:18 +02:00
Robert Muir 25dd64250b painless: remove input, support params instead 2016-05-11 21:32:10 -04:00
Robert Muir c5532d3df0 add a rest test for this that seems to work, fix the documentation. thanks @s1monw 2016-05-11 16:07:08 -04:00
Adrien Grand 866a5459f0 Make significant terms work on fields that are indexed with points. #18031
It will keep using the caching terms enum for keyword/text fields and falls back
to IndexSearcher.count for fields that do not use the inverted index for
searching (such as numbers and ip addresses). Note that this probably means that
significant terms aggregations on these fields will be less efficient than they
used to be. It should be ok under a sampler aggregation though.

This moves tests back to the state they were in before numbers started using
points, and also adds a new test that significant terms aggs fail if a field is
not indexed.

In the long term, we might want to follow the approach that Robert initially
proposed that consists in collecting all documents from the background filter in
order to compute frequencies using doc values. This would also mean that
significant terms aggregations do not require fields to be indexed anymore.
2016-05-11 16:52:58 +02:00
Isabel Drost-Fromm 85f1ab44d9 Convert rest of query-dsl docs to be run in tests 2016-05-11 14:37:19 +02:00
Clinton Gormley 97a41ee973 First pass at improving analyzer docs (#18269)
* Docs: First pass at improving analyzer docs

I've rewritten the intro to analyzers plus the docs
for all analyzers to provide working examples.

I've also removed:

* analyzer aliases (see #18244)
* analyzer versions (see #18267)
* snowball analyzer (see #8690)

Next steps will be tokenizers, token filters, char filters

* Fixed two typos
2016-05-11 14:17:56 +02:00
Jason Tedor 2bf585e642 Require /bin/bash in packaging
This commit adds a hard requirement to the RPM and Debian packages for
/bin/bash to be present, and adds a note regarding this to the migration
docs.

Relates #18259
2016-05-10 21:17:09 -04:00
Lee Hinman 1c54033e92 Merge branch 'pr/18068' 2016-05-10 08:27:43 -06:00
Alexander Kazakov 667a091205 Add note about cat field data API changes into migration doc 2016-05-10 16:41:21 +03:00
Isabel Drost-Fromm ab4367c07e Add CONSOLE to span queries.
... and range, and terms...
2016-05-10 12:59:44 +02:00
Isabel Drost-Fromm 754a677ff5 Use TESTSETUP instead of TEST[continued] 2016-05-10 12:46:00 +02:00
Isabel Drost-Fromm a865090cf3 CONSOLE is the new AUTOSENSE 2016-05-10 12:42:17 +02:00
Clinton Gormley 02cf429e53 Update full-text-queries.asciidoc
Updated the full text query intro to add `match_phrase` and `match_phrase_prefix`
2016-05-10 12:18:21 +02:00
Isabel Drost-Fromm e486560ea8 Add Autosense annotation for query dsl testing
this adds the autosense annotation to a couple of query dsl
docs files and fixes the snippets to work in the tests along
the way.
2016-05-10 11:54:48 +02:00
jasonquick 7bf8e6719c Fix minor typo (#18222) 2016-05-10 09:39:42 +02:00
Robert Muir ba2fe156e8 Switch over dynamic method calls, loads and stores to invokedynamic.
Remove performance hack for accessing a document's fields, its not needed.
Add support for accessing is-getter methods like List.isEmpty() as .empty

Closes #18201
2016-05-09 21:44:32 -04:00
Zack Dever 5a7edf992c Add missing comma in JSON data in curl example
Relates #18227
2016-05-09 21:20:37 -04:00
Jason Tedor 7d1fd17172 Remove plugin script parsing of system properties
The plugin script parses command-line options looking for Java system
properties and extracts these arguments to pass to the java command when
starting the JVM. Since elasticsearch-plugin allows arbitrary user
arguments to the JVM via ES_JAVA_OPTS, this parsing is unnecessary. This
commit removes this unnecessary 

Relates #18207
2016-05-09 13:06:18 -04:00
Clinton Gormley 3f594089c2 Renamed all AUTOSENSE snippets to CONSOLE (#18210) 2016-05-09 15:42:23 +02:00
Nik Everett 2528934411 CONSOLE is the new AUTOSENSE
This makes the test generation support both while we move from
`// AUTOSENSE` to `// CONSOLE`.

Will bother #18160
2016-05-09 09:23:23 -04:00
Clinton Gormley b352a90454 Correct docs for dynamic mapping of fields
Floating point numbers are added as `float`, and Strings are added as `text` with `keyword sub-field
2016-05-07 17:16:31 +02:00
Jason Tedor 4ae5c0d93d Add link to field data docs from field data fields
This commit adds a link to the field data docs from the field data
fields docs for completeness.

Closes #17672
2016-05-07 11:09:17 -04:00
Tanguy Leroux 0ff5652fff Add node name to Cat Recovery
closes #8041
2016-05-06 16:59:53 +02:00
Britta Weber 901c25b7ff Merge pull request #18183 from brwe/exclude-all-but-text-from-wildcard-highlighting
Exclude all but string fields from highlighting if wildcards are used…
2016-05-06 16:57:25 +02:00
Britta Weber ddebbb9536 add string to documentation 2016-05-06 16:50:34 +02:00
Alexander Reelsen 473be01373 Documentation: Switch to https for debian repository 2016-05-06 16:05:19 +02:00
Nik Everett cb40b986d1 Allow leading `/` in AUTOSENSE path
Relates to #18160
2016-05-06 09:26:19 -04:00
Britta Weber d3c5f865be Exclude all but string fields from highlighting if wildcards are used in fieldname
We should prevent highlighting if a field is anything but a text or keyword field.
However, someone might implement a custom field type that has text and still want to
highlight on that. We cannot know in advance if the highlighter will be able to
highlight such a field and so we do the following:
If the field is only highlighted because the field matches a wildcard we assume
it was a mistake and do not process it.
If the field was explicitly given we assume that whoever issued the query knew
what they were doing and try to highlight anyway.

closes #17537
2016-05-06 13:41:16 +02:00
Clinton Gormley 4ddf916aab Removed scripting docs for docs[field].multiValued
Closes #18164
2016-05-06 10:40:34 +02:00
Adrien Grand de8354dd7f Allow binary sort values. #17959
The `ip` field uses a binary representation internally. This breaks when
rendering sort values in search responses since elasticsearch tries to write a
binary byte[] as an utf8 json string. This commit extends the `DocValueFormat`
API in order to give fields a chance to choose how to render values.

Closes #6077
2016-05-06 09:27:02 +02:00
Clinton Gormley c55df195c5 Fixed bad asciidoc 2016-05-06 09:25:58 +02:00
Adrien Grand 7d8708716e QueryBuilder does not need generics. #18133
QueryBuilder has generics, but those are never used: all call sites use
`QueryBuilder<?>`. Only `AbstractQueryBuilder` needs generics so that the base
class can contain a default implementation for setters that returns `this`.
2016-05-06 08:38:20 +02:00
Nik Everett f3b2ab822d Another wait_for_yellow to the docs
All in service of the snippets passing consistently.
2016-05-05 19:03:23 -04:00
Robert Muir e3ce6c9048 Painless: add fielddata accessors (.value/.values/.distance()/etc)
This gives better coverage and consistency with the scripting APIs, by
whitelisting the primary search scripting API classes and using them instead
of only Map and List methods.

For example, accessing fields can now be done with `.value` instead of `.0`
because `getValue()` is whitelisted. For now, access to a document's fields in
this way (loads) are fast-pathed in the code, to avoid dynamic overhead.

Access to geo fields and geo distance functions is now supported.

TODO: date support (e.g. whitelist ReadableDateTime methods as a start)
TODO: improve docs (like expressions and groovy have for document's fields)
TODO: remove fast-path hack

Closes #18169

Squashed commit of the following:

commit ec9f24b2424891a7429bb4c0a03f9868cba0a213
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 17:59:37 2016 -0400

    cutover to <Def> instead of <Object> here

commit 9edb1550438acd209733bc36f0d2e0aecf190ecb
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 17:03:02 2016 -0400

    add fast-path for docvalues field loads

commit f8e38c0932fccc0cfa217516130ad61522e59fe5
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 16:47:31 2016 -0400

    Painless: add fielddata accessors (.value/.values/.distance()/etc)
2016-05-05 18:38:41 -04:00
Nik Everett 3912761572 [docs] Add wait_until_yellow to fix build failure
The snippet in the docs creates and index and uses it with the
_analyze api. The trouble is that if the index hasn't been created
fully the _analyze API will fail. This adds a
GET _cluster/health?wait_for_status=yellow
which fixes the issue.

While this does make the docs more cluttered, it also makes the snippets
actually runnable.

Closes #18165
2016-05-05 16:02:00 -04:00
Nik Everett 4b1c116461 Generate and run tests from the docs
Adds infrastructure so `gradle :docs:check` will extract tests from
snippets in the documentation and execute the tests. This is included
in `gradle check` so it should happen on CI and during a normal build.

By default each `// AUTOSENSE` snippet creates a unique REST test. These
tests are executed in a random order and the cluster is wiped between
each one. If multiple snippets chain together into a test you can annotate
all snippets after the first with `// TEST[continued]` to have the
generated tests for both snippets joined.

Snippets marked as `// TESTRESPONSE` are checked against the response
of the last action.

See docs/README.asciidoc for lots more.

Closes #12583. That issue is about catching bugs in the docs during build.
This catches *some* bugs in the docs during build which is a good start.
2016-05-05 13:58:03 -04:00
Jack Conradson 2cae575f53 Added single-quoted strings.
Closes #18150
2016-05-05 09:26:02 -07:00
Nik Everett 230697c202 [reindex] Switch throttle to Float.POSITIVE_INFITINTY/"unlimited"
All other values are errors.

Add java test for throttling. We had a REST test but it only ran against
one node so it didn't catch serialization errors.

Add Simple round trip test for rethrottle request
2016-05-04 16:14:32 -04:00
Ali Beyad 67c0734bf3 Update misc.asciidoc
Added documentation for the cluster.indices.tombstones.size property for maximum tombstones in the cluster state.
2016-05-04 15:21:47 -04:00
Robert Muir 7656d7ea73 docs: remove null from expressions case.
Expressions don't have nulls, only doubles. If the field is missing, then its
treated as 0.0. You can query .empty to see if its missing and substitute something else.

See https://github.com/elastic/elasticsearch/pull/18132#discussion_r62068494
2016-05-04 12:50:12 -04:00
Clinton Gormley 34d90b041f Reorganise scripting docs (#18132)
* Reorganize scripting documentation

* Further changes to tidy up scripting docs

Closes #18116

* Add note about .lat/lon potentially returning null

* Added .value to expressions example

* Fixed two bad ASCIIDOC links
2016-05-04 18:17:10 +02:00
Jim Ferenczi 052191f2a2 Add the ability to use the breadth_first mode with nested aggregations (such as `top_hits`) which require access to score information.
The score is recomputed lazily for each document belonging to a top bucket.
Relates to #9825
2016-05-04 15:35:45 +02:00
Isabel Drost-Fromm 9a90bdce01 Add deprecation warnings for ignore_malformed to reference docs 2016-05-04 12:04:28 +02:00
Isabel Drost-Fromm 6b9ac46402 Merge branch 'master' into enhancement/switch_geodistancesortbuilder_to_geovalidationmethod 2016-05-04 11:30:15 +02:00
Adrien Grand 51a53c55cb Update store documentation after #17616. 2016-05-04 08:53:11 +02:00
Adrien Grand 80dbe31d59 Add note about using ipv6 addresses in `query_string`. 2016-05-04 08:53:11 +02:00
Kelsey DeJesus-Banos 929187c71b Update function-score-query.asciidoc (#18115) 2016-05-03 19:29:57 +02:00
Lee Hinman 5648253d45 Add documentation for shard store output in allocation explain API
Relates to #17689
2016-05-03 09:51:15 -06:00
Clinton Gormley d01526b6ab Bumped docs version 2016-05-03 15:35:33 +02:00
Daniel Mitterdorfer 0a6f40c7f5 Enable HTTP compression by default with compression level 3
With this commit we compress HTTP responses provided the client
supports it (as indicated by the HTTP header 'Accept-Encoding').

We're also able to process compressed HTTP requests if needed.

The default compression level is lowered from 6 to 3 as benchmarks
have indicated that this reduces query latency with a negligible
increase in network traffic.

Closes #7309
2016-05-03 08:53:15 +02:00
Robert Muir fff82db681 Add tests/doc for boolean fields with expressions 2016-05-02 18:13:03 -04:00
Robert Muir 693c1f6671 Support geo_point fields in lucene expressions.
Closes #18096
2016-05-02 17:49:21 -04:00
Robert Muir 28409e4509 Add support for .empty to expressions, and some docs improvements
Closes #18077
2016-05-02 09:07:25 -04:00
Clinton Gormley 9fee8c76af Added release notes for 5.0.0-alpha2 2016-05-02 14:21:59 +02:00
Clinton Gormley 7c8397d99b Update keyword.asciidoc
`ignore_above` doesn't apply to analyzed `text` fields
2016-05-02 13:47:14 +02:00
Todd Dicken 9162fd27a0 Change qoutes to quotes (#18078)
Updated misspelling in documentation
2016-05-02 13:06:07 +02:00
Josef Salyer b2ce2f5afa Fixed the documentation formatting. (#17990)
The source code example for the initial example was missing the correct JSON object formatting and syntax.  That has been fixed with my change.
2016-04-29 18:07:48 +02:00
Robin Joseph e322903f2c Fix typo in include-in-all.asciidoc (#18055) 2016-04-29 18:03:22 +02:00
Martijn van Groningen 7aca1389e2 ingest: Add `date_index_name` processor.
Closes #17814
2016-04-29 17:20:48 +02:00
David Pilato 2232a7cdf3 Merge branch 'pr/cat-size-time-units' 2016-04-29 15:09:14 +02:00
Shane Connelly 713c0df3a3 Merge pull request #17994 from eskibars/master
Add new IPv6 types to docs where it's supported
2016-04-29 06:00:32 -07:00
Clinton Gormley 04bd55d61c Added perl migration script for indexed scripts to migration docs 2016-04-29 14:18:27 +02:00
Martijn van Groningen 6c3beaa2eb Drop top level inner hits in favour of inner hits defined in the query dsl.
Fix a limitation that prevent from hierarchical inner hits be defined in query dsl.

Removed the nested_path, parent_child_type and query options from inner hits dsl. These options are only set by ES
upon parsing the has_child, has_parent and nested queries are using their respective query builders.

These options are still used internally, when these options are set a new private copy is created based on the
provided InnerHitBuilder and configuring either nested_path or parent_child_type and the inner query of the query builder
being used.

Closes #11118
2016-04-29 11:17:24 +02:00
Tal Levy 6302fb65a3 add ability to disable ability to override values of existing fields in set processor 2016-04-28 13:50:19 -07:00
Isabel Drost-Fromm c1fa9cd18e Add note that coerce and ignore_malformed are deprecated for geo
distance sorting
2016-04-28 14:13:58 +02:00
Isabel Drost-Fromm a19c426e0f Deprecate coerce/ignore_malformed for GeoBoundingBoxQuery 2016-04-28 14:10:59 +02:00
Isabel Drost-Fromm 3f743a30cf Deprecate coerce/ignore_malformed in GeoDistanceQueryBuilder 2016-04-28 14:06:27 +02:00
Isabel Drost-Fromm 3160798084 Deprecate coerce/ignore_malformed for GeoDistanceRangeQuery 2016-04-28 14:01:54 +02:00
Isabel Drost-Fromm 5306de3ce3 Deprecate coerce/ignore_malformed for GeoPolygonQueryBuilder
Includes update to parsing code, tests, migration docs and reference
docs.
2016-04-28 13:56:50 +02:00
Clinton Gormley 84a2b4e17e Update id-field.asciidoc
Clarified which queries support the `_id` field
2016-04-28 13:36:14 +02:00
Shane Connelly 10b45852e9 Revert IPv6 here 2016-04-27 08:07:44 -07:00
Jim Ferenczi 573c4f3ed1 Extend field stats:
* Add isSearchable and isAggregatable (collapsed to true if any of the instances of that field are searchable or aggregatable).
* Accept wildcards in field names.
* Add a section named conflicts for fields with the same name but with incompatible types (instead of throwing an exception).
2016-04-27 16:51:53 +02:00
Alexander Kazakov a8a33a1a94 Row-centric output for _cat/fielddata 2016-04-27 13:29:02 +03:00
Christoph Büscher a2c3b5cae1 Update keyword.asciidoc 2016-04-27 12:10:19 +02:00
Shane Connelly aff148f532 Add new IPv6 types to docs where it's supported 2016-04-26 11:38:49 -07:00
Rafał Bigaj ead431f524 Duplicated colon was removed (#17988)
Hope this help :-)
2016-04-26 20:31:00 +02:00
Areek Zillur afacc18dcc Merge branch 'master' into docs/completion_suggester 2016-04-26 10:16:38 -04:00
Areek Zillur cc99b24bf7 Document completion suggest breaking changes 2016-04-26 10:15:21 -04:00
Jason Tedor efeec4d096 Merge pull request #17017 from jasontedor/generic-thread-pool
Actually bound the generic thread pool
2016-04-26 08:27:48 -04:00
Martijn Laarman 166cfcee8a Document missing shard version in routing table of cluster state (#17945)
as breaking change

removed as per: https://github.com/elastic/elasticsearch/pull/16243

because of: https://github.com/elastic/elasticsearch/issues/14739
2016-04-26 10:50:38 +02:00
Pius 66686040ca Update settings.asciidoc
Changed "must bet set" to "must be set" under Discovery Settings.
2016-04-26 00:15:39 -07:00
Pius f6656aa6ff Update settings.asciidoc
Added the 2 missing ` under Request Cache Settings section.
2016-04-25 23:57:42 -07:00
Pius 1364cc89f1 Update mapping.asciidoc
Changed "referrer to" to "refer to".
2016-04-25 23:35:42 -07:00
Sergii Golubev 2f6405ee27 serial-diff-aggregation.asciidoc: fix a mistake (#17950) 2016-04-25 07:45:54 -04:00
Clinton Gormley d56a8e5dd8 Update index-apis.asciidoc
Asciidoc typo
2016-04-25 13:06:57 +02:00
Jason Tedor 5608fa7ac1 Actually bound the generic thread pool
This commit actually bounds the size of the generic thread pool. The
generic thread pool was of type cached, a thread pool with an unbounded
number of workers and an unbounded work queue. With this commit, the
generic thread pool is now of type scaling. As such, the cached thread
pool type has been removed. By default, the generic thread pool is
constructed with a core pool size of four, a max pool size of 128 and
idle workers can be reaped after a keep-alive time of thirty seconds
expires. The work queue for this thread pool remains unbounded.
2016-04-25 06:47:26 -04:00
Lee Hinman 5fe1916be9 Merge pull request #17924 from elastic/russcam-patch-1
Update settings.asciidoc
2016-04-24 18:25:08 -06:00
Ryan Ernst 30eb72b9cb Merge pull request #17942 from rjernst/aggs_example_keyword
Docs: Update example aggs to use dynamic keyword field
2016-04-23 16:20:07 -07:00
Clinton Gormley b9978ace40 Update settings.asciidoc
Asciidoc typo
2016-04-23 13:44:42 +02:00
Russ Cam fb58ae3b4f Update settings.asciidoc
Add note for removal of index.translog.interval
2016-04-23 11:44:59 +10:00
Ryan Ernst 034adeb947 Docs: Update example aggs to use dynamic keyword field
The getting started docs use dynamic mappings. With the recent change to
string split into text and keyword, text lost the default ability to do
aggs. This was added back in #17188. This change updates the getting
started examples to use the keyword multi field added to dynamically
mapped text fields.

closes #17941
2016-04-22 16:34:15 -07:00
chenxiang a0aea5baf7 Update terms-query.asciidoc
user id of tweet hould exist in the `followers`, otherwise the search result is empty
2016-04-22 10:56:13 -06:00
ericamick 069eb72604 Update bucket.asciidoc 2016-04-22 10:54:25 -06:00
ericamick f081bf4e26 Update bulk.asciidoc 2016-04-22 10:51:33 -06:00
ericamick 3004c45f7b Update update.asciidoc 2016-04-22 10:50:42 -06:00
ericamick 276b89242c Update get.asciidoc 2016-04-22 10:48:58 -06:00
Nik Everett 61f0b665b8 Fix fallback setting for two get/2 2016-04-22 11:10:01 -04:00
Christoph Büscher a1c9025eaa Update completion-suggest.asciidoc
Removed trailing comma.
2016-04-22 14:00:37 +02:00
Martijn van Groningen c5ad2e2865 Changed indexed scripts to be stored in the cluster state instead of the `.scripts` index.
Also added max script size soft limit for stored scripts.

Closes #16651
2016-04-22 13:42:55 +02:00
Clinton Gormley e4df68b627 Added cautionary note to match_phrase_prefix explaining its shortcomings
Closes #17655
2016-04-22 12:45:12 +02:00
Christoph Büscher 0ec4ffcb3a Remove QueryFilterBuilder section from migration docs.
This query builder was deprecated in 2.0 and has been removed.
2016-04-21 18:11:01 +02:00
Martijn van Groningen dd2184ab25 ingest: Streamline option naming for several processors:
* `rename` processor, renamed `to` to `target_field`
* `date` processor, renamed `match_field` to `field` and renamed `match_formats` to `formats`
* `geoip` processor, renamed `source_field` to `field` and renamed `fields` to `properties`
* `attachment` processor, renamed `source_field` to `field` and renamed `fields` to `properties`

Closes #17835
2016-04-21 13:40:43 +02:00
Jun Ohtani 9eb242a5fe Analyze API : Rename filters/token_filters/char_filter to filter/token_filter/char_filter
Closes #15189
2016-04-21 18:05:11 +09:00
Zachary Tong 80288ad60c Add `fingerprint` token filter and `fingerprint` analyzer
Adds a `fingerprint` token filter which uses Lucene's FingerprintFilter,
and a `fingerprint` analyzer that combines the Fingerprint filter with
lowercasing, stop word removal and asciifolding.

Closes #13325
2016-04-20 16:10:56 -04:00
Martijn van Groningen 81449fc912 percolator: renamed `percolator` query to `percolate` query 2016-04-20 15:23:54 +02:00
Clinton Gormley ca8ea36b30 Updated decay-function image in function_score query
Closes #17479
2016-04-20 13:37:52 +02:00
Clinton Gormley b89e6cd5d8 Added link to breaking changes to release notes 2016-04-19 20:05:18 +02:00
Lee Hinman b8899cdb78 Merge remote-tracking branch 'dakrone/allow-bad-json' 2016-04-19 10:02:53 -06:00
Martijn van Groningen ba08313417 settings: Removed `action.get.realtime` setting
Closes #12543
2016-04-19 17:14:23 +02:00
Lee Hinman a1e8fb794c Allow JSON with unquoted field names by enabling system property
In Elasticsearch 5.0.0, by default unquoted field names in JSON will be
rejected. This can cause issues, however, for documents that were
already indexed with unquoted field names. To alleviate this, a system
property has been added that can be enabled so migration can occur.

This system property will be removed in Elasticsearch 6.0.0

Resolves #17674
2016-04-19 09:14:13 -06:00
Clinton Gormley 102a398d9f Fixed split processor example 2016-04-19 14:11:45 +02:00
Clinton Gormley 68f96868a6 Percolator docs missing a callout 2016-04-19 14:11:23 +02:00
Russ Cam e53131dd79 Update has-parent-query.asciidoc (#17841)
Change reference to `score_mode` to `score`
2016-04-19 11:56:05 +02:00
Clinton Gormley c024504842 Update search.asciidoc
Corrected breaking changes for `has_parent`.  Relates to https://github.com/elastic/elasticsearch/pull/17841
2016-04-19 11:54:48 +02:00
Martijn van Groningen 8e63ce00f0 docs: removed confusing statement. 2016-04-19 11:49:51 +02:00
Martijn van Groningen 40c22fc654 percolator: removed .percolator type instead a field of type `percolator` should be configured before indexing percolator queries
* Added an extra `field` parameter to the `percolator` query to indicate what percolator field should be used. This must be an existing field in the mapping of type `percolator`.
* The `.percolator` type is now forbidden. (just like any type that starts with a `.`)

This only applies for new indices created on 5.0 and later. Indices created on previous versions the .percolator type is still allowed to exist.
The new `percolator` field type isn't active in such indices and the `PercolatorQueryCache` knows how to load queries from these legacy indices.
The `PercolatorQueryBuilder` will not enforce that the `field` parameter is of type `percolator`.
2016-04-19 11:20:31 +02:00
Clinton Gormley a2ab13ddd1 Update ingest-node.asciidoc
Documented `separator` in the `split processor

Closes https://github.com/elastic/elasticsearch/issues/17831
2016-04-19 11:11:58 +02:00
Clinton Gormley 40b84d2ef6 Update mapping.asciidoc
Correct `fielddata.frequency.regex` to `fielddata.filter.regex` in breaking changes
2016-04-18 21:00:27 +02:00
Sergii Golubev 5ce3eb96b0 tophits-aggregation.asciidoc: fix a typo 2016-04-18 09:23:39 +02:00
David Pilato 5e1f26c22a Add support for documented byte/size units and for micros as a time unit in _cat API
We advertise in our documentation that byte units are like `kb`, `mb`... But we actually only support the simple notation `k` or `m`.
This commit adds support for the documented form and keeps the non documented options to avoid any breaking change.

It also adds support for `micros`, `nanos` and `d` as a time unit in `_cat` API.

Remove the support for `b` as a SizeValue unit. Actually, for numbers, when using raw numbers without unit, there is no text to add/parse after the number. For example, you don't write `10` as `10b`. We support option like `size=` in `_cat` API which means that we want to display raw data without unit (singles).

Documentation updated accordingly.

Add test for the empty size option.

Fix missing TimeValues options for some cat APIs
2016-04-15 20:55:41 +02:00
LeonardGC 0b8be7f894 Update field-mapping.asciidoc (#17670) 2016-04-15 09:22:38 +02:00
bloublou 83944c5628 Typo correction heap_size.asciidoc (#17745)
* Typo correction Xms Xmx

Typo correction on "-Xms4000mb -Xmx4000mb"

* Change mb to m for Xms/Xmx
2016-04-14 20:37:37 +02:00
Adrien Grand d84c643f58 Use the new points API to index numeric fields. #17746
This makes all numeric fields including `date`, `ip` and `token_count` use
points instead of the inverted index as a lookup structure. This is expected
to perform worse for exact queries, but faster for range queries. It also
requires less storage.

Notes about how the change works:
 - Numeric mappers have been split into a legacy version that is essentially
   the current mapper, and a new version that uses points, eg.
   LegacyDateFieldMapper and DateFieldMapper.
 - Since new and old fields have the same names, the decision about which one
   to use is made based on the index creation version.
 - If you try to force using a legacy field on a new index or a field that uses
   points on an old index, you will get an exception.
 - IP addresses now support IPv6 via Lucene's InetAddressPoint and store them
   in SORTED_SET doc values using the same encoding (fixed length of 16 bytes
   and sortable).
 - The internal MappedFieldType that is stored by the new mappers does not have
   any of the points-related properties set. Instead, it keeps setting the index
   options when parsing the `index` property of mappings and does
   `if (fieldType.indexOptions() != IndexOptions.NONE) { // add point field }`
   when parsing documents.

Known issues that won't fix:
 - You can't use numeric fields in significant terms aggregations anymore since
   this requires document frequencies, which points do not record.
 - Term queries on numeric fields will now return constant scores instead of
   giving better scores to the rare values.

Known issues that we could work around (in follow-up PRs, this one is too large
already):
 - Range queries on `ip` addresses only work if both the lower and upper bounds
   are inclusive (exclusive bounds are not exposed in Lucene). We could either
   decide to implement it, or drop range support entirely and tell users to
   query subnets using the CIDR notation instead.
 - Since IP addresses now use a different representation for doc values,
   aggregations will fail when running a terms aggregation on an ip field on a
   list of indices that contains both pre-5.0 and 5.0 indices.
 - The ip range aggregation does not work on the new ip field. We need to either
   implement range aggs for SORTED_SET doc values or drop support for ip ranges
   and tell users to use filters instead. #17700

Closes #16751
Closes #17007
Closes #11513
2016-04-14 17:56:23 +02:00
Colin Goodheart-Smithe c595322d90 Adds ignore_unmapped option to geo queries
The change adds a new option to the geo_* queries: ignore_unmapped. If this option is set to false, the toQuery method on the QueryBuilder will throw an exception if the field specified in the query is unmapped. If the option is set to true, the toQuery method on the QueryBuilder will return a MatchNoDocsQuery. The default value is false so the queries work how they do today (throwing an exception on unmapped field)
2016-04-14 15:29:07 +01:00
Colin Goodheart-Smithe 686aff1545 Adds ignore_unmapped option to nested and P/C queries
The change adds a new option to the `nested`, `has_parent`, `has_children` and `parent_id` queries: `ignore_unmapped`. If this option is set to false, the `toQuery` method on the QueryBuilder will throw an exception if the type/path specified in the query is unmapped. If the option is set to true, the `toQuery` method on the QueryBuilder will return a MatchNoDocsQuery. The default value is `false`so the queries work how they do today (throwing an exception on unmapped paths/types)
2016-04-14 10:34:30 +01:00
Clinton Gormley acec464eb8 Docs: Clarified the purpose of the parent_id query 2016-04-14 11:25:26 +02:00
Sergii Golubev 434a563fe0 terms-aggregation.asciidoc tiny edit 2016-04-13 16:51:47 -06:00
Martijn van Groningen 16fa3e546e docs: remove mention of file based grok pattern 2016-04-13 22:51:12 +02:00
Clinton Gormley 447f099544 Improve glossary to not refer to types as "like a table" (#17704)
Closes #17673
2016-04-13 14:29:47 +02:00
Nik Everett 0f9804b0e2 reindex: gracefully handle when _source is disabled
Closes #17666
2016-04-13 08:19:58 -04:00
Sergii Golubev 39b914bd77 histogram-aggregation.asciidoc: tiny edit (#17706) 2016-04-13 14:19:05 +02:00
Martijn van Groningen ca5bd89581 docs: adjust grok processor docs to not mention pattern files as these no longer exist
Closes #17692
2016-04-13 12:37:50 +02:00
Daniel Mitterdorfer 0c7795f53d Merge remote-tracking branch 'danielmitterdorfer/bulk-size-limit'
Closes #17133
2016-04-13 10:43:00 +02:00
Clinton Gormley a62b9296c6 Docs: Fixed link to phonetic plugin 2016-04-13 10:17:46 +02:00
Clinton Gormley bdf62b5615 More asciidoc errors 2016-04-13 10:14:09 +02:00
Clinton Gormley 1a15e55f94 More asciidoc errors 2016-04-13 10:02:09 +02:00
Clinton Gormley b201605a81 Fix bad asciidoc 2016-04-13 09:57:00 +02:00
Daniel Mitterdorfer 52b2016447 Limit request size on transport level
With this commit we limit the size of all in-flight requests on
transport level. The size is guarded by a circuit breaker and is
based on the content size of each request.

By default we use 100% of available heap meaning that the parent
circuit breaker will limit the maximum available size. This value
can be changed by adjusting the setting

network.breaker.inflight_requests.limit

Relates #16011
2016-04-13 09:54:59 +02:00
Clinton Gormley d26b7457cf Docs: Added note about older versions of RPM not being supported, and mentioned CentOS 5 2016-04-13 09:43:38 +02:00
Clinton Gormley 29b75960df Docs: Added note about RPMs not being supported on SLES 11 2016-04-13 09:34:01 +02:00
Jason Tedor a581d7cca4 Merge pull request #17675 from jasontedor/java-opts
Add JVM options configuration file
2016-04-12 23:07:40 -04:00
Jason Tedor 3879aa2a98 Add JVM options configuration file
This commit adds a new configuration file jvm.options to centralize and
simplify management of JVM options. This separates the configuration of
the JVM from the packaging scripts (bin/elasticsearch*, bin/service.bat,
and init.d/elasticsearch) simplifying end-user operational management of
custom JVM options.
2016-04-12 11:19:16 -04:00
Adrien Grand 0eb1a816c8 Allow the query cache to be disabled. #16268
This replaces the internal `index.queries.cache.type` setting with
a new `index.queries.cache.enabled` setting, which is documented.

Closes #15802
2016-04-11 18:06:16 +02:00
Adrien Grand 4adc31fe11 Use `mmapfs` by default.
I case any problem was discovered, you can still enable the legacy `default`
directory instead. But the plan is to get rid of it in 6.0.

Closes #16983
2016-04-08 20:23:27 +02:00
Greg Marzouka 37cb00a0b5 Merge pull request #17576 from gmarz/docs/windows-install
[DOCS] Update instructions for running on Windows
2016-04-07 18:01:42 -04:00
gmarz 6d9ed8ebf6 [DOCS] Update instructions for running on Windows
Closes #16455
2016-04-07 17:59:58 -04:00
Igor Motov 81c59cae18 Add _cat/tasks
Adds new _cat endpoint that lists all tasks
2016-04-07 09:28:21 -06:00
Adrien Grand 1d0239c125 Add a warning about the impact of sorting terms aggregations on the accuracy of doc counts. 2016-04-07 16:57:44 +02:00
Clinton Gormley 88c5dfeca4 Docs: Removed references to deprecated functionality
* search_type=count
* DFS in term vectors
* Replaced string with text/keyword as appropriate
2016-04-07 13:33:35 +02:00
Herbert Fischer c0ebce0ba0 Add some clarification regarding docs.count
* Add some clarification regarding docs.count

* Some improvements as suggested by @jasontedor
2016-04-07 11:15:32 +02:00
Ibrahim Awwal 5121060e75 Fix typo in templates.asciidoc
The doc mentions match_path in one place but the correct syntax is path_match which is mentioned everywhere else. Using the wrong string leads to errors because the mapping becomes too greedy, and matches things it shouldn't.
2016-04-06 16:40:20 -06:00
Lee Hinman 6e44ddaf0e Merge branch 'pr/11338' 2016-04-06 15:06:30 -06:00
Dmitrii Izgurskii 272f3eb140 Add missing comma
Added missing comma
2016-04-06 15:03:37 -06:00
Jimmy Jones f157dae053 Disallow unquoted field names, fix testcases using unquoted JSON 2016-04-06 14:37:15 -06:00
Colin Goodheart-Smithe c20c49963f [DOCS] update match query documentation
Now the `match` query has been split out into `match`, `match_phrase` and `match_phrase_prefix` we need to update the docs to remove the deprecated syntax
2016-04-06 15:44:22 +01:00
Mahmoud Lababidi 20b398daea Add a more descriptive example to Index Template
Closes #17513
The current example in the documentation for Index Templates lacks any properties values. This is helpful to many devs that aren't sure how to take a regular Index Mapping and convert it to a template.
2016-04-06 12:18:10 +02:00
Igor Motov f599ac5d5a Expose whether a task is cancellable in the _tasks list API
Closes #17369
2016-04-05 19:16:08 -06:00
Sergii Golubev 8430b379d8 string.asciidoc: fix for `position_increment_gap`
Remove  outdated and duplicate description for the `position_increment_gap` parameter.
2016-04-05 16:23:42 -04:00
Jeff ba34faa1ef Call out where we are making a setting change.
IMHO the original text here was incomplete. Adding the simple words 'in the index mapping' makes this sentence more clear. Perhaps a be more clear to make this a link.
2016-04-05 13:51:31 -06:00
Dimitrios Liappis b86d0983bb Merge pull request #17505 from dliappis/docs_fixes
Fix locahost typo and distro clarifications
2016-04-05 19:24:33 +03:00
Clinton Gormley b5b00c5d66 Updated 5.0.0-alpha1 release notes 2016-04-05 16:08:22 +02:00
Luca Cavanna 67ccfc354e Switch to using ParseField to parse query names
* [TEST] check registered queries one by one in SearchModuleTests

* Switch to using ParseField to parse query names

If we have a deprecated query name, at the moment we don't have a way to log any deprecation warning nor fail when we are in strict mode. With this change we use ParseField, which will take care of the camel casing that we currently do manually (so that one day we can remove it more easily). This also means, that each query will have a unique preferred name, and all the other names are deprecated.

Terms query "in" synonym is now formally deprecated, as well as fuzzy_match, match_fuzzy, match_phrase and match_phrase_prefix for match query, mlt for more_like_this and geo_bbox for geo_bounding_box. All these will be removed in 6.0.

Every QueryParser holds now a ParseField constant called QUERY_NAME_FIELD that holds the name for it. The first name is the preferred one, all the others are deprecated. The first name is taken from the NAME constant already present in each query builder object, so that we somehow keep the serialization constant separated from ParseField. This change also allowed us to remove the names method from the QueryParser interface.
2016-04-05 15:38:53 +02:00
Dimitrios Liappis e6d6eaa1bf Reverting 127.0.0.1 to localhost 2016-04-04 19:03:13 +03:00
Dimitrios Liappis be602ee026 clarify that dnf is mostly for Fedora >=22
RHEL7 and CentOS 7 are still on yum
2016-04-04 16:10:27 +03:00
Dimitrios Liappis 22e6567995 Replacing typo with distro safe address
apart from locahost typo, the issue is that localhost is not 100% safe
for all distros with IPv6.

For example fedora23 defines localhost4 and localhost6 (among other
aliases) so `curl localhost:9200` doesn't work.

For this reason, I think it's safer to replace localhost with 127.0.0.1
2016-04-04 16:01:03 +03:00
Clinton Gormley 06604708d4 Docs: Complete rewrite of setup, installation, and configuration docs 2016-04-03 16:09:48 +02:00
Joe Hillenbrand f355bb2a9b Merge pull request #17431 from joehillen/issue/17430
Set MAX_OPEN_FILES to 65536
2016-03-31 10:33:40 -07:00
Nik Everett 3caa5a929a Documentation for scroll size in reindex 2016-03-31 12:21:32 -04:00
javanna 0ea6dba04b [DOCS] Update _cat/nodes docs after recent changes 2016-03-31 16:36:29 +02:00
Daniel Mitterdorfer 6f0581c67c Update version and major-version variable in reference docs
Checked with @clintongormley
2016-03-31 16:17:46 +02:00
Daniel Mitterdorfer b7b65b93d1 Merge remote-tracking branch 'danielmitterdorfer/update-cluster-stats-docs' 2016-03-31 14:56:07 +02:00
Daniel Mitterdorfer 930ce1bfec Add up-to-date example of cluster stats API output 2016-03-31 14:41:37 +02:00
Martijn van Groningen 91fc9daeda Merge pull request #17443 from martijnvg/reindex_small_doc_fixed
Fix small reindex doc issues
2016-03-31 14:40:16 +02:00
Clinton Gormley 637da86e5f Docs: Updated the upgrade API docs to explain that the reindex API should be used instead 2016-03-31 14:34:31 +02:00
Martijn van Groningen 6bcfc730d7 reindex: fix small documentation issues 2016-03-31 12:59:12 +02:00
Clinton Gormley 18c5ea8599 Update templates.asciidoc
Relates to #17375
2016-03-31 11:15:57 +02:00
Adrien Grand 51d9574916 Fix reference guide about the default value of min_should_match. 2016-03-31 10:49:02 +02:00
Adrien Grand 26a0fb37a4 Add examples of useful dynamic templates to the docs. #17413 2016-03-31 09:45:11 +02:00
Joe Hillenbrand 81b6326891 Set MAX_OPEN_FILES to 65536
Relates to #17430
2016-03-30 16:44:37 -07:00
Igor Motov e073b0c75d Add ability to group tasks by common parent
By default, tasks are grouped by node. However, task execution in elasticsearch can be quite complex and an individual task that runs on a coordinating node can have many subtasks running on other nodes in the cluster. This commit makes it possible to list task grouped by common parents instead of by node. When this option is enabled all subtask are grouped under the coordinating node task that started all subtasks in the group. To group tasks by common parents, use the following syntax:

 GET /tasks?group_by=parents
2016-03-30 17:50:27 -04:00
Clinton Gormley 48c566da73 Documented how tribe node clients inherit their settings
Closes #17309
2016-03-30 20:05:44 +02:00
Adrien Grand d7179cafcc Add a soft limit on the number of shards that can be queried in a single search request. #17396
This commit adds the new `action.search.shard_count.limit` setting which
configures the maximum number of shards that can be queried in a single search
request. It has a default value of 1000.
2016-03-30 16:55:01 +02:00
Martijn van Groningen 7e2696c570 Refactored inner hits parsing and intoduced InnerHitBuilder
Both top level and inline inner hits are now covered by InnerHitBuilder.
Although there are differences between top level and inline inner hits,
they now make use of the same builder logic.

The parsing of top level inner hits slightly changed to be more readable.
Before the nested path or parent/child type had to be specified as encapsuting
json object, now these settings are simple fields. Before this was required
to allow streaming parsing of inner hits without missing contextual information.

Once some issues are fixed with inline inner hits (around multi level hierachy of inner hits),
top level inner hits will be deprecated and removed in the next major version.
2016-03-30 15:15:56 +02:00
Adrien Grand fc47007e17 Add a soft limit on the mapping depth. #17400
This commit adds the new `index.mapping.depth.limit` setting which controls the
maximum mapping depth that is allowed. It has a default value of 20.
2016-03-30 14:37:00 +02:00
Simon Willnauer 2514681f66 updateing filtering.asciidoc to also use 'node.attr' namespace 2016-03-30 14:11:59 +02:00
Simon Willnauer ad24653948 update allocation_awareness.asciidoc to also use 'node.attr' namespace 2016-03-30 13:52:45 +02:00
Simon Willnauer 8b075dbb75 Remove ability to specify arbitrary node attributes with `node.` prefix
Today the basic node settings like `node.data` and `node.master` can't really be fully validated
since we allow to specify custom user attributes on the node level. We have to, in order to
support that, add a wildcard setting for `node.*` to let these setting pass validation.
Instead we should require a more contraint prefix like `node.attr.` that defines a namespace
that is reserved for user attributes.
This commit adds a new namespace for attributes in `node.attr`.

Closes #17280
2016-03-30 13:29:48 +02:00
Isabel Drost-Fromm f27399dc0e Merge pull request #17282 from MaineC/deprecation/sort-option-reverse-removal
Remove deprecated reverse option from sorting
2016-03-30 11:02:19 +02:00
javanna 19eeb68bc4 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 21:53:22 +02:00
Spencer f07b35b328 Merge pull request #17368 from spalger/docs/clarifyMinMasterCheck
[docs] clarify where discovery.zen.minimum_master_node is required
2016-03-29 11:44:50 -07:00
javanna ae34c20a62 add node.client breaking changes to migrate guide 2016-03-29 20:33:59 +02:00
javanna 061f09d9a4 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 20:19:33 +02:00
Yanjun Huang 361adcf387 Add limit to total number of fields in mapping. #17357
This is to prevent mapping explosion when dynamic keys such as UUID are used as field names. index.mapping.total_fields.limit specifies the total number of fields an index can have. An exception will be thrown when the limit is reached. The default limit is 1000. Value 0 means no limit. This setting is runtime adjustable

Closes #11443
2016-03-29 19:39:46 +02:00
Igor Motov c356b30cff Update task management docs to reflect the latest changes in the interface
Brings docs in line with new list task syntax and adds task cancellation API docs.
2016-03-29 12:26:37 -04:00
Tal Levy 9ac3887139 Merge pull request #17263 from talevy/auto-convert
add  type conversion support to ConvertProcessor
2016-03-29 07:57:57 -07:00
Tal Levy 2064fe3985 add type conversion support to ConvertProcessor 2016-03-29 07:56:53 -07:00
Clinton Gormley 798e4281fa Added experimental annotation to the update-by-query and reindex docs 2016-03-29 15:06:27 +02:00
javanna 8fc9dbbb99 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 14:27:04 +02:00
Clinton Gormley 978b24327e Docs: Included Nodes Task API and tidied reindex/update-by-query 2016-03-29 13:51:11 +02:00
Isabel Drost-Fromm 407e2cdcf9 Merge branch 'master' into deprecation/sort-option-reverse-removal
Conflicts:
	core/src/main/java/org/elasticsearch/search/sort/ScoreSortBuilder.java
	core/src/test/java/org/elasticsearch/search/sort/FieldSortBuilderTests.java
2016-03-29 11:04:02 +02:00
javanna de5cbda8e7 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 10:48:47 +02:00
Lee Hinman 80ab366de4 Add API to explain why a shard is or isn't assigned
This adds a new `/_cluster/allocation/explain` API that explains why a
shard can or cannot be allocated to nodes in the cluster. Additionally,
it will show where the master *desires* to put the shard, according to
the `ShardsAllocator`.

It looks like this:

```
GET /_cluster/allocation/explain?pretty
{
  "index": "only-foo",
  "shard": 0,
  "primary": false
}
```

Though, you can optionally send an empty body, which means "explain the
allocation for the first unassigned shard you find".

The output when a shard is unassigned looks like this:

```
{
  "shard" : {
    "index" : "only-foo",
    "index_uuid" : "KnW0-zELRs6PK84l0r38ZA",
    "id" : 0,
    "primary" : false
  },
  "assigned" : false,
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",
    "at" : "2016-03-22T20:04:23.620Z"
  },
  "nodes" : {
    "V-Spi0AyRZ6ZvKbaI3691w" : {
      "node_name" : "Susan Storm",
      "node_attributes" : {
        "bar" : "baz"
      },
      "final_decision" : "NO",
      "weight" : 0.06666675,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    },
    "Qc6VL8c5RWaw1qXZ0Rg57g" : {
      "node_name" : "Slipstream",
      "node_attributes" : {
        "bar" : "baz",
        "foo" : "bar"
      },
      "final_decision" : "NO",
      "weight" : -1.3833332,
      "decisions" : [ {
        "decider" : "same_shard",
        "decision" : "NO",
        "explanation" : "the shard cannot be allocated on the same node id [Qc6VL8c5RWaw1qXZ0Rg57g] on which it already exists"
      } ]
    },
    "PzdyMZGXQdGhqTJHF_hGgA" : {
      "node_name" : "The Symbiote",
      "node_attributes" : { },
      "final_decision" : "NO",
      "weight" : 2.3166666,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    }
  }
}
```

And when the shard *is* assigned, the output looks like:

```
{
  "shard" : {
    "index" : "only-foo",
    "index_uuid" : "KnW0-zELRs6PK84l0r38ZA",
    "id" : 0,
    "primary" : true
  },
  "assigned" : true,
  "assigned_node_id" : "Qc6VL8c5RWaw1qXZ0Rg57g",
  "nodes" : {
    "V-Spi0AyRZ6ZvKbaI3691w" : {
      "node_name" : "Susan Storm",
      "node_attributes" : {
        "bar" : "baz"
      },
      "final_decision" : "NO",
      "weight" : 1.4499999,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    },
    "Qc6VL8c5RWaw1qXZ0Rg57g" : {
      "node_name" : "Slipstream",
      "node_attributes" : {
        "bar" : "baz",
        "foo" : "bar"
      },
      "final_decision" : "CURRENTLY_ASSIGNED",
      "weight" : 0.0,
      "decisions" : [ {
        "decider" : "same_shard",
        "decision" : "NO",
        "explanation" : "the shard cannot be allocated on the same node id [Qc6VL8c5RWaw1qXZ0Rg57g] on which it already exists"
      } ]
    },
    "PzdyMZGXQdGhqTJHF_hGgA" : {
      "node_name" : "The Symbiote",
      "node_attributes" : { },
      "final_decision" : "NO",
      "weight" : 3.6999998,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    }
  }
}
```

Only "NO" decisions are returned by default, but all decisions can be
shown by specifying the `?include_yes_decisions=true` parameter in the
request.

Resolves #14593
2016-03-28 15:21:02 -06:00
spalger ce44bbfadf [docs] clarify where discovery.zen.minimum_master_node is required
https://github.com/elastic/elasticsearch/pull/17288 added a check to enforce that the `discovery.zen.minimum_master_nodes` configuration is set when nodes have the `host`, `port`, or `bind_host` set in either `transport` or general `network` configuration sections. This was documented incorrectly as "nodes that are bound to a non-loopback interface", which lead to confusion as I set `network.host: "localhost"` and the check was still failing.

This change updates the docs to detail the actual check. I think it also highlights how complex the check is and the need for a simpler solution.
2016-03-28 12:53:40 -07:00
javanna a9f4982c40 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-25 20:16:40 +01:00
Clinton Gormley 3da7393b00 Tidied up percolator doc annotations 2016-03-25 15:36:51 +01:00
Boaz Leskes b8227a7222 Enforce `discovery.zen.minimum_master_nodes` is set when bound to a public ip #17288
discovery.zen.minimum_master_nodes is the single most important setting to set on a production cluster. We have no way of supplying a good default so it must be set by the user. Binding a node to a public IP (as opposed to the default local host) is a good enough indication that a node will be part of a production cluster cluster and thus it's a good tradeoff to enforce the settings. Note that nothing prevent users from setting it to 1 in a single node cluster.

Closes #17288
2016-03-25 12:56:20 +01:00
Clinton Gormley 08903f1ed8 Tidied the Painless docs and added the experimental tag 2016-03-24 18:34:40 +01:00
javanna 27d4994aff Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-24 18:10:11 +01:00
Boaz Leskes 6dd164d0bd Include pings from client nodes in master election
We currently have a `discovery.zen.master_election.filter_client` setting that control whether their ping responses are ignored for master election (which is the current default). With the push to treat client nodes as normal nodes (and promote the transport/rest clients for client work), this should be changed. This commit remove this setting and it's companion `discovery.zen.master_election.filter_data` setting (currently defaulting to  false) in favor of singe `discovery.zen.master_election.ignore_non_master_pings` setting with more intuitive name (defaulting to false).

Resolves #17325
Closes #17329
2016-03-24 17:48:05 +01:00
Clinton Gormley 0e7054cf02 Merge pull request #17272 from pengqiuyuan/master
Update template-query.asciidoc
2016-03-24 16:49:49 +01:00
javanna ce86fc5647 Cluster Stats: remove mem section
The available memory metric was always set to `0` since 2.0.beta1 (bug).  was left behind but never set. Turns out the section wasn't that useful, as it would only output the total memory available throughout all nodes in the cluster. We decided to remove the section then.
2016-03-24 15:49:27 +01:00
Isabel Drost-Fromm 08d989d9b6 Merge branch 'master' into deprecation/sort-option-reverse-removal
Conflicts:
	core/src/main/java/org/elasticsearch/search/sort/FieldSortBuilder.java
	core/src/main/java/org/elasticsearch/search/sort/ScoreSortBuilder.java
2016-03-24 12:06:10 +01:00
Isabel Drost-Fromm 801d178ade Remove mention of reverse in docs and add to migration doc 2016-03-24 12:04:31 +01:00
Jim Ferenczi da42f199bd Enforce isolated mode for all plugins
This commit removes the isolated option, each plugin have its own classloader.
2016-03-24 09:17:33 +01:00
Areek Zillur e16e113691 Remove suggest threadpool
In #17198, we removed suggest transport action, which
used the `suggest` threadpool to execute requests. Now
`suggest` threadpool is unused and suggest requests are
executed on the `search` threadpool.
2016-03-23 18:01:45 -04:00
debadair ad28fb9ec0 Docs: Adding Painless to the Scripting documentation. 2016-03-23 13:52:40 -07:00
Areek Zillur 442a6e0009 document suggest stats being merged with search stats 2016-03-23 16:37:57 -04:00
Areek Zillur e7e93f98e3 add migration guide to use search api for suggest 2016-03-23 16:37:57 -04:00
Nik Everett bbe0f3af3b Merge pull request #17255 from simonw/patch-2
Link to named queries docs from bool query page
2016-03-23 11:51:40 -04:00
javanna 030453d320 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-23 11:25:34 +01:00
javanna 4bfef1fde1 [DOCS] clarify that tribe node connects to every node in every cluster
Closes #16756
2016-03-23 10:43:58 +01:00
pengqiuyuan 80ef18c3b2 Update template-query.asciidoc 2016-03-23 17:14:31 +08:00
Simon Willison fdac0c7c6c Link to named queries docs from bool query page
The named queries feature only makes sense with bool queries, but was not cross-referenced from the bool query documentation page.
2016-03-22 12:07:57 -07:00
Nik Everett da96b6e41d [reindex] Add thottling support
The throttle is applied when starting the next scroll request so that its
timeout can include the throttle time.
2016-03-22 12:34:14 -04:00
Colin Goodheart-Smithe d6fe7515fd Merge pull request #17243 from colings86/docs/searchRequestBreakingChanges
added breaking changes for the Java API to the breaking changes doc for 5.0
2016-03-22 15:58:40 +00:00
Colin Goodheart-Smithe 25c4446942 iter 2016-03-22 15:58:12 +00:00
Colin Goodheart-Smithe ee7e84acc3 review comments 2016-03-22 15:34:47 +00:00
Adrien Grand c52b1f3a7c An `exists` query on an object should query a single term.
Currently if you run an `exists` query on an object, it will resolve all sub
fields and create a disjunction for all those fields. However the `_field_names`
mapper indexes paths for objects so we could query object paths directly.

I also changed the query parser to reject `exists` queries if the `_field_names`
field is disabled since it would be a big performance trap.
2016-03-22 16:26:45 +01:00
Adrien Grand b42f66c8ac Document 5.0 mapping changes. 2016-03-22 16:22:58 +01:00
Colin Goodheart-Smithe b8a96d9a65 added breaking changes for the Java API to the breaking changes doc for 5.0 2016-03-22 14:39:16 +00:00
Luca Cavanna 3764b3ff80 Merge pull request #17145 from alexshadow007/fix-17101
Fix column aliases in _cat/indices, _cat/nodes and _cat/shards APIs
2016-03-22 15:37:21 +01:00
javanna eebd0cfccd Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-22 10:34:40 +01:00
Simon Willnauer 7f16a1d9a7 Improve upgrade experience of node level index settings
In 5.0 we don't allow index settings to be specified on the node level ie.
in yaml files or via commandline argument. This can cause problems during
upgrade if this was used extensively. For instance if analyzers where
specified on a node level this might cause the index to be closed when
imported (see #17187). In such a case all indices relying on this
must be updated via `PUT /${index}/_settings`. Yet, this API has slightly
different semantics since it overrides existing settings. To make this less
painful this change adds a `preserve_existing` parameter on that API to ensure
we have the same semantics as if the setting was applied on the node level.

This change also adds a better error message and a change to the migration guide
to ensure upgrades are smooth if index settings are specified on the node level.

If a index setting is detected this change fails the node startup and prints a message
like this:
```
*************************************************************************************
Found index level settings on node level configuration.

Since elasticsearch 5.x index level settings can NOT be set on the nodes
configuration like the elasticsearch.yaml, in system properties or command line
arguments.In order to upgrade all indices the settings must be updated via the
/${index}/_settings API. Unless all settings are dynamic all indices must be closed
in order to apply the upgradeIndices created in the future should use index templates
to set default values.

Please ensure all required values are updated on all indices by executing:

curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
  "index.number_of_shards" : "1",
  "index.query.default_field" : "main_field",
  "index.translog.durability" : "async",
  "index.ttl.disable_purge" : "true"
}'
*************************************************************************************
```
2016-03-21 20:12:18 +01:00
javanna bf390a935e Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-21 17:18:23 +01:00
Martijn van Groningen e3b7e5d75a percolator: Replace percolate api with the new percolator query
Also replaced the PercolatorQueryRegistry with the new PercolatorQueryCache.

The PercolatorFieldMapper stores the rewritten form of each percolator query's xcontext
in a binary doc values field. This make sure that the query rewrite happens only during
indexing (some queries for example fetch shapes, terms in remote indices) and
the speed up the loading of the queries in the percolator query cache.

Because the percolator now works inside the search infrastructure a number of features
(sorting fields, pagination, fetch features) are available out of the box.

The following feature requests are automatically implemented via this refactoring:

Closes #10741
Closes #7297
Closes #13176
Closes #13978
Closes #11264
Closes #10741
Closes #4317
2016-03-21 12:21:50 +01:00
Clinton Gormley 830e2e049a Docs: Build release notes 2016-03-18 15:04:49 +01:00
Clinton Gormley e07b6a2641 Docs: Added 5.0.0-alpha1 release notes 2016-03-18 14:51:49 +01:00
Clinton Gormley 4506b7ad82 Docs: Fixed bad asciidoc link 2016-03-18 13:26:01 +01:00
Clinton Gormley dc21ab7576 Docs: Corrected behaviour of max_token_length in standard tokenizer 2016-03-18 10:58:16 +01:00
Alexander Reelsen 2dffad9ec3 Docs: Display reindex/update by query API and fix build doc issue
The documentation existed, but was not linked anywhere.
Also fixed the docs to make sure they build with this enabled.
2016-03-18 10:44:16 +01:00
Martijn van Groningen 3b17ddcd46 Removed old 1.x parent/child logic that should have been removed.
`0` really means, don't match any child docs.
2016-03-18 10:07:27 +01:00
Martijn van Groningen 1dd2be81c3 nested / parent child: Removed `total` score mode in favour of `sum` score mode.
Closes #17083
2016-03-18 10:07:26 +01:00
Clinton Gormley bd059b8cc3 Clarify how `-Djava.security.policy=someURL` must be passed
Closes #17160
2016-03-17 14:13:19 +01:00
Areek Zillur da165f425f update migration doc for removing gateway.format setting 2016-03-16 18:48:02 -04:00
Alexander Kazakov 51ac97000b Fix column aliases in _cat/indices, _cat/nodes and _cat/shards APIs #17101 2016-03-16 19:18:43 +03:00
Christoph Büscher 39667b5793 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	docs/reference/migration/migrate_5_0/java.asciidoc
2016-03-16 12:06:42 +01:00
Jason Tedor 618441aea3 Merge pull request #17088 from jasontedor/simplify-bootstrap-settings
Bootstrap does not set system properties
2016-03-15 19:25:16 -04:00
Clinton Gormley 432f0cc193 Docs: Added the ingest node to the modules/nodes page
Closes #17113
2016-03-15 19:03:18 +01:00
Jason Tedor 2f7e181318 Fix typo inadvertently introduced 2016-03-15 10:05:28 -04:00
Christoph Büscher bc84cdfed1 Using SortMode enum in all sort builders 2016-03-15 12:43:19 +01:00
Christoph Büscher b4b874f0d8 Merge branch 'master' into feature-suggest-refactoring 2016-03-15 12:11:39 +01:00
Areek Zillur 35f7cfb6c0 Add upgrader to upgrade old indices to new naming convention 2016-03-14 23:24:05 -04:00
Christoph Büscher 40f3501d7f Merge branch 'master' into feature-suggest-refactoring 2016-03-14 14:57:57 +01:00
Clinton Gormley 0ed0fea558 Updated link to Joda time zones 2016-03-14 12:24:58 +01:00
Christoph Büscher 97638c95fc Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	docs/reference/migration/migrate_5_0.asciidoc
2016-03-14 11:13:47 +01:00
Clinton Gormley c3cd8564df Corrected regexp syntax docs for COMPLEMENT 2016-03-14 10:46:31 +01:00
Clinton Gormley c90b4f3bae Docs: Added note about upgrading from 1.x to 5.x 2016-03-14 09:58:46 +01:00
Jason Tedor 8a05c2a2be Bootstrap does not set system properties
Today, certain bootstrap properties are set and read via system
properties. This action-at-distance way of managing these properties is
rather confusing, and completely unnecessary. But another problem exists
with setting these as system properties. Namely, these system properties
are interpreted as Elasticsearch settings, not all of which are
registered. This leads to Elasticsearch failing to startup if any of
these special properties are set. Instead, these properties should be
kept as local as possible, and passed around as method parameters where
needed. This eliminates the action-at-distance way of handling these
properties, and eliminates the need to register these non-setting
properties. This commit does exactly that.

Additionally, today we use the "-D" command line flag to set the
properties, but this is confusing because "-D" is a special flag to the
JVM for setting system properties. This creates confusion because some
"-D" properties should be passed via arguments to the JVM (so via
ES_JAVA_OPTS), and some should be passed as arguments to
Elasticsearch. This commit changes the "-D" flag for Elasticsearch
settings to "-E".
2016-03-13 20:09:15 -04:00
Jason Tedor 8ac5a98b87 Remove links to nonexistent migration docs 2016-03-13 19:12:06 -04:00
Clinton Gormley 5f48b9c86a Removed breaking changes docs for < 5.0 2016-03-13 21:18:44 +01:00
Clinton Gormley 5c845f8bb5 Reworked 5.0 breaking changes docs 2016-03-13 21:17:48 +01:00
Clinton Gormley a5a9bbfe88 Update compound-word-tokenfilter.asciidoc
Only FOP v1.2 compatible hyphenation files are supported by the hyphenation decompounder
2016-03-11 15:08:36 +01:00
Jason Tedor f465d98eb3 Add raw recovery progress to cat recovery API
This commit adds fields bytes_recovered and files_recovered to the cat
recovery API. These fields, respectively, indicate the total number of
bytes and files recovered. Additionally, for consistency, some totals
fields and translog recovery fields have been renamed.

Closes #17064
2016-03-11 08:27:09 -05:00
Christoph Büscher daeffb149c Merge branch 'master' into feature-suggest-refactoring 2016-03-11 10:37:28 +01:00
Daniel Mitterdorfer 94aa025b93 Document breaking change in ClusterHealthResponse in 2.2 2016-03-11 09:47:53 +01:00
Lee Hinman b3bd189cd7 [DOCS] Fix missing table end in function-score-query.asciidoc 2016-03-10 13:35:33 -07:00
Lee Hinman 982a369efc [DOCS] Document the modifiers for `field_value_factor`
Resolves #13511
2016-03-10 11:34:34 -07:00
Yannick Welsch 295d33c2a6 Merge pull request #17021 from ywelsch/fix/block-delete-on-snapshot
Fail closing or deleting indices during a full snapshot
2016-03-10 18:51:04 +01:00
Yannick Welsch 266394c3ab Fail closing or deleting indices during a full snapshot
Closes #16321
2016-03-10 18:11:47 +01:00
Lee Hinman 22e716551b Add -XX+AlwaysPreTouch JVM flag
Enables the touching of all memory pages used by the JVM heap spaces
during initialization of the HotSpot VM, which commits all memory pages
at initialization time. By default, pages are committed only as they are
needed.
2016-03-10 10:11:32 -07:00
Clinton Gormley 2fa573bc58 Missing word in docs 2016-03-10 14:34:05 +01:00
Robin Clarke 046212035c Clarification about precedence of settings
Closes #14559
2016-03-10 14:29:51 +01:00
Nicholas Knize f7a2dbfcaf fixing silly typo in docs 2016-03-10 07:28:13 -06:00
Clinton Gormley 6b4a6be2df Merge pull request #14800 from MaineC/doc-fix/query-dsl
Fix minor typos in query dsl docs
2016-03-10 13:45:13 +01:00
Martijn van Groningen 2fa33d5c47 Added ingest statistics to node stats API
The ingest stats include the following statistics:
* `ingest.total.count`- The total number of document ingested during the lifetime of this node
* `ingest.total.time_in_millis` - The total time spent on ingest preprocessing documents during the lifetime of this node
* `ingest.total.current` - The total number of documents currently being ingested.
* `ingest.total.failed` - The total number ingest preprocessing operations failed during the lifetime of this node

Also these stats are returned on a per pipeline basis.
2016-03-10 13:21:43 +01:00
Clinton Gormley a8c7ae7809 Fixed bad docs link 2016-03-10 13:08:29 +01:00
Clinton Gormley f9622f9acc Docs: Added a note about the update API not supporting external versioning
Closes #12820
2016-03-10 12:33:39 +01:00
Christoph Büscher 69c83b3459 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/action/suggest/TransportSuggestAction.java
        core/src/main/java/org/elasticsearch/search/suggest/SuggestParseElement.java
	core/src/main/java/org/elasticsearch/search/suggest/SuggestionSearchContext.java
	core/src/main/java/org/elasticsearch/search/suggest/phrase/PhraseSuggester.java
2016-03-10 11:44:36 +01:00
Nicholas Knize 55635d5de1 update coerce and breaking changes documentation 2016-03-09 16:09:44 -06:00
Nicholas Knize 61f39e6c92 GeoPointV2 update docs and query builders
This commit updates the documentation for GeoPointField by removing all references to the coerce and doc_values parameters. DocValues are enabled in lucene GeoPointField by default (required for boundary filtering). The QueryBuilders are updated to automatically normalize points (ignoring the coerce parameter) for any index created onOrAfter version 2.2.
2016-03-09 16:09:44 -06:00
Clinton Gormley 06929f8ed4 Merge pull request #17030 from 36degrees/patch-1
Fix typo in clear cache documentation
2016-03-09 15:34:15 +01:00
Jason Tedor c0572c631d Note to configuration docs on number of threads
This commit adds a note to the configuration docs regarding the number
of threads necessary for the Elasticsearch user.

Relates #17003
2016-03-08 09:14:55 -05:00
Christoph Büscher 1264f37a1b Merge branch 'master' into feature-suggest-refactoring 2016-03-08 11:10:08 +01:00