Commit Graph

4576 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe 5e7a79636d
[DOCS] Clarify behaviour of scripted-metric arg with empty parent buckets 2017-06-02 11:00:27 +01:00
Luca Cavanna 018c6c38fe [DOCS] Clarify connections and gateway nodes selection in cross cluster search docs (#24859)
Closes #24836
2017-06-02 11:13:47 +02:00
olcbean 6dea5f14c3 Java api: Remove unneeded getTookInMillis method (#23923)
Some response classes in the java api expose both `getTook()` which returns a `TimeValue` and `getTookInMillis` which returns a `long` value. `getTook()` is enough as one can do `getTook().millis()` to obtain the same result as `getTookInMillis()`, which can be removed.
2017-06-02 11:11:05 +02:00
Colin Goodheart-Smithe 779fb9a1c0 Adds nodes usage API to monitor usages of actions (#24169)
* Adds nodes usage API to monitor usages of actions

The nodes usage API has 2 main endpoints

/_nodes/usage and /_nodes/{nodeIds}/usage return the usage statistics
for all nodes and the specified node(s) respectively.

At the moment only one type of usage statistics is available, the REST
actions usage. This records the number of times each REST action class is
called and when the nodes usage api is called will return a map of rest
action class name to long representing the number of times each of the action
classes has been called.

Still to do:

* [x] Create usage service to store usage statistics
* [x] Record usage in REST layer
* [x] Add Transport Actions
* [x] Add REST Actions
* [x] Tests
* [x] Documentation

* Rafactors UsageService so counts are done by the handlers

* Fixing up docs tests

* Adds a name to all rest actions

* Addresses review comments
2017-06-02 08:46:38 +01:00
Tanguy Leroux 528bd25fa7 Add superset size to Significant Term REST response (#24865)
This commit adds a new bg_count field to the REST response of
SignificantTerms aggregations. Similarly to the bg_count that already
exists in significant terms buckets, this new bg_count field is set at
the aggregation level and is populated with the superset size value.
2017-06-02 09:45:15 +02:00
Adrien Grand bbdf50f6bd Docs: More search speed advices. (#24802) 2017-06-01 17:23:22 +02:00
Adrien Grand ebf806d38f Reorganize docs of global ordinals. (#24982)
Currently global ordinals are documented under `fielddata`. It moves them to
their own file since they also work with doc values and fielddata is on the way
out.

Closes #23101
2017-06-01 16:47:44 +02:00
Jason Tedor 9b4a189147 Add purge option to remove plugin CLI
By default, the remove plugin CLI command preserves configuration
files. This is so that if a user is upgrading the plugin (which is done
by first removing the old version and then installing the new version)
they do not lose their configuration file. Yet, there are circumstances
where preserving the configuration file is not desired. This commit adds
a purge option to the remove plugin CLI command.

Relates #24981
2017-06-01 08:53:39 -04:00
Clinton Gormley 1b0c93b07c Documented the level parameter to nodes stats
Closes #24999
2017-06-01 12:11:21 +02:00
Sergey Novikov a7b21534b1 Docs: Fix typo in docker docs (#24988)
`boostrap.memory_lock` -> `bootstrap.memory_lock`
2017-05-31 13:42:47 -04:00
David Cho-Lerat 491dc1186a Add missing word to terms-query.asciidoc (#24960) 2017-05-30 09:42:07 -04:00
David Cho-Lerat c939bcb7f5 Correct some spelling in match-phrase-prefix docs (#24956) 2017-05-30 09:02:01 -04:00
Tanguy Leroux 28d97df67c Add document count to Matrix Stats aggregation response (#24776)
This commit adds a `doc_count` field to the response body of Matrix
Stats aggregation. It exposes the number of documents involved in
 the computation of statistics, a value that can already be retrieved using
  the method MatrixStats.getDocCount() in the Java API.
2017-05-30 09:39:41 +02:00
propulkit 25516868fe TCorrecting api name (#24924)
As per REST request signature for reroute, API has no underscore.
2017-05-29 13:58:31 +02:00
Jason Tedor bb63577f14 Fix plugin docs for using custom config dir
This commit fixes an issue with the plugin docs incorrectly specifying
how to set a custom configuration directory. The correct way is to use
the environment variable CONF_DIR.
2017-05-26 13:40:35 -04:00
Clinton Gormley 0656d0236b Update context-suggest.asciidoc
Removed incorrect parameter
2017-05-26 17:41:40 +02:00
Matt Weber 601a61a91c Support Multiple Collapse Inner Hits
Support multiple named inner hits on a field collapsing
request.
2017-05-26 13:23:57 +02:00
Tal Levy dfe2ecaa28 add docs example for Ingest scripts manipulating document metadata (#24875)
It may not be clear to users that the Ingest ScriptProcessor context object `ctx` can 
manipulate document metadata like `_index` and `_type`.
2017-05-25 07:45:19 -07:00
Brian Lesperance 959990728b Docs: Fix grammar in aliases doc (#24852) 2017-05-24 10:18:25 -04:00
markharwood b7197f5e21 SignificantText aggregation - like significant_terms, but for text (#24432)
* SignificantText aggregation - like significant_terms but doesn’t require fielddata=true, recommended used with `sampler` agg to limit expense of tokenizing docs and takes optional `filter_duplicate_text`:true setting to avoid stats skew from repeated sections of text in search results.

Closes #23674
2017-05-24 13:46:43 +01:00
António Ribeiro 85a1b2b406 Fix link to perl docs (#24842)
* Fixes Elasticsearch issue #24606.

* Fixes Elasticsearch issue #24606.

* Fixes Elasticsearch issue #24606.

* Fixes Elasticsearch issue #24606.

* Issue #24606 - Changed the link text to Search::Elasticsearch::Client::5_0::Bulk and
Search::Elasticsearch::Client::5_0::Scroll.
2017-05-24 11:43:54 +02:00
Nik Everett 13a86fec99 Add magic $_path stash key to docs tests (#24724)
Adds a "magic" key to the yaml testing stash mostly for use with
documentation tests. When unstashing an object, `$_path` is the
path into the current position in the object you are unstashing.
This means that in docs tests you can use
`// TESTRESPONSEs/somevalue/$body.${_path}/` to mean "replace
`somevalue` with whatever is the response in the same position."

Compare how you must carefully mock out all the numbers in the profile
response without this change:
```
// TESTRESPONSE[s/"id": "\[2aE02wS1R8q_QFnYu6vDVQ\]\[twitter\]\[1\]"/"id": $body.profile.shards.0.id/]
// TESTRESPONSE[s/"rewrite_time": 51443/"rewrite_time": $body.profile.shards.0.searches.0.rewrite_time/]
// TESTRESPONSE[s/"score": 51306/"score": $body.profile.shards.0.searches.0.query.0.breakdown.score/]
// TESTRESPONSE[s/"time_in_nanos": "1873811"/"time_in_nanos": $body.profile.shards.0.searches.0.query.0.time_in_nanos/]
// TESTRESPONSE[s/"build_scorer": 2935582/"build_scorer": $body.profile.shards.0.searches.0.query.0.breakdown.build_scorer/]
// TESTRESPONSE[s/"create_weight": 919297/"create_weight": $body.profile.shards.0.searches.0.query.0.breakdown.create_weight/]
// TESTRESPONSE[s/"next_doc": 53876/"next_doc": $body.profile.shards.0.searches.0.query.0.breakdown.next_doc/]
// TESTRESPONSE[s/"time_in_nanos": "391943"/"time_in_nanos": $body.profile.shards.0.searches.0.query.0.children.0.time_in_nanos/]
// TESTRESPONSE[s/"score": 28776/"score": $body.profile.shards.0.searches.0.query.0.children.0.breakdown.score/]
// TESTRESPONSE[s/"build_scorer": 784451/"build_scorer": $body.profile.shards.0.searches.0.query.0.children.0.breakdown.build_scorer/]
// TESTRESPONSE[s/"create_weight": 1669564/"create_weight": $body.profile.shards.0.searches.0.query.0.children.0.breakdown.create_weight/]
// TESTRESPONSE[s/"next_doc": 10111/"next_doc": $body.profile.shards.0.searches.0.query.0.children.0.breakdown.next_doc/]
// TESTRESPONSE[s/"time_in_nanos": "210682"/"time_in_nanos": $body.profile.shards.0.searches.0.query.0.children.1.time_in_nanos/]
// TESTRESPONSE[s/"score": 4552/"score": $body.profile.shards.0.searches.0.query.0.children.1.breakdown.score/]
// TESTRESPONSE[s/"build_scorer": 42602/"build_scorer": $body.profile.shards.0.searches.0.query.0.children.1.breakdown.build_scorer/]
// TESTRESPONSE[s/"create_weight": 89323/"create_weight": $body.profile.shards.0.searches.0.query.0.children.1.breakdown.create_weight/]
// TESTRESPONSE[s/"next_doc": 2852/"next_doc": $body.profile.shards.0.searches.0.query.0.children.1.breakdown.next_doc/]
// TESTRESPONSE[s/"time_in_nanos": "304311"/"time_in_nanos": $body.profile.shards.0.searches.0.collector.0.time_in_nanos/]
// TESTRESPONSE[s/"time_in_nanos": "32273"/"time_in_nanos": $body.profile.shards.0.searches.0.collector.0.children.0.time_in_nanos/]
```

To how you can cavalierly mock all the numbers at once with this change:
```
// TESTRESPONSE[s/(?<=[" ])\d+(\.\d+)?/$body.$_path/]
```
2017-05-23 15:33:48 -04:00
Clinton Gormley 086abe6216 Marked `fixed_auto_queue_size` as experimental
Relates to https://github.com/elastic/elasticsearch/pull/23884
2017-05-22 10:03:31 +02:00
olcbean e08e92d934 Deleting a document from a non-existing index creates the should not auto create it, unless using EXTERNAL* versioning (#24518)
Currently a `delete document` request against a non-existing index actually **creates** this index.

With this change the `delete document` no longer creates the previously non-existing index and throws an `index_not_found` exception instead.

However as discussed in https://github.com/elastic/elasticsearch/pull/15451#issuecomment-165772026, if an external version is explicitly used, the current behavior is preserved and the index is still created and the document is marked for deletion.

Fixes #15425
2017-05-22 10:00:22 +03:00
archana a5358f34b3 Update mappings.asciidoc
typo
2017-05-20 13:39:05 -05:00
Kristen b18df27d74 Small typo in Painless Dispatch page (#24792) 2017-05-19 10:56:41 -04:00
Oleksandr Chychkan 065d91bccc Typo in setup/configuration.asciidoc (#24797) 2017-05-19 10:49:56 -04:00
Jack Conradson 0aa380b770 Fix search template documentation reference to scripting security. 2017-05-18 14:27:58 -07:00
Jack Conradson 1196dfb6bb Remove Deprecated Script Settings (#24756)
Removes all fine-grained script settings replaced by scripts.types_allowed and scripts.contexts_allowed.
2017-05-18 13:32:46 -07:00
Ryan Ernst b214b80e6c GCS Repository: Remove specifying credential file on disk (#24727)
This commit removes the ability to specify the google credential json
file on disk, which is deprecated in 5.5.0.
2017-05-18 10:22:29 -07:00
Ryan Ernst 26e2e933f5 Scripting: Remove native scripts (#24726)
Native scripts have been replaced in documentation by implementing
a ScriptEngine and they were deprecated in 5.5.0. This commit
removes the native script infrastructure for 6.0.

closes #19966
2017-05-17 14:49:24 -07:00
Ryan Ernst 463fe2f4d4 Scripting: Remove file scripts (#24627)
This commit removes file scripts, which were deprecated in 5.5.

closes #21798
2017-05-17 14:42:25 -07:00
debadair bb3a59fa70 [DOCS] Fixed cross doc xref in plugin docs. 2017-05-16 17:55:47 -07:00
Ryan Ernst d74760c306 GCS Repository: Add secure storage of credentials (#24697)
This commit adds gcs credential settings to the elasticsearch keystore.
The setting name follows the same pattern as the s3 client settings,
beginning with `gcs.client.`, followed by the client name, and then the
setting name, in this case, `credentials_file`. Using the legacy service
file setting is also deprecated.
2017-05-16 17:17:37 -07:00
debadair f80799acc2 [DOCS] Removed API xrefs from Painless GSG 2017-05-16 15:05:40 -07:00
Zachary Tong a2845c86fe
CONSOLEify some more aggregation docs
Related #18160
2017-05-16 17:25:24 -04:00
debadair 5ac2ddd2be [DOCS] Setting up separate Painless book. 2017-05-16 12:46:56 -07:00
Jack Conradson b7f0df626a [DOCS] Added Painless Language Spec content 2017-05-16 12:46:56 -07:00
Lee Hinman d09e64323f Add ability to automatically adjust search threadpool queue_size
This PR adds a new thread pool type: `fixed_auto_queue_size`. This thread pool
behaves like a regular `fixed` threadpool, except that every
`auto_queue_frame_size` operations (default: 10,000) in the thread pool,
[Little's Law](https://en.wikipedia.org/wiki/Little's_law) is calculated and
used to adjust the pool's `queue_size` either up or down by 50. A minimum and
maximum is taken into account also. When the min and max are the same value, a
regular fixed executor is used instead.

The `SEARCH` threadpool is changed to use this new type of thread pool. However,
the min and max are both set to 1000, meaning auto adjustment is opt-in rather
than opt-out.

Resolves #3890
2017-05-16 11:13:16 -06:00
Ryan Ernst 97d2657e18 Remove script access to term statistics (#19462)
In scripts (at least some of the languages), the terms dictionary and
postings can be access with the special _index variable. This is for
very advanced use cases which want to do their own scoring. The problem
is segment level statistics must be recomputed for every document.
Additionally, this is not friendly to the terms index caching as the
order of looking up terms should be controlled by lucene.

This change removes _index from scripts. Anyone using it can and should
instead write a Similarity plugin, which is explicitly designed to allow
doing the calculations needed for a relevance score.

closes #19359
2017-05-16 09:10:09 -07:00
Simon Willnauer 1cae850cf5 Add a cluster block that allows to delete indices that are read-only (#24678)
Today when an index is `read-only` the index is also blocked from
being deleted which sometimes is undesired since in-order to make
changes to a cluster indices must be deleted to free up space. This is
a likely scenario in a hosted environment when disk-space is limited to switch
indices read-only but allow deletions to free up space.
2017-05-16 17:34:37 +02:00
Luis Majano 788d8c1ddc Docs: Link to new native ColdFusion (CFML) client (#24690) 2017-05-16 10:50:26 -04:00
Daniel Mitterdorfer 77762fcbb0 Use correct script name in docs for Windows
With this commit we correct the name of the ES batch script to
`elasticsearch.bat` in the docs and use backslashes in path names.
2017-05-16 15:57:05 +02:00
Ryan Ernst 548a5c1386 Docs: Add migration note about .yaml and .json removal (#24689)
relates #19391
relates #24633
2017-05-15 13:42:28 -07:00
Jack Conradson 43292979fd Add New Security Script Settings (#24637)
Settings are simplified to allowed_types and allowed_contexts.  If a setting is not specified the default is to enable all for that setting.
2017-05-15 13:37:46 -07:00
Vlad Holubiev 557390d7d1 Fix typo in example (grades_count -> types_count) (#24635)
Looks like `doc.grade` was used for examples before. But not anymore - https://www.elastic.co/guide/en/elasticsearch/reference/2.4/search-aggregations-metrics-valuecount-aggregation.html
2017-05-15 14:08:46 -04:00
Jason Tedor 4e21a33689 Clarify disabling swap in docs
Our strong recommendation is disabling swap over any other alternative
to avoid the JVM from landing on disk. This commit clarifies the docs in
this regard.
2017-05-12 16:09:52 -04:00
Jim Ferenczi 279a18a527 Add parent-join module (#24638)
* Add parent-join module

This change adds a new module named `parent-join`.
The goal of this module is to provide a replacement for the `_parent` field but as a first step this change only moves the `has_child`, `has_parent` queries and the `children` aggregation to this module.
These queries and aggregations are no longer in core but they are deployed by default as a module.

Relates #20257
2017-05-12 15:58:06 +02:00
Ryan Ernst c1f1f66509 Scripting: Replace advanced and native scripts with ScriptEngine docs (#24603)
This commit documents how to write a `ScriptEngine` in order to use
expert internal apis, such as using Lucene directly to find index term
statistics. These documents prepare the way to remove both native
scripts and IndexLookup.

The example java code is actually compiled and tested under a new gradle
subproject for example plugins. This change does not yet breakup
jvm-example into the new examples dir, which should be done separately.

relates #19359
relates #19966
2017-05-11 12:15:16 -07:00
qwerty4030 e7d352b489 Compound order for histogram aggregations. (#22343)
This commit adds support for histogram and date_histogram agg compound order by refactoring and reusing terms agg order code. The major change is that the Terms.Order and Histogram.Order classes have been replaced/refactored into a new class BucketOrder. This is a breaking change for the Java Transport API. For backward compatibility with previous ES versions the (date)histogram compound order will use the first order. Also the _term and _time aggregation order keys have been deprecated; replaced by _key.

Relates to #20003: now that all these aggregations use the same order code, it should be easier to move validation to parse time (as a follow up PR).

Relates to #14771: histogram and date_histogram aggregation order will now be validated at reduce time.

Closes #23613: if a single BucketOrder that is not a tie-breaker is added with the Java Transport API, it will be converted into a CompoundOrder with a tie-breaker.
2017-05-11 18:06:26 +01:00
Martijn van Groningen 840da4aebf
Removed deprecated template query.
Relates to #19390
2017-05-11 14:56:45 +02:00
propulkit 77feabb3d5 Machine Permission update on Google Compute
Machine permission can be updated without deleting instances.

Backport of #24607 in master branch
2017-05-11 07:52:34 +02:00
Ryan Ernst 0789a74055 S3 Repository: Remove deprecated settings (#24445)
These settings are deprecated in 5.5. This change removes them for 6.0.
2017-05-10 20:12:17 -07:00
Chris Ivens d447b79e16 Docs: Tiny typo to Painless dispatch justification (#24588) 2017-05-10 22:05:19 -04:00
Ali Beyad 743217a430 Enhances get snapshots API to allow retrieving repository index only (#24477)
Currently, the get snapshots API (e.g. /_snapshot/{repositoryName}/_all)
provides information about snapshots in the repository, including the
snapshot state, number of shards snapshotted, failures, etc.  In order
to provide information about each snapshot in the repository, the call
must read the snapshot metadata blob (`snap-{snapshot_uuid}.dat`) for
every snapshot.  In cloud-based repositories, this can be expensive,
both from a cost and performance perspective.  Sometimes, all the user
wants is to retrieve all the names/uuids of each snapshot, and the
indices that went into each snapshot, without any of the other status
information about the snapshot.  This minimal information can be
retrieved from the repository index blob (`index-N`) without needing to
read each snapshot metadata blob.

This commit enhances the get snapshots API with an optional `verbose`
parameter.  If `verbose` is set to false on the request, then the get
snapshots API will only retrieve the minimal information about each
snapshot (the name, uuid, and indices in the snapshot), and only read
this information from the repository index blob, thereby giving users
the option to retrieve the snapshots in a repository in a more
cost-effective and efficient manner.

Closes #24288
2017-05-10 15:48:40 -04:00
Clinton Gormley 2486086980 Deprecate the Tribe node
The Tribe node is deprecated in favour of Cross Cluster Search and will be removed in 7.0.
2017-05-10 14:05:12 +02:00
Matt Weber b24326271e Add ICUCollationFieldMapper (#24126)
Adds a new "icu_collation" field type that exposes lucene's
ICUCollationDocValuesField.  ICUCollationDocValuesField is the replacement
for ICUCollationKeyFilter which has been deprecated since Lucene 5.
2017-05-10 10:35:11 +02:00
Suhas Karanth 09c5fbfd00 Docs: Correct description of example (#24541)
Copy and paste error.
2017-05-09 15:18:43 -04:00
Clinton Gormley b8dede06b5 Fixed prerelease URLs for RPM and Deb repos in docs 2017-05-09 18:33:23 +02:00
Adrien Grand a72eaa8e0f Identify documents by their `_id`. (#24460)
Now that indices have a single type by default, we can move to the next step
and identify documents using their `_id` rather than the `_uid`.

One notable change in this commit is that I made deletions implicitly create
types. This helps with the live version map in the case that documents are
deleted before the first type is introduced. Otherwise there would be no way
to differenciate `DELETE index/foo/1` followed by `PUT index/foo/1` from
`DELETE index/bar/1` followed by `PUT index/foo/1`, even though those are
different if versioning is involved.
2017-05-09 16:33:52 +02:00
Clinton Gormley 41b10554fd Mark 6.0.0-alpha1 as prerelease 2017-05-09 14:02:11 +02:00
Clinton Gormley d77757fc1b Updated release notes for 6.0.0-alpha1 2017-05-09 14:01:32 +02:00
glefloch 59dd4d288a documentation of preserve existing settings 2017-05-08 12:05:01 +02:00
Anupam 0b36fb052c Update completion-suggest.asciidoc (#24506) 2017-05-05 11:34:41 -04:00
Nicholas Knize 0c4eb0a029 Add new ip_range field type
This commit adds support for indexing and searching a new ip_range field type. Both IPv4 and IPv6 formats are supported. Tests are updated and docs are added.
2017-05-05 09:43:42 -05:00
Clinton Gormley 9f08a553d9 Fixed docs syntax for for-in loop in painless 2017-05-05 16:07:20 +02:00
Clinton Gormley 0174119296 Added release notes for 6.0.0-alpha1 2017-05-05 12:39:50 +02:00
Clinton Gormley c9aecbb8a5 Added removal of JavaScript and Python to breaking changes 2017-05-05 12:39:50 +02:00
Clinton Gormley 6e970db533 Fixed chunking of breaking changes docs 2017-05-05 11:08:55 +02:00
Simon Willnauer 6b67e0bf2f Include all aliases including non-filtering in `_search_shards` response (#24489)
`_search_shards`API today only returns aliases names if there is an alias
filter associated with one of them. Now it can be useful to see which aliases
have been expanded for an index given the index expressions. This change also includes non-filtering aliases even without a filtering alias being present.
2017-05-05 09:34:12 +02:00
Nik Everett a01f846226 CONSOLEify a few more docs
Adds CONSOLE to cross-cluster-search docs but skips them for testing
because we don't have a second cluster set up. This gets us the
`VIEW IN CONSOLE` and `COPY AS CURL` links and makes sure that they
are valid yaml (not json, technically) but doesn't get testing.
Which is better than we had before.

Adds CONSOLE to the dynamic templates docs and ingest-node docs.
The ingest-node docs contain a *ton* of non-console snippets. We
might want to convert them to full examples later, but that can be
a separate thing.

Relates to #18160
2017-05-04 21:01:14 -04:00
Nik Everett 559bec23cc Docs: rewrite the docs/README file
I originally wrote this file when we first added snippets testing
and a lot has changed. We've grown quite fond of the
`// TESTRESPONSE[s/foo/bar/]` construct, for example, but the docs
discouraged its use.

Relates to #18160
2017-05-04 17:57:25 -04:00
Nik Everett 9f431543fc CONSOLEify inner hits docs
Rewrites most of the snippets in the `innert_hits` docs to be
complete examples and enables `VIEW IN CONSOLE`, `COPY AS CURL`,
and automatic testing of the snippets.
2017-05-04 17:30:54 -04:00
Jason Tedor de65f51d34 Simplify file store
Today we go to heroic lengths to workaround bugs in the JDK or around
issues like BSD jails to get information about the underlying file
store. For example, we went to lengths to work around a JDK bug where
the file store returned would incorrectly report whether or not a path
is writable in certain situations in Windows operating
systems. Another bug prevented getting file store information on
Windows on a virtual drive on Windows. We no longer need to work
around these bugs, we could simply try to write to disk and let an I/O
exception arise if we could not write to the disk or take advantage of
the fact that these bugs are fixed in recent releases of the JDK
(e.g., the file store bug is fixed since 8u72). Additionally, we
collected information about all file stores on the system which meant
that if the user had a stale NFS mount, Elasticsearch could hang and
fail on startup if that mount point was not available. Finally, we
collected information through Lucene about whether or not a disk was a
spinning disk versus an SSD, information that we do not need since we
assume SSDs by default. This commit takes into consideration that we
simply do not need this heroic effort, we do not need information
about all file stores, and we do not need information about whether or
not a disk spins to greatly simplfy file store handling.

Relates #24402
2017-05-04 11:19:41 -04:00
James Baiera f5edd5049a Fixing permission errors for `KERBEROS` security mode for HDFS Repository (#23439)
Added missing permissions required for authenticating with Kerberos to HDFS. Also implemented 
code to support authentication in the form of using a Kerberos keytab file. In order to support 
HDFS authentication, users must install a Kerberos keytab file on each node and transfer it to the 
configuration directory. When a user specifies a Kerberos principal in the repository settings the 
plugin automatically enables security for Hadoop and begins the login process. There will be a 
separate PR and commit for the testing infrastructure to support these changes.
2017-05-04 10:51:31 -04:00
Adrien Grand 977016ba25 Do not index `_type` when there is at most one type. (#24363)
This change makes `_type` behave pretty much like `_index` when
`index.mapping.single_type` is true.
2017-05-04 16:29:35 +02:00
Ali Beyad 48031a2c5a [DOCS] Fixes the documentation on leading forward slashes in the (#24478)
[DOCS] Fixes the documentation on leading forward slashes in the
base_path of S3 repositories

Closes #23435
2017-05-03 22:58:43 -04:00
Nik Everett 45dd3780e2 CONSOLEify remaining _cat docs
Relates to #18160
2017-05-03 20:59:27 -04:00
Luca Cavanna 144f96eaeb Open/Close index api to allow_no_indices by default (#24401)
Open/Close index api have allow_no_indices set to false by default, while delete index has it set to true. The flag controls where a wildcard expression that matches no indices will be ignored or an error will be thrown instead. This commit aligns open/close default behaviour to that of delete index.
2017-05-03 16:22:26 +02:00
javanna a45e2efa00 fix typo in migrate_6_0/java.asciidoc 2017-05-03 15:00:44 +02:00
Dimitrios Liappis 79857357bf Docs: Update production notes for Docker
Add info about the base image used and the github repo of
elasticsearch-docker.

Clarify that setting `memlock=-1:-1` is only a requirement when
`bootstrap_memory_lock=true` and the alternatives we document
elsewhere in docs for disabling swap are valid for Docker as well.

Additionally, with latest versions of docker-ce shipping with
unlimited (or high enough) defaults for `nofile` and `nproc`, clarify
that explicitly setting those per ES container is not required, unless
they are not defined in the Docker daemon.

Finally simplify production `docker-compose.yml` example by removing
unneeded options.

Relates #24389
2017-05-03 14:27:31 +03:00
Luca Cavanna 92bfd16c58 Java api: ActionRequestBuilder#execute to return a PlainActionFuture (#24415)
This change makes the request builder code-path same as `Client#execute`. The request builder used to return a `ListenableActionFuture` when calling execute, which allows to associate listeners with the returned future. For async execution though it is recommended to use the `execute` method that accepts an `ActionListener`, like users would do when using `Client#execute`.

Relates to #24412
Relates to #9201
2017-05-03 11:20:53 +02:00
Nik Everett 732741dd8d Build that java api docs from a test (#24354)
We've had `QueryDSLDocumentationTests` for a while but it had a very
hopeful comment at the top about how we want to make sure that the
example in the query-dsl docs match up with the test but we never
had anything that made *sure* that they did. This changes that!

Now the examples from the query-dsl docs are all built from the
`QueryDSLDocumentationTests`. All except for the percolator example
because that is hard to do as it stands now.

To make this easier this change moves `QueryDSLDocumentationTests`
from core and into the high level rest client. This is useful for
two reasons:
1. We expect the high level rest client to be able to use the builders.
2. The code that builds that docs doesn't check out all of
Elasticsearch. It only checks out certain directories. Since we're
already including snippets from that directory we don't have to
make any changes to that process.

Closes #24320
2017-05-02 13:00:56 -04:00
Lisa Cawley d0a10cf140 [DOCS] Update XPack Reference URL for 5.4 (#24425) 2017-05-02 08:33:29 -07:00
Clinton Gormley 582b3c06b6 Added docs for batched_reduce_size
Relates to #23288
2017-05-02 14:25:03 +02:00
Jim Ferenczi 9d8254fadf Fix FieldCaps documentation
Fix the expected output for field_caps call.
Fixes #24413
2017-05-02 10:14:47 +02:00
Nik Everett ae0290bae9 Doc test: use propery regex for file size
The _cat/shards docs asserted that one of the columns looked like
a propery byte size but used a regex like `\d+\.\d+.*` which doesn't
match `0b` which is a possible value. Instead this uses
`\d(\.\d+)?[kmg]?b`.
2017-05-01 15:49:00 -04:00
Zachary Tong 735986c140
[DOCS] Tweak doc test to sync_flush
The response is attempting to illustrate the sync_id marker, but in
the test the index is too "fresh" to have a sync marker. So the test
needs to execute a sync flush behind the scenes so that the marker
is present
2017-05-01 15:15:32 -04:00
Zachary Tong 38273709b5 CONSOLEify some more Indices APIs (#24375)
* CONSOLEify doc testing for some more Indices APIs

Related to #18160
2017-05-01 13:56:39 -04:00
Zachary Tong 4e49c618f2 CONSOLEify Stats Aggregation docs (#24373) 2017-05-01 13:33:24 -04:00
Zachary Tong 130f1a56f1 Re-enable doc testing for Pipeline Aggregations (#24374)
* Re-enable doc testing for Pipeline Aggregations

Also adds a response + test for movavg pipeline
2017-05-01 13:30:51 -04:00
Jim Ferenczi 2508df6cc8 Add missing link for the WordDelimiterGraphFilter 2017-04-28 17:12:38 +02:00
Christoph Büscher 16a7cbe463 Add `count` value to rest output of `geo_centroid` (#24387)
Currently we don't write the count value to the geo_centroid aggregation rest response,
but it is provided via the java api and the count() method in the GeoCentroid interface. 
We should add this parameter to the rest output and also provide it via the getProperty()
method.
2017-04-28 16:25:22 +02:00
Nik Everett 94e3796908 Docs tests: cat/health can have max_task_wait_time
Make the doc test assertions ok with a non `-` value for
`max_task_wait_time`. These are rare, but possible:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-unix-compatibility/os=oraclelinux/900/consoleFull
2017-04-28 09:58:53 -04:00
Guillaume Le Floch 382a617d34 Handle multiple aliases in _cat/aliases api (#23698)
The alias parameter was documented as a list in our rest-spec, yet only the first value out of a list was getting read and processed. This commit adds support for multiple aliases to _cat/aliases

Closes #23661
2017-04-28 15:21:44 +02:00
Nik Everett 2412574e49 Docs: Upserts no longer support version
Closes #16671
2017-04-28 08:43:46 -04:00
Martijn van Groningen b77254871b
docs: document alternative for nested inner hits source
Closes #24110
2017-04-28 11:09:24 +02:00
Nik Everett 06aeb2307f Docs: remove experimental tag from update-by-query
We meant to remove it in #23621 but we removed it from the
documentation for the java client but not the rest documetnaiton.
2017-04-27 10:36:28 -04:00
Adrien Grand 1be2800120 Only allow one type on 7.0 indices (#24317)
This adds the `index.mapping.single_type` setting, which enforces that indices
have at most one type when it is true. The default value is true for 6.0+ indices
and false for old indices.

Relates #15613
2017-04-27 08:43:20 +02:00
Nik Everett 416feeb7f9 Rewrite description of `bool`'s `should` (#24342)
Docs: rewrite description of `bool`'s `should`

Rewrites the description of the `bool` query's `should`
clauses so it is (hopefully) more clear what the defaults
for `minimum_should_match` are.

There is still an `[IMPORTANT]` section about `minimum_should_match`
in a filter context. I think it is worth keeping because it is, well,
important.

Closes #23831
2017-04-26 14:09:26 -04:00
Till Backhaus b744dc3bcc Link to minimum master nodes docs from Zen docs
This commit adds a link to the minimum master nodes section of the
important settings docs from the Zen discovery docs to clarify the
meaning and importance of setting minimum master nodes to a quorum of
master-eligible nodes.

Relates #24311
2017-04-25 16:53:05 -04:00
Danilo Akamine 0adaf9fb4c Drop `search_analyzer` parameter from keyword.asciidoc (#24221)
`search_analyzer` isn't supported by `keyword` fields so this removes
it from the documentation for them.
2017-04-25 12:49:50 -04:00
Bodecker DellaMaria 92d6b2b60d Add newly open-sourced Scala REST client "escalar" (#24174)
Workday recently open-sourced our internal Scala wrapper for the Elasticsearch REST API. We plan to continue maintaining the library and use it in our products. Thought it would be a good idea to link it here in case anyone else is interested in using it!
2017-04-25 14:43:55 +02:00
Guillaume Le Floch 739cb35d1b Allow passing single scrollID in clear scroll API body (#24242)
* Allow single scrollId in string format

Closes #24233
2017-04-25 13:43:21 +02:00
Jason Tedor 508b774d76 Revert "[TEST] mute failing docs test"
This reverts commit f2e31cdeef.
2017-04-25 06:04:02 -04:00
Colin Goodheart-Smithe f2e31cdeef
[TEST] mute failing docs test 2017-04-25 10:30:08 +01:00
Nik Everett 3ae671aaf3 Docs test: Be ok with different _seq_nos
The test wanted specific _primary_terms and _seq_nos but there is
no need to specify that.
2017-04-24 17:22:37 -04:00
Nik Everett db93735321 CONSOLEify some of the docs documentation
delete, index, and update.

Relates to #18160
2017-04-24 17:06:54 -04:00
Nik Everett e429d66956 CONSOLEify some more docs
Relates to #18160
2017-04-24 16:08:19 -04:00
Nik Everett 5fbc86e2aa Allow painless to load stored fields (#24290)
We document that painless can load stored fields but it can't
because the classes that make that work aren't whitelisted.
2017-04-24 14:22:39 -04:00
Christoph Büscher 026bf2e3ee Remove getCountAsString() from InternalStats and Stats interface (#24291)
The `count` value in the stats aggregation represents a simple doc count
that doesn't require a formatted version. We didn't render an "as_string"
version for count in the rest response, so the method should also be
removed in favour of just using String.valueOf(getCount()) if a string
version of the count is needed.

Closes #24287
2017-04-24 18:40:57 +02:00
farisk 931198688c Document that painless doesn't support the "advanced text scoring" (#24271)
I just spent ages debugging a script I wrote after following the documentation. It was not clear to me that _index is not defined when using painless; if it was mentioned on this page I would have saved myself a lot of time.
2017-04-24 10:29:49 -04:00
Michael Despotopoulos 94b079ed42 Docs: Replace deprecated pluginList with Arrays.asList (#24270)
ESIntegTestCase#pluginList was remove removed in ES 5.0. We are using Arrays.asList instead.
2017-04-24 13:30:37 +02:00
Jim Ferenczi ca9aebac72 Update docs to include index sorting link 2017-04-23 20:40:24 +02:00
Jason Tedor 108d8905e2 Add note to docs regarding JAVA_HOME on Windows
For the Windows service, JAVA_HOME should be set to the path to the
JDK. We should make this clear in the docs to help users avoid
frustrating startup problems.

Relates #24260
2017-04-21 18:22:55 -04:00
Fabien Baligand 4a45579506 token_count type : add an option to count tokens (fix #23227) (#24175)
Add option "enable_position_increments" with default value true.
If option is set to false, indexed value is the number of tokens
(not position increments count)
2017-04-21 00:53:28 +02:00
Clinton Gormley ba552a0736 Added "release-state" support to plugin docs 2017-04-20 15:04:20 +02:00
Clinton Gormley 710cd05253 Added examples to cross cluster search of using cluster settings 2017-04-20 15:02:52 +02:00
Jason Tedor 446124b9d5 Reword note on whitespace in Log4j settings
This commit rewords the note on whitespace in Log4j settings to not
refer to only of the examples on the page, but instead be clear that the
note applies to all the examples on the page.
2017-04-20 07:34:30 -04:00
Jason Tedor 7a934bd6f4 Add note to docs on whitespace in Log4j settings
A confusing thing that can happen when configuring Log4j is that
extraneous whitespace throws off its configuration parsing yet the error
messages that arise give no indication that this is the problem. This
commit adds a note to the docs.

Relates #24198
2017-04-20 07:29:21 -04:00
makeyang 0e322a7e0b Update TransportClient java documentation (#24205)
We are using now in master `TransportAddress` instead of `InetSocketTransportAddress`.
2017-04-20 10:35:35 +02:00
Ryan Ernst 151a65ed17 Ec2 Discovery: Cleanup deprecated settings (#24150)
This commit removes the deprecated cloud.aws.* settings. It also removes
backcompat for specifying `discovery.type: ec2`, and unused aws signer
code which was removed in a previous PR.
2017-04-19 12:06:10 -07:00
Jason Tedor 4796557a30 Add primary term to doc write response
This commit adds the primary term to the doc write response.

Relates #24171
2017-04-19 14:44:22 -04:00
Iliiaz Akhmedov 688fa309bc Changing some grammar in docs (#24164) 2017-04-19 08:49:13 -06:00
Jim Ferenczi f05af0a382 Enable index-time sorting (#24055)
This change adds an index setting to define how the documents should be sorted inside each Segment.
It allows any numeric, date, boolean or keyword field inside a mapping to be used to sort the index on disk.
It is not allowed to use a `nested` fields inside an index that defines an index sorting since `nested` fields relies on the original sort of the index.
This change does not add early termination capabilities in the search layer. This will be added in a follow up.

Relates #6720
2017-04-19 14:36:11 +02:00
Dimitrios Liappis c0ac50eaa4 Clarify elasticsearch user uid:gid mapping in Docker docs
Elasticsearch runs as user elasticsearch with uid:gid 1000:1000 inside
the Docker container. Clarify that bind mounted local directories need
to be accessible by this user.

Relates #24092
2017-04-19 15:12:46 +03:00
Loek van Gool e11d892562 Update field-names-field.asciidoc (#24178)
fix typo in field name
2017-04-19 11:57:37 +02:00
Tanguy Leroux e81bbc288a Remove Ubuntu 12.04 (#24161)
Ubuntu 12.04 will be EOL on April 28, 2017.
2017-04-19 09:39:42 +02:00
Glen Smith 3ff014d07d ingest-node.asciidoc - Clarify json processor (#21876)
Add examples for the json processor.
2017-04-18 23:27:26 -04:00
Ryan Ernst 212f24aa27 Tests: Clean up rest test file handling (#21392)
This change simplifies how the rest test runner finds test files and
removes all leniency.  Previously multiple prefixes and suffixes would
be tried, and tests could exist inside or outside of the classpath,
although outside of the classpath never quite worked. Now only classpath
tests are supported, and only one resource prefix is supported,
`/rest-api-spec/tests`.

closes #20240
2017-04-18 15:07:08 -07:00
Suhas Karanth cee76295ca Update aggs reference documentation for 'keyed' options (#23758)
Add 'keyed' parameter documentation for following:
 - Date Histogram Aggregation
 - Date Range Aggregation
 - Geo Distance Aggregation
 - Histogram Aggregation
 - IP range aggregation
 - Percentiles Aggregation
 - Percentile Ranks Aggregation
2017-04-18 15:57:50 +02:00
Adrien Grand 4632661bc7 Upgrade to a Lucene 7 snapshot (#24089)
We want to upgrade to Lucene 7 ahead of time in order to be able to check whether it causes any trouble to Elasticsearch before Lucene 7.0 gets released. From a user perspective, the main benefit of this upgrade is the enhanced support for sparse fields, whose resource consumption is now function of the number of docs that have a value rather than the total number of docs in the index.

Some notes about the change:
 - it includes the deprecation of the `disable_coord` parameter of the `bool` and `common_terms` queries: Lucene has removed support for coord factors
 - it includes the deprecation of the `index.similarity.base` expert setting, since it was only useful to configure coords and query norms, which have both been removed
 - two tests have been marked with `@AwaitsFix` because of #23966, which we intend to address after the merge
2017-04-18 15:17:21 +02:00
Jim Ferenczi 66202dcf51 Include missing field-caps docs in search 2017-04-18 10:31:35 +02:00
BUPTAnderson a973197503 docs: fix search info in transport API docs (#22045)
fix some error!
2017-04-17 22:19:47 -04:00
Sakthipriyan Vairamani dd3bbfb153 doc: highlight that doc counts come from lucene (#23522)
The docs don't clearly explain that the deleted doc count also comes from lucene.
IMHO, it is worth highlighting this information separately, as a Note.
Apart from that, there should be an official recommended alternative as well.
2017-04-17 21:52:29 -04:00
Emir Beganović f4f6ce4b91 Docs: fix some case problems in aliases.asciidoc (#23657)
Make more things uppercase.
2017-04-17 21:45:19 -04:00
Himanshu Bhandoh c53a2b870e Fixing typo in spelling of rollover (#24146)
rolllover -> rollover
2017-04-17 21:31:11 -04:00
Andrew Selden f8b15abe9a Update reference docs for geocentroid aggregation. (#24141)
This includes a link to the Wikipedia page explaining what a centroid
is.

Closes #24140
2017-04-17 21:27:43 -04:00
Ulugbek Baymuradov 9cb477d387 Update filter-aggregation.asciidoc (#24138)
Fix a discrepancy between the example and the prose.
2017-04-17 18:46:13 -04:00
Ryan Ernst 1629c9fd5c S3 Repository: Cleanup deprecated settings (#24097)
This commit removes all deprecated settings which start with
`cloud.aws`, `repositories.s3` and repository level client settings.
2017-04-17 14:18:49 -07:00
Nik Everett 718e332c64 Docs: Be ok with long recovery times
The _cat docs were asserting that an index took only some number of
milliseconds to recovery. In this build it took a whole second:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+5.3+multijob-intake/192/consoleFull

So this changes the assertion to be ok with a second.
2017-04-17 16:56:12 -04:00
Suhas Karanth f97d8bc78d Update reference docs for Highlighter fragmenter (#23754)
Explain the fragmenter and add examples.
2017-04-17 14:00:24 -04:00
Lee Hinman 162ce85ff2 Remove customization of ES_USER and ES_GROUP
This removes the ability to configure Elasticsearch to use custom username
and/or group when it is run.

Resolves #23848
2017-04-14 13:24:21 -06:00
Jason Tedor 99e0268e0a Remove support for default settings
Today Elasticsearch allows default settings to be used only if the
actual setting is not set. These settings are trappy, and the complexity
invites bugs. This commit removes support for default settings with the
exception of default.path.data, default.path.conf, and default.path.logs
which are maintainted to support packaging. A follow-up will remove
support for these as well.

Relates #24093
2017-04-13 14:25:45 -04:00
debadair 7e1903469e [DOCS] Added note about Elastic Cloud to improve 'elastic aws' SERP results. 2017-04-12 17:57:03 -07:00
Nik Everett c17bee571f Update scripts/security docs for sandboxed world (#23977)
Drops any mention of non-sandboxed scripting languages other than a
brief "we don't support them and we shouldn't because A and B"
statement.

Relates to #23930
2017-04-11 14:55:57 -04:00
Lee Hinman 5cace8e48a Remove shadow replicas
Resolves #22024
2017-04-11 11:26:26 -06:00
Dmitriy Troyan 1ea4c285b6 Update aggregation.asciidoc (#24042)
There are four kinds of aggregations now, not three.
2017-04-11 09:02:38 -04:00
Simon Willnauer f22e0dc30b Add cross-cluster search remote cluster info API (#23969)
This commit adds an API to discover information like seed nodes,
http addresses and connection status of a configured remote cluster.

Closes #23925
2017-04-11 09:24:40 +02:00
Simon Willnauer 040b86a76b Set shard count limit to unlimited (#24012)
Now that we have incremental reduce functions for topN and aggregations
we can set the default for `action.search.shard_count.limit` to unlimited.
This still allows users to restrict these settings while by default we executed
across all shards matching the search requests index pattern.
2017-04-10 17:09:21 +02:00
Jason Tedor 669fcd05e6 Add note to docs on duplicate keys in config
This commit adds a note to the migration docs that duplicate keys in the
configuration file are no longer permitted.

Relates #24022
2017-04-10 10:58:59 -04:00
Jim Ferenczi 9b3c85dd88 Deprecate _field_stats endpoint (#23914)
_field_stats has evolved quite a lot to become a multi purpose API capable of retrieving the field capabilities and the min/max value for a field.
In the mean time a more focused API called `_field_caps` has been added, this enpoint is a good replacement for _field_stats since he can
retrieve the field capabilities by just looking at the field mapping (no lookup in the index structures).
Also the recent improvement made to range queries makes the _field_stats API obsolete since this queries are now rewritten per shard based on the min/max found for the field.
This means that a range query that does not match any document in a shard can return quickly and can be cached efficiently.
For these reasons this change deprecates _field_stats. The deprecation should happen in 5.4 but we won't remove this API in 6.x yet which is why
 this PR is made directly to 6.0.
 The rest tests have also been adapted to not throw an error while this change is backported to 5.4.
2017-04-10 10:10:16 +02:00
Jason Tedor 61c5976aee Upgrade to Log4j 2.8.2
This commit upgrades the Log4j dependencies from version 2.7 to version
2.8.2. This release includes a fix for a case where Log4j could lose
exceptions in the presence of a security manager.

Relates #23995
2017-04-09 07:19:16 -04:00
Jason Tedor 9056e0cb49 Remove hidden file leniency from plugin service
This commit removes some leniency from the plugin service which skips
hidden files in the plugins directory. We really want to ensure the
integrity of the plugin folder, so hasta la vista leniency.

Relates #23982
2017-04-08 18:22:44 -04:00
Ryan Ernst 83ba677e7f Discovery EC2: Remove region setting (#23991)
We have both endpoint and region settings. Region was removed from s3 to
simplify configuration. This is the ec2 equivalent.

closes #22758
2017-04-07 22:06:40 -07:00
Ryan Ernst 05e2ea1aef AWS Plugins: Remove signer type setting (#23984)
This commit removes support for s3 signer type in 6.0, and adds a note
to the migration guide.

closes #22599
2017-04-07 16:46:17 -07:00
Nik Everett 7fad7c675d Rewrite the scripting security docs (#23930)
They needed to be updated now that Painless is the default and
the non-sandboxed scripting languages are going away or gone.

I dropped the entire section about customizing the classloader
whitelists. In master this barely does anything (exposes more
things to expressions).
2017-04-07 11:46:41 -04:00
Martijn van Groningen 3d9671a668
[PERCOLATOR] Allowing range queries with now ranges inside percolator queries.
Before now ranges where forbidden, because the percolator query itself could get cached and then the percolator queries with now ranges that should no longer match, incorrectly will continue to match.
By disabling caching when the `percolator` is being used, the percolator can now correctly support range queries with now based ranges.

 I think this is the right tradeoff. The percolator query is likely to not be the same between search requests and disabling range queries with now ranges really disabled people using the percolator for their use cases.

 Also fixed an issue that existed in the percolator fieldmapper, it was unable to find forbidden queries inside `dismax` queries.

 Closes #23859
2017-04-07 08:44:43 +02:00
Nik Everett 048191ceb6 CONSOLEify highlighting a function_score docs
Converts many of the partial examples into full search requests.

Relates #18160
2017-04-06 08:13:56 -04:00
Nik Everett 471af48170 Docs: fix response object name
In #22810 I renamed a response object used by reindex and friends
but didn't update the java-api documentation that uses it. This
makes that update.
2017-04-05 16:03:49 -04:00
Nik Everett 0b20a59391 Docs test: defend against round numbers
If a shard has a nice, round number the test in the `_cat/shards`
reference file would fail. They should be ok with it. A failure:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+5.3+multijob-unix-compatibility/os=fedora/93/console
2017-04-05 15:31:11 -04:00
Suhas Karanth 777b5a3c16 Correct documentation for Min Bucket Aggregation (#23867) 2017-04-05 12:39:37 +02:00
Jayant Ameta 47160ba6ed Update secure-settings.asciidoc (#23891) 2017-04-05 11:18:23 +02:00
Alex Chan 7f43b75d2b Add a missing apostrophe (let us ~> let's) (#23905) 2017-04-04 14:58:32 -06:00
Jason Tedor 51b5dbffb7 Disable bootstrap checks for single-node discovery
While there are use-cases where a single-node is in production, there
are also use-cases for starting a single-node that binds transport to an
external interface where the node is not in production (for example, for
testing the transport client against a node started in a Docker
container). It's tricky to balance the desire to always enforce the
bootstrap checks when a node might be in production with the need for
the community to perform testing in situations that would trip the
bootstrap checks. This commit enables some flexibility for these
users. By setting the discovery type to "single-node", we disable the
bootstrap checks independently of how transport is bound. While this
sounds like a hole in the bootstrap checks, the bootstrap checks can
already be avoided in the single-node use-case by binding only HTTP but
not transport. For users that are genuinely in production on a
single-node use-case with transport bound to an external use-case, they
can set the system property "es.enable.bootstrap.checks" to force
running the bootstrap checks. It would be a mistake for them not to do
this.

Relates #23598
2017-04-04 09:39:04 -04:00
wyukawa 3274eab41d Fix bulk queue size in thread pool docs
This commit fixes an incorrect specification for the default queue size
for the bulk thread pool in the thread pool docs.

Relates #23870
2017-04-03 22:39:24 -04:00
Lee Hinman b6b9ef8e26 [DOCS] Remove line about eager loading global ordinals
Fielddata can no longer be configured to be loaded eagerly (it only accepts
`true` and `false`), so this line is a little misleading because it talks about
a procedure we can no longer do.
2017-04-03 12:56:21 -06:00
Nik Everett 9d2293b381 CONSOLEify the "using scripts" documentation
I found an error in one of the Painless scripts as part of
the conversion.

Relates to #18160
2017-04-03 10:15:29 -04:00
David Pilato 0951e20a87 Merge branch 'pr/22728-azure-backoff' 2017-04-03 10:54:03 +02:00
David Pilato 17be03e85e Add Backoff policy to azure repository
With this commit, Azure repositories are now using an Exponential Backoff policy before failing the backup.
It uses Azure SDK default values for this policy:

* `30s` delta backoff base with
   * `3s` min
   * `90s` max
* `3` retries max

Users can define the number of retries they wish by setting `cloud.azure.storage.xxx.max_retries` where `xxx` is the azure named account.

Closes #22728.
2017-04-03 10:52:44 +02:00
Nik Everett ad69503dce CONSOLEify analysis docs
Converts the analysis docs to that were marked as json into `CONSOLE`
format. A few of them were in yaml but marked as json for historical
reasons. I added more complete examples for a few of the less obvious
sounding ones.

Relates to #18160
2017-04-02 11:17:14 -04:00
Nik Everett 514187be8e Fix language in some docs
The pattern-analyzer docs contained a snippet that was an expanded
regex that was marked as `[source,js]`. This changes it to
`[source,regex]`.

The htmlstrip-charfilter and pattern-replace-charfilter docs had
examples that were actually a list of tokens but marked `[source,js]`.
This marks them as `[source,text]` so they don't count as unconverted
CONSOLE snippets.

The pattern-replace-charfilter also had a doc who's test was
skipped because of funny interaction with the test framework. This
fixes the test.

Three more down, eighty-two to go.

Relates to #18160
2017-04-01 14:45:44 -04:00
Nik Everett 9baa48a928 CONSOLEify lang-analyzer docs
CONSOLEifies the lang-analyzer docs and replaces the (invalid)
empty `keyword_marker` setups that were on the page with one
that contains the word "example" translated into the appropriate
language.

Relates to #18160
2017-04-01 14:21:58 -04:00
Jim Ferenczi a8250b26e7 Add FieldCapabilities (_field_caps) API (#23007)
This change introduces a new API called `_field_caps` that allows to retrieve the capabilities of specific fields.

Example:

````
GET t,s,v,w/_field_caps?fields=field1,field2
````
... returns:
````
{
   "fields": {
      "field1": {
         "string": {
            "searchable": true,
            "aggregatable": true
         }
      },
      "field2": {
         "keyword": {
            "searchable": false,
            "aggregatable": true,
            "non_searchable_indices": ["t"]
            "indices": ["t", "s"]
         },
         "long": {
            "searchable": true,
            "aggregatable": false,
            "non_aggregatable_indices": ["v"]
            "indices": ["v", "w"]
         }
      }
   }
}
````

In this example `field1` have the same type `text` across the requested indices `t`, `s`, `v`, `w`.
Conversely `field2` is defined with two conflicting types `keyword` and `long`.
Note that `_field_caps` does not treat this case as an error but rather return the list of unique types seen for this field.
2017-03-31 15:34:46 +02:00
Clinton Gormley 5eba90f37c Doc values can no longer be updated
Closes #23820
2017-03-31 15:29:01 +02:00
Nik Everett 653f50973a CONSOLEify geo-shape docs
`CONSOLE`ify geo-shape type and geo-shape query docs.

Relates to #18160
2017-03-31 09:11:54 -04:00
David Pilato f5d41dfc9d Merge branch 'pr/remove-repositories-azure-settings' 2017-03-31 12:33:12 +02:00
Nik Everett 9abb125417 Fix exists query doc
I managed to push the last one without testing it because I'd changed
the way I run tests locally and hadn't picked it up. Ooops. This one
works better.
2017-03-30 22:26:10 -04:00
Nik Everett bc33753aee Mark exists-query dsl doc properly
All the docs for the `exists` query that aren't marked as `CONSOLE`
aren't actually `CONSOLE`-worthy so this marks them as `NOTCONSOLE`.

It also rewrites the text around `missing` query. Since it was
removed in 5.0 we don't need to talk about it in the 6.0 docs.

Relates to #18160
2017-03-30 22:01:07 -04:00
Nik Everett 5f91241f57 CONSOLEify geo aggregation docs
Turns the top example in each of the geo aggregation docs into a working
example that can be opened in CONSOLE. Subsequent examples can all also
be opened in console and will work after you've run the first example.
All examples are tested as part of the build.
2017-03-30 21:28:52 -04:00
Christoph Wurm 2720fc0b43 Clarify task cancellation command (#23667)
Makes it explicit that the node_id has to be included when canceling a task.
2017-03-30 20:21:21 +02:00
Ali Beyad a4b37bf7fe [DOCS] Multiple clusters connected to the same repository (#23807) 2017-03-30 13:08:41 -04:00
Abdon Pijpelink ef1329727d Update compound-word-tokenfilter.asciidoc (#23817)
Updated URL to OFFO Sourceforge project
2017-03-30 12:27:32 +02:00
Glen Smith c62d4b7b0f Clarify preference docs
This commit clarifies the preference docs regarding the explanation of
how operations are routed by default. In particular, the previous use of
"shard replicas" was confusing as it could imply an operation would only
be routed to replicas by default.

Relates #23794
2017-03-29 12:55:47 -04:00
Ali Beyad 2120086d82 Adds pattern keyword marker filter support (#23600)
This commit adds support for the pattern keyword marker filter in
Lucene.  Previously, the keyword marker filter in Elasticsearch
supported specifying a keywords set or a path to a set of keywords.
This commit exposes the regular expression pattern based keyword marker
filter also available in Lucene, so that any token matching the pattern
specified by the `keywords_pattern` setting is excluded from being
stemmed by any stemming filters.

Closes #4877
2017-03-28 11:13:34 -04:00
Dimitris Athanasiou 34f116eae3 Require explicit query in _delete_by_query API (#23632)
As the query of a search request defaults to match_all,
calling _delete_by_query without an explicit query may
result in deleting all data.

In order to protect users against falling into that
pitfall, this commit adds a check to require the explicit
setting of a query.

Closes #23629
2017-03-28 15:44:57 +01:00
Ali Beyad 8359dd05c9 Adds boolean similarity to Elasticsearch (#23637)
This commit adds the boolean similarity scoring from Lucene to
Elasticsearch.  The boolean similarity provides a means to specify that
a field should not be scored with typical full-text ranking algorithms,
but rather just whether the query terms match the document or not.
Boolean similarity scores a query term equal to its query boost only.
Boolean similarity is available as a default similarity option and thus
a field can be specified to have boolean similarity by declaring in its
mapping:
    "similarity": "boolean"

Closes #6731
2017-03-28 10:17:23 -04:00
Nik Everett efc62f193f Docs: fix health response test
I managed to test the wrong snippet before pushing the last
commit.... This fixes the error in it.
2017-03-28 08:29:10 -04:00
Nik Everett d505c20373 Docs: Clean up response test in getting_started
It had drifted from a fix applied to 5.x. The 5.x fix was more
readable. This applies the 5.x fix.
2017-03-28 08:29:10 -04:00
Shane Connelly 16a8d5245f Reflect cross-cluster search in "dedicated" terminology (#23771)
* Reflects cross-cluster search in dedicated node settings

* Fix space issue
2017-03-27 23:14:34 -04:00
Robin Clarke 677fd68f16 Fix docs for plugin install via proxy on Windows
This commit addresses an issue with the docs for plugin install via a
proxy on Windows where the HTTP proxy options were incorrectly
specified.

Relates #23757
2017-03-27 07:40:11 -04:00
Jason Tedor a6c4234575 Add early-access check
The OpenJDK project provides early-access builds of upcoming
releases. These early-access builds are not suitable for
production. These builds sometimes end up on systems due to aggressive
packaging (e.g., Ubuntu). This commit adds a bootstrap check to ensure
these early-access builds are not being used in production.

Relates #23743
2017-03-24 14:52:50 -04:00
Igor Motov 63e1403017 Docs: add description of possible snapshot states 2017-03-23 15:20:38 -04:00
Nik Everett a783c6c85c CONSOLEify some more docs
And expand on the `stemmer_override` examples, including the
file on disk and an example of specifying the rules inline.

Relates to #18160
2017-03-22 17:58:06 -04:00
Igor Motov f927a2708d Make it possible to validate a query on all shards instead of a single random shard (#23697)
This is especially useful when we rewrite the query because the result of the rewrite can be very different on different shards. See #18254 for example.
2017-03-22 17:39:21 -04:00
Nik Everett e860fe7363 CONSOLEify some more docs
Relates to #18160
2017-03-22 17:15:14 -04:00
Nik Everett 1dee2f32a4 Docs: CONSOLEify synonym tokenfiler docs
Relates to #18160
2017-03-22 16:30:52 -04:00
Nik Everett 1c1b29400b Docs: Fix language on a few snippets
They aren't `js`, they are their own thing.

Relates to #18160
2017-03-22 15:57:28 -04:00
Stefan Gorgiovski 798c19dd7f Deprecate request_cache for clear-cache (#23638)
It is called `request` now.
2017-03-22 08:28:04 -04:00
Clinton Gormley 5cf13f29bb Update ingest-node.asciidoc
Fixed docs typo
2017-03-22 10:44:11 +01:00
Martijn van Groningen b116b8f0cb
[DOCS] Update the docs about the fact that global ordinals for _parent field are loaded eagerly instead of lazily by default.
Relates to #8053
2017-03-22 10:39:39 +01:00
Ryan Ernst 105bc0ee1f Docs: Add note about updating plugins requiring removal and reinstallation (#23597)
closes #20321
2017-03-21 11:59:21 -07:00
Florian Hopf 09753d6a86 Fix typo in allocation explain API docs
This commit addresses a simple typo in the application explain API docs.

Relates #23669
2017-03-21 08:41:54 -04:00
Minwoo Kim 6b46334d31 Docs: fix a typo in transport client's put-mapping.asciidoc (#23607)
It was talking about the wrong type.
2017-03-20 15:05:40 -04:00
Nik Everett 23b8b97275 Use include-tagged macro for high level client docs (#23438)
This should make it simpler to include example snippets from
tests.
2017-03-20 14:50:19 -04:00
Max Goodridge faa6ade035 Fix typo 'Elastisearch' -> 'Elasticsearch' (#23633)
Corrects `Elastisearch` to `Elasticsearch` in some docs and a packaging test utility.
2017-03-20 10:07:59 -04:00
Jim Ferenczi b8c352fc3f Add support for fragment_length in the unified highlighter (#23431)
* Add support for fragment_length in the unified highlighter

This commit introduce a new break iterator (a BoundedBreakIterator) designed for the unified highlighter
 that is able to limit the size of fragments produced by generic break iterator like `sentence`.
The `unified` highlighter now supports `boundary_scanner` which can `words` or `sentence`.
The `sentence` mode will use the bounded break iterator in order to limit the size of the sentence to `fragment_length`.
When sentences bigger than `fragment_length` are produced, this mode will break the sentence at the next word boundary **after**
 `fragment_length` is reached.
2017-03-17 18:10:13 +01:00
Arnaud Venturi 89cbb0fed5 Docs: Corrected path to elasticsearch-plugin (#23622) 2017-03-17 10:57:58 -04:00
Christoph Büscher 413bf05956 Docs: Add comma to reverse nested agg snippet 2017-03-17 14:07:18 +01:00
Clinton Gormley e37cdab87f Update scripting.asciidoc
Fixed bad asciidoc
2017-03-16 19:37:38 -07:00
Jason Tedor b9ac69cbd8 Unmark reindex as experimental
The reindex API is mature now, and we will work to maintain backwards
compatibility in accordance with our backwards compatibility
policy. This commit unmarks the reindex API as experimental.

Relates #23621
2017-03-16 22:02:15 -04:00
Robin Stocker 6e9dfb3348 Docs: Specify that byte units use powers of 1024 (#23574)
In SI units, "kilobyte" or "kB" would mean 1000 bytes, whereas "KiB" is
used for 1024. Add a note in `api-conventions.asciidoc` to clarify the
meaning in Elasticsearch.
2017-03-16 12:39:45 -04:00
Md.Abdulla-Al-Sun 8b6d521037 Remove Settings.settingsBuilder (#23575)
In this repository, `Settings.builder` is used everywhere although it does exactly same as `Settings.settingsBuilder`. With the reference of the commit 42526ac28e , I think mistakenly this `Settings.settingsBuilder` remains in.
2017-03-16 11:20:42 +01:00
Jack Conradson 8e04561c0d Change params._source to params['_source'] in example. 2017-03-15 17:29:31 -07:00
Jack Conradson 4c11ebc8b9 Fix example in documentation for Painless using _source. (#21322) 2017-03-15 17:18:34 -07:00
Jason Tedor f7b8128f92 Enable explicitly enforcing bootstrap checks
This commit adds a system property that enables end-users to explicitly
enforce the bootstrap checks, independently of the binding of the
transport protocol. This can be useful for single-node production
systems that do not bind the transport protocol (and thus the bootstrap
checks would not be enforced).

Relates #23585
2017-03-15 10:36:17 -07:00
Pavel Chertorogov ff1530592e Docs: Fix indentation in has-child-query.asciidoc (#23565) 2017-03-13 08:41:18 -07:00
Pavel Chertorogov 5da7cefbe2 Docs: Fix indentation in has-parent-query.asciidoc 2017-03-13 08:17:11 -07:00
NFM f8fa5c96aa Fix indentation in sort docs
This commit fixes the indentation in an example query in the sort docs.

Relates #23561
2017-03-12 17:08:06 -07:00
David Pilato e3b1c95623 Fix settingsBuilder() with builder() 2017-03-09 02:52:19 -08:00
Jason Tedor f4a432e456 Add note regarding out-of-sync replicas
This commit adds a note to the resiliency status page regarding the fact
that replicas can fall out of sync with the primary shard after primary
promotion occurs due to a failing primary shard.

Relates #23503
2017-03-07 14:25:23 -05:00
Shashank Singh 7420bda8ed Fix link to Debian install instructions
The link for the Debian install instructions was mistakenly pointing to
the RPM instructions. This commit fixes the reference.

Relates #23485
2017-03-06 14:41:30 -08:00
Ali Beyad 577d2a6a1d Adds cluster state size to /_cluster/state response (#23440)
This commit adds the size of the cluster state to the response for the
get cluster state API call (GET /_cluster/state).  The size that is
returned is the size of the full cluster state in bytes when compressed.
This is the same size of the full cluster state when serialized to
transmit over the network.  Specifying the ?human flag displays the
compressed size in a more human friendly manner.  Note that even if the
cluster state request filters items from the cluster state (so a subset
of the cluster state is returned), the size that is returned is the
compressed size of the entire cluster state.

Closes #3415
2017-03-02 14:20:29 -05:00
Jason Tedor e256ce452b Add documentation for Bash requirement
This commit adds a note to the docs regarding the requirements for Bash.

Relates #23464
2017-03-02 13:36:39 -05:00
Luca Cavanna cc65a94fd4 [TEST] improve yaml test sections parsing (#23407)
Throw error when skip or do sections are malformed, such as they don't start with the proper token (START_OBJECT). That signals bad indentation, which would be ignored otherwise. Thanks (or due to) our pull parsing code, we were still able to properly parse the sections, yet other runners weren't able to.

Closes #21980

* [TEST] fix indentation in matrix_stats yaml tests

* [TEST] fix indentation in painless yaml test

* [TEST] fix indentation in analysis yaml tests

* [TEST] fix indentation in generated docs yaml tests

* [TEST] fix indentation in multi_cluster_search yaml tests
2017-03-02 12:43:20 +01:00
msancho a37c759ba2 Fixed typo in documentation (#23406)
* Fixed typo in documentation

The option in "gap_policy" "insert_zeros" was missing a trailing "s"

* Update movavg-aggregation.asciidoc
2017-03-01 15:22:26 +01:00
Randall Britten c54fa177ef Docs: Fixed Parameters tables to use defaults col (#23396)
Occurred in a few places for pipeline aggregates.
2017-03-01 14:47:21 +01:00
gabriel-tessier 42f6d1e900 fix typo (#23424) 2017-03-01 14:13:00 +01:00
David Pilato 8d31a2fa06 Merge branch 'doc/hlclient-delete' 2017-03-01 13:39:32 +01:00
David Pilato ee4a17a0e2 Adapt per Nik's comments 2017-03-01 11:51:06 +01:00
David Pilato 798bf966a3 Don't mention cluster compatibility 2017-03-01 11:28:13 +01:00
David Pilato 6c526adaed Merge branch 'master' into doc/hlclient-delete 2017-03-01 11:26:36 +01:00
Adrien Grand b388389ada Remove support for the include/pattern syntax. (#23141)
Relates #22933
2017-03-01 10:00:38 +01:00
Jürgen van Dijk 7609d67c7d Typo (#23344) 2017-02-28 16:33:09 +01:00
Randall Britten 05fd2eca6f Docs: corrected "and" --> "an" (#23376) 2017-02-27 14:38:29 -05:00
Randall Britten 98e19cced4 Docs: Corrected definition of type param of children agg (#23377) 2017-02-27 14:38:28 -05:00
javanna 4f487ab1b9 [TEST] randomize request content_type between all of the supported formats 2017-02-27 12:27:03 +01:00
David Pilato c373ed102f Extract documentation from test code 2017-02-24 17:38:18 +01:00
David Pilato 60661ec9cc Add first High level client documentation
The REST Client is split into 2 parts:

* Low level
* High level

The High level client has a main common section and the document delete API documentation as a start.
2017-02-24 13:52:36 +01:00
Jim Ferenczi 63bdd01eb7 Expose WordDelimiterGraphTokenFilter (#23327)
This change exposes the new Lucene graph based word delimiter token filter in the analysis filters.
Unlike the `word_delimiter` this token filter named `word_delimiter_graph` correctly handles multi terms expansion at query time.

Closes #23104
2017-02-24 00:53:38 +01:00
Shai Erera eeac6d27f2 Add BreakIteratorBoundaryScanner support for FVH (#23248)
This commit adds a boundary_scanner property to the search highlight
request so the user can specify different boundary scanners:

* `chars` (default,  current behavior)
* `word` Use a WordBreakIterator
* `sentence` Use a SentenceBreakIterator

This commit also adds "boundary_scanner_locale" to define which locale
should be used when scanning the text.
2017-02-23 23:32:22 +01:00
Lee Hinman 6c9b89b882 [TEST] Fix incorrect test cluster name in cluster health doc tests 2017-02-22 17:18:11 -07:00
Ryan Ernst 175bda64a0 Build: Rework integ test setup and shutdown to ensure stop runs when desired (#23304)
Gradle's finalizedBy on tasks only ensures one task runs after another,
but not immediately after. This is problematic for our integration tests
since it allows multiple project's integ test clusters to be
simultaneously. While this has not been a problem thus far (gradle 2.13
happened to keep the finalizedBy tasks close enough that no clusters
were running in parallel), with gradle 3.3 the task graph generation has
changed, and numerous clusters may be running simultaneously, causing
memory pressure, and thus generally slower tests, or even failure if the
system has a limited amount of memory (eg in a vagrant host).

This commit reworks how integ tests are configured. It adds an
`integTestCluster` extension to gradle which is equivalent to the current
`integTest.cluster` and moves the rest test runner task to
`integTestRunner`.  The `integTest` task is then just a dummy task,
which depends on the cluster runner task, as well as the cluster stop
task. This means running `integTest` in one project will both run the
rest tests, and shut down the cluster, before running `integTest` in
another project.
2017-02-22 12:43:15 -08:00
Christopher Best eeaa0ccec2 Update getting-started.asciidoc (#23296) 2017-02-22 11:06:27 +01:00
Alexander Reelsen 6781c4320c Documentation: Consoleify cat shards/recovery API docs (#23116)
Relates #23001
2017-02-22 09:18:10 +01:00
Glen Smith a590a22ea3 Add note and link to 'tune for disk usage' (#23252)
* Add note and link to 'tune for disk usage'

* Changed formatting as suggested

Thanks, @ clintongormley!
2017-02-20 20:31:19 +01:00
David Pilato da907e7a7d Remove global `repositories.azure` settings
Today we have multiple ways to define settings when a user needs to create a repository:

* in `elasticsearch.yml` file using `repositories.azure` prefix
* when creating the repository itself with `PUT _snaphot/repo`

The plan is to:

* Deprecate `repositories.azure` settings in 5.x (done with #22856)
* Remove in 6.x (this PR)

Related to #22800
2017-02-20 12:22:54 +01:00
Jay Modi b234644035 Enforce Content-Type requirement on the rest layer and remove deprecated methods (#23146)
This commit enforces the requirement of Content-Type for the REST layer and removes the deprecated methods in transport
requests and their usages.

While doing this, it turns out that there are many places where *Entity classes are used from the apache http client
libraries and many of these usages did not specify the content type. The methods that do not specify a content type
explicitly have been added to forbidden apis to prevent more of these from entering our code base.

Relates #19388
2017-02-17 14:45:41 -05:00
Jason Tedor 21181d7865 Fix more missing markup in configuration docs
This commit fixes some missing markup in the configuration docs in the
logging configuration section.
2017-02-16 19:00:49 -05:00
Andreas Roussos 788c64848b [DOCS] Fixed various typos in the 'cat APIs' section (#23216) 2017-02-16 20:41:42 +01:00
Jason Tedor 0e3e111921 Replace HTTP link with HTTPS link
This commit replaces an HTTP link in the configuration docs with an
HTTPS link.
2017-02-16 12:19:30 -05:00
Jason Tedor 69dc730341 Fix missing markup in configuration docs
This commit adds a missing backtick in the configuration docs.
2017-02-16 12:19:30 -05:00
markwalkom ced99dde50 Update stop-analyzer.asciidoc (#23195)
Clarified where the stopwords file needs to live
2017-02-16 13:36:15 +01:00
Lee Hinman 5443f7d625 Console-ify curl statements for allocation explain API docs (#23190)
* Console-ify curl statements for allocation explain API docs

Relates to #23001

* Fix tests

* Remove exclusion from build.gradle

* Call out index creation in prose

* Add console back and skip test
2017-02-15 17:18:07 -07:00
Nik Everett a6cf170384 Add content-type header to curl in getting-started
Relates to #23001
2017-02-15 18:14:25 -05:00
Ali Beyad 71739623d3 Consolify snapshot documentation (#23189)
This commit brings the snapshot documentation in conformity
with the CONSOLE format, and fixes the docs so that the documentation
tests can be run against them.
2017-02-15 18:13:27 -05:00
Lee Hinman ac3cc9641b
[DOCS] Console-ify shadow replica documentation
Relates to #23001
2017-02-15 11:58:53 -07:00
Areek Zillur 2a38f155de Docs: CONSOLEify multi-get.asciidoc (#23122)
* Docs: CONSOLEify multi-get.asciidoc

relates #23001

* incorporate feedback
2017-02-15 13:36:07 -05:00
sungjunyoung ec686b52c1 Update getting-started.asciidoc (#23180)
The "sort" and "_score" fields are contained within the "hits" field.
2017-02-15 10:55:09 +01:00
AlexNodex 139eb69fe4 Typo (#23179)
autoGeneratePhraseQueries should be auto_generate_phrase_queries
2017-02-15 10:10:06 +01:00
Catherine Snow 51bad4300c Fix typo (#23171) 2017-02-15 09:38:10 +01:00
Adrien Grand 8d6a41f671 Nested queries should avoid adding unnecessary filters when possible. (#23079)
When nested objects are present in the mappings, many queries get deoptimized
due to the need to exclude documents that are not in the right space. For
instance, a filter is applied to all queries that prevents them from matching
non-root documents (`+*:* -_type:__*`). Moreover, a filter is applied to all
child queries of `nested` queries in order to make sure that the child query
only matches child documents (`_type:__nested_path`), which is required by
`ToParentBlockJoinQuery` (the Lucene query behing Elasticsearch's `nested`
queries).

These additional filters slow down `nested` queries. In 1.7-, the cost was
somehow amortized by the fact that we cached filters very aggressively. However,
this has proven to be a significant source of slow downs since 2.0 for users
of `nested` mappings and queries, see #20797.

This change makes the filtering a bit smarter. For instance if the query is a
`match_all` query, then we need to exclude nested docs. However, if the query
is `foo: bar` then it may only match root documents since `foo` is a top-level
field, so no additional filtering is required.

Another improvement is to use a `FILTER` clause on all types rather than a
`MUST_NOT` clause on all nested paths when possible since `FILTER` clauses
are more efficient.

Here are some examples of queries and how they get rewritten:

```
"match_all": {}
```

This query gets rewritten to `ConstantScore(+*:* -_type:__*)` on master and
`ConstantScore(_type:AutomatonQuery {\norg.apache.lucene.util.automaton.Automaton@4371da44})`
with this change. The automaton is the complement of `_type:__*` so it matches
the same documents, but is faster since it is now a positive clause. Simplistic
performance testing on a 10M index where each root document has 5 nested
documents on average gave a latency of 420ms on master and 90ms with this change
applied.

```
"term": {
  "foo": {
    "value": "0"
  }
}
```

This query is rewritten to `+foo:0 #(ConstantScore(+*:* -_type:__*))^0.0` on
master and `foo:0` with this change: we do not need to filter nested docs out
since the query cannot match nested docs. While doing performance testing in
the same conditions as above, response times went from 250ms to 50ms.

```
"nested": {
  "path": "nested",
  "query": {
    "term": {
      "nested.foo": {
        "value": "0"
      }
    }
  }
}
```

This query is rewritten to
`+ToParentBlockJoinQuery (+nested.foo:0 #_type:__nested) #(ConstantScore(+*:* -_type:__*))^0.0`
on master and `ToParentBlockJoinQuery (nested.foo:0)` with this change. The
top-level filter (`-_type:__*`) could be removed since `nested` queries only
match documents of the parent space, as well as the child filter
(`#_type:__nested`) since the child query may only match nested docs since the
`nested` object has both `include_in_parent` and `include_in_root` set to
`false`. While doing performance testing in the same conditions as above,
response times went from 850ms to 270ms.
2017-02-14 16:05:19 +01:00
Boaz Leskes 70a3ac1767 Add a note about `cluster.routing.allocation.node_concurrent_recoveries` (#23160)
Closes #23152
2017-02-14 14:14:41 +02:00
Loek van Gool 214a3536aa Update redirects.asciidoc (#23148) 2017-02-13 16:23:25 +01:00
Giuseppe ecbeffcb1e Add note about min_score filtering efficiency (#23109)
* Add note about min_score filtering efficiency

* Reword to mention 'HAVING'

* Remove reference to HAVING
2017-02-13 12:15:01 +01:00
Adrien Grand f3509b8003 Consolify docs/reference/analysis/tokenfilters/pattern-capture-tokenfilter.asciidoc. (#23050) 2017-02-13 11:00:12 +01:00
Ryan Ernst c91848e6a7 Docs: Consoleify cluster and indices settings docs (#23030)
relates #23001
2017-02-10 14:57:43 -08:00
Tanguy Leroux e2e5937455 Use `typed_keys` parameter to prefix suggester names by type in search responses (#23080)
This pull request reuses the typed_keys parameter added in #22965, but this time it applies it to suggesters. When set to true, the suggester names in the search response will be prefixed with a prefix that reflects their type.
2017-02-10 10:53:38 +01:00
Tanguy Leroux 63ea6f7168 [Docs] Remove unnecessary // TEST[continued] in search-template doc
It has been explained in e39b96f257
2017-02-10 10:08:24 +01:00
Nik Everett e39b96f257 Fix confusing section in docs/README
It looked like it was telling you to add both `// TEST[continued]`
and `// TESTRESPONSE` to the same snippet which doesn't do anything.
Instead it was trying to say that you can have many snippets included
into the same test by doing:

```
request1
```
// CONSOLE

```
response1
```
// TESTRESPONSE

``
request2
```
// CONSOLE
// TEST[continued]

```
response2
```
// TESTRESPONSE
2017-02-09 16:17:14 -05:00
Clinton Gormley d43417ef47 Docs: Deleted redundant word in scripting 2017-02-09 22:02:42 +01:00
Jonathan D Strootman cb35b3785a Adding `ansible-elasticsearch` to list of CM tools (#23058) 2017-02-09 21:14:30 +01:00
Clinton Gormley 78d3028bb7 Fixed bad asciidoc in delete-by-query 2017-02-09 20:14:56 +01:00
Jim Ferenczi 94087b3274 Removes ExpandCollapseSearchResponseListener, search response listeners and blocking calls
This changes removes the SearchResponseListener that was used by the ExpandCollapseSearchResponseListener to expand collapsed hits.
The removal of SearchResponseListener is not a breaking change because it was never released.
This change also replace the blocking call in ExpandCollapseSearchResponseListener by a single asynchronous multi search request. The parallelism of the expand request can be set via CollapseBuilder#max_concurrent_group_searches

Closes #23048
2017-02-09 18:06:10 +01:00
Tanguy Leroux 3553522328 Add parameter to prefix aggs name with type in search responses (#22965)
This pull request adds a new parameter to the REST Search API named `typed_keys`. When set to true, the aggregation names in the search response will be prefixed with a prefix that reflects the internal type of the aggregation.

Here is a simple example:
```
GET /_search?typed_keys
{
    "aggs": {
        "tweets_per_user": {
            "terms": {
                "field": "user"
            }
        }
    },
    "size": 0
}
```

And the response:

```
{
    "aggs": {
        "sterms:tweets_per_user": {
            ...
        }
    }
}
```

This parameter is intended to make life easier for REST clients that could parse back the prefix and could detect the type of the aggregation to parse. It could also be implemented for suggesters.
2017-02-09 11:19:04 +01:00
Tanguy Leroux 832952cb29 [Docs] Fix consoleify search-template.asciidoc
It does not reproduce well, hopefully this will fix the failure on DELETE _search/template/<templatename>.
2017-02-08 21:23:38 +01:00
Igor Motov 1fc4fa5729 Docs: CONSOLEify native script docs
Relates #23001
2017-02-08 13:30:39 -05:00
Jay Modi 7f3769c745 Remove ldjson support and document ndjson for bulk/msearch (#23049)
This commit removes support for the `application/x-ldjson` Content-Type header as this was only used in the first draft
of the spec and had very little uptake. Additionally, the docs for bulk and msearch have been updated to specifically
call out ndjson and mention that the newline character may be preceded by a carriage return.

Finally, the bulk request handling of the carriage return has been improved to remove this character from the source.

Closes #23025
2017-02-08 11:55:50 -05:00
Clinton Gormley 40f40d7676 Docs: Fix termvectors by removing example blocks with embedded CONSOLE tests 2017-02-08 17:12:40 +01:00
Tanguy Leroux 477d1aa8bf [Docs] Consoleify multi-search and search-template docs (#23047)
Relates #23001
2017-02-08 17:05:22 +01:00
Christoph Büscher e177d2ca40 Docs: CONSOLEify termvectors.asciidoc (#23046) 2017-02-08 16:06:11 +01:00
Daniel Mitterdorfer 8e4b89cdbe SENSE -> CONSOLE 2017-02-08 11:49:15 +01:00
Daniel Mitterdorfer 88c5627a1b CONSOLEify multi-termvectors docs
Relates #23001
2017-02-08 11:47:42 +01:00
Yannick Welsch 9154686623 Remove legacy primary shard allocation mode based on versions (#23016)
Elasticsearch v5.0.0 uses allocation IDs to safely allocate primary shards whereas prior versions of ES used a version-based mode instead. Elasticsearch v5 still has support for version-based primary shard allocation as it needs to be able to load 2.x shards. ES v6 can drop the legacy support.
2017-02-08 10:00:55 +01:00
Lee Hinman b3c27a7fdd Disallow include_in_all for 6.0+ indices
Since `_all` is now deprecated and cannot be set for new indices, we should also
disallow any field that has the `include_in_all` parameter set.

Resolves #22923
2017-02-07 19:31:51 -07:00
Tim Brooks ad4bfa2307 Docs: CONSOLEify transport docs (#23027)
This is related to #23001.
2017-02-07 20:06:28 -06:00
Jordan Robinson 693b0017af Small typo fix in Windows service documentation
This commit removes a duplicate definite article in the Windows service
documentation.

Relates #23028
2017-02-07 17:25:46 -05:00
Nik Everett 0e98c9107a Docs: CONSOLEify some more docs
These need to be CONSOLEified *now* because we're starting to
require Content-Type headers and they didn't have any.

* cluster/reroute: Marked as CONSOLE but skipped because the docs
build runs with a single node.
* docs/bulk: Marked as NOTCONSOLE because the snippets describe
either examples or `curl` commands. Fixed the `curl` command to
include the `Content-Type` header.
* query-dsl/terms-query: Marked as CONSOLE.
* search/request/rescore: Marked as CONSOLE. Fixed deprecated
syntax.

Relates #23001
Relates #18160
2017-02-07 16:49:01 -05:00
Nik Everett 0c011cb290 Docs: CONSOLEify histogram aggregation docs
This adds the `COPY AS CURL` and `VIEW IN CONSOLE` links to the docs
and causes the snippets to be tested during Elasticsearch's build.

Relates to #18160
2017-02-07 16:09:32 -05:00
Nik Everett 245aa0404a Docs: CONSOLEify sum aggregation docs
This adds the `COPY AS CURL` and `VIEW IN CONSOLE` buttons to the
docs and makes the build execute the snippets as part of `docs:check`.

Relates to #18160
2017-02-07 14:18:54 -05:00
Jim Ferenczi bbf62e3472 CONSOLify indices/analyze.asciidoc and search/field-stats.asciidoc
Relates #23001
2017-02-07 20:15:09 +01:00
Nik Everett 274ee30d34 Docs: CONSOLEify the avg aggregation docs
This creates the `COPY AS CURL` and `VIEW IN CONSOLE` buttons and
makes the build test the examples.

Relates to #18160
2017-02-07 13:48:27 -05:00
Nik Everett a2ed676862 Docs: Explain painless's method dispatch (#23021)
Painless uses Ruby-like method dispatch (reciever type, method name,
and arity) rather than Java-like (reciever type, method name, and
argument compile time types) or Groovy-like method dispatch (receiver
type, method name, and argument run time types). We do this for
mostly good reasons but we never documented it.

Relates to #22720
2017-02-07 12:09:22 -05:00
Clinton Gormley f5e7c25e24 Update normalizers.asciidoc
analyzers -> normalizers
2017-02-07 12:09:39 +01:00
Simon Willnauer dc659feeb4 Add a setting to disable remote cluster connections on a node (#23005)
Today either all nodes in the cluster connect to remote clusters of only nodes
that have remote clusters configured in their node config. To allow global remote
cluster configuration but restrict connections to a set of nodes in the cluster
this change adds a new setting `search.remote.connect` (defaults to `true`) to allow
to disable remote cluster connections on a per node basis.
2017-02-07 09:59:24 +01:00
Nik Everett 0d6e622242 Make dates be ReadableDateTimes in scripts (#22948)
Instead of longs. If you want millis since epoch you can call doc.date_field.value.millis.

Relates to #22875
2017-02-06 16:44:56 -05:00
Nicholas Knize bc884c1e7b [Docs] Remove ignore_malformed from Geo Query DSL docs
This commit removes the ignore_malformed parameter from the Geo Query DSL documentation.
2017-02-06 14:27:15 -06:00
javanna b9cf6333bd [TEST] fix typo in cross cluster search docs 2017-02-05 15:56:45 +01:00
Clinton Gormley bbaaf1af45 Add a preface title 2017-02-04 21:23:41 +01:00
Clinton Gormley a8f7938128 Defguide link should not use {branch} 2017-02-04 21:21:46 +01:00
Clinton Gormley e181a020a9 Replaced absolute URLs in docs with attributes 2017-02-04 12:05:03 +01:00
Clinton Gormley 0fea2a207e Use Versions.asciidoc for groovy docs too 2017-02-04 11:42:45 +01:00
Clinton Gormley c1be26f2e1 Centralised doc versions in docs/Versions.asciidoc 2017-02-04 11:16:19 +01:00
Chris Earle d2428b1e0d Remove note about future ES version for content-type on master 2017-02-03 20:10:04 -05:00
Chris Earle dabc51f988 [DOCS] Low Level REST Client should emphasize ContentType (#22940)
This adds a callout to note that the Low Level REST Client example sets the `ContentType` explicitly and that users should do the same.
2017-02-03 19:49:35 -05:00
Lee Hinman 39e7c30912 Change certain replica failures not to fail the replica shard
This changes the way that replica failures are handled such that not all
failures will cause the replica shard to be failed or marked as stale.

In some cases such as refresh operations, or global checkpoint syncs, it is
"okay" for the operation to fail without the shard being failed (because no data
is out of sync). In these cases, instead of failing the shard we should simply
fail the operation, and, in the event it is a user-facing operation, return a
5xx response code including the shard-specific failures.

This was accomplished by having two forms of the `Replicas` proxy, one that is
for non-write operations that does not fail the shard, and one that is for write
operations that will fail the shard when an operation fails.

Relates to #10708
2017-02-03 14:39:46 -07:00
Nik Everett 70e3cce904 Fix name of `enable_position_increments` (#22895)
It was accidentally renamed `enabled_position_increment` in the cleanups
for 5.0. This adds `enable_position_increment` as a deprecated alias
so it will continue to work.
2017-02-03 16:28:27 -05:00
Nicholas Knize b1a6b227e1 Remove deprecated geo query parameters, and GeoPointDistanceRangeQuery
This commit removes the following queries and parameters (which were deprecated in 5.0):

* GeoPointDistanceRangeQuery
* coerce, and ignore_malformed for GeoBoundingBoxQuery, GeoDistanceQuery, GeoPolygonQuery, and GeoDistanceSort
2017-02-03 10:08:00 -06:00
Nicholas Knize f1e1975882 [DOCS] Add sloppy_arc and factor removal to 6.0 migration docs 2017-02-03 09:49:12 -06:00
AlexNodex fb8bdbc57a Update typo in date (#22955)
your example has yyy and it should be yyyy
2017-02-03 13:16:17 +01:00
Clinton Gormley 8ace37e214 Fix asciidoc in stored fields 2017-02-03 10:18:01 +01:00
Jim Ferenczi 4876448e39 Consilify get-field-mapping docs (#22936)
This change also removes the reference to the difference bewteen full name and index name.
They are always the same since 2.x and `name` does not refer anymore to `author.name` automatically.
A simple pattern must be used instead.
Remove redundant code that checks the field name twice.
2017-02-03 10:04:31 +01:00
Jun Ohtani 7ea457955d Merge pull request #22879 from johtani/fix_documentation_error_in_date_histogram
[Doc]Not support "M" time unit in offset param
2017-02-03 16:40:08 +09:00
Jay Modi 7520a107be Optionally require a valid content type for all rest requests with content (#22691)
This change adds a strict mode for xcontent parsing on the rest layer. The strict mode will be off by default for 5.x and in a separate commit will be enabled by default for 6.0. The strict mode, which can be enabled by setting `http.content_type.required: true` in 5.x, will require that all incoming rest requests have a valid and supported content type header before the request is dispatched. In the non-strict mode, the Content-Type header will be inspected and if it is not present or not valid, we will continue with auto detection of content like we have done previously.

The content type header is parsed to the matching XContentType value with the only exception being for plain text requests. This value is then passed on with the content bytes so that we can reduce the number of places where we need to auto-detect the content type.

As part of this, many transport requests and builders were updated to provide methods that
accepted the XContentType along with the bytes and the methods that would rely on auto-detection have been deprecated.

In the non-strict mode, deprecation warnings are issued whenever a request with body doesn't provide the Content-Type header.

See #19388
2017-02-02 14:07:13 -05:00
Nicholas Knize b41d5747f0 Reduce GeoDistance insanity
GeoDistance query, sort, and scripts make use of a crazy GeoDistance enum for handling 4 different ways of computing geo distance: SLOPPY_ARC, ARC, FACTOR, and PLANE. Only two of these are necessary: ARC, PLANE. This commit removes SLOPPY_ARC, and FACTOR and cleans up the way Geo distance is computed.
2017-02-02 12:39:42 -06:00
Nik Everett dacc150934 Expose multi-valued dates to scripts and document painless's date functions (#22875)
Implemented by wrapping an array of reused `ModuleDateTime`s that
we grow when needed. The `ModuleDateTime`s are reused when we
move to the next document.

Also improves the error message returned when attempting to modify
the `ScriptdocValues`, removes a couple of allocations, and documents
that the date functions are available in Painless.

Relates to #22162
2017-02-01 21:57:07 -05:00
Rob Moore 40e1c136de Update search.asciidoc (#22865) 2017-02-01 09:24:19 +01:00
Jack Conradson 3d2626c4c6 Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.

The new behavior is the following:

When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.

Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].

When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.

Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.

When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 13:27:02 -08:00
Jim Ferenczi f6d38d480a Integrate UnifiedHighlighter (#21621)
* Integrate UnifiedHighlighter

This change integrates the Lucene highlighter called "unified" in the list of supported highlighters for ES.
This highlighter can extract offsets from either postings, term vectors, or via re-analyzing text.
The best strategy is picked automatically at query time and depends on the field and the query to highlight.
2017-01-31 19:06:03 +01:00
Nik Everett f90051e6e0 Docs: Add a note about `<` and `>` in query_string
`<` and `>` can't be escaped at all in `query_string`. If we're not
going to fix that we should at least document it.

Relates to #21703
2017-01-31 12:23:18 -05:00
alamzeeshan a1cc683cff Updated document as per code change. (#22878)
Updated document as per this change : https://github.com/elastic/elasticsearch/pull/15235
2017-01-31 13:36:09 +01:00
Shubham Aggarwal d80e05f4d2 Minor fix in Create Index desc (#22877) 2017-01-31 12:53:15 +01:00
Shubham Aggarwal e07e4cc4dd Fix incorrect heading for Whitespace Tokenizer (#22883) 2017-01-31 12:51:37 +01:00
markharwood 9e8e556b08 Build fix for broken docs build 2017-01-31 10:27:06 +00:00
markharwood c0d525b108 [DOCS] [TEST] enhancement - added CONSOLE scripts for sampler aggs (#22869)
Added missing CONSOLE scripts to documentation for sampler and diversified_sampler aggs.
Includes new StackOverflow index setup in build.gradle

Closes #22746

* Formatting tweaks
2017-01-31 09:45:25 +00:00
Jun Ohtani 94933f9d19 [Doc]Not support "M" time unit in offset param 2017-01-31 18:23:38 +09:00
David Pilato 858333246d Merge branch 'pr/remove-azure-container-auto-creation'
# Conflicts:
#	docs/reference/migration/migrate_6_0/plugins.asciidoc
2017-01-31 09:05:43 +01:00
Ryan Ernst 003805e875 Docs: Add setup section for the keystore tool and secure settings (#22838) 2017-01-30 14:56:45 -08:00
Ryan Ernst cf7747c338 S3 Repository: Remove region setting (#22853)
This change removes the ability to set region for s3 repositories.
Endpoint should be used instead if a custom s3 location needs to be
used.

closes #22758
2017-01-30 14:34:59 -08:00
David Pilato 4d1e291933 Merge remote-tracking branch 'origin/master' 2017-01-30 17:53:13 +01:00
David Pilato 72e15b1c7c Merge branch 'pr/fix-hdfs-settings' 2017-01-30 17:52:34 +01:00
Jim Ferenczi e7e871acdd Fix link to keyword and numeric type 2017-01-30 13:57:28 +01:00
Mathieu Berube e0b8e45cc5 Fix typo - mergins to margins (#22839) 2017-01-30 13:52:32 +01:00
Clinton Gormley b5a0e59af4 Fixed bad asciidoc 2017-01-30 11:56:40 +01:00
Clinton Gormley 938f5194ef Include field-collapsing docs in request-body search 2017-01-30 11:47:12 +01:00
David Pilato 1898dc2554 Remove auto creation of container for azure repository
Follow up of #22857 where we deprecate automatic creation of azure containers.

BTW I found that the `AzureSnapshotRestoreServiceIntegTests` does not bring any value because it runs basically a Snapshot/Restore operation on local files which we already test in core.

So instead of trying to fix it to make it pass with this PR, I simply removed it.
2017-01-30 11:47:08 +01:00
Adrien Grand dc62255ddd Document upcoming scoring changes. (#22806) 2017-01-30 11:08:49 +01:00
Ryan Ernst fe4043c8ff S3 Repository: Remove bucket auto create (#22846)
closes #22761
2017-01-28 11:13:21 -08:00
Ryan Ernst c921bebc4a S3 Repository: Remove env var and sysprop credentials support (#22842)
These are deprecated in 5.x. This commit removes support for them in 6.0.
2017-01-27 13:43:16 -08:00
Clinton Gormley 19ce039d2d Update type-field.asciidoc
Wildcard type names are not supported
2017-01-27 17:50:28 +01:00
David Pilato 59b34ba71c Merge branch 'pr/22566-ec2-meta-host' 2017-01-26 17:37:53 +01:00
David Pilato 17930930a7 Update after review 2017-01-26 17:10:37 +01:00
David Pilato 3804bfcc60 Read ec2 discovery address from aws instance tags
This PR adds a new option for `host_type`: `tag:TAGNAME` where `TAGNAME` is the tag field you defined for your ec2 instance.

For example if you defined a tag `my-elasticsearch-host` in ec2 and set it to `myhostname1.mydomain.com`, then
setting `host_type: tag:my-elasticsearch-host` will tell Discovery Ec2 plugin to read the host name from the
`my-elasticsearch-host` tag. In this case, it will be resolved to `myhostname1.mydomain.com`.

Closes #22566.
2017-01-26 17:10:37 +01:00
Nik Everett a383bc1be0 Add remaining generated painless API
This was generated by the generated added in
8a2d424d68 but not committed in that
change for review purposes.
2017-01-26 10:43:22 -05:00
Nik Everett 8a2d424d68 Generate reference links for painless API (#22775)
Adds "Appending B. Painless API Reference", a reference of all classes
and methods available from Painless. Removes links to java packages
because they contain methods that we don't expose and don't contain
methods that we do expose (the ones in Augmentation). Instead this
generates a list of every class and every exposed method using the same
type information available to the
interpreter/compiler/whatever-we-call-it. From there you can jump to
the relevant docs.

Right now you build all the asciidoc files by running
```
gradle generatePainlessApi
```

These files are expected to be committed because we build the docs
without running `gradle`.

Also changes the output of `Debug.explain` so that it is easy to
search for the class in the generated reference documentation.

You can also run it in an IDE safely if you pass the path to the
directory in which to generate the docs as the first parameter. It'll
blow away the entire directory an recreate it from scratch so be careful.

And then you can build the docs by running something like:
```
../docs/build_docs.pl --out ../built_docs/ --doc docs/reference/index.asciidoc --open
```

That is, if you have checked out https://github.com/elastic/docs in
`../docs`. Wait a minute or two and your browser will pop open in with
all of Elasticsearch's reference documentation. If you go to
`http://localhost:8000/painless-api-reference.html` you can see this
list. Or you can get there by following the links to `Modules` and
`Scripting` and `Painless` and then clicking the link in the paragraphs
below titled `Appendix B. Painless API Reference`.

I like having these in asciidoc because we can deep link to them from the
rest of the guide with constructs like
`<<painless-api-reference-Object-hashCode-0>>` and
`<<painless-api-reference->>` and we get link checking. Then the only
brittle link maintenance bit is the link generation for javadoc. Which
sucks. But I think it is important that we link to the methods directly
so they are easy to find.

Relates to #22720
2017-01-26 10:39:19 -05:00
Clinton Gormley f5a0d18c4c Docs: Cluster allocation explain should be on one page 2017-01-26 11:38:19 +01:00
David Pilato 7d68779655 repositories.hdfs.path can not be set
Reported at: https://discuss.elastic.co/t/combine-elasticsearch-5-1-1-and-repository-hdfs/69659

If you define as described [in our docs](https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-hdfs-config.html) the following `elasticsearch.yml` settings:

```yml
repositories:
  hdfs:
    uri: "hdfs://es-master:9000/" # optional - Hadoop file-system URI
    path: "some/path" # required - path with the file-system where data is stored/loaded
```

It fails at startup because we don't register the global setting `repositories.hdfs.path` in `HdfsPlugin`.

This PR removes that from our docs so people must provide those settings only when registering the repository with:

```
PUT _snapshot/my_hdfs_repository
{
  "type": "hdfs",
  "settings": {
    "uri": "hdfs://namenode:8020/",
    "path": "elasticsearch/respositories/my_hdfs_repository",
    "conf.dfs.client.read.shortcircuit": "true"
  }
}
```

Based on issue #22800.

Closes #22301
2017-01-26 09:22:43 +01:00
Nik Everett d704a880e7 Add tests for top_hits aggregation (#22754)
Add unit tests for `TopHitsAggregator` and convert some snippets in
docs for `top_hits` aggregation to `// CONSOLE`.

Relates to #22278
Relates to #18160
2017-01-25 16:15:50 -05:00
Anil Bawa-Cavia 86b3ff229c Add new ruby search library to community clients doc (#22765)
* Add new Ruby search lib from Artsy.

* update ruby library reference following rename
2017-01-25 11:49:39 +01:00
Chris Earle f0f75b187a Support Preemptive Authentication with RestClient (#21336)
This adds the necessary `AuthCache` needed to support preemptive authorization. By adding every host to the cache, the automatically added `RequestAuthCache` interceptor will add credentials on the first pass rather than waiting to do it after _each_ anonymous request is rejected (thus always sending everything twice when basic auth is required).
2017-01-24 11:34:05 -05:00