Commit Graph

4128 Commits

Author SHA1 Message Date
Sho Yoshida db13b4c2bc Added Smalltalk client (#21361)
> Pharo Smalltalk
> Pharo emerged as a fork of Squeak Smalltalk. It focuses on modern software engineering and development techniques.

project:
- http://pharo.org/ Pharo Smalltalk
2016-11-28 11:05:17 +01:00
Carney Wu 2c0db3909f include not work in 5.x anymore (#21815)
include not work in 5.x anymore use includes instead
2016-11-28 11:02:59 +01:00
Adrien Grand 4c46ffcecf Document that min/max operate on the double representation of the data.
Relates #9545
2016-11-28 10:34:43 +01:00
Clinton Gormley 5555e85619 Document that the PUT mapping API with the _default_ type overwrites instead of merging
Closes #8215
2016-11-26 12:43:56 +01:00
Clinton Gormley 5ae6845d4d Update percolate-query.asciidoc
Add missing callout to percolate query
2016-11-26 12:35:33 +01:00
Ali Beyad 51bfecc7cb [DOCS] fixes word usage in allocation awareness docs 2016-11-25 11:47:40 -05:00
Achraf d81a928b1f Correction of the names of numirals (#21531)
What was called Arabic numerals is actually Hindu - Eastern Arabic notation. And the Latin numerals you refer to is the Arabic numbers.
2016-11-25 14:30:49 +01:00
Adrin Jalali 953928b2c5 typo fix (it self -> itself) (#21781)
* typo fix.

* apply "stored field value"

* replaced "whereas" with "on the contrary"
2016-11-24 17:11:43 +01:00
Adrin Jalali 3bb9317ca2 clarify ambiguous sentence. (#21734) 2016-11-24 16:47:14 +01:00
Clinton Gormley 9b8e3c6878 Add docs for the batch mode of plugin installation (#21700)
* Add docs for the batch mode of plugin installation

Closes https://github.com/elastic/elasticsearch/issues/21490

* Updated to clarify that all permissions are granted automatically in batch mode
2016-11-24 16:41:05 +01:00
markharwood aa60e5cc07 Aggregations - support for partitioning set of terms used in aggregations so that multiple requests can be done without trying to compute everything in one request.
Closes #21487
2016-11-24 15:10:46 +00:00
Adrin Jalali 0871073f9b clarification on geo distance sorting (#21779)
* clarification on geo distance sorting

* applying the suggested change
2016-11-24 16:06:10 +01:00
Nik Everett 434fa4bd26 Docs and tests for painless lack of boxing for ?: and ?. (#21756)
NOTE: The result of `?.` and `?:` can't be assigned to primitives. So
`int[] someArray = null; int l = someArray?.length` and
`int s = params.size ?: 100` don't work. Do
`def someArray = null; def l = someArray?.length` and
`def s = params.size ?: 100` instead.

Relates to #21748
2016-11-23 14:33:32 -05:00
Ahmad Ragab 49e1ca249a Update usage.asciidoc
Reordered parameters for last async call based on implementation

Backport of #21360 in master branch (6.0).

Closes #21360.
2016-11-23 17:34:45 +01:00
David Pilato bfb2116aa6 Add missing change for sniffer page
Related to #21737
2016-11-23 17:13:01 +01:00
David Pilato 5bc92643ee Merge branch 'doc/gradle-rest-client' 2016-11-23 17:03:58 +01:00
David Pilato 77e8e1d6f2 Update layout 2016-11-23 17:00:41 +01:00
Jason Tedor 32e6fcf256 Fix markup in Zen discovery docs
This commit fixes a markup issue in the Zen discovery docs where a link
and its referring text were not on the same line tripping the renderer.
2016-11-23 10:02:13 -05:00
Nik Everett 76a804e589 Revert "it's a `noop` operation, not a `none` operation. (#21736)"
This reverts commit 7f77214ced.

`none` is indeed how you trigger the `noop` operation in the `_update`
API.
2016-11-23 09:26:53 -05:00
Ryan Ernst 6940b2b8c7 Remove groovy scripting language (#21607)
* Scripting: Remove groovy scripting language

Groovy was deprecated in 5.0. This change removes it, along with the
legacy default language infrastructure in scripting.
2016-11-22 19:24:12 -08:00
Nik Everett 1791623700 Document `error_trace`
The `error_trace` parameter turns on the `stack_trace` field
in errors which returns stack traces.

Removes documentation for `camelCase` because it hasn't worked
in a while....

Documents the internal parameters used to render stack traces as
internal only.

Closes #21708
2016-11-22 19:16:07 -05:00
Luca Cavanna db5a72774b Add indices and filter information to search shards api output (#21738)
Add indices and filter information to search shards api output

The search shards api returns info about which shards are going to be hit by executing a search with provided parameters: indices, routing, preference. Indices can also be aliases, which can also hold filters. The output includes an array of shards and a summary of all the nodes the shards are allocated on. This commit adds a new indices section to the search shards output that includes one entry per index, where each index can be associated with an optional filter in case the index was hit through a filtered alias.

This is relevant since we have moved parsing of alias filters to the coordinating node.

Relates to #20916
2016-11-22 23:00:25 +01:00
Jason Tedor 9dc65037bc Lazy resolve unicast hosts
Today we eagerly resolve unicast hosts. This means that if DNS changes,
we will never find the host at the new address. Moreover, a single host
failng to resolve causes startup to abort. This commit introduces lazy
resolution of unicast hosts. If a DNS entry changes, there is an
opportunity for the host to be discovered. Note that under the Java
security manager, there is a default positive cache of infinity for
resolved hosts; this means that if a user does want to operate in an
environment where DNS can change, they must adjust
networkaddress.cache.ttl in their security policy. And if a host fails
to resolve, we warn log the hostname but continue pinging other
configured hosts.

When doing DNS resolutions for unicast hostnames, we wait until the DNS
lookups timeout. This appears to be forty-five seconds on modern JVMs,
and it is not configurable. If we do these serially, the cluster can be
blocked during ping for a lengthy period of time. This commit introduces
doing the DNS lookups in parallel, and adds a user-configurable timeout
for these lookups.

Relates #21630
2016-11-22 14:17:04 -05:00
Nik Everett 83ea1be185 Remove a bad callout from the new debugging docs
It was a leftover and doesn't break the usual build but breaks the docs
build.
2016-11-22 13:13:16 -05:00
Nik Everett 457c2d8fb0 Add Debug.explain to painless
You can use `Debug.explain(someObject)` in painless to throw an
`Error` that can't be caught by painless code and contains an
object's class. This is useful because painless's sandbox doesn't
allow you to call `someObject.getClass()`.

Closes #20263
2016-11-22 12:46:02 -05:00
Luca Cavanna db8b2dceea Remove ignored type parameter in search_shards api (#21688)
The `type` parameter has always been accepted by the search_shards api, probably to make the api and its urls the same as search. Truth is that the type never had any effect, it's been ignored from day one while accepting it may make users think that we actually do something with it.

This commit removes support for the type parameter from the REST layer and the Java API. Backwards compatibility is maintained on the transport layer though.

The new added serialization test also uncovered a bug in the java API where the `ClusterSearchShardsRequest` could be created with no arguments, but the indices were required to be not null otherwise the request couldn't be serialized as `writeTo` would throw NPE. Fixed by setting a default value (empty array) for indices.
2016-11-22 17:22:33 +01:00
Adrin Jalali 7f77214ced it's a `noop` operation, not a `none` operation. (#21736)
It works I guess cause it's ignored as an invalid operation.
2016-11-22 10:41:24 -05:00
Adrin Jalali 982f7cb067 fixing an ambiguous sentence. (#21729) 2016-11-22 16:35:58 +01:00
Adrin Jalali 4bb7091f64 `force` is deprecated be mentioned at the end. (#21731) 2016-11-22 16:35:57 +01:00
David Pilato 5e1780061c Add Gradle coordinates for the REST client 2016-11-22 16:34:23 +01:00
Clinton Gormley 284dedfb5f Update get-settings.asciidoc
Fix syntax for filtering settings by name

Closes https://github.com/elastic/elasticsearch/issues/21713
2016-11-22 15:10:09 +01:00
Erel2 b538bbd2c8 Update index.asciidoc (#21727) 2016-11-22 14:17:57 +01:00
Lee Hinman 11da09e9bc Allow overriding all-field leniency when `lenient` option is specified
As part of #20925 and #21341 we added an "all-fields" mode to the
`query_string` and `simple_query_string`. This would expand the query to
all fields and automatically set `lenient` to true.

However, we should still allow a user to override the `lenient` flag to
whichever value they desire, should they add it in the request. This
commit does that.
2016-11-21 21:32:25 -07:00
Nik Everett c79371fd5b Remove lang-python and lang-javascript (#20734)
They were deprecated in 5.0. We are concentrating on making
Painless awesome rather than supporting every language possible.

Closes #20698
2016-11-21 22:13:25 -05:00
Chris Earle 3f2d22bc61 [DOCS] Show EC2's auto attribute (#21474)
* [DOCS] Show EC2's auto attribute

This documents the `aws_availability_zone` node attribute as part of the `discovery-ec2` plugin. Also fixes outdated usage of "cloud aws".
2016-11-21 11:04:53 -05:00
Adrien Grand c7fc688096 Add information about the removal of store throttling to the migration guide.
Relates to #21573
2016-11-21 15:03:07 +01:00
Adrien Grand 52408fc389 Add a recommendation against large documents to the docs. (#21652) 2016-11-21 15:01:36 +01:00
Trey Tacon 3ef7f0dec6 Fixing indentation in geospatial querying example. (#21682)
Specifically the example which shows providing an array of an array of values.
2016-11-21 13:09:21 +01:00
Chris Fritz 546fa92d61 Fix typo in filters aggregation docs (#21690) 2016-11-21 12:52:45 +01:00
Clinton Gormley 496afbbda5 Docs: Added offline install link for discovery-file plugin
Closes #21696
2016-11-21 11:47:48 +01:00
Ludovic Dubost 366241508e Adding XWiki Elasticsearch Macro (#21505)
Added link to Elasticsearch Macro with description
2016-11-19 16:51:05 +01:00
Jeffery Bradberry 9322c4fe62 The alias is switched to point to the new index (#21512)
If the index satisfies the specified conditions then a new index is created and the alias is switched to point to the new index.
2016-11-19 16:47:39 +01:00
Shubham Aggarwal dce51e2062 Update getting-started.asciidoc (#21677) 2016-11-19 12:45:28 +01:00
Nik Everett ae468441dc Implement the ?: operator in painless (#21506)
Implements a null coalescing operator in painless that looks like `?:`. This form was chosen to emulate Groovy's `?:` operator. It is different in that it only coalesces null values, instead of Groovy's `?:` operator which coalesces all falsy values. I believe that makes it the same as Kotlin's `?:` operator. In other languages this operator looks like `??` (C#) and `COALESCE` (SQL) and `:-` (bash).

This operator is lazy, meaning the right hand side is only evaluated at all if the left hand side is null.
2016-11-18 13:54:26 -05:00
Sakthipriyan Vairamani 52d4cd504a minor grammatical improvements (#21640) 2016-11-18 08:55:04 -07:00
ismael-hasan 7906db83d5 Update BulkProcessor size in the example
By default, it is recommended to start bulk with a size of 10-15MB, and increase it gradually to get the right size for the environment. The example shows originally 1GB, which can lead to some users to just copy-paste the code snippet and start with excessively big sizes.

Backport of #21664 in master branch.
2016-11-18 16:52:46 +01:00
Luca Wintergerst 7a533577b8 Merge pull request #21651 from elastic/suggester-docs-example-fix
fix two errors in suggester docs
2016-11-18 13:13:13 +01:00
Jason Tedor 5ca60fa78b Remove stale mention of minimum master nodes check
This commit removes a stale mention of the minimum master nodes boostrap
check from the docs.

Relates #21653
2016-11-18 06:56:23 -05:00
Luca Wintergerst 277f4b8d24 fix two errors in suggester docs
The first changed referred to an example of the 2.4 documentation. I removed the no longer relevant parts. We should consider adding a little more here. 

The second change was just then->than in the suggest_mode popular section
2016-11-18 12:05:49 +01:00
Ryan Ernst 914664d89a Fix leftover reference to ScriptModule in native script docs 2016-11-17 08:22:15 -08:00
Nik Everett df2a5e7691 Mark snippet in docker docs NOTCONSOLE
That keeps us from failing the build because it is "unconverted".

Relates to #21497
2016-11-17 10:07:42 -05:00
Dimitrios Liappis 6c9ea0877d [docs] Document Docker installation method
Initial commit of the Docker installation method for Elasticsearch.

Relates #21497
2016-11-17 15:06:32 +02:00
David Pilato 475a7ca84f Add documentation for lenient in multimatch
`lenient` option is documented for `match` query but not for `multi_match` query.
2016-11-17 08:35:20 +01:00
David Pilato c946094d5b Add documentation for lenient in multimatch
`lenient` option is documented for `match` query but not for `multi_match` query.
2016-11-16 16:15:28 +01:00
Jason Tedor d06a8903fd Merge branch 'master' into feature/seq_no
* master: (22 commits)
  Add proper toString() method to UpdateTask (#21582)
  Fix `InternalEngine#isThrottled` to not always return `false`. (#21592)
  add `ignore_missing` option to SplitProcessor (#20982)
  fix trace_match behavior for when there is only one grok pattern (#21413)
  Remove dead code from GetResponse.java
  Fixes date range query using epoch with timezone (#21542)
  Do not cache term queries. (#21566)
  Updated dynamic mapper section
  Docs: Clarify date_histogram bucket sizes for DST time zones
  Handle release of 5.0.1
  Fix skip reason for stats API parameters test
  Reduce skip version for stats API parameter tests
  Strict level parsing for indices stats
  Remove cluster update task when task times out (#21578)
  [DOCS] Mention "all-fields" mode doesn't search across nested documents
  InternalTestCluster: when restarting a node we should validate the cluster is formed via the node we just restarted
  Fixed bad asciidoc in boolean mapping docs
  Fixed bad asciidoc ID in node stats
  Be strict when parsing values searching for booleans (#21555)
  Fix time zone rounding edge case for DST overlaps
  ...
2016-11-16 09:10:35 -05:00
Tal Levy 6796464f16 add `ignore_missing` option to SplitProcessor (#20982)
Closes #20840.
2016-11-16 15:46:09 +02:00
Tal Levy 04b712bdc5 fix trace_match behavior for when there is only one grok pattern (#21413)
There is an issue in the Grok Processor, where trace_match: true does not inject the _ingest._grok_match_index into the ingest-document when there is just one pattern provided. This is due to an optimization in the regex construction. This commit adds a check for when this is the case, and injects a static index value of "0", since there is only one pattern matched (at the first index into the patterns).

To make this clearer, more documentation was added to the grok-processor docs.

Fixes #21371.
2016-11-16 15:41:54 +02:00
David Pilato 2842e2752a Updated dynamic mapper section
Backport of #21574 in master (6.0)
2016-11-16 09:52:08 +01:00
Christoph Büscher 4ccd8e79c1 Docs: Clarify date_histogram bucket sizes for DST time zones
Added a warning note that clarifies bucket sizes diverging from the intended
`interval` size when using a time zone that has DST changes.

Closes #18805
2016-11-16 09:40:07 +01:00
Lee Hinman 17a2fffc9b [DOCS] Mention "all-fields" mode doesn't search across nested documents 2016-11-15 11:02:43 -07:00
Clinton Gormley a4e88bb64a Fixed bad asciidoc in boolean mapping docs 2016-11-15 17:50:23 +00:00
Clinton Gormley cfabc95f59 Fixed bad asciidoc ID in node stats 2016-11-15 17:39:15 +00:00
Lee Hinman 96122aa518 Be strict when parsing values searching for booleans (#21555)
This changes only the query parsing behavior to be strict when searching on
boolean values. We continue to accept the variety of values during index time,
but searches will only be parsed using `"true"` or `"false"`.

Resolves #21545
2016-11-15 10:36:57 -07:00
Bryan Karlovitz f03723a812 Minor style change to getting-started.asciidoc (#21571)
* In line 525, change the phrase "... with as little network roundtrips as possible." to "... with as few network roundtrips as possible."
2016-11-15 12:18:49 -05:00
Jason Tedor f5ac0e5076 Remove lenient stats parsing
Today when parsing a stats request, Elasticsearch silently ignores
incorrect metrics. This commit removes lenient parsing of stats requests
for the nodes stats and indices stats APIs.

Relates #21417
2016-11-15 12:17:26 -05:00
Boaz Leskes 2c0338fa87 Merge remote-tracking branch 'upstream/master' into feature/seq_no 2016-11-15 17:09:08 +00:00
Lee Hinman 568a7ea5f1 Fix incorrect instructions for disabling deprecation logging (#21569)
We log deprecation events at "WARN", so setting it to `info` means the events
are still logged. It must be set to `error` in order to disable the logging.
2016-11-15 09:57:26 -07:00
Nik Everett 7dcff27aea Update docs for scripted metric agg
Now that the default language is painless the examples didn't work at
all. This fixes them.

Closes #21536
2016-11-15 11:47:17 -05:00
Adrien Grand 6db683a4bd Fix recurring doc test failures with the cat API. (#21561)
This failure is due to the fact that we sort on store size, which is cached. So
it might happen that the store size that is taken into account is not the right
one, which makes the indices sorted in the wrong order. This changes the doc
example to sort on the number of docs instead.

Closes #21062
2016-11-15 16:00:44 +01:00
Boaz Leskes c9f49039d3 Merge remote-tracking branch 'upstream/master' into feature/seq_no 2016-11-15 10:14:47 +00:00
Sumit Gupta e53405f4f3 Update geohashgrid-aggregation.asciidoc (#21530) 2016-11-15 10:49:02 +01:00
Daniel Mitterdorfer 9d3d6c5409 [docs] clients need to add jackson-databind (#21527)
With ES 5.0 we do not include Jackson
Databind anymore with ES core. This commit
updates our docs to state that users need
to add this artifact now in their projects.
2016-11-14 10:07:07 +01:00
Clinton Gormley 30d342c87c Update significantterms-aggregation.asciidoc
Fix scripted significant terms example to use `params.` prefix for painless
2016-11-14 09:40:04 +01:00
Jason Tedor 1ea69b1a80 Merge branch 'master' into feature/seq_no
* master:
  Set vm.max_map_count on systemd package install
  [TEST] reduce the number of snapshotted shards to 1 in testSnapshotSucceedsAfterSnapshotFailure() so that we are more likely to trigger I/O exceptions on writing the control files during the finalize phase of snapshotting (with the aim of triggering an I/O failure when writing pending-index-*).
  Add documentation for Logger with Transport Client
  Enable appender exceptions in UpdateSettingsIT
  [TEST] remove AwaitsFix from testSnapshotSucceedsAfterSnapshotFailure, turns out the issue is specific to Java 9 v143
  Cleanup formatting in UpdateSettingsIT.java
  [TEST] mute the testSnapshotSucceedsAfterSnapshotFailure() test until its clear what is going wrong.
  Mark SearchQueryIT test as awaits fix
  Makes snapshot throttling test go much faster (#21485)
  Breaking changes docs for template index_patterns
  [TEST] adds randomness between atomic and non-atomic move operations in MockRepository
  Cache successful shard deletion checks (#21438)
  Task cancellation command should wait for all child nodes to receive cancellation request before returning
2016-11-11 17:03:01 -05:00
David Pilato 5336e72fbb Add documentation for Logger with Transport Client
Backport of #21477 in master branch
2016-11-11 21:23:23 +01:00
Jason Tedor 33f7cd5a16 Remove shard ID from doc write response
This commit removes the shard ID from doc write response; this was
useful for debugging but its time has passed.

Relates #21508
2016-11-11 15:18:25 -05:00
Jason Tedor 1e7c424479 Merge branch 'master' into feature/seq_no
* master:
  ShardActiveResponseHandler shouldn't hold to an entire cluster state
  Ensures cleanup of temporary index-* generational blobs during snapshotting (#21469)
  Remove (again) test uses of onModule (#21414)
  [TEST] Add assertBusy when checking for pending operation counter after tests
  Revert "Add trace logging when aquiring and releasing operation locks for replication requests"
  Allows multiple patterns to be specified for index templates (#21009)
  [TEST] fixes rebalance single shard check as it isn't guaranteed that a rebalance makes sense and the method only tests if rebalance is allowed
  Document _reindex with random_score
2016-11-11 11:25:27 -05:00
Nik Everett 7c3886769f Breaking changes docs for template index_patterns
0219a211d3 added support for templates
to have multiple patterns and renamed `template` to `index_patterns`.
This adds the breaking changes docs for that.
2016-11-11 10:13:19 -05:00
Jason Tedor d3417fb022 Merge branch 'master' into feature/seq_no
* master: (516 commits)
  Avoid angering Log4j in TransportNodesActionTests
  Add trace logging when aquiring and releasing operation locks for replication requests
  Fix handler name on message not fully read
  Remove accidental import.
  Improve log message in TransportNodesAction
  Clean up of Script.
  Update Joda Time to version 2.9.5 (#21468)
  Remove unused ClusterService dependency from SearchPhaseController (#21421)
  Remove max_local_storage_nodes from elasticsearch.yml (#21467)
  Wait for all reindex subtasks before rethrottling
  Correcting a typo-Maan to Man-in README.textile (#21466)
  Fix InternalSearchHit#hasSource to return the proper boolean value (#21441)
  Replace all index date-math examples with the URI encoded form
  Fix typos (#21456)
  Adapt ES_JVM_OPTIONS packaging test to ubuntu-1204
  Add null check in InternalSearchHit#sourceRef to prevent NPE (#21431)
  Add VirtualBox version check (#21370)
  Export ES_JVM_OPTIONS for SysV init
  Skip reindex rethrottle tests with workers
  Make forbidden APIs be quieter about classpath warnings (#21443)
  ...
2016-11-10 23:40:33 -05:00
Ryan Ernst 48bfb142b9 Remove (again) test uses of onModule (#21414)
This change was reverted after it caused random test failures. This was
due to a copy/paste error in the original PR which caused the mock
version of ClusterInfoService to be used whenever the mock *ZenPing* was
used, and the real ClusterInfoService to be used when MockZenPing was
not used.
2016-11-10 16:06:14 -08:00
Alexander Lin 0219a211d3 Allows multiple patterns to be specified for index templates (#21009)
* Allows for an array of index template patterns to be provided to an
index template, and rename the field from 'template' to 'index_pattern'.

Closes #20690
2016-11-10 18:00:30 -05:00
Nik Everett eeb6602c98 Document _reindex with random_score
You can use `_reindex` and `random_score` to extract a random
subset of an index but you have to be careful to sort by `_score`
or it won't work.

Closes #21432
2016-11-10 16:14:30 -05:00
Clinton Gormley 860efb2ad2 Replace all index date-math examples with the URI encoded form
Closes #21454
2016-11-10 12:23:19 +01:00
lslxdx e560c5be43 Fix typos (#21456)
Delete repeated words " between nodes and".
2016-11-10 11:55:04 +01:00
Lee Hinman 2674e415c8 Merge remote-tracking branch 'dakrone/sqs-all-field-mode' 2016-11-09 13:13:41 -07:00
Lee Hinman 7420fd0be3 Add "all fields" execution mode to simple_query_string query
This commit introduces a new execution mode for the
`simple_query_string` query, which is intended down the road to be a
replacement for the current _all field.

It now does auto-field-expansion and auto-leniency when the following criteria
are ALL met:

    The _all field is disabled
    No default_field has been set in the index settings
    No fields are specified in the request

Additionally, a user can force the "all-like" execution by setting the
all_fields parameter to true.

When executing in all field mode, the `simple_query_string` query will
look at all the fields in the mapping that are not metafields and can be
searched, and automatically expand the list of fields that are going to
be queried.

Relates to #20925, which is the `query_string` version of this work.
This is basically the same behavior, but for the `simple_query_string`
query.

Relates to #19784
2016-11-09 10:38:51 -07:00
Clinton Gormley 474f63ebac Added note to ICU docs about the possible need to reindex (#21426)
when upgrading from a previous major version.

Relates #21352
2016-11-09 15:20:15 +01:00
Nik Everett d03b8e4abb Implement reading from null safe dereferences
Null safe dereferences make handling null or missing values shorter.
Compare without:
```
if (ctx._source.missing != null && ctx._source.missing.foo != null) {
  ctx._source.foo_length = ctx.source.missing.foo.length()
}
```

To with:
```
Integer length = ctx._source.missing?.foo?.length();
if (length != null) {
  ctx._source.foo_length = length
}
```

Combining this with the as of yet unimplemented elvis operator allows
for very concise defaults for nulls:
```
ctx._source.foo_length = ctx._source.missing?.foo?.length() ?: 0;
```

Since you have to start somewhere, we started with null safe dereferenes.

Anyway, this is a feature borrowed from groovy. Groovy allows writing to
null values like:
```
def v = null
v?.field = 'cat'
```
And the writes are simply ignored. Painless doesn't support this at this
point because it'd be complex to implement and maybe not all that useful.

There is no runtime cost for this feature if it is not used. When it is
used we implement it fairly efficiently, adding a jump rather than a
temporary variable.

This should also work fairly well with doc values.
2016-11-09 07:20:11 -05:00
Jason Tedor b743ab0b07 Remove 5.x references from cat API migration doc
This commit removes some references to 5.x that were picked up when the
migration docs for the cat API were migrated from 5.x to master.

Relates #21342
2016-11-09 07:09:59 -05:00
Jason Tedor 168c54fa6e Add migration docs for cat API
This commit adds migration docs for the cat API, including a note
regarding the change in response in the cat thread pool API for
unbounded queue sizes.

Relates #21342
2016-11-09 07:07:52 -05:00
Clinton Gormley f5013e0872 Updated install docs to add missing empty cell in directory layout 2016-11-09 11:54:56 +01:00
Clinton Gormley 931d54f04d Update zip-targz.asciidoc
Added missing empty cell in directory layout table
2016-11-09 11:53:16 +01:00
javanna 2f32c1173b Revert "Tests: Remove a couple test uses of onModule (#21414)"
This reverts commit b326f0bc51.
2016-11-09 11:32:16 +01:00
Ryan Ernst b326f0bc51 Tests: Remove a couple test uses of onModule (#21414)
There were still a couple test use cases and examples that were using
onModule. This change cleans those cases up.
2016-11-08 13:50:13 -08:00
Jason Tedor aec09a76d6 Clarify requesting all stats in node stats docs
This commit clarifies how to explicitly obtain all stats from the node
stats API.
2016-11-08 13:47:15 -05:00
Luca Cavanna 293a3cab01 Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries (#21378)
* Rest client: don't reuse that same HttpAsyncResponseConsumer across multiple retries

Turns out that AbstractAsyncResponseConsumer from apache async http client is stateful and cannot be reused across multiple requests. The failover mechanism was mistakenly reusing that same instance, which can be provided by users, across retries in case nodes are down or return 5xx errors. The downside is that we have to change the signature of two public methods, as HttpAsyncResponseConsumer cannot be provided directly anymore, rather its factory needs to be provided which is going to be used to create one instance of the consumer per request attempt.

Up until now we tested our RestClient against multiple nodes only in a mock environment, where we don't really send http requests. In that scenario we can verify that retries etc. work properly but the interaction with the http client library in a real scenario is different and can catch other problems. With this commit we also add an integration test that sends requests to multiple hosts, and some of them may also get stopped meanwhile. The specific test for pathPrefix was also removed as pathPrefix is now randomly applied by default, hence implicitly tested. Moved also a small test method that checked the validity of the path argument to the unit test RestClientSingleHostTests.

Also increase default buffer limit to 100MB and make it required in default consumer

The default buffer limit used to be 10MB but that proved not to be high enough for scroll requests (see reindex from remote). With this commit we increase the limit to 100MB and make it a bit more visibile in the consumer factory.
2016-11-08 16:42:42 +01:00
Jason Tedor 6f6c633298 Fix markup in sysconfig-file docs
This commit fixes a markup issue in the sysconfig-file docs, adding a
missing backtick.
2016-11-08 08:33:45 -05:00
Jason Tedor cc2743743c Fix version constant in Groovy API docs
This commit fixes the version constant in the Groovy API docs from
5.0.0-alpha5 to 6.0.0-alpha1.
2016-11-08 08:08:45 -05:00
Joeyn414 aefffca763 minor grammatical fix (#21384) 2016-11-08 11:52:52 +01:00
Ryan Ernst 7a2c984bcc Test: Remove multi process support from rest test runner (#21391)
At one point in the past when moving out the rest tests from core to
their own subproject, we had multiple test classes which evenly split up
the tests to run. However, we simplified this and went back to a single
test runner to have better reproduceability in tests. This change
removes the remnants of that multiplexing support.
2016-11-07 15:07:34 -08:00
Ryan Ernst edb9d8fc03 Docs: Update discovery plugins example configuration with discovery.zen.hosts_provider (#21390)
This new setting was added with deguicing unicast hosts providers, but
the docs were not updated.
2016-11-07 13:33:08 -08:00
Jason Tedor 1f0c6cd137 Remove double space in bootstrap checks docs
This commit removes a double space in the G1GC check section of the
bootstrap check docs.
2016-11-07 16:28:04 -05:00
Jason Tedor b30732c464 Migrate G1GC JVM check to bootstrap check
This commit fixes an assertion in G1GCCheck#jvmVersion that was
mistakenly asserting on itself.

Relates #21388
2016-11-07 16:19:05 -05:00
Adrien Grand 263af27d76 Fix docs example after #21218. 2016-11-07 14:57:20 +01:00
Clinton Gormley a36465d479 Removed left over ID and [float] from the setup docs 2016-11-07 14:48:50 +01:00
markharwood dd21aa41be Docs fix - Diversified sampler agg had incorrect title and example
Closes #21347
2016-11-07 10:46:22 +00:00
Nik Everett 593d47efe2 Make it clear _suggest doesn't support source filtering (#21268)
We plan to deprecate `_suggest` during 5.0 so it isn't worth fixing
it to support the `_source` parameter for `_source` filtering. But we
should fix the docs so they are accurate.

Since this removes the last non-`// CONSOLE` line in
`completion-suggest.asciidoc` this also removes it from the list of
files that have non-`// CONSOLE` docs.

Closes #20482
2016-11-06 20:15:45 -05:00
Clinton Gormley 9d56c1b766 Groovy is no longer the default scripting language
Closes #21208
2016-11-05 16:31:34 +01:00
Loek van Gool 1a23739211 Update store.asciidoc (#21353)
* Update store.asciidoc

* Update store.asciidoc

* Update store.asciidoc
2016-11-05 14:58:16 +01:00
Nik Everett 7ff9ba1604 Fix asciidoc structure for sliced reindex
Asciidoc likes headings just so and will complain and fail the
docs build without it.

Related to #20767
2016-11-04 21:59:19 -04:00
Nik Everett a13a050271 Add automatic parallelization support to reindex and friends (#20767)
Adds support for `?slices=N` to reindex which automatically
parallelizes the process using parallel scrolls on `_uid`. Performance
testing sees a 3x performance improvement for simple docs
on decent hardware, maybe 30% performance improvement
for more complex docs. Still compelling, especially because
clusters should be able to get closer to the 3x than the 30%
number.

Closes #20624
2016-11-04 20:59:15 -04:00
Luca Cavanna c2160a88b5 Remove support for controversial ignore_unavailable and allow_no_indices from indices exists api (#20712)
Exist requests are supposed to never throw an exception, but rather return true or false depending on whether some resource exists or not. Indices exists does that for indices and accepts wildcard expressions too. The way the api works internally is by resolving indices and catching IndexNotFoundException: if an exception is thrown the index does not exist hence it returns false, otherwise it returns true. That works ok only if ignore_unavailable and allow_no_indices indices options are both set to false, meaning that they are strict and any missing index or wildcard expressions that resolves to no indices will lead to an exception that can be thrown and cause false to be returned.

Unfortunately the indices options have  been configurable up until now for this request, meaning that one can set ignore_unavailable or allow_no_indices to true and have the indices exist request return true for indices that really don't exist, which makes very little sense in the context of this api.

This commit removes the indicesOptions setter from the IndicesExistsRequest and makes settable only expandWildcardsOpen and expandWildcardsClosed, hence a subset of the available indices options. This way we can guarantee more consistent behaviour of the indices exists api. We can then remove the ignore_unavailable and allow_no_indices option from indices exists api spec
2016-11-04 19:26:37 +01:00
Andreas Creten 1ab4e0cba4 Add elasticsearcher for PHP (#21143)
Disclaimer: it is a library written by one of the engineers in my company.
2016-11-04 17:27:31 +01:00
Nik Everett a4b3a95f5a Move flush in _cat/indices docs tests (#21117)
Moves the `_flush` in the `_cat/indices` snippets testing framework
to the very first test. We need to flush super early because index
size is cached for a few seconds so we really need to read a
consistent size on the first read so we can sort by it properly.

Closes #21062
2016-11-04 10:32:07 -04:00
Lee Hinman 6666fb1614 Add "all field" execution mode to query_string query
This commit introduces a new execution mode for the query_string query, which
is intended down the road to be a replacement for the current _all field.

It now does auto-field-expansion and auto-leniency when the following criteria
are ALL met:

    The _all field is disabled
    No default_field has been set in the index settings
    No default_field has been set in the request
    No fields are specified in the request

Additionally, a user can force the "all-like" execution by setting the
all_fields parameter to true.

When executing in all field mode, the query_string query will look at all the
fields in the mapping that are not metafields and can be searched, and
automatically expand the list of fields that are going to be queried.

Relates to #19784
2016-11-04 05:46:18 -06:00
Chris Earle e6462ea4ca [DOCS] "service" should be "elasticsearch-service" (#21311)
The Windows `service.bat` script was renamed to `elasticsearch-service.bat` in ES 5.0+.
2016-11-03 15:37:37 -04:00
Ryan Ernst dc6ed7b8d4 Remove pluggability of ZenPing (#21049)
Plugins: Remove pluggability of ZenPing

ZenPing is the part of zen discovery which knows how to ping nodes.
There is only one alternative implementation, which is just for testing.
This change removes the ability to add custom zen pings, and instead
hooks in the MockZenPing for tests through an overridden method in
MockNode. This also folds in the ZenPingService (which was really just a
single method) into ZenDiscovery, and removes the idea of having
multiple ZenPing instances. Finally, this was the last usage of the
ExtensionPoint classes, so that is also removed here.
2016-11-03 08:20:20 -07:00
Adrien Grand 7ec51d628d Make the default S3 buffer size depend on the available memory. (#21299)
Currently the default S3 buffer size is 100MB, which can be a lot for small
heaps. This pull request updates the default to be 100MB for heaps that are
greater than 2GB and 5% of the heap size otherwise.
2016-11-03 16:07:52 +01:00
Jason Tedor 548c12d60f Remove broken docs link
The important settings docs previously referred to a section regarding
the node.max_local_storage_nodes setting. This section was removed, but
the link was not. This commit removes that link.
2016-11-03 10:14:46 -04:00
Jason Tedor 886eba076c Remove node.max_local_storage_nodes from setup doc
Previously node.max_local_storage_nodes defaulted to fifty, and this
permitted users to start multiple instances of Elasticsearch sharing the
same data folder. This can be dangerous, and usually it does not make
sense to run more than one instance of Elasticsearch on a single
server. Because of this, we had a note in the important settings docs
advising users to set this setting to one. However, we have since
changed the default value of this setting to one so this advise is no
longer needed.

Relates #21305
2016-11-03 09:03:42 -04:00
Clinton Gormley ef2453360e Fixed rollover docs test for date math indices 2016-11-03 13:48:34 +01:00
Clinton Gormley 6902eafbfe Specify date math with URI encoding in rollover API docs
Closes #21181
2016-11-03 10:48:50 +01:00
Shane Connelly 9f1f0e971f Merge pull request #21267 from eskibars/master
Updates plugin documentation for 5.0
2016-11-02 11:18:33 -07:00
Christoph Büscher a0c094d0c1 Add deprecation logging message for 'fuzzy' query
This query is deprecated from 5.0 on. Similar to IndicesQueryBuilder we should
log a deprecation warning whenever this query is used.

Relates to #15760
2016-11-02 15:45:33 +01:00
Shane Connelly c2062494a9 Fix commit message that got into ingegrations doc 2016-11-02 07:01:21 -07:00
Nik Everett b6db00efd8 Update docs after test-framework moved
Closes #21263
2016-11-02 09:51:55 -04:00
Shane Connelly eac04e2adc Drops any plugins that haven't been updated since 2.0. Updates 5.0 links/verbiage 2016-11-02 06:38:50 -07:00
Adrien Grand 52de0645fb Remove `lowercase_expanded_terms` and `locale` from query-parser options. (#20208)
Lucene 6.2 introduces the new `Analyzer.normalize` API, which allows to apply
only character-level normalization such as lowercasing or accent folding, which
is exactly what is needed to process queries that operate on partial terms such
as `prefix`, `wildcard` or `fuzzy` queries. As a consequence, the
`lowercase_expanded_terms` option is not necessary anymore. Furthermore, the
`locale` option was only needed in order to know how to perform the lowercasing,
so this one can be removed as well.

Closes #9978
2016-11-02 14:25:08 +01:00
Adrien Grand 68b0e395b2 Add recommendations about getting consistent scores despite shards and replicas. (#21167)
This is a topic that has triggered many questions recently so it would be good
to have these recommendations documented.
2016-11-02 10:50:38 +01:00
Jim Ferenczi 9d6fac809c Expose splitOnWhitespace in `Query String Query` (#20965)
This change adds an option called `split_on_whitespace` which prevents the query parser to split free text part on whitespace prior to analysis. Instead the queryparser would parse around only real 'operators'. Default to true.
For instance the query `"foo bar"` would let the analyzer of the targeted field decide how the tokens should be splitted.
Some options are missing in this change but I'd like to add them in a follow up PR in order to be able to simplify the backport in 5.x. The missing options (changes) are:
* A `type` option which similarly to the `multi_match` query defines how the free text should be parsed when multi fields are defined.
* Simple range query with additional tokens like ">100 50" are broken when `split_on_whitespace` is set to false. It should be possible to preserve this syntax and make the parser aware of this special syntax even when `split_on_whitespace` is set to false.
* Since all this options would make the `query_string_query` very similar to a match (multi_match) query we should be able to share the code that produce the final Lucene query.
2016-11-02 10:00:40 +01:00
Alexander Lin f86d784bbc Add extra note about Java8 requirement to sub-install docs (#21190)
Closes #20005
2016-11-01 15:52:09 -06:00
Adriel Dean-Hall b72a708c0d Add docs with up to date instructions on updating default similarity (#21242)
* Add docs with up to date instructions on updating default similarity

The default similarity can no longer be set in the configuration file
(you will get an error on startup). Update the docs with the method
that works.

* Add instructions for changing similarity on index creation
2016-11-01 16:14:20 -04:00
Craig Squire 1f1daf59bc Documentation updates for scroll API size parameter (#21229)
* Document size parameter for scroll API

* Fix size parameter behavior description for scroll
2016-11-01 15:55:09 -04:00
Nik Everett a612e5988e Bump reindex-from-remote's buffer to 200mb
It was 10mb and that was causing trouble when folks reindex-from-remoted
with large documents.

We also improve the error reporting so it tells folks to use a smaller
batch size if they hit a buffer size exception. Finally, adds some docs
to reindex-from-remote mentioning the buffer and giving an example of
lowering the size.

Closes #21185
2016-11-01 13:19:28 -04:00
LakumiNarayanan 5af6deb5b5 Fix typo in keyword.asciidoc (#21237) 2016-11-01 10:15:12 -04:00
Jason Tedor f81d28d0a0 Fix transport host setting
This commit fixes a reference to the transport.host setting which was
mistakenly identified as transport.tcp.host.
2016-10-31 20:48:55 -04:00
Jason Tedor 5fab17c7bc Fix markup for bootstrap checks docs
This commit adds a missing backtick to the bootstrap check docs.
2016-10-31 17:33:15 -04:00
Jack Conradson 185dff7346 Cleanup ScriptType (#21179)
Refactored ScriptType to clean up some of the variable and method names. Added more documentation. Deprecated the 'in' ParseField in favor of 'stored' to match the indexed scripts being replaced by stored scripts.
2016-10-31 13:48:51 -07:00
Jason Tedor 1d8d8ea55f Clarify production mode for bootstrap checks
This commit clarifies that production mode for the bootstrap checks is
only tripped if transport is bound to an external interface.

Relates #21220
2016-10-31 16:34:21 -04:00
Christoph Büscher 1f5adaa824 Docs: Adding Ukrainian analyzer 2016-10-31 18:20:39 +01:00
Lee Hinman 6a8bad8a06 [DOCS] Document all date formats (#21164)
Resolves #21046
2016-10-31 09:15:36 -06:00
Stanislav Mamontov 7738af27e8 Fix malformed JSON in Delete API example (#21168)
Obviously, there should be

    "result": "deleted"

instead of

    "result: deleted"
2016-10-31 09:13:46 -06:00
Nik Everett b5f86f6f05 Add note to getting-started doc about CONSOLE
Link to the documentation for CONSOLE in getting-started so the
layout of the snippet makes some sense.

Relates to #21158
2016-10-30 00:09:09 -04:00
Nik Everett 3a7a218e8f Support negative array ofsets in painless
Adds support for indexing into lists and arrays with negative
indexes meaning "counting from the back". So for if
`x = ["cat", "dog", "chicken"]` then `x[-1] == "chicken"`.

This adds an extra branch to every array and list access but
some performance testing makes it look like the branch predictor
successfully predicts the branch every time so there isn't a
in execution time for this feature when the index is positive.
When the index is negative performance testing showed the runtime
is the same as writing `x[x.length - 1]`, again, presumably thanks
to the branch predictor.

Those performance metrics were calculated for lists and arrays but
`def`s get roughly the same treatment though instead of inlining
the test they need to make a invoke dynamic so we don't screw up
maps.

Closes #20870
2016-10-29 16:12:40 -04:00
Boaz Leskes 75ee2bb61d Update resiliency page for the release of v5 (#21177) 2016-10-28 18:46:54 +02:00
Adrien Grand bafedc93fa Fix Lucene version in the docs. 2016-10-28 14:50:46 +02:00
Adrien Grand b3cc54cf0d Upgrade to lucene-6.3.0-snapshot-ed102d6 (#21150)
Lucene 6.3 is expected to be released in the next weeks so it'd be good to give
it some integration testing. I had to upgrade randomized-testing too so that
both Lucene and Elasticsearch are on the same version.
2016-10-28 14:47:15 +02:00
Adrien Grand 9cbbddb6dc Add support for `quote_field_suffix` to `simple_query_string`. (#21060)
Closes #18641
2016-10-28 09:11:57 +02:00
Jun Ohtani a66c76eb44 Merge pull request #20704 from johtani/remove_request_params_in_analyze_api
Removing request parameters in _analyze API
2016-10-27 17:43:18 +09:00
Colin Goodheart-Smithe c1a9833445 Correct similarity default for 5.0 (#21144) 2016-10-27 09:33:21 +01:00
Igor Motov 17ad88d539 Makes search action cancelable by task management API
Long running searches now can be cancelled using standard task cancellation mechanism.
2016-10-25 12:27:34 -10:00
Nik Everett 44c3b04bef Convert more docs to // CONSOLE
Converts docs for `_cat/segments`, `_cat/plugins` and `_cat/repositories`
from `curl` to `// CONSOLE` so they are tested as part of the build and
are cleaner to use in Console. They should work fine with `curl` with
the `COPY AS CURL` link.

Also swaps the `source` type of the response from `js` to `txt` because
that is more correct. The syntax highlighter doesn't care. It looks at
the text to figure out the language. So it looks a little funny for `_cat`
responses regardless.

Relates to #18160
2016-10-25 11:17:24 -04:00
Jason Tedor 900ee0536e Strengthen handling of unavailable cgroup stats
On some systems, cgroups will be available but not configured. And in
some cases, cgroups will be configured, but not for the subsystems that
we are expecting (e.g., cpu and cpuacct). This commit strengthens the
handling of cgroup stats on such systems.

Relates #21094
2016-10-24 16:36:51 -04:00
Praveen Shukla 2e18f2e818 [DOCS] clarifies master nodes to be master eligible nodes 2016-10-24 08:58:44 -04:00
Jason Tedor 3d642ab0eb Add basic cgroup CPU metrics
This commit adds basic cgroup CPU metrics to the node stats API.

Relates #21029
2016-10-24 08:26:56 -04:00
Jason Tedor 904dcc7127 Remove timeout parameter from plugin script docs
Support for this parameter was removed but the docs were not
updated. This commit removes this stale parameter from the docs.

Relates #21068
2016-10-21 10:56:46 -04:00
Lukáš Vlček d88e6c62aa [DOC] Fix HttpHost constructor arguments (#21056)
Protocol is the last argument.
Oficial API doc: https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/HttpHost.html
Your own use here: 7560101ec7/client/rest/src/test/java/org/elasticsearch/client/RequestLoggerTests.java (L54)
2016-10-21 09:53:08 +02:00
Igor Motov a541f0187d Docs: add documentation about removal of cluster.routing.allocation.snapshot.relocation_enabled 2016-10-20 14:19:12 -10:00
Jim Ferenczi d0bbe89c16 Optimize query with types filter in the URL (t/t/_search) (#20979)
This change adds a TypesQuery that checks if the disjunction of types should be rewritten to a MatchAllDocs query. The check is done only if the number of terms is below a threshold (16 by default and configurable via max_boolean_clause).
2016-10-20 12:33:32 +02:00
Nik Everett acf7c7430b Add "simple match" support for reindex-from-remote whitelist
This allows you to whitelist `localhost:*` or `127.0.10.*:9200`.
It explicitly checks for patterns like `*` in the whitelist and
refuses to start if the whitelist would match everything. Beyond
that the user is on their own designing a secure whitelist.
2016-10-18 21:47:21 -04:00
Tal Levy 38c650f376 make painless the default scripting language for ScriptProcessor (#20981)
- fixes a bug in the docs that mentions `lang` as optional
- now `lang` defaults to "painless"
2016-10-18 16:22:01 -07:00
Chris Earle c2894d2362 Add missing '// CONSOLE' to code snippet 2016-10-18 12:57:03 -04:00
Chris Earle 9cf7214380 [DOCS] Add "version" to template and pipeline docs (#20407)
* [DOCS] Add "version" to template and pipeline docs

This adds details about the "version" to both the template and pipeline pages.
2016-10-18 11:56:18 -04:00
Joshua Rich 8133c5e85d Merge pull request #20958 from joshuar/joshuar-docs/clearer-shard-allocation-filtering
Docs: Cluster Allocation Filtering
2016-10-18 14:24:41 +11:00
Joshua Rich cdb156e691 Merge pull request #20794 from joshuar/doc/fix_highlighter_ambiguities
[DOCS] Use a better name for fields in examples to avoid ambiguity
2016-10-18 14:23:27 +11:00
Clinton Gormley 5ec2ba3166 Update scripted-metric-aggregation.asciidoc
Removed docs for `reduce_params`

Closes #20917
2016-10-17 19:31:30 +02:00
David Pilato 1be7bdf721 Merge pull request #20975 from zhenxing914/2.3
node.client() should not be used in the documentation
2016-10-17 18:28:57 +02:00
Clinton Gormley 43ad8de75b Fixed typo in index rollover docs
Closes #20935
2016-10-17 17:56:54 +02:00
Jason Tedor 96aa5e33ce Fix slowlog docs
This commit fixes two issues with the slow log docs:
 - clarifies that these settings are per index
 - updates index slow log configuration for Log4j 2

Relates #20976
2016-10-17 10:50:32 -04:00
Quinn Shanahan 1bef6c7fee Update regexp-syntax.asciidoc (#20973) 2016-10-17 16:32:17 +02:00
Joshua Rich 63f484ffa3 Docs: Cluster Allocation Filtering
Put more emphasis on the fact that multiple values can be specified
and move examples after explanation of settings.
2016-10-17 11:02:33 +11:00
Nik Everett 4fbe1a8819 CONSOLEify _cat/pending_tasks docs
Relates to #18160
2016-10-14 14:12:35 -04:00
Jason Tedor 370253f95a Add doc note regarding processors bound
This commit expands the thread pool docs regarding the processor
setting.

Relates #20895
2016-10-14 10:32:31 -04:00
Nik Everett 68ed183381 CONSOLEify a few more _cat docs
`_cat/master`, `_cat/nodeattrs`, `_cat/nodes`.
2016-10-13 16:43:06 -04:00
Nik Everett 279baa0284 Add a flush to test in _cat/indices.asciidoc
We test that sorting by `store.size` works but sometimes the
sizes aren't what we expect. At least in CI:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-unix-compatibility/os=opensuse/101/console
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+5.x+multijob-unix-compatibility/os=centos/100/console

I haven't been able to reproduce it locally but adding a `_flush`
won't hurt and might make the inconsistency vanish.
2016-10-13 13:21:57 -04:00
Nik Everett 3bba7dbe07 Docs: note about snapshot version compatibility (#20896)
It is important that folks understand that snapshot/restore isn't
for archiving. It is appropriate for backup and disaster recovery
but not for archival over long periods of time because of version
incompatibility.

Closes #20866
2016-10-13 10:49:32 -04:00
Uli Fahrer 85094d9190 Fix wrong heading
Relates #20906
2016-10-13 07:06:53 -04:00
Clinton Gormley 101ea0549a Add note about retention periods to reindex-upgrade docs 2016-10-13 12:13:55 +02:00
Thibaud BARDIN 1bcd26627c [DOCS] Fix typo in "Wait For Active Shards" part (#20900)
Add missing closing backtick
2016-10-13 08:53:30 +02:00
Robin Clarke bbe6555b7a Docs: your -> you're (#20883) 2016-10-12 11:09:34 -04:00
Nik Everett 42a7a554b1 Don't mind pending cluster tasks in docs build
This removes an assertion that the cluster doesn't have any
pending cluster state tasks from the `_cat/health` docs.

Relates to #18160
2016-10-11 17:44:50 -04:00
Nik Everett 298cf1cf21 CONSOLEify _cat/indices docs
Relates to #18160
Uses the new sorting (#20658) in the `_cat` API to support all use
cases natively. We can still resort to piping things through `sort`
if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 14:55:37 -04:00
Thibaud BARDIN 82881ef99a [DOCS] Fix typo in "Cluster Health" part (#20864)
Replace "we can see and total of ..." by "we can see a total of ..."
2016-10-11 13:53:11 -04:00
Lee Hinman f0a2726dcd [DOCS] Remove documentation for `force` version-type
This option should not be recommended to anyone, and should never be
used, upon chance of primary/replica divergence.

Relates to #19769
2016-10-11 11:21:32 -06:00
Alexander Lin 10ddcc4616 Adding built-in sorting capability to _cat apis. (#20658)
* Adding built-in sorting capability to _cat apis.

Closes #16975

* addressing pr comments

* changing value types back to original implementation and fixing cosmetic issues

* Changing compareTo, hashCode of value types to a better implementation

* Changed value compareTos to use Double.compare instead of if statements + fixed some failed unit tests
2016-10-11 12:29:22 -04:00
Nik Everett 3528a514e2 Fix docs build
Removes a non-existant file from the non-`// CONSOLE` list. If you
remove all the non-`// CONSOLE` snippets from a file it must be
removed from the whitelist for the build to pass. Removing the file
entirely counts as removing the snippets.

Related to #18160
2016-10-11 08:58:51 -04:00
Clinton Gormley 82e2f6e747 Document the ctx._now variable in the update API
Relates to #20835
2016-10-11 13:13:03 +02:00
Clinton Gormley 5b5306ea82 Add an indices-upgrade section to the docs redirects. 2016-10-11 12:39:02 +02:00
Clinton Gormley a7d8de14bb Removed the upgrade API docs
Relates to #20675
2016-10-11 12:21:46 +02:00
Clinton Gormley 02a739d3c9 Added upgrade docs explaining how to reindex in place or reindex from remote
Closes #20675
2016-10-11 12:14:35 +02:00
Pascal Borreli fcb01deb34 Fixed typos (#20843) 2016-10-10 14:51:47 -06:00
Nik Everett c414faf2fb Reword note about windows and FDs
Make it clear why this isn't a thing in windows and remove value
judgement about file descriptors.

Relates to #20737
2016-10-10 08:47:25 -04:00
Nik Everett 741ecf80ff Loosen assertion in getting-started.asciidoc
We were asserting that the size of an index had a decimal point but it
doesn't always:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+5.0+java9-periodic/240/console
2016-10-09 13:47:59 -04:00
Lucas Bremgartner 4bf45e53a5 Documentation: comments in index templates (#20701)
Fixes #20513
2016-10-08 12:38:45 +02:00
Clinton Gormley 22f1acde94 Docs: Pattern analyzer does not support a max_token_length parameter
Closes #20713
2016-10-08 12:27:33 +02:00
David Pilato 2e1539b879 Fix URL for plugins
Reported at https://discuss.elastic.co/t/ingest-geoip-for-beta1/62505
2016-10-07 23:33:13 +02:00
Nik Everett 06049283a0 CONSOLEify some _cat docs
`/_cat/count`, `/_cat/fielddata`, and `/_cat/health`.

Three more files down, 141 to go.

Relates to #18160
2016-10-07 16:30:45 -04:00
Shane Connelly 78ae19ae5e Merge pull request #20784 from eskibars/master
Add a note that reindex does not set up mappings, etc
2016-10-07 09:39:10 -07:00
Clinton Gormley 6cf7a93837 Better formatting of date-match URL encoding 2016-10-07 16:33:30 +02:00
Clinton Gormley eb62f90578 Document date math use in the rollover API (#20780)
* Document date math use in the rollover API

Relates to #20709
2016-10-07 15:49:28 +02:00
Adrien Grand 7a403f640b Clarify some docs about geo-distance sorting. (#20735)
This also improves formatting a bit.
2016-10-07 15:26:34 +02:00
Jason Tedor d01a62908a Change separator for shards preference
The shards preference on a search request enables specifying a list of
shards to hit, and then a secondary preference (e.g., "_primary") can be
added. Today, the separator between the shards list and the secondary
preference is ';'. Unfortunately, this is also a valid separtor for URL
query parameters. This means that a preference like "_shards:0;_primary"
will be parsed into two URL parameters: "_shards:0" and "_primary". With
the recent change to strict URL parsing, the second parameter will be
rejected, "_primary" is not a valid URL parameter on a search
request. This means that this feature has never worked (unless the ';'
is escaped, but no one does that because our docs do not that, and there
was no indication from Elasticsearch that this did not work). This
commit changes the separator to '|'.

Relates #20786
2016-10-07 07:17:01 -05:00
Simon Willnauer 194a6b1df0 Remove LocalTransport in favor of MockTcpTransport (#20695)
This change proposes the removal of all non-tcp transport implementations. The
mock transport can be used by default to run tests instead of local transport that has
roughly the same performance compared to TCP or at least not noticeably slower.

This is a master only change, deprecation notice in 5.x will be committed as a
separate change.
2016-10-07 11:27:47 +02:00
Jun Ohtani 945fa499d2 Deprecating request parameters in _analyze API
Remove params in indices.analyze.json
Fix REST changes

Closes #20246
2016-10-07 16:23:24 +09:00
Jun Ohtani 99236b7627 Removing request parameters in _analyze API
Fix small English issue in breaking changes

Closes #20246
2016-10-07 16:23:24 +09:00
Jun Ohtani eca9894c5f Removing request parameters in _analyze API
Remove unused imports
Replace POST method by GET method in docs
 Add breaking changes explanation
 Fix small issue in Kuromoji docs

Closes #20246
2016-10-07 16:23:24 +09:00
Jun Ohtani 370f0b885e Removing request parameters in _analyze API
Remove request params in _analyze API without index param
Change rest-api-test using JSON
Change docs using JSON

Closes #20246
2016-10-07 16:23:24 +09:00
Joshua Rich e06a40ccbd [DOCS] Use a better name for fields in examples to avoid ambiguity
Previously, this doc was using a field called "content". This is
confusing, especially when the doc starts talking about the content of
the content field.  This change makes the field name "comment" which
is less ambiguous and also changes some related field names in the doc
to make a consistent example theme of editing docs around blog posts.
2016-10-07 14:46:55 +11:00
Clinton Gormley e83e8e890e Fixed released_state docs switch to build correctly and include missing links 2016-10-06 21:42:11 +02:00
Shane Connelly 3164917fd4 Adds a note that reindex does not set up mappings, etc. Closes #20783 2016-10-06 12:27:08 -07:00
Clinton Gormley d822a642b0 Document how to run a single docs test 2016-10-06 21:10:49 +02:00
Nik Everett 6d42e197b8 CONSOLEify remaining getting-started docs
Moves some test data used by those docs into the Elasticsearch
repository so we can use it when we test the docs during the build.

Relates to #18160
2016-10-06 14:06:48 -04:00
Nik Everett d9781bd069 Fix broken regex in doc tests
Regexes are hard.
2016-10-06 13:53:20 -04:00
Nik Everett d7d5df8863 CONSOLEify some _cat docs
Added `// NOTCONSOLE` to some `_cat` docs that rely on `sort` or
are otherwise too difficult for us to test at this point.

Relates to #20717
2016-10-06 13:37:21 -04:00
Clinton Gormley 40bf2c206c Add placeholder to the docs to distinguish released from unreleased from prerelease versions (#20644)
Closes https://github.com/elastic/stack/issues/18
2016-10-06 17:58:03 +02:00
Clinton Gormley 43a851f1f6 Most date-math chars in index names need to be percent encoded
Updated docs to provide encodings
2016-10-06 15:44:42 +02:00
Nik Everett 41d6529d06 CONSOLEify scroll docs
This causes the snippets to be tested during the build and gives
helpful links to the reader to open the docs in console or copy them
as curl commands.

Relates to #18160
2016-10-05 11:21:54 -04:00
Anatolii Stepaniuk f895abcf40 Fix grammar issues in some docs
This commit fixes some grammar issues in various docs.

Closes #20751
Closes #20752
Closes #20754
Closes #20755
2016-10-05 11:20:45 -04:00
Jason Tedor 51d53791fe Remove lenient URL parameter parsing
Today when parsing a request, Elasticsearch silently ignores incorrect
(including parameters with typos) or unused parameters. This is bad as
it leads to requests having unintended behavior (e.g., if a user hits
the _analyze API and misspell the "tokenizer" then Elasticsearch will
just use the standard analyzer, completely against intentions).

This commit removes lenient URL parameter parsing. The strategy is
simple: when a request is handled and a parameter is touched, we mark it
as such. Before the request is actually executed, we check to ensure
that all parameters have been consumed. If there are remaining
parameters yet to be consumed, we fail the request with a list of the
unconsumed parameters. An exception has to be made for parameters that
format the response (as opposed to controlling the request); for this
case, handlers are able to provide a list of parameters that should be
excluded from tripping the unconsumed parameters check because those
parameters will be used in formatting the response.

Additionally, some inconsistencies between the parameters in the code
and in the docs are corrected.

Relates #20722
2016-10-04 12:45:29 -04:00
Nik Everett 2b3760f78b File descriptors limit doesn't apply to Windows
On Windows the JDK uses `CreateFileW` which has a stupidly high
limit for the number of `Handle`s it can make - `16 * 1024 * 1024`.
So this isn't really a problem on Windows at all.

Closes #20732
2016-10-04 11:32:08 -04:00
Simon Willnauer 56f35baf47 Add date-math support to `_rollover` (#20709)
today it's not possible to use date-math efficiently with the `_rollover`
API. This change adds support for date-math in the target index as well as
support for preserving the math logic when an existing index that was created with
a date math expression all subsequent indices are created with the same expression.
2016-10-03 16:52:33 +02:00
Nik Everett 9271c0302f CONSOLEify some aggs docs
Cleans up the example result in `children-aggregation` so that
it matches the example data.

Relates to #18160
2016-10-03 09:22:56 -04:00
Tanguy Leroux 91835400b5 [Docs] Adapt //TESTREPONSE for Windows platforms
This commit adapts a //TESTRESPONSE check in the getting-started.asciidoc because the Load Average is not available on Windows platforms.
2016-10-03 11:30:23 +02:00
Nik Everett 7cfc2f126c CONSOLEify the first half of getting-started
Adds a `// TESTRESPONSE[_cat]` syntax that sets up some substitutions
useful for asserting that _cat APIs match.
2016-10-01 12:56:23 +02:00
Boaz Leskes 27eab74510 merge from master 2016-09-30 17:19:30 +02:00
Martijn van Groningen 55dce523c2 docs: marked `foreach` processor as experimental
Closes #19602
2016-09-30 12:23:42 +02:00
Simon Willnauer 74184cb1b0 Stabelize tests in phrase-suggest.asciidoc 2016-09-29 11:13:17 +02:00
Simon Willnauer f2e6862803 Add a hard limit for `index.number_of_shard` (#20682)
this change adds a hard limit to `index.number_of_shard` that prevents
indices from being created that have more than 1024 shards. This is still
a huge limit and can only be changed via settings a system property.
2016-09-29 11:03:30 +02:00
Jason Tedor 8879360f66 Fix failing doc tests in feature/seq_no
This commit fixes failing doc tests in feature/seq_no after merging
master into this branch.
2016-09-29 03:58:02 +02:00
Tal Levy f3a5ee671b [docs] [fix] `field` is no longer an option for the script processor (#20614) 2016-09-29 03:04:43 +02:00
Jason Tedor 25fd9e26c4 Merge branch 'master' into feature/seq_no
* master: (1199 commits)
  [DOCS] Remove non-valid link to mapping migration document
  Revert "Default `include_in_all` for numeric-like types to false"
  test: add a test with ipv6 address
  docs: clearify that both ip4 and ip6 addresses are supported
  Include complex settings in settings requests
  Add production warning for pre-release builds
  Clean up confusing error message on unhandled endpoint
  [TEST] Increase logging level in testDelayShards()
  change health from string to enum (#20661)
  Provide error message when plugin id is missing
  Document that sliced scroll works for reindex
  Make reindex-from-remote ignore unknown fields
  Remove NoopGatewayAllocator in favor of a more realistic mock (#20637)
  Remove Marvel character reference from guide
  Fix documentation for setting Java I/O temp dir
  Update client benchmarks to log4j2
  Changes the API of GatewayAllocator#applyStartedShards and (#20642)
  Removes FailedRerouteAllocation and StartedRerouteAllocation
  IndexRoutingTable.initializeEmpty shouldn't override supplied primary RecoverySource (#20638)
  Smoke tester: Adjust to latest changes (#20611)
  ...
2016-09-29 00:22:31 +02:00
Tal Levy 550a0449bc [docs] [fix] `field` is no longer an option for the script processor (#20614) 2016-09-28 21:50:32 +02:00
Lee Hinman 85402d5220 [DOCS] Remove non-valid link to mapping migration document 2016-09-28 09:09:19 -06:00
Lee Hinman 3f77eacab1 Revert "Default `include_in_all` for numeric-like types to false"
This reverts commit 6666892038.
2016-09-28 07:07:46 -06:00
Martijn van Groningen a7ad11c2e7 docs: clearify that both ip4 and ip6 addresses are supported 2016-09-28 10:03:43 +02:00
Lee Hinman 27496d6b92 Merge remote-tracking branch 'dakrone/breaker-done-resiliency' 2016-09-26 08:39:55 -06:00
Nik Everett 560fba1b28 Document that sliced scroll works for reindex
Surprise! You can use sliced scroll to easily parallelize reindex
and friend. They support it because they use the same infrastructure
as a regular search to parse the search request. While we would like
to make an "automatic" option for parallelizing reindex, this manual
option works right now and is pretty convenient!
2016-09-26 05:27:44 +02:00
Pius 660d9d0de3 Remove Marvel character reference from guide
Relates #20654
2016-09-25 11:28:13 -04:00
Jason Tedor a7c777f58a Fix documentation for setting Java I/O temp dir
This commit fixes the documentation for configuring the Java I/O temp
dir which incorrectly suggested using the -D flag as a parameter on the
command line; these flags have been removed and should now be specified
as arguments to the JVM using either the ES_JAVA_OPTS environment
variable or using the jvm.options configuration file.

Closes #20652
2016-09-24 21:13:22 -04:00
Clinton Gormley e3b7b4f032 Reorganised docs for mapping safeguard settings 2016-09-22 14:58:17 +02:00
Martijn van Groningen ad7c22198c docs: describe more explicitly what happens when indexing queries that fetch terms 2016-09-22 10:00:11 +00:00
Nik Everett 3ed3e5e660 Convert more docs to CONSOLE
* plugins/discovery-azure-class.asciidoc
* reference/cluster.asciidoc
* reference/modules/cluster/misc.asciidoc
* reference/modules/indices/request_cache.asciidoc

After this is merged there will be no unconvereted snippets outside
of `reference`.

Related to #18160
2016-09-21 09:36:21 -04:00
Clinton Gormley c753a0bfe0 Change APT/YUM urls to point to the prerelease repositories
Closes #20593
2016-09-21 14:49:14 +02:00
Lee Hinman 45d4d08f32 [DOCS] Mark combinatorial explosion in aggs as 'done'
This marks the "Prevent combinatorial explosion in aggregations from
causing OOM" task as done in 5.0.0.

Relates to #8081 and #19394
2016-09-20 16:07:43 -06:00
Nik Everett c97f4f38e7 Weaken overspecified tests in response snippets
The docs in api-conventions contained some very specific response
snippets that made the tests a bit fragile. This removes some of
that fragility.
2016-09-20 13:53:07 -04:00
Nik Everett af285f1aa4 Convert more docs to CONSOLE
* reference/api-conventions.asciidoc
* reference/redirects.asciidoc

Relates to #18160
2016-09-20 13:12:12 -04:00
Clinton Gormley 57d40a4691 Fix JVM heap size docs for 5.0
Closes #20587
2016-09-20 18:51:56 +02:00
David Pilato c2d621dfdf Merge branch 'doc/plugins-offline-url' 2016-09-20 14:53:05 +02:00
Tanguy Leroux 33348a9c7c [DOC] Add /cat/templates to list of excluded docs 2016-09-20 11:02:55 +02:00
Alexander Lin d31a8e6558 Provides a cat api endpoint for templates. (#20545)
Adds a cat api endpoint: /_cat/templates and its more specific version, /_cat/templates/{name}.

It looks something like:

$ curl "localhost:9200/_cat/templates?v"
name                  template     order version
sushi_california_roll *avocado*    1     1
pizza_hawaiian        *pineapples* 1
pizza_pepperoni       *pepperoni*  1

The specified version (only allows * globs) looks like:

$ curl "localhost:9200/_cat/templates/pizza*"
name            template     order version
pizza_hawaiian  *pineapples* 1
pizza_pepperoni *pepperoni*  1

Partially specified columns:

$ curl "localhost:9200/_cat/templates/pizza*?v=true&h=name,template"
name            template
pizza_hawaiian  *pineapples*
pizza_pepperoni *pepperoni*

The help text:

$ curl "localhost:9200/_cat/templates/pizza*?help"
name     | n | template name
template | t | template pattern string
order    | o | template application order number
version  | v | version

Closes #20467
2016-09-20 10:40:23 +02:00
Tanguy Leroux 24fda8a928 [DOC] Remove 'reference/docs/get.asciidoc' from list of excluded docs 2016-09-20 10:23:09 +02:00
Jim Ferenczi f98d5b6261 Add CONSOLE tests for snippets in get and bulk API docs (#20473)
* Add CONSOLE tests for snippets in get and bulk API docs

This change adds tests for the snippets in the get and bulk API documentation.
2016-09-20 09:18:08 +02:00
Clinton Gormley 95bca7c390 Remove redundant .x from apt/rpm repo docs
Relates to #20574
2016-09-20 08:11:02 +02:00
Ryan Ernst b77c16086c Convert old download links to unified release urls (#20574)
With the unified release process across the elastic stack, download
links for all products are changing. This change updates docs referring
to the old download and packages urls.

Note that this change also updates the plugin installation command as
the url for downloads is being changed to be consistent with that for
packages (both plural).
2016-09-19 18:10:45 -07:00
Jason Tedor 05b4e0c0e3 Add serial collector bootstrap check
The serial collector is not suitable for running with a server
application like Elasticsearch and can decimate performance and lead to
cluster instability. This commit adds a bootstrap check to prevent usage
of the serial collector when Elasticsearch is running in production
mode.

Relates #20558
2016-09-19 20:25:50 -04:00
Nik Everett 156393be0e Fail build if new doc snippets aren't `// CONSOLE`
This tracks the snippets that probably should be converted to
`// CONSOLE` or `// TESTRESPONSE` and fails the build if the list
of files with such snippets doesn't match the list in `docs/build.gradle`.
Setting the file looks like
```
/* List of files that have snippets that probably should be converted to
 * `// CONSOLE` and `// TESTRESPONSE` but have yet to be converted. Try and
 * only remove entries from this list. When it is empty we'll remove it
 * entirely and have a party! There will be cake and everything.... */
buildRestTests.expectedUnconvertedCandidates = [
  'plugins/discovery-azure-classic.asciidoc',
...
  'reference/search/suggesters/completion-suggest.asciidoc',
]
```

This list is in `build.gradle` because we expect it to be fairly
temporary. In a few months we'll have converted all of the docs and won't
ned it any more.

From now on if you add now docs that contain a snippet that shows an
interaction with elasticsearch you have three choices:
1. Stick `// CONSOLE` on the interactions and `// TESTRESPONSE` on the
responses. The build (specifically (`gradle docs:check`) will test that
these interactions "work". If there isn't a `// TESTRESPONSE` snippet
then "work" just means "Elasticsearch responds with a 200-level response
code and no `WARNING` headers. This is way better than nothing.

2. Add `// NOTCONSOLE` if the snippet isn't actually interacting with
Elasticsearch. This should only be required for stuff like javascript
source code or `curl` against an external service like AWS or GCE. The
snippet will not get "OPEN IN CONSOLE" or "COPY AS CURL" buttons or be
tested.

3. Add `// TEST[skip:reason]` under the snippet. This will just skip the
snippet in the test phase. This should really be reserved for snippets
where we can't test them because they require an external service that
we don't have at testing time.

Please, please, please, please don't add more things to the list. After
all, it sais there'll be cake when we remove it entirely!

Relates to #18160
2016-09-19 16:43:43 -04:00
David Pilato ed4d0881b1 Add profile and explain parameters to template API
We can now run templates using `explain` and/or `profile` parameters.
Which is interesting when you have defined a complicated profile but want to debug it in an easier way than running the full query again.

You can use `explain` parameter when running a template:

```js
GET /_search/template
{
  "file": "my_template",
  "params": {
    "status": [ "pending", "published" ]
  },
  "explain": true
}
```

You can use `profile` parameter when running a template:

```js
GET /_search/template
{
  "file": "my_template",
  "params": {
    "status": [ "pending", "published" ]
  },
  "profile": true
}
```
2016-09-19 17:52:13 +02:00
David Pilato 4c9a77e6c0 Add link to "offline install" and show the full URL 2016-09-19 15:04:29 +02:00
David Pilato 9e58ca835c Merge branch 'master' into doc/plugins-offline-url
# Conflicts:
#	docs/plugins/mapper-attachments.asciidoc
2016-09-19 14:51:55 +02:00
David Pilato a530399ae5 Merge branch 'pr/remove-mapper-attachments' 2016-09-19 14:47:14 +02:00
Jason Tedor 7874463db0 Merge pull request #20553 from jasontedor/node-attributes
Node attributes and REST API spec README
2016-09-19 08:31:09 -04:00
Alexander Lin 7cd0316b51 Fix minhash docs level
Relates #20547
2016-09-19 07:54:04 -04:00
Jason Tedor b456823434 Fix leftover node attributes usage
Previously node attributes could be set via node.* but this now requires
using node.attr.*. This commit fixes some leftover usages of the old
way.
2016-09-19 07:45:31 -04:00
Tanguy Leroux 656596c2a9 [DOC] Remove obsolete node names from documentation
Funny node names have been removed in #19456 and replaced by UUID. This commit removes these obsolete node names and replace them by real UUIDs in the documentation.

closes #20065
2016-09-19 11:56:28 +02:00
Tanguy Leroux 1894489832 [DOC] Update /_cat/nodes doc
closes #20162
2016-09-19 09:31:48 +02:00
David Pilato dfd1eebdd0 Remove mapper attachments plugin
We now have in 5.0.0 `ingest-attachment` plugin.
We can remove `mapper-attachments` plugin for 6.0.

Closes #18837.
2016-09-19 09:01:16 +02:00
Alexander Guz c1e8b6a8ba Fixed aggregation by "gender" request example.
`gender.keyword` should be used instead of just `gender` or we
get an error `Fielddata is disabled on text fields by default. Set
fielddata=true on [gender] in order to load fielddata in memory by
uninverting the inverted index. Note that this can however use

Closes #20535
significant memory.`
2016-09-17 17:23:27 -04:00
Ali Beyad f608e6c6cf Improves the documentation for the (#20531)
`cluster.routing.allocation.cluster_concurrent_rebalance` setting,
clarifying in which shard allocation situations the rebalance limit
takes effect.

Closes #20529
2016-09-16 16:06:18 -04:00
Alexander Guz 8ec94a4ba0 Edited response structure on indexing a document (#20517)
Added "_shards" and "result" keys to the response.
2016-09-16 10:16:40 -06:00
Boaz Leskes 577dcb3237 Add current cluster state version to zen pings and use them in master election (#20384)
During a networking partition, cluster states updates (like mapping changes or shard assignments)
are committed if a majority of the masters node received the update correctly. This means that the current master has access to enough nodes in the cluster to continue to operate correctly. When the network partition heals, the isolated nodes catch up with the current state and get the changes they couldn't receive before. However, if a second partition happens while the cluster
is still recovering from the previous one *and* the old master is put in the minority side, it may be that a new master is elected which did not yet catch up. If that happens, cluster state updates can be lost.

This commit fixed 95% of this rare problem by adding the current cluster state version to `PingResponse` and use them when deciding which master to join (and thus casting the node's vote). 

Note: this doesn't fully mitigate the problem as a cluster state update which is issued concurrently with a network partition can be lost if the partition prevents the commit message (part of the two phased commit of cluster state updates) from reaching any single node in the majority side *and* the partition does allow for the master to acknowledge the change. We are working on a more comprehensive fix but that requires considerate work  and is targeted at 6.0.
2016-09-15 23:39:11 +02:00
Areek Zillur 0556c93920 Merge pull request #20488 from areek/docs/clarify_plugins_installation
[DOC] Add note requiring full cluster restart for installing plugins with custom metadata
2016-09-15 17:05:27 -04:00
Areek Zillur 9b17242b5d [DOC] Add note for full cluster restart for installing plugins using custom metadata
Currently, we check if a node has the same set of custom metadata as the master
before joining the cluster. This implies freshly installing a plugin that has its
custom metadata requires a full cluster restart.
2016-09-15 17:04:45 -04:00
Nik Everett e4c80c94e9 Convert more search docs to CONSOLE
`profile.asciidoc` now runs all of its command but it doesn't validate
all of the results. Writing the validation is time consuming so I only
did some of it.
2016-09-15 11:58:21 -04:00
Luca Cavanna 37489c3274 Add clusterUUID to RestMainAction output (#20503)
Add clusterUUID to RestMainAction output

GET / now returns the clusterUUID as well as part of its output for monitoring purposes
2016-09-15 16:25:17 +02:00
gfyoung b03c807368 Rename service.bat to elasticsearch-service.bat (#20496)
Closes gh-17528.
2016-09-15 15:53:44 +02:00
Clinton Gormley 960efe6202 Fixed typo in cat indices docs 2016-09-14 18:36:58 +01:00
Nik Everett 2d568ece2d CONSOLEify some search docs
* search/search.asciidoc
* search/request-body.asciidoc
* search/explain.asciidoc
* search/search-shards.asciidoc
2016-09-14 13:06:37 -04:00
Tobias Günther 3a7a437594 Update rescoring docs in respect to sort (#20477)
* Update rescoring docs in respect to sort

If sort is present in a query the rescore query is not executed. As long as this feature is neither implemented (see discussion in #6788) nor  the combination of sort and rescoring raises an error, we should warn the user in the documentation about this.

* Missed a dot
2016-09-14 17:07:10 +01:00
Jason Tedor 412c61c402 Update logger names in docs
In 7560101ec7, the Elasticsearch logger
names were modified to be their fully-qualified class name (with some
exceptions for special loggers like the slow logs and the transport
tracer). This commit updates the docs accordingly.

Relates #20475
2016-09-14 08:08:49 -04:00
Tal Levy 4704efaef4 [ingest-geoip] do not insert null-valued fields in geoip response
update geoip to not include null-valued results from database

Originally, the plugin would still insert all the requested fields, but
assign null to each one. This fixes that by not writing the fields at
all. Makes for a better experience when the null fields conflict with
the typical geo_point field mapping.
2016-09-13 18:12:02 -07:00
Ali Beyad 730f9058a1 Documentation for file-based discovery plugin (#20462)
Documentation for file-based discovery plugin.

Closes #20458
2016-09-13 20:53:10 -04:00
debadair ee4798d852 Docs: Added redirect for geohash_cell query topic. 2016-09-13 17:40:24 -07:00
Nicholas Knize 598bab93ae [DOC] Cleanup dangling references to deprecated geo parameters
With the cut over to LatLonPoint the geohash, geohash_precision, lat_lon, and geohash_prefix parameters have been removed. This commit fixes the doc build by removing the remaining dangling references to these removed parameters.
2016-09-13 16:38:38 -05:00
Jim Ferenczi 1764ec56b3 Fixed naming inconsistency for fields/stored_fields in the APIs (#20166)
This change replaces the fields parameter with stored_fields when it makes sense.
This is dictated by the renaming we made in #18943 for the search API.

The following list of endpoint has been changed to use `stored_fields` instead of `fields`:
* get
* mget
* explain

The documentation and the rest API spec has been updated to cope with the changes for the following APIs:
* delete_by_query
* get
* mget
* explain

The `fields` parameter has been deprecated for the following APIs (it is replaced by _source filtering):
* update: the fields are extracted from the _source directly.
* bulk: the fields parameter is used but fields are extracted from the source directly so it is allowed to have non-stored fields.

Some APIs still have the `fields` parameter for various reasons:
* cat.fielddata: the fields paramaters relates to the fielddata fields that should be printed.
* indices.clear_cache: used to indicate which fielddata fields should be cleared.
* indices.get_field_mapping: used to filter fields in the mapping.
* indices.stats: get stats on fields (stored or not stored).
* termvectors: fields are retrieved from the stored fields if possible and extracted from the _source otherwise.
* mtermvectors:
* nodes.stats: the fields parameter is used to concatenate completion_fields and fielddata_fields so it's not related to stored_fields at all.

Fixes #20155
2016-09-13 20:54:41 +02:00
Nicholas Knize 1a60e1c3d2 Update docs for LatLonPoint cut over
This commit removes documentation for:

* geohash cell query
* lat_lon parameter
* geohash parameter
* geohash_precision parameter
* geohash_prefix parameter

It also updates failing tests that reference these parameters for backcompat.
2016-09-13 12:18:21 -05:00
Tanguy Leroux 6090c51fc5 Add quiet option to disable console logging (#20422)
This commit adds a -q/--quiet option to Elasticsearch so that it does not log anything in the console and closes stdout & stderr streams. This is useful for SystemD to avoid duplicate logs in both journalctl and /var/log/elasticsearch/elasticsearch.log while still allows the JVM to print error messages in stdout/stderr if needed.

closes #17220
2016-09-13 14:08:24 +02:00
Jason Tedor c7bfbe3e69 Add health status parameter to cat indices API
This commit adds a health status parameter to the cat indices API for
filtering on indices that match the specified status (green|yellow|red).

Relates #20393
2016-09-13 07:57:18 -04:00
Yevhen Bobrov 786508be08 Documentation for field_masking_span query (#20395)
* Documentation for field_masking_span query. Fixes #20293

* After review fixes
2016-09-13 12:27:33 +01:00
Isabel Drost-Fromm 0e707f241e Add docs to template support for _msearch (#17382)
Add docs to template support for _msearch

Relates to #10885
Relates to #15674

* Reference those docs from the rest api spec for _msearch/template support.
2016-09-13 13:19:25 +02:00
Lee Hinman 94625d74e4 No longer allow cluster name in data path
In 5.x we allowed this with a deprecation warning. This removes the code
added for that deprecation, requiring the cluster name to not be in the
data path.

Resolves #20391
2016-09-12 15:47:01 -06:00
Nik Everett 69bf08f6c6 Disable regexes by default in painless
Adds a new node level, non-dynamic setting, `script.painless.regex.enabled`
can be used to enable regexes.

Closes #20397
2016-09-12 14:09:43 -04:00
David Pilato 472c679981 Add offline install instructions for plugins
Follow up of https://github.com/elastic/elasticsearch/issues/15925#issuecomment-171250150

This commit adds offline install instructions for plugins.
2016-09-12 15:36:16 +02:00
Tal Levy 9f1f5fdedc introduce the JSON Processor (#20128)
introduce the JSON Processor
2016-09-09 14:34:32 -07:00
Tal Levy dda32545bb add ignore_missing option to relevant processors (#20194) 2016-09-09 12:20:18 -07:00
Clinton Gormley 9c3007496c Update search-template.asciidoc
Fixed typo
2016-09-09 13:06:49 +02:00
Clinton Gormley 2f6d0119f1 Added warning messages about the dangers of pathological regexes to:
* pattern-replace charfilter
* pattern-capture and pattern-replace token filters
* pattern tokenizer
* pattern analyzer

Relates to #20038
2016-09-09 09:53:07 +02:00
Clinton Gormley add2fbd7b2 Update painless.asciidoc
Asciidoc typo
2016-09-09 09:30:04 +02:00
Lee Hinman 49695af2ac Remove FORCE version_type
This was an error-prone version type that allowed overriding previous
version semantics. It could cause primaries and replicas to be out of
sync however, so it has been removed.

Resolves #19769
2016-09-08 15:29:47 -06:00
Jason Tedor 27ff4f327c Remove allow unquoted JSON
Previous versions of Elasticsearch permitted unquoted JSON field names even though this is against the JSON spec. This leniency was disabled by default in the 5.x series of Elasticsearch but a backwards compatibility layer was added via a system property with the intention of removing this layer in 6.0.0. This commit removes this backwards compatibility layer.

Relates #20388
2016-09-08 13:36:31 -04:00
Clinton Gormley 100cdf0ba9 Use a comment block to comment out release notes 2016-09-08 18:30:37 +02:00
Clinton Gormley 9974e3f3d8 Bumped doc versions to 6.0.0-alpha1 2016-09-08 18:29:18 +02:00
Clinton Gormley 891a15db22 Add breaking changes for 6.0 2016-09-08 18:27:01 +02:00
Clinton Gormley b9d20a039f Comment out release notes section 2016-09-08 18:17:27 +02:00
Clinton Gormley 1f11a5d93c Remove links to release notes 2016-09-08 18:07:39 +02:00
Jason Tedor 21e1ae8611 Remove 5.x release notes
This commit removes the series 5.x release notes from master as that
branch is currently on the 6.x series.
2016-09-08 11:31:28 -04:00
Lee Hinman 40b088d728 Rework documentation example for _all to be less ambigious with numerics 2016-09-08 09:09:48 -06:00
Lee Hinman 6666892038 Default `include_in_all` for numeric-like types to false
This includes:

- All regular numeric types such as int, long, scaled-float, double, etc
- IP addresses
- Dates
- Geopoints and Geoshapes

Relates to #19784
2016-09-08 09:09:48 -06:00
Jason Tedor 8e7dfae7d1 Remove collect payloads parameter
The collect_payloads parameter of the span_near query was previously
deprecated with the intention to be removed. This commit removes this
parameter.

Relates #20385
2016-09-08 09:37:36 -04:00
Lee Hinman b4cc3cd35d Remove FORCE version_type
This was an error-prone version type that allowed overriding previous
version semantics. It could cause primaries and replicas to be out of
sync however, so it has been removed.

Resolves #19769
2016-09-07 13:05:18 -06:00
Nik Everett bebdec570f [docs] Mark percolator response snippets properly
Now the docs tests will catch any errors in the responses. This
would have caught the error fixed in
https://github.com/elastic/elasticsearch/pull/20351
2016-09-07 09:45:50 -04:00
antonisppn e77f4710e4 [docs] Percolator samples are not working. Mapping is wrong.
Hi all, 

I was trying to run the percolate examples, but I figured that because of the "type":"keyword" , the code wasn't working.
In the saerch query the "message" : "A new bonsai tree in the office" is a pure string. 

I changed it to "text".
2016-09-07 08:15:20 -04:00
Alexander Lin f825e8f4cb Exposing lucene 6.x minhash filter. (#20206)
Exposing lucene 6.x minhash tokenfilter

Generate min hash tokens from an incoming stream of tokens that can
be used to estimate document similarity.

Closes #20149
2016-09-07 09:38:12 +02:00
Lee Hinman fd3392aef8 [DOCS] Mark cluster allocation explain API as experimental in docs 2016-09-06 11:29:33 -06:00
Martijn van Groningen 245882cde3 * Removed `script.default_lang` setting and made `painless` the hardcoded default script language.
** The default script language is now maintained in `Script` class.
* Added `script.legacy.default_lang` setting that controls the default language for scripts that are stored inside documents (for example percolator queries).  This defaults to groovy.
** Added `QueryParseContext#getDefaultScriptLanguage()` that manages the default scripting language. Returns always `painless`, unless loading query/search request in legacy mode then the returns what is configured in `script.legacy.default_lang` setting.
** In the aggregation parsing code added `ParserContext` that also holds the default scripting language like `QueryParseContext`. Most parser don't have access to `QueryParseContext`. This is for scripts in aggregations.
* The `lang` script field is always serialized (toXContent).

Closes #20122
2016-09-06 18:44:48 +02:00
Nik Everett e03fb602cd Add CONSOLE places where it is obviously missing
These places already have other annotations like `// TEST` and
`// TESTSETUP` so they are already in console format.
2016-09-06 10:48:19 -04:00
Nik Everett 5cff2a046d Remove most of the need for `// NOTCONSOLE`
and be much more stingy about what we consider a console candidate.

* Add `// CONSOLE` to check-running
* Fix version in some snippets
* Mark groovy snippets as groovy
* Fix versions in plugins
* Fix language marker errors
* Fix language parsing in snippets

  This adds support for snippets who's language is written like
  `[source, txt]` and `["source","js",subs="attributes,callouts"]`.

  This also makes language required for snippets which is nice because
  then we can be sure we can grep for snippets in a particular language.
2016-09-06 10:32:54 -04:00
Martijn van Groningen 6f6d17dc9c ingest: Add `dot_expander` processor that can turn fields with dots in the field name into object fields. 2016-09-05 07:28:38 +02:00
Nik Everett bfd072bc10 Switch more docs to CONSOLE
Related to #18160
2016-09-02 18:22:30 -04:00
Jun Ohtani c4759bcc02 Merge pull request #20285 from johtani/fix/remove_token_filter_param_in_analyze_api
Remove `token_filter` in _analyze API
2016-09-03 02:03:51 +09:00
Areek Zillur c92f82e624 Merge pull request #20169 from areek/doc/fix_completion_breaking_changes
Update breaking changes for completion suggester
2016-09-02 12:39:16 -04:00
Areek Zillur af215b528f move completion performance tips from migration docs to completion docs 2016-09-02 12:37:56 -04:00
Clinton Gormley 4885709e10 Update rollover-index.asciidoc
Fixed weirdly formatted callouts in rollover docs
2016-09-02 16:08:34 +02:00
Greg Ichneumon Brown 639b7278d9 Docs: clarify calculation of sigma and lambda in function_score (#20267)
- Using log() to indicate natural log can add some confusion when trying to further adjust/tweak scores. Other parts of the API (field_value_factor on this same page) use 'ln' and 'log', so this change should be more consistent
- Fixes #20027
- I generated the images using http://latex2png.com/ at a resolution of 150 which seemed to be about the same size as before
2016-09-02 14:41:07 +02:00
Adrien Grand cdc27b75b8 Add more information to the how-to docs. #20297
- use auto-generated ids for indexing #20211
 - use rounded dates in queries #20115
2016-09-02 14:28:47 +02:00
Clinton Gormley 234ca6b1fd Fix bad link in plugin docs 2016-09-02 10:54:05 +02:00
Luca Cavanna faa03ad9fa Merge pull request #20255 from javanna/enhancement/cluster_stats_available_memory
Add mem section back to cluster stats
2016-09-02 10:19:51 +02:00
Jun Ohtani aef2e5d90e Remove `token_filter` in _analyze API
Fix wording in docs
Refactoring RestAnalyzeActionTests using expectThrows()

Closes #20283
2016-09-02 15:08:28 +09:00
Areek Zillur c869ca18eb Update breaking changes for completion suggester
To indicate we removed completion payload option
in favour of returning document source with
completion suggestions
2016-09-01 21:48:30 -04:00
nrichers e5bf02b155 Fix broken link reportedby Twitter user (#20291) 2016-09-01 17:41:44 -07:00
Jack Conradson 7c552f8127 Merge branch 'master' into deprecate 2016-09-01 14:53:49 -07:00
Nik Everett c8b984aee9 Add CONSOLE to more docs
Relates to #18160
2016-09-01 17:08:18 -04:00
Jason Tedor 5fe4cb6adc Size limit deprecation logs
This commit configures the deprecation logs to be size-limited to 1 GB,
and compress these logs when they roll. The default configuration will
preserve up to four rolled logs.

Relates #20287
2016-09-01 14:25:04 -04:00
Florian Hopf 359e76f7e7 Fixed wording 2016-09-01 11:22:44 -06:00
Nik Everett 4b219d15d2 Add CONSOLE to a few snippets in reference docs
This allows them to be run in Console and adds them to the list of
docs that are automatically tested as part of the build.

Relates to #18160
2016-09-01 13:08:36 -04:00
Jun Ohtani 3d9f8ed764 Remove `token_filter` in _analyze API
Remove the param and change docs

Closes #20283
2016-09-02 01:36:45 +09:00
Jack Conradson 71d8ee5eac Merge branch 'master' into deprecate 2016-09-01 08:51:29 -07:00
javanna 5f299ff46f add mem section back to cluster stats
The mem section was buggy in cluster stats and removed. It is now added back with the same structure as in node stats, containing total memory, available memory, used memory and percentages. All the values are the sum of all the nodes across the cluster (or at least the ones that we were able to get the values from).
2016-09-01 11:26:03 +02:00
Simon Willnauer a0becd26b1 Optimize indexing for the autogenerated ID append-only case (#20211)
If elasticsearch controls the ID values as well as the documents
version we can optimize the code that adds / appends the documents
to the index. Essentially we an skip the version lookup for all
documents unless the same document is delivered more than once.

On the lucene level we can simply call IndexWriter#addDocument instead
of #updateDocument but on the Engine level we need to ensure that we deoptimize
the case once we see the same document more than once.

This is done as follows:

1. Mark every request with a timestamp. This is done once on the first node that
receives a request and is fixed for this request. This can be even the
machine local time (see why later). The important part is that retry
requests will have the same value as the original one.

2. In the engine we make sure we keep the highest seen time stamp of "retry" requests.
This is updated while the retry request has its doc id lock. Call this `maxUnsafeAutoIdTimestamp`

3. When the engine runs an "optimized" request comes, it compares it's timestamp with the
current `maxUnsafeAutoIdTimestamp` (but doesn't update it). If the the request
timestamp is higher it is safe to execute it as optimized (no retry request with the same
timestamp has been run before). If not we fall back to "non-optimzed" mode and run the request as a retry one
and update the `maxUnsafeAutoIdTimestamp` unless it's been updated already to a higher value

Relates to #19813
2016-09-01 10:39:40 +02:00
Jack Conradson 1cdfc3ccfe Merge branch 'master' into deprecate 2016-08-31 15:58:14 -07:00
Jack Conradson 3b3baa6e6c Made deprecation of Groovy, Javascript, and Python more explicit. 2016-08-31 15:56:31 -07:00
Jason Tedor 76ab02e002 Merge branch 'master' into log4j2
* master:
  Avoid NPE in LoggingListener
  Randomly use Netty 3 plugin in some tests
  Skip smoke test client on JDK 9
  Revert "Don't allow XContentBuilder#writeValue(TimeValue)"
  [docs] Remove coming in 2.0.0
  Don't allow XContentBuilder#writeValue(TimeValue)
  [doc] Remove leftover from CONSOLE conversion
  Parameter improvements to Cluster Health API wait for shards (#20223)
  Add 2.4.0 to packaging tests list
  Docs: clarify scale is applied at origin+offest (#20242)
2016-08-31 16:37:55 -04:00
Jason Tedor 750033dc4b Update docs for Log4j 2
This commit updates the logging docs for Elasticsearch to reflect the
migration to Log4j 2.
2016-08-31 15:51:52 -04:00
Nik Everett 1128776b3c [docs] Remove coming in 2.0.0
2.0.0's been out for a long time.

Closes #20268
2016-08-31 14:43:04 -04:00
Nik Everett dcaed58f90 [doc] Remove leftover from CONSOLE conversion 2016-08-31 13:14:47 -04:00
Ali Beyad 4641254ea6 Parameter improvements to Cluster Health API wait for shards (#20223)
* Params improvements to Cluster Health API wait for shards

Previously, the cluster health API used a strictly numeric value
for `wait_for_active_shards`. However, with the introduction of
ActiveShardCount and the removal of write consistency level for
replication operations, `wait_for_active_shards` is used for
write operations to represent values for ActiveShardCount. This
commit moves the cluster health API's usage of `wait_for_active_shards`
to be consistent with its usage in the write operation APIs.

This commit also changes `wait_for_relocating_shards` from a
numeric value to a simple boolean value `wait_for_no_relocating_shards`
to set whether the cluster health operation should wait for
all relocating shards to complete relocation.

* Addresses code review comments

* Don't be lenient if `wait_for_relocating_shards` is set
2016-08-31 11:58:19 -04:00
Greg Ichneumon Brown 92c54aa4a1 Docs: clarify scale is applied at origin+offest (#20242)
- fixes #19832
2016-08-31 17:02:59 +02:00
Jason Tedor e166459bbe Merge branch 'master' into log4j2
* master:
  Increase visibility of deprecation logger
  Skip transport client plugin installed on JDK 9
  Explicitly disable Netty key set replacement
  percolator: Fail indexing percolator queries containing either a has_child or has_parent query.
  Make it possible for Ingest Processors to access AnalysisRegistry
  Allow RestClient to send array-based headers
  Silence rest util tests until the bogusness can be simplified
  Remove unknown HttpContext-based test as it fails unpredictably on different JVMs
  Tests: Improve rest suite names and generated test names for docs tests
  Add support for a RestClient base path
2016-08-31 10:59:27 -04:00
Jason Tedor 1a805bb675 Increase visibility of deprecation logger
The deprecation logger is an important way to make visible features of
Elasticsearch that are deprecated. Yet, the default logging makes the
log messages for the deprecation logger invisible. We want these log
messages to be visible, so the default logging for the deprecation
logger should enable these log messages. This commit changes the log
level of deprecation log message to warn, and configures the deprecation
logger so that these log messages are visible out of the box.

Relates #20254
2016-08-31 10:51:17 -04:00
Jack Conradson 2b9c3a1a98 Fixed a doc test. 2016-08-30 16:51:37 -07:00
Jack Conradson c8f57df5cf Merge branch 'master' into deprecate 2016-08-30 09:11:18 -07:00
Jack Conradson 7930233527 Deprecate Groovy, Python, and Javascript scripts. 2016-08-30 09:06:18 -07:00
Chris Earle 6ad92c0f9d [DOCS] Document performRequest being renamed to performRequestAsync
This updates the docs to reflect that the asynchronous variants were renamed to have "Async" at the end.
2016-08-30 11:29:52 -04:00
Nik Everett df73292256 Add an alias action to delete an index
While removing an index isn't actually an alias action, if we add
an alias action that deletes an index then we can delete and index
and add an alias with the same name as the index atomically, in
the same cluster state update.

Closes #20064
2016-08-30 10:15:21 -04:00
Clinton Gormley 70f4d718f8 Update percolate.asciidoc
Tidy up percolator docs
2016-08-30 13:05:02 +02:00
Jim Ferenczi accb636824 Merge pull request #20213 from jimferenczi/painless_list_add
Fix docs that uses += to add an element in a list even though painless does not accept it.
2016-08-30 09:33:59 +02:00
Tanguy Leroux b4245c7ad9 Add exclusion filters support to filter_path
This commit adds the support for exclusion filter to the response filtering (filter_path) feature. It changes the XContentBuilder APIs so that it now accepts two types of filters: inclusive and exclusive. Filters are no more String arrays but sets of String instead.
2016-08-30 09:08:30 +02:00
Nik Everett 9c3f6d58ac Support downgrading keyword/text into string
This changes Elasticsearch to automatically downgrade `text` and
`keyword` fields into appropriate `string` fields when changing the
mapping of indexes imported from 2.x. This allows users to use the
modern, documented syntax against 2.x indexes. It also makes it clear
that reindexing in order to recreate the index in 5.0 is required for
any long lived indexes. This change is useful for the times when you
can't (cluster is just starting, not stable enough for reindex) or
shouldn't (index will only live 90 days or something).
2016-08-29 11:27:37 -04:00
Jim Ferenczi dc663a432b Fix docs that uses += to add an element in a list even though painless does not accept it. 2016-08-29 16:00:11 +02:00
markwalkom a8e7d57a84 Update api-conventions.asciidoc (#20203) 2016-08-29 15:10:20 +02:00
Nik Everett c24ad1df4e Mark lots of stuff `// NOTCONSOLE` 2016-08-29 08:28:16 -04:00
Nik Everett 5357d9d9ce Add `// TESTRESPONSE` to mapper-attachements docs 2016-08-29 08:28:16 -04:00
Munish Goyal 81b815ff76 Correct grammar in parent field doc 2016-08-29 07:51:39 -04:00
Jun Ohtani 2a00c9dc46 Merge pull request #19860 from johtani/fix/validate_empty_field_name
Validate blank field name
2016-08-29 11:52:18 +09:00
Nik Everett 3fe42beb64 Cleanup some docs
Mark one `// NOTCONSOLE`, mark another `[source,painless]`, and
another `// TESTRESPONSE` and fix a bug in it.
2016-08-26 16:01:07 -04:00
Nik Everett 52f23918c2 Use `painless` as language for painless snippets (#20185)
The syntax highlighter does a decent job when you do this. This lets
us `grep` for painless snippets in the docs.

Closes #20025
2016-08-26 15:39:44 -04:00
Nicolas Ruflin 4ab1093564 Add reindex example on how to reindex daily indices (#18654)
This can be a common case with beats in case the template changes between two versions and the old data should be reindex with the new templates.
2016-08-26 13:08:52 -04:00
Josh Becker 3c24ea43fd Docs: Remove extra word from phrase-suggester 2016-08-26 13:02:54 -04:00
Jun Ohtani 450f47d5b5 Validate blank field name
add validation and validate only 5.0+
Add tests before 5.0

Closes #19251
2016-08-26 20:10:33 +09:00
Martijn van Groningen 7c9af98a3c docs: add sort workaround 2016-08-26 10:55:42 +02:00
Dominik Stadler f0db4d9942 Add an example call of how to stop a snapshot or restore operation (#20153) 2016-08-25 13:01:04 +02:00
Igor Motov b36fbc4452 Add support for parameters to the script ingest processor
The script processor should support `params` to be consistent with all other script consumers.
2016-08-24 16:49:48 -04:00
Jim Ferenczi 9bedbbaa6a Fixed doc links 2016-08-24 22:37:59 +02:00
Jim Ferenczi 4682fc34ae Add the ability to disable the retrieval of the stored fields entirely
This change adds a special field named _none_ that allows to disable the retrieval of the stored fields in a search request or in a TopHitsAggregation.

To completely disable stored fields retrieval (including disabling metadata fields retrieval such as _id or _type) use _none_ like this:

````
POST _search
{
   "stored_fields": "_none_"
}
````
2016-08-24 16:40:08 +02:00
Clinton Gormley 9cce45d4af Add cloud to S3/EC2 plugin docs 2016-08-24 15:38:43 +02:00
Simon Willnauer c499427166 Use _refresh instead of reading from Translog in the RT GET case (#20102)
Today we do a lot of accounting inside the engine to maintain locations
of documents inside the transaction log. This is only needed to ensure
we can return the documents source from the engine if it hasn't been refreshed.
Aside of the added complexity to be able to read from the currently writing translog,
maintainance of pointers into the translog this also caused inconsistencies like different values
of the `_ttl` field if it was read from the tlog or not. TermVectors are totally different if
the document is fetched from the tranlog since copy fields are ignored etc.

This chance will simply call `refresh` if the documents latest version is not in the index. This
streamlines the semantics of the `_get` API and allows for more optimizations inside the engine
and on the transaction log. Note: `_refresh` is only called iff the requested document is not refreshed
yet but has recently been updated or added.

#Relates to #19787
2016-08-24 15:30:08 +02:00
Jason Tedor 90c2aae28b Remove minimum master nodes bootstrap docs
The minimum master nodes bootstrap check was removed in
069fc22696 but the docs were left behind.
This commit removes these stale docs.

Relates #20127
2016-08-23 14:31:58 -04:00
ddddn 6228b002c5 Update index_.asciidoc (#20125) 2016-08-23 20:08:52 +02:00
Nik Everett 5b34bec92a Add deprecation warnings to docs for geohash
Relates to #20126
2016-08-23 13:43:35 -04:00
Lee Hinman 3298a4ed38 Revert "Merge remote-tracking branch 'dakrone/exclude-numerics-from-all'"
This reverts commit 514585290c, reversing
changes made to 8563c8d897.
2016-08-23 09:24:33 -06:00
Nicholas Knize 8234fad9ca Deprecate geohash parameters for geo_point parser
This commit deprecates all geohash parameters in the geo_point field parser.
2016-08-23 09:19:21 -05:00
Nicholas Knize 28ed0e7abf Deprecate optimize_bbox on geodistance queries
Deprecates the optimize_bbox parameter on geodistance queries. This has no longer been needed since version 2.2 because lucene geo distance queries (postings and LatLonPoint) already optimize by bounding box.
2016-08-23 09:14:54 -05:00
Simon Willnauer d685847b73 Use `refresh=true` in mapping/fields examples (#20120)
Fix field examples to make documents actually visible

This commit adds refresh calls to field examples an removes not working
`_routing` and `_field_names` script access.

Closes #20118
2016-08-23 13:32:14 +02:00
Jack Conradson 131e370a16 Make Painless the default scripting language.
Closes #20017
2016-08-22 17:38:02 -07:00
Lee Hinman 514585290c Merge remote-tracking branch 'dakrone/exclude-numerics-from-all' 2016-08-22 12:36:25 -06:00
Munish Goyal f9c17dd976 Correct sentence (#20088) 2016-08-22 16:20:14 +02:00
Jim Ferenczi 4bee565535 Fix docs stating that index.mapper.dynamic can be set for all nodes in the elasticsearch.yml file. This is not supported in 5.x (index settings cannot be set at the cluster level) and should be replace with a template for all indices. 2016-08-22 10:20:43 +02:00
Lee Hinman b6ec1ae6eb Rework documentation example for _all to be less ambigious with numerics 2016-08-19 16:44:38 -06:00
Lee Hinman d7e516c0b4 Default `include_in_all` for numeric-like types to false
This includes:

- All regular numeric types such as int, long, scaled-float, double, etc
- IP addresses
- Dates
- Geopoints and Geoshapes

Relates to #19784
2016-08-19 15:50:38 -06:00
Tanguy Leroux 807da19150 [Doc] Update plugins intro about categories of plugins core/community 2016-08-19 14:12:10 +02:00
Adrien Grand a4ea7e7223 Switch indices.exists_type from `{index}/{type}` to `{index}/_mapping/{type}`. #20055
This will help remove types as we will need `{index}/{id}` to tell whether a
document exists.

Relates #15613
2016-08-19 09:18:24 +02:00
Clinton Gormley de208cf78c Fied bad asciidoc 2016-08-18 14:08:58 +02:00
Munish Goyal 0ee3a479e9 Update wildcard-query.asciidoc (#20057)
Update sentence grammar
2016-08-18 14:04:46 +02:00
Clinton Gormley 31e5e0b17f Document that pipeline aggs cannot be used for sorting
Closes #20037
2016-08-18 13:52:45 +02:00
Lee Hinman f6b166f19e Merge remote-tracking branch 'dakrone/forbid-simpleregex-in-index-name' 2016-08-17 16:01:09 -06:00
Lee Hinman 6030acb43b Disallow creating indices starting with '-' or '+'
Previously this was possible, which was problematic when issuing a
request like `DELETE /-myindex`, which was interpretted as "delete
everything except for myindex".

Resolves #19800
2016-08-17 15:13:03 -06:00
Jason Tedor d4dec26aa0 Update max local storage nodes docs
This commit updates the max local storage nodes docs to reflect that the
default is now one after 1f0673c9bd.

Relates #20029
2016-08-17 12:13:09 -04:00
Nik Everett 066afcf3c3 Add NOTCONSOLE to a few of the docs
These are docs for the java client and console doesn't make much
sense there.
2016-08-17 10:13:06 -04:00
Nik Everett f27a7f4c1a Test response snippets in kuromoji docs
Relates to #18160
2016-08-17 09:56:00 -04:00
Nik Everett c66db9a81e Add `// CONSOLE` to much of pipeline agg docs
Most of the examples in the pipeline aggregation docs use a small
"sales" test data set and I converted all of the examples that use
it to `// CONSOLE`. There are still a bunch of snippets in the pipeline
aggregation docs that aren't `// CONSOLE` so they aren't tested. Most
of them are "this is the most basic form of this aggregation" so they
are more immune to errors and bit rot then the examples that I converted.
I'd like to do something with them as well but I'm not sure what.

Also, the moving average docs and serial diff docs didn't get a lot of
love from this pass because they don't use the test data set or follow
the same general layout.

Relates to #18160
2016-08-17 09:26:41 -04:00
markwalkom f556424ab9 Update synonym-tokenfilter.asciidoc (#19988)
* Update synonym-tokenfilter.asciidoc

* Update synonym-tokenfilter.asciidoc
2016-08-17 13:39:22 +02:00
Adrien Grand d894db1590 Only use `PUT` for index creation, not POST. #20001
Currently both `PUT` and `POST` can be used to create indices. This commit
removes support for `POST index_name` so that we can use it to index documents
with auto-generated ids once types are removed.

Relates #15613
2016-08-17 10:15:42 +02:00
polyfractal 2c1b9b67db [DOCS] Add missing index to test setup
#19884 fixed the aliases, but didn't add `index2` to the test setup which causes the
alias command to fail
2016-08-16 11:55:56 -04:00
Ludo eea1bc719b Swap a index for the same alias (#19884)
In the example there was a alias removed and then a different alias created for the same index, but I think actually swapping a index by another one for the same alias would make more sense as an example here.
2016-08-16 15:24:42 +02:00
Nicholas Knize a93af8651c add geo distance script breaking changes to migration docs 2016-08-15 19:12:24 -05:00
Nik Everett 7542ef3173 [docs] Don't allow `[source,javascript]`
The syntax highlighter doesn't support it. Just use `js`.
2016-08-12 17:08:43 -04:00
Nik Everett ffd226efa0 Add `// NOTCONSOLE` to docs
We have 1074 snippets that look like they should be converted to
`// CONSOLE`. At least that is what `gradle docs:listConsoleCandidates`
says. This adds `// NOTCONSOLE` to explicitly mark snippets that
*shouldn't* be converted to `// CONSOLE`. After marking the blindingly
obvious ones this cuts the remaining snippet count to 1032.
2016-08-12 16:49:57 -04:00
Clinton Gormley 15c96df5b5 Replaced "true" with true in snapshot restore docs
Closes #19947
2016-08-12 16:56:56 +02:00
Jason Tedor 1f0673c9bd Default max local storage nodes to one
This commit defaults the max local storage nodes to one. The motivation
for this change is that a default value greather than one is dangerous
as users sometimes end up unknowingly starting a second node and start
thinking that they have encountered data loss.

Relates #19964
2016-08-12 09:26:20 -04:00
Nik Everett 9f8f2ea54b Remove ESIntegTestCase#pluginList
It was a useful method in 1.7 when javac's type inference wasn't as
good, but now we can just replace it with `Arrays.asList`.
2016-08-11 15:44:02 -04:00
Gytis Šk 8a97f05e41 Fix typos in inner-hits documentation (#19910) 2016-08-11 21:15:11 +02:00
Tal Levy bf046f8f93 update ingest date index name processor with runnable CONSOLE examples (#19935) 2016-08-11 11:36:14 -07:00
Colin Goodheart-Smithe 2904562b01 [DOCS] Fix shard request cache docs
Docs have been changed to reflect the fact that shard request cache is now enabled by default

Closes #19695
2016-08-11 14:25:34 +01:00
Clinton Gormley 2e3bc656e6 Update inner-hits.asciidoc
Typo

Closes #19775
2016-08-11 12:36:31 +02:00
Nik Everett 3793d0573e Add complete examples to ingest-geoip docs
Adds `// CONSOLE` and example responses to both make usage more clear
and to test the snippets.

Relates to #18160
2016-08-10 17:13:15 -04:00
David Pilato acc50d5817 Update documentation after merge with master 2016-08-10 19:07:22 +02:00
Nik Everett f204ad026c Add complete examples to some ingest docs
These examples should make it more clear what the plugins do and they
test that the snippets actually work.

Relates to #18160
2016-08-10 12:09:25 -04:00
Jason Tedor e899e8b4e0 Reword expect header bug notice
This commit rewords the expect header bug notice to provide the precise
details for the bug arising. In particular, the bug does not impact any
request over 1024 bytes, but instead impacts any request with a body
that is sent in two requests, the first with an Expect: 100-continue
header. The size is irrelevant, and requests with bodies larger than
1024 bytes are okay as long as the Expect: 100-continue header is not
also sent.

Relates #19911
2016-08-10 10:42:58 -04:00
Thomas Decaux bf2e5cb988 [docs] Remove extra "s" at buckets_path snippet
Closes #19907
2016-08-10 08:56:00 -04:00
Clinton Gormley 83532ac377 Documented netty4 Expect bug in release notes and breaking changes 2016-08-10 10:16:25 +02:00
Pius 40f3b5ab76 Update snapshots.asciidoc
>However, the version of the new cluster should be the same or newer than the cluster that was

Afaik, you can't restore a snapshot to a newer cluster that is not consecutively newer (i.e. can't restore 1.x snapshot to a 5.x cluster).  This is to clarify the statement above moving forward.
2016-08-09 18:22:29 -07:00
Deb Adair c522568d1b Docs: Fixed typos in example buckets_paths > buckets_path. 2016-08-09 14:37:37 -07:00
Areek Zillur 16d93e5a53 Merge pull request #19877 from areek/fix/remove_completion_payload
Remove payload option from completion suggester
2016-08-09 15:27:29 -04:00
Lee Hinman 5849c488b5 Merge remote-tracking branch 'dakrone/compliation-breaker' 2016-08-09 11:57:26 -06:00
Ali Beyad d915230a06 Clarifies the documentation for the `http.cors.enabled` setting (#19890)
Clarifies the documentation for the `http.cors.enabled` setting
2016-08-09 12:54:38 -05:00
Lee Hinman 2be52eff09 Circuit break the number of inline scripts compiled per minute
When compiling many dynamically changing scripts, parameterized
scripts (<https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html#prefer-params>)
should be preferred. This enforces a limit to the number of scripts that
can be compiled within a minute. A new dynamic setting is added -
`script.max_compilations_per_minute`, which defaults to 15.

If more dynamic scripts are sent, a user will get the following
exception:

```json
{
  "error" : {
    "root_cause" : [
      {
        "type" : "circuit_breaking_exception",
        "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
        "bytes_wanted" : 0,
        "bytes_limit" : 0
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "i",
        "node" : "a5V1eXcZRYiIk8lecjZ4Jw",
        "reason" : {
          "type" : "general_script_exception",
          "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
          "caused_by" : {
            "type" : "circuit_breaking_exception",
            "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
            "bytes_wanted" : 0,
            "bytes_limit" : 0
          }
        }
      }
    ],
    "caused_by" : {
      "type" : "general_script_exception",
      "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
      "caused_by" : {
        "type" : "circuit_breaking_exception",
        "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
        "bytes_wanted" : 0,
        "bytes_limit" : 0
      }
    }
  },
  "status" : 500
}
```

This also fixes a bug in `ScriptService` where requests being executed
concurrently on a single node could cause a script to be compiled
multiple times (many in the case of a powerful node with many shards)
due to no synchronization between checking the cache and compiling the
script. There is now synchronization so that a script being compiled
will only be compiled once regardless of the number of concurrent
searches on a node.

Relates to #19396
2016-08-09 10:26:27 -06:00
Clinton Gormley 1760e00489 Bumped version in docs 2016-08-09 15:57:35 +02:00
Areek Zillur d107141bf6 Remove payload option from completion suggester
The payload option was introduced with the new completion
suggester implementation in v5, as a stop gap solution
to return additional metadata with suggestions.

Now we can return associated documents with suggestions
(#19536) through fetch phase using stored field (_source).
The additional fetch phase ensures that we only fetch
the _source for the global top-N suggestions instead of
fetching _source of top results for each shard.
2016-08-08 16:04:06 -04:00
David Pilato 97dfa2ba40 Fix typo
Reported at https://discuss.elastic.co/t/little-error-in-documentation-page-mapping-parameters-format/57424
2016-08-08 10:52:09 +02:00
javanna 73d0a1b777 [DOCS] clarify behaviour when routing is required and no routing value is specified
This note in the delete api about broadcasting to all shards is a leftover that should have been removed when the broadcasting feature was removed

Relates to #10136
2016-08-08 10:41:59 +02:00
Nicholas Knize 2d590af593 Deprecate GeoDistance enumerators and remove geo distance script helpers
GeoDistance is implemented using a crazy enum that causes issues with the scripting modules. This commit moves all distance calculations to arcDistance and planeDistance static methods in GeoUtils. It also removes unnecessary distance helper methods from ScriptDocValues.GeoPoints.
2016-08-05 18:42:06 -05:00
Areek Zillur 469eb2546d Merge pull request #19536 from areek/enhancement/completion_suggester_documents
Add support for returning documents with completion suggester
2016-08-05 18:55:08 -04:00
Areek Zillur fee013c07c Add support for returning documents with completion suggester
This commit enables completion suggester to return documents
associated with suggestions. Now the document source is returned
with every suggestion, which respects source filtering options.

In case of suggest queries spanning more than one shard, the
suggest is executed in two phases, where the last phase fetches
the relevant documents from shards, implying executing suggest
requests against a single shard is more performant due to the
document fetch overhead when the suggest spans multiple shards.
2016-08-05 17:51:45 -04:00
Nik Everett 1e587406d8 Fail yaml tests and docs snippets that get unexpected warnings
Adds `warnings` syntax to the yaml test that allows you to expect
a `Warning` header that looks like:
```
    - do:
        warnings:
            - '[index] is deprecated'
            - quotes are not required because yaml
            - but this argument is always a list, never a single string
            - no matter how many warnings you expect
        get:
            index:    test
            type:    test
            id:        1
```

These are accessible from the docs with:
```
// TEST[warning:some warning]
```

This should help to force you to update the docs if you deprecate
something. You *must* add the warnings marker to the docs or the build
will fail. While you are there you *should* update the docs to add
deprecation warnings visible in the rendered results.
2016-08-04 15:23:05 -04:00
Clinton Gormley 2cceb0a5f4 Updated v5.0.0-alpha5 release notes 2016-08-04 17:17:53 +02:00
Ryan Biesemeyer 9f1525255a Update link to mapper-murmur3 plugin in card docs (#19788) 2016-08-04 15:56:59 +02:00
Jason Tedor 533412e36f Improve cat thread pool API
Today, when listing thread pools via the cat thread pool API, thread
pools are listed in a column-delimited format. This is unfriendly to
command-line tools, and inconsistent with other cat APIs. Instead,
thread pools should be listed in a row-delimited format.

Additionally, the cat thread pool API is limited to a fixed list of
thread pools that excludes certain built-in thread pools as well as all
custom thread pools. These thread pools should be available via the cat
thread pool API.

This commit improves the cat thread pool API by listing all thread pools
(built-in or custom), and by listing them in a row-delimited
format. Finally, for each node, the output thread pools are sorted by
thread pool name.

Relates #19721
2016-08-03 23:02:13 -04:00
debadair bcc5c7c07a Docs: Fixed callout error that broke the build. 2016-08-03 17:20:00 -07:00
Nik Everett 3be1e7ec35 CONSOLify the completion suggester docs (#19758)
* CONSOLEify search/suggesters/completion
* CONSOLEify context suggester docs
2016-08-03 18:40:17 -04:00
Mary fa3420c2a5 Update term-level-queries.asciidoc
Typo fix
2016-08-03 10:18:13 -06:00
Ali Beyad 4f70ee521f Migration Guide changes for BlobContainer (#19731)
Adds a notice in the migration guide for removing
two deleteBlobs and one writeBlob method from the
BlobContainer interface.
2016-08-03 10:41:25 -04:00
Adrien Grand a0818d3b87 Split regular histograms from date histograms. #19551
Currently both aggregations really share the same implementation. This commit
splits the implementations so that regular histograms can support decimal
intervals/offsets and compute correct buckets for negative decimal values.

However the response API is still the same. So for intance both regular
histograms and date histograms will produce an
`org.elasticsearch.search.aggregations.bucket.histogram.Histogram`
aggregation.

The optimization to compute an identifier of the rounded value and the
rounded value itself has been removed since it was only used by regular
histograms, which now do the rounding themselves instead of relying on the
Rounding abstraction.

Closes #8082
Closes #4847
2016-08-03 08:39:48 +02:00
Lee Hinman 0ade5a207d Add documentation for the 'elasticsearch-translog' tool
This adds documentation to the translog page for the CLI truncation
tool.
2016-08-02 16:26:28 -06:00
Luca Cavanna c0b71d26cf Merge pull request #19687 from javanna/docs/java_api_version
[DOCS] replace {es.version} in maven coordinates with current version
2016-08-02 15:31:08 +02:00
javanna 772c16702e [DOCS] replace {es.version} in maven coordinates with current version 2016-08-02 15:17:21 +02:00
Ali Beyad a21dd80f1b Documentation changes for wait_for_active_shards (#19581)
Documentation changes and migration doc changes for introducing 
wait_for_active_shards and removing write consistency level.

Closes #19581
2016-08-02 09:15:01 -04:00
Ali Beyad a7e68e36cf Fixes header level for migration doc entry 2016-08-01 16:15:12 -04:00
Adrien Grand a4cb63b98c Remove `_missing_` from the docs.
It is removed in 5.0, see #15153.
2016-08-01 16:57:37 +02:00
Alexander Lin 9ac6389e43 Rename operation to result and reworking responses
* Rename operation to result and reworking responses
* Rename DocWriteResponse.Operation enum to DocWriteResponse.Result

These are just easier to interpret names.

Closes #19664
2016-08-01 10:42:58 -04:00
Isabel Drost-Fromm 672ffb6e4d Revert "Add console to docs for inner hits, explain, and friends" 2016-08-01 14:09:54 +02:00
Isabel Drost-Fromm 5104437e4f Merge pull request #18519 from MaineC/docs/add_console_to_search
Add console to docs for inner hits, explain, and friends
2016-08-01 12:10:09 +02:00