Commit Graph

3809 Commits

Author SHA1 Message Date
Boaz Leskes 70a3ac1767 Add a note about `cluster.routing.allocation.node_concurrent_recoveries` (#23160)
Closes #23152
2017-02-14 14:14:41 +02:00
Loek van Gool 214a3536aa Update redirects.asciidoc (#23148) 2017-02-13 16:23:25 +01:00
Giuseppe ecbeffcb1e Add note about min_score filtering efficiency (#23109)
* Add note about min_score filtering efficiency

* Reword to mention 'HAVING'

* Remove reference to HAVING
2017-02-13 12:15:01 +01:00
Adrien Grand f3509b8003 Consolify docs/reference/analysis/tokenfilters/pattern-capture-tokenfilter.asciidoc. (#23050) 2017-02-13 11:00:12 +01:00
Ryan Ernst c91848e6a7 Docs: Consoleify cluster and indices settings docs (#23030)
relates #23001
2017-02-10 14:57:43 -08:00
Tanguy Leroux e2e5937455 Use `typed_keys` parameter to prefix suggester names by type in search responses (#23080)
This pull request reuses the typed_keys parameter added in #22965, but this time it applies it to suggesters. When set to true, the suggester names in the search response will be prefixed with a prefix that reflects their type.
2017-02-10 10:53:38 +01:00
Tanguy Leroux 63ea6f7168 [Docs] Remove unnecessary // TEST[continued] in search-template doc
It has been explained in e39b96f257
2017-02-10 10:08:24 +01:00
Clinton Gormley d43417ef47 Docs: Deleted redundant word in scripting 2017-02-09 22:02:42 +01:00
Jonathan D Strootman cb35b3785a Adding `ansible-elasticsearch` to list of CM tools (#23058) 2017-02-09 21:14:30 +01:00
Clinton Gormley 78d3028bb7 Fixed bad asciidoc in delete-by-query 2017-02-09 20:14:56 +01:00
Jim Ferenczi 94087b3274 Removes ExpandCollapseSearchResponseListener, search response listeners and blocking calls
This changes removes the SearchResponseListener that was used by the ExpandCollapseSearchResponseListener to expand collapsed hits.
The removal of SearchResponseListener is not a breaking change because it was never released.
This change also replace the blocking call in ExpandCollapseSearchResponseListener by a single asynchronous multi search request. The parallelism of the expand request can be set via CollapseBuilder#max_concurrent_group_searches

Closes #23048
2017-02-09 18:06:10 +01:00
Tanguy Leroux 3553522328 Add parameter to prefix aggs name with type in search responses (#22965)
This pull request adds a new parameter to the REST Search API named `typed_keys`. When set to true, the aggregation names in the search response will be prefixed with a prefix that reflects the internal type of the aggregation.

Here is a simple example:
```
GET /_search?typed_keys
{
    "aggs": {
        "tweets_per_user": {
            "terms": {
                "field": "user"
            }
        }
    },
    "size": 0
}
```

And the response:

```
{
    "aggs": {
        "sterms:tweets_per_user": {
            ...
        }
    }
}
```

This parameter is intended to make life easier for REST clients that could parse back the prefix and could detect the type of the aggregation to parse. It could also be implemented for suggesters.
2017-02-09 11:19:04 +01:00
Tanguy Leroux 832952cb29 [Docs] Fix consoleify search-template.asciidoc
It does not reproduce well, hopefully this will fix the failure on DELETE _search/template/<templatename>.
2017-02-08 21:23:38 +01:00
Igor Motov 1fc4fa5729 Docs: CONSOLEify native script docs
Relates #23001
2017-02-08 13:30:39 -05:00
Jay Modi 7f3769c745 Remove ldjson support and document ndjson for bulk/msearch (#23049)
This commit removes support for the `application/x-ldjson` Content-Type header as this was only used in the first draft
of the spec and had very little uptake. Additionally, the docs for bulk and msearch have been updated to specifically
call out ndjson and mention that the newline character may be preceded by a carriage return.

Finally, the bulk request handling of the carriage return has been improved to remove this character from the source.

Closes #23025
2017-02-08 11:55:50 -05:00
Clinton Gormley 40f40d7676 Docs: Fix termvectors by removing example blocks with embedded CONSOLE tests 2017-02-08 17:12:40 +01:00
Tanguy Leroux 477d1aa8bf [Docs] Consoleify multi-search and search-template docs (#23047)
Relates #23001
2017-02-08 17:05:22 +01:00
Christoph Büscher e177d2ca40 Docs: CONSOLEify termvectors.asciidoc (#23046) 2017-02-08 16:06:11 +01:00
Daniel Mitterdorfer 88c5627a1b CONSOLEify multi-termvectors docs
Relates #23001
2017-02-08 11:47:42 +01:00
Yannick Welsch 9154686623 Remove legacy primary shard allocation mode based on versions (#23016)
Elasticsearch v5.0.0 uses allocation IDs to safely allocate primary shards whereas prior versions of ES used a version-based mode instead. Elasticsearch v5 still has support for version-based primary shard allocation as it needs to be able to load 2.x shards. ES v6 can drop the legacy support.
2017-02-08 10:00:55 +01:00
Lee Hinman b3c27a7fdd Disallow include_in_all for 6.0+ indices
Since `_all` is now deprecated and cannot be set for new indices, we should also
disallow any field that has the `include_in_all` parameter set.

Resolves #22923
2017-02-07 19:31:51 -07:00
Tim Brooks ad4bfa2307 Docs: CONSOLEify transport docs (#23027)
This is related to #23001.
2017-02-07 20:06:28 -06:00
Jordan Robinson 693b0017af Small typo fix in Windows service documentation
This commit removes a duplicate definite article in the Windows service
documentation.

Relates #23028
2017-02-07 17:25:46 -05:00
Nik Everett 0e98c9107a Docs: CONSOLEify some more docs
These need to be CONSOLEified *now* because we're starting to
require Content-Type headers and they didn't have any.

* cluster/reroute: Marked as CONSOLE but skipped because the docs
build runs with a single node.
* docs/bulk: Marked as NOTCONSOLE because the snippets describe
either examples or `curl` commands. Fixed the `curl` command to
include the `Content-Type` header.
* query-dsl/terms-query: Marked as CONSOLE.
* search/request/rescore: Marked as CONSOLE. Fixed deprecated
syntax.

Relates #23001
Relates #18160
2017-02-07 16:49:01 -05:00
Nik Everett 0c011cb290 Docs: CONSOLEify histogram aggregation docs
This adds the `COPY AS CURL` and `VIEW IN CONSOLE` links to the docs
and causes the snippets to be tested during Elasticsearch's build.

Relates to #18160
2017-02-07 16:09:32 -05:00
Nik Everett 245aa0404a Docs: CONSOLEify sum aggregation docs
This adds the `COPY AS CURL` and `VIEW IN CONSOLE` buttons to the
docs and makes the build execute the snippets as part of `docs:check`.

Relates to #18160
2017-02-07 14:18:54 -05:00
Jim Ferenczi bbf62e3472 CONSOLify indices/analyze.asciidoc and search/field-stats.asciidoc
Relates #23001
2017-02-07 20:15:09 +01:00
Nik Everett 274ee30d34 Docs: CONSOLEify the avg aggregation docs
This creates the `COPY AS CURL` and `VIEW IN CONSOLE` buttons and
makes the build test the examples.

Relates to #18160
2017-02-07 13:48:27 -05:00
Nik Everett a2ed676862 Docs: Explain painless's method dispatch (#23021)
Painless uses Ruby-like method dispatch (reciever type, method name,
and arity) rather than Java-like (reciever type, method name, and
argument compile time types) or Groovy-like method dispatch (receiver
type, method name, and argument run time types). We do this for
mostly good reasons but we never documented it.

Relates to #22720
2017-02-07 12:09:22 -05:00
Clinton Gormley f5e7c25e24 Update normalizers.asciidoc
analyzers -> normalizers
2017-02-07 12:09:39 +01:00
Simon Willnauer dc659feeb4 Add a setting to disable remote cluster connections on a node (#23005)
Today either all nodes in the cluster connect to remote clusters of only nodes
that have remote clusters configured in their node config. To allow global remote
cluster configuration but restrict connections to a set of nodes in the cluster
this change adds a new setting `search.remote.connect` (defaults to `true`) to allow
to disable remote cluster connections on a per node basis.
2017-02-07 09:59:24 +01:00
Nik Everett 0d6e622242 Make dates be ReadableDateTimes in scripts (#22948)
Instead of longs. If you want millis since epoch you can call doc.date_field.value.millis.

Relates to #22875
2017-02-06 16:44:56 -05:00
Nicholas Knize bc884c1e7b [Docs] Remove ignore_malformed from Geo Query DSL docs
This commit removes the ignore_malformed parameter from the Geo Query DSL documentation.
2017-02-06 14:27:15 -06:00
javanna b9cf6333bd [TEST] fix typo in cross cluster search docs 2017-02-05 15:56:45 +01:00
Clinton Gormley e181a020a9 Replaced absolute URLs in docs with attributes 2017-02-04 12:05:03 +01:00
Clinton Gormley c1be26f2e1 Centralised doc versions in docs/Versions.asciidoc 2017-02-04 11:16:19 +01:00
Lee Hinman 39e7c30912 Change certain replica failures not to fail the replica shard
This changes the way that replica failures are handled such that not all
failures will cause the replica shard to be failed or marked as stale.

In some cases such as refresh operations, or global checkpoint syncs, it is
"okay" for the operation to fail without the shard being failed (because no data
is out of sync). In these cases, instead of failing the shard we should simply
fail the operation, and, in the event it is a user-facing operation, return a
5xx response code including the shard-specific failures.

This was accomplished by having two forms of the `Replicas` proxy, one that is
for non-write operations that does not fail the shard, and one that is for write
operations that will fail the shard when an operation fails.

Relates to #10708
2017-02-03 14:39:46 -07:00
Nik Everett 70e3cce904 Fix name of `enable_position_increments` (#22895)
It was accidentally renamed `enabled_position_increment` in the cleanups
for 5.0. This adds `enable_position_increment` as a deprecated alias
so it will continue to work.
2017-02-03 16:28:27 -05:00
Nicholas Knize b1a6b227e1 Remove deprecated geo query parameters, and GeoPointDistanceRangeQuery
This commit removes the following queries and parameters (which were deprecated in 5.0):

* GeoPointDistanceRangeQuery
* coerce, and ignore_malformed for GeoBoundingBoxQuery, GeoDistanceQuery, GeoPolygonQuery, and GeoDistanceSort
2017-02-03 10:08:00 -06:00
Nicholas Knize f1e1975882 [DOCS] Add sloppy_arc and factor removal to 6.0 migration docs 2017-02-03 09:49:12 -06:00
AlexNodex fb8bdbc57a Update typo in date (#22955)
your example has yyy and it should be yyyy
2017-02-03 13:16:17 +01:00
Clinton Gormley 8ace37e214 Fix asciidoc in stored fields 2017-02-03 10:18:01 +01:00
Jim Ferenczi 4876448e39 Consilify get-field-mapping docs (#22936)
This change also removes the reference to the difference bewteen full name and index name.
They are always the same since 2.x and `name` does not refer anymore to `author.name` automatically.
A simple pattern must be used instead.
Remove redundant code that checks the field name twice.
2017-02-03 10:04:31 +01:00
Jun Ohtani 7ea457955d Merge pull request #22879 from johtani/fix_documentation_error_in_date_histogram
[Doc]Not support "M" time unit in offset param
2017-02-03 16:40:08 +09:00
Jay Modi 7520a107be Optionally require a valid content type for all rest requests with content (#22691)
This change adds a strict mode for xcontent parsing on the rest layer. The strict mode will be off by default for 5.x and in a separate commit will be enabled by default for 6.0. The strict mode, which can be enabled by setting `http.content_type.required: true` in 5.x, will require that all incoming rest requests have a valid and supported content type header before the request is dispatched. In the non-strict mode, the Content-Type header will be inspected and if it is not present or not valid, we will continue with auto detection of content like we have done previously.

The content type header is parsed to the matching XContentType value with the only exception being for plain text requests. This value is then passed on with the content bytes so that we can reduce the number of places where we need to auto-detect the content type.

As part of this, many transport requests and builders were updated to provide methods that
accepted the XContentType along with the bytes and the methods that would rely on auto-detection have been deprecated.

In the non-strict mode, deprecation warnings are issued whenever a request with body doesn't provide the Content-Type header.

See #19388
2017-02-02 14:07:13 -05:00
Nicholas Knize b41d5747f0 Reduce GeoDistance insanity
GeoDistance query, sort, and scripts make use of a crazy GeoDistance enum for handling 4 different ways of computing geo distance: SLOPPY_ARC, ARC, FACTOR, and PLANE. Only two of these are necessary: ARC, PLANE. This commit removes SLOPPY_ARC, and FACTOR and cleans up the way Geo distance is computed.
2017-02-02 12:39:42 -06:00
Nik Everett dacc150934 Expose multi-valued dates to scripts and document painless's date functions (#22875)
Implemented by wrapping an array of reused `ModuleDateTime`s that
we grow when needed. The `ModuleDateTime`s are reused when we
move to the next document.

Also improves the error message returned when attempting to modify
the `ScriptdocValues`, removes a couple of allocations, and documents
that the date functions are available in Painless.

Relates to #22162
2017-02-01 21:57:07 -05:00
Jack Conradson 3d2626c4c6 Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.

The new behavior is the following:

When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.

Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].

When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.

Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.

When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 13:27:02 -08:00
Jim Ferenczi f6d38d480a Integrate UnifiedHighlighter (#21621)
* Integrate UnifiedHighlighter

This change integrates the Lucene highlighter called "unified" in the list of supported highlighters for ES.
This highlighter can extract offsets from either postings, term vectors, or via re-analyzing text.
The best strategy is picked automatically at query time and depends on the field and the query to highlight.
2017-01-31 19:06:03 +01:00
Nik Everett f90051e6e0 Docs: Add a note about `<` and `>` in query_string
`<` and `>` can't be escaped at all in `query_string`. If we're not
going to fix that we should at least document it.

Relates to #21703
2017-01-31 12:23:18 -05:00
alamzeeshan a1cc683cff Updated document as per code change. (#22878)
Updated document as per this change : https://github.com/elastic/elasticsearch/pull/15235
2017-01-31 13:36:09 +01:00
Shubham Aggarwal d80e05f4d2 Minor fix in Create Index desc (#22877) 2017-01-31 12:53:15 +01:00
Shubham Aggarwal e07e4cc4dd Fix incorrect heading for Whitespace Tokenizer (#22883) 2017-01-31 12:51:37 +01:00
markharwood 9e8e556b08 Build fix for broken docs build 2017-01-31 10:27:06 +00:00
markharwood c0d525b108 [DOCS] [TEST] enhancement - added CONSOLE scripts for sampler aggs (#22869)
Added missing CONSOLE scripts to documentation for sampler and diversified_sampler aggs.
Includes new StackOverflow index setup in build.gradle

Closes #22746

* Formatting tweaks
2017-01-31 09:45:25 +00:00
Jun Ohtani 94933f9d19 [Doc]Not support "M" time unit in offset param 2017-01-31 18:23:38 +09:00
David Pilato 858333246d Merge branch 'pr/remove-azure-container-auto-creation'
# Conflicts:
#	docs/reference/migration/migrate_6_0/plugins.asciidoc
2017-01-31 09:05:43 +01:00
Ryan Ernst 003805e875 Docs: Add setup section for the keystore tool and secure settings (#22838) 2017-01-30 14:56:45 -08:00
Ryan Ernst cf7747c338 S3 Repository: Remove region setting (#22853)
This change removes the ability to set region for s3 repositories.
Endpoint should be used instead if a custom s3 location needs to be
used.

closes #22758
2017-01-30 14:34:59 -08:00
Jim Ferenczi e7e871acdd Fix link to keyword and numeric type 2017-01-30 13:57:28 +01:00
Mathieu Berube e0b8e45cc5 Fix typo - mergins to margins (#22839) 2017-01-30 13:52:32 +01:00
Clinton Gormley b5a0e59af4 Fixed bad asciidoc 2017-01-30 11:56:40 +01:00
Clinton Gormley 938f5194ef Include field-collapsing docs in request-body search 2017-01-30 11:47:12 +01:00
David Pilato 1898dc2554 Remove auto creation of container for azure repository
Follow up of #22857 where we deprecate automatic creation of azure containers.

BTW I found that the `AzureSnapshotRestoreServiceIntegTests` does not bring any value because it runs basically a Snapshot/Restore operation on local files which we already test in core.

So instead of trying to fix it to make it pass with this PR, I simply removed it.
2017-01-30 11:47:08 +01:00
Adrien Grand dc62255ddd Document upcoming scoring changes. (#22806) 2017-01-30 11:08:49 +01:00
Ryan Ernst fe4043c8ff S3 Repository: Remove bucket auto create (#22846)
closes #22761
2017-01-28 11:13:21 -08:00
Ryan Ernst c921bebc4a S3 Repository: Remove env var and sysprop credentials support (#22842)
These are deprecated in 5.x. This commit removes support for them in 6.0.
2017-01-27 13:43:16 -08:00
Clinton Gormley 19ce039d2d Update type-field.asciidoc
Wildcard type names are not supported
2017-01-27 17:50:28 +01:00
Nik Everett a383bc1be0 Add remaining generated painless API
This was generated by the generated added in
8a2d424d68 but not committed in that
change for review purposes.
2017-01-26 10:43:22 -05:00
Nik Everett 8a2d424d68 Generate reference links for painless API (#22775)
Adds "Appending B. Painless API Reference", a reference of all classes
and methods available from Painless. Removes links to java packages
because they contain methods that we don't expose and don't contain
methods that we do expose (the ones in Augmentation). Instead this
generates a list of every class and every exposed method using the same
type information available to the
interpreter/compiler/whatever-we-call-it. From there you can jump to
the relevant docs.

Right now you build all the asciidoc files by running
```
gradle generatePainlessApi
```

These files are expected to be committed because we build the docs
without running `gradle`.

Also changes the output of `Debug.explain` so that it is easy to
search for the class in the generated reference documentation.

You can also run it in an IDE safely if you pass the path to the
directory in which to generate the docs as the first parameter. It'll
blow away the entire directory an recreate it from scratch so be careful.

And then you can build the docs by running something like:
```
../docs/build_docs.pl --out ../built_docs/ --doc docs/reference/index.asciidoc --open
```

That is, if you have checked out https://github.com/elastic/docs in
`../docs`. Wait a minute or two and your browser will pop open in with
all of Elasticsearch's reference documentation. If you go to
`http://localhost:8000/painless-api-reference.html` you can see this
list. Or you can get there by following the links to `Modules` and
`Scripting` and `Painless` and then clicking the link in the paragraphs
below titled `Appendix B. Painless API Reference`.

I like having these in asciidoc because we can deep link to them from the
rest of the guide with constructs like
`<<painless-api-reference-Object-hashCode-0>>` and
`<<painless-api-reference->>` and we get link checking. Then the only
brittle link maintenance bit is the link generation for javadoc. Which
sucks. But I think it is important that we link to the methods directly
so they are easy to find.

Relates to #22720
2017-01-26 10:39:19 -05:00
Clinton Gormley f5a0d18c4c Docs: Cluster allocation explain should be on one page 2017-01-26 11:38:19 +01:00
Nik Everett d704a880e7 Add tests for top_hits aggregation (#22754)
Add unit tests for `TopHitsAggregator` and convert some snippets in
docs for `top_hits` aggregation to `// CONSOLE`.

Relates to #22278
Relates to #18160
2017-01-25 16:15:50 -05:00
Clinton Gormley 12f5309041 Docs: Made cross-cluster search experimental 2017-01-24 12:44:01 +01:00
Nik Everett ee264c6957 Fix parsing for `max_determinized_states` (#22749)
There was a typo in the `ParseField` declaration. I know
we want to port these parsers to `ObjectParser` eventually
but I don't have the energy for that today and want to get
this fixed.

Closes #22722
2017-01-23 11:57:43 -05:00
William Webber f1a902865f Update span-multi-term-query.asciidoc (#22733)
"term" is not actually a multi-term query (perhaps confusion with "term range")
2017-01-23 17:33:40 +01:00
William Webber abaf728882 "from" => "gte", "to" => "lte" in bool example (#22735) 2017-01-23 17:29:00 +01:00
Jason Tedor d9a3ffcea0 Add note regarding custom jvm.options
When users need to specify a custom location for configuration files,
they also need to specify a custom location for the jvm.options file yet
our docs are absent in this regard. This commit adds a note to the
rolling upgrade docs explaining this situation.

Relates #22747
2017-01-23 10:39:11 -05:00
Jim Ferenczi e48bc2eed7 Add field collapsing for search request (#22337)
* Add top hits collapsing to search request

The field collapsing is done with a custom top docs collector that "collapse" search hits with same field value.
The distributed aspect is resolve using the two passes that the regular search uses. The first pass "collapse" the top hits, then the coordinating node merge/collapse the top hits from each shard.

```
GET _search
{
   "collapse": {
      "field": "category",
   }
}
```

This change also adds an ExpandCollapseSearchResponseListener that intercepts the search response and expands collapsed hits using the CollapseBuilder#innerHit} options.
The retrieval of each inner_hits is done by sending a query to all shards filtered by the collapse key.

```
GET _search
{
   "collapse": {
      "field": "category",
      "inner_hits": {
	"size": 2
      }
   }
}
```
2017-01-23 16:33:51 +01:00
Nik Everett da8740128b Docs: CONSOLE-ify value_count aggregation docs
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the snippets
in the `value_count` docs and causes the build to execute the snippets
for testing.

Release #18160
2017-01-23 10:07:29 -05:00
markharwood 87495750ff Docs fix - Added missing link to new Adjacency-matrix agg 2017-01-23 10:18:30 +00:00
Nik Everett a99bddcc7e CONSOLE-ify filter aggregation docs
This adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the
snippet and causes the build to execute the snippet as a test.

Relates to #18160
2017-01-23 01:32:56 -05:00
Nik Everett 40e2645177 CONSOLE-ify date_range aggregation docs
This adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the
snippets in the docs for the `date_range` aggregation and tests
those snippets as part of the build.

Relates to #18160
2017-01-22 23:38:45 -05:00
Nik Everett f7524fbdef CONSOLE-ify date histogram docs
This adds the `VIEW IN SENSE` and `COPY AS CURL` links and has
the build automatically execute the snippets and verify that they
work.

Relates to #18160
2017-01-20 16:23:28 -05:00
Nik Everett c2a580304b CONSOLE-ify min and max aggregation docs
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the docs
and makes the build automatically test them.

Relates to #18160
2017-01-20 15:33:00 -05:00
Nik Everett 8c856eaa9f CONSOLE-ify global-aggregation.asciidoc
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the example
`global` aggregation. Also improves the example by adding a
non-`global` aggregation to compare it to.

Relates to #18160
2017-01-20 14:36:51 -05:00
Folusho Oladipo daf1f184d4 Corrected a plural verb to a singular one. (#22681) 2017-01-20 19:00:10 +01:00
Patryk Krawaczyński 42c0e8947f Fix duplicates from search.query (#22701)
search.query_current, search.query_time and search.query_total have wrong aliases.
2017-01-20 18:45:10 +01:00
markharwood f01784205f New AdjacencyMatrix aggregation
Similar to the Filters aggregation but only supports "keyed" filter buckets and automatically "ANDs" pairs of filters to produce a form of adjacency matrix.
The intersection of buckets "A" and "B" is named "A&B" (the choice of separator is configurable). Empty intersection buckets are removed from the final results.

Closes #22169
2017-01-20 15:49:31 +00:00
Yannick Welsch 881993de3a [Docs] Remove outdated info about enabling/disabling doc_values (#22694) 2017-01-19 17:33:40 +01:00
Daniel Mitterdorfer aece89d6a1 Make boolean conversion strict (#22200)
This PR removes all leniency in the conversion of Strings to booleans: "true"
is converted to the boolean value `true`, "false" is converted to the boolean
value `false`. Everything else raises an error.
2017-01-19 07:59:18 +01:00
Michael McCandless 1d1bdd476c Finish exposing FlattenGraphTokenFilter (#22667) 2017-01-18 11:05:34 -05:00
Boaz Leskes 1227044ddd Add a deprecation notice to shadow replicas (#22647)
Relates to #22024

On top of documentation, the PR adds deprecation loggers and deals with the resulting warning headers.

The yaml test is set exclude versions up to 6.0. This is need to make sure bwc tests pass until this is backported to 5.2.0 . Once that's done, I will change the yaml test version limits
2017-01-18 12:28:09 +01:00
Christoph Büscher 9ed867ea83 [DOCS] Fix inconsistent formatting for fieldnames in profile.asciidoc 2017-01-18 10:41:22 +01:00
Clinton Gormley e415170989 Docs: Fix link syntax in cross-cluster docs
Also tidy up settings list
2017-01-18 10:13:48 +01:00
Clinton Gormley 5d70df855d Docs: Cross-cluster search doc wasn't being included 2017-01-18 10:02:51 +01:00
Simon Willnauer 19f9cb307a Merge branch 'master' into feature/multi_cluster_search 2017-01-18 09:24:35 +01:00
Scott Somerville 372812da98 Allow an index to be partitioned with custom routing (#22274)
This change makes it possible for custom routing values to go to a subset of shards rather than
just a single shard. This enables the ability to utilize the spatial locality that custom routing can
provide while mitigating the likelihood of ending up with an imbalanced cluster or suffering
from a hot shard.

This is ideal for large multi-tenant indices with custom routing that suffer from one or both of
the following:
- The big tenants cannot fit into a single shard or there is so many of them that they will likely
end up on the same shard
- Tenants often have a surge in write traffic and a single shard cannot process it fast enough

Beyond that, this should also be useful for use cases where most queries are done under the context
of a specific field (e.g. a category) since it gives a hint at how the data can be stored to minimize
the number of shards to check per query. While a similar solution can be achieved with multiple
concrete indices or aliases per value today, those approaches breakdown for high cardinality fields.

A partitioned index enforces that mappings have routing required, that the partition size does not
change when shrinking an index (the partitions will shrink proportionally), and rejects mappings
that have parent/child relationships.

Closes #21585
2017-01-18 08:51:23 +01:00
Elijah 3b92179e09 Improve wording in recipes docs
This commit improves some of the wording the recipes docs.

Relates #22661
2017-01-17 21:00:36 -05:00
Elijah b56605d68c Correct grammar in list in how-to docs
This commit corrects the grammar in a list in the how-to docs; namely,
the word "and" was missing preceding the final element in a list.

Relates #22663
2017-01-17 20:57:22 -05:00
Ali Beyad 099d229138 [TEST] fix documentation checking tests to account for possible
pending tasks in the cluster state
2017-01-17 17:29:14 -05:00
Elijah 297b1b7d9a Capitalize "Elasticsearch" in indexing speed docs
This commit fixes the capitalization of "Elasticsearch" in the indexing
speed docs.

Relates #22659
2017-01-17 12:33:01 -05:00
Alex a0c83c4511 Minor doc changes to clarify mapping index param for string type (#22652)
* Grammatical correction

* Add note for legacy string mapping type

* Update truncate token filter to not mention the keyword tokenizer

The advice predates the existence of the keyword field

Closes #22650
2017-01-17 16:43:11 +01:00
Simon Willnauer 709cb9a39e Merge branch 'master' into feature/multi_cluster_search 2017-01-17 12:34:36 +01:00
Clinton Gormley 401438819e Docs: Fix the first highlighting example to work
Closes #22642
2017-01-17 12:20:03 +01:00
Clinton Gormley 519a9c469d Update truncate token filter to not mention the keyword tokenizer
The advice predates the existence of the keyword field

Closes #22650
2017-01-17 12:15:22 +01:00
Simon Willnauer d7eee637d9 fix some docs issues 2017-01-17 11:47:29 +01:00
Tim Brooks 16a76d9bc0 Remove blocking TCP clients and servers (#22639)
This commit removes the option to use the blocking variants of the TCP
transport server, TCP transport client, or http server.
2017-01-16 18:38:51 -06:00
Christoph Büscher 2791c69960 Update profile.asciidoc
Making the "Human readable output" section a note instead of an own section.
2017-01-16 16:19:07 +01:00
Boaz Leskes f88ab76067 Revert "Add a deprecation notice to shadow replicas (#22025)"
This reverts commit 0da190234c.
2017-01-16 16:15:41 +01:00
Boaz Leskes 0da190234c Add a deprecation notice to shadow replicas (#22025)
Also adds deprecation logging.

See #22024
2017-01-16 15:40:05 +01:00
Christoph Büscher 49a49da3f5 [Docs] Fix section title in profile.asciidoc 2017-01-16 14:53:06 +01:00
Christoph Büscher 59a48ffc41 ProfileResult and CollectorResult should print machine readable timing information (#22561)
Currently both ProfileResult and CollectorResult print the time field in a human readable string format
 (e.g. "time": "55.20315000ms"). When trying to parse this back to a long value, for example to use in 
the planned high level java rest client, we can lose precision because of conversion and rounding issues. 
This change adds a new additional field (`time_in_nanos`) to the profile response to be able to get the 
original time value in nanoseconds back. 

The old `time` field is only printed when the `?`human=true` flag in the url is set. This follow the behaviour for 
all other stats-related apis. Also the format of the `time` field is slightly changed. Instead of always formatting 
the output as a 10-digit ms value, by using the `XContentBuilder#timeValueField()` method we now print 
the largest time unit present is used (e.g. "s", "ms", "micros").
2017-01-16 14:27:55 +01:00
Jason Tedor fc3280b3cf Expose logs base path
For certain situations, end-users need the base path for Elasticsearch
logs. Exposing this as a property is better than hard-coding the path
into the logging configuration file as otherwise the logging
configuration file could easily diverge from the Elasticsearch
configuration file. Additionally, Elasticsearch will only have
permissions to write to the log directory configured in the
Elasticsearch configuration file. This commit adds a property that
exposes this base path.

One use-case for this is configuring a rollover strategy to retain logs
for a certain period of time. As such, we add an example of this to the
documentation.

Additionally, we expose the property es.logs.cluster_name as this is
used as the name of the log files in the default configuration.

Finally, we expose es.logs.node_name in cases where node.name is
explicitly set in case users want to include the node name as part of
the name of the log files.

Relates #22625
2017-01-16 07:39:37 -05:00
Simon Willnauer 895124e67e Merge branch 'master' into feature/multi_cluster_search 2017-01-16 13:20:45 +01:00
Elijah 5430f84b64 Inclusion of link to Multi Delete (#22619)
Based on the 2 categories it looks like the docs-delete-by-query was to be listed here under the Multi-document APIs section.
2017-01-16 12:58:59 +01:00
Jake 6ee72ae72d Fix formatting in update-by-query (#22628)
The automatic slicing section wasn't displaying the bullet list correctly.
2017-01-16 12:44:51 +01:00
Colin Goodheart-Smithe aa0c8ce895 [DOCS] Fiexes typo in getting started doc 2017-01-16 09:32:07 +00:00
Colin Goodheart-Smithe 04f19bbaa4 [DOCS]cMake it clear that bulk API actions are processed sequentially on each shard rather than sequentially overall (#22550) 2017-01-16 09:15:24 +00:00
Simon Willnauer 63e4552c0d Merge branch 'master' into feature/multi_cluster_search 2017-01-13 23:07:20 +01:00
maciejkula b4c8c21553 State default sort order on missing values
Closes #19099
2017-01-13 17:05:13 +01:00
Simon Willnauer 6779ea9c2a Merge branch 'master' into feature/multi_cluster_search 2017-01-13 12:10:23 +01:00
Lee Hinman 2db01b6127 Merge remote-tracking branch 'dakrone/disable-all-by-default' 2017-01-12 10:17:51 -07:00
Tanguy Leroux df703dce0a [DOC] Document {{url}} mustache function (#22549)
This function introduced in #20838 wasn't documented at all.

Related to #22459
2017-01-12 14:57:03 +01:00
Francesc Gil 17342c403f Indentation error on example of dist_max (#22578)
There was a problem with the indentation on the example of the `dist_max` query
2017-01-12 09:38:36 +01:00
Lee Hinman 7a18bb50fc Disable _all by default
This change disables the _all meta field by default.

Now that we have the "all-fields" method of query execution, we can save both
indexing time and disk space by disabling it.

_all can no longer be configured for indices created after 6.0.

Relates to #20925 and #21341
Resolves #19784
2017-01-11 16:47:13 -07:00
Simon Willnauer 00781d24ce Merge branch 'master' into feature/multi_cluster_search 2017-01-11 23:40:46 +01:00
Jason Tedor b7995fbc0d Fix default port for unicast zen ping hosts
Today when you do not specify a port for an entry in
discovery.zen.ping.unicast.hosts, the default port is the value of the
setting transport.profiles.default.port and falls back to the value of
transport.tcp.port if this is not set. For a node that is explicitly
bound to a different port than the default port, this means that the
default port will be equal to this explicitly bound port. Yet, the docs
say that we fall back to 9300 here. This commit corrects the docs.

Relates #22568
2017-01-11 17:10:56 -05:00
Matt Weber 609d2aab15 QueryString and SimpleQueryString Graph Support (#22541)
Add support for graph token streams to "query_String" and
"simple_query_string" queries.
2017-01-11 18:59:43 +01:00
Simon Willnauer 46713fb9aa remove leftover 2017-01-11 15:10:36 +01:00
Simon Willnauer ea0e72f3d9 improve docs based on review commments: 2017-01-11 14:52:41 +01:00
Simon Willnauer 2aae409508 Merge branch 'master' into feature/multi_cluster_search 2017-01-11 12:41:26 +01:00
Simon Willnauer 4c61f1d75d Cut over to use affix setting for remote cluster configuration
Instead of `search.remote.seeds.${clustername}` we now specify the seeds as:
`search.remote.${clustername}.seeds` which is a real list setting compared to an unvalidated
group setting before.
2017-01-11 12:38:46 +01:00
Dimitrios Liappis 91b5669d99 [docs] Add missing RUN command from custom docker config
Fix the `Dockerfile` example in the `Customizing image` third configuration
method by adding the missing RUN instruction.

Originally reported by Shankar Vasudevan (@vshank77).

Relates #21973
2017-01-11 11:13:05 +02:00
Simon Willnauer 6d2d878068 Merge branch 'master' into feature/multi_cluster_search 2017-01-11 09:28:00 +01:00
Lee Hinman b0331e1146 Merge remote-tracking branch 'dakrone/document-sqs-negation-better' 2017-01-10 12:47:32 -07:00
David Cho-Lerat d6a23de002 Update getting-started.asciidoc (#22370) 2017-01-10 20:12:55 +01:00
Chris Adams 61213f7917 Update Debian package installation documentation (#22405)
#9261 added a warning about the use of `add-apt-repository` which is becoming obsolete over time as new distribution releases include later versions of `add-apt-repository` which don't automatically add the `deb-src` line. This change updates the documentation to make the block a note rather than a warning and adds two other reasons for avoiding `add-apt-repository` which are still relevant: avoiding edits to a system shared file and not requiring a large number of non-default packages to add one line of text to a file.
2017-01-10 19:49:57 +01:00
Lee Hinman 66cf3d3220 Document simple_query_string negation with default_operator of OR
This can be confusing when unexpected.

Resolves #4707
2017-01-10 10:27:00 -07:00
Gytis Šk 501a782131 Update api-conventions.asciidoc (#22439)
The example

```
/<logstash-{now/d-2d}>,<logstash-{now/d-1d}>,<logstash-{now/d}>/_search
```

shows escaped URL where `,  => %2C`, so I assume it should be escaped and be present in the table
2017-01-10 17:59:50 +01:00
Jake d7cc6e28e7 Document `must_not` context and scoring (#22532)
Document that `must_not` uses filter context and returns a score of `0`.
2017-01-10 17:26:48 +01:00
Nik Everett 75d5b3d9eb Fix parent_id example in docs
And fix some indentation I noticed while looking up the query.
2017-01-10 10:01:31 -05:00
Ali Beyad 26f92f8482 Cluster allocation explain API documentation (#22436)
This commit updates the cluster allocation explain API documentation to
explain the new request parameters and response formats, and gives
examples of the explain API responses under various scenarios.
2017-01-10 08:55:39 -06:00
Achraf 5dc85c25d9 Hindu-Arabico-Latino Numerals (#22476)
Hi, same edit as for : https://www.elastic.co/guide/en/elasticsearch/reference/current/analyzer-anatomy.html
2017-01-10 15:24:56 +01:00
Clinton Gormley cb7952e71d Docs: Parent field is no longer indexed and should use parent_id instead of term query
Closes #22517
2017-01-10 13:48:07 +01:00
A.N 6e1df967be Update delete-by-query.asciidoc (#22519)
Fixed some grammatical mistakes
2017-01-10 13:38:34 +01:00
Boaz Leskes 61fe6b32e4 Add a high level description of ES's data replication model (#22327)
This commit adds a document describing our data replication model in high level terms. The goal is give people basic insight into how things work in order to better understand how read and writes interact, both during normal operations and under failures.
2017-01-10 13:17:48 +01:00
Jim Ferenczi 433c822d4f Promote longs to doubles when a terms agg mixes decimal and non-decimal numbers (#22449)
* Promote longs to doubles when a terms agg mixes decimal and non-decimal number

This change makes the terms aggregation work when the buckets coming from different indices are a mix of decimal numbers and non-decimal numbers. In this case non-decimal number (longs) are promoted to decimal (double) which can result in a loss of precision for big numbers.

Fixes #22232
2017-01-10 11:50:56 +01:00
Martijn van Groningen cb2333dacd percolator: remove deprecated percolate and mpercolate apis 2017-01-10 11:18:27 +01:00
javanna d3d81fcab5 fix another typo in the docs 2017-01-09 18:22:11 +01:00
javanna 56c0bd659d fix couple of typos in the docs 2017-01-09 18:17:43 +01:00
Simon Willnauer 1ef98ede17 Merge branch 'master' into feature/multi_cluster_search 2017-01-09 12:09:23 +01:00
Nik Everett 923820c6c9 Document the `detailed` parameter of tasks API (#22425)
Provides an example of using is and an example return description
and explains that we've added descriptions for some tasks but not
even close to all of them. And that we expect to change the
descriptions as we learn more.

Closes #22407

* Fix example

Getting a single task is always detailed, no need to specify.

* Rewrite like imotov wants it
2017-01-06 10:24:52 -05:00
Simon Willnauer 418ec62bfb Merge branch 'master' into feature/multi_cluster_search 2017-01-06 10:24:40 +01:00
Johannes Kanavin 27c57aeebe Fixed id's of 'worked example' in scripted metric aggs docs (#22430) 2017-01-05 14:37:27 -05:00
Jason Veatch 20f90178fe Docs: Detail on false/strict dynamic mapping setting (#22451)
Reference: https://www.elastic.co/guide/en/elasticsearch/guide/master/dynamic-mapping.html
2017-01-05 14:36:18 -05:00
Simon Willnauer 349ea0f9b6 cut over to use : instead of | for cross cluster search 2017-01-05 17:03:12 +01:00
Simon Willnauer dca54734ac add basic docs 2017-01-05 16:10:34 +01:00
Mike Dias 1286d1ac83 Fix listing format in reindex docs (#22420) 2017-01-03 12:44:49 -05:00
Christoph Büscher a773d46c69 Remove deprecated `minimum_number_should_match` in BoolQueryBuilder
After deprecating getters and setters and the query DSL parameter in 5.x,
support for `minimum_number_should_match` can be removed entirely. Also
consolidated comments with the ones on 5.x branch and added an entry to the
migration docs.
2017-01-03 15:14:33 +01:00
Dave Richardson eaefb5f99b Descriptions for scripting enabled (#22388)
The `Script source settings` section currently states that `false` means scripting is ENABLED.
The other sections seem to indicate that `false` means scripting is DISABLED.

If the current documentation is correct, that would imply that `inline` and `stored` scripting are ENABLED by default, which seems to conflict with all the other sections in the document.
2016-12-30 12:50:17 -05:00
Adrien Grand 3f805d68cb Add the ability to set an analyzer on keyword fields. (#21919)
This adds a new `normalizer` property to `keyword` fields that pre-processes the
field value prior to indexing, but without altering the `_source`. Note that
only the normalization components that work on a per-character basis are
applied, so for instance stemming filters will be ignored while lowercasing or
ascii folding will be applied.

Closes #18064
2016-12-30 09:36:10 +01:00
Paweł Bobruk 117b63ed41 Docs fix native script usage in 5.x (#22362) 2016-12-29 09:41:26 -08:00
Clinton Gormley 3999e5ba6b Docs: Added link from bool and constant score query to filter context
Closes #22353
2016-12-29 11:05:28 +01:00
Itamar Syn-Hershko dea8cee70f Fixing a second "second" reference in docs (#22345) 2016-12-28 18:50:10 +01:00
Jason Tedor 2713549533 Use reader for doc stats
Today we try to pull stats from index writer but we do not get a
consistent view of stats. Under heavy indexing, this inconsistency can
be very skewed indeed. In particular, it can lead to the number of
deleted docs being reported as negative and this leads to serialization
issues. Instead, we should provide a consistent view of the stats by
using an index reader.

Relates #22317
2016-12-23 09:44:56 -05:00
David Pilato 08f556c133 Merge pull request #22310 from gameldar/patch-3
Add ingest-attachment-with-arrays section to ingest attachments doc
2016-12-23 10:33:23 +01:00
Adrien Grand e39942fc02 `value_type` is useful regardless of scripting. (#22160)
Today we only expose `value_type` in scriptable aggregations, however it is
also useful with unmapped fields. I suspect we never noticed because
`value_type` was not documented (fixed) and most aggregations are scriptable.

Closes #20163
2016-12-22 14:35:12 +01:00
Gameldar e3eb363882 Link directly to the attachments in arrays section
The link should be made to the relevant section of the ingest attachments documentation, rather than the top of the page.
2016-12-22 20:52:08 +08:00
Francesc Gil dec6fc2d40 Repeated language analyzers (#22240)
* Repeated language analyzers

The `catalan` analyzer was repeated on the supported list :)

* Reordered the languages to have alphabetic order

* Added space for format

* Reordered the languages and removed repeated
2016-12-21 17:32:02 +01:00
gameldar d404ee3533 Add ingest-attachment-with-arrays section to ingest attachments doc
Added a new section detailing how to use the attachment processor
within an array.

This reverts commit #22296 and instead links to the foreach processor.
2016-12-22 00:18:33 +08:00
Adrien Grand 84edf36f11 Make `-0` compare less than `+0` consistently. (#22173)
Our `float`/`double` fields generally assume that `-0` compares less than `+0`,
except when bounds are exclusive: an exclusive lower bound on `-0` excludes
`+0` and an exclusive upper bound on `+0` excludes `-0`.

Closes #22167
2016-12-21 16:51:45 +01:00
Tal Levy c53b2ee9cd introduce KV Processor in Ingest Node (#22272)
Now you can parse field values of the `key=value` variety and have
`key` be inserted as a field name in an ingest document.

Closes #22222.
2016-12-20 13:26:17 -08:00
Tal Levy ad4b1ecdeb [docs] update ingest-node delete docs to mention wildcarding (#22270) 2016-12-20 10:52:17 -08:00
Adrien Grand b2e93d2870 Be explicit about the fact backslashes need to be escaped. (#22257)
Relates #22255
2016-12-19 14:21:21 +01:00
Clinton Gormley f96769f97b Update painless-syntax.asciidoc
Fix asciidoc syntax
2016-12-19 10:09:24 +01:00
Daniel Mitterdorfer 3ce7b119d2 Enable strict duplicate checks for all XContent types (#22225)
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default for all XContent types (not only JSON).

We have also changed the name of the system property to disable this feature
from `es.json.strict_duplicate_detection` to the now more appropriate name
`es.xcontent.strict_duplicate_detection`.

Relates elastic/elasticsearch#19614
Relates elastic/elasticsearch#22073
2016-12-19 09:29:47 +01:00
Daniel Mitterdorfer 6327e35414 Change type of ingest doc meta-data field 'TIMESTAMP' to `Date` (#22234)
With this commit we change the data type of the 'TIMESTAMP'
meta-data field from a formatted date string to a plain
`java.util.Date` instance. The main reason for this change is
that our benchmarks have indicated that this contributes
significantly to the time spent in the ingest pipeline.

The overhead in terms of indexing throughput of the ingest
pipeline is about 15% and breaks down roughly as follows:

* 5% overhead caused by the conversion from `XContent` -> `Map`
* 5% overhead caused by the timestamp formatting
* 5% overhead caused by the conversion `Map` -> `XContent`

Relates #22074
2016-12-19 09:10:58 +01:00
Jason Tedor 30806af6bd Rename bootstrap.seccomp to bootstrap.system_call_filter
We try to install a system call filter on various operating systems
(Linux, macOS, BSD, Solaris, and Windows) but the setting
(bootstrap.seccomp) to control this is named after the Linux
implementation (seccomp). This commit replaces this setting with
bootstrap.system_call_filter. For backwards compatibility reasons, we
fallback to bootstrap.seccomp and log a deprecation message if
bootstrap.seccomp is set. We intend to remove this fallback in
6.0.0. Note that now is the time to make this change it's likely that
most users are not making this setting anyway as prior to version 5.2.0
(currently unreleased) it was not necessary to configure anything to
enable a node to start up if the system call filter failed to install
(we marched on anyway) but starting in 5.2.0 it will be necessary in
this case.

Relates #22226
2016-12-16 18:22:54 -05:00
Tal Levy bb37167946 Enables the ability to inject serialized json fields into root of document. (#22179)
The JSON processor has an optional field called "target_field".
If you don't specify target_field then target_field becomes what you specified as "field".
There isn't anyway to add the fields to the root of a document. By
setting `add_to_root`, now serialized fields will be inserted into the
top-level fields of the ingest document.

Closes #21898.
2016-12-16 10:17:27 -08:00
Pablo Musa 152efe95e6 Small typo fix in the docs. (#22180)
There is a small typo in the convert processor code example.
2016-12-16 14:50:06 +01:00
Florian Hopf 0e18782d11 Update bucket-script-aggregation.asciidoc (#22219)
Example is missing "params." for painless
2016-12-16 12:39:22 +01:00
Masaru Hasegawa a0185c83a7 Merge pull request #21393 from masaruh/alias_boost
Resolve index names in indices_boost
2016-12-16 15:07:51 +09:00
Jason Tedor 41ffb008ad Fix doc bug for cgroup cpuacct usage metric
This commit fixes a silly doc bug where the field that represents the
total CPU time consumed by all tasks in the same cgroup was mistakenly
reported as "usage" instead of "usage_nanos".

Relates #21029
2016-12-15 23:22:54 -05:00
Nik Everett 61597f2c20 Send error_trace by default when testing (#22195)
Sends the `error_trace` parameter with all requests sent by the
yaml test framework, including the doc snippet tests. This can be
overridden by settings `error_trace: false`. While this drift's
core's handling of the yaml tests from the client's slightly this
should only be a problem for tests that rely on the default value,
both of which I've fixed by setting the value explicitly.

This also escapes `\n` and `\t` in the `Stash dump on failure` so
the `stack_trace` is more readable.

Also fixes `RestUpdateSettingsAction` to not think of the `error_trace`
parameter as a setting.
2016-12-15 13:35:14 -05:00
Areek Zillur cdd5fbe3a1 Deprecate _suggest endpoint in favour of _search (#20305)
* Replace _suggest endpoint to _search in docs

In 5.0, the _suggest endpoint is just sugar for _search
with suggestions specified. Users should move away from
using the _suggest endpoint, as it is marked as deprecated in 5.x and
will be removed in 6.0

* update docs to use _search endpoint instead of _suggest

* Add deprecation logging to RestSuggestAction

* Use search endpoint instead of suggest endpoint in rest tests
2016-12-14 21:49:53 -05:00
Clinton Gormley 571b20137a Document using round-robin DNS for discovery (#21810)
* Document using round-robin DNS for discovery

Closes #8427

* Improve wording around round robin DNS

* Included jason's suggestion
2016-12-14 11:29:39 +01:00
Daniel Mitterdorfer 7e5058037b Enable strict duplicate checks for JSON content
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default. This ensures that JSON keys are always unique. While this has
a performance impact, benchmarking has indicated that the typical drop in
indexing throughput is around 1 - 2%.

As a last resort, we allow users to still disable strict duplicate checks
by setting `-Des.json.strict_duplicate_detection=false` which is
intentionally undocumented.

Closes #19614
2016-12-14 09:35:53 +01:00
Lee Hinman a4e8b5d952 Throw an exception on unrecognized "match_mapping_type"
When using dynamic templates, ES will now throw an exception if a
`match_mapping_type` is used that doesn't correspond to an actual type.

Relates to #17285
2016-12-12 09:59:48 -07:00
Matt Sicker 32fcb7220e Clarify first search API explanation of parameters (#21958)
Added info about the sort parameter and included the pretty parameter in the example.
2016-12-12 15:00:53 +01:00
yutaono abf718832d add missing url-access-control link (#21960) 2016-12-12 14:56:52 +01:00
Luca Cavanna 6d987a9b69 Remove support for empty queries (#22092)
Our query DSL supports empty queries (`{}`), which have a different meaning depending on the query that holds it, either ignored, match_all or match_none. We deprecated the support for empty queries in 5.0, where we log a deprecation warning wherever they are used.

The way we supported it once we moved query parsing to the coordinating node was having an Optional<QueryBuilder> return type in all of our parse methods (called fromXContent). See #17624. The central place for this was QueryParseContext#parseInnerQueryBuilder. We can now remove all the optional return types and simply throw an exception whenever an empty query is found.
2016-12-12 12:37:12 +01:00
Luca Cavanna 73cf002293 Un-deprecate fuzzy query (#22088)
When we decided to deprecate and remove fuzzy query in #15760, we didn't realize we would take away the possibililty for uses to use a fuzzy query as part of a span query, which is not possible using match query. This means we have to go back and un-deprecate fuzzy query, which will not be removed.

Closes #15760
2016-12-12 12:09:16 +01:00
Masaru Hasegawa 3df2a086d4 Resolve index names in indices_boost
This change allows specifying alias/wildcard expression in indices_boost.
And added another format for specifying indices_boost. It accepts array of index name and boost pair.
If an index is included in multiple aliases/wildcard expressions, the first match will be used.
With new format, old format is marked as deprecated.

Closes #4756
2016-12-11 21:41:49 +09:00
Matthew Haugen ca8d89af5a Correct typo in "detect_noop" documentation (#22039)
The documentation reads:

> You can disable this behavior by setting "detect_noop": false like this:

Followed by a code example, that originally set `"detect_noop": true`.

Please correct me if I got the change backwards (i.e. the paragraph should be changed to `true`), but this seems like it makes the most sense.
2016-12-09 15:24:08 -05:00
Igor Motov 93b5e55660 Restores the original default format of search slow log
In 5.0, the search slow log switched to the multi-line format with no option to get back to the origin single-line format that was used prior to 5.0 by default. This commit removes the reformat option from the search slow log and returns the search slow log back to the single-line format.

Closes #21711
2016-12-09 12:38:28 -05:00
Matias Anaya beb794cb0f Fix typo in percolated-query.asciidoc (#21991) 2016-12-09 13:45:57 +01:00
Juan Paulo Gutierrez acc39b1496 Fixed typo (#22066) 2016-12-09 10:34:29 +01:00
Adrien Grand 787519ee4c Fix `other_bucket` on the `filters` agg to be enabled if a key is set. (#21994)
Closes #21951
2016-12-09 09:48:48 +01:00
Adrien Grand 9524c81af9 Document the `locale` option of the `date` field. (#22050)
This also adds another level of protection against using the default locale.
Relates to https://discuss.elastic.co/t/mapping-for-12h-date-format/68433/3.
2016-12-09 09:45:53 +01:00
Jared Carey 317866894e Fix systemd override example in configuring docs
When overriding a systemd configuration via a drop-in file, the
[Service] header is required. This commit adds this to an example
drop-in override in the configuring docs.

Relates #22038
2016-12-07 19:41:59 -05:00
Thibault Pierre e494d6a94e Fix wrong link (#22019) 2016-12-07 17:58:46 +01:00
Colin Goodheart-Smithe 8006b105f3 Update order examples to use max instead of avg (#22032)
The use of the avg aggregation for sorting the terms aggregation is not encouraged since it has unbounded error. This changes the examples to use the max aggregation which does not suffer the same issues
2016-12-07 16:00:24 +00:00
Nikhil Patel b5e3d351d9 Fix typos in threads docs
This commit fixes a typo in the threads docs where the past tense form
of a verb was used when current tense is needed.

Relates #22016
2016-12-07 08:22:49 -05:00
Lucas Bremgartner 0086b99797 [Docs] Correct setting name in snapshot/restore documentation (#22023)
There is no setting include_cluster_state for snapshot restore. The correct name for this setting is include_global_state.
2016-12-07 14:12:10 +01:00
Jim Ferenczi b42ca6bcc9 Include unindexed field in FieldStats response (#21821)
* Include unindexed field in FieldStats response

This change adds non-searchable fields to the FieldStats response. These fields do not have min/max informations but they can be aggregatable. Fields that are only stored in _source (store:no, index:no, doc_values:no) will still be missing since they do not have any useful information to show. Indices and clients must be at least on V_5_2_0 to see this change.
2016-12-06 13:32:57 +01:00
Nik Everett 2087234d74 Timeout improvements for rest client and reindex (#21741)
Changes the default socket and connection timeouts for the rest
client from 10 seconds to the more generous 30 seconds.

Defaults reindex-from-remote to those timeouts and make the
timeouts configurable like so:
```
POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "socket_timeout": "1m",
      "connect_timeout": "10s"
    },
    "index": "source",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}
```

Closes #21707
2016-12-05 10:54:51 -05:00
Jason Tedor 0afef53a17 Add system call filter bootstrap check
Today if system call filters fail to install on startup, we log a
message but otherwise march on. This might leave users without system
call filters installed not knowing that they have implicitly accepted
the additional risk. We should not be lenient like this, instead clearly
informing the user that they have to either fix their configuration or
accept the risk of not having system call filters installed. This commit
adds a bootstrap check that if system call filters are enabled, they
must successfully install.

Relates #21940
2016-12-02 14:27:54 -05:00
Allen Torres 887fbb6387 Update lowercase-tokenizer.asciidoc (#21896)
Fixed typo
2016-12-02 10:49:51 -05:00
Arthur a30efacca4 [DOC] Update node names (#21892)
Update the node names in the documentation.
2016-12-02 14:02:23 +01:00
Jason Tedor 1e645bc2d6 Fix reference to jvm.options docs
The reference for the jvm.options docs recently changed from
es-java-opts to jvm-options. This commit fixes a broken reference that
arose as a result of this change.
2016-12-01 20:05:19 -05:00
Jason Tedor b9df2e2287 Improve the out-of-the-box experience
Elasticsearch can be run in a few different ways:
 - from the command line on Linux and Windows
 - as a service on Linux and Windows

on both 32-bit client and 64-bit server VMs. We strive for a great
out-of-the-box experience any of these combinations but today it is
lacking on 32-bit client JVMs and on the Windows service. There are two
deficiencies that arise:
 - on any 32-bit client JVM we fail to start out of the box because we
   force the server JVM in jvm.options
 - when installing the Windows service, the thread stack size must be
   specified in jvm.options

This commit attempts to address these deficiencies.

We should continue to force the server JVM because there are systems
where the server JVM is not active by default (e.g., the 32-bit JDK on
Windows). This does mean that if a user tries to run with a client JVM
they will see a failure message at startup but this is the best that we
can do if we want to continue to force the server JVM. Thus, this commit
at least documents this situation.

To improve the situation with installing the Windows service, this
commit adds a default setting for the thread stack size. This default is
chosen based on the default thread stack size across all 64-bit server
JVMs. This means that if a user tries to run with a 32-bit JVM they
could otherwise see significantly higher memory usage (this situation is
complicated, it's really only on Windows where the extra memory usage is
egregious, but cutting into the 32-bit address space on any system is
bad). So this commit makes it so that the out-of-the-box experience is
improved for the Windows service on 64-bit server JVMs and we document
the need to adjust this setting on 32-bit JVMs.

Again, we are focusing on the out-of-the-box experience here and this
means optimizing for the best experience on any 64-bit server JVM as
this covers the vast majority of the user base. The users that are on
32-bit JVMs will suffer a little bit but at least now any user on any
64-bit server JVM can start Elasticsearch out of the box.

Finally, we fix some references to the jvm.options documentation.

Relates #21920
2016-12-01 17:26:29 -05:00
Jason Tedor 32df032c59 Add option to skip kernel parameters on install
During package install on systemd-based systems, we try to set
vm.max_map_count. On some systems (e.g., containers), users do not have
the ability to tune these parameters from within the container. This
commit provides an option for these users to skip setting such kernel
parameters.

Relates #21899
2016-12-01 17:23:51 -05:00
Luca Cavanna 103984a4a1 Remove indices query (#21837)
The indices query is deprecated since 5.0.0 (#17710). It can now be removed in master (future 6.0 version).
2016-11-30 19:37:01 +01:00
Jason Tedor 0fe6b12553 Remove misplaced parenthesis from configuring docs
This commit removes a misplaced parenthesis from the systemd section of
the configuring docs.
2016-11-30 12:11:47 -05:00
Adrien Grand eed5de20e0 Remove docs for the removed `geo_distance_range` query. 2016-11-30 16:36:55 +01:00
Adrien Grand c5b9c98b99 Remove the `default` store type. (#21616)
It used to be a hybrid store between `niofs` and `mmapfs`, which we removed when
we switched to `fs` by default (which is `mmapfs` on 64-bits systems).
2016-11-30 15:33:26 +01:00
Adrien Grand 90ab477f19 The `terms` query should always map to a Lucene `TermsQuery`. (#21786)
Currently, the `terms` query is just syctactic sugar for a `bool` query when
used in a query context. This change proposes to always generate the same query
in query and filter contexts, which is less confusing.
2016-11-30 15:29:09 +01:00
Adrien Grand 6231009a8f Remove 2.x backward compatibility of mappings. (#21670)
For the record, I also had to remove the geo-hash cell and geo-distance range
queries to make the code compile. These queries already throw an exception in
all cases with 5.x indices, so that does not hurt any more.

I also had to rename all 2.x bwc indices from `index-${version}` to
`unsupported-${version}` to make `OldIndexBackwardCompatibilityIT`
happy.
2016-11-30 13:34:46 +01:00
Adrin Jalali 235e6acd73 typo fix (and -> any) (#21860) 2016-11-30 12:56:00 +01:00
javanna bbebc644f9 [DOCS] document breaking changes added with #21852 2016-11-29 20:05:30 +01:00
Luca Cavanna f253621feb Remove deprecated query names: in, geo_bbox, mlt, fuzzy_match and match_fuzzy (#21852)
These query names were all deprecated in 5.0.0:
- in is removed in favour of terms
- geo_bbox is removed in favour of geo_bounding_box
- mlt is removed in favour of more_like_this
- fuzzy_match and match_fuzzy are removed in favour of match
2016-11-29 19:07:01 +01:00
Jim Ferenczi d791ddf704 Upgrade to lucene-6.4.0-snapshot-ec38570 (#21853)
Set lucene version to 6.4.0-snapshot-ec38570 and update all the sha1s/license
Fix invalid combo after upgrade in query_string query. split_on_whitespace=false is disallowed if auto_generate_phrase_queries=true
Adapt the expectations of some tests to the new format of the Lucene explain output
2016-11-29 18:40:31 +01:00
Nicholas Knize af1ab68b64 Add RangeFieldMapper for numeric and date range types
Lucene 6.2 added index and query support for numeric ranges. This commit adds a new RangeFieldMapper for indexing numeric (int, long, float, double) and date ranges and creating appropriate range and term queries. The design is similar to NumericFieldMapper in that it uses a RangeType enumerator for implementing the logic specific to each type. The following range types are supported by this field mapper: int_range, float_range, long_range, double_range, date_range.

Lucene does not provide a DocValue field specific to RangeField types so the RangeFieldMapper implements a CustomRangeDocValuesField for handling doc value support.

When executing a Range query over a Range field, the RangeQueryBuilder has been enhanced to accept a new relation parameter for defining the type of query as one of: WITHIN, CONTAINS, INTERSECTS. This provides support for finding all ranges that are related to a specific range in a desired way. As with other spatial queries, DISJOINT can be achieved as a MUST_NOT of an INTERSECTS query.
2016-11-29 10:10:14 -06:00
Itamar Syn-Hershko c3a95a6666 Fixing cut-in-middle paragraph (#21850) 2016-11-29 14:00:26 +01:00
Luca Cavanna c25f9b5fba [DOCS] add source filtering example to reindex docs (#21835) 2016-11-29 09:22:54 +01:00
Matt Weber 04e07bcdb6 Synonym Graph Support (LUCENE-6664) (#21517)
Integrate the patch from LUCENE-6664 into elasticsearch and
add support for handling a graph token stream in match/multi-match
queries.

This fixes longstanding bugs with multi-token synonyms returning
incorrect results with proximity queries.
2016-11-28 09:25:49 -08:00
Adrin Jalali eec05ec208 then -> than (#21829) 2016-11-28 17:04:56 +01:00
Carney Wu 2c0db3909f include not work in 5.x anymore (#21815)
include not work in 5.x anymore use includes instead
2016-11-28 11:02:59 +01:00
Adrien Grand 4c46ffcecf Document that min/max operate on the double representation of the data.
Relates #9545
2016-11-28 10:34:43 +01:00
Clinton Gormley 5555e85619 Document that the PUT mapping API with the _default_ type overwrites instead of merging
Closes #8215
2016-11-26 12:43:56 +01:00
Clinton Gormley 5ae6845d4d Update percolate-query.asciidoc
Add missing callout to percolate query
2016-11-26 12:35:33 +01:00
Ali Beyad 51bfecc7cb [DOCS] fixes word usage in allocation awareness docs 2016-11-25 11:47:40 -05:00
Achraf d81a928b1f Correction of the names of numirals (#21531)
What was called Arabic numerals is actually Hindu - Eastern Arabic notation. And the Latin numerals you refer to is the Arabic numbers.
2016-11-25 14:30:49 +01:00
Adrin Jalali 953928b2c5 typo fix (it self -> itself) (#21781)
* typo fix.

* apply "stored field value"

* replaced "whereas" with "on the contrary"
2016-11-24 17:11:43 +01:00
Adrin Jalali 3bb9317ca2 clarify ambiguous sentence. (#21734) 2016-11-24 16:47:14 +01:00
markharwood aa60e5cc07 Aggregations - support for partitioning set of terms used in aggregations so that multiple requests can be done without trying to compute everything in one request.
Closes #21487
2016-11-24 15:10:46 +00:00
Adrin Jalali 0871073f9b clarification on geo distance sorting (#21779)
* clarification on geo distance sorting

* applying the suggested change
2016-11-24 16:06:10 +01:00
Nik Everett 434fa4bd26 Docs and tests for painless lack of boxing for ?: and ?. (#21756)
NOTE: The result of `?.` and `?:` can't be assigned to primitives. So
`int[] someArray = null; int l = someArray?.length` and
`int s = params.size ?: 100` don't work. Do
`def someArray = null; def l = someArray?.length` and
`def s = params.size ?: 100` instead.

Relates to #21748
2016-11-23 14:33:32 -05:00
Jason Tedor 32e6fcf256 Fix markup in Zen discovery docs
This commit fixes a markup issue in the Zen discovery docs where a link
and its referring text were not on the same line tripping the renderer.
2016-11-23 10:02:13 -05:00
Nik Everett 76a804e589 Revert "it's a `noop` operation, not a `none` operation. (#21736)"
This reverts commit 7f77214ced.

`none` is indeed how you trigger the `noop` operation in the `_update`
API.
2016-11-23 09:26:53 -05:00
Ryan Ernst 6940b2b8c7 Remove groovy scripting language (#21607)
* Scripting: Remove groovy scripting language

Groovy was deprecated in 5.0. This change removes it, along with the
legacy default language infrastructure in scripting.
2016-11-22 19:24:12 -08:00
Nik Everett 1791623700 Document `error_trace`
The `error_trace` parameter turns on the `stack_trace` field
in errors which returns stack traces.

Removes documentation for `camelCase` because it hasn't worked
in a while....

Documents the internal parameters used to render stack traces as
internal only.

Closes #21708
2016-11-22 19:16:07 -05:00
Luca Cavanna db5a72774b Add indices and filter information to search shards api output (#21738)
Add indices and filter information to search shards api output

The search shards api returns info about which shards are going to be hit by executing a search with provided parameters: indices, routing, preference. Indices can also be aliases, which can also hold filters. The output includes an array of shards and a summary of all the nodes the shards are allocated on. This commit adds a new indices section to the search shards output that includes one entry per index, where each index can be associated with an optional filter in case the index was hit through a filtered alias.

This is relevant since we have moved parsing of alias filters to the coordinating node.

Relates to #20916
2016-11-22 23:00:25 +01:00
Jason Tedor 9dc65037bc Lazy resolve unicast hosts
Today we eagerly resolve unicast hosts. This means that if DNS changes,
we will never find the host at the new address. Moreover, a single host
failng to resolve causes startup to abort. This commit introduces lazy
resolution of unicast hosts. If a DNS entry changes, there is an
opportunity for the host to be discovered. Note that under the Java
security manager, there is a default positive cache of infinity for
resolved hosts; this means that if a user does want to operate in an
environment where DNS can change, they must adjust
networkaddress.cache.ttl in their security policy. And if a host fails
to resolve, we warn log the hostname but continue pinging other
configured hosts.

When doing DNS resolutions for unicast hostnames, we wait until the DNS
lookups timeout. This appears to be forty-five seconds on modern JVMs,
and it is not configurable. If we do these serially, the cluster can be
blocked during ping for a lengthy period of time. This commit introduces
doing the DNS lookups in parallel, and adds a user-configurable timeout
for these lookups.

Relates #21630
2016-11-22 14:17:04 -05:00
Nik Everett 83ea1be185 Remove a bad callout from the new debugging docs
It was a leftover and doesn't break the usual build but breaks the docs
build.
2016-11-22 13:13:16 -05:00
Nik Everett 457c2d8fb0 Add Debug.explain to painless
You can use `Debug.explain(someObject)` in painless to throw an
`Error` that can't be caught by painless code and contains an
object's class. This is useful because painless's sandbox doesn't
allow you to call `someObject.getClass()`.

Closes #20263
2016-11-22 12:46:02 -05:00
Luca Cavanna db8b2dceea Remove ignored type parameter in search_shards api (#21688)
The `type` parameter has always been accepted by the search_shards api, probably to make the api and its urls the same as search. Truth is that the type never had any effect, it's been ignored from day one while accepting it may make users think that we actually do something with it.

This commit removes support for the type parameter from the REST layer and the Java API. Backwards compatibility is maintained on the transport layer though.

The new added serialization test also uncovered a bug in the java API where the `ClusterSearchShardsRequest` could be created with no arguments, but the indices were required to be not null otherwise the request couldn't be serialized as `writeTo` would throw NPE. Fixed by setting a default value (empty array) for indices.
2016-11-22 17:22:33 +01:00
Adrin Jalali 7f77214ced it's a `noop` operation, not a `none` operation. (#21736)
It works I guess cause it's ignored as an invalid operation.
2016-11-22 10:41:24 -05:00
Adrin Jalali 982f7cb067 fixing an ambiguous sentence. (#21729) 2016-11-22 16:35:58 +01:00
Adrin Jalali 4bb7091f64 `force` is deprecated be mentioned at the end. (#21731) 2016-11-22 16:35:57 +01:00
Clinton Gormley 284dedfb5f Update get-settings.asciidoc
Fix syntax for filtering settings by name

Closes https://github.com/elastic/elasticsearch/issues/21713
2016-11-22 15:10:09 +01:00
Lee Hinman 11da09e9bc Allow overriding all-field leniency when `lenient` option is specified
As part of #20925 and #21341 we added an "all-fields" mode to the
`query_string` and `simple_query_string`. This would expand the query to
all fields and automatically set `lenient` to true.

However, we should still allow a user to override the `lenient` flag to
whichever value they desire, should they add it in the request. This
commit does that.
2016-11-21 21:32:25 -07:00
Nik Everett c79371fd5b Remove lang-python and lang-javascript (#20734)
They were deprecated in 5.0. We are concentrating on making
Painless awesome rather than supporting every language possible.

Closes #20698
2016-11-21 22:13:25 -05:00
Adrien Grand c7fc688096 Add information about the removal of store throttling to the migration guide.
Relates to #21573
2016-11-21 15:03:07 +01:00
Adrien Grand 52408fc389 Add a recommendation against large documents to the docs. (#21652) 2016-11-21 15:01:36 +01:00
Trey Tacon 3ef7f0dec6 Fixing indentation in geospatial querying example. (#21682)
Specifically the example which shows providing an array of an array of values.
2016-11-21 13:09:21 +01:00
Chris Fritz 546fa92d61 Fix typo in filters aggregation docs (#21690) 2016-11-21 12:52:45 +01:00
Jeffery Bradberry 9322c4fe62 The alias is switched to point to the new index (#21512)
If the index satisfies the specified conditions then a new index is created and the alias is switched to point to the new index.
2016-11-19 16:47:39 +01:00
Shubham Aggarwal dce51e2062 Update getting-started.asciidoc (#21677) 2016-11-19 12:45:28 +01:00
Nik Everett ae468441dc Implement the ?: operator in painless (#21506)
Implements a null coalescing operator in painless that looks like `?:`. This form was chosen to emulate Groovy's `?:` operator. It is different in that it only coalesces null values, instead of Groovy's `?:` operator which coalesces all falsy values. I believe that makes it the same as Kotlin's `?:` operator. In other languages this operator looks like `??` (C#) and `COALESCE` (SQL) and `:-` (bash).

This operator is lazy, meaning the right hand side is only evaluated at all if the left hand side is null.
2016-11-18 13:54:26 -05:00
Sakthipriyan Vairamani 52d4cd504a minor grammatical improvements (#21640) 2016-11-18 08:55:04 -07:00
Luca Wintergerst 7a533577b8 Merge pull request #21651 from elastic/suggester-docs-example-fix
fix two errors in suggester docs
2016-11-18 13:13:13 +01:00
Jason Tedor 5ca60fa78b Remove stale mention of minimum master nodes check
This commit removes a stale mention of the minimum master nodes boostrap
check from the docs.

Relates #21653
2016-11-18 06:56:23 -05:00
Luca Wintergerst 277f4b8d24 fix two errors in suggester docs
The first changed referred to an example of the 2.4 documentation. I removed the no longer relevant parts. We should consider adding a little more here. 

The second change was just then->than in the suggest_mode popular section
2016-11-18 12:05:49 +01:00
Ryan Ernst 914664d89a Fix leftover reference to ScriptModule in native script docs 2016-11-17 08:22:15 -08:00
Nik Everett df2a5e7691 Mark snippet in docker docs NOTCONSOLE
That keeps us from failing the build because it is "unconverted".

Relates to #21497
2016-11-17 10:07:42 -05:00
Dimitrios Liappis 6c9ea0877d [docs] Document Docker installation method
Initial commit of the Docker installation method for Elasticsearch.

Relates #21497
2016-11-17 15:06:32 +02:00
David Pilato 475a7ca84f Add documentation for lenient in multimatch
`lenient` option is documented for `match` query but not for `multi_match` query.
2016-11-17 08:35:20 +01:00
David Pilato c946094d5b Add documentation for lenient in multimatch
`lenient` option is documented for `match` query but not for `multi_match` query.
2016-11-16 16:15:28 +01:00
Jason Tedor d06a8903fd Merge branch 'master' into feature/seq_no
* master: (22 commits)
  Add proper toString() method to UpdateTask (#21582)
  Fix `InternalEngine#isThrottled` to not always return `false`. (#21592)
  add `ignore_missing` option to SplitProcessor (#20982)
  fix trace_match behavior for when there is only one grok pattern (#21413)
  Remove dead code from GetResponse.java
  Fixes date range query using epoch with timezone (#21542)
  Do not cache term queries. (#21566)
  Updated dynamic mapper section
  Docs: Clarify date_histogram bucket sizes for DST time zones
  Handle release of 5.0.1
  Fix skip reason for stats API parameters test
  Reduce skip version for stats API parameter tests
  Strict level parsing for indices stats
  Remove cluster update task when task times out (#21578)
  [DOCS] Mention "all-fields" mode doesn't search across nested documents
  InternalTestCluster: when restarting a node we should validate the cluster is formed via the node we just restarted
  Fixed bad asciidoc in boolean mapping docs
  Fixed bad asciidoc ID in node stats
  Be strict when parsing values searching for booleans (#21555)
  Fix time zone rounding edge case for DST overlaps
  ...
2016-11-16 09:10:35 -05:00
Tal Levy 6796464f16 add `ignore_missing` option to SplitProcessor (#20982)
Closes #20840.
2016-11-16 15:46:09 +02:00
Tal Levy 04b712bdc5 fix trace_match behavior for when there is only one grok pattern (#21413)
There is an issue in the Grok Processor, where trace_match: true does not inject the _ingest._grok_match_index into the ingest-document when there is just one pattern provided. This is due to an optimization in the regex construction. This commit adds a check for when this is the case, and injects a static index value of "0", since there is only one pattern matched (at the first index into the patterns).

To make this clearer, more documentation was added to the grok-processor docs.

Fixes #21371.
2016-11-16 15:41:54 +02:00
David Pilato 2842e2752a Updated dynamic mapper section
Backport of #21574 in master (6.0)
2016-11-16 09:52:08 +01:00
Christoph Büscher 4ccd8e79c1 Docs: Clarify date_histogram bucket sizes for DST time zones
Added a warning note that clarifies bucket sizes diverging from the intended
`interval` size when using a time zone that has DST changes.

Closes #18805
2016-11-16 09:40:07 +01:00
Lee Hinman 17a2fffc9b [DOCS] Mention "all-fields" mode doesn't search across nested documents 2016-11-15 11:02:43 -07:00
Clinton Gormley a4e88bb64a Fixed bad asciidoc in boolean mapping docs 2016-11-15 17:50:23 +00:00
Clinton Gormley cfabc95f59 Fixed bad asciidoc ID in node stats 2016-11-15 17:39:15 +00:00
Lee Hinman 96122aa518 Be strict when parsing values searching for booleans (#21555)
This changes only the query parsing behavior to be strict when searching on
boolean values. We continue to accept the variety of values during index time,
but searches will only be parsed using `"true"` or `"false"`.

Resolves #21545
2016-11-15 10:36:57 -07:00
Bryan Karlovitz f03723a812 Minor style change to getting-started.asciidoc (#21571)
* In line 525, change the phrase "... with as little network roundtrips as possible." to "... with as few network roundtrips as possible."
2016-11-15 12:18:49 -05:00
Jason Tedor f5ac0e5076 Remove lenient stats parsing
Today when parsing a stats request, Elasticsearch silently ignores
incorrect metrics. This commit removes lenient parsing of stats requests
for the nodes stats and indices stats APIs.

Relates #21417
2016-11-15 12:17:26 -05:00
Boaz Leskes 2c0338fa87 Merge remote-tracking branch 'upstream/master' into feature/seq_no 2016-11-15 17:09:08 +00:00
Lee Hinman 568a7ea5f1 Fix incorrect instructions for disabling deprecation logging (#21569)
We log deprecation events at "WARN", so setting it to `info` means the events
are still logged. It must be set to `error` in order to disable the logging.
2016-11-15 09:57:26 -07:00
Nik Everett 7dcff27aea Update docs for scripted metric agg
Now that the default language is painless the examples didn't work at
all. This fixes them.

Closes #21536
2016-11-15 11:47:17 -05:00
Adrien Grand 6db683a4bd Fix recurring doc test failures with the cat API. (#21561)
This failure is due to the fact that we sort on store size, which is cached. So
it might happen that the store size that is taken into account is not the right
one, which makes the indices sorted in the wrong order. This changes the doc
example to sort on the number of docs instead.

Closes #21062
2016-11-15 16:00:44 +01:00
Boaz Leskes c9f49039d3 Merge remote-tracking branch 'upstream/master' into feature/seq_no 2016-11-15 10:14:47 +00:00
Sumit Gupta e53405f4f3 Update geohashgrid-aggregation.asciidoc (#21530) 2016-11-15 10:49:02 +01:00
Clinton Gormley 30d342c87c Update significantterms-aggregation.asciidoc
Fix scripted significant terms example to use `params.` prefix for painless
2016-11-14 09:40:04 +01:00
Jason Tedor 1ea69b1a80 Merge branch 'master' into feature/seq_no
* master:
  Set vm.max_map_count on systemd package install
  [TEST] reduce the number of snapshotted shards to 1 in testSnapshotSucceedsAfterSnapshotFailure() so that we are more likely to trigger I/O exceptions on writing the control files during the finalize phase of snapshotting (with the aim of triggering an I/O failure when writing pending-index-*).
  Add documentation for Logger with Transport Client
  Enable appender exceptions in UpdateSettingsIT
  [TEST] remove AwaitsFix from testSnapshotSucceedsAfterSnapshotFailure, turns out the issue is specific to Java 9 v143
  Cleanup formatting in UpdateSettingsIT.java
  [TEST] mute the testSnapshotSucceedsAfterSnapshotFailure() test until its clear what is going wrong.
  Mark SearchQueryIT test as awaits fix
  Makes snapshot throttling test go much faster (#21485)
  Breaking changes docs for template index_patterns
  [TEST] adds randomness between atomic and non-atomic move operations in MockRepository
  Cache successful shard deletion checks (#21438)
  Task cancellation command should wait for all child nodes to receive cancellation request before returning
2016-11-11 17:03:01 -05:00
Jason Tedor 33f7cd5a16 Remove shard ID from doc write response
This commit removes the shard ID from doc write response; this was
useful for debugging but its time has passed.

Relates #21508
2016-11-11 15:18:25 -05:00
Jason Tedor 1e7c424479 Merge branch 'master' into feature/seq_no
* master:
  ShardActiveResponseHandler shouldn't hold to an entire cluster state
  Ensures cleanup of temporary index-* generational blobs during snapshotting (#21469)
  Remove (again) test uses of onModule (#21414)
  [TEST] Add assertBusy when checking for pending operation counter after tests
  Revert "Add trace logging when aquiring and releasing operation locks for replication requests"
  Allows multiple patterns to be specified for index templates (#21009)
  [TEST] fixes rebalance single shard check as it isn't guaranteed that a rebalance makes sense and the method only tests if rebalance is allowed
  Document _reindex with random_score
2016-11-11 11:25:27 -05:00
Nik Everett 7c3886769f Breaking changes docs for template index_patterns
0219a211d3 added support for templates
to have multiple patterns and renamed `template` to `index_patterns`.
This adds the breaking changes docs for that.
2016-11-11 10:13:19 -05:00
Jason Tedor d3417fb022 Merge branch 'master' into feature/seq_no
* master: (516 commits)
  Avoid angering Log4j in TransportNodesActionTests
  Add trace logging when aquiring and releasing operation locks for replication requests
  Fix handler name on message not fully read
  Remove accidental import.
  Improve log message in TransportNodesAction
  Clean up of Script.
  Update Joda Time to version 2.9.5 (#21468)
  Remove unused ClusterService dependency from SearchPhaseController (#21421)
  Remove max_local_storage_nodes from elasticsearch.yml (#21467)
  Wait for all reindex subtasks before rethrottling
  Correcting a typo-Maan to Man-in README.textile (#21466)
  Fix InternalSearchHit#hasSource to return the proper boolean value (#21441)
  Replace all index date-math examples with the URI encoded form
  Fix typos (#21456)
  Adapt ES_JVM_OPTIONS packaging test to ubuntu-1204
  Add null check in InternalSearchHit#sourceRef to prevent NPE (#21431)
  Add VirtualBox version check (#21370)
  Export ES_JVM_OPTIONS for SysV init
  Skip reindex rethrottle tests with workers
  Make forbidden APIs be quieter about classpath warnings (#21443)
  ...
2016-11-10 23:40:33 -05:00
Ryan Ernst 48bfb142b9 Remove (again) test uses of onModule (#21414)
This change was reverted after it caused random test failures. This was
due to a copy/paste error in the original PR which caused the mock
version of ClusterInfoService to be used whenever the mock *ZenPing* was
used, and the real ClusterInfoService to be used when MockZenPing was
not used.
2016-11-10 16:06:14 -08:00
Alexander Lin 0219a211d3 Allows multiple patterns to be specified for index templates (#21009)
* Allows for an array of index template patterns to be provided to an
index template, and rename the field from 'template' to 'index_pattern'.

Closes #20690
2016-11-10 18:00:30 -05:00
Nik Everett eeb6602c98 Document _reindex with random_score
You can use `_reindex` and `random_score` to extract a random
subset of an index but you have to be careful to sort by `_score`
or it won't work.

Closes #21432
2016-11-10 16:14:30 -05:00
Clinton Gormley 860efb2ad2 Replace all index date-math examples with the URI encoded form
Closes #21454
2016-11-10 12:23:19 +01:00
lslxdx e560c5be43 Fix typos (#21456)
Delete repeated words " between nodes and".
2016-11-10 11:55:04 +01:00
Lee Hinman 2674e415c8 Merge remote-tracking branch 'dakrone/sqs-all-field-mode' 2016-11-09 13:13:41 -07:00
Lee Hinman 7420fd0be3 Add "all fields" execution mode to simple_query_string query
This commit introduces a new execution mode for the
`simple_query_string` query, which is intended down the road to be a
replacement for the current _all field.

It now does auto-field-expansion and auto-leniency when the following criteria
are ALL met:

    The _all field is disabled
    No default_field has been set in the index settings
    No fields are specified in the request

Additionally, a user can force the "all-like" execution by setting the
all_fields parameter to true.

When executing in all field mode, the `simple_query_string` query will
look at all the fields in the mapping that are not metafields and can be
searched, and automatically expand the list of fields that are going to
be queried.

Relates to #20925, which is the `query_string` version of this work.
This is basically the same behavior, but for the `simple_query_string`
query.

Relates to #19784
2016-11-09 10:38:51 -07:00
Nik Everett d03b8e4abb Implement reading from null safe dereferences
Null safe dereferences make handling null or missing values shorter.
Compare without:
```
if (ctx._source.missing != null && ctx._source.missing.foo != null) {
  ctx._source.foo_length = ctx.source.missing.foo.length()
}
```

To with:
```
Integer length = ctx._source.missing?.foo?.length();
if (length != null) {
  ctx._source.foo_length = length
}
```

Combining this with the as of yet unimplemented elvis operator allows
for very concise defaults for nulls:
```
ctx._source.foo_length = ctx._source.missing?.foo?.length() ?: 0;
```

Since you have to start somewhere, we started with null safe dereferenes.

Anyway, this is a feature borrowed from groovy. Groovy allows writing to
null values like:
```
def v = null
v?.field = 'cat'
```
And the writes are simply ignored. Painless doesn't support this at this
point because it'd be complex to implement and maybe not all that useful.

There is no runtime cost for this feature if it is not used. When it is
used we implement it fairly efficiently, adding a jump rather than a
temporary variable.

This should also work fairly well with doc values.
2016-11-09 07:20:11 -05:00
Jason Tedor b743ab0b07 Remove 5.x references from cat API migration doc
This commit removes some references to 5.x that were picked up when the
migration docs for the cat API were migrated from 5.x to master.

Relates #21342
2016-11-09 07:09:59 -05:00
Jason Tedor 168c54fa6e Add migration docs for cat API
This commit adds migration docs for the cat API, including a note
regarding the change in response in the cat thread pool API for
unbounded queue sizes.

Relates #21342
2016-11-09 07:07:52 -05:00
Clinton Gormley f5013e0872 Updated install docs to add missing empty cell in directory layout 2016-11-09 11:54:56 +01:00
Clinton Gormley 931d54f04d Update zip-targz.asciidoc
Added missing empty cell in directory layout table
2016-11-09 11:53:16 +01:00
javanna 2f32c1173b Revert "Tests: Remove a couple test uses of onModule (#21414)"
This reverts commit b326f0bc51.
2016-11-09 11:32:16 +01:00
Ryan Ernst b326f0bc51 Tests: Remove a couple test uses of onModule (#21414)
There were still a couple test use cases and examples that were using
onModule. This change cleans those cases up.
2016-11-08 13:50:13 -08:00
Jason Tedor aec09a76d6 Clarify requesting all stats in node stats docs
This commit clarifies how to explicitly obtain all stats from the node
stats API.
2016-11-08 13:47:15 -05:00
Jason Tedor 6f6c633298 Fix markup in sysconfig-file docs
This commit fixes a markup issue in the sysconfig-file docs, adding a
missing backtick.
2016-11-08 08:33:45 -05:00
Joeyn414 aefffca763 minor grammatical fix (#21384) 2016-11-08 11:52:52 +01:00
Jason Tedor 1f0c6cd137 Remove double space in bootstrap checks docs
This commit removes a double space in the G1GC check section of the
bootstrap check docs.
2016-11-07 16:28:04 -05:00
Jason Tedor b30732c464 Migrate G1GC JVM check to bootstrap check
This commit fixes an assertion in G1GCCheck#jvmVersion that was
mistakenly asserting on itself.

Relates #21388
2016-11-07 16:19:05 -05:00
Adrien Grand 263af27d76 Fix docs example after #21218. 2016-11-07 14:57:20 +01:00
Clinton Gormley a36465d479 Removed left over ID and [float] from the setup docs 2016-11-07 14:48:50 +01:00
markharwood dd21aa41be Docs fix - Diversified sampler agg had incorrect title and example
Closes #21347
2016-11-07 10:46:22 +00:00
Nik Everett 593d47efe2 Make it clear _suggest doesn't support source filtering (#21268)
We plan to deprecate `_suggest` during 5.0 so it isn't worth fixing
it to support the `_source` parameter for `_source` filtering. But we
should fix the docs so they are accurate.

Since this removes the last non-`// CONSOLE` line in
`completion-suggest.asciidoc` this also removes it from the list of
files that have non-`// CONSOLE` docs.

Closes #20482
2016-11-06 20:15:45 -05:00
Clinton Gormley 9d56c1b766 Groovy is no longer the default scripting language
Closes #21208
2016-11-05 16:31:34 +01:00
Loek van Gool 1a23739211 Update store.asciidoc (#21353)
* Update store.asciidoc

* Update store.asciidoc

* Update store.asciidoc
2016-11-05 14:58:16 +01:00
Nik Everett 7ff9ba1604 Fix asciidoc structure for sliced reindex
Asciidoc likes headings just so and will complain and fail the
docs build without it.

Related to #20767
2016-11-04 21:59:19 -04:00
Nik Everett a13a050271 Add automatic parallelization support to reindex and friends (#20767)
Adds support for `?slices=N` to reindex which automatically
parallelizes the process using parallel scrolls on `_uid`. Performance
testing sees a 3x performance improvement for simple docs
on decent hardware, maybe 30% performance improvement
for more complex docs. Still compelling, especially because
clusters should be able to get closer to the 3x than the 30%
number.

Closes #20624
2016-11-04 20:59:15 -04:00
Luca Cavanna c2160a88b5 Remove support for controversial ignore_unavailable and allow_no_indices from indices exists api (#20712)
Exist requests are supposed to never throw an exception, but rather return true or false depending on whether some resource exists or not. Indices exists does that for indices and accepts wildcard expressions too. The way the api works internally is by resolving indices and catching IndexNotFoundException: if an exception is thrown the index does not exist hence it returns false, otherwise it returns true. That works ok only if ignore_unavailable and allow_no_indices indices options are both set to false, meaning that they are strict and any missing index or wildcard expressions that resolves to no indices will lead to an exception that can be thrown and cause false to be returned.

Unfortunately the indices options have  been configurable up until now for this request, meaning that one can set ignore_unavailable or allow_no_indices to true and have the indices exist request return true for indices that really don't exist, which makes very little sense in the context of this api.

This commit removes the indicesOptions setter from the IndicesExistsRequest and makes settable only expandWildcardsOpen and expandWildcardsClosed, hence a subset of the available indices options. This way we can guarantee more consistent behaviour of the indices exists api. We can then remove the ignore_unavailable and allow_no_indices option from indices exists api spec
2016-11-04 19:26:37 +01:00
Nik Everett a4b3a95f5a Move flush in _cat/indices docs tests (#21117)
Moves the `_flush` in the `_cat/indices` snippets testing framework
to the very first test. We need to flush super early because index
size is cached for a few seconds so we really need to read a
consistent size on the first read so we can sort by it properly.

Closes #21062
2016-11-04 10:32:07 -04:00
Lee Hinman 6666fb1614 Add "all field" execution mode to query_string query
This commit introduces a new execution mode for the query_string query, which
is intended down the road to be a replacement for the current _all field.

It now does auto-field-expansion and auto-leniency when the following criteria
are ALL met:

    The _all field is disabled
    No default_field has been set in the index settings
    No default_field has been set in the request
    No fields are specified in the request

Additionally, a user can force the "all-like" execution by setting the
all_fields parameter to true.

When executing in all field mode, the query_string query will look at all the
fields in the mapping that are not metafields and can be searched, and
automatically expand the list of fields that are going to be queried.

Relates to #19784
2016-11-04 05:46:18 -06:00
Chris Earle e6462ea4ca [DOCS] "service" should be "elasticsearch-service" (#21311)
The Windows `service.bat` script was renamed to `elasticsearch-service.bat` in ES 5.0+.
2016-11-03 15:37:37 -04:00
Jason Tedor 548c12d60f Remove broken docs link
The important settings docs previously referred to a section regarding
the node.max_local_storage_nodes setting. This section was removed, but
the link was not. This commit removes that link.
2016-11-03 10:14:46 -04:00
Jason Tedor 886eba076c Remove node.max_local_storage_nodes from setup doc
Previously node.max_local_storage_nodes defaulted to fifty, and this
permitted users to start multiple instances of Elasticsearch sharing the
same data folder. This can be dangerous, and usually it does not make
sense to run more than one instance of Elasticsearch on a single
server. Because of this, we had a note in the important settings docs
advising users to set this setting to one. However, we have since
changed the default value of this setting to one so this advise is no
longer needed.

Relates #21305
2016-11-03 09:03:42 -04:00
Clinton Gormley ef2453360e Fixed rollover docs test for date math indices 2016-11-03 13:48:34 +01:00
Clinton Gormley 6902eafbfe Specify date math with URI encoding in rollover API docs
Closes #21181
2016-11-03 10:48:50 +01:00
Christoph Büscher a0c094d0c1 Add deprecation logging message for 'fuzzy' query
This query is deprecated from 5.0 on. Similar to IndicesQueryBuilder we should
log a deprecation warning whenever this query is used.

Relates to #15760
2016-11-02 15:45:33 +01:00
Nik Everett b6db00efd8 Update docs after test-framework moved
Closes #21263
2016-11-02 09:51:55 -04:00
Adrien Grand 52de0645fb Remove `lowercase_expanded_terms` and `locale` from query-parser options. (#20208)
Lucene 6.2 introduces the new `Analyzer.normalize` API, which allows to apply
only character-level normalization such as lowercasing or accent folding, which
is exactly what is needed to process queries that operate on partial terms such
as `prefix`, `wildcard` or `fuzzy` queries. As a consequence, the
`lowercase_expanded_terms` option is not necessary anymore. Furthermore, the
`locale` option was only needed in order to know how to perform the lowercasing,
so this one can be removed as well.

Closes #9978
2016-11-02 14:25:08 +01:00
Adrien Grand 68b0e395b2 Add recommendations about getting consistent scores despite shards and replicas. (#21167)
This is a topic that has triggered many questions recently so it would be good
to have these recommendations documented.
2016-11-02 10:50:38 +01:00
Jim Ferenczi 9d6fac809c Expose splitOnWhitespace in `Query String Query` (#20965)
This change adds an option called `split_on_whitespace` which prevents the query parser to split free text part on whitespace prior to analysis. Instead the queryparser would parse around only real 'operators'. Default to true.
For instance the query `"foo bar"` would let the analyzer of the targeted field decide how the tokens should be splitted.
Some options are missing in this change but I'd like to add them in a follow up PR in order to be able to simplify the backport in 5.x. The missing options (changes) are:
* A `type` option which similarly to the `multi_match` query defines how the free text should be parsed when multi fields are defined.
* Simple range query with additional tokens like ">100 50" are broken when `split_on_whitespace` is set to false. It should be possible to preserve this syntax and make the parser aware of this special syntax even when `split_on_whitespace` is set to false.
* Since all this options would make the `query_string_query` very similar to a match (multi_match) query we should be able to share the code that produce the final Lucene query.
2016-11-02 10:00:40 +01:00
Alexander Lin f86d784bbc Add extra note about Java8 requirement to sub-install docs (#21190)
Closes #20005
2016-11-01 15:52:09 -06:00
Adriel Dean-Hall b72a708c0d Add docs with up to date instructions on updating default similarity (#21242)
* Add docs with up to date instructions on updating default similarity

The default similarity can no longer be set in the configuration file
(you will get an error on startup). Update the docs with the method
that works.

* Add instructions for changing similarity on index creation
2016-11-01 16:14:20 -04:00
Craig Squire 1f1daf59bc Documentation updates for scroll API size parameter (#21229)
* Document size parameter for scroll API

* Fix size parameter behavior description for scroll
2016-11-01 15:55:09 -04:00
Nik Everett a612e5988e Bump reindex-from-remote's buffer to 200mb
It was 10mb and that was causing trouble when folks reindex-from-remoted
with large documents.

We also improve the error reporting so it tells folks to use a smaller
batch size if they hit a buffer size exception. Finally, adds some docs
to reindex-from-remote mentioning the buffer and giving an example of
lowering the size.

Closes #21185
2016-11-01 13:19:28 -04:00
LakumiNarayanan 5af6deb5b5 Fix typo in keyword.asciidoc (#21237) 2016-11-01 10:15:12 -04:00
Jason Tedor f81d28d0a0 Fix transport host setting
This commit fixes a reference to the transport.host setting which was
mistakenly identified as transport.tcp.host.
2016-10-31 20:48:55 -04:00
Jason Tedor 5fab17c7bc Fix markup for bootstrap checks docs
This commit adds a missing backtick to the bootstrap check docs.
2016-10-31 17:33:15 -04:00
Jack Conradson 185dff7346 Cleanup ScriptType (#21179)
Refactored ScriptType to clean up some of the variable and method names. Added more documentation. Deprecated the 'in' ParseField in favor of 'stored' to match the indexed scripts being replaced by stored scripts.
2016-10-31 13:48:51 -07:00
Jason Tedor 1d8d8ea55f Clarify production mode for bootstrap checks
This commit clarifies that production mode for the bootstrap checks is
only tripped if transport is bound to an external interface.

Relates #21220
2016-10-31 16:34:21 -04:00
Christoph Büscher 1f5adaa824 Docs: Adding Ukrainian analyzer 2016-10-31 18:20:39 +01:00
Lee Hinman 6a8bad8a06 [DOCS] Document all date formats (#21164)
Resolves #21046
2016-10-31 09:15:36 -06:00
Stanislav Mamontov 7738af27e8 Fix malformed JSON in Delete API example (#21168)
Obviously, there should be

    "result": "deleted"

instead of

    "result: deleted"
2016-10-31 09:13:46 -06:00
Nik Everett b5f86f6f05 Add note to getting-started doc about CONSOLE
Link to the documentation for CONSOLE in getting-started so the
layout of the snippet makes some sense.

Relates to #21158
2016-10-30 00:09:09 -04:00
Nik Everett 3a7a218e8f Support negative array ofsets in painless
Adds support for indexing into lists and arrays with negative
indexes meaning "counting from the back". So for if
`x = ["cat", "dog", "chicken"]` then `x[-1] == "chicken"`.

This adds an extra branch to every array and list access but
some performance testing makes it look like the branch predictor
successfully predicts the branch every time so there isn't a
in execution time for this feature when the index is positive.
When the index is negative performance testing showed the runtime
is the same as writing `x[x.length - 1]`, again, presumably thanks
to the branch predictor.

Those performance metrics were calculated for lists and arrays but
`def`s get roughly the same treatment though instead of inlining
the test they need to make a invoke dynamic so we don't screw up
maps.

Closes #20870
2016-10-29 16:12:40 -04:00
Adrien Grand bafedc93fa Fix Lucene version in the docs. 2016-10-28 14:50:46 +02:00
Adrien Grand b3cc54cf0d Upgrade to lucene-6.3.0-snapshot-ed102d6 (#21150)
Lucene 6.3 is expected to be released in the next weeks so it'd be good to give
it some integration testing. I had to upgrade randomized-testing too so that
both Lucene and Elasticsearch are on the same version.
2016-10-28 14:47:15 +02:00
Adrien Grand 9cbbddb6dc Add support for `quote_field_suffix` to `simple_query_string`. (#21060)
Closes #18641
2016-10-28 09:11:57 +02:00
Jun Ohtani a66c76eb44 Merge pull request #20704 from johtani/remove_request_params_in_analyze_api
Removing request parameters in _analyze API
2016-10-27 17:43:18 +09:00
Colin Goodheart-Smithe c1a9833445 Correct similarity default for 5.0 (#21144) 2016-10-27 09:33:21 +01:00
Igor Motov 17ad88d539 Makes search action cancelable by task management API
Long running searches now can be cancelled using standard task cancellation mechanism.
2016-10-25 12:27:34 -10:00
Nik Everett 44c3b04bef Convert more docs to // CONSOLE
Converts docs for `_cat/segments`, `_cat/plugins` and `_cat/repositories`
from `curl` to `// CONSOLE` so they are tested as part of the build and
are cleaner to use in Console. They should work fine with `curl` with
the `COPY AS CURL` link.

Also swaps the `source` type of the response from `js` to `txt` because
that is more correct. The syntax highlighter doesn't care. It looks at
the text to figure out the language. So it looks a little funny for `_cat`
responses regardless.

Relates to #18160
2016-10-25 11:17:24 -04:00
Jason Tedor 900ee0536e Strengthen handling of unavailable cgroup stats
On some systems, cgroups will be available but not configured. And in
some cases, cgroups will be configured, but not for the subsystems that
we are expecting (e.g., cpu and cpuacct). This commit strengthens the
handling of cgroup stats on such systems.

Relates #21094
2016-10-24 16:36:51 -04:00
Praveen Shukla 2e18f2e818 [DOCS] clarifies master nodes to be master eligible nodes 2016-10-24 08:58:44 -04:00
Jason Tedor 3d642ab0eb Add basic cgroup CPU metrics
This commit adds basic cgroup CPU metrics to the node stats API.

Relates #21029
2016-10-24 08:26:56 -04:00
Igor Motov a541f0187d Docs: add documentation about removal of cluster.routing.allocation.snapshot.relocation_enabled 2016-10-20 14:19:12 -10:00
Jim Ferenczi d0bbe89c16 Optimize query with types filter in the URL (t/t/_search) (#20979)
This change adds a TypesQuery that checks if the disjunction of types should be rewritten to a MatchAllDocs query. The check is done only if the number of terms is below a threshold (16 by default and configurable via max_boolean_clause).
2016-10-20 12:33:32 +02:00
Nik Everett acf7c7430b Add "simple match" support for reindex-from-remote whitelist
This allows you to whitelist `localhost:*` or `127.0.10.*:9200`.
It explicitly checks for patterns like `*` in the whitelist and
refuses to start if the whitelist would match everything. Beyond
that the user is on their own designing a secure whitelist.
2016-10-18 21:47:21 -04:00
Tal Levy 38c650f376 make painless the default scripting language for ScriptProcessor (#20981)
- fixes a bug in the docs that mentions `lang` as optional
- now `lang` defaults to "painless"
2016-10-18 16:22:01 -07:00
Chris Earle c2894d2362 Add missing '// CONSOLE' to code snippet 2016-10-18 12:57:03 -04:00
Chris Earle 9cf7214380 [DOCS] Add "version" to template and pipeline docs (#20407)
* [DOCS] Add "version" to template and pipeline docs

This adds details about the "version" to both the template and pipeline pages.
2016-10-18 11:56:18 -04:00
Joshua Rich 8133c5e85d Merge pull request #20958 from joshuar/joshuar-docs/clearer-shard-allocation-filtering
Docs: Cluster Allocation Filtering
2016-10-18 14:24:41 +11:00
Joshua Rich cdb156e691 Merge pull request #20794 from joshuar/doc/fix_highlighter_ambiguities
[DOCS] Use a better name for fields in examples to avoid ambiguity
2016-10-18 14:23:27 +11:00
Clinton Gormley 5ec2ba3166 Update scripted-metric-aggregation.asciidoc
Removed docs for `reduce_params`

Closes #20917
2016-10-17 19:31:30 +02:00
Clinton Gormley 43ad8de75b Fixed typo in index rollover docs
Closes #20935
2016-10-17 17:56:54 +02:00
Jason Tedor 96aa5e33ce Fix slowlog docs
This commit fixes two issues with the slow log docs:
 - clarifies that these settings are per index
 - updates index slow log configuration for Log4j 2

Relates #20976
2016-10-17 10:50:32 -04:00
Quinn Shanahan 1bef6c7fee Update regexp-syntax.asciidoc (#20973) 2016-10-17 16:32:17 +02:00
Joshua Rich 63f484ffa3 Docs: Cluster Allocation Filtering
Put more emphasis on the fact that multiple values can be specified
and move examples after explanation of settings.
2016-10-17 11:02:33 +11:00
Nik Everett 4fbe1a8819 CONSOLEify _cat/pending_tasks docs
Relates to #18160
2016-10-14 14:12:35 -04:00
Jason Tedor 370253f95a Add doc note regarding processors bound
This commit expands the thread pool docs regarding the processor
setting.

Relates #20895
2016-10-14 10:32:31 -04:00
Nik Everett 68ed183381 CONSOLEify a few more _cat docs
`_cat/master`, `_cat/nodeattrs`, `_cat/nodes`.
2016-10-13 16:43:06 -04:00
Nik Everett 279baa0284 Add a flush to test in _cat/indices.asciidoc
We test that sorting by `store.size` works but sometimes the
sizes aren't what we expect. At least in CI:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-unix-compatibility/os=opensuse/101/console
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+5.x+multijob-unix-compatibility/os=centos/100/console

I haven't been able to reproduce it locally but adding a `_flush`
won't hurt and might make the inconsistency vanish.
2016-10-13 13:21:57 -04:00
Nik Everett 3bba7dbe07 Docs: note about snapshot version compatibility (#20896)
It is important that folks understand that snapshot/restore isn't
for archiving. It is appropriate for backup and disaster recovery
but not for archival over long periods of time because of version
incompatibility.

Closes #20866
2016-10-13 10:49:32 -04:00
Clinton Gormley 101ea0549a Add note about retention periods to reindex-upgrade docs 2016-10-13 12:13:55 +02:00
Thibaud BARDIN 1bcd26627c [DOCS] Fix typo in "Wait For Active Shards" part (#20900)
Add missing closing backtick
2016-10-13 08:53:30 +02:00
Robin Clarke bbe6555b7a Docs: your -> you're (#20883) 2016-10-12 11:09:34 -04:00
Nik Everett 42a7a554b1 Don't mind pending cluster tasks in docs build
This removes an assertion that the cluster doesn't have any
pending cluster state tasks from the `_cat/health` docs.

Relates to #18160
2016-10-11 17:44:50 -04:00
Nik Everett 298cf1cf21 CONSOLEify _cat/indices docs
Relates to #18160
Uses the new sorting (#20658) in the `_cat` API to support all use
cases natively. We can still resort to piping things through `sort`
if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 14:55:37 -04:00
Thibaud BARDIN 82881ef99a [DOCS] Fix typo in "Cluster Health" part (#20864)
Replace "we can see and total of ..." by "we can see a total of ..."
2016-10-11 13:53:11 -04:00
Lee Hinman f0a2726dcd [DOCS] Remove documentation for `force` version-type
This option should not be recommended to anyone, and should never be
used, upon chance of primary/replica divergence.

Relates to #19769
2016-10-11 11:21:32 -06:00
Alexander Lin 10ddcc4616 Adding built-in sorting capability to _cat apis. (#20658)
* Adding built-in sorting capability to _cat apis.

Closes #16975

* addressing pr comments

* changing value types back to original implementation and fixing cosmetic issues

* Changing compareTo, hashCode of value types to a better implementation

* Changed value compareTos to use Double.compare instead of if statements + fixed some failed unit tests
2016-10-11 12:29:22 -04:00
Clinton Gormley 82e2f6e747 Document the ctx._now variable in the update API
Relates to #20835
2016-10-11 13:13:03 +02:00
Clinton Gormley 5b5306ea82 Add an indices-upgrade section to the docs redirects. 2016-10-11 12:39:02 +02:00
Clinton Gormley a7d8de14bb Removed the upgrade API docs
Relates to #20675
2016-10-11 12:21:46 +02:00
Clinton Gormley 02a739d3c9 Added upgrade docs explaining how to reindex in place or reindex from remote
Closes #20675
2016-10-11 12:14:35 +02:00
Pascal Borreli fcb01deb34 Fixed typos (#20843) 2016-10-10 14:51:47 -06:00
Nik Everett c414faf2fb Reword note about windows and FDs
Make it clear why this isn't a thing in windows and remove value
judgement about file descriptors.

Relates to #20737
2016-10-10 08:47:25 -04:00
Nik Everett 741ecf80ff Loosen assertion in getting-started.asciidoc
We were asserting that the size of an index had a decimal point but it
doesn't always:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+5.0+java9-periodic/240/console
2016-10-09 13:47:59 -04:00
Lucas Bremgartner 4bf45e53a5 Documentation: comments in index templates (#20701)
Fixes #20513
2016-10-08 12:38:45 +02:00
Clinton Gormley 22f1acde94 Docs: Pattern analyzer does not support a max_token_length parameter
Closes #20713
2016-10-08 12:27:33 +02:00
Nik Everett 06049283a0 CONSOLEify some _cat docs
`/_cat/count`, `/_cat/fielddata`, and `/_cat/health`.

Three more files down, 141 to go.

Relates to #18160
2016-10-07 16:30:45 -04:00
Shane Connelly 78ae19ae5e Merge pull request #20784 from eskibars/master
Add a note that reindex does not set up mappings, etc
2016-10-07 09:39:10 -07:00
Clinton Gormley 6cf7a93837 Better formatting of date-match URL encoding 2016-10-07 16:33:30 +02:00
Clinton Gormley eb62f90578 Document date math use in the rollover API (#20780)
* Document date math use in the rollover API

Relates to #20709
2016-10-07 15:49:28 +02:00
Adrien Grand 7a403f640b Clarify some docs about geo-distance sorting. (#20735)
This also improves formatting a bit.
2016-10-07 15:26:34 +02:00
Jason Tedor d01a62908a Change separator for shards preference
The shards preference on a search request enables specifying a list of
shards to hit, and then a secondary preference (e.g., "_primary") can be
added. Today, the separator between the shards list and the secondary
preference is ';'. Unfortunately, this is also a valid separtor for URL
query parameters. This means that a preference like "_shards:0;_primary"
will be parsed into two URL parameters: "_shards:0" and "_primary". With
the recent change to strict URL parsing, the second parameter will be
rejected, "_primary" is not a valid URL parameter on a search
request. This means that this feature has never worked (unless the ';'
is escaped, but no one does that because our docs do not that, and there
was no indication from Elasticsearch that this did not work). This
commit changes the separator to '|'.

Relates #20786
2016-10-07 07:17:01 -05:00
Simon Willnauer 194a6b1df0 Remove LocalTransport in favor of MockTcpTransport (#20695)
This change proposes the removal of all non-tcp transport implementations. The
mock transport can be used by default to run tests instead of local transport that has
roughly the same performance compared to TCP or at least not noticeably slower.

This is a master only change, deprecation notice in 5.x will be committed as a
separate change.
2016-10-07 11:27:47 +02:00
Jun Ohtani 945fa499d2 Deprecating request parameters in _analyze API
Remove params in indices.analyze.json
Fix REST changes

Closes #20246
2016-10-07 16:23:24 +09:00
Jun Ohtani 99236b7627 Removing request parameters in _analyze API
Fix small English issue in breaking changes

Closes #20246
2016-10-07 16:23:24 +09:00