Commit Graph

4077 Commits

Author SHA1 Message Date
Chris Earle f0f75b187a Support Preemptive Authentication with RestClient (#21336)
This adds the necessary `AuthCache` needed to support preemptive authorization. By adding every host to the cache, the automatically added `RequestAuthCache` interceptor will add credentials on the first pass rather than waiting to do it after _each_ anonymous request is rejected (thus always sending everything twice when basic auth is required).
2017-01-24 11:34:05 -05:00
Clinton Gormley 12f5309041 Docs: Made cross-cluster search experimental 2017-01-24 12:44:01 +01:00
Nik Everett ee264c6957 Fix parsing for `max_determinized_states` (#22749)
There was a typo in the `ParseField` declaration. I know
we want to port these parsers to `ObjectParser` eventually
but I don't have the energy for that today and want to get
this fixed.

Closes #22722
2017-01-23 11:57:43 -05:00
William Webber f1a902865f Update span-multi-term-query.asciidoc (#22733)
"term" is not actually a multi-term query (perhaps confusion with "term range")
2017-01-23 17:33:40 +01:00
William Webber abaf728882 "from" => "gte", "to" => "lte" in bool example (#22735) 2017-01-23 17:29:00 +01:00
Jason Tedor d9a3ffcea0 Add note regarding custom jvm.options
When users need to specify a custom location for configuration files,
they also need to specify a custom location for the jvm.options file yet
our docs are absent in this regard. This commit adds a note to the
rolling upgrade docs explaining this situation.

Relates #22747
2017-01-23 10:39:11 -05:00
Jim Ferenczi e48bc2eed7 Add field collapsing for search request (#22337)
* Add top hits collapsing to search request

The field collapsing is done with a custom top docs collector that "collapse" search hits with same field value.
The distributed aspect is resolve using the two passes that the regular search uses. The first pass "collapse" the top hits, then the coordinating node merge/collapse the top hits from each shard.

```
GET _search
{
   "collapse": {
      "field": "category",
   }
}
```

This change also adds an ExpandCollapseSearchResponseListener that intercepts the search response and expands collapsed hits using the CollapseBuilder#innerHit} options.
The retrieval of each inner_hits is done by sending a query to all shards filtered by the collapse key.

```
GET _search
{
   "collapse": {
      "field": "category",
      "inner_hits": {
	"size": 2
      }
   }
}
```
2017-01-23 16:33:51 +01:00
Nik Everett da8740128b Docs: CONSOLE-ify value_count aggregation docs
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the snippets
in the `value_count` docs and causes the build to execute the snippets
for testing.

Release #18160
2017-01-23 10:07:29 -05:00
markharwood 87495750ff Docs fix - Added missing link to new Adjacency-matrix agg 2017-01-23 10:18:30 +00:00
Nik Everett a99bddcc7e CONSOLE-ify filter aggregation docs
This adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the
snippet and causes the build to execute the snippet as a test.

Relates to #18160
2017-01-23 01:32:56 -05:00
Nik Everett 40e2645177 CONSOLE-ify date_range aggregation docs
This adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the
snippets in the docs for the `date_range` aggregation and tests
those snippets as part of the build.

Relates to #18160
2017-01-22 23:38:45 -05:00
Nik Everett f7524fbdef CONSOLE-ify date histogram docs
This adds the `VIEW IN SENSE` and `COPY AS CURL` links and has
the build automatically execute the snippets and verify that they
work.

Relates to #18160
2017-01-20 16:23:28 -05:00
Nik Everett c2a580304b CONSOLE-ify min and max aggregation docs
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the docs
and makes the build automatically test them.

Relates to #18160
2017-01-20 15:33:00 -05:00
Nik Everett 8c856eaa9f CONSOLE-ify global-aggregation.asciidoc
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the example
`global` aggregation. Also improves the example by adding a
non-`global` aggregation to compare it to.

Relates to #18160
2017-01-20 14:36:51 -05:00
Folusho Oladipo daf1f184d4 Corrected a plural verb to a singular one. (#22681) 2017-01-20 19:00:10 +01:00
Patryk Krawaczyński 42c0e8947f Fix duplicates from search.query (#22701)
search.query_current, search.query_time and search.query_total have wrong aliases.
2017-01-20 18:45:10 +01:00
Nik Everett 00707b0b0d Readd unconverted snippets mark for doc
I accidentally removed it when cleaning up something else.
2017-01-20 12:39:37 -05:00
Nik Everett 025b1a0fc5 Remove unused scripts from docs build
These were added in 5.x to support testing the lang-python and
lang-js plugins which have since been removed.
2017-01-20 11:31:52 -05:00
markharwood f01784205f New AdjacencyMatrix aggregation
Similar to the Filters aggregation but only supports "keyed" filter buckets and automatically "ANDs" pairs of filters to produce a form of adjacency matrix.
The intersection of buckets "A" and "B" is named "A&B" (the choice of separator is configurable). Empty intersection buckets are removed from the final results.

Closes #22169
2017-01-20 15:49:31 +00:00
Yannick Welsch 881993de3a [Docs] Remove outdated info about enabling/disabling doc_values (#22694) 2017-01-19 17:33:40 +01:00
Daniel Mitterdorfer aece89d6a1 Make boolean conversion strict (#22200)
This PR removes all leniency in the conversion of Strings to booleans: "true"
is converted to the boolean value `true`, "false" is converted to the boolean
value `false`. Everything else raises an error.
2017-01-19 07:59:18 +01:00
Michael McCandless 1d1bdd476c Finish exposing FlattenGraphTokenFilter (#22667) 2017-01-18 11:05:34 -05:00
Boaz Leskes 1227044ddd Add a deprecation notice to shadow replicas (#22647)
Relates to #22024

On top of documentation, the PR adds deprecation loggers and deals with the resulting warning headers.

The yaml test is set exclude versions up to 6.0. This is need to make sure bwc tests pass until this is backported to 5.2.0 . Once that's done, I will change the yaml test version limits
2017-01-18 12:28:09 +01:00
Christoph Büscher 9ed867ea83 [DOCS] Fix inconsistent formatting for fieldnames in profile.asciidoc 2017-01-18 10:41:22 +01:00
Clinton Gormley e415170989 Docs: Fix link syntax in cross-cluster docs
Also tidy up settings list
2017-01-18 10:13:48 +01:00
Clinton Gormley 5d70df855d Docs: Cross-cluster search doc wasn't being included 2017-01-18 10:02:51 +01:00
Simon Willnauer 19f9cb307a Merge branch 'master' into feature/multi_cluster_search 2017-01-18 09:24:35 +01:00
Scott Somerville 372812da98 Allow an index to be partitioned with custom routing (#22274)
This change makes it possible for custom routing values to go to a subset of shards rather than
just a single shard. This enables the ability to utilize the spatial locality that custom routing can
provide while mitigating the likelihood of ending up with an imbalanced cluster or suffering
from a hot shard.

This is ideal for large multi-tenant indices with custom routing that suffer from one or both of
the following:
- The big tenants cannot fit into a single shard or there is so many of them that they will likely
end up on the same shard
- Tenants often have a surge in write traffic and a single shard cannot process it fast enough

Beyond that, this should also be useful for use cases where most queries are done under the context
of a specific field (e.g. a category) since it gives a hint at how the data can be stored to minimize
the number of shards to check per query. While a similar solution can be achieved with multiple
concrete indices or aliases per value today, those approaches breakdown for high cardinality fields.

A partitioned index enforces that mappings have routing required, that the partition size does not
change when shrinking an index (the partitions will shrink proportionally), and rejects mappings
that have parent/child relationships.

Closes #21585
2017-01-18 08:51:23 +01:00
Elijah 3b92179e09 Improve wording in recipes docs
This commit improves some of the wording the recipes docs.

Relates #22661
2017-01-17 21:00:36 -05:00
Elijah b56605d68c Correct grammar in list in how-to docs
This commit corrects the grammar in a list in the how-to docs; namely,
the word "and" was missing preceding the final element in a list.

Relates #22663
2017-01-17 20:57:22 -05:00
Ali Beyad 099d229138 [TEST] fix documentation checking tests to account for possible
pending tasks in the cluster state
2017-01-17 17:29:14 -05:00
Elijah 297b1b7d9a Capitalize "Elasticsearch" in indexing speed docs
This commit fixes the capitalization of "Elasticsearch" in the indexing
speed docs.

Relates #22659
2017-01-17 12:33:01 -05:00
Alex a0c83c4511 Minor doc changes to clarify mapping index param for string type (#22652)
* Grammatical correction

* Add note for legacy string mapping type

* Update truncate token filter to not mention the keyword tokenizer

The advice predates the existence of the keyword field

Closes #22650
2017-01-17 16:43:11 +01:00
Simon Willnauer 709cb9a39e Merge branch 'master' into feature/multi_cluster_search 2017-01-17 12:34:36 +01:00
Clinton Gormley 401438819e Docs: Fix the first highlighting example to work
Closes #22642
2017-01-17 12:20:03 +01:00
Clinton Gormley 519a9c469d Update truncate token filter to not mention the keyword tokenizer
The advice predates the existence of the keyword field

Closes #22650
2017-01-17 12:15:22 +01:00
Simon Willnauer d7eee637d9 fix some docs issues 2017-01-17 11:47:29 +01:00
Tim Brooks 16a76d9bc0 Remove blocking TCP clients and servers (#22639)
This commit removes the option to use the blocking variants of the TCP
transport server, TCP transport client, or http server.
2017-01-16 18:38:51 -06:00
Luca Cavanna 193111919c move ignore parameter support from yaml test client to low level rest client (#22637)
All the language clients support a special ignore parameter that doesn't get passed to elasticsearch with the request, but used to indicate which error code should not lead to an exception if returned for a specific request.

Moving this to the low level REST client will allow the high level REST client to make use of it too, for instance so that it doesn't have to intercept ResponseExceptions when the get api returns a 404.
2017-01-16 18:54:44 +01:00
Christoph Büscher 2791c69960 Update profile.asciidoc
Making the "Human readable output" section a note instead of an own section.
2017-01-16 16:19:07 +01:00
Boaz Leskes f88ab76067 Revert "Add a deprecation notice to shadow replicas (#22025)"
This reverts commit 0da190234c.
2017-01-16 16:15:41 +01:00
Boaz Leskes 0da190234c Add a deprecation notice to shadow replicas (#22025)
Also adds deprecation logging.

See #22024
2017-01-16 15:40:05 +01:00
Christoph Büscher 49a49da3f5 [Docs] Fix section title in profile.asciidoc 2017-01-16 14:53:06 +01:00
Christoph Büscher 59a48ffc41 ProfileResult and CollectorResult should print machine readable timing information (#22561)
Currently both ProfileResult and CollectorResult print the time field in a human readable string format
 (e.g. "time": "55.20315000ms"). When trying to parse this back to a long value, for example to use in 
the planned high level java rest client, we can lose precision because of conversion and rounding issues. 
This change adds a new additional field (`time_in_nanos`) to the profile response to be able to get the 
original time value in nanoseconds back. 

The old `time` field is only printed when the `?`human=true` flag in the url is set. This follow the behaviour for 
all other stats-related apis. Also the format of the `time` field is slightly changed. Instead of always formatting 
the output as a 10-digit ms value, by using the `XContentBuilder#timeValueField()` method we now print 
the largest time unit present is used (e.g. "s", "ms", "micros").
2017-01-16 14:27:55 +01:00
Jason Tedor fc3280b3cf Expose logs base path
For certain situations, end-users need the base path for Elasticsearch
logs. Exposing this as a property is better than hard-coding the path
into the logging configuration file as otherwise the logging
configuration file could easily diverge from the Elasticsearch
configuration file. Additionally, Elasticsearch will only have
permissions to write to the log directory configured in the
Elasticsearch configuration file. This commit adds a property that
exposes this base path.

One use-case for this is configuring a rollover strategy to retain logs
for a certain period of time. As such, we add an example of this to the
documentation.

Additionally, we expose the property es.logs.cluster_name as this is
used as the name of the log files in the default configuration.

Finally, we expose es.logs.node_name in cases where node.name is
explicitly set in case users want to include the node name as part of
the name of the log files.

Relates #22625
2017-01-16 07:39:37 -05:00
Simon Willnauer 895124e67e Merge branch 'master' into feature/multi_cluster_search 2017-01-16 13:20:45 +01:00
Elijah 5430f84b64 Inclusion of link to Multi Delete (#22619)
Based on the 2 categories it looks like the docs-delete-by-query was to be listed here under the Multi-document APIs section.
2017-01-16 12:58:59 +01:00
Jake 6ee72ae72d Fix formatting in update-by-query (#22628)
The automatic slicing section wasn't displaying the bullet list correctly.
2017-01-16 12:44:51 +01:00
Colin Goodheart-Smithe aa0c8ce895 [DOCS] Fiexes typo in getting started doc 2017-01-16 09:32:07 +00:00
Colin Goodheart-Smithe 04f19bbaa4 [DOCS]cMake it clear that bulk API actions are processed sequentially on each shard rather than sequentially overall (#22550) 2017-01-16 09:15:24 +00:00
Simon Willnauer 63e4552c0d Merge branch 'master' into feature/multi_cluster_search 2017-01-13 23:07:20 +01:00
maciejkula b4c8c21553 State default sort order on missing values
Closes #19099
2017-01-13 17:05:13 +01:00
Simon Willnauer 6779ea9c2a Merge branch 'master' into feature/multi_cluster_search 2017-01-13 12:10:23 +01:00
Lee Hinman 2db01b6127 Merge remote-tracking branch 'dakrone/disable-all-by-default' 2017-01-12 10:17:51 -07:00
Tanguy Leroux df703dce0a [DOC] Document {{url}} mustache function (#22549)
This function introduced in #20838 wasn't documented at all.

Related to #22459
2017-01-12 14:57:03 +01:00
Francesc Gil 17342c403f Indentation error on example of dist_max (#22578)
There was a problem with the indentation on the example of the `dist_max` query
2017-01-12 09:38:36 +01:00
Lee Hinman 7a18bb50fc Disable _all by default
This change disables the _all meta field by default.

Now that we have the "all-fields" method of query execution, we can save both
indexing time and disk space by disabling it.

_all can no longer be configured for indices created after 6.0.

Relates to #20925 and #21341
Resolves #19784
2017-01-11 16:47:13 -07:00
Simon Willnauer 00781d24ce Merge branch 'master' into feature/multi_cluster_search 2017-01-11 23:40:46 +01:00
Jason Tedor b7995fbc0d Fix default port for unicast zen ping hosts
Today when you do not specify a port for an entry in
discovery.zen.ping.unicast.hosts, the default port is the value of the
setting transport.profiles.default.port and falls back to the value of
transport.tcp.port if this is not set. For a node that is explicitly
bound to a different port than the default port, this means that the
default port will be equal to this explicitly bound port. Yet, the docs
say that we fall back to 9300 here. This commit corrects the docs.

Relates #22568
2017-01-11 17:10:56 -05:00
Matt Weber 609d2aab15 QueryString and SimpleQueryString Graph Support (#22541)
Add support for graph token streams to "query_String" and
"simple_query_string" queries.
2017-01-11 18:59:43 +01:00
Simon Willnauer 46713fb9aa remove leftover 2017-01-11 15:10:36 +01:00
Simon Willnauer ea0e72f3d9 improve docs based on review commments: 2017-01-11 14:52:41 +01:00
Simon Willnauer 2aae409508 Merge branch 'master' into feature/multi_cluster_search 2017-01-11 12:41:26 +01:00
Simon Willnauer 4c61f1d75d Cut over to use affix setting for remote cluster configuration
Instead of `search.remote.seeds.${clustername}` we now specify the seeds as:
`search.remote.${clustername}.seeds` which is a real list setting compared to an unvalidated
group setting before.
2017-01-11 12:38:46 +01:00
Dimitrios Liappis 91b5669d99 [docs] Add missing RUN command from custom docker config
Fix the `Dockerfile` example in the `Customizing image` third configuration
method by adding the missing RUN instruction.

Originally reported by Shankar Vasudevan (@vshank77).

Relates #21973
2017-01-11 11:13:05 +02:00
Simon Willnauer 6d2d878068 Merge branch 'master' into feature/multi_cluster_search 2017-01-11 09:28:00 +01:00
Lee Hinman b0331e1146 Merge remote-tracking branch 'dakrone/document-sqs-negation-better' 2017-01-10 12:47:32 -07:00
David Cho-Lerat d6a23de002 Update getting-started.asciidoc (#22370) 2017-01-10 20:12:55 +01:00
Chris Adams 61213f7917 Update Debian package installation documentation (#22405)
#9261 added a warning about the use of `add-apt-repository` which is becoming obsolete over time as new distribution releases include later versions of `add-apt-repository` which don't automatically add the `deb-src` line. This change updates the documentation to make the block a note rather than a warning and adds two other reasons for avoiding `add-apt-repository` which are still relevant: avoiding edits to a system shared file and not requiring a large number of non-default packages to add one line of text to a file.
2017-01-10 19:49:57 +01:00
Lee Hinman 66cf3d3220 Document simple_query_string negation with default_operator of OR
This can be confusing when unexpected.

Resolves #4707
2017-01-10 10:27:00 -07:00
Gytis Šk 501a782131 Update api-conventions.asciidoc (#22439)
The example

```
/<logstash-{now/d-2d}>,<logstash-{now/d-1d}>,<logstash-{now/d}>/_search
```

shows escaped URL where `,  => %2C`, so I assume it should be escaped and be present in the table
2017-01-10 17:59:50 +01:00
Jake d7cc6e28e7 Document `must_not` context and scoring (#22532)
Document that `must_not` uses filter context and returns a score of `0`.
2017-01-10 17:26:48 +01:00
Nik Everett 75d5b3d9eb Fix parent_id example in docs
And fix some indentation I noticed while looking up the query.
2017-01-10 10:01:31 -05:00
Ali Beyad 26f92f8482 Cluster allocation explain API documentation (#22436)
This commit updates the cluster allocation explain API documentation to
explain the new request parameters and response formats, and gives
examples of the explain API responses under various scenarios.
2017-01-10 08:55:39 -06:00
Achraf 5dc85c25d9 Hindu-Arabico-Latino Numerals (#22476)
Hi, same edit as for : https://www.elastic.co/guide/en/elasticsearch/reference/current/analyzer-anatomy.html
2017-01-10 15:24:56 +01:00
Clinton Gormley cb7952e71d Docs: Parent field is no longer indexed and should use parent_id instead of term query
Closes #22517
2017-01-10 13:48:07 +01:00
A.N 6e1df967be Update delete-by-query.asciidoc (#22519)
Fixed some grammatical mistakes
2017-01-10 13:38:34 +01:00
Boaz Leskes 61fe6b32e4 Add a high level description of ES's data replication model (#22327)
This commit adds a document describing our data replication model in high level terms. The goal is give people basic insight into how things work in order to better understand how read and writes interact, both during normal operations and under failures.
2017-01-10 13:17:48 +01:00
Jim Ferenczi 433c822d4f Promote longs to doubles when a terms agg mixes decimal and non-decimal numbers (#22449)
* Promote longs to doubles when a terms agg mixes decimal and non-decimal number

This change makes the terms aggregation work when the buckets coming from different indices are a mix of decimal numbers and non-decimal numbers. In this case non-decimal number (longs) are promoted to decimal (double) which can result in a loss of precision for big numbers.

Fixes #22232
2017-01-10 11:50:56 +01:00
Martijn van Groningen cb2333dacd percolator: remove deprecated percolate and mpercolate apis 2017-01-10 11:18:27 +01:00
javanna d3d81fcab5 fix another typo in the docs 2017-01-09 18:22:11 +01:00
javanna 56c0bd659d fix couple of typos in the docs 2017-01-09 18:17:43 +01:00
Simon Willnauer 1ef98ede17 Merge branch 'master' into feature/multi_cluster_search 2017-01-09 12:09:23 +01:00
animageofmine e3546d59c4 Add support for ca-central-1 region to EC2 and S3 plugins
Closes #22458 #22454
2017-01-06 16:27:08 -06:00
Ryan Ernst cd6e3f4cea Merge branch 'master' into keystore 2017-01-06 09:32:08 -08:00
Nik Everett 923820c6c9 Document the `detailed` parameter of tasks API (#22425)
Provides an example of using is and an example return description
and explains that we've added descriptions for some tasks but not
even close to all of them. And that we expect to change the
descriptions as we learn more.

Closes #22407

* Fix example

Getting a single task is always detailed, no need to specify.

* Rewrite like imotov wants it
2017-01-06 10:24:52 -05:00
Simon Willnauer 418ec62bfb Merge branch 'master' into feature/multi_cluster_search 2017-01-06 10:24:40 +01:00
Johannes Kanavin 27c57aeebe Fixed id's of 'worked example' in scripted metric aggs docs (#22430) 2017-01-05 14:37:27 -05:00
Jason Veatch 20f90178fe Docs: Detail on false/strict dynamic mapping setting (#22451)
Reference: https://www.elastic.co/guide/en/elasticsearch/guide/master/dynamic-mapping.html
2017-01-05 14:36:18 -05:00
Simon Willnauer 349ea0f9b6 cut over to use : instead of | for cross cluster search 2017-01-05 17:03:12 +01:00
Simon Willnauer dca54734ac add basic docs 2017-01-05 16:10:34 +01:00
Mike Dias 1286d1ac83 Fix listing format in reindex docs (#22420) 2017-01-03 12:44:49 -05:00
Christoph Büscher a773d46c69 Remove deprecated `minimum_number_should_match` in BoolQueryBuilder
After deprecating getters and setters and the query DSL parameter in 5.x,
support for `minimum_number_should_match` can be removed entirely. Also
consolidated comments with the ones on 5.x branch and added an entry to the
migration docs.
2017-01-03 15:14:33 +01:00
Dave Richardson eaefb5f99b Descriptions for scripting enabled (#22388)
The `Script source settings` section currently states that `false` means scripting is ENABLED.
The other sections seem to indicate that `false` means scripting is DISABLED.

If the current documentation is correct, that would imply that `inline` and `stored` scripting are ENABLED by default, which seems to conflict with all the other sections in the document.
2016-12-30 12:50:17 -05:00
Adrien Grand 3f805d68cb Add the ability to set an analyzer on keyword fields. (#21919)
This adds a new `normalizer` property to `keyword` fields that pre-processes the
field value prior to indexing, but without altering the `_source`. Note that
only the normalization components that work on a per-character basis are
applied, so for instance stemming filters will be ignored while lowercasing or
ascii folding will be applied.

Closes #18064
2016-12-30 09:36:10 +01:00
Paweł Bobruk 117b63ed41 Docs fix native script usage in 5.x (#22362) 2016-12-29 09:41:26 -08:00
Clinton Gormley 3999e5ba6b Docs: Added link from bool and constant score query to filter context
Closes #22353
2016-12-29 11:05:28 +01:00
Itamar Syn-Hershko dea8cee70f Fixing a second "second" reference in docs (#22345) 2016-12-28 18:50:10 +01:00
David Pilato 4fba1c562f Merge pull request #22235 from dadoonet/doc/dbq-java-api
Add documentation for Delete By Query Java API
2016-12-23 16:04:19 +01:00
Jason Tedor 2713549533 Use reader for doc stats
Today we try to pull stats from index writer but we do not get a
consistent view of stats. Under heavy indexing, this inconsistency can
be very skewed indeed. In particular, it can lead to the number of
deleted docs being reported as negative and this leads to serialization
issues. Instead, we should provide a consistent view of the stats by
using an index reader.

Relates #22317
2016-12-23 09:44:56 -05:00
David Pilato 2511442a92 Merge pull request #22300 from dadoonet/doc/ingest-attachment
Adds more information about ingest attachment properties extraction
2016-12-23 11:30:19 +01:00
David Pilato e1b6166f21 Makes more obvious that we expect an array 2016-12-23 11:28:12 +01:00
David Pilato 08f556c133 Merge pull request #22310 from gameldar/patch-3
Add ingest-attachment-with-arrays section to ingest attachments doc
2016-12-23 10:33:23 +01:00
gameldar b100f18505 Fix the ingest attachment array examples
Fix up the ingest attachment array handling example so they are full
examples and validated by the build system correctly.
2016-12-23 13:51:14 +08:00
Ryan Ernst fb690ef748 Settings: Add infrastructure for elasticsearch keystore
This change is the first towards providing the ability to store
sensitive settings in elasticsearch. It adds the
`elasticsearch-keystore` tool, which allows managing a java keystore.
The keystore is loaded upon node startup in Elasticsearch, and used by
the Setting infrastructure when a setting is configured as secure.

There are a lot of caveats to this PR. The most important is it only
provides the tool and setting infrastructure for secure strings. It does
not yet provide for keystore passwords, keypairs, certificates, or even
convert any existing string settings to secure string settings. Those
will all come in follow up PRs. But this PR was already too big, so this
at least gets a basic version of the infrastructure in.

The two main things to look at.  The first is the `SecureSetting` class,
which extends `Setting`, but removes the assumption for the raw value of the
setting to be a string. SecureSetting provides, for now, a single
helper, `stringSetting()` to create a SecureSetting which will return a
SecureString (which is like String, but is closeable, so that the
underlying character array can be cleared). The second is the
`KeyStoreWrapper` class, which wraps the java `KeyStore` to provide a
simpler api (we do not need the entire keystore api) and also extend
the serialized format to add metadata needed for loading the keystore
with no assumptions about keystore type (so that we can change this in
the future) as well as whether the keystore has a password (so that we
can know whether prompting is necessary when we add support for keystore
passwords).
2016-12-22 16:28:34 -08:00
Nik Everett f5f2149ff2 Remove much ceremony from parsing client yaml test suites (#22311)
* Remove a checked exception, replacing it with `ParsingException`.
* Remove all Parser classes for the yaml sections, replacing them with static methods.
* Remove `ClientYamlTestFragmentParser`. Isn't used any more.
* Remove `ClientYamlTestSuiteParseContext`, replacing it with some static utility methods.

I did not rewrite the parsers using `ObjectParser` because I don't think it is worth it right now.
2016-12-22 11:00:34 -05:00
Adrien Grand e39942fc02 `value_type` is useful regardless of scripting. (#22160)
Today we only expose `value_type` in scriptable aggregations, however it is
also useful with unmapped fields. I suspect we never noticed because
`value_type` was not documented (fixed) and most aggregations are scriptable.

Closes #20163
2016-12-22 14:35:12 +01:00
Gameldar e3eb363882 Link directly to the attachments in arrays section
The link should be made to the relevant section of the ingest attachments documentation, rather than the top of the page.
2016-12-22 20:52:08 +08:00
Francesc Gil dec6fc2d40 Repeated language analyzers (#22240)
* Repeated language analyzers

The `catalan` analyzer was repeated on the supported list :)

* Reordered the languages to have alphabetic order

* Added space for format

* Reordered the languages and removed repeated
2016-12-21 17:32:02 +01:00
gameldar d404ee3533 Add ingest-attachment-with-arrays section to ingest attachments doc
Added a new section detailing how to use the attachment processor
within an array.

This reverts commit #22296 and instead links to the foreach processor.
2016-12-22 00:18:33 +08:00
David Pilato 2adb310508 Merge pull request #22308 from nicpalmer/master
Support for eu-west-2 (London) cloud-aws plugin

See:

* http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region
* http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region
2016-12-21 16:57:42 +01:00
Adrien Grand 84edf36f11 Make `-0` compare less than `+0` consistently. (#22173)
Our `float`/`double` fields generally assume that `-0` compares less than `+0`,
except when bounds are exclusive: an exclusive lower bound on `-0` excludes
`+0` and an exclusive upper bound on `+0` excludes `-0`.

Closes #22167
2016-12-21 16:51:45 +01:00
Nic Palmer 3894ec9bae Fixed eu-west-2 entries for discovery-ec2 and repository-s3 also updated the asciidocs 2016-12-21 15:48:07 +00:00
Rui Hu 80f8dfe852 Fixed document mistake and fit for 5.1.1 API 2016-12-21 08:18:16 -05:00
David Pilato 6d96cdb87c Add // CONSOLE to the snippet 2016-12-21 12:46:01 +01:00
David Pilato 80843afb19 Adds more information about ingest attachment properties extraction
This is coming from thsi thread on discuss: https://discuss.elastic.co/t/ingest-attachment-plugin-exception/69167/10
2016-12-21 12:13:16 +01:00
David Pilato 345125237d Merge pull request #22296 from gameldar/patch-2
Add link to foreach processor to ingest-attachment.asciidoc
2016-12-21 11:41:28 +01:00
Clinton Gormley 9b22ec1c6a Add ID for percolate query to Java API docs 2016-12-21 10:54:27 +01:00
Gameldar 71c223c383 Add link to foreach processor to ingest-attachment.asciidoc
Add a Note to the foreach processor for processing an array of attachments, as this is required to be able to process an array of attachments.
2016-12-21 15:15:19 +08:00
Chris Earle 61e1678e66 [DOCS] Update Percolate Java example
The percolator's Java example was note quite right. This updates it to use working code.
2016-12-20 19:07:36 -05:00
Tal Levy c53b2ee9cd introduce KV Processor in Ingest Node (#22272)
Now you can parse field values of the `key=value` variety and have
`key` be inserted as a field name in an ingest document.

Closes #22222.
2016-12-20 13:26:17 -08:00
Tal Levy 5a90d9d7e6 add `ignore_missing` flag to ingest plugins (#22273)
added `ignore_missing` flag to:

- Attachment Processor
- GeoIP Processor
- User-Agent Processor
2016-12-20 10:53:28 -08:00
Tal Levy ad4b1ecdeb [docs] update ingest-node delete docs to mention wildcarding (#22270) 2016-12-20 10:52:17 -08:00
Adrien Grand b2e93d2870 Be explicit about the fact backslashes need to be escaped. (#22257)
Relates #22255
2016-12-19 14:21:21 +01:00
Clinton Gormley f96769f97b Update painless-syntax.asciidoc
Fix asciidoc syntax
2016-12-19 10:09:24 +01:00
Daniel Mitterdorfer 655a95a2bb Cache results of geoip lookups (#22231)
With this commit, we introduce a cache to the geoip ingest processor.
The cache is enabled by default and caches the 1000 most recent items.
The cache size is controlled by the setting `ingest.geoip.cache_size`.

Closes #22074
2016-12-19 10:06:12 +01:00
Daniel Mitterdorfer 3ce7b119d2 Enable strict duplicate checks for all XContent types (#22225)
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default for all XContent types (not only JSON).

We have also changed the name of the system property to disable this feature
from `es.json.strict_duplicate_detection` to the now more appropriate name
`es.xcontent.strict_duplicate_detection`.

Relates elastic/elasticsearch#19614
Relates elastic/elasticsearch#22073
2016-12-19 09:29:47 +01:00
Daniel Mitterdorfer 6327e35414 Change type of ingest doc meta-data field 'TIMESTAMP' to `Date` (#22234)
With this commit we change the data type of the 'TIMESTAMP'
meta-data field from a formatted date string to a plain
`java.util.Date` instance. The main reason for this change is
that our benchmarks have indicated that this contributes
significantly to the time spent in the ingest pipeline.

The overhead in terms of indexing throughput of the ingest
pipeline is about 15% and breaks down roughly as follows:

* 5% overhead caused by the conversion from `XContent` -> `Map`
* 5% overhead caused by the timestamp formatting
* 5% overhead caused by the conversion `Map` -> `XContent`

Relates #22074
2016-12-19 09:10:58 +01:00
David Pilato efe5a75d26 Merge remote-tracking branch 'origin/master' 2016-12-17 08:31:57 +01:00
Jason Tedor 30806af6bd Rename bootstrap.seccomp to bootstrap.system_call_filter
We try to install a system call filter on various operating systems
(Linux, macOS, BSD, Solaris, and Windows) but the setting
(bootstrap.seccomp) to control this is named after the Linux
implementation (seccomp). This commit replaces this setting with
bootstrap.system_call_filter. For backwards compatibility reasons, we
fallback to bootstrap.seccomp and log a deprecation message if
bootstrap.seccomp is set. We intend to remove this fallback in
6.0.0. Note that now is the time to make this change it's likely that
most users are not making this setting anyway as prior to version 5.2.0
(currently unreleased) it was not necessary to configure anything to
enable a node to start up if the system call filter failed to install
(we marched on anyway) but starting in 5.2.0 it will be necessary in
this case.

Relates #22226
2016-12-16 18:22:54 -05:00
Tal Levy bb37167946 Enables the ability to inject serialized json fields into root of document. (#22179)
The JSON processor has an optional field called "target_field".
If you don't specify target_field then target_field becomes what you specified as "field".
There isn't anyway to add the fields to the root of a document. By
setting `add_to_root`, now serialized fields will be inserted into the
top-level fields of the ingest document.

Closes #21898.
2016-12-16 10:17:27 -08:00
David Pilato 72ee65f914 Add documentation for Delete By Query Java API
Closes #22114
2016-12-16 18:11:55 +01:00
David Pilato e32c7f1d72 Explain how to use bulk processor in a test context
When using a bulk processor in test, you might write something like:

```java
BulkProcessor bulkProcessor = BulkProcessor.builder(client, new BulkProcessor.Listener() {
    @Override public void beforeBulk(long executionId, BulkRequest request) {}
    @Override public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {}
    @Override public void afterBulk(long executionId, BulkRequest request, Throwable failure) {}
})
        .setBulkActions(10000)
        .setFlushInterval(TimeValue.timeValueSeconds(10))
        .build();

for (int i = 0; i < 10000; i++) {
    bulkProcessor.add(new IndexRequest("foo", "bar", "doc_" + i)
            .source(jsonBuilder().startObject().field("foo", "bar").endObject()
    ));
}

bulkProcessor.flush();
client.admin().indices().prepareRefresh("foo").get();
SearchResponse response = client.prepareSearch("foo").get();
// response does not contain any hit
```

The problem is that by default bulkProcessor defines the number of concurrent requests to 1 which is using behind the scene an Async BulkRequestHandler.
When you call `flush()` in a test, you expect it to flush all the content of the bulk so you can search for your docs.
But because of the async handling, there is a great chance that none of the documents has been indexed yet when you call the `refresh` method.

We should advice in our Java guide to explicitly set concurrent requests to `0` so users will use behind the scene the Sync BulkRequestHandler.

```java
BulkProcessor bulkProcessor = BulkProcessor.builder(client, new BulkProcessor.Listener() {
    @Override public void beforeBulk(long executionId, BulkRequest request) {}
    @Override public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {}
    @Override public void afterBulk(long executionId, BulkRequest request, Throwable failure) {}
})
        .setBulkActions(5000)
        .setFlushInterval(TimeValue.timeValueSeconds(10))
        .setConcurrentRequests(0)
        .build();
```

Closes #22158.
2016-12-16 16:45:56 +01:00
David Pilato 43f9cd1fd4 Merge branch 'pr/update-docs-51' 2016-12-16 16:22:07 +01:00
David Pilato 14976555f2 Fix Link to virtual hosting of buckets
Was using markdown style instead of asciidoc
2016-12-16 15:25:41 +01:00
Pablo Musa 152efe95e6 Small typo fix in the docs. (#22180)
There is a small typo in the convert processor code example.
2016-12-16 14:50:06 +01:00
Florian Hopf 0e18782d11 Update bucket-script-aggregation.asciidoc (#22219)
Example is missing "params." for painless
2016-12-16 12:39:22 +01:00
Masaru Hasegawa a0185c83a7 Merge pull request #21393 from masaruh/alias_boost
Resolve index names in indices_boost
2016-12-16 15:07:51 +09:00
Jason Tedor 41ffb008ad Fix doc bug for cgroup cpuacct usage metric
This commit fixes a silly doc bug where the field that represents the
total CPU time consumed by all tasks in the same cgroup was mistakenly
reported as "usage" instead of "usage_nanos".

Relates #21029
2016-12-15 23:22:54 -05:00
Nik Everett 61597f2c20 Send error_trace by default when testing (#22195)
Sends the `error_trace` parameter with all requests sent by the
yaml test framework, including the doc snippet tests. This can be
overridden by settings `error_trace: false`. While this drift's
core's handling of the yaml tests from the client's slightly this
should only be a problem for tests that rely on the default value,
both of which I've fixed by setting the value explicitly.

This also escapes `\n` and `\t` in the `Stash dump on failure` so
the `stack_trace` is more readable.

Also fixes `RestUpdateSettingsAction` to not think of the `error_trace`
parameter as a setting.
2016-12-15 13:35:14 -05:00
Areek Zillur cdd5fbe3a1 Deprecate _suggest endpoint in favour of _search (#20305)
* Replace _suggest endpoint to _search in docs

In 5.0, the _suggest endpoint is just sugar for _search
with suggestions specified. Users should move away from
using the _suggest endpoint, as it is marked as deprecated in 5.x and
will be removed in 6.0

* update docs to use _search endpoint instead of _suggest

* Add deprecation logging to RestSuggestAction

* Use search endpoint instead of suggest endpoint in rest tests
2016-12-14 21:49:53 -05:00
Isabel Drost-Fromm c18f032b08 Fix typo in command for checking single doc file
The recommended command line option for running the doc check on a single documentation file contained a typo.

h/t to @nik9000 for finding the typo.
2016-12-14 13:44:57 +01:00
David Pilato 320dacd6d2 Update script metric aggregation for 5.0
From 5.0, we are now using painless.
2016-12-14 11:44:10 +01:00
David Pilato 11a6248344 Update script query doc for 5.1
From 5.1, we changed the order of Script class ctor.

Related to https://github.com/elastic/elasticsearch/pull/21321#issuecomment-266432519
2016-12-14 11:36:55 +01:00
David Pilato 87a016a155 Update search template doc for 5.1
From 5.1, we can't use anymore `ScriptService.ScriptType` but `ScriptType`.

Related to https://github.com/elastic/elasticsearch/pull/21136#issuecomment-266429243
2016-12-14 11:36:32 +01:00
Clinton Gormley 571b20137a Document using round-robin DNS for discovery (#21810)
* Document using round-robin DNS for discovery

Closes #8427

* Improve wording around round robin DNS

* Included jason's suggestion
2016-12-14 11:29:39 +01:00
Daniel Mitterdorfer 7e5058037b Enable strict duplicate checks for JSON content
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default. This ensures that JSON keys are always unique. While this has
a performance impact, benchmarking has indicated that the typical drop in
indexing throughput is around 1 - 2%.

As a last resort, we allow users to still disable strict duplicate checks
by setting `-Des.json.strict_duplicate_detection=false` which is
intentionally undocumented.

Closes #19614
2016-12-14 09:35:53 +01:00
Lee Hinman a4e8b5d952 Throw an exception on unrecognized "match_mapping_type"
When using dynamic templates, ES will now throw an exception if a
`match_mapping_type` is used that doesn't correspond to an actual type.

Relates to #17285
2016-12-12 09:59:48 -07:00
Matt Sicker 32fcb7220e Clarify first search API explanation of parameters (#21958)
Added info about the sort parameter and included the pretty parameter in the example.
2016-12-12 15:00:53 +01:00
yutaono abf718832d add missing url-access-control link (#21960) 2016-12-12 14:56:52 +01:00
Luca Cavanna 6d987a9b69 Remove support for empty queries (#22092)
Our query DSL supports empty queries (`{}`), which have a different meaning depending on the query that holds it, either ignored, match_all or match_none. We deprecated the support for empty queries in 5.0, where we log a deprecation warning wherever they are used.

The way we supported it once we moved query parsing to the coordinating node was having an Optional<QueryBuilder> return type in all of our parse methods (called fromXContent). See #17624. The central place for this was QueryParseContext#parseInnerQueryBuilder. We can now remove all the optional return types and simply throw an exception whenever an empty query is found.
2016-12-12 12:37:12 +01:00
Luca Cavanna 73cf002293 Un-deprecate fuzzy query (#22088)
When we decided to deprecate and remove fuzzy query in #15760, we didn't realize we would take away the possibililty for uses to use a fuzzy query as part of a span query, which is not possible using match query. This means we have to go back and un-deprecate fuzzy query, which will not be removed.

Closes #15760
2016-12-12 12:09:16 +01:00
Masaru Hasegawa 3df2a086d4 Resolve index names in indices_boost
This change allows specifying alias/wildcard expression in indices_boost.
And added another format for specifying indices_boost. It accepts array of index name and boost pair.
If an index is included in multiple aliases/wildcard expressions, the first match will be used.
With new format, old format is marked as deprecated.

Closes #4756
2016-12-11 21:41:49 +09:00
Matthew Haugen ca8d89af5a Correct typo in "detect_noop" documentation (#22039)
The documentation reads:

> You can disable this behavior by setting "detect_noop": false like this:

Followed by a code example, that originally set `"detect_noop": true`.

Please correct me if I got the change backwards (i.e. the paragraph should be changed to `true`), but this seems like it makes the most sense.
2016-12-09 15:24:08 -05:00
Igor Motov 93b5e55660 Restores the original default format of search slow log
In 5.0, the search slow log switched to the multi-line format with no option to get back to the origin single-line format that was used prior to 5.0 by default. This commit removes the reformat option from the search slow log and returns the search slow log back to the single-line format.

Closes #21711
2016-12-09 12:38:28 -05:00
Matias Anaya beb794cb0f Fix typo in percolated-query.asciidoc (#21991) 2016-12-09 13:45:57 +01:00
Juan Paulo Gutierrez acc39b1496 Fixed typo (#22066) 2016-12-09 10:34:29 +01:00
Adrien Grand 787519ee4c Fix `other_bucket` on the `filters` agg to be enabled if a key is set. (#21994)
Closes #21951
2016-12-09 09:48:48 +01:00
Adrien Grand 9524c81af9 Document the `locale` option of the `date` field. (#22050)
This also adds another level of protection against using the default locale.
Relates to https://discuss.elastic.co/t/mapping-for-12h-date-format/68433/3.
2016-12-09 09:45:53 +01:00
David Pilato 18a3d6b4f3 S3/Azure snapshot repo documentation wrong for "read_only"
We used to write that people should use `read_only` although it should be `readonly`.

Closes #22007.
2016-12-08 16:57:50 +01:00
Jared Carey 317866894e Fix systemd override example in configuring docs
When overriding a systemd configuration via a drop-in file, the
[Service] header is required. This commit adds this to an example
drop-in override in the configuring docs.

Relates #22038
2016-12-07 19:41:59 -05:00
Guilherme 9fbfe540d5 add Lua client (#22028)
Add entry for elasticsearch-lua (https://github.com/DhavalKapil/elasticsearch-lua)
2016-12-07 11:24:28 -07:00
Thibault Pierre e494d6a94e Fix wrong link (#22019) 2016-12-07 17:58:46 +01:00
Colin Goodheart-Smithe 8006b105f3 Update order examples to use max instead of avg (#22032)
The use of the avg aggregation for sorting the terms aggregation is not encouraged since it has unbounded error. This changes the examples to use the max aggregation which does not suffer the same issues
2016-12-07 16:00:24 +00:00
David Pilato 8923b36780 Merge pull request #21956 from alexshadow007/aws_read_timeout
Add setting to set read timeout for EC2 discovery and S3 repository plugins
2016-12-07 16:00:48 +01:00
Nikhil Patel b5e3d351d9 Fix typos in threads docs
This commit fixes a typo in the threads docs where the past tense form
of a verb was used when current tense is needed.

Relates #22016
2016-12-07 08:22:49 -05:00
Lucas Bremgartner 0086b99797 [Docs] Correct setting name in snapshot/restore documentation (#22023)
There is no setting include_cluster_state for snapshot restore. The correct name for this setting is include_global_state.
2016-12-07 14:12:10 +01:00
Alexander Kazakov 0a03a62ab6 Using ClientConfiguration.DEFAULT_SOCKET_TIMEOUT as default value for read timeout 2016-12-06 21:13:28 +03:00
Jim Ferenczi b42ca6bcc9 Include unindexed field in FieldStats response (#21821)
* Include unindexed field in FieldStats response

This change adds non-searchable fields to the FieldStats response. These fields do not have min/max informations but they can be aggregatable. Fields that are only stored in _source (store:no, index:no, doc_values:no) will still be missing since they do not have any useful information to show. Indices and clients must be at least on V_5_2_0 to see this change.
2016-12-06 13:32:57 +01:00
Alexander Kazakov 1491e2dec9 Remove default value for read_timeout setting
Fix tests and docs
2016-12-05 21:29:17 +03:00
Nik Everett 2087234d74 Timeout improvements for rest client and reindex (#21741)
Changes the default socket and connection timeouts for the rest
client from 10 seconds to the more generous 30 seconds.

Defaults reindex-from-remote to those timeouts and make the
timeouts configurable like so:
```
POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "socket_timeout": "1m",
      "connect_timeout": "10s"
    },
    "index": "source",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}
```

Closes #21707
2016-12-05 10:54:51 -05:00
Alexander Kazakov 23550f277b Add us-east-2 AWS region 2016-12-04 20:02:05 +03:00
Alexander Kazakov 5695eaf19e Add setting to set read timeout for EC2 discovery and S3 repository plugins 2016-12-04 01:58:53 +03:00
Jason Tedor 0afef53a17 Add system call filter bootstrap check
Today if system call filters fail to install on startup, we log a
message but otherwise march on. This might leave users without system
call filters installed not knowing that they have implicitly accepted
the additional risk. We should not be lenient like this, instead clearly
informing the user that they have to either fix their configuration or
accept the risk of not having system call filters installed. This commit
adds a bootstrap check that if system call filters are enabled, they
must successfully install.

Relates #21940
2016-12-02 14:27:54 -05:00
Allen Torres 887fbb6387 Update lowercase-tokenizer.asciidoc (#21896)
Fixed typo
2016-12-02 10:49:51 -05:00
nelsonSchwarz ee35bba6b6 elasticsearch.R is abandoned (#21897)
README file reads: (This project is no longer maintained, if you would like to take over feel free to send me an email)
2016-12-02 10:39:20 -05:00
Arthur a30efacca4 [DOC] Update node names (#21892)
Update the node names in the documentation.
2016-12-02 14:02:23 +01:00
Jason Tedor 1e645bc2d6 Fix reference to jvm.options docs
The reference for the jvm.options docs recently changed from
es-java-opts to jvm-options. This commit fixes a broken reference that
arose as a result of this change.
2016-12-01 20:05:19 -05:00
Jason Tedor b9df2e2287 Improve the out-of-the-box experience
Elasticsearch can be run in a few different ways:
 - from the command line on Linux and Windows
 - as a service on Linux and Windows

on both 32-bit client and 64-bit server VMs. We strive for a great
out-of-the-box experience any of these combinations but today it is
lacking on 32-bit client JVMs and on the Windows service. There are two
deficiencies that arise:
 - on any 32-bit client JVM we fail to start out of the box because we
   force the server JVM in jvm.options
 - when installing the Windows service, the thread stack size must be
   specified in jvm.options

This commit attempts to address these deficiencies.

We should continue to force the server JVM because there are systems
where the server JVM is not active by default (e.g., the 32-bit JDK on
Windows). This does mean that if a user tries to run with a client JVM
they will see a failure message at startup but this is the best that we
can do if we want to continue to force the server JVM. Thus, this commit
at least documents this situation.

To improve the situation with installing the Windows service, this
commit adds a default setting for the thread stack size. This default is
chosen based on the default thread stack size across all 64-bit server
JVMs. This means that if a user tries to run with a 32-bit JVM they
could otherwise see significantly higher memory usage (this situation is
complicated, it's really only on Windows where the extra memory usage is
egregious, but cutting into the 32-bit address space on any system is
bad). So this commit makes it so that the out-of-the-box experience is
improved for the Windows service on 64-bit server JVMs and we document
the need to adjust this setting on 32-bit JVMs.

Again, we are focusing on the out-of-the-box experience here and this
means optimizing for the best experience on any 64-bit server JVM as
this covers the vast majority of the user base. The users that are on
32-bit JVMs will suffer a little bit but at least now any user on any
64-bit server JVM can start Elasticsearch out of the box.

Finally, we fix some references to the jvm.options documentation.

Relates #21920
2016-12-01 17:26:29 -05:00
Jason Tedor 32df032c59 Add option to skip kernel parameters on install
During package install on systemd-based systems, we try to set
vm.max_map_count. On some systems (e.g., containers), users do not have
the ability to tune these parameters from within the container. This
commit provides an option for these users to skip setting such kernel
parameters.

Relates #21899
2016-12-01 17:23:51 -05:00
Luca Cavanna 103984a4a1 Remove indices query (#21837)
The indices query is deprecated since 5.0.0 (#17710). It can now be removed in master (future 6.0 version).
2016-11-30 19:37:01 +01:00
Jason Tedor 0fe6b12553 Remove misplaced parenthesis from configuring docs
This commit removes a misplaced parenthesis from the systemd section of
the configuring docs.
2016-11-30 12:11:47 -05:00
Adrien Grand eed5de20e0 Remove docs for the removed `geo_distance_range` query. 2016-11-30 16:36:55 +01:00
Adrien Grand c5b9c98b99 Remove the `default` store type. (#21616)
It used to be a hybrid store between `niofs` and `mmapfs`, which we removed when
we switched to `fs` by default (which is `mmapfs` on 64-bits systems).
2016-11-30 15:33:26 +01:00
Adrien Grand 90ab477f19 The `terms` query should always map to a Lucene `TermsQuery`. (#21786)
Currently, the `terms` query is just syctactic sugar for a `bool` query when
used in a query context. This change proposes to always generate the same query
in query and filter contexts, which is less confusing.
2016-11-30 15:29:09 +01:00
David Pilato 8b5223c37a Update Java documentation for 5.0
Remove old links
Fix template search layout
2016-11-30 14:51:06 +01:00
David Pilato e967fdaa13 Merge branch 'doc/java-api' 2016-11-30 14:15:27 +01:00
David Pilato 13f4b96ff2 Update Java documentation for 5.0
Fix after review
2016-11-30 14:13:52 +01:00
Adrien Grand 6231009a8f Remove 2.x backward compatibility of mappings. (#21670)
For the record, I also had to remove the geo-hash cell and geo-distance range
queries to make the code compile. These queries already throw an exception in
all cases with 5.x indices, so that does not hurt any more.

I also had to rename all 2.x bwc indices from `index-${version}` to
`unsupported-${version}` to make `OldIndexBackwardCompatibilityIT`
happy.
2016-11-30 13:34:46 +01:00
Adrin Jalali 235e6acd73 typo fix (and -> any) (#21860) 2016-11-30 12:56:00 +01:00
javanna bbebc644f9 [DOCS] document breaking changes added with #21852 2016-11-29 20:05:30 +01:00
Luca Cavanna f253621feb Remove deprecated query names: in, geo_bbox, mlt, fuzzy_match and match_fuzzy (#21852)
These query names were all deprecated in 5.0.0:
- in is removed in favour of terms
- geo_bbox is removed in favour of geo_bounding_box
- mlt is removed in favour of more_like_this
- fuzzy_match and match_fuzzy are removed in favour of match
2016-11-29 19:07:01 +01:00
Jim Ferenczi d791ddf704 Upgrade to lucene-6.4.0-snapshot-ec38570 (#21853)
Set lucene version to 6.4.0-snapshot-ec38570 and update all the sha1s/license
Fix invalid combo after upgrade in query_string query. split_on_whitespace=false is disallowed if auto_generate_phrase_queries=true
Adapt the expectations of some tests to the new format of the Lucene explain output
2016-11-29 18:40:31 +01:00
Nicholas Knize af1ab68b64 Add RangeFieldMapper for numeric and date range types
Lucene 6.2 added index and query support for numeric ranges. This commit adds a new RangeFieldMapper for indexing numeric (int, long, float, double) and date ranges and creating appropriate range and term queries. The design is similar to NumericFieldMapper in that it uses a RangeType enumerator for implementing the logic specific to each type. The following range types are supported by this field mapper: int_range, float_range, long_range, double_range, date_range.

Lucene does not provide a DocValue field specific to RangeField types so the RangeFieldMapper implements a CustomRangeDocValuesField for handling doc value support.

When executing a Range query over a Range field, the RangeQueryBuilder has been enhanced to accept a new relation parameter for defining the type of query as one of: WITHIN, CONTAINS, INTERSECTS. This provides support for finding all ranges that are related to a specific range in a desired way. As with other spatial queries, DISJOINT can be achieved as a MUST_NOT of an INTERSECTS query.
2016-11-29 10:10:14 -06:00
Itamar Syn-Hershko c3a95a6666 Fixing cut-in-middle paragraph (#21850) 2016-11-29 14:00:26 +01:00
Luca Cavanna c25f9b5fba [DOCS] add source filtering example to reindex docs (#21835) 2016-11-29 09:22:54 +01:00
Matt Weber 04e07bcdb6 Synonym Graph Support (LUCENE-6664) (#21517)
Integrate the patch from LUCENE-6664 into elasticsearch and
add support for handling a graph token stream in match/multi-match
queries.

This fixes longstanding bugs with multi-token synonyms returning
incorrect results with proximity queries.
2016-11-28 09:25:49 -08:00
David Pilato f8cf9f790b Update Java documentation for 5.0
Some of the methods have been removed or deprecated.

Also related to #21825.
2016-11-28 17:33:40 +01:00
Adrin Jalali eec05ec208 then -> than (#21829) 2016-11-28 17:04:56 +01:00
Sho Yoshida db13b4c2bc Added Smalltalk client (#21361)
> Pharo Smalltalk
> Pharo emerged as a fork of Squeak Smalltalk. It focuses on modern software engineering and development techniques.

project:
- http://pharo.org/ Pharo Smalltalk
2016-11-28 11:05:17 +01:00