Commit Graph

631 Commits

Author SHA1 Message Date
Martijn van Groningen 7e2696c570 Refactored inner hits parsing and intoduced InnerHitBuilder
Both top level and inline inner hits are now covered by InnerHitBuilder.
Although there are differences between top level and inline inner hits,
they now make use of the same builder logic.

The parsing of top level inner hits slightly changed to be more readable.
Before the nested path or parent/child type had to be specified as encapsuting
json object, now these settings are simple fields. Before this was required
to allow streaming parsing of inner hits without missing contextual information.

Once some issues are fixed with inline inner hits (around multi level hierachy of inner hits),
top level inner hits will be deprecated and removed in the next major version.
2016-03-30 15:15:56 +02:00
Simon Willnauer 8b075dbb75 Remove ability to specify arbitrary node attributes with `node.` prefix
Today the basic node settings like `node.data` and `node.master` can't really be fully validated
since we allow to specify custom user attributes on the node level. We have to, in order to
support that, add a wildcard setting for `node.*` to let these setting pass validation.
Instead we should require a more contraint prefix like `node.attr.` that defines a namespace
that is reserved for user attributes.
This commit adds a new namespace for attributes in `node.attr`.

Closes #17280
2016-03-30 13:29:48 +02:00
Isabel Drost-Fromm f27399dc0e Merge pull request #17282 from MaineC/deprecation/sort-option-reverse-removal
Remove deprecated reverse option from sorting
2016-03-30 11:02:19 +02:00
javanna 19eeb68bc4 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 21:53:22 +02:00
javanna ae34c20a62 add node.client breaking changes to migrate guide 2016-03-29 20:33:59 +02:00
Isabel Drost-Fromm 407e2cdcf9 Merge branch 'master' into deprecation/sort-option-reverse-removal
Conflicts:
	core/src/main/java/org/elasticsearch/search/sort/ScoreSortBuilder.java
	core/src/test/java/org/elasticsearch/search/sort/FieldSortBuilderTests.java
2016-03-29 11:04:02 +02:00
spalger ce44bbfadf [docs] clarify where discovery.zen.minimum_master_node is required
https://github.com/elastic/elasticsearch/pull/17288 added a check to enforce that the `discovery.zen.minimum_master_nodes` configuration is set when nodes have the `host`, `port`, or `bind_host` set in either `transport` or general `network` configuration sections. This was documented incorrectly as "nodes that are bound to a non-loopback interface", which lead to confusion as I set `network.host: "localhost"` and the check was still failing.

This change updates the docs to detail the actual check. I think it also highlights how complex the check is and the need for a simpler solution.
2016-03-28 12:53:40 -07:00
Boaz Leskes b8227a7222 Enforce `discovery.zen.minimum_master_nodes` is set when bound to a public ip #17288
discovery.zen.minimum_master_nodes is the single most important setting to set on a production cluster. We have no way of supplying a good default so it must be set by the user. Binding a node to a public IP (as opposed to the default local host) is a good enough indication that a node will be part of a production cluster cluster and thus it's a good tradeoff to enforce the settings. Note that nothing prevent users from setting it to 1 in a single node cluster.

Closes #17288
2016-03-25 12:56:20 +01:00
Boaz Leskes 6dd164d0bd Include pings from client nodes in master election
We currently have a `discovery.zen.master_election.filter_client` setting that control whether their ping responses are ignored for master election (which is the current default). With the push to treat client nodes as normal nodes (and promote the transport/rest clients for client work), this should be changed. This commit remove this setting and it's companion `discovery.zen.master_election.filter_data` setting (currently defaulting to  false) in favor of singe `discovery.zen.master_election.ignore_non_master_pings` setting with more intuitive name (defaulting to false).

Resolves #17325
Closes #17329
2016-03-24 17:48:05 +01:00
javanna ce86fc5647 Cluster Stats: remove mem section
The available memory metric was always set to `0` since 2.0.beta1 (bug).  was left behind but never set. Turns out the section wasn't that useful, as it would only output the total memory available throughout all nodes in the cluster. We decided to remove the section then.
2016-03-24 15:49:27 +01:00
Isabel Drost-Fromm 08d989d9b6 Merge branch 'master' into deprecation/sort-option-reverse-removal
Conflicts:
	core/src/main/java/org/elasticsearch/search/sort/FieldSortBuilder.java
	core/src/main/java/org/elasticsearch/search/sort/ScoreSortBuilder.java
2016-03-24 12:06:10 +01:00
Isabel Drost-Fromm 801d178ade Remove mention of reverse in docs and add to migration doc 2016-03-24 12:04:31 +01:00
Jim Ferenczi da42f199bd Enforce isolated mode for all plugins
This commit removes the isolated option, each plugin have its own classloader.
2016-03-24 09:17:33 +01:00
Areek Zillur e16e113691 Remove suggest threadpool
In #17198, we removed suggest transport action, which
used the `suggest` threadpool to execute requests. Now
`suggest` threadpool is unused and suggest requests are
executed on the `search` threadpool.
2016-03-23 18:01:45 -04:00
Areek Zillur 442a6e0009 document suggest stats being merged with search stats 2016-03-23 16:37:57 -04:00
Areek Zillur e7e93f98e3 add migration guide to use search api for suggest 2016-03-23 16:37:57 -04:00
Colin Goodheart-Smithe d6fe7515fd Merge pull request #17243 from colings86/docs/searchRequestBreakingChanges
added breaking changes for the Java API to the breaking changes doc for 5.0
2016-03-22 15:58:40 +00:00
Colin Goodheart-Smithe 25c4446942 iter 2016-03-22 15:58:12 +00:00
Colin Goodheart-Smithe ee7e84acc3 review comments 2016-03-22 15:34:47 +00:00
Adrien Grand c52b1f3a7c An `exists` query on an object should query a single term.
Currently if you run an `exists` query on an object, it will resolve all sub
fields and create a disjunction for all those fields. However the `_field_names`
mapper indexes paths for objects so we could query object paths directly.

I also changed the query parser to reject `exists` queries if the `_field_names`
field is disabled since it would be a big performance trap.
2016-03-22 16:26:45 +01:00
Adrien Grand b42f66c8ac Document 5.0 mapping changes. 2016-03-22 16:22:58 +01:00
Colin Goodheart-Smithe b8a96d9a65 added breaking changes for the Java API to the breaking changes doc for 5.0 2016-03-22 14:39:16 +00:00
Simon Willnauer 7f16a1d9a7 Improve upgrade experience of node level index settings
In 5.0 we don't allow index settings to be specified on the node level ie.
in yaml files or via commandline argument. This can cause problems during
upgrade if this was used extensively. For instance if analyzers where
specified on a node level this might cause the index to be closed when
imported (see #17187). In such a case all indices relying on this
must be updated via `PUT /${index}/_settings`. Yet, this API has slightly
different semantics since it overrides existing settings. To make this less
painful this change adds a `preserve_existing` parameter on that API to ensure
we have the same semantics as if the setting was applied on the node level.

This change also adds a better error message and a change to the migration guide
to ensure upgrades are smooth if index settings are specified on the node level.

If a index setting is detected this change fails the node startup and prints a message
like this:
```
*************************************************************************************
Found index level settings on node level configuration.

Since elasticsearch 5.x index level settings can NOT be set on the nodes
configuration like the elasticsearch.yaml, in system properties or command line
arguments.In order to upgrade all indices the settings must be updated via the
/${index}/_settings API. Unless all settings are dynamic all indices must be closed
in order to apply the upgradeIndices created in the future should use index templates
to set default values.

Please ensure all required values are updated on all indices by executing:

curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
  "index.number_of_shards" : "1",
  "index.query.default_field" : "main_field",
  "index.translog.durability" : "async",
  "index.ttl.disable_purge" : "true"
}'
*************************************************************************************
```
2016-03-21 20:12:18 +01:00
Martijn van Groningen e3b7e5d75a percolator: Replace percolate api with the new percolator query
Also replaced the PercolatorQueryRegistry with the new PercolatorQueryCache.

The PercolatorFieldMapper stores the rewritten form of each percolator query's xcontext
in a binary doc values field. This make sure that the query rewrite happens only during
indexing (some queries for example fetch shapes, terms in remote indices) and
the speed up the loading of the queries in the percolator query cache.

Because the percolator now works inside the search infrastructure a number of features
(sorting fields, pagination, fetch features) are available out of the box.

The following feature requests are automatically implemented via this refactoring:

Closes #10741
Closes #7297
Closes #13176
Closes #13978
Closes #11264
Closes #10741
Closes #4317
2016-03-21 12:21:50 +01:00
Martijn van Groningen 3b17ddcd46 Removed old 1.x parent/child logic that should have been removed.
`0` really means, don't match any child docs.
2016-03-18 10:07:27 +01:00
Martijn van Groningen 1dd2be81c3 nested / parent child: Removed `total` score mode in favour of `sum` score mode.
Closes #17083
2016-03-18 10:07:26 +01:00
Areek Zillur da165f425f update migration doc for removing gateway.format setting 2016-03-16 18:48:02 -04:00
Christoph Büscher 39667b5793 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	docs/reference/migration/migrate_5_0/java.asciidoc
2016-03-16 12:06:42 +01:00
Jason Tedor 618441aea3 Merge pull request #17088 from jasontedor/simplify-bootstrap-settings
Bootstrap does not set system properties
2016-03-15 19:25:16 -04:00
Jason Tedor 2f7e181318 Fix typo inadvertently introduced 2016-03-15 10:05:28 -04:00
Christoph Büscher bc84cdfed1 Using SortMode enum in all sort builders 2016-03-15 12:43:19 +01:00
Christoph Büscher b4b874f0d8 Merge branch 'master' into feature-suggest-refactoring 2016-03-15 12:11:39 +01:00
Areek Zillur 35f7cfb6c0 Add upgrader to upgrade old indices to new naming convention 2016-03-14 23:24:05 -04:00
Christoph Büscher 97638c95fc Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	docs/reference/migration/migrate_5_0.asciidoc
2016-03-14 11:13:47 +01:00
Clinton Gormley c90b4f3bae Docs: Added note about upgrading from 1.x to 5.x 2016-03-14 09:58:46 +01:00
Jason Tedor 8a05c2a2be Bootstrap does not set system properties
Today, certain bootstrap properties are set and read via system
properties. This action-at-distance way of managing these properties is
rather confusing, and completely unnecessary. But another problem exists
with setting these as system properties. Namely, these system properties
are interpreted as Elasticsearch settings, not all of which are
registered. This leads to Elasticsearch failing to startup if any of
these special properties are set. Instead, these properties should be
kept as local as possible, and passed around as method parameters where
needed. This eliminates the action-at-distance way of handling these
properties, and eliminates the need to register these non-setting
properties. This commit does exactly that.

Additionally, today we use the "-D" command line flag to set the
properties, but this is confusing because "-D" is a special flag to the
JVM for setting system properties. This creates confusion because some
"-D" properties should be passed via arguments to the JVM (so via
ES_JAVA_OPTS), and some should be passed as arguments to
Elasticsearch. This commit changes the "-D" flag for Elasticsearch
settings to "-E".
2016-03-13 20:09:15 -04:00
Jason Tedor 8ac5a98b87 Remove links to nonexistent migration docs 2016-03-13 19:12:06 -04:00
Clinton Gormley 5f48b9c86a Removed breaking changes docs for < 5.0 2016-03-13 21:18:44 +01:00
Clinton Gormley 5c845f8bb5 Reworked 5.0 breaking changes docs 2016-03-13 21:17:48 +01:00
Jason Tedor f465d98eb3 Add raw recovery progress to cat recovery API
This commit adds fields bytes_recovered and files_recovered to the cat
recovery API. These fields, respectively, indicate the total number of
bytes and files recovered. Additionally, for consistency, some totals
fields and translog recovery fields have been renamed.

Closes #17064
2016-03-11 08:27:09 -05:00
Christoph Büscher daeffb149c Merge branch 'master' into feature-suggest-refactoring 2016-03-11 10:37:28 +01:00
Daniel Mitterdorfer 94aa025b93 Document breaking change in ClusterHealthResponse in 2.2 2016-03-11 09:47:53 +01:00
Yannick Welsch 295d33c2a6 Merge pull request #17021 from ywelsch/fix/block-delete-on-snapshot
Fail closing or deleting indices during a full snapshot
2016-03-10 18:51:04 +01:00
Yannick Welsch 266394c3ab Fail closing or deleting indices during a full snapshot
Closes #16321
2016-03-10 18:11:47 +01:00
Lee Hinman 22e716551b Add -XX+AlwaysPreTouch JVM flag
Enables the touching of all memory pages used by the JVM heap spaces
during initialization of the HotSpot VM, which commits all memory pages
at initialization time. By default, pages are committed only as they are
needed.
2016-03-10 10:11:32 -07:00
Nicholas Knize f7a2dbfcaf fixing silly typo in docs 2016-03-10 07:28:13 -06:00
Christoph Büscher 69c83b3459 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/action/suggest/TransportSuggestAction.java
        core/src/main/java/org/elasticsearch/search/suggest/SuggestParseElement.java
	core/src/main/java/org/elasticsearch/search/suggest/SuggestionSearchContext.java
	core/src/main/java/org/elasticsearch/search/suggest/phrase/PhraseSuggester.java
2016-03-10 11:44:36 +01:00
Nicholas Knize 55635d5de1 update coerce and breaking changes documentation 2016-03-09 16:09:44 -06:00
Christoph Büscher 7ec5075a87 Merge branch 'master' into feature-suggest-refactoring 2016-03-07 10:45:43 +01:00
Christoph Büscher 30a788d87c Suggestions: Make field name mandatory ctor argument
The field name is a required argument for all suggesters, but
it was specified via a field() setter in SuggestionBuilder so far.
This changes field name to being a mandatory constructor argument
and lets suggestion builders throw an error if field name is missing
or the empty string.
2016-03-03 21:35:53 +01:00
Lee Hinman 6adbbff97c Fix organization rename in all files in project
Basically a query-replace of "https://github.com/elasticsearch/" with "https://github.com/elastic/"
2016-03-03 12:04:13 -07:00
Adrien Grand 2c3e4840f2 Store _all payloads on 1 byte instead of 4. #16899
This changes the `_all` field to store per-field boosts using a single byte
similarly to norms.
2016-03-03 15:00:23 +01:00
Christoph Büscher a7053afdb9 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	docs/reference/migration/migrate_5_0.asciidoc
2016-03-01 12:37:05 +01:00
Andrey Ryaguzov f744c3f724 Docs: Added migration description for custom analysis file path
Closes #15597
Closes #15556
2016-02-29 20:56:19 +01:00
Nik Everett fd729604ab [docs] Add breaking change notes for logging
ba5be0332d removed support for degrading
to slf4j and j.u.l but didn't document this as a breaking change because
it is only breaking for folks using Elasticsearch's jar as a java client.
People do that so this counts as a breaking change.

Also, if anyone was brave enough to try and replace log4j on an installed
version of Elasticsearch that will no longer work and this documents that
as well. It doens't get a full heading and instead lives with the java
client notes. Mostly because I can't imagine it worked consistently enough
for anyone to actually do it in the first place. We just never tested it
well enough to make sure we didn't break it after it was implemented.
2016-02-29 11:28:18 -05:00
Jason Tedor 0de66da2d4 Fix formatting in migration 5.0 docs 2016-02-29 11:05:42 -05:00
Nik Everett 1e1645d22b Move migrate_3_0 to migrate_5_0 2016-02-29 09:43:11 -05:00
Jason Tedor 5d8631848b Add note on Groovy dependencies to migration docs
This commit adds a note to the migration docs regarding the reduction of
the Groovy dependencies from the groovy-all artifact to the groovy
artifact that was previously done in
180ab2493e.

Closes #16858
2016-02-29 09:39:01 -05:00
Ali Beyad a7f6488216 Merge remote-tracking branch 'upstream/master' into
feature-suggest-refactoring
2016-02-26 17:21:12 -05:00
Konstantin Delchev b451d5eb07 Fix typo 2016-02-26 13:36:13 -05:00
Jason Tedor c978335968 Remove es.useLinkedTransferQueue
This commit removes the system property "es.useLinkedTransferQueue" that
defaulted to false and was used to control the queue implementation used
in a few places.

Closes #16786
2016-02-23 17:29:15 -08:00
Christoph Büscher 78534847a3 Merge branch 'master' into feature-suggest-refactoring 2016-02-23 16:47:13 -08:00
Jason Tedor 6e840b39f5 Remove ability to disable Netty gathering writes
Java NIO has the notion of gathering writes. These are writes that
gather data from multiple buffers into a single channel. These gathering
writes in Netty have been enabled by default with the possibility to
disable them using "es.netty.gathering". This flag was added in case
having gathering writes on by default did not work out. We have not
published this ability and sufficient time has passed to render
judgement that using gathering writes is okay.

Closes #16774
2016-02-22 16:17:44 -08:00
Lee Hinman 99052c3fef Limit the accepted length of the _id
Elasticsearch should reject ids that are this long, to ensure a document
always remains retrievable for clients that impose a maximum URI length

Closes #16034
2016-02-22 12:34:18 -07:00
Christoph Büscher d888b5c267 Merge branch 'master' into feature-suggest-refactoring 2016-02-22 11:22:28 -08:00
Jason Tedor fa885f2e96 Remove es.max-open-files flag
This commit removes the es.max-open-files flag as the same information
can be obtained from the cluster nodes info API, and is warn logged on
startup if it's set too low anyway.

Closes #16757
2016-02-21 21:01:08 -08:00
Clinton Gormley aedfa10b43 Change docs example with script on/off to true/false
Closes #15469
2016-02-15 17:53:59 +01:00
Ali Beyad 5320d538f1 Merge remote-tracking branch 'upstream/master' into feature-suggest-refactoring 2016-02-15 11:03:33 -05:00
David Pilato 55d9b6878b Deprecate Mapper Attachment Plugin
Now that we have the ingest-attachment plugin (https://github.com/elastic/elasticsearch/pull/16490)  we should deprecate the mapper-attachment plugin.

Closes #16650.
2016-02-15 16:40:12 +01:00
Clinton Gormley cca611171c Document breaking change in terms query/filter in 2.0
Closes #14830
2016-02-14 18:13:03 +01:00
Jason Tedor 3bbd1c129e Remove host from cat nodes API
As the host and ip fields are always equal by design, the host field in
the cat nodes API is redundant and should be removed.

Closes #16656
2016-02-14 09:21:32 -05:00
Ali Beyad beea413820 Merge remote-tracking branch 'upstream/master' into feature-suggest-refactoring 2016-02-12 11:54:44 -05:00
Simon Willnauer b5aee2075f Merge pull request #16610 from s1monw/test_indices_request_cache
Refactor IndicesRequestCache to make it testable.
2016-02-12 09:27:20 +01:00
Ali Beyad eed557742f Refactors building query specific objects from the term suggestion
builder and merges changes between the suggestion builders to context
object implementations.
2016-02-11 18:30:58 -05:00
Simon Willnauer c50586599e Make security non-optional
2.x has show so far that running with security manager is the way to go.
This commit make this non-optional. Users that need to pass their own rules
can still do this via the system configuration for the security manager. They
can even opt out of all security that way.
2016-02-11 17:11:24 +01:00
Simon Willnauer d538dd64c2 Refactor IndicesRequestCache to make it testable.
This commit moves IndicesRequestCache into o.e.indics and makes all API in this
class package private. All references to SearchReqeust, SearchContext etc. have been factored
out and relevant glue code has been added to IndicesService. The IndicesRequestCache is not a
simple class without any hard dependencies on ThreadPool nor SearchService or IndexShard. This now
allows to add unittests.
This commit also removes two settings `indices.requests.cache.clean_interval` and `indices.fielddata.cache.clean_interval`
in favor of `indices.cache.clean_interval` which cleans both caches.
2016-02-11 16:51:44 +01:00
Jim Ferenczi b146f3ecb3 Pack all the plugin files into a single folder named elasticsearch at the root of the plugin zip. 2016-02-10 10:13:05 +01:00
Dongjoon Hyun 21ea552070 Fix typos in docs. 2016-02-09 02:07:32 -08:00
Spencer cc0d0525fc Fix asciidoc typo 2016-02-08 10:49:27 -07:00
Clinton Gormley 0c41a68690 Fixed asciidoc issues 2016-02-05 14:47:44 +01:00
Jason Tedor 21e22a2ddc Reword cat API breaking changes 2016-02-05 06:28:39 -05:00
Andrej Kazakov 7f2b369dfd Use Accept header field in cat API
The cat API previously used the Content-Type header field for
determining the media type of the response. This is in opposition to the
HTTP spec which specifies the Accept header field for this purpose. This
commit replaces the use of the Content-Type header field with the Accept
header field in the cat API.

Closes #14421
2016-02-05 06:28:39 -05:00
Jim Ferenczi 7d0181b5d4 Rename bin/plugin in bin/elasticsearch-plugin 2016-02-05 10:09:14 +01:00
Simon Willnauer 3de0004ea3 add migration notice 2016-02-04 16:23:58 +01:00
Yannick Welsch 4937531a17 Remove obsolete version in ShardRouting
Closes #16243
2016-02-04 15:50:25 +01:00
Simon Willnauer e60b0bb8fa Apply feedback from @clintongormley 2016-02-03 11:38:28 +01:00
Simon Willnauer 4a4e523357 Merge branch 'master' into make_settings_strict 2016-02-03 11:34:12 +01:00
Simon Willnauer 9a74443238 Add notice in migration docs 2016-02-03 11:20:53 +01:00
Clinton Gormley c7a818d691 Merge pull request #16370 from pra85/patch-1
Fix minor typo in migrate_3_0.asciidoc
2016-02-02 16:15:30 +01:00
Clinton Gormley f640bf0636 Merge pull request #16371 from clintongormley/deprecate-multicast
Deprecate the discovery-multicast plugin
2016-02-02 15:23:58 +01:00
Prayag Verma 10b3071e16 Fix minor typo in migrate_3_0.asciidoc
Remove extra `on`
2016-02-02 18:16:58 +05:30
Clinton Gormley 70f9d60585 Merge pull request #16020 from mfussenegger/migrate-2_0_settings
2.0 Breaking Changes - Settings docs don't mention index.gateway.local.sync
2016-02-02 13:19:17 +01:00
Ryan Ernst 7b5ed21d0d Remove reference to multicast docs, which no longer exist. 2016-02-01 11:27:03 -08:00
Ryan Ernst 3787f437ec Merge branch 'master' into remove_multicast 2016-02-01 07:25:45 -08:00
Christoph Büscher 61c435e6a9 PhraseSuggestionBuilde: Refactor DirectCandidateGenerator
As a prerequisite for refactoring the whole PhraseSuggestionBuilder
to be able to be parsed and streamed from the coordinating node, the
DirectCandidateGenerator must implement Writeable, be able to parse
a new instance (fromXContent()) and later when transported to the
shard to generate a PhraseSuggestionContext.DirectCandidateGenerator.
Also adding equals/hashCode and tests and moving DirectCandidateGenerator
to its own DirectCandidateGeneratorBuilder class.
2016-02-01 12:22:48 +01:00
Clinton Gormley 880c60a982 Added missing breaking changes to 2.2 2016-02-01 09:51:36 +01:00
Clinton Gormley 676078c53d Docs: Java Security Manager and scripting
Added docs explaining the impact of the Java Security Manager on scripting
languages, how to disable the JSM, and how to customise the classloader
whitelist.

Closes https://github.com/elastic/elasticsearch/issues/16094
Closes https://github.com/elastic/elasticsearch/issues/14290
2016-02-01 09:38:21 +01:00
Ryan Ernst b8f08c35ec Plugin: Remove multicast plugin
closes #16310.
2016-01-29 18:41:31 -08:00
Martijn van Groningen 9ec1b11148 term vectors: The term vector APIs no longer modify the mappings if an unmapped field is found 2016-01-29 09:32:40 +01:00
Ryan Ernst 60180fecf8 Add migration note for site plugins removal 2016-01-27 08:11:57 -08:00
Simon Willnauer 1df7d4d5b5 Merge pull request #16257 from s1monw/no_fsync_on_every_operation
Remove the ability to fsync on every operation and only schedule fsync task if really needed
2016-01-27 15:48:30 +01:00
Simon Willnauer 84ce9f3618 Remove the ability to fsync on every operation and only schedule fsync task if really needed
This commit limits the `index.translog.sync_interval` to a value not less than `100ms` and
removes the support for fsync on every operation which used to be enabled if `index.translog.sync_interval` was set to `0s`
Now this pr also only schedules an async fsync if the durability is set to `async`. By default not async task is scheduled.

Closes #16152
2016-01-27 12:28:38 +01:00
Jason Tedor 284cc3a048 Script mode settings as booleans
This commit modifies the accept values for script mode settings from
"on", "off", and "sandbox" to "true", "false", and "sandbox".
2016-01-27 06:26:58 -05:00
Jason Tedor 087e55cc51 Script mode settings
This commit converts the script mode settings to the new settings
infrastructure. This is a major refactoring of the handling of script
mode settings. This refactoring is necessary because these settings are
determined at runtime based on the registered script engines and the
registered script contexts.
2016-01-27 06:26:04 -05:00
Adrien Grand 209860854d Make the `index` property a boolean.
With the split of `string` into `text` and `keyword`, the `index` property can
only have two values and should be a boolean.
2016-01-27 09:06:00 +01:00
Adrien Grand 2aaa5e6448 Remove the ability to enable doc values with the `fielddata.format` setting.
Doc values can now only be enabled by setting `doc_values: true` in the
mappings. Removing this feature also means that we can now fail mapping updates
that try to disable doc values.
2016-01-27 09:06:00 +01:00
Adrien Grand f959d39ac3 Fix default doc values to be enabled when a field is not indexed.
Doc values currently default to `true` if the field is indexed and not analyzed.
So setting `index:no` automatically disables doc values, which is not explicit
in the documentation.

This commit makes doc values default to true for numerics, booleans regardless
of whether they are indexed. Not indexed strings still don't have doc values,
since we can't know whether it is rather a text or keyword field. This
potential source of confusion should go away when we split `string` into `text`
and `keyword`.
2016-01-27 09:06:00 +01:00
Christoph Büscher 97114af8ae Merge pull request #16014 from cbuescher/refactor/rescore-fromXContent
RescoreBuilder: Add parsing and creating of RescoreSearchContext

Adding the ability to parse from xContent to the rescore builder. Also making RescoreBuilder an abstract base class that encapsulates the window_size setting, with QueryRescoreBuilder as its only implementation at the moment.

Relates to #15559
2016-01-26 15:42:35 +01:00
Christoph Büscher 98446e7dd3 Adding java api changes to migrate_3_0.asciidoc 2016-01-26 15:26:47 +01:00
Martijn van Groningen 8b02f214c4 percolator: The percolate api shouldn't add field mappings for unmapped fields inside the document being percolated to the mapping.
Closes #15751
2016-01-26 10:26:46 +01:00
Jason Tedor fa4e6020cd Break out packaging breaking changes 2016-01-22 06:28:13 -05:00
Jason Tedor 26e5cd651e Default standard output to the journal in systemd
This commit modifies the default setting for standard output in the
systemd configuration to the journal instead of /dev/null. This is to
address a user pain point where Elasticsearch would fail to start but
the error message would be sent to standard output and therefore
/dev/null leading to difficult-to-debug situations.
2016-01-21 15:09:57 -05:00
Martijn van Groningen 7914d0f940 Several MapperService cleanups:
* Cleaned up MapperService#searchFilter(...) and moved it DefaultSearchContext, since it that class was the only user. As part of the cleanup percolate query documents are no longer excluded from the search response.
* Removed resolveClosestNestedObjectMapper(...) method as it was no longer used.
* Removed DocumentTypeListener infrastructure. Before it was used by the percolator and parent/child, but these features no longer use it.

Closes #15924
2016-01-20 22:26:56 +01:00
Martijn van Groningen b9dc5acf2c Cleanup ParentFieldMapper:
* Remove remaining 1.x bwc logic.
* Stop storing stored fields and indexed terms. The _parent field's only purpose is to support joins between parent and child type and only storing doc values is sufficient.
* In the mapping the parent field mapper is now known under '{parent}#{child}' key, because this is the field the parent/child join uses too.
* Added new sub fetch phase to lookup that _parent field from doc values field if that is required (before this was fetched from stored _parent field)
* Removed the ability to query directly on `_parent` in the query dsl. Instead the `{parent}#{child}` field should be used. Under the hood a doc values query is used instead of a term query, because only doc values fields are stored now.
* Added a new `parent_id` query to easily query child documents with a specific parent id without having to know what join field to use
* Also in aggregations `_parent` field can't be used any more and `{parent}#{child}` field name should be used instead to aggregate directly on the _parent join field.
2016-01-20 16:56:22 +01:00
Yannick Welsch d5b691b68e Extend reroute with an option to force assign stale primary shard copies
Closes #15708
2016-01-19 12:07:01 +01:00
Yannick Welsch a1b8dd2de9 Add per-index setting to limit number of nested fields
Closes #14983
2016-01-19 10:03:48 +01:00
Mathias Fussenegger d31abab827 add note about `index.gateway.local.sync` to 2.0 migration/settings docs 2016-01-18 20:39:09 +01:00
Jason Tedor df598e8129 Modify load average formats
This commit modifies the load_average in the node stats API response
to be an object containing the one-minute, five-minute and
fifteen-minute load averages as fields (if those values are
available). Additionally, this commit modifies the cat nodes API
response to format the one-minute, five-minute and fifteen-minute load
averages as null if any of the respective values are not available.
2016-01-18 11:41:34 -05:00
Nik Everett a8ceb37d07 Merge pull request #15913 from nik9000/deprecate_collect_payloads
Deprecate span_near's collect_payloads
2016-01-12 09:56:19 -05:00
Michael McCandless b4a095d430 Merge pull request #14121 from mikemccand/fair_indexing_buffers
The indexing buffer on a node (default: 10% of the JVM heap) is now a "shared pool" across all shards on that node.  This way, shards doing intense indexing can use much more than other shards doing only light indexing, and only once the sum of all indexing buffers across all shards exceeds the node's indexing buffer will we ask shards to move recently indexed documents to segments on disk.
2016-01-12 04:59:10 -05:00
Jason Tedor d9fd6e2fe3 Merge pull request #15907 from jasontedor/load-average
Reintroduce five-minute and fifteen-minute load averages on Linux

Relates #12049, relates #14741
2016-01-11 23:43:40 -05:00
Jason Tedor 1de2081ed3 Reintroduce five-minute and fifteen-minute load averages on Linux
This commit reintroduces the five-minute and fifteen-minute load stats
on Linux, and changes the format of the load_average field back to an
array.
2016-01-11 23:42:47 -05:00
Nik Everett ae0bf1b32f Deprecate span_near's collect_payloads 2016-01-11 22:11:38 -05:00
Michael McCandless 07e8370322 feedback 2016-01-11 17:16:28 -05:00
Clinton Gormley f782c8ac07 Merge pull request #15891 from sromocki/patch-1
Update crud.asciidoc, nouns mixed up
2016-01-11 20:01:16 +01:00
Adrien Grand 67d233cecd Remove warmers and the warmer API.
Warmers are now barely useful and will be removed in 3.0. Note that this only
removes the warmer API and query-based warmers. We still have warmers internally
for eg. global ordinals.

Close #15607
2016-01-07 09:57:07 +01:00
Martijn van Groningen 2d6adf6428 Percolator refactoring:
* Added percolator field mapper that extracts the query terms and indexes these terms with the percolator query.
* At percolate time these extracted terms are used to query percolator queries that are like to be evaluated. This can significantly cut down the time it takes to percolate. Whereas before all percolator queries were evaluated if they matches with the document being percolated.
* Changes made to percolator queries are no longer immediately visible, a refresh needs to happen before the changes are visible.
* By default the percolate api only returns upto 10 matches instead of returning all matching percolator queries.
* Made percolate more modular, so that it is easier to add unit tests.
* Added unit tests for the percolator.

Closes #12664
Closes #13646
2016-01-06 16:08:10 +01:00
Jim Ferenczi 992ffac509 Merge pull request #15446 from jimferenczi/classic_similarity
Renames `default` similarity into `classic`
2015-12-30 08:42:20 -08:00
Adrien Grand a7e1eeb13d Merge pull request #15684 from jpountz/deprecate/field_boost
Remove object notation for core types.
2015-12-30 09:58:01 +01:00
Adrien Grand bffbad3c19 Remove object notation for core types.
When specifying a string field, you can either do:

```
{
  "foo": "bar"
}
```

or

```
{
  "foo": {
    "value": "bar",
    "boost": 42
  }
}
```

The latter option is now removed.

Closes #15388
2015-12-28 19:49:18 +01:00
Simon Willnauer f5e4cd4616 Remove recovery threadpools and throttle outgoing recoveries on the master
Today we throttle recoveries only for incoming recoveries. Nodes that have a lot
of primaries can get overloaded due to too many recoveries. To still keep that at bay
we limit the number of threads that are sending files to the target to overcome this problem.

The right solution here is to also throttle the outgoing recoveries that are today unbounded on
the master and don't start the recovery until we have enough resources on both source and target nodes.

The concurrency aspects of the recovery source also added a lot of complexity and additional threadpools
that are hard to configure. This commit removes the concurrent streamns notion completely and sends files
in the thread that drives the recovery simplifying the recovery code considerably.
Outgoing recoveries are not throttled on the master via a allocation decider.
2015-12-22 14:59:43 +01:00
Simon Willnauer 3a5d3a3bb0 Remove deprecated query cache settings 2015-12-22 09:39:12 +01:00
Simon Willnauer fcfd98e9e8 Drop support for simple translog and hard-wire buffer to 8kb
Today we have two variants of translogs for indexing. We only recommend the buffered
one which also has a 20% advantage in indexing speed. This commit removes the option and defaults
to the buffered case. It also hard-wires the translog buffer to 8kb instead of 64kb. We used to
adjust that buffer based on if the shard is active or not, this code has also been removed and
instead we just keep an 8kb buffer arround.
2015-12-21 16:44:35 +01:00
Jim Ferenczi 81fd2169cf Renames "default" similarity into "classic".
Replaces deprecated DefaultSimilarity by ClassicSimilarity.
Fixes #15102
2015-12-21 16:22:53 +01:00
Simon Willnauer 613e8a9782 add note to migration guide 2015-12-21 15:20:32 +01:00
Yannick Welsch 3a442db9bd Allocate primary shards based on allocation ids
Closes #15281
2015-12-17 15:55:50 +01:00
Adrien Grand 9fd843748e Remove the `format` option of the `_source` field.
This option allows to force the xcontent type to use to store the `_source`
document. The default is to use the same format as the input format.

This commit makes this option ignored for 2.x indices and rejected for 3.0
indices.
2015-12-11 18:06:20 +01:00
Jim Ferenczi d78e24689b Merge pull request #15364 from jimferenczi/missing_query_removal
Remove the MissingQueryBuilder which was deprecated in 2.2.0.
2015-12-11 17:20:47 +01:00
Jim Ferenczi 437488ae64 Remove the MissingQueryBuilder which was deprecated in 2.2.0.
As a replacement use ExistsQueryBuilder inside a mustNot() clause.
So instead of using `new ExistsQueryBuilder(name)` now use:
`new BoolQueryBuilder().mustNot(new ExistsQueryBuilder(name))`.

Closes #14112
2015-12-10 12:55:05 +01:00
David Pilato 414fccb7d1 Merge branch 'fix/15268-proxy-auth' 2015-12-09 23:21:00 +01:00
Adrien Grand f2d8a35888 Dynamically map floating-point numbers as floats instead of doubles.
Close #13851
2015-12-08 18:19:47 +01:00
Britta Weber e0aa481bf5 Merge pull request #15213 from brwe/copy-to-in-multi-fields-exception
throw exception if a copy_to is within a multi field

Copy to within multi field is ignored from 2.0 on, see #10802.
Instead of just ignoring it, we should throw an exception if this
is found in the mapping when a mapping is added. For already
existing indices we should at least log a warning.
We remove the copy_to in any case.

related to #14946
2015-12-08 14:41:07 +01:00
David Pilato 7dcb40bcac Add support for proxy authentication for s3 and ec2
When using S3 or EC2, it was possible to use a proxy to access EC2 or S3 API but username and password were not possible to be set.

This commit adds support for this. Also, to make all that consistent, proxy settings for both plugins have been renamed:

* from `cloud.aws.proxy_host` to `cloud.aws.proxy.host`
* from `cloud.aws.ec2.proxy_host` to `cloud.aws.ec2.proxy.host`
* from `cloud.aws.s3.proxy_host` to `cloud.aws.s3.proxy.host`
* from `cloud.aws.proxy_port` to `cloud.aws.proxy.port`
* from `cloud.aws.ec2.proxy_port` to `cloud.aws.ec2.proxy.port`
* from `cloud.aws.s3.proxy_port` to `cloud.aws.s3.proxy.port`

New settings are `proxy.username` and `proxy.password`.

```yml
cloud:
    aws:
        protocol: https
        proxy:
            host: proxy1.company.com
            port: 8083
            username: myself
            password: theBestPasswordEver!
```

You can also set different proxies for `ec2` and `s3`:

```yml
cloud:
    aws:
        s3:
            proxy:
                host: proxy1.company.com
                port: 8083
                username: myself1
                password: theBestPasswordEver1!
        ec2:
            proxy:
                host: proxy2.company.com
                port: 8083
                username: myself2
                password: theBestPasswordEver2!
```

Note that `password` is filtered with `SettingsFilter`.

We also fix a potential issue in S3 repository. We were supposed to accept key/secret either set under `cloud.aws` or `cloud.aws.s3` but the actual code never implemented that.

It was:

```java
account = settings.get("cloud.aws.access_key");
key = settings.get("cloud.aws.secret_key");
```

We replaced that by:

```java
String account = settings.get(CLOUD_S3.KEY, settings.get(CLOUD_AWS.KEY));
String key = settings.get(CLOUD_S3.SECRET, settings.get(CLOUD_AWS.SECRET));
```

Also, we extract all settings for S3 in `AwsS3Service` as it's already the case for `AwsEc2Service` class.

Closes #15268.
2015-12-07 23:10:54 +01:00
Simon Willnauer 4e80a5e099 Merge pull request #15234 from s1monw/remove_ancient_settings
Remove ancient deprecated and alternative recovery settings
2015-12-07 21:15:49 +01:00
Simon Willnauer c1f7f8c03c add settings to migration guide 2015-12-07 09:54:23 +01:00
Lee Hinman f709b7283f Remove `GET` option for /_forcemerge
POST should be used to indicate this is not just a retrieval operation.

Resolves #15165
2015-12-03 13:56:15 -07:00
Clinton Gormley e84b61358f Fix version order for breaking changes docs 2015-12-03 14:24:19 +01:00
Jim Ferenczi 731833cfc6 Fixes #14489
Do not to load fields from _source when using the `fields` option.
  Non stored (non existing) fields are ignored by the fields visitor when using the `fields` option.

Fixes #10783
  Support * wildcard to retrieve stored fields when using the `fields` option.
  Supported pattern styles are "xxx*", "*xxx", "*xxx*" and "xxx*yyy".
2015-11-30 11:00:32 +01:00
Martijn van Groningen 48771f1a76 field stats: Added `min_value_as_string` and `max_value_as_string` response elements for all number based fields. The existing `min_value` and `max_value` will return the values as numbers instead.
Closes #14404
2015-11-23 08:48:28 +01:00
Christoph Büscher 991ab030dc Adding notes about ShapeBuilder changes to migration doc. 2015-11-21 12:51:28 +01:00
David Pilato 052285ac88 Revert 3cacec4468
We actually want to keep the test when using deprecated setting in 3.0.
We will keep this setting deprecated as well so people will be able to update in a smoother way.

Also add the deprecating information to the migration documentation.
2015-11-18 19:14:26 +01:00
Clinton Gormley 062abf813b Update packaging.asciidoc
Remove mention of following symlinks under `path.scripts`

Closes #14733
2015-11-18 13:48:43 +01:00
Jason Tedor 6872d545ac Add system CPU percent to OS stats
This commit adds the system CPU percent reflecting the recent CPU usage
for the whole system.
2015-11-17 13:48:46 -05:00
Peter Lebedev 1201843d72 Update java.asciidoc
Add few more breaking changes.

Closes #14696
2015-11-17 12:27:44 +01:00
Shaunak Kashyap 636c88d09f Fixing setting name 2015-11-13 13:36:14 -08:00
Clinton Gormley bf3f9c778c Docs: Added plugin-descriptor.properties to breaking changes
Closes #14569
2015-11-09 12:17:44 +01:00
Clinton Gormley f0fda74083 Moved a breaking change to the correct section. Closes #14510 2015-11-08 21:53:56 +01:00
Clinton Gormley 1849c645f3 Merge pull request #14509 from elastic/markw/typo-parentchild-releasenotes-14508
Fixed minor typo
2015-11-08 21:48:40 +01:00
Clinton Gormley 28015d9e03 Docs: Made the list of breaking changes in each version more obvious 2015-11-08 12:59:07 +01:00
Clinton Gormley 6fa799e694 Docs: Correct docs around CONF_DIR and -Des.path.conf 2015-11-05 14:05:52 +01:00
Mark Walkom 4f2154b0e0 Removed / from commands 2015-11-04 14:56:06 +11:00
Mark Walkom ab0cba4bbf Removed / from command 2015-11-04 14:54:40 +11:00
Mark Walkom 97a066829c Fixed minor typo 2015-11-04 14:46:37 +11:00
javanna ce6aa258a9 Remove support for query_binary and filter_binary
query_binary and filter_binary are unused at this point, as we only parse on the coordinating node and the java api only holds structured java objects for queries and filters, meaning they all implement Writeable and get natively serialized.

Relates to #14308
Closes #14433
2015-11-03 14:05:45 +01:00
Jason Tedor e3b8dc7121 Forbid changing thread pool types
This commit forbids the changing of thread pool types for any thread
pool. The motivation here is that these are expert settings with
little practical advantage.

Closes #14294, relates #2509, relates #2858, relates #5152
2015-11-02 20:52:48 -05:00
Nik Everett 28633fae21 [mapping] Remove transform
Removes the mapping transform feature which when used made debugging very
difficult. Users should transform their documents on the way into
Elasticsearch rather than having Elasticsearch do it.

Closes #12674
2015-10-30 11:46:54 -04:00
Mark Walkom 10ad6ae10f Mention changes to repos.
Fixes 14376
2015-10-30 18:46:31 +11:00
Lee Hinman 3b5058017e Merge branch 'remove-optimize-rest' 2015-10-29 15:18:03 -06:00
Clinton Gormley 16fb84bc34 Merge pull request #14296 from Dwaligon/master
Fixing misspelled words in documentation
2015-10-29 20:24:23 +01:00
javanna 93b57089d0 update migrate guide according to recent deprecation removals 2015-10-28 10:50:29 +01:00
Lee Hinman 3a458af0b7 Remove /_optimize REST API endpoint
The `/_optimize` endpoint was deprecated in 2.1.0 and can now be removed
entirely.
2015-10-27 10:17:16 -06:00
javanna dc900a08a6 Remove "query" query and fix related parsing bugs
We have two types of parse methods for queries: one for the inner query, to be used once the parser is positioned within the query element, and one for the whole query source, including the query element that wraps the actual query.

With the search refactoring we ended up using the former in count, cat count and delete by query, whereas we should have used the former.  It ends up working properly given that we have a registered (deprecated) query called "query", which used to allow to wrap a filter into a query, but this has the following downsides:
1) prevents us from removing the deprecated "query" query
2) we end up supporting a top level query that is not wrapped within a query element (pre 1.0 syntax iirc that shouldn't be supported anymore)

This commit finally removes the "query" query and fixes the related parsing bugs. We also had some tests that were providing queries in the wrong format, those have been fixed too.

Closes #13326
Closes #14304
2015-10-27 14:54:30 +01:00
Jason O'Donnell 05d5d9bd12 Fixing typo 2015-10-26 16:42:45 -04:00
Robert Muir 6c8e290322 Allow binding to multiple addresses
* Allow for multiple host specifications (e.g. _en0_,192.168.1.2,_site_).
* Add _site_ and _global_ scopes as counterparts to _local_.
* Warn on heuristic selection of publish address.
* Remove the arbitrary _non_loopback_ setting.

Closes #13954
2015-10-23 23:43:37 -04:00
javanna 75cedca0da Remove search exists api
Closes #13682
Closes #13911
2015-10-21 17:39:32 +02:00
Lee Hinman b3646f9bfc Merge remote-tracking branch 'dakrone/use-the-force-luke' 2015-10-20 21:13:18 -06:00
Christoph Büscher 5d25bc30cd Query DSL: Remove NotQueryBuilder
The NotQueryBuilder has been deprecated on the 2.x branches
and can be removed with the next major version. It can be
replaced by boolean query with added mustNot() clause.

Closes #13761
2015-10-20 19:43:16 +02:00
Lee Hinman 9ea4909035 Add Force Merge API, deprecate Optimize API
This adds an API for force merging lucene segments. The `/_optimize` API is now
deprecated and replaced by the `/_forcemerge` API, which has all the same flags
and action, just a different name.
2015-10-20 09:00:24 -06:00
Jason Tedor 18b32f0ba5 Remove cache concurrency level settings that no longer apply
This commit removes some cache concurrency level settings that were
applicable when the cache was backed by the Guava cache implementation,
but no longer apply with the cache implementation completed in #13717.

Relates #7836, relates #13224, relates #13717
2015-10-20 10:15:03 -04:00
javanna a6e7a5f307 Java api: remove the count api
Closes #14166
Closes #13928
2015-10-19 14:40:52 +02:00
Adrien Grand fc0a69273d Merge pull request #14066 from sarwarbhuiyan/mapping-spelling-error
changed ben to been
2015-10-13 09:42:54 +02:00
Adrien Grand d3aa3565db Deprecate `index.analysis.analyzer.default_index` in favor of `index.analysis.analyzer.default`.
Close #11861
2015-10-12 22:19:16 +02:00
Sarwar Bhuiyan b876a4d571 changed he to the 2015-10-12 15:16:49 +01:00
Sarwar Bhuiyan 460c74f584 changed ben to been 2015-10-12 14:39:55 +01:00
Clinton Gormley 499d53c506 Docs: Document ES_CLASSPATH removal in breaking changes
Closes #14008
2015-10-08 18:22:53 +02:00
Clinton Gormley c208add625 Merge pull request #14019 from ankon/patch-1
Fix minor typo
2015-10-08 18:19:00 +02:00
Clinton Gormley 8afddec2c9 Merge pull request #14031 from ankon/patch-3
Make the sentence actually understandable
2015-10-08 18:17:40 +02:00
Andreas Kohn 2ffc2979d7 [doc] Make the sentence actually understandable 2015-10-08 16:37:17 +02:00
Andreas Kohn 3bb24921a9 Remove excess comma in code block 2015-10-08 13:23:29 +02:00
Andreas Kohn a119e82ed0 Fix minor typo 2015-10-08 11:47:55 +02:00
David Pilato 8b03d261ff Add missing breaking doc for azure and aws 2015-10-08 06:53:37 +02:00
David Pilato c73ab50df1 Rename cloud-gce plugin to discovery-gce plugin
Follow up azure and aws splits, we need to be consistent and rename `cloud-gce` to `discovery-gce`.
2015-10-08 06:53:37 +02:00
Nik Everett b32022dce1 Merge pull request #13977 from nik9000/doc_breaking_java
Add a note about shard failure in the api
2015-10-07 10:36:50 -04:00
Boaz Leskes bcb3fab6ac Engine: Remove Engine.Create
The `_create` API is handy way to specify an index operation should only be done if the document doesn't exist. This is currently implemented in explicit code paths all the way down to the engine. However, conceptually this is no different than any other versioned operation - instead of requiring a document is on a specific version, we require it to be deleted (or non-existent). This PR removes Engine.Create in favor of a slight extension in the VersionType logic.

There are however a couple of side effects:
- DocumentAlreadyExistsException is removed and VersionConflictException is used instead (with an improved error message)
- Update will reject version parameters if the upsert option is used (it doesn't compute anyway).
- Translog.Create is also removed infavor of Translog.Index (that's OK because their binary format was the same, so we can just read Translog.Index of the translog file)

Closes #13955
2015-10-07 12:37:34 +02:00
Nik Everett 36a6211523 [doc] Add a note about shard failure in the api
Closes #13674
2015-10-06 16:24:39 -04:00
Britta Weber df81497e9f add paragraph about removal of conf file option 2015-10-06 19:36:48 +02:00
javanna e8653f5156 Java api: IdsQueryBuilder to accept only non null ids and types
Types are still optional, but if you do provide them, they can't be null. Split the existing constructor that accepted nnull into two, one that accepts no arguments, and another one that accepts the types argument, which must be not null.

Also trimmed down different ways of setting ids, some were misleading as they would always add the ids to the existing ones and not set them, the add prefix makes that clear. Left `addIds` method that accepts a varargs argument. Added check for ids not be null.
2015-10-05 15:10:30 +02:00
javanna eaafc62f53 [DOCS] added query-refactoring changes to 3.0 migrate guide 2015-09-25 14:55:37 +02:00
javanna 629ac8e014 Java api: remove TermsLookupQueryBuilder
TermsLookupQueryBuilder was left around only for bw comp reasons, but TermsQueryBuilder is its replacement. We can remove it now that it is clear query refactoring goes in master (3.0).
2015-09-25 11:44:05 +02:00
javanna 34de79370f Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/ElasticsearchException.java
2015-09-25 09:38:24 +02:00
Jason Tedor 4fe243a5ca Merge pull request #13701 from jasontedor/fix-ping-timeout-settings-inconsistency
Fix ping timeout settings inconsistencies
2015-09-23 13:17:46 -04:00
Jason Tedor d8b29f7beb Fix ping timeout settings inconsistencies
This commit fixes ping timeout settings inconsistencies in
ZenDiscovery. In particular, the documentation refers to the ping
timeout setting as discovery.zen.ping_timeout but the code was
ultimately using discovery.zen.ping.timeout if this was set.

This commit also changes all instances of the raw string
“discovery.zen.ping_timeout” to the constant
o.e.d.z.ZenDiscovery.SETTING_PING_TIMEOUT.

Finally, this commit removes the legacy setting
"discovery.zen.initial_ping_timeout".

Closes #6579, #9581, #9908
2015-09-23 12:58:23 -04:00
Christoph Büscher dbb01f5b43 Query Refactoring: Move MoreLikeThisQueryBuilder validation to constructor
The current MoreLikeThisQueryBuilder validation checks for existence of at
least one `like` text or item. This is hard to check in setters, so this PR
tries to change the construction of the query so that we can do these checks
already at construction time.

Changing to using arrays for fieldnames, likeTexts, likeItems, unlikeTexts
and unlikeItems. `likeTexts` and/or `likeItems` need to be specified at
construction time to validate we have at least one item there.

Relates to #10217
2015-09-23 18:20:46 +02:00
Antonio Bonuccelli 0149ed41f4 Update settings.asciidoc
generalising to single dash
2015-09-23 14:54:51 +02:00
Antonio Bonuccelli 8da0f89739 Order of command line flags
specifying the order required for cli -Des notation arguments
2015-09-23 14:54:50 +02:00
javanna 7eedd84dc3 Query refactoring: refactor query_string query and score functions
Refactor the function_score query so it can be parsed on the coordinating node, split parse into fromXContent and toQuery, make FunctionScoreQueryBuilder Writeable.

Closes #13653
2015-09-23 11:32:54 +02:00
Christoph Büscher 03035a28a3 Query Refactoring: moving validation to setters and constructors
Moving validation from validate() to constructors and setters for the
following query builders:

* GeoDistanceQueryBuilder
* GeoDistanceRangeQueryBuilder
* GeoPolygonQueryBuilder
* GeoShapeQueryBuilder
* GeohashCellQuery
* TermsQueryBuilder

Relates to #10217
2015-09-22 11:05:56 +02:00
Christoph Büscher cc69de5c5f Merge branch 'master' into feature/query-refactoring 2015-09-21 15:38:43 +02:00
Thomas Cucchietti 03488e3902 Docs: Update query_dsl.asciidoc
Minor orthographic fix

Closes #13452
2015-09-19 16:09:53 +02:00
Christoph Büscher 37b46fb79c Query Refactoring: moving validation to constructors and setters
This PR is the second batch in moving the query validation we started
to collect in the validate() method to the corresponding setters
and constructors.
2015-09-17 23:30:42 +02:00
Christoph Büscher 887399eebf Query Refactoring: moving validation to constructors and setters
This PR is an initial step to move the query validation we started
to collect in the validate() method to the corresponding setters
and constructors.
2015-09-17 13:57:34 +02:00
Alex Ksikes a3c68f690b Refactors MultiMatchQueryBuilder and Parser
Relates to #10217

This PR is against the query-refactoring branch.

Closes #13405
2015-09-16 13:41:22 +02:00
Christoph Büscher 8fb1aa9781 Refactors GeoDistanceQueryBuilder/-Parser
Splits parsing and Lucene query generation. Switches from storing lat/lon
separately to using GeoPoint instead.

Relates to #10217
2015-09-16 11:15:32 +02:00
Alex Ksikes a13336da54 Refactors MoreLikeThisQueryBuilder and Parser
Relates to #10217

This PR is against the query-refactoring branch.

Closes #13486
2015-09-16 02:09:38 +02:00
javanna 73f7df510e Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/FunctionScoreQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/FunctionScoreQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/factor/FactorParser.java
	core/src/test/java/org/elasticsearch/ExceptionSerializationTests.java
	core/src/test/java/org/elasticsearch/benchmark/search/child/ChildSearchBenchmark.java
	core/src/test/java/org/elasticsearch/benchmark/search/child/ChildSearchShortCircuitBenchmark.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
	core/src/test/java/org/elasticsearch/percolator/PercolatorIT.java
	core/src/test/java/org/elasticsearch/search/child/ChildQuerySearchIT.java
	docs/reference/query-dsl/has-parent-query.asciidoc
2015-09-11 14:15:12 +02:00
Adrien Grand 86f1b07df0 Docs: Remove docs for the `filtered`, `and`, `or` and `(f)query` queries. 2015-09-11 11:00:54 +02:00
Nik Everett e4981968ad [search] Limit the size of the result window
Requesting a million hits, or page 100,000 is always a bad idea, but users
may not be aware of this. This adds a per-index limit on the maximum size +
from that can be requested which defaults to 10,000.

This should not interfere with deep-scrolling.

Closes #9311
2015-09-10 15:38:29 -04:00
Martijn van Groningen ab0847e0df parent/child: several cleanups
* Dropped ScoreType in favour of Lucene's ScoreMode
* Removed `score_type` option from `has_child` and `has_parent` queries in favour for the already existing `score_mode` option.
* Removed the score mode `sum` in favour for the already existing `total` score mode. (`sum` doesn't exist in Lucene's ScoreMode class)
* If `max_children` is set to `0` it now really means that zero children are allowed to match.
2015-09-10 17:15:41 +02:00
javanna 56b3db6ba3 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/AndQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/AndQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/FQueryFilterParser.java
	core/src/main/java/org/elasticsearch/index/query/FilteredQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/FilteredQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/IndicesQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/LimitQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/LimitQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MultiMatchQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/QueryFilterBuilder.java
	core/src/main/java/org/elasticsearch/index/query/QueryFilterParser.java
	core/src/main/java/org/elasticsearch/index/query/QueryParseContext.java
	core/src/main/java/org/elasticsearch/index/query/ScriptQueryParser.java
	core/src/main/java/org/elasticsearch/indices/query/IndicesQueriesRegistry.java
	core/src/main/java/org/elasticsearch/search/sort/GeoDistanceSortParser.java
	core/src/main/java/org/elasticsearch/search/sort/SortParseElement.java
	core/src/test/java/org/elasticsearch/benchmark/search/child/ChildSearchShortCircuitBenchmark.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
2015-09-10 15:52:28 +02:00
Martijn van Groningen 2eadc6d595 nested sorting: If sorting by nested field then the `nested_path` should always be specified.
Closes #13420
2015-09-10 12:21:12 +02:00
Christoph Büscher 90fac17a2d Query refactoring: MatchQueryBuilder
This add equals, hashcode, read/write methods, separates toQuery and JSON parsing and adds tests.
Also moving MatchQueryBuilder.Type to MatchQuery to MatchQuery, adding serialization and hashcode,
equals there.

Relates to #10217
2015-09-10 12:14:55 +02:00
javanna 484fcd49e5 Query refactoring: query_string
Relates to #10217
2015-09-10 10:09:24 +02:00
javanna db705ab460 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/mapper/internal/ParentFieldMapper.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MoreLikeThisQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/MoreLikeThisQueryParser.java
	core/src/main/java/org/elasticsearch/index/search/child/ChildrenQuery.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
	core/src/test/java/org/elasticsearch/index/search/child/AbstractChildTestCase.java
	core/src/test/java/org/elasticsearch/index/search/child/ChildrenConstantScoreQueryTests.java
	core/src/test/java/org/elasticsearch/index/search/child/ChildrenQueryTests.java
	core/src/test/java/org/elasticsearch/search/child/ChildQuerySearchBwcIT.java
	core/src/test/java/org/elasticsearch/search/child/ChildQuerySearchIT.java
	core/src/test/resources/org/elasticsearch/index/query/simple-query-string.json
2015-09-09 17:08:22 +02:00
Martijn van Groningen 2fb2a12c52 Removed pre 2.x parent child implementation 2015-09-09 13:21:40 +02:00
Alex Ksikes a45ee273e3 MLT: builder takes a new Item object like its parser
Previously the parser could take any Term Vectors request, but this would be
not the case of the builder which would still use MultiGetRequest.Item. This
introduces a new Item class which is used by both the builder and parser.

Beyond that the rest is mostly cleanups such as:

1) Deprecating the ignoreLike methods, in favor to using unlike.

2) Deprecating and renaming MoreLikeThisBuilder#addItem to addLikeItem.

3) Ordering the methods of MoreLikeThisBuilder more logically.

This change is needed for the upcoming query refactoring of MLT.

Closes #13372
2015-09-09 11:40:27 +02:00
Ryan Ernst 55795f8ec2 Merge pull request #13383 from rjernst/go_away_transport_paths
Remove environment from transport client
2015-09-08 13:28:21 -07:00
David Pilato 81223179df [doc] Java add missing breaking changes
Closes #13151
2015-09-08 21:23:20 +02:00
Ryan Ernst 86ffa38ec7 Add migration guide note about transport client changes to
settings/environment
2015-09-08 10:22:46 -07:00
Simon Willnauer 18bec264f9 Split HasChildQueryParser into toQuery and formXContent
This is an intial commit that splits HasChildQueryParser / Builder into
the two seperate steps. This one is particularly nasty since it transports
a pretty wild InnerHits object that needs heavy refactoring. Yet, this commit
has still some nocommits and needs more tests and maybe another cleanup but
it's a start to get the code out there.
2015-09-08 17:11:57 +02:00
javanna f2605b34d6 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/ScriptQueryParser.java
2015-09-08 16:11:22 +02:00
Clinton Gormley 2c20658204 Docs: Added deprecation notice for _timestamp and _ttl 2015-09-07 21:16:19 +02:00
Adrien Grand 0c26e7cd83 Remove the scan and count search types.
These search types have been deprecated in 2.1 and 2.0 respectively, and will
be removed in 3.0.
2015-09-07 15:18:45 +02:00
javanna 52be313c69 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/BoolQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/IdsQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/SimpleQueryStringParser.java
	core/src/main/java/org/elasticsearch/index/query/TemplateQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/TermsQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/fieldvaluefactor/FieldValueFactorFunctionParser.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/GND.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/JLHScore.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/NXYSignificanceHeuristic.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/PercentageScore.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/ScriptHeuristic.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/SignificantTermsSignificanceScoreIT.java
2015-09-02 16:00:35 +02:00
xuzha f46e66e7d0 Remove the experimental indices.fielddata.cache.expire
closes #10781
2015-09-01 00:40:04 -07:00
javanna faf526dccb Merge branch 'master' into feature/query-refactoring 2015-08-31 13:28:48 +02:00
Clinton Gormley 40dbba4e64 Update settings.asciidoc
Added removal of merge and merge throttling settings to breaking changes
2015-08-28 18:31:54 +02:00
Clinton Gormley 0331e444e6 Update mapping.asciidoc
Added fielddata format removal to breaking changes docs
2015-08-28 18:12:54 +02:00
Nik Everett 54639478c0 Merge pull request #13180 from nik9000/cat_non_verbose
Default _cat verbose to false
2015-08-28 11:59:29 -04:00
Clinton Gormley a46368310a Update mapping.asciidoc
Add type name length restriction to breaking changes docs.
2015-08-28 17:52:55 +02:00
Nik Everett 8a75d33ee6 Remove migration note about cat verbosity
Its no longer true.
2015-08-28 11:24:58 -04:00
javanna 78d097de39 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/apache/lucene/queryparser/classic/MapperQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/AndQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/AndQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/BoolQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/DisMaxQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/DisMaxQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/GeoPolygonQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MoreLikeThisQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MultiMatchQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/NestedQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/PrefixQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/RegexpQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/SpanNearQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/SpanOrQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/TermQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/TermsQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/support/InnerHitsQueryParserHelper.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
2015-08-28 10:49:28 +02:00
Nik Everett 9eb684da51 Default detect_noop to true
detect_noop is pretty cheap and noop updates compartively expensive so this
feels like a sensible default.

Also had to do some testing and documentation around how _ttl works with
detect_noop.

Closes #11282
2015-08-27 10:34:18 -04:00
Dan Tuffery d8298e1d3a Update query_dsl.asciidoc
Fixed typo.'
2015-08-27 12:47:15 +02:00
Ryan Ernst fdf8e67de3 Merge branch 'master' into docs/multicast 2015-08-26 19:42:13 -07:00
Ryan Ernst 7a61adf8c4 Add links to multicast and dbq plugins 2015-08-26 19:40:00 -07:00
xuzha fb2be6d6a1 The name "position_offset_gap" is confusing because Lucene has three
similar sounding things:

* Analyzer#getPositionIncrementGap
* Analyzer#getOffsetGap
* IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS and
* FieldType#storeTermVectorOffsets

Rename position_offset_gap to position_increment_gap
closes #13056
2015-08-26 14:56:35 -07:00
Alex Ksikes 0872a6ea39 removes TermsLookup changes as it was used internally 2015-08-26 14:07:55 +02:00
Alex Ksikes 1af0a39221 Refactors TermsQueryBuilder and Parser
Refactors TermsQueryBuilder and Parser for #10217.

This PR is against the query-refactoring branch.

Closes #12042
2015-08-26 13:58:46 +02:00
Christoph Büscher efadf87371 Merge branch 'master' into feature/query-refactoring 2015-08-26 10:23:03 +02:00
Nik Everett 3f6108ab74 [Mapping] Move position_offset_gap default change
Until a couple of hours ago we expected the position_offset_gap to default
to 0 in 2.0 and 100 in 2.1. We decided it was worth backporting that new
default to 2.0. So now that its backported we need to teach 2.1 that 2.0
also defaults to 100.

Closes #7268
2015-08-25 18:01:10 -04:00