Commit Graph

4162 Commits

Author SHA1 Message Date
Simon Willnauer 8c1fe37e92 Remove in-place rewriting of SearchSourceBuilder
SearchSourceBuilder should be cloned when rewriting, the in-place rewrite
is failing with local transport assertions.
2016-02-13 08:57:32 +01:00
Simon Willnauer 25d6a143d4 Merge branch 'master' into add_rewrite_infra 2016-02-12 22:13:34 +01:00
Zachary Tong 2ebd2ba616 [TEST] Temporarily silence failing test
Failure ticket at #16645
2016-02-12 15:36:37 -05:00
Martijn van Groningen 299179e704 logging: exception is first argument 2016-02-12 18:23:22 +01:00
Luca Cavanna f589fa8013 Merge pull request #16617 from javanna/enhancement/function_score_strict_parsing
Function Score Query: make parsing stricter
2016-02-12 17:45:10 +01:00
Luca Cavanna ca23ee6104 Merge pull request #16640 from javanna/enhancement/remove_copy_constructors
Remove copy constructors from request classes and TransportMessage type
2016-02-12 17:21:55 +01:00
javanna cb1ca92618 move function score to ParseField 2016-02-12 17:15:04 +01:00
javanna 320f356030 use assertNull rather than assertTrue(object == null) 2016-02-12 17:15:04 +01:00
javanna 507b868692 [TEST] make SearchSourceBuilderTests pickier, check that the search source has been read completely
Same check is already performed in AbstractQueryTestCase, makes sense to have it here too.
2016-02-12 17:15:04 +01:00
javanna a624410450 Function Score Query: make parsing stricter
Function Score Query now checks the type of token that we are parsing, which makes parsing stricter and allows to throw useful errors in case the json is malformed. It also makes code more readable as in what gets parsed when.

Closes #16583
2016-02-12 17:15:03 +01:00
Nicholas Knize a3d425573c Upgrade GeoPointIndexFieldData to support new PREFIX Encoding
This commit updates the OrdinalsBuilder and GeoPoint FieldData loader to work with the new PREFIX_ENCODING introduced in lucene-5.5.0. Backcompat is included to support legacy encoding types.

closes #16634
2016-02-12 10:13:17 -06:00
javanna 8d0089c8d8 Remove unused generics type from TransportMessage
followup of #15776, the type is not needed anymore.
2016-02-12 16:31:35 +01:00
Colin Goodheart-Smithe 83885423a0 [TEST] mute DuelFieldDataTests.testDuelGeoPoints()
Bug URL = https://github.com/elastic/elasticsearch/issues/16634
2016-02-12 15:20:13 +00:00
javanna 5849646baa Java api: remove needless copy constructors from request classes
After #15776 got in, we don't need these copy constructors anymore. When we used to copy requests it was to make sure that headers and context were copied from the parent requests (e.g. index/delete as part of update). This is not a problem anymore.
2016-02-12 16:16:39 +01:00
Simon Willnauer 6c6ed135c5 wrap lines 2016-02-12 15:37:25 +01:00
Simon Willnauer 84d91c87a1 fix test 2016-02-12 15:34:50 +01:00
Simon Willnauer 9e0ceaa63e [TEST] fix test to be env agnostic 2016-02-12 15:25:00 +01:00
Simon Willnauer 8d568ce3e1 Move optimization out of BoolQueryBuilder into tests 2016-02-12 15:14:28 +01:00
Simon Willnauer 387a55d5f9 wrap lines 2016-02-12 13:03:21 +01:00
Simon Willnauer b906c8a389 apply fixes after review 2016-02-12 12:53:12 +01:00
Colin Goodheart-Smithe 86b0777392 removes a lot of warnings form the aggregations 2016-02-12 10:47:08 +00:00
Colin Goodheart-Smithe d9c1e88ba9 PipelineAggregatorBuilder.setMetaData is now chain able 2016-02-12 10:06:51 +00:00
Colin Goodheart-Smithe ed3f7903f4 Merge branch 'master' into feature/aggs-refactoring 2016-02-12 09:48:59 +00:00
David Pilato aabb124209 Add filtering support within Setting class
Now we have a nice Setting infra, we can define in Setting class if a setting should be filtered or not.
So when we register a setting, setting filtering would be automatically done.

Instead of writing:

```java
Setting<String> KEY_SETTING = Setting.simpleString("cloud.aws.access_key", false, Setting.Scope.CLUSTER);
settingsModule.registerSetting(AwsEc2Service.KEY_SETTING, false);
settingsModule.registerSettingsFilterIfMissing(AwsEc2Service.KEY_SETTING.getKey());
```

We could simply write:

```java
Setting<String> KEY_SETTING = Setting.simpleString("cloud.aws.access_key", false, Setting.Scope.CLUSTER, true);
settingsModule.registerSettingsFilterIfMissing(AwsEc2Service.KEY_SETTING.getKey());
```

It also removes `settingsModule.registerSettingsFilterIfMissing` method.

The plan would be to remove as well `settingsModule.registerSettingsFilter` method but it still used with wildcards. For example in Azure Repository plugin:

```java
module.registerSettingsFilter(AzureStorageService.Storage.PREFIX + "*.account");
module.registerSettingsFilter(AzureStorageService.Storage.PREFIX + "*.key");
```

Closes #16598.
2016-02-12 10:35:54 +01:00
Simon Willnauer 685bee3081 Merge branch 'master' into add_rewrite_infra 2016-02-12 09:46:01 +01:00
Simon Willnauer b5aee2075f Merge pull request #16610 from s1monw/test_indices_request_cache
Refactor IndicesRequestCache to make it testable.
2016-02-12 09:27:20 +01:00
Simon Willnauer 691c5c49ed Fix registerSettingsFilterIfMissing and add unittest for settings filter registration 2016-02-12 09:25:24 +01:00
Nik Everett 821a20f582 Merge branch 'master' into feature/reindex 2016-02-11 17:41:05 -05:00
Boaz Leskes 8bc2332d9a Decouple recovery source/target logic and transport piping
The current logic for doing recovery from a source to a target shourd is tightly coupled with the underlying network pipes. This changes decouple the two, making it easier to add unit tests for shard recovery that doesn't involve the node and network environment.

On top that, RecoveryTarget is renamed to RecoveryTargetService leaving space to renaming RecoveryStatus to RecoveryTarget (and thus avoid the confusion we have today with RecoveryState).

Correspondingly RecoverySource is renamed to RecoverySourceService.

Closes #16605
2016-02-11 21:36:48 +01:00
Nicholas Knize 52ee4c7027 upgrade to lucene 5.5.0-snapshot-850c6c2 2016-02-11 14:28:50 -06:00
Nik Everett 30107b4a74 Teach reindex to stop when cancelled
All we do is check the cancelled flag and stop the request at a few key
points.

Adds the cancellation cause to the status so any request that is cancelled
but doesn't die can be seen in the task list.
2016-02-11 15:21:22 -05:00
David Pilato df50371c34 Merge branch 'pr/16477-aws-settings' 2016-02-11 19:47:43 +01:00
gmarz 4a96f4f310 Fix failing stylecheck 2016-02-11 13:09:08 -05:00
javanna 37ff6af759 environemnt -> environment 2016-02-11 18:31:21 +01:00
javanna 8045e51ff3 Corrected typos and removed unused import 2016-02-11 18:31:20 +01:00
Greg Marzouka 2304952647 Merge pull request #16516 from gmarz/fix/pipeline-aggs-meta
Set meta data for pipeline aggregations
2016-02-11 12:27:46 -05:00
Adrien Grand 70f19b2c64 Merge pull request #16589 from jpountz/feature/keyword
Add a new `keyword` field.
2016-02-11 18:23:32 +01:00
Adrien Grand bc47c577d2 Add a new `keyword` field.
The `keyword` field is intended to replace `not_analyzed` string fields. It is
indexed and has doc values by default, and doesn't support enabling term
vectors.

Although it doesn't support setting an analyzer for now, there are plans for
it to support basic normalization in the future such as case folding.
2016-02-11 18:19:53 +01:00
Simon Willnauer d04ec6eb2c Merge pull request #16176 from s1monw/make_security_non_optional
Make security non-optional
2016-02-11 17:37:48 +01:00
Adrien Grand a1e251af20 Remove the MapperBuilders utility class.
We can just call constructors directly.
2016-02-11 17:32:58 +01:00
Simon Willnauer c50586599e Make security non-optional
2.x has show so far that running with security manager is the way to go.
This commit make this non-optional. Users that need to pass their own rules
can still do this via the system configuration for the security manager. They
can even opt out of all security that way.
2016-02-11 17:11:24 +01:00
Simon Willnauer d538dd64c2 Refactor IndicesRequestCache to make it testable.
This commit moves IndicesRequestCache into o.e.indics and makes all API in this
class package private. All references to SearchReqeust, SearchContext etc. have been factored
out and relevant glue code has been added to IndicesService. The IndicesRequestCache is not a
simple class without any hard dependencies on ThreadPool nor SearchService or IndexShard. This now
allows to add unittests.
This commit also removes two settings `indices.requests.cache.clean_interval` and `indices.fielddata.cache.clean_interval`
in favor of `indices.cache.clean_interval` which cleans both caches.
2016-02-11 16:51:44 +01:00
Colin Goodheart-Smithe ca6058efab Adds validation to the Aggregator Builder implementations 2016-02-11 14:04:24 +00:00
Colin Goodheart-Smithe 4a9c84d86d Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/search/SearchService.java
#	test/framework/src/main/java/org/elasticsearch/search/MockSearchService.java
2016-02-11 14:04:04 +00:00
Simon Willnauer 46cab6d2ec Merge pull request #16603 from s1monw/move_cache_to_indicesservice
Move IndicesQueryCache and IndicesRequestCache into IndicesService
2016-02-11 14:21:36 +01:00
Boaz Leskes efb4582eb1 Move reminder settings in NettyHttpServerTransport to the new infra
Some bw incompatible setting changes:

http.netty.http.blocking_server -> http.tcp.blocking_server
http.netty.host (removed, we just have http.host)
http.netty.bind_host (removed, we just have http.bind_host)
http.netty.publish_host (removed, we just have http.publish_host)
http.netty.tcp_no_delay -> http.tcp.no_delay
http.netty.tcp_keep_alive -> http.tcp.keep_alive
http.netty.reuse_address -> http.txp.reuse_address
http.netty.tcp_send_buffer_size -> http.tcp.send_buffer_size
http.netty.tcp_receive_buffer_size -> http.tcp.receive_buffer_size

Closes #16531
2016-02-11 14:08:10 +01:00
Yannick Welsch 94f19d7e37 Reuse existing allocation id for primary shard allocation
Closes #16530
2016-02-11 14:00:21 +01:00
Simon Willnauer de7b152736 Fix test 2016-02-11 12:09:30 +01:00
David Pilato 37b0fc4f10 Migrate AWS settings to new settings infrastructure
Reintroducing commit fb7723c but now deals with setting names conflicts
Also adds java documentation for each setting

Closes #16293.
Related to https://github.com/elastic/elasticsearch/pull/16477#discussion_r52469084
2016-02-11 12:03:09 +01:00
Simon Willnauer 7835525f45 Move IndicesQueryCache and IndicesRequestCache into IndicesService
this is a minor cleanup that detaches `IndicesRequestCache` and `IndicesQueryCache`
from guice and moves it into `IndicesService`. It also decouples the `IndexShard` and `IndexService`
from these caches which are unnecessary dependencies.
2016-02-11 12:02:21 +01:00
Simon Willnauer 8bdc7362a6 Add infrastructure to rewrite query buidlers
QueryBuilders today do all their heavy lifting in toQuery() which
can be too late for several operations. For instance if we want to fetch geo shapes
on the coordinating node we need to do all this before we create the actual lucene query
which happens on the shard itself. Also optimizations for request caching need to be done
to the query builder rather than the query which then in-turn needs to be serialized again.
This commit adds the basic infrastructure for query rewriting and moves the heavy lifting into
the rewrite method for the following queries:

 * `WrapperQueryBuilder`
 * `GeoShapeQueryBuilder`
 * `TermsQueryBuilder`
 * `TemplateQueryBuilder`

Other queries like `MoreLikeThisQueryBuilder` still need to be fixed / converted. The nice
sideeffect of this is that queries like template queries will now also match the request cache
if their non-template equivalent has been cached befoore. In the future this will allow to
add optimizataion like rewriting time-based queries into primitives like `match_all_docs` or `match_no_docs`
based on the currents shards bounds. This is especially appealing for indices that are read-only ie. never change.
2016-02-11 10:45:49 +01:00
Dongjoon Hyun ebcbe5d4c5 Fix typos in comments
Closes #16592
2016-02-10 21:22:53 -05:00
Igor Motov a5406e1ffa Adds wait for task registration to testCanFetchIndexStatus
In the testCanFetchIndexStatus the task check can occur before the indexing process is started making the test to fail. This commit adds an additional lock to make sure we check tasks only after at least one of the tasks is registered.
2016-02-10 18:33:12 -05:00
gmarz 2adce71d32 Fix MetricsAggregationBuilder missing the ability to set meta data 2016-02-10 17:52:57 -05:00
Jason Tedor b2b02f14cb Add valid duplicate classes JARs
This commit adds JARs containing valid duplicate classes for testing
JAR hell. These JARs can be recreated with
https://github.com/jasontedor/duplicate-classes.
2016-02-10 15:10:43 -05:00
Nik Everett 0da30d5eae Merge branch 'master' into feature/reindex 2016-02-10 14:07:41 -05:00
Nik Everett 4642111bac Teach reindex to retry on rejection
And count it in the status too!
2016-02-10 13:59:17 -05:00
Dongjoon Hyun 06e7c9c005 Fix typos in comments in elasticsearch/common. 2016-02-10 10:29:54 -08:00
Jason Tedor 374ecea435 Remove dead support for Java Service Wrapper
This commit removes bootstrap support for Java Service Wrapper. The
implementation of this has been moved to its own repository where it was
deprecated, does not work with Elasticsearch 2.x, and is untested and
therefore unmaintained.

Closes #16580
2016-02-10 12:10:43 -05:00
Jason Tedor 346ff0435a Fail demoted primary shards and retry request
This commit handles the scenario where a replication action fails on a
replica shard, the primary shard attempts to fail the replica shard
but the primary shard is notified of demotion by the master. In this
scenario, the demoted primary shard must be failed, and then the
request rerouted again to the new primary shard.

Closes #16415, closes #14252
2016-02-10 11:39:32 -05:00
Tal Levy 321c463929 Merge pull request #16557 from talevy/ingest_hide_null_metadata
hide null-valued metadata fields from WriteableIngestDocument#toXContent
2016-02-10 08:29:16 -08:00
Colin Goodheart-Smithe cef9b015e8 fix merge error 2016-02-10 12:20:10 +00:00
Colin Goodheart-Smithe 1f760bd1bd Merge branch 'master' into feature/aggs-refactoring 2016-02-10 12:16:26 +00:00
Colin Goodheart-Smithe 37ce7406d7 Renames PipelineAggregatorFactory to PipelineAggregatorBuilder
Also renames all the implementations appropriately
2016-02-10 11:03:55 +00:00
Colin Goodheart-Smithe 2d1f97beae Made AggregatorFactory fields final and removed AggregationContext from createInternal() parameters 2016-02-10 10:59:59 +00:00
Luca Cavanna 09c32fb1c9 Merge pull request #16539 from javanna/enhancement/ingest_tp
Ingest: use bulk thread pool for bulk request processing (was index before)
2016-02-10 11:49:21 +01:00
Isabel Drost-Fromm e0023a6692 Remove deprecated parameter from field sort builder.
This removes the deprecated ignore_unmapped parameter from field
sort builder.

This is in preparation of #16127
2016-02-10 11:31:23 +01:00
Isabel Drost-Fromm f354ab8e11 Merge pull request #16572 from MaineC/geosort-serialisation
Make GeoDistanceSortBuilder serializable, 2nd try
2016-02-10 11:25:51 +01:00
Colin Goodheart-Smithe 54e2f31189 Parse Aggregations on Coordinating node 2016-02-10 10:00:16 +00:00
Adrien Grand a5e5211341 Merge pull request #16264 from jpountz/fix/update_mapping_meta_on_all_types
Put mapping operations must update metadata of all types.
2016-02-10 10:41:01 +01:00
Jim Ferenczi e283ac8588 Fix sporadic SimpleQueryStringBuilderTests failures. 2016-02-10 10:33:45 +01:00
Colin Goodheart-Smithe ccee05792b Clean up of some NORELEASE comments for Agg refactoring 2016-02-10 09:25:34 +00:00
javanna 9ed32f715f Ingest: use bulk thread pool for bulk request processing (was index before)
Closes #16503
2016-02-10 10:19:21 +01:00
Jim Ferenczi b146f3ecb3 Pack all the plugin files into a single folder named elasticsearch at the root of the plugin zip. 2016-02-10 10:13:05 +01:00
Isabel Drost-Fromm 9432d57409 Make GeoDistanceSortBuilder serializable
Adds to GeoDistanceSortBuilder:

* equals
* hashcode
* writeto/readfrom
* moves xcontent parsing logic over
* adds roundtrip tests
* fixes roundtrip test for xcontent by keeping points just as geopoints not geohashes internally
* fixes xcontent parsing of ignore_malformed if coerce is set/unset
* adds exception to sortMode setter to avoid setting invalid sort modes

Relates to #15178
2016-02-10 10:10:56 +01:00
Adrien Grand 4ab7a18738 Put mapping operations must update metadata of all types.
Today put mapping operations only update metadata of the type that is being
modified, which is not enough since some modifications may have side-effects
on other types.

Closes #16239
2016-02-10 09:41:12 +01:00
Igor Motov 99a7d8e41f Add task cancellation mechanism
Only tasks that extend CancellableTask can be cancelled using this mechanism. If a cancellable task has children it can elect to cancel all child tasks as well. In this case a special ban parent request is sent to all nodes. This request does two things: 1) it prevents any tasks with the banned parent task from being started, and 2) it cancels all currently running tasks that have the banned task as a parent. The ban is lifted as soon as the coordinating node notifies all other nodes that the cancelled task has finished executing. If the coordinating node leaves the cluster before it has a chance to lift its bans, all bans set by this coordinating node are automatically removed.

As an option a task can elect to automatically cancel all child tasks if their parent task was running on a node that just left the cluster. This option makes sense for cancellable heavy tasks that have no side-effects and only return results to the coordinating node. With the coordinating node gone, it doesn't make sense to run such tasks any longer since their results will be most likely discarded.
2016-02-09 22:30:57 -05:00
Nik Everett 5c80dd5048 Merge pull request #16567 from nik9000/checkstyle_equals_hashcode
Fail build if we define equals but not hashCode
2016-02-09 22:00:51 -05:00
Nik Everett f76366909a Fail build if we define equals but not hashCode
That is like some kind of cardinal sin or something, right?

We had two violations though they weren't super likely to be keys in a hashmap
any time soon.
2016-02-09 21:29:08 -05:00
Dongjoon Hyun 1d26b4de22 Fix typos in the names of some variables
Closes #16566
2016-02-09 20:35:42 -05:00
Tal Levy f9453e594a hide null-valued metadata fields from WriteableIngestDocument#toXContent 2016-02-09 12:57:58 -08:00
Yannick Welsch 0d11443aba Fix filters and null parameters in _aliases command
Closes #16549
Closes #16547
2016-02-09 21:43:42 +01:00
Jason Tedor 40789d818e Remove unused import from o.e.e.Environment 2016-02-09 13:26:32 -05:00
Jason Tedor 1046a43445 Remove unused import from o.e.c.h.MessageDigests 2016-02-09 12:59:26 -05:00
Jason Tedor e6545b63d4 Add Javadocs for o.e.c.h.MessageDigests 2016-02-09 12:56:50 -05:00
Alexander Reelsen 0d4711c2fc Ingest: Add attachment processor
This is a simple port of the mapper attachment plugin to the ingest
functionality, no new features. The only option is to limit
the number of chars to prevent indexing of huge documents.

Fields can be selected in the processor as well.

Close #16303
2016-02-09 17:03:30 +01:00
Simon Willnauer 5cbc7bb93e Merge pull request #16508 from s1monw/issues/16378
Add field names to several mapping errors
2016-02-09 16:17:33 +01:00
Simon Willnauer 35fc6006f7 fix similarity tests 2016-02-09 15:24:57 +01:00
Colin Goodheart-Smithe 80bbb4a385 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/percolator/PercolateContext.java
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/filter/FilterParser.java
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/filters/FiltersParser.java
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsParametersParser.java
#	core/src/main/java/org/elasticsearch/search/internal/DefaultSearchContext.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridParserTests.java
#	test/framework/src/main/java/org/elasticsearch/test/TestSearchContext.java
2016-02-09 10:37:20 +00:00
Colin Goodheart-Smithe e5a506752d Fix where metadata wasn't being passed through to the AggregatorFactory 2016-02-09 10:28:55 +00:00
Dongjoon Hyun 3bad16c1ca Rename variables.
This PR renames the following three variables to fix a typo `settting` into `setting`.
 * Rename a static class member:
   INDEX_TRANSLOG_FLUSH_THRESHOLD_SIZE_SETTTING -> INDEX_TRANSLOG_FLUSH_THRESHOLD_SIZE_SETTING
 * Rename a parameter: aSettting --> aSetting
 * Rename a local variable: indexSetttings -> indexSettings
2016-02-08 23:13:33 -08:00
Jason Tedor c121af63b4 BootstrapSettings final with private constructor 2016-02-08 17:24:09 -05:00
Simon Willnauer 3dd827017c Merge pull request #16501 from s1monw/trash_thread_local
Detach QueryShardContext from IndexShard and remove obsolete threadlocals
2016-02-08 21:01:35 +01:00
Simon Willnauer 10f55a2df1 simplify method signature 2016-02-08 20:27:22 +01:00
Jason Tedor 06e526ba58 Register bootstrap settings
This commit registers bootstrap settings used on startup. Without
registration, setting any of these settings causes node startup to
fail. By registering these settings (rather than clearing) after use, we
enable them to be visible in any APIs that show all settings.

Closes #16513
2016-02-08 14:15:31 -05:00
gmarz 2051817780 Set meta data for pipeline aggregations
Closes #16484
2016-02-08 13:25:50 -05:00
Colin Goodheart-Smithe 94e594c095 Make AggregatorFactory fully immutable by requiring the parent, subFactories and metadata in the constructor 2016-02-08 17:51:56 +00:00
Jason Tedor 46ee568695 Log warning if max file descriptors too low
This commit adds logging for a warning message if the max file
descriptor count is too low.

Closes #16506
2016-02-08 11:50:38 -05:00
Simon Willnauer 7cb4baebf5 Add field names to several mapping errors
Closes #16378
2016-02-08 17:46:32 +01:00
Colin Goodheart-Smithe 323088d022 Decouples the AggregatorBuilder from the AggregatorFactory 2016-02-08 16:36:27 +00:00
Yannick Welsch 7ea4f7400b Require that relocation source is marked as relocating before starting recovery to relocation target 2016-02-08 17:30:33 +01:00
Simon Willnauer 2412dbe599 apply feedback from @colings86 2016-02-08 17:16:12 +01:00
Jason Tedor cbeae225b9 Speedup MessageDigestTests#testToHexString
The purpose of this commit is to speed up the runtime of
MessageDigestTests#testToHexString. As written, the test contains a loop
that creates 1024 test cases leading to a test runtime on the order of a
few seconds. Given build infrastructure, a single test case should
suffice. Therefore, this commit removes this loop so that the test can
execute on the order of a couple hundred milliseconds.
2016-02-08 10:58:34 -05:00
Adrien Grand bf382ce5d8 Merge pull request #16255 from jpountz/fix/search_analyzer_serialization
Fix serialization of `search_analyzer`.
2016-02-08 16:54:08 +01:00
Adrien Grand 12477f38b4 Fix serialization of `search_analyzer`.
We currently have a bug that it will be omitted if the index analyzer is the
default analyzer.
2016-02-08 16:51:36 +01:00
Jason Tedor ef88943e6e Cleanup JavaVersion
This commit includes a few minor cleanups to o/e/b/JavaVersion.java:
 - Stronger argument checking in JavaVersion#parse
 - Use JDK 8 string joiner
 - Keep an immutable copy of the version sequence
2016-02-08 10:43:46 -05:00
Colin Goodheart-Smithe 02ecfd6279 Merge branch 'master' into feature/aggs-refactoring 2016-02-08 15:41:02 +00:00
Colin Goodheart-Smithe a0b004acb5 Split AggregatorFactory into AggregatorBuilder and AggregatorFactory 2016-02-08 14:27:34 +00:00
Simon Willnauer 96bcb47fc9 Detach QueryShardContext from IndexShard and remove obsolete threadlocals
IndexShard currently holds an arbitraritly used `getQueryShardContext` that comes
out of a ThreadLocal. It's usage is undefined and arbitraty since there is also
such a method with different semantics on `IndexService` This commit removes the threadLocal on
IndexShard as well as on the context itself. It's types are now a member and the QueryShardContext
lifecycle is managed byt SearchContext which passes the types on from the SearchRequest.
2016-02-08 15:05:03 +01:00
Isabel Drost-Fromm 0b3cab6da3 Move sorting tests w/o scripting back to core
In preparation of #16127 this moves sorting tests back to core
that don't actually need scripting or groovy to work.
2016-02-08 14:45:33 +01:00
Boaz Leskes 4decc72da6 Fix recovery translog stats totals when recovering from store
Recovery from store fails to correctly set the translog recovery stats. This fixes it and tightens up the logic bringing it all to IndexShard (previously it was set by the recovery logic).

Closes #15974
Closes #16493
2016-02-08 11:36:59 +01:00
Simon Willnauer 7bfb1dfcc1 [TEST] Don't assert on null value. It's fine to not always see an exception in this part of the test. 2016-02-07 19:58:03 +01:00
Jason Tedor 16fe948a15 Objects#requireNonNull guard in MessageDigests 2016-02-07 11:20:53 -05:00
Jason Tedor d0c03b26f6 Fix trace logging statement in ZenDiscovery 2016-02-06 09:17:49 -05:00
Jason Tedor 542373a779 Avoid cloning MessageDigest instances
This commit modifies the MessageDigests message digest provider to
return a thread local instance of MessageDigest instances instead of
using clone since some providers do not support clone.

Closes #16479
2016-02-06 09:10:31 -05:00
Simon Willnauer b185209b63 Make IndicesWarmer a private class of IndexService
There is no need for IndicesWarmer to be a global accessible class. All it needs
access to is inside IndexService. It also doesn't need to be mutable once it's not a per node
instance. This commit move IndicesWarmer to IndexWarmer and makes the default impls like field data and
norms warming an impl detail. Also the IndexShard doesn't depend on this class anymore, instead it accepts
an Engine.Warmer as a ctor argument which delegates to the actual warmer from the index.
2016-02-05 15:24:17 +01:00
Simon Willnauer 778ec84a7c Merge pull request #16469 from s1monw/simplify_indices_field_data_cache
Simplify IndicesFieldDataCache and detach from guice
2016-02-05 14:24:19 +01:00
Simon Willnauer 9b2f40f627 Merge pull request #16471 from s1monw/no_sys_props_leaked
Apply system properties after all arguemnts are parsed in BootstrapCLIParser
2016-02-05 14:23:00 +01:00
Christoph Büscher 97172aa2fd Merge pull request #16465 from cbuescher/issue-13884-test
Tests: Add test for minimum_should_match for SimpleQueryStringQuery
2016-02-05 13:12:53 +01:00
Christoph Büscher d885fe62ea Add test for minimum_should_match, one term and multiple fields
This adds a test case similar to the issue in #13884 which was
fixed in #16155.
2016-02-05 12:33:13 +01:00
Andrej Kazakov 7f2b369dfd Use Accept header field in cat API
The cat API previously used the Content-Type header field for
determining the media type of the response. This is in opposition to the
HTTP spec which specifies the Accept header field for this purpose. This
commit replaces the use of the Content-Type header field with the Accept
header field in the cat API.

Closes #14421
2016-02-05 06:28:39 -05:00
Colin Goodheart-Smithe 19d9c588e1 more line length fixes 2016-02-05 10:41:06 +00:00
Colin Goodheart-Smithe 92d958c46e fixed 140 character line limit 2016-02-05 10:26:49 +00:00
Simon Willnauer 1e18c9a7b6 Apply system properties after all arguemnts are parsed in BootstrapCLIParser
One of our tests leaked a system property here since we failed after appling some
system properties in BootstrapCLIParser. This is not a huge deal in production since
we exit the JVM if we fail on that. Yet for correctnes we should only apply them if
we manage to parse them all.
This also caused a test failure lately on CI but on an unrelated test:
  https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+periodic/314/console
2016-02-05 11:21:00 +01:00
Simon Willnauer b5ef48ce57 fix more tests 2016-02-05 10:36:02 +01:00
Colin Goodheart-Smithe f06f17f328 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamInput.java
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamOutput.java
2016-02-05 09:22:14 +00:00
Simon Willnauer 9d07501c96 fix test 2016-02-05 09:54:55 +01:00
Simon Willnauer 00be9e58f2 Simplify IndicesFieldDataCache and detach from guice
Indices level field data cacheing belongs into IndicesService and doesn't need to be
wired by guice. This commit also moves the async cache refresh out of the class into
IndicesService such that threadpool dependencies are removed and testing / creation becomes
simpler.
2016-02-05 09:43:33 +01:00
Martijn van Groningen 7a6adfd93a ingest: Added foreach processor.
This processor is useful when all elements of a json array need to be processed in the same way.
This avoids that a processor needs to be defined for each element in an array.
Also it is very likely that it is unknown how many elements are inside an json array.
2016-02-04 23:44:01 +01:00
Simon Willnauer 59a20015d6 Make TermVectorService static 2016-02-04 17:17:16 +01:00
Simon Willnauer 75802515ea Merge branch 'master' into detach_shard_from_term_vectors 2016-02-04 17:12:09 +01:00
Simon Willnauer 450ee70038 Remove DFS support from TermVector API
Retrieving distributed DF for TermVectors is beside it's esotheric justification
a very slow process and can cause serious load on the cluster. We also don't have nearly
enough testing for this stuff and given the complexity we should remove it rather than carrying it
around.
2016-02-04 16:20:24 +01:00
Yannick Welsch 15507580bb Merge pull request #16243 from ywelsch/fix/remove-shardrouting-version
Remove obsolete version in ShardRouting
2016-02-04 15:52:00 +01:00
Yannick Welsch 4937531a17 Remove obsolete version in ShardRouting
Closes #16243
2016-02-04 15:50:25 +01:00
Boaz Leskes 4cdab159f1 Remove plugability of the gateway
In the spirit of simplification, remove the ability to have a custom gateway. We haven't seen any valid use for it so far.

closes #16448
2016-02-04 15:45:05 +01:00
Nik Everett bb4d8b79fe Merge branch 'master' into feature/reindex 2016-02-04 09:23:49 -05:00
Boaz Leskes 924e880480 Remove `gateway.initial_meta` and always rely on min master nodes
During initial cluster forming, when  a master is elected, it reaches out to all other masters nodes and ask the last cluster state they persisted. To make sure we select the right state, we must successfully read from a `min_master_nodes` nodes. The gateway currently have specific settings to override this behavior, but I don't think they are ever used. We can drop them and reach out to the discovery layer, the single source of truth for the min master nodes settings.

Closes #16446
2016-02-04 13:42:28 +01:00
Simon Willnauer 30f569f958 Remove dependency from IndexShard on TermVectorService
This dependency is just syntactic sugar and complicates IndexShard creation.
This commit move it out where it's actually used and needed.
2016-02-04 12:55:45 +01:00
Simon Willnauer e69350f2ba Fix method visibility in SettingsModule.java 2016-02-04 11:53:06 +01:00
Simon Willnauer cb811a85da Add javadocs for SettingsModule 2016-02-04 11:49:59 +01:00
Simon Willnauer e19833171b Add more javadocs to Setting.java 2016-02-04 11:44:36 +01:00
Simon Willnauer c416b23739 Merge pull request #16445 from s1monw/filter_settings_on_node_info
Add settings filtering to node info requests
2016-02-04 11:26:04 +01:00
Simon Willnauer 26a0c72323 add test fore node info settings filtering 2016-02-04 11:15:29 +01:00
Ryan Ernst 5c0f09b03a Merge pull request #16443 from rjernst/cli_terminal
CliTool: Cleanup and document Terminal
2016-02-04 02:05:28 -08:00
Ryan Ernst e97345984d Remove format args from readText and readSecret 2016-02-04 02:03:17 -08:00
Simon Willnauer ca1a12e4c3 Add settings filtering to node info requests
Today we don't filter settings in NodeInfo so transport clients get unfiltered settings.
2016-02-04 10:45:32 +01:00
Simon Willnauer cc520b20e1 Merge branch 'master' into fix_settings_filter 2016-02-04 09:28:41 +01:00
Ryan Ernst b80200a363 CliTool: Cleanup and document Terminal
This change documents the Terminal abstraction that cli tools use, as
well as simplifies the api to be a minimal set of methods to interact
with a terminal.
2016-02-03 22:22:56 -08:00
Ryan Ernst b0ed528be8 Plugin cli: Delete temp zip file once it is unzipped 2016-02-03 20:17:42 -08:00
Nik Everett a2f07679fd Add task status
Implements a simple task status for superclasses of ReplicationRequest to
show how you can do use the status.
2016-02-03 18:21:42 -05:00
Simon Willnauer e1cf5e745d some plugins share settings - make it easy to filter them 2016-02-03 21:50:06 +01:00
Simon Willnauer 686627be5a Move ModuleTestCase into framework 2016-02-03 21:04:44 +01:00
Ali Beyad 592f5499cd More robust handling of CORS HTTP Access Control
Uses a refactored version of Netty's CORS implementation to provide more
robust cross-origin resource request functionality.  The CORS specific
Elasticsearch parameters remain the same, just the underlying
implementation has changed.

It has also been refactored in a way that allows dropping in Netty's
CORS handler as a replacement once Elasticsearch is upgraded to Netty 4.
2016-02-03 14:50:20 -05:00
Simon Willnauer e02d2e004e Rewrite SettingsFilter to be immutable
This change rewrites the entire settings filtering mechanism to be immutable.
All filters must be registered up-front in the SettingsModule. Filters that are comma-sparated are
not allowed anymore and check on registration.
This commit also adds settings filtering to the default settings recently added to ensure we don't render
filtered settings.
2016-02-03 20:05:55 +01:00
Colin Goodheart-Smithe 761adbb28d Fixed merge errors 2016-02-03 15:08:24 +00:00
Colin Goodheart-Smithe 5d9d91b761 Merge branch 'master' into feature/aggs-refactoring 2016-02-03 14:45:16 +00:00
Nik Everett 5e98094db1 Remove long lines from ElasticsearchException
Mostly just wrapping the exception list.

Also:
* Reworks the docs on ElasticsearchExceptionHandle
* Removes long lines from ExceptionSerializationTests
 * Switches one method from arrow shaped to early returns
 * Adds line breaks
2016-02-03 09:33:02 -05:00
Martijn van Groningen 0d9f535044 ingest: seal all processor and processor factory implementations 2016-02-03 14:53:11 +01:00
Isabel Drost-Fromm 7d787ad1d1 Merge pull request #16411 from MaineC/test-fix/geo-sort-messy
Move non-groovy geo tests from messy tests to core
2016-02-03 14:52:01 +01:00
Simon Willnauer bac6c7c764 Remove dead code 2016-02-03 14:42:11 +01:00
Simon Willnauer 1972487deb Merge pull request #16365 from s1monw/make_settings_strict
Make settings validation strict
2016-02-03 14:29:25 +01:00
Isabel Drost-Fromm f99570b0f4 Split messy geo distance/ geo distance sorting tests.
This splits the geo distance and geo distance sorting tests marked messy:
Test cases that don't really need Groovy support are moved back to the
core test suite closer to the code they actually test.

Relates to #15178
2016-02-03 13:54:07 +01:00
Konrad Beiske 2c247e9d56 Merge pull request #16017 from beiske/xcontent_exclude
Add option to exclude based on paths in XContent
2016-02-03 13:51:49 +01:00
Simon Willnauer a77344d742 Merge branch 'master' into make_settings_strict 2016-02-03 13:21:49 +01:00
Jason Tedor eb9bdd5f8a Shard state action request logging 2016-02-03 06:56:01 -05:00
Jason Tedor 0b474c6cfc Shard state action request logging
This commit modifies the string representation of a shard state action
request. The issue being addressed is that the previous logging would
log "failure: [Unknown]" for shard started actions but this just leads
to confusion that there is a failure but its cause is unknown.

Closes #16396
2016-02-03 06:52:54 -05:00
Robert Muir 20b584c7e1 Merge pull request #16400 from rmuir/lucene_4de5f1d
Upgrade to lucene 5.5.0-snapshot-4de5f1d
2016-02-03 06:45:50 -05:00
Konrad Beiske 42833b0f71 Adjusted indentation 2016-02-03 12:37:38 +01:00
Konrad Beiske 42a44b19aa Fixed parameter order in FilterPathBasedFilter constructor 2016-02-03 12:23:14 +01:00
Konrad Beiske 6f915549c6 Fixed typo in javadoc 2016-02-03 12:18:05 +01:00
Konrad Beiske 0dfb55d72c exclude{Filters}, filters => filters, exclude 2016-02-03 12:11:27 +01:00
Simon Willnauer d5ece77f2a remove dead code 2016-02-03 11:36:11 +01:00
Simon Willnauer 4a4e523357 Merge branch 'master' into make_settings_strict 2016-02-03 11:34:12 +01:00
Konrad Beiske db4c6c96a3 Minor refactoring and indentation fixes 2016-02-03 11:14:00 +01:00
Boaz Leskes 125029bf12 make testTimeoutSendExceptionWithDelayedResponse less timing sensitive 2016-02-03 11:09:50 +01:00
Simon Willnauer ed492901ed Apply feedback from @bleskes and add fix some minor issues 2016-02-03 10:44:06 +01:00
Robert Muir bca0c88df3 Fix versions to match reality 2016-02-02 23:03:38 -05:00
Robert Muir a1edf73205 Revert "Test awaits Lucene snapshot upgrade"
This reverts commit e6a5e79ede.
2016-02-02 22:54:36 -05:00
Robert Muir d5dc05f69e Upgrade to lucene 5.5.0-snapshot-1725675 2016-02-02 22:53:39 -05:00
Tal Levy b653e80314 fix ingest client put error test 2016-02-02 15:24:58 -08:00
Jason Tedor 8b37827ac6 Guard against null routing node iterator 2016-02-02 18:14:37 -05:00
Jason Tedor a3a49a12ef Illegal shard failure requests
Today, shard failure requests are blindly handled on the master without
any validation that the request is a legal request. A legal request is a
shard failure request for which the shard requesting the failure is
either the local allocation or the primary allocation. This is because
shard failure requests are classified into only two sets: requests that
correspond to shards that exist, and requests that correspond to shards
that do not exist. Requests that correspond to shards that do not exist
are immediately marked as successful (there is nothing to do), and
requests that correspond to shards that do exist are sent to the
allocation service for handling the failure.

This pull request adds a third classification for shard failure requests
to separate out illegal shard failure requests and enables the master to
validate shard failure requests. The master communicates the illegality
of a shard failure request via a new exception:
NoLongerPrimaryShardException. This exception can be used by shard
failure listeners to discover when they've sent a shard failure request
that they were not allowed to send (e.g., if they are no longer the
primary allocation for the shard).

Closes #16275
2016-02-02 17:41:09 -05:00
Tal Levy 9e7e2ab10b remove DeDotProcessor from Ingest 2016-02-02 14:16:01 -08:00
Tal Levy 3191fc7347 Merge pull request #16355 from talevy/fix_ingest_exception
revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation
2016-02-02 14:11:24 -08:00
Tal Levy 0a1580eefa revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation 2016-02-02 14:08:22 -08:00
Ryan Ernst 9f47b376da Plugin cli: Improve maven coordinates detection
Identifying when a plugin id is maven coordinates is currently done by
checking if the plugin id contains 2 colons. However, a valid url could
have 2 colons, for example when a port is specified. This change adds
another check, ensuring the plugin id with maven coordinates does not
contain a slash, which only a url would have.

closes #16376
2016-02-02 12:17:56 -08:00
Tanguy Leroux 865bbc2096 Remove string formatting from Terminal print methods
This can be trappy and wrong formating strings can throws format exceptions and hide the real message.
2016-02-02 19:57:16 +01:00
Jason Tedor e6a5e79ede Test awaits Lucene snapshot upgrade
This commit marks OldIndexBackwardsCompatibilityIT#testOldIndexes as
awaiting a Lucene snapshot upgrade to reflect the fact that
Elasticsearch 2.2.0 is built against Lucene 5.4.1 but the current Lucene
snapshot in master/2.x does not contain the Lucene version 5.4.1 field.

Relates #16373
2016-02-02 10:01:37 -05:00
Jason Tedor 1e6b2d4f1d Fix JVM GC monitor missing settings test
This commit fixes a test bug in
JvmGcMonitorServiceSettingsTests#testMissingSetting. The purpose of the
test is to test that if settings are provided for a collector for at
least one of warn, info, and debug then it is provided for all of warn,
info, and debug. However, for a collector setting to be valid it must be
a positive time value but the randomization in the test construction
could produce zero time values.

Closes #16369
2016-02-02 08:49:47 -05:00
Clinton Gormley 58b6db8d82 Added versions 2.1.3-SNAPSHOT and 2.2.1-SNAPSHOT, and bwc indices for 2.1.2 and 2.2.0 2016-02-02 14:39:21 +01:00
Yannick Welsch 13ad335416 Fix compilation in TransportReplicationActionTests 2016-02-02 14:08:19 +01:00
Yannick Welsch 8cca0395ef Revert "Make GeoDistanceSortBuilder serializable"
This reverts commit c108a4ce6d.
2016-02-02 14:04:45 +01:00
Yannick Welsch 089ab7d9bf Revert "Remove deprecation for geohash setter"
This reverts commit b5a4e99344.
2016-02-02 14:03:58 +01:00
Yannick Welsch c5a6ddfdc2 Merge pull request #16274 from ywelsch/fix/endless-index-loop
Prevent TransportReplicationAction to route request based on stale local routing table
2016-02-02 13:59:54 +01:00
Yannick Welsch cd53772178 Use allocation ids to prevent repeated recovery of failed shards
Closes #16346
2016-02-02 13:54:27 +01:00
Yannick Welsch af1f637547 Prevent TransportReplicationAction to route request based on stale local routing table
Closes #16274
Closes #12573
Closes #12574
2016-02-02 13:52:45 +01:00
Isabel Drost-Fromm b5a4e99344 Remove deprecation for geohash setter
This removes the deprecation for the geohash based setter to quickly fix the failure here: http://build-us-00.elastic.co/job/es_core_master_suse/3312 

Reintroducing postponed until related test in groovy module is fixed. Need to figure out what went wrong when I ran the build locally w/o failure before.
2016-02-02 13:09:34 +01:00
Yannick Welsch 51862ce5ae Fix IndexShardTests.testStressRelocated
Closes #16364
2016-02-02 12:57:19 +01:00
Boaz Leskes 58825773eb Remove `discovery.zen.rejoin_on_master_gone`
That setting was built as an escape hatch for the work done in 1.4. I never heard it being used once. We can safely remove it.

Closes #16353
2016-02-02 12:15:56 +01:00
Isabel Drost-Fromm df45c75594 Merge pull request #16151 from MaineC/enhancement/15178-sort-geosortbuilder
Refactor GeoSortBuilder
2016-02-02 12:14:35 +01:00
Alexander Reelsen 2366b3593f BWC: Added 1.7 version constants and bwc indices 2016-02-02 12:08:11 +01:00
Simon Willnauer 818a9eefb2 Make settings validation strict
This commit enableds strict settings validation on node startup. All settings
passed to elasticsearch either through system properties, yaml files or any other
way to pass settings must be registered and valid. Settings that are unknown ie. due to
typos or due to deprecation or removal will cause the node to NOT start up. Plugins
have to declare all their settings on the `SettingsModule#registerSetting` and settings for
plugins that are not installed must be removed.

This commit also removes the ability to specify the nodes name via `-Des.name` or just `name` in the
configuration files. The node name must be prefixed with the node prexif like `node.name: Boom`. Left over
usage of `name` will also cause startup to fail.
2016-02-02 11:32:44 +01:00
Isabel Drost-Fromm c108a4ce6d Make GeoDistanceSortBuilder serializable
Adds to GeoDistanceSortBuilder:

* equals
* hashcode
* writeto/readfrom
* moves xcontent parsing logic over
* adds roundtrip tests
* fixes roundtrip test for xcontent by keeping points just as geopoints not geohashes internally
* fixes xcontent parsing of ignore_malformed if coerce is set/unset
* adds exception to sortMode setter to avoid setting invalid sort modes

Relates to #15178
2016-02-02 11:24:32 +01:00
Tanguy Leroux a7fb5a27cc Mute IndexShardTests.testStressRelocated 2016-02-02 11:12:58 +01:00
Yannick Welsch 10b5ffcda5 Add proper handoff between old and new copy of relocating primary shard
When primary relocation completes, a cluster state is propagated that deactivates the old primary and marks the new primary as active.
As cluster state changes are not applied synchronously on all nodes, there can be a time interval where the relocation target has processed
the cluster state and believes to be the active primary and the relocation source has not yet processed the cluster state update and
still believes itself to be the active primary. This commit ensures that, before completing the relocation, the reloction source deactivates
writing to its store and delegates requests to the relocation target.

Closes #15900
2016-02-02 09:46:55 +01:00
Yannick Welsch e1006ea400 Add operation counter for IndexShard
Adds a container that represents a resource with reference counting capabilities. Provides operations to suspend acquisition of new references. Useful for resource management when resources are intermittently unavailable.

Closes #15956
2016-02-02 09:46:55 +01:00
Nik Everett d91a898f6a Merge pull request #16330 from nik9000/line_length_1
Start to break lines at 140 characters
2016-02-01 21:41:47 -05:00
Nik Everett 07cba65c1b Start to break lines at 140 characters
Its what we say our maximum line length is in CONTRIBUTING.md and it'd be
nice to have a check on that. Unfortunately, we don't actually wrap all lines
at 140.
2016-02-01 21:09:35 -05:00
Ryan Ernst a2c37c0989 CliTool: Allow unexpected exceptions to propagate
Cli tools currently catch all exceptions, and only print the exception
message, except when a special system property is set. Even with this
flag set, certain exceptions, like IOException, are captured and their
stack trace is always lost.

This change adds a UserError class, which can be used a cli tools to
specify a message to the user, as well as an exit status. All other
exceptions are propagated out of main, so java will exit with non-zero
and print the stack trace.
2016-02-01 16:35:22 -08:00
Nik Everett c1d4478067 Merge branch 'master' into feature/reindex 2016-02-01 18:33:09 -05:00
Tal Levy dcb88909d5 Merge pull request #16324 from talevy/ingest_on_failure_tag
Add processor tags to on_failure metadata in ingest pipeline
2016-02-01 10:13:26 -08:00
Jason Tedor 24a841a075 ShardId equality and hash code inconsistency
This commit fixes an inconsistency between ShardId#equals(Object),
ShardId#hashCode, and ShardId#compareTo(ShardId). In particular,
ShardId#equals(Object) compared only the numerical shard ID and the
index name, but did not compare the index UUID; a similar situation
applies to ShardId#compareTo(ShardId). However, ShardId#hashCode
incorporated the indexUUID into its calculation. This can lead to
situations where two ShardIds compare as equal yet have different hash
codes.

Closes #16319
2016-02-01 11:56:50 -05:00
Ryan Ernst 01b80b100e Merge pull request #16333 from rjernst/single_node_static
Make single node utility methods non-static
2016-02-01 08:46:54 -08:00
Ryan Ernst 3787f437ec Merge branch 'master' into remove_multicast 2016-02-01 07:25:45 -08:00
Ryan Ernst 661e3c99f7 Merge pull request #16336 from rjernst/demangle
Reduce complexity of plugin cli
2016-02-01 07:20:04 -08:00
Colin Goodheart-Smithe 2d2e9e0621 fixed issue from merge 2016-02-01 13:57:38 +00:00
Colin Goodheart-Smithe a241983c9d Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/aggregations/AggregationsBinaryIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/FilterIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/FiltersIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/GeoDistanceIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/NestedIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/SamplerIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/pipeline/DerivativeIT.java
#	modules/lang-groovy/src/test/java/org/elasticsearch/messy/tests/BucketSelectorTests.java
#	modules/lang-groovy/src/test/java/org/elasticsearch/messy/tests/TDigestPercentileRanksTests.java
2016-02-01 13:38:33 +00:00
Jason Tedor ed7bc5ba16 Unused variables in TransportReplicationAction 2016-02-01 08:12:34 -05:00
Colin Goodheart-Smithe 745e8f96e7 Moved http settings to the new settings infrastructure
The following settings were moved to Setting contents in HttpTransportSettings:

* http.cors.allow-origin
* http.port
* http.publish_port
* http.detailed_errors.enabled
* http.max_content_length
* http.max_chunk_size
* http.max_header_size
* http.max_initial_line_length

The following settings were removed:

* http.port
* http.netty.port
* http.netty.publish_port
* http.netty.max_content_length
* http.netty.max_chunk_size
* http.netty.max_header_size
* http.netty.max_initial_line_length
2016-02-01 12:56:41 +00:00
Simon Willnauer 008aa2c316 Merge pull request #16341 from s1monw/cache_settings
Convert PageCacheRecycler settings
2016-02-01 13:51:19 +01:00
Christoph Büscher 61c435e6a9 PhraseSuggestionBuilde: Refactor DirectCandidateGenerator
As a prerequisite for refactoring the whole PhraseSuggestionBuilder
to be able to be parsed and streamed from the coordinating node, the
DirectCandidateGenerator must implement Writeable, be able to parse
a new instance (fromXContent()) and later when transported to the
shard to generate a PhraseSuggestionContext.DirectCandidateGenerator.
Also adding equals/hashCode and tests and moving DirectCandidateGenerator
to its own DirectCandidateGeneratorBuilder class.
2016-02-01 12:22:48 +01:00
Simon Willnauer aa5f9d5158 Convert PageCacheRecycler settings 2016-02-01 11:34:19 +01:00
Boaz Leskes 856883fcc5 Local Discovery - don't create a local DiscoNode, but use the one from cluster service
Long ago (#7834) the owner ship of the local disco node was centralized to the cluster service. LocalDiscovery is still created it's own disco node, which is not used by the cluster service and thus creating confusion (two nodes same name but different ids).

This commit also removes and optimization where when joining a new master we would first copy the master's metadata and only then pull in the rest of the cluster state (and it's nodes).

Closes #16317
2016-02-01 11:05:40 +01:00
Boaz Leskes df80e8f215 Migrate the rest of NettyTransport settings to the new infra
Also does some consistency clean up, renaming trasnport.netty.* settings to transport.*

Closes #16307
2016-02-01 10:26:52 +01:00
Simon Willnauer af0e40ec7d Merge pull request #16316 from s1monw/isseus/13685_2nd_try
Ensure all resoruces are closed on Node#close()
2016-02-01 10:17:57 +01:00
Ryan Ernst a052dfeb38 Plugins: Reduce complexity of plugin cli
The plugin cli currently is extremely lenient, allowing most errors to
simply be logged. This can lead to either corrupt installations (eg
partially installed plugins), or confused users.

This change rewrites the plugin cli to have almost no leniency.
Unfortunately it was not possible to remove all leniency, due in
particular to how config files are handled.

The following functionality was simplified:
* The format of the name argument to install a plugin is now an official
  plugin name, maven coordinates, or a URL.
* Checksum files are required, and only checked, for official plugins
  and maven plugins. Checksums are also only SHA1.
* Downloading no longer uses a separate thread, and no longer has a timeout.
* Installation, and removal, attempts to be atomic. This only truly works
  when no config or bin files exist.
* config and bin directories are verified before copying is attempted.
* Permissions and user/group are no longer set on config and bin files.
  We rely on the users umask.
* config and bin directories must only contain files, no subdirectories.
* The code is reorganized so each command is a separate class. These
  classes already existed, but were embedded in the plugin cli class, as
  an extra layer between the cli code and the code running for each command.
2016-01-31 20:30:45 -08:00
Ryan Ernst a398ba839c Tests: Make single node utility methods non-static
With the recent change to allow ESSingleNodeTestCase to specify plugins
and Version for the node, node creation became lazy, where it now
happens before the first test to run. However, there were many static
methods on ESSingleNodeTestCase that still try to access the node. This
change makes those methods non-static.
2016-01-31 09:58:41 -08:00
Jason Tedor 105411060c Uppercase ells ('L') in long literals
This commit removes and forbids the use of lowercase ells ('l') in long
literals because they are often hard to distinguish from the digit
representing one ('1').

Closes #16329
2016-01-30 22:16:02 -05:00
Lee Hinman 4e8623f8f4 Merge remote-tracking branch 'dakrone/add-ex-name-to-msg' 2016-01-30 16:28:03 -07:00
Lee Hinman 7e05b34708 Add Exception class name to message in `NotSerializableExceptionWrapper`
Previously it's possible to get errors like:

```
Caused by: NotSerializableExceptionWrapper[d:\shared_data\afs-issue-1-1\23]
```

Which doesn't tell us what the underlying exception type was that could
not be serialized.

This changes the message to prepend the
`ElasticsearchException.getExceptionName()` of the exception (which is a
underscore case of the class with the leading 'Elasticsearch' removed)
2016-01-30 16:27:24 -07:00
Jason Tedor 2a78e02fe9 Log shard ID in TransportReplicationAction
This commit amends a logging statement in TransportReplicationAction to
log the full shard ID instead of just the numerical shard ID (without
the index name).
2016-01-30 09:27:38 -05:00
Ryan Ernst b8f08c35ec Plugin: Remove multicast plugin
closes #16310.
2016-01-29 18:41:31 -08:00
Tal Levy 61e4283a16 Add processor tags to on_failure metadata in ingest pipeline
closes #16202
2016-01-29 14:36:11 -08:00
Tal Levy fca442f4d1 Introduce Pipeline Factory Error Responses in Node Ingest
When there is an exception thrown during pipeline creation within
Rest calls (in put pipeline, and simulate) We now return a structured
error response to the user with details around which processor's
configuration is the cause of the issue, or which configuration property
is misconfigured, etc.
2016-01-29 13:37:27 -08:00
Nik Everett ee35972f3d Less wrapping 2016-01-29 13:05:25 -05:00
Simon Willnauer 6814f24009 Cleanup Relesables now that we can delegate to IOUtils 2016-01-29 17:02:40 +01:00
Simon Willnauer 15841081f6 Disambiguate TestCluster implementation since Client is now also Closeable and if we call IOUtils it might interpret it as a Iterable<Closeable> 2016-01-29 16:51:06 +01:00
Nik Everett ab79ff73af Log any errors in reindex task
Does so by introducing TaskListener which is just like ActionListener but
gets the Task as each parameter. Unlike ActionListener which is used
_everywhere_ you can only use TaskListener directly with TransportAction.
TransportAction under the covers uses an ActionListener implemetation that
closes over the task to call the TaskListener.
2016-01-29 10:33:08 -05:00
Simon Willnauer ac87751b3d Use IOUtils#close() where needed 2016-01-29 16:26:37 +01:00
Jason Tedor c7b8fecb0e Merge pull request #16313 from jasontedor/monitor-settings
Monitor settings
2016-01-29 10:21:38 -05:00
Simon Willnauer 87737d3e02 Ensure all resoruces are closed on Node#close()
We are leaking all kinds of resources if something during Node#close() barfs.
This commit cuts over to a list of closeables to release resources that
also closed remaining services if one or more services fail to close.

Closes #13685
2016-01-29 16:18:21 +01:00
Jason Tedor a1fd01d4cc No leniency in parsing JVM GC monitor settings
This commit removes some leniency in the parsing of the JVM GC monitor
settings, and adds tests for the JVM GC monitor settings.
2016-01-29 10:07:34 -05:00
Simon Willnauer f89e63f988 Merge pull request #16315 from s1monw/issues/16299
Use index name rather than the index object to check against existence in the set
2016-01-29 15:19:43 +01:00
Boaz Leskes 9aceddb9e2 Missing CLUSTER_ROUTING_ALLOCATION_ALLOW_REBALANCE_SETTING.getKey() 2016-01-29 15:17:46 +01:00
Simon Willnauer 838edab168 Use index name rather than the index object to check against existence in the set
This is an issue due to some refactoring we did along the lines of Index.java etc.
We pass the index object to Set#contains which should actually be only it's name.

Closes #16299
2016-01-29 15:09:48 +01:00
Simon Willnauer 14017d4ad8 Merge pull request #16311 from s1monw/cut_over_tribe_settings
Cut over tribe node settings to new settings infra
2016-01-29 14:23:50 +01:00
Jason Tedor ea56bfad9a JVM GC monitor settings
This commit converts the GC monitor settings "monitor.jvm.gc.*" to the
new settings infrastructure.
2016-01-29 08:07:27 -05:00
Jason Tedor 3f53e143a5 FS monitor refresh interval setting
This commit converts the FS monitor refresh interval setting
"monitor.fs.refresh_interval" to the new settings infrastructure.
2016-01-29 07:58:57 -05:00
Jason Tedor aac53c9210 JVM monitor refresh interval setting
This commit converts the JVM monitor refresh interval setting
"monitor.jvm.refresh_interval" to the new settings infrastructure.
2016-01-29 07:58:56 -05:00
Simon Willnauer 65c0f80b13 Merge pull request #16308 from s1monw/never_interrupt_store_checks
Prevent interruption while store checks lucene files for consistency
2016-01-29 13:04:55 +01:00
Simon Willnauer 4c8768eeb7 Cut over tribe node settings to new settings infra 2016-01-29 13:03:55 +01:00
Colin Goodheart-Smithe 859f9e69b7 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/action/search/SearchRequestBuilder.java
#	core/src/main/java/org/elasticsearch/percolator/PercolateContext.java
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/ScriptHeuristic.java
#	core/src/main/java/org/elasticsearch/search/internal/DefaultSearchContext.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificanceHeuristicTests.java
2016-01-29 11:58:19 +00:00
Jason Tedor 4eed71cad2 Process monitor refresh interval setting
This commit converts the process monitor refresh interval setting
"monitor.process.refresh_interval" to the new settings infrastructure.
2016-01-29 06:52:29 -05:00
Jason Tedor d13739fcb9 OS monitor refresh interval setting
This commit converts the OS monitor refresh interval setting
"monitor.os.refresh_interval" to the new settings infrastructure.
2016-01-29 06:49:43 -05:00
Simon Willnauer 74a1959c3b Merge pull request #16305 from s1monw/issues/16301
Add a hard check to ensure we are running with the expected lucene version
2016-01-29 12:17:35 +01:00
Simon Willnauer f6b922a97a Prevent interruption while store checks lucene files for consistency
This can cause premature closing of the underlying file descriptor immediately
if at the same time the thread is blocked on IO. The file descriptor will remain closed
and subsequent access to NIOFSDirectory will throw a ClosedChannelException.
2016-01-29 12:09:00 +01:00
Simon Willnauer fa17a84a89 Add a hard check to ensure we are running with the expected lucene version
Closes #16301
2016-01-29 11:59:31 +01:00
Martijn van Groningen ce89039926 percolator: If `index.percolator.map_unmapped_fields_as_string` is enabled then don't replace found fields with dummy string field
Closes #10500
2016-01-29 11:45:54 +01:00
Martijn van Groningen 6c34452409 inner hits: Query and top level inner hit definitions shouldn't overwrite each other.
Closes #16218
2016-01-29 10:52:04 +01:00
Christoph Büscher b039360d2e Merge pull request #16130 from cbuescher/refactor-smoothingModel
Enable Writeable serialization and parsing from xContent for SmoothingModels
2016-01-29 10:49:34 +01:00
Christoph Büscher aefdee17fd Adding builder method to SmoothingModel implementations
Adds a method that emits a WordScorerFactory to all of the
three SmoothingModel implementatins that will be needed when
we switch to parsing the PhraseSuggestion on the coordinating
node and need to delay creating the WordScorer on the shards.
2016-01-29 10:21:31 +01:00
Martijn van Groningen f5e89f7242 mappings: remove fly weight 2016-01-29 10:12:39 +01:00
Martijn van Groningen 9ec1b11148 term vectors: The term vector APIs no longer modify the mappings if an unmapped field is found 2016-01-29 09:32:40 +01:00
Ali Beyad e3816d58ae TribeIT.testOnConflictDrop test awaits fix until #16299 is resolved 2016-01-28 17:10:04 -05:00
Jason Tedor 90284e6f01 Merge pull request #16061 from jasontedor/normalize-unavailable-load-average
Normalize unavailable load average

Relates #12049, #14741, #15907, #15932, #15934
2016-01-28 15:38:36 -05:00
Igor Motov e035dabd4d Extend tracking of parent tasks to master node, replication and broadcast actions
Now MasterNodeOperations, ReplicationAllShards, ReplicationSingleShard,  BroadcastReplication and BroadcastByNode actions keep track of their parent tasks.
2016-01-28 12:13:36 -05:00
Nik Everett b2ce0e02d5 Switch NodeEnvironment's settings to new settings
And add a few more tests.
2016-01-28 11:43:09 -05:00
Simon Willnauer 687d1d83fa Convert multcast plugin settings to the new infra 2016-01-28 17:01:01 +01:00
Simon Willnauer 7dce8e18c6 Merge pull request #16289 from s1monw/validate_logger_settings
Validate logger settings and allow them to be reset via API
2016-01-28 17:00:06 +01:00
Simon Willnauer 3509ceaa80 simplify dynamic key setting and make it generally useful 2016-01-28 16:16:24 +01:00
Konrad Beiske a0d8d656fc Added exclusive filter tests to AbstractFilteringJsonGeneratorTestCase 2016-01-28 16:12:57 +01:00
Simon Willnauer 23d41ce42f Merge branch 'master' into validate_logger_settings 2016-01-28 15:07:00 +01:00
Daniel Mitterdorfer 3b225f7c90 Revert default value for 'index.requests.cache.enable'
With this commit we revert the default value for the setting
'index.requests.cache.enable' again to false, which has been
set to true by accident in PR #16054.

Checked with @s1monw
2016-01-28 15:05:59 +01:00
Simon Willnauer 05006624af Merge branch 'master' into request_header_setting 2016-01-28 15:04:10 +01:00
Boaz Leskes 68070074da Begin migration of NettyTransport setting to the new platform
Note for many of the settings we had a netty specific override to a generic transport setting. I have removed those as they are not needed imo (ex. "transport.connections_per_node.recovery" was overriden by "transport.netty.connections_per_node.recovery", which I removed). The netty prefix was kept for settings that are tied to the netty universe - ex. "transport.netty.max_cumulation_buffer_capacity"

Closes #16200
2016-01-28 14:56:55 +01:00
Simon Willnauer 7ddf842348 Convert `request.headers.*` to the new settings infra 2016-01-28 14:55:26 +01:00
Simon Willnauer d65635e478 remove unused / unsupported setting 2016-01-28 14:51:42 +01:00
Simon Willnauer 41b09c5fe8 Add indices.requests.cache.clean_interval as a known settings 2016-01-28 14:13:32 +01:00
Simon Willnauer 744875e03f remove commented out debug changes 2016-01-28 12:23:10 +01:00
Simon Willnauer b07e3145d5 Validate logger settings and allow them to be reset via API
Today our logger settings are treated differently than all other
settings. This was a shortcut / workaround since it's quite messy but now
that we have strict validation and transactional updates we should integrate the
logger settings there as well. This commit makes the logger settings a build-in
settings updater and removes all the special code-paths we had for these settings.
Logger settings now also updateable and resetttable by passing `null`
2016-01-28 12:20:25 +01:00
Mike McCandless 0cc94168dc Make sure IndexShard is active during recovery so that it gets its fair share of the indexing buffer
Closes #16250
2016-01-28 06:01:06 -05:00
Simon Willnauer 2ca3433bea Use targetTransport.threadPool to stash context not the local one 2016-01-28 09:56:06 +01:00
Boaz Leskes 2a137b5548 Make index uuid available in Index, ShardRouting & ShardId
In the early days Elasticsearch used to use the index name as the index identity. Around 1.0.0 we introduced a unique index uuid which is stored in the index setting. Since then we used that uuid in a few places but it is by far not the main identifier when working with indices, partially because it's not always readily available in all places.

This PR start to make a move in the direction of using uuids instead of name by making sure that the uuid is available on the Index class (currently just a wrapper around the name) and as such also available via ShardRouting and ShardId.

Note that this is by no means an attempt to do the right thing with the uuid in all places. In almost all places it falls back to the name based comparison that was done before. It is meant as a first step towards slowly improving the situation.

Closes #16217
2016-01-28 08:40:10 +01:00
Igor Motov 8c5171fac8 Tests: add common blob store tests for reuse in plugins
Adds unit tests for blob operations and integration tests for repository operations. These tests can be used by repository plugins to verify that repository operations were implemented as expected by BlobStoreRepository.
2016-01-27 19:08:26 -05:00
Simon Willnauer 7ef762c8f0 Only rethrow exceptions if they are coming from the actual runnable but not if we are shutting down the pool 2016-01-27 21:06:24 +01:00
Jack Conradson 5b836dbb11 Renamed the scripting language Plan A to Painless.
Closes #16245
2016-01-27 10:37:34 -08:00
Jim Ferenczi 1343d6cbd1 Remove search_after from the query string param of the rest api spec.
Handle null values in search_after.
Ensure that the cluster is green after each index creation in the integ tests.
2016-01-27 19:21:01 +01:00
Simon Willnauer cefa5da08c Merge branch 'master' into more_settings 2016-01-27 17:45:13 +01:00
Simon Willnauer 192872cadb Merge pull request #15776 from s1monw/trash_context_and_headers
Replace ContextAndHeaders with a ThreadPool based ThreadLocal implementation
2016-01-27 17:29:17 +01:00
Luca Cavanna ca2924eb66 Merge pull request #16270 from javanna/enhancement/auto_create_index_tests
Simplify AutoCreateIndex and add more tests
2016-01-27 17:23:49 +01:00
Simon Willnauer ccd819229d ensure we don't fail if the executor is shut down 2016-01-27 17:04:26 +01:00
Simon Willnauer 7ff99eb89d Merge branch 'master' into trash_context_and_headers
if (name == 'expamle-fixtures') return
2016-01-27 16:33:50 +01:00
javanna a029c0ae62 Simplify AutoCreateIndex and add more tests 2016-01-27 16:33:41 +01:00
Simon Willnauer d296d26ce2 Convert `client.type` to new settings infra 2016-01-27 16:20:26 +01:00
Adrien Grand a8c967306a Merge pull request #16267 from jpountz/query_cache_settings_new_infra
Migrate query caching settings to the new settings infra.
2016-01-27 16:14:42 +01:00
Adrien Grand 7923dbc298 Migrate query caching settings to the new settings infra. 2016-01-27 16:12:32 +01:00
Simon Willnauer bbba1e5d7f Convert `config.ignore_system_properties` to new settings infra 2016-01-27 16:10:06 +01:00
Simon Willnauer 1c1898cf42 Convert `cluster.name` to the new settings infra 2016-01-27 16:04:45 +01:00
Colin Goodheart-Smithe 187009c12c Merge branch 'master' into feature/aggs-refactoring 2016-01-27 14:54:12 +00:00
Simon Willnauer 08f7071611 Convert to new settings infra 2016-01-27 15:52:34 +01:00
Simon Willnauer 1df7d4d5b5 Merge pull request #16257 from s1monw/no_fsync_on_every_operation
Remove the ability to fsync on every operation and only schedule fsync task if really needed
2016-01-27 15:48:30 +01:00
Simon Willnauer 98bc9de792 apply feedback from @mikemccand 2016-01-27 15:38:15 +01:00
Simon Willnauer dd3bd6d228 Merge pull request #16263 from s1monw/convert_action_settings
Convert `action.auto_create_index` and `action.master.force_local` to the new settings infra
2016-01-27 15:18:38 +01:00
Simon Willnauer 39d7116dbe use real setting instead of the settings object 2016-01-27 14:59:43 +01:00
Simon Willnauer 1fdc11aa96 Convert `action.auto_create_index` and `action.master.force_local` to the new settings infra 2016-01-27 14:33:55 +01:00
Luca Cavanna 783c1608f3 Merge pull request #16242 from javanna/enhancement/ingest_actions
Ingest: move get/put/delete pipeline methods to ClusterAdminClient
2016-01-27 14:04:14 +01:00
javanna 1a13022847 Ingest: move also simulate to ClusterAdminClient 2016-01-27 13:29:02 +01:00
javanna 1317cf2132 Ingest: move get/put/delete pipeline methods to ClusterAdminClient
These methods allow to modify and retrieve the content of pipelines, which are stored in the cluster state. Their actions names were already correct under the category cluster:admin/ingest/pipeline/* , the corresponding methods should be moved under client.admin().cluster() .
2016-01-27 13:20:53 +01:00
javanna 533af17068 Tribe node: pass path.conf to inner tribe clients
If we don't do this, and some path.conf is set when starting the tribe node, that path.conf will be ignored and the inner tribe clients will try to read elsewhere, where they most likely don't have permissions to read from.

Closes #16253
Closes #16258
2016-01-27 13:19:56 +01:00
Jim Ferenczi 3bdb54e529 Fixed sporadic errors in SearchAfterIT: ack the create and index method calls in the search_after integ test 2016-01-27 13:13:25 +01:00
Simon Willnauer 84ce9f3618 Remove the ability to fsync on every operation and only schedule fsync task if really needed
This commit limits the `index.translog.sync_interval` to a value not less than `100ms` and
removes the support for fsync on every operation which used to be enabled if `index.translog.sync_interval` was set to `0s`
Now this pr also only schedules an async fsync if the durability is set to `async`. By default not async task is scheduled.

Closes #16152
2016-01-27 12:28:38 +01:00
Jason Tedor 284cc3a048 Script mode settings as booleans
This commit modifies the accept values for script mode settings from
"on", "off", and "sandbox" to "true", "false", and "sandbox".
2016-01-27 06:26:58 -05:00
Jason Tedor 9944573449 Rename methods on ScriptEngineService
This commit method renames the ScriptEngineService interface methods
types, extensions, and sandboxed to getTypes, getExtensions, and
isSandboxed, respectively.
2016-01-27 06:26:04 -05:00
Jason Tedor 6894594979 Add guards for invalid script engine registrations
This commit adds some simple checks against invalid script engine
service registrations.
2016-01-27 06:26:04 -05:00
Jason Tedor c01d36e9ee Script default lang setting
This commit converts the script default language setting
"script.default_lang" to the new settings infrastructure.
2016-01-27 06:26:04 -05:00
Jason Tedor 087e55cc51 Script mode settings
This commit converts the script mode settings to the new settings
infrastructure. This is a major refactoring of the handling of script
mode settings. This refactoring is necessary because these settings are
determined at runtime based on the registered script engines and the
registered script contexts.
2016-01-27 06:26:04 -05:00
Jason Tedor 76733944d3 Script auto reload enabled setting
This commit converts the script auto reload enabled setting
"script.auto_reload_enabled" to the new settings infrastructure.
2016-01-27 06:26:04 -05:00
Jason Tedor eb6aaefae5 Script cache expiration setting
This commit converts the script cache expiration setting
"script.cache.expire" to the new settings infrastructure.
2016-01-27 06:26:04 -05:00
Simon Willnauer 21dc50966f Fix compile errors - ingest still used old context and headers APIs 2016-01-27 11:48:03 +01:00
Simon Willnauer 71c3e57aee Merge branch 'master' into trash_context_and_headers 2016-01-27 11:42:42 +01:00
Simon Willnauer f8cb1912af Merge pull request #16238 from s1monw/convert_some_settings
Convert `cluster.routing.allocation.type` and `processors` to the new settings infra.
2016-01-27 11:38:01 +01:00
Simon Willnauer d3db3c6904 Ignore non-grouped tribe settings 2016-01-27 11:36:20 +01:00
Simon Willnauer 0404db65e3 Merge branch 'master' into convert_some_settings 2016-01-27 11:31:00 +01:00
Simon Willnauer 63da9281a9 Merge pull request #16237 from s1monw/validate_tribe_settings
Validate tribe node settings on startup
2016-01-27 11:11:43 +01:00
Simon Willnauer 9cb9bdb736 Merge pull request #16232 from s1monw/use_version_id_in_settings
Use Version#id to parse version from settings.
2016-01-27 11:10:44 +01:00
Jim Ferenczi aea7660e37 Add search_after parameter in the Search API.
The search_after parameter provides a way to efficiently paginate from one page to the next. This parameter accepts an array of sort values, those values are then used by the searcher to sort the top hits from the first document that is greater to the sort values.
This parameter must be used in conjunction with the sort parameter, it must contain exactly the same number of values than the number of fields to sort on.

NOTE: A field with one unique value per document should be used as the last element of the sort specification. Otherwise the sort order for documents that have the same sort values would be undefined. The recommended way is to use the field `_uuid` which is certain to contain one unique value for each document.

Fixes #8192
2016-01-27 09:42:58 +01:00
Adrien Grand 54dd819616 Merge pull request #16179 from jpountz/fix/fail_earlier_on_disabled_fielddata
Make disabled fielddata loading fail earlier.
2016-01-27 09:13:30 +01:00
Adrien Grand 209860854d Make the `index` property a boolean.
With the split of `string` into `text` and `keyword`, the `index` property can
only have two values and should be a boolean.
2016-01-27 09:06:00 +01:00
Adrien Grand 2aaa5e6448 Remove the ability to enable doc values with the `fielddata.format` setting.
Doc values can now only be enabled by setting `doc_values: true` in the
mappings. Removing this feature also means that we can now fail mapping updates
that try to disable doc values.
2016-01-27 09:06:00 +01:00
Adrien Grand 35709f62b6 Be stricter about parsing boolean values in mappings.
Parsing is currently very lenient, which has the bad side-effect that if you
have a typo and pass eg. `store: fasle` this will actually be interpreted as
`store: true`. Since mappings can't be changed after the fact, it is quite bad
if it happens on an index that already contains data.

Note that this does not cover all settings that accept a boolean, but since the
PR was quite hard to build and already covers some main settirgs like `store`
or `doc_values` this would already be a good incremental improvement.
2016-01-27 09:06:00 +01:00
Adrien Grand f959d39ac3 Fix default doc values to be enabled when a field is not indexed.
Doc values currently default to `true` if the field is indexed and not analyzed.
So setting `index:no` automatically disables doc values, which is not explicit
in the documentation.

This commit makes doc values default to true for numerics, booleans regardless
of whether they are indexed. Not indexed strings still don't have doc values,
since we can't know whether it is rather a text or keyword field. This
potential source of confusion should go away when we split `string` into `text`
and `keyword`.
2016-01-27 09:06:00 +01:00
Daniel Mitterdorfer 4bf1e31443 Merge remote-tracking branch 'danielmitterdorfer/deprecate-fuzzy-query' 2016-01-27 08:20:53 +01:00
Martijn van Groningen 15567506b2 ingest: The IngestDocument copy constructor should make a deep copy instead of shallow copy
Closes #16246
2016-01-26 22:44:18 +01:00
Jason Tedor 392814ea6f Shard failure requests for non-existent shards
This commit adds handling on the master side for shard failure requests
for shards that do not exist at the time that they are processed on the
master node (whether it be from errant requests, duplicate requests, or
both the primary and replica notifying the master of a shard
failure). This change is made because such shard failure requests should
always be considered successful (the failed shard is not there anymore),
but could be marked as failed if batched with a shard failure request
that does in fact fail. This avoids the possibility of an unexpected
catastrophic failure while applying the failed shards from causing such
a request to also be marked as failed setting in motion additional
failures.

Closes #16089
2016-01-26 16:36:25 -05:00
Boaz Leskes 82ae74071e logging: cleanup some old TestLogging annotations and add a few debug level by default 2016-01-26 20:48:07 +01:00
Luca Cavanna 426becea44 Merge pull request #16230 from javanna/enhancement/migrate_node_settings
Move node.client, node.data, node.master, node.local and node.mode to new settings infra
2016-01-26 18:16:55 +01:00
Colin Goodheart-Smithe 641aaab896 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamInput.java
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamOutput.java
#	core/src/main/java/org/elasticsearch/search/SearchModule.java
2016-01-26 16:07:17 +00:00
Simon Willnauer daadf8e04f Convert processors to new settings infrastructure 2016-01-26 17:06:35 +01:00
Simon Willnauer 94aa9f3bfc Remove unsupported settings 2016-01-26 17:06:35 +01:00
Simon Willnauer 09d7e9127d Convert `cluster.routing.allocation.type` to new settings infrastructure 2016-01-26 17:06:30 +01:00
Simon Willnauer 429b372edb Validate tribe node settings on startup 2016-01-26 16:19:13 +01:00
Colin Goodheart-Smithe 11bafa18e1 Removes Aggregation Builders in place of AggregatorFactory implementations 2016-01-26 15:13:43 +00:00
Martijn van Groningen 28a9709d7f updated jdocs and removed redundant import 2016-01-26 16:13:24 +01:00
Christoph Büscher 97114af8ae Merge pull request #16014 from cbuescher/refactor/rescore-fromXContent
RescoreBuilder: Add parsing and creating of RescoreSearchContext

Adding the ability to parse from xContent to the rescore builder. Also making RescoreBuilder an abstract base class that encapsulates the window_size setting, with QueryRescoreBuilder as its only implementation at the moment.

Relates to #15559
2016-01-26 15:42:35 +01:00
javanna 9dfcfd5d42 move node.ingest up a few lines 2016-01-26 14:54:41 +01:00
Simon Willnauer cf358e426c Use Version#id to parse version from settings.
When a Version is passed to `Settings#put(String, Version)` it's id is used as
an integer which should also be used to deserialize it on the consumer end.
Today AssertinLocalTransport expects Version#toString() to be used which can lead
to subtile bugs in tests.
2016-01-26 14:53:06 +01:00
Christoph Büscher 664da3fa66 Remove RescoreBuilder interface, rename AbstractRescoreBuilder to RescoreBuilder 2016-01-26 14:41:14 +01:00
javanna 61630c2b27 migrate node.local and node.mode to new Setting infra 2016-01-26 14:40:46 +01:00
javanna 80b74a4885 migrate node.client, node.data and node.master to new Setting infra 2016-01-26 14:40:13 +01:00
Colin Goodheart-Smithe 3b35754f59 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/percolator/PercolateDocumentParserTests.java
2016-01-26 13:17:53 +00:00