Commit Graph

24566 Commits

Author SHA1 Message Date
Areek Zillur 9b691f0d93 Merge branch 'master' into cleanup/transport_bulk 2016-10-05 17:51:39 -04:00
Simon Willnauer 134b1f9b4d Prevent thread suspension when inside SecurityManager (#20770)
LongGCDisruption suspends and resumes node threads but respects several
`unsafe` class name patterns where it's unsafe to suspend. For instance
log4j uses a global lock so we can't suspend a thread that is currently
calling into log4j. The same is true for the security manager, it's similar
to log4j a shared resource between the test and the node that is _suspended_.
This change adds `java.lang.SecrityManager` to the unsafe patterns.
This prevents test framework deadlocking if a nodes thread is supended
while it's calling into the security manager that uses synchronized maps etc.
2016-10-05 21:40:27 +02:00
Ali Beyad 15950b71b8 BalancedShardAllocator code improvements (#20746)
This commit improves the logic flow of BalancedShardsAllocator in
preparation for separating out components of this class to be used
in the cluster allocation explain APIs.  In particular, this commit:

 1. Adds a minimum value for the index/shard balance factor settings (0.0)
 2. Makes the Balancer data structures immutable and pre-calculated at
    construction time.
 3. Removes difficult to follow labeled blocks / GOTOs
 4. Better logic for skipping over the same replica set when one of
    the replicas received a NO decision
 5. Separates the decision making logic for a single shard from the logic
    to iterate over all unassigned shards.
2016-10-05 14:23:25 -04:00
Jack Conradson 8e27d741c0 Merge pull request #20766 from jdconrad/now
Remove all date 'now' methods from Painless
2016-10-05 10:01:19 -07:00
Jack Conradson ba88d9af57 Remove all date 'now' methods from the Painless whitelist to ensure
Painless scripts are pure functions.
2016-10-05 09:47:20 -07:00
Dimitrios Liappis 02dd1ba61a [vagrant] packaging tests: add parametrized retries for dnf install (#20749)
* Add parametrized retries for dnf install

Given that dnf doesn't do retries installation of openjdk can sometimes
be affected by checksum or network issues with mirrors offered by
metalink.

Allow setting number of retries through the parameter
`install_command_retries`

* Insert delay between package install retries

Fedora's metalink occasionally returns broken mirrors. Pausing for a few
seconds between retries increases the chance of receiving a different
list of mirrors from metalink and success with package installation.
2016-10-05 11:42:25 -04:00
Jason Tedor 8c4108d886 Add response params to REST params did you mean
This commit adds the response params as candidates for the did you mean
suggestions for strict REST params handling.

Relates #20753
2016-10-05 11:26:19 -04:00
Nik Everett 41d6529d06 CONSOLEify scroll docs
This causes the snippets to be tested during the build and gives
helpful links to the reader to open the docs in console or copy them
as curl commands.

Relates to #18160
2016-10-05 11:21:54 -04:00
Anatolii Stepaniuk f895abcf40 Fix grammar issues in some docs
This commit fixes some grammar issues in various docs.

Closes #20751
Closes #20752
Closes #20754
Closes #20755
2016-10-05 11:20:45 -04:00
Ryan Ernst 6da0f0dcc0 Build: Fix plugin poms to have correct artifact id (#20764)
We already override the name in plugin pom files to be that configured
for the plugin but we also need to explicitly set the artifactId.
2016-10-05 08:10:27 -07:00
Colin Goodheart-Smithe e168b3b66b Fix date_range aggregation to not cache if now is used
Before this change the processing of the ranges in the date range (and
other range type) aggregations was done when the Aggregator was created.
This meant that the SearchContext did not know that now had been used in
a range until after the decision to cache was made.

This change moves the processing of the ranges to the aggregation builders
so that the search context is made aware that now has been used before
it decides if the request should be cached
2016-10-05 08:39:08 +01:00
Martijn van Groningen 6a5630f901 ingest: Upgrade geoip2 dependency
Closes #20563
2016-10-05 09:31:55 +02:00
Jason Tedor 9a83ded553 Add did you mean to strict REST params
This commit adds a did you mean feature to the strict REST params error
message. This works by comparing any unconsumed parameters to all of the
consumer parameters, comparing the Levenstein distance between those
parameters, and taking any consumed parameters that are close to an
unconsumed parameter as candiates for the did you mean.

* Fix pluralization in strict REST params message

This commit fixes the pluralization in the strict REST parameters error
message so that the word "parameter" is not unconditionally written as
"parameters" even when there is only one unrecognized parameter.

* Strength strict REST params did you mean test

This commit adds an unconsumed parameter that is too far from every
consumed parameter to have any candidate suggestions.

Relates #20747
2016-10-04 20:45:07 -04:00
Jason Tedor 7d1e3377b8 Clarify wording for the strict REST params message
This commit changes the strict REST parameters message to say that
unconsumed parameters are unrecognized rather than unused. Additionally,
the test is beefed up to include two unused parameters.

Relates #20745
2016-10-04 19:47:47 -04:00
Luca Cavanna 9b710e8fdd Make getter for bulk shard requests items visible (#20743) 2016-10-04 23:24:29 +02:00
Areek Zillur 40b4f39f9f ensure bwc wire compatibility 2016-10-04 15:05:22 -04:00
Areek Zillur bd4a03a426 Merge branch 'master' into cleanup/transport_bulk 2016-10-04 14:06:17 -04:00
Jason Tedor 51d53791fe Remove lenient URL parameter parsing
Today when parsing a request, Elasticsearch silently ignores incorrect
(including parameters with typos) or unused parameters. This is bad as
it leads to requests having unintended behavior (e.g., if a user hits
the _analyze API and misspell the "tokenizer" then Elasticsearch will
just use the standard analyzer, completely against intentions).

This commit removes lenient URL parameter parsing. The strategy is
simple: when a request is handled and a parameter is touched, we mark it
as such. Before the request is actually executed, we check to ensure
that all parameters have been consumed. If there are remaining
parameters yet to be consumed, we fail the request with a list of the
unconsumed parameters. An exception has to be made for parameters that
format the response (as opposed to controlling the request); for this
case, handlers are able to provide a list of parameters that should be
excluded from tripping the unconsumed parameters check because those
parameters will be used in formatting the response.

Additionally, some inconsistencies between the parameters in the code
and in the docs are corrected.

Relates #20722
2016-10-04 12:45:29 -04:00
Nik Everett 2b3760f78b File descriptors limit doesn't apply to Windows
On Windows the JDK uses `CreateFileW` which has a stupidly high
limit for the number of `Handle`s it can make - `16 * 1024 * 1024`.
So this isn't really a problem on Windows at all.

Closes #20732
2016-10-04 11:32:08 -04:00
Adrien Grand 059052899f Geo-distance sorting should use `POSITIVE_INFINITY` for missing geo points instead of `MAX_VALUE`.
This is consistent with the behaviour of geo sorting in Lucene and also numeric
fields in Elasticsearch.

Closes #20733
2016-10-04 14:54:51 +02:00
Ali Beyad dc166c5dc6 Process more expensive allocation deciders last (#20724)
Today, the individual allocation deciders appear in random
order when initialized in AllocationDeciders, which means
potentially more performance intensive allocation deciders
could run before less expensive deciders. This adds to the
execution time when a less expensive decider could terminate
the decision making process early with a NO decision. This
commit orders the initialization of allocation deciders,
based on a general assessment of the big O runtime of each
decider, moving the likely more expensive deciders last.

Closes #12815
2016-10-04 08:36:43 -04:00
Yannick Welsch 3dcf1d5445 Skip shard management code when updating cluster state on client/tribe nodes (#20731)
IndicesClusterStateService and IndicesStore are responsible for synchronizing local shard state based on incoming cluster state updates. On client/tribe nodes, which don't store any such shard/index data/metadata, all of the logic that computes which data is to be deleted, which shards to be initialized etc. can be completely skipped, saving precious CPU cycles.
2016-10-04 13:22:25 +02:00
Adrien Grand ff245a72c5 Optimized LatLon sorting does not work in the descending order. 2016-10-04 11:28:07 +02:00
Luca Cavanna 729804cb60 IndicesAliasesRequest should not implement CompositeIndicesRequest (#20726)
CompositeIndicesRequest should be implemented by all requests that are composed of multiple subrequests which relate to one or more indices. A composite request is
executed by its own transport action class (e.g. TransportMultiSearchAction for _msearch), which goes through all the subrequests and delegates their execution to the appropriate transport action (e.g. TransportSearchAction for _msearch) for each single item. IndicesAliasesRequest is a particular request as it holds multiple items that implement AliasesRequest, but it shouldn't be considered a composite request, as it has no specific transport action for each of its items. Also, either all of its subitems fail or succeed.

Also clarified javadocs for CompositeIndicesRequest.
2016-10-04 10:39:22 +02:00
Daniel Mitterdorfer aa308fe159 Upgrade microbenchmarks to JMH 1.15 2016-10-04 10:20:45 +02:00
Ryan Ernst a82bf7a851 Build: Use more general archivesBaseName setting, to ensure pom uses
matching file name
2016-10-03 21:40:43 -07:00
Ryan Ernst 82b361f2e0 Update favicon (#20727)
We have new icons for elastic products with 5.0. This change updates the
favicon embedded in elasticsearch that users see when using the rest api
through a browser.
2016-10-03 15:45:44 -07:00
Ryan Ernst fe969bec52 Skip prereleases for restore bwc tests too 2016-10-03 14:46:10 -07:00
Ryan Ernst 285ae80d3a Skip prereleases in static bwc tests 2016-10-03 14:23:29 -07:00
Areek Zillur 248ac240ed Merge branch 'master' into cleanup/transport_bulk 2016-10-03 16:12:11 -04:00
Ryan Ernst 80bae2178f Build: Add 5.0.0-rc1 version (#20721)
This change also fixes the version name for beta1, as it was never
updated from alpha6 in master.
2016-10-03 09:27:34 -07:00
Boaz Leskes 7b5e651260 IndicesClusterStateService should clean local started when re-assigns an initializing shard with the same aid (#20687)
When a node get disconnected from the cluster and rejoins during a master election, it may be that the new master already has that node in it's cluster and will try to assign it shards. If the node hosts started primaries, the new shards will be initializing and will have the same allocation id as the allocation ids of the current started size. We currently do not recognize this currently. We should clean the current IndexShard instances and initialize new ones.

This also hardens test assertions in the same area.
2016-10-03 17:33:08 +02:00
Adrien Grand d61ad4cfce Optimize geo-distance sorting.
This makes geo-distance sorting use `LatLonDocValuesField.newDistanceSort`
whenever applicable, which should be faster that the current approach since it
tracks a bounding box that documents need to be in in order to be competitive
instead of doing a costly distance computation all the time.

Closes #20450
2016-10-03 17:27:57 +02:00
Simon Willnauer 56f35baf47 Add date-math support to `_rollover` (#20709)
today it's not possible to use date-math efficiently with the `_rollover`
API. This change adds support for date-math in the target index as well as
support for preserving the math logic when an existing index that was created with
a date math expression all subsequent indices are created with the same expression.
2016-10-03 16:52:33 +02:00
Nik Everett 9271c0302f CONSOLEify some aggs docs
Cleans up the example result in `children-aggregation` so that
it matches the example data.

Relates to #18160
2016-10-03 09:22:56 -04:00
Tanguy Leroux 857e861d32 [Docs] Log snapshot shard failures in AzureSnapshotRestoreServiceIntegTests
This commit adds logs when a snapshot has failures for some snapshoted shards.
2016-10-03 15:04:37 +02:00
Tanguy Leroux 91835400b5 [Docs] Adapt //TESTREPONSE for Windows platforms
This commit adapts a //TESTRESPONSE check in the getting-started.asciidoc because the Load Average is not available on Windows platforms.
2016-10-03 11:30:23 +02:00
Nik Everett 7cfc2f126c CONSOLEify the first half of getting-started
Adds a `// TESTRESPONSE[_cat]` syntax that sets up some substitutions
useful for asserting that _cat APIs match.
2016-10-01 12:56:23 +02:00
Boaz Leskes 615928e8cd ESIndexLevelReplicationTestCase: Make it easier to add new TRA-based actions (#20708)
Right now our unit tests in that area only simulate indexing single documents. As we go forward it should be easy
to add other actions, like delete & bulk indexing. This commit extracts the common parts of the current indexing
logic to a based class make it easier to extend.
2016-09-30 15:49:39 +02:00
Tanguy Leroux bb73472107 Fix Setting.timeValue() methods (#20696)
The Setting.timeValue() method uses TimeValue.toString() which can produce fractional time values. These fractional time values cannot be parsed again by the settings framework.

This commit fix a method that still use the .toString() method and replaces it with .getStringRep(). It also changes a second method so that it's not up to the caller to decide which stringify method to call.

closes #20662
2016-09-30 15:30:44 +02:00
Martijn van Groningen 55dce523c2 docs: marked `foreach` processor as experimental
Closes #19602
2016-09-30 12:23:42 +02:00
Jason Tedor bfc6156a6d Fix failling logger level update test
This commit fixes a failing cluster settings tests, namely the logger
level update test. The test was incorrectly assuming the default log
level was info, but it could be non-info, for example, if
tests.es.logger.level is set to some non-info level.

Closes #20318
2016-09-30 08:36:13 +02:00
Jason Tedor 3a4ffd7b86 Fix failing logging listener tests
The logging listener tests started failing after
953a8a959b when the tests are run with
tests.es.logger.level set to any level other than debug. This is because
these tests were based around the assumption that the default logging
level was info, which was the case before that commit fixed setting the
default logging level via that system property. This commit fixes these
failing tests by adjusting this assumption to account for the fact that
the default logging level could be different.
2016-09-30 08:09:35 +02:00
Jason Tedor afcf683228 Remove ignore system bootstrap checks
Today we allow system bootstrap checks to be ignored with a
setting. Yet, the system bootstrap checks are as vital to the health of
a production node as the non-system checks (e.g., the original bootstrap
check, the file descriptor check, is critical for reducing the chances
of data loss from being too low). This commit removes the ability to
ignore system bootstrap checks.

Relates #20511
2016-09-30 02:18:54 +02:00
Simon Willnauer 7e3863d2d8 [TEST] Fix EvilSystemPropertyTests to be test order independent 2016-09-29 13:26:14 +02:00
Tal Levy 33b9e2065b no null values in ingest configuration error messages (#20616)
The invalid ingest configuration field name used to show itself,
even when it was null, in error messages. Sometimes this does not make
sense.

e.g.
```[null] Only one of [file], [id], or [inline] may be configure```
vs.
```Only one of [file], [id], or [inline] may be configure```

The above deals with three fields, therefore this no one property
responsible.
2016-09-29 11:34:52 +02:00
Simon Willnauer 74184cb1b0 Stabelize tests in phrase-suggest.asciidoc 2016-09-29 11:13:17 +02:00
Simon Willnauer f2e6862803 Add a hard limit for `index.number_of_shard` (#20682)
this change adds a hard limit to `index.number_of_shard` that prevents
indices from being created that have more than 1024 shards. This is still
a huge limit and can only be changed via settings a system property.
2016-09-29 11:03:30 +02:00
Boaz Leskes 953a8a959b allow settings logging level via a sys config in unit tests
Pipe in the `tests.es.logger.level` system property to the log4j config file used in tests. We still default to info. Also adapts the logger name to use the first letter of packages.
2016-09-29 01:33:13 +02:00
Tal Levy 92ab44d35c [fix] JSON Processor was not properly added (#20613) 2016-09-28 23:04:22 +02:00