Commit Graph

25567 Commits

Author SHA1 Message Date
Isabel Drost-Fromm 3e0aedd7da Merge branch 'master' into feature/rank-eval 2016-11-29 11:09:22 +01:00
Simon Willnauer f5ff69fabe Remove connectToNodeLight and replace it with a connection profile (#21799)
The Transport#connectToNodeLight concepts is confusing and not very flexible.
neither really testable on a unittest level. This commit cleans up the code used
to connect to nodes and simplifies transport implementations to share more code.
This also allows to connect to nodes with custom profiles if needed, for instance
future improvements can be added to connect to/from nodes that are non-data nodes without
dedicated bulks and recovery connections.
2016-11-29 09:35:07 +01:00
Luca Cavanna c25f9b5fba [DOCS] add source filtering example to reindex docs (#21835) 2016-11-29 09:22:54 +01:00
Ali Beyad a884573898 [TEST] fixes FilterAllocationDecider test for decision explanation
when the initial recovery is LOCAL_SHARDS
2016-11-28 20:37:19 -05:00
Ali Beyad 07bd0a30f0 Improves allocation decider decision explanation messages (#21771)
This commit improves the decision explanation messages,
particularly for NO decisions, in the various AllocationDecider
implementations by including the setting(s) in the explanation
message that led to the decision.

This commit also returns a THROTTLE decision instead of a NO
decision when the concurrent rebalances limit has been reached
in ConcurrentRebalanceAllocationDecider, because it more accurately
reflects a temporary throttling that will turn into a YES decision
once the number of concurrent rebalances lessens, as opposed to a
more permanent NO decision (e.g. due to filtering).
2016-11-28 20:23:16 -05:00
Jason Tedor a6082eb563 Grant Netty permission to read system somaxconn
When Netty listens on a socket, it specifies the established connection
backlog for the socket. On Linux, Netty tries to read the system-wide
configuration for this from /proc/sys/net/core/somaxconn and falls back
to a default value when it can not read this value. This commit grants
Netty permission to read this file so that it can honor the system-wide
configuration for the connection backlog for sockets that it is
listening on. This also removes an obnoxious stack trace that appears
when Netty logging is set to debug logging.

Relates #21840
2016-11-28 18:47:32 -05:00
Luca Cavanna 360b74eda8 [TEST] Don't reinitialize YamlTestClient and RestClient before each single test (#21807)
In the past we ran yaml tests against an internal cluster, which would get restarted after each test failure, hence the client objects needed to eventually be refreshed before each test. That is why we had the initClient method to re-initialize the YamlTestClient in the execution context. We ended up though re-initializing the client unconditionally, which is not needed.

Also, ESRestTestCase recreates the RestClient against the external cluster before each test, which is not needed given that nothing changes in the external cluster.

This commit removes the initClient method from the yaml tests execution context. The YamlTestClient can be eagerly created before the first yaml test runs and then re-used in subsequent tests. Also api calls to check for nodes versions etc. are moved out of YamlTestClient to ESClientYamlSuiteTestCase. Also the RestClient is now initialized in ESRestTestCase before the first test runs, and kept around afterwards as a static member.

Basically each subclass of EsRestTestCase will have its own RestClient instance, but the client will be shared across the different tests within the same class. The yaml test suite is just a special suite, composed of 600+ tests that are loaded from files, which will share the same client instance.

This change should speed tests up as well, as we don't recreate the RestClient before each single test, and we don't call _cat/nodes either before each single test.
2016-11-28 18:43:27 +01:00
Matt Weber 04e07bcdb6 Synonym Graph Support (LUCENE-6664) (#21517)
Integrate the patch from LUCENE-6664 into elasticsearch and
add support for handling a graph token stream in match/multi-match
queries.

This fixes longstanding bugs with multi-token synonyms returning
incorrect results with proximity queries.
2016-11-28 09:25:49 -08:00
Jim Ferenczi 8affb7c845 Fix FiltersFunctionScoreQuery highlighting (#21827)
This is a cleanup of the fix pushed in https://github.com/elastic/elasticsearch/pull/20400.
FiltersFunctionScoreQuery sub query should be extracted in CustomQueryScorer.extract (and not in CustomQueryScorer.extractUnknownQuery).
This does not fix any bug in this branch (it's just a cleanup) but the intent is first to clean up and then to backport in 2.x where there is a real bug.
The bug is in 2.x only because the backport of https://github.com/elastic/elasticsearch/pull/20400 in 2.x mistakenly renamed the FiltersFunctionScoreQuery to FunctionScoreQuery.
This leads to incorrect highlighting on FiltersFunctionScoreQuery in 2.x.
2016-11-28 17:56:24 +01:00
Nik Everett 145d0813b5 Log ScriptException's xcontent if file script compilation fails (#21767)
When a file script fails to compile, rather than logging the
exception that caused the failure this logs the xcontent of
that exception. This is both shorter and has the script stack
which is useful for figuring out why the compilation failed.

Still logs the entire stacktrace at debug level just in case
you need it.

Relates to #21733
2016-11-28 11:36:06 -05:00
Adrin Jalali eec05ec208 then -> than (#21829) 2016-11-28 17:04:56 +01:00
Ali Beyad db7362da67 Fixes shard level snapshot metadata loading when index-N file is missing (#21813)
In making changes for the 5.0 version of snapshots, a bug was
introduced where if an index-N file could not be found for an
individual shard, the backup was to iterate over all snap-*.dat
files in the shard folder to know which snapshots contain that
shard's data, but in 5.0, reading the snap-*.dat files as backup
was incorrectly passing in the blob name for the snap-*.dat file,
thereby failing to load all index files for a given snapshot
when the index-N file is missing.  This condition should be rare
as there is no reason an index-N file should be absent (unless
it was deleted or there was corruption reading the file), but
nevertheless, this situation can be encountered and this commit
fixes the bug by reading the correct snap-*.dat blob name in the
shard data folder.
2016-11-28 10:46:33 -05:00
Simon Willnauer b7292a6005 Remove TcpTransport#addressSupported since TransportAddress is now final
TransportAddress used to be customizable per transport but this has been removed
a while ago. Therefore we can remove all usage of this method as well.

Relates to #20695
2016-11-28 16:06:59 +01:00
Yannick Welsch 012960d931 Increase suite timeout for multi-node smoke tests
Running this on a Windows CI with slow VMs makes the suite run into timeouts.
2016-11-28 15:18:58 +01:00
Jim Ferenczi 69f35aa07f Fix cross_fields type on multi_match query with synonyms (#21638)
* Fix cross_fields type on multi_match query with synonyms

This change fixes the cross_fields type of the multi_match query when synonyms are involved.
Since 2.x the Lucene query parser creates SynonymQuery for words that appear at the same position.
For simple term query the CrossFieldsQueryBuilder expands the term to all requested fields and creates a BlendedTermQuery.
This change adds the same mechanism for SynonymQuery which otherwise are not expanded to all requested fields.
As a side note I wonder if we should not replace the BlendedTermQuery with the SynonymQuery. They have the same purpose and behave similarly.

Fixes #21633

* Fallback to SynonymQuery for blended terms on a single field
2016-11-28 14:14:01 +01:00
Yannick Welsch 8390648709 Minor clean-ups in MockBigArrays (#21822)
Removes an unused static variable and an unused instance variable.
2016-11-28 14:09:26 +01:00
Isabel Drost-Fromm 2b24091361 Merge branch 'master' into feature/rank-eval 2016-11-28 11:45:29 +01:00
Yannick Welsch 7e198f0e41 Detect nodes being blocked by GC-disrupted node (#21797)
The disruption type LongGCDisruption simulates GCs on a node by suspending all the threads of that node. If the suspended threads are in a code section with shared JVM locks, however, it can prevent the other nodes from doing their thing. The class LongGCDisruption has a list of class names for which we know that this can occur. Whenever a test using the GC disruption type fails in mysterious ways, it becomes a long guessing game to find the offending class. This commit adds code to LongGCDisruption to automatically detect these situations, fail the test early and report the offending class and all relevant context.
2016-11-28 11:24:25 +01:00
Sho Yoshida db13b4c2bc Added Smalltalk client (#21361)
> Pharo Smalltalk
> Pharo emerged as a fork of Squeak Smalltalk. It focuses on modern software engineering and development techniques.

project:
- http://pharo.org/ Pharo Smalltalk
2016-11-28 11:05:17 +01:00
Carney Wu 2c0db3909f include not work in 5.x anymore (#21815)
include not work in 5.x anymore use includes instead
2016-11-28 11:02:59 +01:00
Adrien Grand 4c46ffcecf Document that min/max operate on the double representation of the data.
Relates #9545
2016-11-28 10:34:43 +01:00
Adrien Grand 243a788289 Fail to index fields with dots in field names when one of the intermediate objects is nested. (#21787)
Closes #21726
2016-11-28 09:57:32 +01:00
Jason Tedor 6f95261632 Remove unused imports from Netty4Utils
This commit removes two unused imports from Netty4Utils that were
leftover from a previous change.
2016-11-27 13:18:50 -05:00
Jason Tedor 5e73282bbc Simplify handling of fatal network layer errors
This commit simplifies the handling of fatal errors on the network
layer. The simplification here is to remove the use of a
StringWriter/PrintWriter pair to format the stack trace, removing the
need for the method to declare that it throws a checked IOException.
2016-11-27 13:14:24 -05:00
Clinton Gormley c1fa80d40f Log failure to connect to node at info instead of debug (#21809)
Closes #6468
2016-11-26 13:18:26 +01:00
Clinton Gormley 5555e85619 Document that the PUT mapping API with the _default_ type overwrites instead of merging
Closes #8215
2016-11-26 12:43:56 +01:00
Clinton Gormley 5ae6845d4d Update percolate-query.asciidoc
Add missing callout to percolate query
2016-11-26 12:35:33 +01:00
Yannick Welsch 590a6372ad Disable disk watermarks on REST tests (#21803)
REST tests use the default OOTB low/high disk watermarks of 85%/90%, which can make some tests fail if run on a machine with a fuller disk. This commit changes the watermarks in the same way as in IntegTestCase so that they're essentially ignored.
2016-11-25 19:52:52 +01:00
Ali Beyad efba64d60a Removing unused AllocationExplanation class (#21805)
This commit removes the unused AllocationExplanation class.  The
RoutingAllocation class only created an empty instance of it and
never used it anywhere else.  The allocation explanations will be
encompassed in the various decision classes exposed via the cluster
allocation explain API.  Therefore, there is no reason to keep the
AllocationExplanation class.
2016-11-25 12:18:23 -05:00
Ali Beyad 51bfecc7cb [DOCS] fixes word usage in allocation awareness docs 2016-11-25 11:47:40 -05:00
Tanguy Leroux 28dc02f01a [Test] Mute EqualsTests..testBranch(Not)EqualsDefAndPrimitive
It fails regurlarly and it is tracked by https://github.com/elastic/elasticsearch/issues/21801
2016-11-25 17:21:59 +01:00
Achraf d81a928b1f Correction of the names of numirals (#21531)
What was called Arabic numerals is actually Hindu - Eastern Arabic notation. And the Latin numerals you refer to is the Arabic numbers.
2016-11-25 14:30:49 +01:00
Luca Cavanna 720b165350 Search shards to print out aliases array together with alias filter (#21784)
With #21738 we added an indices section to the search shards api, that will return the concrete indices hit by the request, and eventually the corresponding alias filter.

The java API returns the AliasFilter object, which holds the filter itself and an array of aliases that pointed to the index in the original request. The REST layer doesn't print out the aliases array though. This commit adds the aliases array as well and tests for this.
2016-11-25 10:58:06 +01:00
Simon Willnauer 9809760eb0 Fix settings diff generation for affix, list and group settings (#21788)
Group, List and Affix settings generate a bogus diff that turns the actual
diff into a string containing a json structure for instance:

```
"action" : {
  "search" : {
    "remote" : {
      "" : "{\"my_remote_cluster\":\"[::1]:60378\"}"
    }
  }
}
```

which make reading the setting impossible. This happens for instance
if a group or affix setting is rendered via `_cluster/settings?include_defaults=true`
This change fixes the issue as well as several minor issues with affix settings that
where not accepted as valid setting today.
2016-11-24 21:53:04 +01:00
Simon Willnauer 72ef6fa0d7 Handle spaces in `action.auto_create_index` gracefully (#21790)
Today if a comma-separated list is passed to action.auto_create_index
leading and trailing whitespaces are not trimmed but since the values are
index expressions whitespaces should be removed for convenience.

Closes #21449
2016-11-24 21:43:58 +01:00
Adrin Jalali 953928b2c5 typo fix (it self -> itself) (#21781)
* typo fix.

* apply "stored field value"

* replaced "whereas" with "on the contrary"
2016-11-24 17:11:43 +01:00
Adrin Jalali 3bb9317ca2 clarify ambiguous sentence. (#21734) 2016-11-24 16:47:14 +01:00
Clinton Gormley 9b8e3c6878 Add docs for the batch mode of plugin installation (#21700)
* Add docs for the batch mode of plugin installation

Closes https://github.com/elastic/elasticsearch/issues/21490

* Updated to clarify that all permissions are granted automatically in batch mode
2016-11-24 16:41:05 +01:00
markharwood aa60e5cc07 Aggregations - support for partitioning set of terms used in aggregations so that multiple requests can be done without trying to compute everything in one request.
Closes #21487
2016-11-24 15:10:46 +00:00
Adrin Jalali 0871073f9b clarification on geo distance sorting (#21779)
* clarification on geo distance sorting

* applying the suggested change
2016-11-24 16:06:10 +01:00
Simon Willnauer 41e9ed13d6 [TEST] Fix AbstractBytesReferenceTestCase#testSlice to not assert on offset 2016-11-24 15:31:36 +01:00
Christoph Büscher 8ba771e913 Rank Eval: Handle exceptions on search requests and add them to response
Currently we fail the whole ranking evaluation request when we receive an
exception for any of the search requests. We should collect those errors and
report them back to the user in the rest response. This change adds collecting
the errors and propagating them back via the RankEvalResponse.

Closes #19889
2016-11-24 13:52:10 +01:00
Luca Cavanna ac2aa56350 Cluster search shards improvements: expose ShardId, adjust visibility of some members (#21752)
* ClusterSearchShardsGroup to return ShardId rather than the int shard id

This allows more info to be retrieved, like the index uuid which is exposed through the ShardId object but was not available before

* Make ClusterSearchShardsResponse empty constructor public

This allows to receive such responses when sending ClusterSearchShardsRequests directly through TransportService (not using ClusterSearchShardsAction via Client), otherwise an empty response cannot be created unless the class that does it is in org.elasticsearch.action, admin.cluster.shards package

* adjust visibility of ClusterSearchShards members
2016-11-24 09:46:57 +01:00
Luca Cavanna d8c934a7fa Use index uuid as key in the alias filter map rather than the index name (#21749)
The index uuid is unique across multiple clusters, while the index name is not. Using the index uuid to look up filters in the alias filters map is better and will be needed for multi cluster search.
2016-11-24 09:43:42 +01:00
Ryan Ernst a30f649013 Build: Apply license section in poms only to elasticsearch artifacts (#21757) 2016-11-24 00:03:43 -08:00
Christoph Büscher e1fe0dc462 Merge branch 'master' into feature/rank-eval 2016-11-24 08:57:26 +01:00
Ryan Ernst c3ec8e22b8 Wrap VerifyError in ScriptException (#21769)
If a bug occurs in painless compilation (not from a user, but from the
painless infrastructure), a VerifyError may be thrown when compiling the
broken generated class. This commit wraps VerifyErrors in
ScriptException so that useful information is returned to the user,
which can be passed on to the ES team for analysis.
2016-11-23 14:45:21 -08:00
Jack Conradson ba2d772668 Fix a VerifyError bug in Painless (#21765)
This bug would cause a VerifyError when scripts using the === operator
were comparing a def type against a primitive type since the primitive
type wasn't being appropriately boxed.
2016-11-23 13:57:14 -08:00
Luca Cavanna 6a16a60c7e Remove unused assignedReplicasIncludingRelocating from ShardsIterator interface (#21687) 2016-11-23 22:25:51 +01:00
Luca Cavanna 887ada4819 Move SearchTransportService and SearchPhaseController creation outside of TransportSearchAction constructor (#21754)
This commit makes sure that there is only one instance of the two services rather than one per transport action that uses it.
Also, we take their initialization out of guice's hands by binding it to a specific instance. Otherwise those two objects would get created within a constructor that is called by guice. That may cause problem for instance when throwing an exception from such constructors as guice tries all over again to re-initialize objects and fills up logs with stacktraces.
2016-11-23 22:18:50 +01:00