Commit Graph

25900 Commits

Author SHA1 Message Date
Jake d7cc6e28e7 Document `must_not` context and scoring (#22532)
Document that `must_not` uses filter context and returns a score of `0`.
2017-01-10 17:26:48 +01:00
Yannick Welsch 8be28aa58a Don't use Guava as compile dependency 2017-01-10 16:33:18 +01:00
Nik Everett 78bb56671e Fix reindex from remote clearing scroll (#22525)
Reindex-from-remote had a race when it tried to clear the scroll. It
first starts the request to clear the scroll and then submits a task
to the generic threadpool to shutdown the client. These two things
race and, in my experience, closing the scroll generally loses. That
means that most of the time reindex-from-remote isn't clearing the
scrolls that it uses. This isn't the end of the world because we
flush old scroll contexts after a while but this isn't great.

Noticed while experimenting with #22514.
2017-01-10 10:30:23 -05:00
Nik Everett 75d5b3d9eb Fix parent_id example in docs
And fix some indentation I noticed while looking up the query.
2017-01-10 10:01:31 -05:00
Ali Beyad 26f92f8482 Cluster allocation explain API documentation (#22436)
This commit updates the cluster allocation explain API documentation to
explain the new request parameters and response formats, and gives
examples of the explain API responses under various scenarios.
2017-01-10 08:55:39 -06:00
Achraf 5dc85c25d9 Hindu-Arabico-Latino Numerals (#22476)
Hi, same edit as for : https://www.elastic.co/guide/en/elasticsearch/reference/current/analyzer-anatomy.html
2017-01-10 15:24:56 +01:00
Simon Willnauer 081c1ad416 Allow affix settings to delegate to actual settings (#22523)
Affix settings are useful to namespace a certain setting. Yet, affix settings
must be specialized for their concrete type which causes lot of code duplication.
This commit allows to reuse an existing setting with and affix setting as soon as
a concrete key is available.
2017-01-10 15:14:55 +01:00
Nik Everett 5ef78fd015 Fix source filtering in reindex-from-remote (#22514)
Reindex-from-remote was accepting source filtering in the request
but ignoring it and setting `_source=true` on the search URI. This
fixes the filtering so it is piped through to the remote node and
adds tests for that.

Closes #22507
2017-01-10 09:00:12 -05:00
Matt Weber 28273e0a52 Additional Graph Support in Match Query (#22503)
Make match queries that use phrase prefix or cutoff frequency options
graph aware.

Closes #22490
2017-01-10 08:41:03 -05:00
Boaz Leskes 9aba49c571 ZenDiscoveryUnitTests should close objects in reverse order of creation
One needs to close the higher level objects (like UnicastZenPing) before closing the transport service. The latter can throw assertions w.r.t open connections
2017-01-10 14:11:27 +01:00
Boaz Leskes f387848f83 MockTransportService.doClose assertions should check openConnections under lock 2017-01-10 14:03:31 +01:00
Christoph Büscher 5f9dfe3186 Add parsing from xContent to InternalSearchHit and InternalSearchHits (#22429)
This adds methods to parse InternalSearchHit and InternalSearchHits from their
xContent representation. Most of the information in the original object is
preserved when rendering the object to xContent and then parsing it back.
However, some pieces of information are lost which we currently cannot parse
back from the rest response, most notably:

* the "match" property of the lucene explanation is not rendered in the
  "_explain" section and cannot be reconstructed on the client side
* the original "shard" information (SearchShardTarget) is only rendered if the
  "explanation" is also set, also we loose the indexUUID of the contained
  ShardId because we don't write it out. As a replacement we can use
  ClusterState.UNKNOWN_UUID on the receiving side
2017-01-10 14:00:04 +01:00
Clinton Gormley cb7952e71d Docs: Parent field is no longer indexed and should use parent_id instead of term query
Closes #22517
2017-01-10 13:48:07 +01:00
A.N 6e1df967be Update delete-by-query.asciidoc (#22519)
Fixed some grammatical mistakes
2017-01-10 13:38:34 +01:00
Yannick Welsch 9fc1a735cc Keep NodeConnectionsService in sync with current nodes in the cluster state (#22509)
The NodeConnectionsService currently determines which nodes to connect to / disconnect from by inspecting cluster state changes and connecting to added nodes / disconnecting from removed nodes. When a master steps down (for example due to another master-eligible node shutting down which brings the number of master-eligible nodes below minimum_master_master), and the connection to other existing nodes was dropped while pinging, however, the connection to these nodes is not re-established while publishing the first cluster state that establishes the node as master.
This commit changes the NodeConnectionsService connect / disconnect logic to always rely on the state that is to be / was published, looking not only at the added / removed nodes, but validating that exactly all nodes that are currently registered in NodeConnectionsService are connected (corresponds to a NOOP if the node is already connected).
2017-01-10 13:29:49 +01:00
Boaz Leskes 61fe6b32e4 Add a high level description of ES's data replication model (#22327)
This commit adds a document describing our data replication model in high level terms. The goal is give people basic insight into how things work in order to better understand how read and writes interact, both during normal operations and under failures.
2017-01-10 13:17:48 +01:00
Tanguy Leroux b9061c1cc9 [TESTS] Fix GetResultTests.testGetSourceAsBytes() (#22524)
The document in the randomized GetResult can exist with no source (like if the _source was disabled in mappings), that's why the test should not always expect a non null source when the doc exists.
2017-01-10 13:12:00 +01:00
Jim Ferenczi 433c822d4f Promote longs to doubles when a terms agg mixes decimal and non-decimal numbers (#22449)
* Promote longs to doubles when a terms agg mixes decimal and non-decimal number

This change makes the terms aggregation work when the buckets coming from different indices are a mix of decimal numbers and non-decimal numbers. In this case non-decimal number (longs) are promoted to decimal (double) which can result in a loss of precision for big numbers.

Fixes #22232
2017-01-10 11:50:56 +01:00
Martijn van Groningen cb2333dacd percolator: remove deprecated percolate and mpercolate apis 2017-01-10 11:18:27 +01:00
henakamaMSFT 72ec3d2661 Fixing the error message to report the number of docs correctly for each node (#22515)
There is a bug in the error message that is thrown if the number of docs differs between the source and target shards when recovering a shard with a syncId. The source and target doc counts are swapped around.

Closes #21893
2017-01-10 09:57:57 +01:00
Tanguy Leroux dc81afd4f8 [TESTS] Create vagrant up task for all boxes (#22496)
Currently, such tasks are only created for default boxes (centos-7, ubuntu-1404) and not all boxes and this can be misleading for developers who want to debug testing scripts on non-default boxes.
2017-01-10 09:45:33 +01:00
Nik Everett 3fb9254b95 Replace Suggesters with namedObject (#22491)
Removes another parser registery type thing in favor of
`XContentParser#namedObject`.
2017-01-09 16:51:08 -05:00
Nik Everett 057194f9ab Fix test under windows
Silly `\r`.
2017-01-09 16:29:59 -05:00
Nik Everett d623df9372 Remove AggregatorParsers
Meant to remove it with e3f77b4795.
It is no longer used.
2017-01-09 15:13:49 -05:00
Nik Everett e3f77b4795 Replace AggregatorParsers with namedObject (#22397)
Removes `AggregatorParsers`, replacing all of its functionality with
`XContentParser#namedObject`.

This is the third bit of payoff from #22003, one less thing to pass
around the entire application.
2017-01-09 13:59:38 -05:00
Nik Everett f75ef7adfd Use namedObject to parse AllocationCommands (#22489)
This removes `AllocationCommandRegistry` entirely and replaces
it with `XContentParser#namedObject`, removing another class from
guice.
2017-01-09 12:26:57 -05:00
Nik Everett fc1f7c2147 Remove content-type detection from reindex-from-remote (#22504)
If the remote doesn't return a content type then reindex
tried to guess the content-type. This didn't work most
of the time and produced a rather useless error message.
Given that Elasticsearch always returns the content-type
we are dropping content-type detection in favor of just
failing the request if the remote didn't return a content-type.

Closes #22329
2017-01-09 11:50:20 -05:00
Boaz Leskes be0c461b50 UnicastZenPingTests didn't properly wait for pinging rounds to be closed
The test ping and waited for the ping results to be returned but since we first return the result and then close temporary connections, assertions are tripped that expects all connections to close by end of test .

Closes #22497
2017-01-09 17:42:28 +01:00
Martijn van Groningen cb8f8fc9ad Don't ignore `ignore_unmapped` options on nested, has_child and has_parent queries if the wrapped query gets rewritten. 2017-01-09 17:17:34 +01:00
Jay Modi b04f8fe159 prevent NPE when calling GetResponse#getSourceAsBytesRef
This commit checks for a null BytesReference as the value for `source`
in GetResult#sourceRef and simply returns null. Previously this would
have resulted in a NPE. While this does seem internal at first glance, it can affect
user code as a GetResponse could trigger this when the document is missing.

Additionally, the CompressorFactory#uncompressIfNeeded now requires a
non-null argument.
2017-01-09 09:19:05 -05:00
Nik Everett f4884e0726 Replace SearchExtRegistry with namedObject (#22492)
This is one of the last things in `SearchRequestParsers`.
2017-01-09 08:35:54 -05:00
Karel Minarik 4f4b76cd41 [TEST] Fixed the incorrect indentation for the `skip` clauses in the REST tests
This patch fixes the incorrect indentation in the REST tests, which makes tests in language runners (eg. Ruby, Python) to fail, since the skip clause is parsed as an empty value. Tha Java YAML parser is smarter/lenient about whitespace, so it doesn't catch this.
2017-01-08 14:21:02 +01:00
Yannick Welsch 8741691511 Fix primary relocation for shadow replicas (#22474)
The recovery process started during primary relocation of shadow replicas accesses the engine on the source shard after it's been closed, which results in the source shard failing itself.
2017-01-08 12:18:52 +01:00
Ali Beyad b8934945b4 Fixes 5.3.0-SNAPSHOT typo 2017-01-06 20:37:35 -05:00
Ali Beyad 056adbd8cc Updates backwards compatibility 5.0 tests to pull
the latest 5.x version - 5.3.0-SNAPSHOT
2017-01-06 20:36:38 -05:00
Nik Everett 12923ef896 Close and flush refresh listeners on shard close
Right now closing a shard looks like it strands refresh listeners,
causing tests like
`delete/50_refresh/refresh=wait_for waits until changes are visible in search`
to fail. Here is a build that fails:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+multi_cluster_search+multijob-darwin-compatibility/4/console

This attempts to fix the problem by implements `Closeable` on
`RefreshListeners` and rejecting listeners when closed. More importantly
the act of closing the instance flushes all pending listeners
so we shouldn't have any stranded listeners on close.

Because it was needed for testing, this also adds the number of
pending listeners to the `CommonStats` object and all API to which
that flows: `_cat/nodes`, `_cat/indices`, `_cat/shards`, and
`_nodes/stats`.
2017-01-06 20:03:32 -05:00
Ali Beyad b0c009ae76 Gracefully handles pre 2.x compressed snapshots
In pre 2.x versions, if the repository was set to compress snapshots,
then snapshots would be compressed with the LZF algorithm.  In 5.x,
Elasticsearch no longer supports the LZF compression algorithm.  This
presents an issue when retrieving snapshots in a repository or upgrading
repository data to the 5.x version, because Elasticsearch throws an
exception when it tries to read the snapshot metadata because it was
compressed using LZF.

This commit gracefully handles the situation by introducing a new
incompatible-snapshots blob to the repository.  For any pre-2.x snapshot
that cannot be read, that snapshot is removed from the list of active
snapshots, because the snapshot could not be restored anyway.  Instead,
the snapshot is recorded in the incompatible-snapshots blob.  When
listing snapshots, both active snapshots and incompatible snapshots will
be listed, with incompatible snapshots showing a `INCOMPATIBLE` state.
Any attempt to restore an incompatible snapshot will result in an
exception.
2017-01-06 17:52:10 -05:00
javanna ded694fc83 Make StatusToXContent extend ToXContentObject and rename it to StatusToXContentObject
This also allows to make RestToXContentListener require ToXContentObject rather than ToXContent
2017-01-06 23:31:48 +01:00
javanna 4e49860f68 Make PercolateResponse a ToXContentObject 2017-01-06 23:31:48 +01:00
javanna 8edf59c9e7 Make DocWriteResponse a ToXContentObject
This involved changing also index, delete, update so that bulk can print them out in its own format.
2017-01-06 23:31:48 +01:00
javanna d5510701a0 Make SearchResponse a ToXContentObject 2017-01-06 23:31:48 +01:00
javanna 3393d1b409 Make TermVectorsResponse a ToXContentObject 2017-01-06 23:31:48 +01:00
javanna 45d4938fcc Migrate some more responses to ToXContentObject 2017-01-06 23:31:48 +01:00
javanna f4aab0138d introduce ToXContentObject interface
`ToXContentObject` extends `ToXContent` without adding new methods to it, while allowing to mark classes that output complete xcontent objects to distinguish them from classes that require starting and ending an anonymous object externally.

Ideally ToXContent would be renamed to ToXContentFragment, but that would be a huge change in our codebase, hence we simply document the fact that toXContent outputs fragments with no guarantees that the output is valid per se without an external ancestor.

Relates to #16347
2017-01-06 23:31:48 +01:00
animageofmine e3546d59c4 Add support for ca-central-1 region to EC2 and S3 plugins
Closes #22458 #22454
2017-01-06 16:27:08 -06:00
Ryan Ernst 5adb3f992c Merge pull request #22443 from rjernst/disable_assertions
Use tests.asserts flag to allow disabling assertions
2017-01-06 12:03:45 -08:00
Ryan Ernst d235e489e6 Merge pull request #22335 from rjernst/keystore
Settings: Add infrastructure for elasticsearch keystore
2017-01-06 11:48:17 -08:00
Ryan Ernst cd6e3f4cea Merge branch 'master' into keystore 2017-01-06 09:32:08 -08:00
Ryan Ernst 42ebfe7bdb fix NPE 2017-01-06 09:11:07 -08:00
Tim B b9c2c2f6f0 Move IfConfig.logIfNecessary call into bootstrap (#22455)
This is related to #22116. A logIfNecessary() call makes a call to
NetworkInterface.getInterfaceAddresses() requiring SocketPermission
connect privileges. By moving this to bootstrap the logging call can be
made before installing the SecurityManager.
2017-01-06 11:10:53 -06:00