Commit Graph

25518 Commits

Author SHA1 Message Date
Jared Carey 317866894e Fix systemd override example in configuring docs
When overriding a systemd configuration via a drop-in file, the
[Service] header is required. This commit adds this to an example
drop-in override in the configuring docs.

Relates #22038
2016-12-07 19:41:59 -05:00
Ali Beyad e6e7bab58c Prepares allocator decision objects for use with the allocation explain API (#21691)
This commit enhances the allocator decision result objects (namely,
AllocateUnassignedDecision, MoveDecision, and RebalanceDecision)
to enable them to be used directly by the cluster allocation explain API. In
particular, this commit does the following:

- Adds serialization and toXContent methods to the response objects,
which will form the explain API responses.
- Moves the calculation of the final explanation to the response
object itself, removing it from the responsibility of the allocators.
- Adds shard store information to the NodeAllocationResult, so that
store information is available for each node, when explaining a
shard allocation by the PrimaryShardAllocator or the ReplicaShardAllocator.
- Removes RebalanceDecision in favor of using MoveDecision for both
moving and rebalancing shards.
- Removes NodeRebalanceResult in favor of using NodeAllocationResult.
- Changes the notion of weight ranking to be relative to the current node,
instead of an absolute weight that doesn't convey any added value to the
API user and can be confusing.
- Introduces a new enum AllocationDecision to convey the decision type,
which enables conveying unassigned, moving, and rebalancing scenarios
with more detail as opposed to just Decision.Type and AllocationStatus.
2016-12-07 17:37:51 -05:00
Guilherme 9fbfe540d5 add Lua client (#22028)
Add entry for elasticsearch-lua (https://github.com/DhavalKapil/elasticsearch-lua)
2016-12-07 11:24:28 -07:00
Ali Beyad 05f64c550a [TEST] fixes line length issue in BulkRequestModifierTests 2016-12-07 13:11:55 -05:00
Thibault Pierre e494d6a94e Fix wrong link (#22019) 2016-12-07 17:58:46 +01:00
Ryan Ernst f02a2b6546 Ingest: Moved ingest invocation into index/bulk actions (#22015)
* Ingest: Moved ingest invocation into index/bulk actions

Ingest was originally setup as a plugin, and in order to hook into the
index and bulk actions, action filters were used. However, ingest was
later moved into core, but the action filters were never removed. This
change moves the execution of ingest into the index and bulk actions.

* Address PR comments

* Remove forwarder direct dependency on ClusterService
2016-12-07 08:43:26 -08:00
Colin Goodheart-Smithe 8006b105f3 Update order examples to use max instead of avg (#22032)
The use of the avg aggregation for sorting the terms aggregation is not encouraged since it has unbounded error. This changes the examples to use the max aggregation which does not suffer the same issues
2016-12-07 16:00:24 +00:00
Christoph Büscher 7454a9647b Add fromXContent to HighlightField
This adds a fromXContent method and unit test to the HighlightField class so we
can parse it as part of a serch response. This is part of the preparation for
parsing search responses on the client side.
2016-12-07 16:32:44 +01:00
David Pilato 8923b36780 Merge pull request #21956 from alexshadow007/aws_read_timeout
Add setting to set read timeout for EC2 discovery and S3 repository plugins
2016-12-07 16:00:48 +01:00
Yannick Welsch c87cc15d49 Add toString() for TransportReplicationAction.ConcreteShardRequest 2016-12-07 15:58:22 +01:00
Christoph Büscher 31a1c2e240 Remove redundant source setters from IndexRequestBuilder 2016-12-07 15:20:36 +01:00
Nikhil Patel b5e3d351d9 Fix typos in threads docs
This commit fixes a typo in the threads docs where the past tense form
of a verb was used when current tense is needed.

Relates #22016
2016-12-07 08:22:49 -05:00
Lucas Bremgartner 0086b99797 [Docs] Correct setting name in snapshot/restore documentation (#22023)
There is no setting include_cluster_state for snapshot restore. The correct name for this setting is include_global_state.
2016-12-07 14:12:10 +01:00
Yannick Welsch 9630b1a6e7 Promote shadow replica to primary when initializing primary fails (#22021)
Failing an initializing primary when shadow replicas are enabled for the index can leave the primary unassigned with replicas being active. Instead, a replica should be promoted to primary, which is fixed by this commit.
2016-12-07 13:59:43 +01:00
Yannick Welsch 13e1a6fd40 Trim in-sync allocations set only when it grows (#21976)
This commit makes two changes to how the in-sync allocations set is updated:

- the set is only trimmed when it grows. This prevents trimming too eagerly when the number of replicas was decreased while shards were unassigned.
- the allocation id of an active primary that failed is only removed from the in-sync set if another replica gets promoted to primary. This prevents the situation where the only available shard copy in the cluster gets removed the in-sync set.

Closes #21719
2016-12-07 10:59:11 +01:00
Adrien Grand c746854e03 Pre-built analysis factories do not implement MultiTermAware correctly. (#21981)
We had tests for the regular factories, but not for the pre-built ones, that
ship by default without requiring users to define them in the analysis settings.
2016-12-07 10:32:25 +01:00
Adrien Grand 33b8d7a19d Expose `ip` fields as strings in scripts. (#21997)
Currently we expose the internal representation that we use for ip addresses,
which are the ipv6 bytes. However, this is not really usable, exposes internal
implementation details and also does not work fine with other APIs that expect
that the values can be `toString`'d.

Closes #21977
2016-12-07 10:32:11 +01:00
Nik Everett ef83dbfbe6 Reindex: Better error message for pipeline in wrong place (#21985)
`_update_by_query` supports specifying the `pipeline` to process the
documents as a url parameter but `_reindex` doesn't. It doesn't because
everything about the `_reindex` request that has to do with writing
the documents is grouped under the `dest` object in the request body.
This changes the response parameter from
`request [_reindex] contains unrecognized parameter: [pipeline]` to
`_reindex doesn't support [pipeline] as a query parmaeter. Specify it in the [dest] object instead.`
2016-12-06 14:55:46 -05:00
Boaz Leskes 4519bdfeb0 InternalTestCluster shouldn't auto heal an active disruption when a new one is set
Instead people should explicitly clear the existing one so it's clear what's going on.
2016-12-06 19:58:11 +01:00
Alexander Kazakov 0a03a62ab6 Using ClientConfiguration.DEFAULT_SOCKET_TIMEOUT as default value for read timeout 2016-12-06 21:13:28 +03:00
shaie 6da44c8164 Fix _termvectors with preference to not hit NPE (#21959)
When you submit a _termvectors request for an artificial document and
specify the 'preference' parameter to send the request to a particular
shard, the request sometimes hits NPE. Fix this case by ignoring the
auto-generated artificial document ID and pick a shard per the
preference parameter, or a random shard.

This closes #21928
2016-12-06 17:29:09 +01:00
Jim Ferenczi b42ca6bcc9 Include unindexed field in FieldStats response (#21821)
* Include unindexed field in FieldStats response

This change adds non-searchable fields to the FieldStats response. These fields do not have min/max informations but they can be aggregatable. Fields that are only stored in _source (store:no, index:no, doc_values:no) will still be missing since they do not have any useful information to show. Indices and clients must be at least on V_5_2_0 to see this change.
2016-12-06 13:32:57 +01:00
Boaz Leskes a7050b2d56 Remove `InternalTestCluster.startNode(s)Async` (#21846)
Since the removal of local discovery of #https://github.com/elastic/elasticsearch/pull/20960 we rely on minimum master nodes to be set in our test cluster. The settings is automatically managed by the cluster (by default) but current management doesn't work with concurrent single node async starting. On the other hand, with `MockZenPing` and the `discovery.initial_state_timeout` set to `0s` node starting and joining is very fast making async starting an unneeded complexity. Test that still need async starting could, in theory, still do so themselves via background threads.

Note that this change also removes the usage of `INITIAL_STATE_TIMEOUT_SETTINGS` as the starting of nodes is done concurrently (but building them is sequential)
2016-12-06 12:06:15 +01:00
Daniel Mitterdorfer a02bc8ed1c Document thread-safety for ingest processors
With this commit we document that ingest processors need to be
thread-safe. Previously this could be inferred from reading the
source code but we got several user questions about this so
it is stated explicitly in the Javadocs of Processor now.
2016-12-06 10:07:51 +01:00
Adrien Grand 26cbda41ea AsciiFoldingFilter's multi-term component should never preserve the original token. (#21982)
This ports the fix of https://issues.apache.org/jira/browse/LUCENE-7536 to
Elasticsearch's ASCIIFoldingTokenFilterFactory.
2016-12-06 10:01:04 +01:00
Ryan Ernst c8f241f284 Plugins: Remove response action filters (#21950)
Action filters currently have the ability to filter both the request and
response. But the response side was not actually used. This change
removes support for filtering responses with action filters.
2016-12-05 16:14:04 -08:00
Alexander Kazakov 1491e2dec9 Remove default value for read_timeout setting
Fix tests and docs
2016-12-05 21:29:17 +03:00
Nik Everett 2087234d74 Timeout improvements for rest client and reindex (#21741)
Changes the default socket and connection timeouts for the rest
client from 10 seconds to the more generous 30 seconds.

Defaults reindex-from-remote to those timeouts and make the
timeouts configurable like so:
```
POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "socket_timeout": "1m",
      "connect_timeout": "10s"
    },
    "index": "source",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}
```

Closes #21707
2016-12-05 10:54:51 -05:00
Jim Ferenczi 03a0a0aebb Undeprecate GetResponse#getFields and GetResponse#getField
These functions should not have been deprecated as they can be used to retrieve stored and doc-value field.
2016-12-05 15:31:53 +01:00
David Pilato 6bddd426cc Merge pull request #21961 from alexshadow007/us-east-2
Add us-east-2 AWS region
2016-12-05 11:13:33 +01:00
Ali Beyad ff9959c865 Don't output null source node in RecoveryFailedException (#21963)
The RecoveryFailedException's output prints the source and
target nodes for the recovery.  However, sometimes there is
no source node for the recovery, only a target node (such as
when recovering a primary shard from disk).  In this case,
we don't want to display the source node.  This commit fixes
this by displaying "Recovery failed on target node.." instead
of "Recovery failed from null to target node" which is what the
output currently displays.
2016-12-04 15:23:35 -05:00
Alexander Kazakov 23550f277b Add us-east-2 AWS region 2016-12-04 20:02:05 +03:00
Jason Tedor 60aa14f48e Increase test logging on test simple pings test
This commit increases the test logging on the unicast zeng ping test of
simple pings to gather more info for chasing a race condition that is
happening in this test.
2016-12-04 08:06:01 -05:00
Jason Tedor 040c05df36 Increase timeouts in UnicastZenPingTests
Sadly, the timeouts here need to be increased to reduce the likelihood
of spurious test failures (test hosts under load are especially prone to
this). This does slow down this test suite a bit, but it's still not as
slow as it was before this endeavor of lowering these timeouts started.
2016-12-03 22:19:55 -05:00
Jason Tedor 2c8229fcaf Cleanup unicast zen ping unknown hosts cached test
This commit cleans up the unicast zen ping unknown hosts cached test:
 - send pings from the same node to more clearly indicate DNS lookups
   are not cached (within the same UnicastZenPing instance)
 - increase ping and wait timeout to 500ms to address race conditions
   (on a test host under load, the timeout was too short for the
   connect/handshake/ping cycle to complete)
2016-12-03 22:00:30 -05:00
Alexander Kazakov 5695eaf19e Add setting to set read timeout for EC2 discovery and S3 repository plugins 2016-12-04 01:58:53 +03:00
Jason Tedor 460e787049 Increase resolve timeout in unknown hosts test
The port limit test is a simple test that fakes that resolving an
address with a port range results the correct address collection.  This
test is subject to a race condition where the timeout on the resolve
request can fire before the resolve code finishes executing (this race
is exceptionally rare, because there are not actually any DNS lookups
being done here since we are just resolving addresses). This commit
increases the timeout here to significantly reduce the chance of a
losing race causing a spurious test failure. This increased timeout
should not increase the runtime of the test, just make failures less
likely.
2016-12-03 09:02:44 -05:00
Jason Tedor f5cbc36896 Increase resolve timeout in unknown hosts test
The unknown hosts test is a simple test that fakes that resolving an
address results in an unknown host exception. The main purpose of this
test is to ensure that we log (and do not silently drop) when a host
fails to resolve. This test is subject to a race condition where the
timeout on the resolve request can fire before the resolve code finishes
executing (this race is exceptionally rare, because there are not
actually any DNS lookups being done here, just a mock resolve
implementation that throws an exception and that's where losing the race
can arise). This commit increases the timeout here to significantly
reduce the chance of a losing race causing a spurious test failure. This
increased timeout should not increase the runtime of the test, just make
failures less likely.
2016-12-03 08:46:24 -05:00
Igor Motov c391b3fff6 Add proper descriptions to reindex, update-by-query and delete-by-query tasks.
Related to #21768
2016-12-02 21:46:38 -05:00
Igor Motov bb9317253a Add descriptions to create snapshot and restore snapshot tasks.
Related to #21768
2016-12-02 21:13:54 -05:00
Jason Tedor c6efd4eb42 Rename method in InternalEngine
This commit renames InternalEngine#loadSeqNoStatsLucene to
InternalEngine#loadSeqNoStatsFromLucene to make this name consistent
with the method InternalEngine#loadSeqNoStatsFromLuceneAndTranslog.
2016-12-02 20:46:26 -05:00
Ryan Ernst 34eb23e98e Plugins: Replace Rest filters with RestHandler wrapper (#21905)
* Plugins: Replace Rest filters with RestHandler wrapper

RestFilters are a complex way of allowing plugins to add extra code
before rest actions are executed. This change removes rest filters, and
replaces with a wrapper which a single plugin may provide.
2016-12-02 14:54:51 -08:00
Jack Conradson 0ecdef026d Test fix for def equals test in Painless. (#21945)
Closes #21801
2016-12-02 14:41:13 -08:00
Jason Tedor b0e8696143 Clarify global checkpoint recovery
Today when starting a new engine, we read the global checkpoint from the
translog only if we are opening an existing translog. This commit
clarifies this situation by distinguishing the three cases of engine
creation in the constructor leading to clearer code.

Relates #21934
2016-12-02 15:00:16 -05:00
Jason Tedor 0afef53a17 Add system call filter bootstrap check
Today if system call filters fail to install on startup, we log a
message but otherwise march on. This might leave users without system
call filters installed not knowing that they have implicitly accepted
the additional risk. We should not be lenient like this, instead clearly
informing the user that they have to either fix their configuration or
accept the risk of not having system call filters installed. This commit
adds a bootstrap check that if system call filters are enabled, they
must successfully install.

Relates #21940
2016-12-02 14:27:54 -05:00
Nik Everett 0c724b1878 Keep context during reindex's retries (#21941)
* Keep context during reindex's retries

This fixes reindex and friend's retries to keep the context.

* Docs
2016-12-02 13:48:51 -05:00
Jay Modi 429e517476 Do not lose host information when pinging
In #21828, serialization of the host string was added to preserve this information when
a TransportAddress gets serialized. However, there is still a case where this did not always
work. In UnicastZenPings, DiscoveryNode instances are created for the ping hosts with the
minimum compatibility version, which is currently less than the version required to preserve
the host information. This means that when a node is received from a PingResponse that the
host information is no longer set correctly on the InetSocketAddress contained in the
DiscoveryNode.

This commit adds a workaround for this situation by allowing the host string to be passed
into the TransportAddress constructor that takes a StreamInput and using that as the host
for the InetAddress that is created during deserialization.
2016-12-02 12:21:53 -05:00
Allen Torres 887fbb6387 Update lowercase-tokenizer.asciidoc (#21896)
Fixed typo
2016-12-02 10:49:51 -05:00
nelsonSchwarz ee35bba6b6 elasticsearch.R is abandoned (#21897)
README file reads: (This project is no longer maintained, if you would like to take over feel free to send me an email)
2016-12-02 10:39:20 -05:00
Ke Li 7cc9833606 Avoid some redundant unboxing and object creation (#21909) 2016-12-02 16:11:41 +01:00