Commit Graph

5359 Commits

Author SHA1 Message Date
Christoph Büscher 757ccf00b2 Enforce MatchQueryBuilder#maxExpansions() to be strictly positive 2016-05-19 16:59:37 +02:00
Jeff Evans 3cf4214255 Add better error message when analyzer created without tokenizer or analyzer type (#18455)
Closes #15492
2016-05-19 15:47:07 +02:00
Ali Beyad fc6df23fea Rename AggregatorBuilder and all of its subclasses to
AggregationBuilder, in keeping consistent with the Java APIs.

Closes #18377
Closes #18367
2016-05-19 09:25:29 -04:00
Tanguy Leroux 35d3bdab84 Add Google Cloud Storage repository plugin
Closes #12880
2016-05-19 13:26:23 +02:00
Martijn van Groningen e2691d7e5c test: Don't generate a value of 0, because FuzzyQuery constructor does't allow that 2016-05-19 13:08:30 +02:00
Martijn van Groningen 050145f61b parent/child: Allow adding additional child types that point to an existing parent type
From 2.0 adding child types to existing types was forbidden because the`_parent` field stores the join between parent and child at index time.
This is to protect from the fact that types that weren't a parent before become a parent while previously indexed documents would not have a join field.
 This would break the parent/child queries.

The restriction was a bit too strict in the sense that also if a type was a parent type the restriction would forbid adding child types that point to a parent type (so child points already point to it).
This change make sure that the restriction only applies if that type isn't a parent type already.

Closes #17956
2016-05-19 11:05:17 +02:00
Simon Willnauer d77c299cb9 Register `indices.query.bool.max_clause_count` setting (#18341)
* Register `indices.query.bool.max_clause_count` setting

This commit registers `indices.query.bool.max_clause_count` as a node
level setting and removes support for its synonym setting
`index.query.bool.max_clause_count`.

Closes #18336
2016-05-19 10:42:35 +02:00
Simon Willnauer 2b972f1f75 FSync translog outside of the writers global lock (#18360)
FSync translog outside of the writers global lock

Today we aquire a write global lock that blocks all modification to the
translog file while we fsync / checkpoint the file. Yet, we don't necessarily
needt to block concurrent operations here. This can lead to a lot of blocked
threads if the machine has high concurrency (lot os CPUs) but uses slow disks
(spinning disks) which is absolutely unnecessary. We just need to protect from
fsyncing / checkpointing concurrently but we can fill buffers and write to the
underlying file in a concurrent fashion.

This change introduces an additional lock that we hold while fsyncing but moves
the checkpointing code outside of the writers global lock.
2016-05-19 09:40:10 +02:00
Simon Willnauer 9a9301f7d8 Remove dead BloomFilter code
We don't use this class for a quite a while. lets trash it.
2016-05-18 23:00:57 +02:00
Clinton Gormley cec9a94b96 Added version 2.3.3 with bwc indices 2016-05-18 17:33:21 +02:00
Christoph Büscher 7c665a010b Fix TimeZoneRounding#nextRoundingValue for hour, minute and second units
Currently rounding intervals obtained by nextRoundingValue() for hour, minute and
second units can include an extra hour when happening at DST transitions that add
an extra hour (eg CEST -> CET). This changes the rounding logic for time units
smaller or equal to an hour to fix this.

Closes #18326
2016-05-18 17:29:02 +02:00
markharwood a846ff93e9 Aggregations fix: support include/exclude strings formatted for IP and date fields in terms and significant_terms aggregations.
Closes #17705
2016-05-18 16:21:55 +01:00
Tanguy Leroux 27b65e90ca Merge pull request #18443 from tlrx/fix-18433
Add missing builder.endObject() in FsInfo
2016-05-18 15:40:55 +02:00
Jason Tedor cad0608cdb Add GC overhead logging
This commit adds simple GC overhead logging. This logging captures
intervals where the JVM is spending a lot of time performing GC but it
is not necessarily the case that each GC is large. For a start, this
logging is simple and does not attempt to incorporate whether or not the
collections were efficient (that is, we are only capturing that a lot of
GC is happening, not that a lot of useless GC is happening).

Relates #18419
2016-05-18 09:31:28 -04:00
Daniel Mitterdorfer 3954306af2 Merge pull request #18432 from danielmitterdorfer/fix-circuit-breaker-it
Clear all caches after testing parent breaker
2016-05-18 15:23:15 +02:00
Tanguy Leroux d7a31c8cf7 Add missing builder.endObject() in FsInfo
closes #18433
2016-05-18 15:19:30 +02:00
Christoph Büscher 808ef6cec7 Fix parsing single `rescore` element in SearchSourceBuilder
We are currently only parsing the array-syntax for the rescore part
in SearchSourceBuilder ("rescore" : [ {...}, {...} ]) . We also need
to support "rescore" : {...}

Closes #18439
2016-05-18 15:08:28 +02:00
Clinton Gormley c03dd8a290 Make the index-too-old exception more explicit (#18438)
Closes #18418
2016-05-18 13:33:25 +02:00
Daniel Mitterdorfer de3e7d161f Add tests for null precondition check in BulkRequest
Relates #18347

Checked with @javanna
2016-05-18 12:10:13 +02:00
Yannick Welsch 6dacac49b3 Simplify recovery logic in IndicesClusterStateService (#18405)
- Moves recovery logic into IndexShard
- Simplifies logic to cancel peer recovery of shard where recovery source node changed
- Ensures routing entry is set on initialization of IndexShard
2016-05-18 10:51:57 +02:00
Daniel Mitterdorfer c13df3b6c5 Clear all caches after testing parent breaker
With this commit we clear all caches after testing the parent circuit breaker.
This is necessary as caches hold on to circuit breakers internally. Additionally,
due to usage of CircuitBreaker#addWithoutBreaking() in caches, it's even possible
to go above the limit. As a consequence, all subsequent requests fall victim to
the limit.

Hence, right after the parent circuit breaker tripped, we clear all caches to
reduce these circuit breakers to 0 again. We also exclude the clear caches
transport request from limit check in order to ensure it will succeed. As this is
typically a very small and low-volume request, it is deemed ok to exclude it.

Closes #18325
2016-05-18 09:31:35 +02:00
Jason Tedor ecce53f0df Add I/O statistics on Linux
This commit adds a variety of real disk metrics for the block devices
that back Elasticsearch data paths. A collection of statistics are read
from /proc/diskstats and are used to report the raw metrics for
operations and read/write bytes.

Relates #15915
2016-05-17 16:16:39 -04:00
Jason Tedor 584be0b3f8 Refactor JvmGcMonitorService for testing
This commit refactors the JvmGcMonitorService so that it can be
tested. In particular, hooks are added to verify that the
JvmMonitorService correctly observes slow GC events, and that the
JvmGcMonitorService logs the correct messages.

Relates #18378
2016-05-17 13:05:36 -04:00
Yannick Welsch 9ba554dfd2 Expose previous cluster state only in RoutingAllocation (#18390)
Instead of re-exposing index metadata and blocks in RoutingNodes (which is part of the cluster state before rerouting), expose it as part of the RoutingAllocation which is known to be only temporarily used during reroute.
2016-05-17 19:02:28 +02:00
polyfractal c755a77022 [TEST] Use a reproducible source of randomness in shuffle 2016-05-17 12:55:07 -04:00
Zachary Tong 7c46b57ff2 Add a Sort ingest processor
Sorts an array of values in ascending or descending order. If all elements are numerics, they will be sorted numerically. If values are strings, or mixtures of strings/numbers, the elements will be sorted lexicographically.
2016-05-17 12:06:48 -04:00
Colin Goodheart-Smithe 8c9ca8b518 Moves query profiler classes into their own package
The change also renames fields and methods in the Profilers class.

Note that I had to make ProfileResult a public class (it was package private before) because now classes that call it are in a different package.
2016-05-17 14:20:05 +01:00
Ali Beyad 3764789d96 Removed unused AllocationService member in
TransportClusterAllocationExplainAction

Closes #18381
2016-05-16 18:41:36 -04:00
Robert Muir 8d4c1befe5 Merge pull request #18364 from rmuir/nukeRunAsFloat
Remove LeafSearchScript.runAsFloat(): Nothing calls it.
2016-05-16 17:08:25 -04:00
Adrien Grand 864ed04059 Lessen leniency of the query dsl. #18276
This change does the following:
 - Queries that are currently unsupported such as prefix queries on numeric
   fields or term queries on geo fields now throw an error rather than returning
   a query that does not match anything.
 - Fuzzy queries on numeric, date and ip fields are now unsupported: they used
   to create range queries, we now expect users to use range queries directly.
   Fuzzy, regexp and prefix queries are now only supported on text/keyword
   fields (including `_all`).
 - The `_uid` and `_id` fields do not support prefix or range queries anymore as
   it would prevent us to store them more efficiently in the future, eg. by
   using a binary encoding.

Note that it is still possible to ignore these errors by using the `lenient`
option of the `match` or `query_string` queries.
2016-05-16 17:37:00 +02:00
Colin Goodheart-Smithe e37e8af5e2 Refactor of query profile classes to make way for other profile implementations 2016-05-16 16:15:50 +01:00
Colin Goodheart-Smithe 6eda9f5df6 more tests following review 2016-05-16 09:07:22 +01:00
Colin Goodheart-Smithe 0c449fee4a small fix following rebase on master 2016-05-16 09:07:22 +01:00
Colin Goodheart-Smithe 66d0bdab0c review comments 2016-05-16 09:07:22 +01:00
Colin Goodheart-Smithe ab3121c871 Adds a methods to find (and dynamically create) the mappers for the parents of a field with dots in the field name 2016-05-16 09:07:22 +01:00
Robert Muir 8edf213492 Remove LeafSearchScript.runAsFloat(): Nothing calls it. 2016-05-15 22:59:28 -04:00
Michael McCandless 0d570352dd Merge pull request #18355 from mikemccand/iterables_flatten
Iterables.flatten should not pre-cache the first iterator
2016-05-15 10:21:35 -04:00
Mike McCandless 8d7db7fd7a remove whitespace 2016-05-14 18:50:10 -04:00
Mike McCandless ded8b400b0 Fix concurrency bug in IMC that could lead to negative total indexing bytes 2016-05-14 18:47:26 -04:00
Mike McCandless 48dca45564 leave Iterables.flatten pre-caching the outer Iterable 2016-05-14 17:10:17 -04:00
Mike McCandless 53c2f8b4b6 improve javadocs 2016-05-14 13:46:34 -04:00
Mike McCandless cf2af8961b Iterables.flatten should not pre-cache the first iterator 2016-05-14 13:39:48 -04:00
Ali Beyad d3d57da89f Removes unused methods in the o/e/common/Strings class.
Closes #18346
2016-05-14 08:08:30 -04:00
Daniel Mitterdorfer 009cf434a2 Merge pull request #18347 from danielmitterdorfer/bulk-req-precondition-check
Add not-null precondition check in BulkRequest
2016-05-14 11:25:02 +02:00
Daniel Mitterdorfer a2381640da Add not-null precondition check in BulkRequest
With this commit we add a precondition check to BulkRequest so
we fail early if users pass `null` for the request object.

For a more detailed discussion, see #12038.
This supersedes #12038.

Relates #12038.
2016-05-14 09:59:53 +02:00
Robert Muir 2028691e66 painless: improve exception stacktraces
closes #18319
2016-05-13 15:40:45 -04:00
Lee Hinman 864ba8dac1 Merge remote-tracking branch 'dakrone/there-can-be-only-one2' 2016-05-13 10:28:41 -06:00
Adrien Grand b4dec0ddbe Remove dead code. 2016-05-13 18:27:12 +02:00
Jason Tedor 81898a2e3e Avoid race while retiring executors
Today, a race condition exists when retiring executors. Namely, if an
executor is retired and then the thread pool is terminated, the retiring
of the executor and the termination of the thread pool can race to
remove the retired executor from the queue of retired executors. More
precisely, when the executor is initially retired, it is placed on a
queue of retired executors, and then removed when it is successfully
shutdown. When the pool is terminated, it will also drain the queue of
retired executors. This leads to a time-of-check-time-of-use race where
the draining can see a retired executor on the queue but that retired
executor can be removed upon successful shutdown of that executor. This
leads to the draining attempting to remove an element from the queue
when there is none. This commit addresses this race condition by instead
safely polling the queue.

Relates #18333
2016-05-13 12:26:18 -04:00
Lee Hinman 9bcdafedda Allow only a single extension for a scripting engine
Previously multiple extensions could be provided, however, this can lead
to confusion with on-disk scripts (ie, "foo.js" and "foo.javascript")
having different content. Only a single extension is now supported.

The only language currently supporting multiple extensions was the
Javascript engine ("js" and "javascript"). It now only supports the
`.js` extension.

Relates to #10598
2016-05-13 09:54:31 -06:00
Lee Hinman d5b75491dc Merge remote-tracking branch 'dakrone/remove-script-sandbox' 2016-05-13 09:50:39 -06:00
Britta Weber e7c17fc9fb [TEST] increase logger level until we know what is going on
We have an issue for it too: https://github.com/elastic/elasticsearch/issues/18121
2016-05-13 17:37:17 +02:00
Christoph Büscher a40c397c67 Don't allow `fuzziness` for `multi_match` types cross_fields, phrase and phrase_prefix
Currently `fuzziness` is not supported for the `cross_fields` type
of the `multi_match` query since it complicates the logic that
blends the term queries that cross_fields uses internally. At the
moment using this combination is silently ignored, which can lead to
confusions. Instead we should throw an exception in this case.
The same is true for phrase and phrase_prefix type.

Closes #7764
2016-05-13 17:32:14 +02:00
Jason Tedor 786a6a00d9 Add test for fixed executor rejected count
This commit adds a test that a fixed executors rejected count behaves as
expected. In particular, we test that if we consume the executor, then
stuff the executor queue, further tasks will be rejected and the
rejected stats are updated appropriately. This test also asserts that if
we resize the queue the rejected count is reset to zero.

Relates #18301
2016-05-13 11:27:12 -04:00
Lee Hinman efff3918d8 Remove support for mulitple languages per scripting engine 2016-05-13 09:24:31 -06:00
Lee Hinman a4060f7436 Remove vestiges of script engine sandboxing
This removes all the mentions of the sandbox from the script engine
services and permissions model. This means that the following settings
are no longer supported:

```yaml
script.inline: sandbox
script.stored: sandbox
```

Instead, only a `true` or `false` value can be specified.

Since this would otherwise break the default-allow parameter for
languages like expressions, painless, and mustache, all script engines
have been updated to have individual settings, for instance:

```yaml
script.engine.groovy.inline: true
```

Would enable all inline scripts for groovy. (they can still be
overridden on a per-operation basis).

Expressions, Painless, and Mustache all default to `true` for inline,
file, and stored scripts to preserve the old scripting behavior.

Resolves #17114
2016-05-13 09:24:31 -06:00
Adrien Grand 638da06c1d Add back support for `ip` range aggregations. #17859
This commit adds support for range aggregations on `ip` fields. However it will
only work on 5.x indices.

Closes #17700
2016-05-13 17:22:01 +02:00
Clinton Gormley f2797dbccb Fixed grammar in index-too-old exception (#18327) 2016-05-13 15:08:15 +02:00
Adrien Grand 61b1f4ad0b Fix xcontent rendering of ip terms aggs. #18003
Currently terms on an ip address try to put their binary representation in the
json response. With this commit, they would return a formatted ip address:

```
      "buckets": [
        {
          "key": "192.168.1.7",
          "doc_count": 1
        }
      ]
```
2016-05-13 14:59:36 +02:00
Daniel Mitterdorfer ddbfda2c68 Exclude specific transport actions from request size limit check
We add support to explicitly exclude specific transport actions
from the request size limit check.

We also exclude the following request types currently:

*MasterPingRequest
* PingRequest
2016-05-13 14:21:24 +02:00
Britta Weber d3efe37814 [TEST] mute test for now, we have an issue for it
https://github.com/elastic/elasticsearch/issues/18325
2016-05-13 14:08:17 +02:00
Britta Weber 0d5a2f25d3 [TEST] muste test, we have an issue for it
https://github.com/elastic/elasticsearch/issues/18293
2016-05-13 12:09:30 +02:00
Olivier Bourgain df43230844 Add index name and uuid in IndexAlreadyExistsException default message
Relates #18274
2016-05-12 14:32:30 -04:00
Jason Tedor 0830bd4885 Remove period in min master node check log message
As most of our log messages are not sentences and do not end with
periods, this commit removes a period from the end of the min master
node bootstrap check log message.
2016-05-12 12:48:58 -04:00
Zachary Tong 5ee5cc25cc Move AsciiFolding earlier in FingerprintAnalyzer filter chain
Rearranges the FingerprintAnalyzer so that AsciiFolding comes earlier in the chain (after lowercasing, before stop removal, for maximum deduping power)

Closes #18266
2016-05-12 09:34:15 -04:00
Robert Muir 3b66d40f7c Merge pull request #18284 from rmuir/painless_value_aggregations
_value support in painess?
2016-05-11 20:37:35 -04:00
Robert Muir 6b4e47bf96 this makes aggregations per-document _value fast (bypass hash put, hash get, etc) for painless.
but i have no clue how to test it, it seems this feature never worked via REST?

Should we drop the feature instead?
2016-05-11 15:39:00 -04:00
Ali Beyad 189341da10 CORS handling triggered whether User-Agent is a browser or not
This commit ensures that if CORS is enabled, then Origin headers are
checked regardless of whether the request came from a browser or not.
In the past, we only proceeded with CORS checks if the User-Agent was a
browser.
2016-05-11 15:30:15 -04:00
Ali Beyad fced8dac72 When CORS is enabled, permit requests from the same origin as the
request host, as the request is not a cross origin.

Relates #18256
2016-05-11 15:24:36 -04:00
Ali Beyad 5189eb41c7 Dangling indices are not imported if a tombstone for the same index
(same name and UUID) exists in the cluster state.  This resolves a
situation where if an index data folder was copied into a node's data
directory while the node is running and that index had a tombstone in
the cluster state, the index would still get imported.

Closes #18250
Closes #18249
2016-05-11 12:56:19 -04:00
Adrien Grand ce4af4be42 Remove dead code. 2016-05-11 18:38:07 +02:00
Adrien Grand 866a5459f0 Make significant terms work on fields that are indexed with points. #18031
It will keep using the caching terms enum for keyword/text fields and falls back
to IndexSearcher.count for fields that do not use the inverted index for
searching (such as numbers and ip addresses). Note that this probably means that
significant terms aggregations on these fields will be less efficient than they
used to be. It should be ok under a sampler aggregation though.

This moves tests back to the state they were in before numbers started using
points, and also adds a new test that significant terms aggs fail if a field is
not indexed.

In the long term, we might want to follow the approach that Robert initially
proposed that consists in collecting all documents from the background filter in
order to compute frequencies using doc values. This would also mean that
significant terms aggregations do not require fields to be indexed anymore.
2016-05-11 16:52:58 +02:00
Jason Tedor d0edd13f7b Log setting key not setting object in IMC
This commit modifies two logging statements in the
IndexingMemoryController to log the key for the setting
indices.memory.index_buffer_size instead of the object.

Relates #18191
2016-05-11 10:37:23 -04:00
Yannick Welsch ae01a4f7f9 Increased logging level for testDelayedAllocationChangeWithSettingTo100ms 2016-05-11 11:59:41 +02:00
Jason Tedor 5f0cc79562 Sort plugins in list plugins command
This commit modifies the list plugins command to produce deterministic
output by sorting the plugins by comparing paths.

Relates #18260
2016-05-10 22:06:37 -04:00
Yannick Welsch a0ffe6ea89 [TEST] Ensure creation of valid routing table
An additional sanity check introduced by #17821 makes some tests fail. This check verifies that
only one shard with same shard id is allocated to a node. This commit fixes a bug in
ClusterStateCreationUtils which would construct a cluster state that allocated two shards with same
id to the same node.
2016-05-10 19:40:19 +02:00
Yannick Welsch 7753420540 Make ShardRouting and UnassignedInfo immutable (#17821)
This makes defensive copying of ShardRouting objects obsolete whenever we do a reroute and trashes less objects.
2016-05-10 19:11:04 +02:00
Jason Tedor 81c0b7bfa9 Log exception when join validation fails
Today when join validation fails, we log a warning but do not log the
exception that led to the join validation failing. This commit modifies
this so that we do log this exception.
2016-05-10 11:56:57 -04:00
Yannick Welsch ad5ce598db Use uppercase 'L' for long literal 2016-05-10 17:32:21 +02:00
Gabriel Moskovicz 0660386976 Add plugin information for Verbose mode
Relates #18051
2016-05-10 11:23:17 -04:00
Lee Hinman a45e1cc750 Add test for NullPointerException in SQS when analyzing text produces null query 2016-05-10 08:35:48 -06:00
Lee Hinman 1c54033e92 Merge branch 'pr/18068' 2016-05-10 08:27:43 -06:00
Colin Goodheart-Smithe 319ca82510 Improving parsing of sigma param for Extended Stats Bucket Aggregatio
Improving parsing of sigma param for Extended Stats Bucket Aggregation
2016-05-10 15:11:02 +01:00
Alexander Kazakov 93d1b385a4 Improving parsing of sigma param for Extended Stats Bucket Aggregation #17499 2016-05-10 14:54:54 +03:00
Daniel Mitterdorfer 8972f39a9c Reenable CircuitBreakerServiceIT#testParentChecking() 2016-05-10 12:24:49 +02:00
Colin Goodheart-Smithe 17225d9ac1 Removes the now obsolete SearchParseElement implementations
All implementations of SearchParseElement have been removed since they are no longer used now that parsing is done on the coordinating node. The SearchParseElement and FetchSubPhaseParseElement classes are not removed as currently they are needed for plugins that add a custom fetch sub phase. These will be removed in a follow up PR that will allow fetch sub phase plugins to register a parser in a different way.
2016-05-10 10:06:54 +01:00
Adrien Grand b6556e275a Remove code duplication in FieldsVisitor. #18218
It currently has twice the same method, once with a MapperService instance and
once with a DocumentMapper. This commits only keeps the former.
2016-05-10 08:17:59 +02:00
Adrien Grand f481492af3 Remove FieldMapper.Builder.indexName. #18219
The ability to configure index names that are different from the full name was
removed in 2.0.
2016-05-10 08:17:00 +02:00
Adrien Grand 68e7ac4166 Remove old backward compatibility layer for version lookups. #18215
The current code tries to handle the case that document versions are either
missing or stored in payloads rather than doc values. Since none of the 2.x
releases allowed this, we can remove this logic.
2016-05-10 08:16:05 +02:00
Adrien Grand 5d8f684319 Mapping cleanups. #18180
This removes dead/duplicate code and makes the `_index` field not configurable.
(Configuration used to jus be ignored, now we would throw an exception if any
is provided.)
2016-05-10 08:14:18 +02:00
Jason Tedor eed5b0be4f Increaes timeout on Netty handshake tests
This commit increases the timeout on the Netty handshake tests because
the previous value could cause timeout exceptions on slow machines.
2016-05-09 19:02:54 -04:00
Johnny Lim 0970c59608 Use Strings.toString() for toString() in SearchResponse and GetResponse (#18102) 2016-05-09 16:56:02 -04:00
Jim Ferenczi aef78ceb13 Do not return fieldstats information for fields that exist in the mapping
but not in the index.
2016-05-09 19:24:03 +02:00
Daniel Mitterdorfer 0f36c744d0 Merge remote-tracking branch 'danielmitterdorfer/eager-content-length' 2016-05-09 15:10:01 +02:00
Daniel Mitterdorfer 3a46812a33 Free bytes reserved on request breaker
With this commit we free all bytes reserved on the request
circuit breaker.

Closes #18144
2016-05-09 15:07:41 +02:00
Daniel Mitterdorfer 000dd62318 Determine content length eagerly in HttpServer
With this commit we eagerly evaluate content length in HttpServer
and also pass the same value to ResourceHandlingHttpChannel. With
this change it easier to reason about the content length that is
freed leaving no doubt that it must be identical to the reserved
amount.
2016-05-09 11:11:59 +02:00
Ryan Ernst c740feb3a7 Mappings: Fix ip mapper to correctly serialize a null null_value
We recently added correct serialization for null values, but the helper
method used does not allow null. This fixes serialization to handle the
null.
2016-05-06 17:07:24 -07:00
Tal Levy c1afcb543e add check for non-existent pipelines provided to simulate requests (#18190)
fixes #18139
2016-05-06 13:34:22 -07:00
Jason Tedor d0d2d2be8c Fix exception message in lifecycle
This commit fixes the exception messages for lifecycles when stopping in
illegal states.

Relates #18189
2016-05-06 16:14:43 -04:00
Ryan Ernst 2c3d3d91cb Remove unnecessary nebula source or javadoc plugins 2016-05-06 13:09:48 -07:00
Ryan Ernst 3d1be071c9 Merge branch 'master' into pom_gen 2016-05-06 12:56:51 -07:00
Chris Earle 5be79ed02c Add Failure Details to every NodesResponse
Most of the current implementations of BaseNodesResponse (plural Nodes) ignore FailedNodeExceptions.

- This adds a helper function to do the grouping to TransportNodesAction
- Requires a non-null array of FailedNodeExceptions within the BaseNodesResponse constructor
- Reads/writes the array to output
- Also adds StreamInput and StreamOutput methods for generically reading and writing arrays
2016-05-06 14:59:43 -04:00
Chris Earle 1d5b9f1ce6 Test was not updated with #18187 2016-05-06 13:17:43 -04:00
Tanguy Leroux 0ff5652fff Add node name to Cat Recovery
closes #8041
2016-05-06 16:59:53 +02:00
Britta Weber 901c25b7ff Merge pull request #18183 from brwe/exclude-all-but-text-from-wildcard-highlighting
Exclude all but string fields from highlighting if wildcards are used…
2016-05-06 16:57:25 +02:00
Britta Weber f270ed26c3 fix highlighing for old version indices with string fields 2016-05-06 16:27:31 +02:00
Jason Tedor d52537dc7b Add semicolon query string parameter delimiter
This commit adds support for the semicolon character as a valid query
string parameter delimiter.

Relates #18186
2016-05-06 09:54:26 -04:00
Jason Tedor e90d00ffce Remove handshake from transport client
This commit removes handshaking from the transport client. This
handshaking is not needed because of the existence of the liveness
check.

Relates #18174
2016-05-06 09:17:18 -04:00
Nik Everett b6698c3145 Random script fields can't overlap
This causes round tripping through xcontent to fail.

Closes #18166
2016-05-06 09:01:09 -04:00
Alexander Reelsen b12a42351e Pipeline Stats: Fix concurrent modification exception (#18177)
Due to trying to modify a map while iterating it, a concurrent modification
in the pipeline stats could be thrown. This uses an iterator to prevent this.

Closes #18126
2016-05-06 15:00:41 +02:00
Britta Weber 7b69e4ef43 keyword fields should also be highlighted 2016-05-06 14:29:19 +02:00
Britta Weber d3c5f865be Exclude all but string fields from highlighting if wildcards are used in fieldname
We should prevent highlighting if a field is anything but a text or keyword field.
However, someone might implement a custom field type that has text and still want to
highlight on that. We cannot know in advance if the highlighter will be able to
highlight such a field and so we do the following:
If the field is only highlighted because the field matches a wildcard we assume
it was a mistake and do not process it.
If the field was explicitly given we assume that whoever issued the query knew
what they were doing and try to highlight anyway.

closes #17537
2016-05-06 13:41:16 +02:00
Adrien Grand 93567a2f1b Remove StringBuilder reuse for uid creation. #18181
This would be better handled by escape analysis.
2016-05-06 13:16:58 +02:00
Adrien Grand e88ac11633 Add back Version.V_5_0_0. #18176
This was lost whene releasing alpha2 since the version constant got renamed.
2016-05-06 12:30:22 +02:00
Christoph Büscher 7d14728960 Add xContent shuffling to some more tests
This adds some random shuffling of xContent to some more test cases.

Relates to #5831
2016-05-06 10:46:39 +02:00
Adrien Grand b91df36a62 Fix and test handling of `null_value`. #18090
This was mostly untested and had some bugs.

Closes #18085
2016-05-06 09:28:23 +02:00
Adrien Grand de8354dd7f Allow binary sort values. #17959
The `ip` field uses a binary representation internally. This breaks when
rendering sort values in search responses since elasticsearch tries to write a
binary byte[] as an utf8 json string. This commit extends the `DocValueFormat`
API in order to give fields a chance to choose how to render values.

Closes #6077
2016-05-06 09:27:02 +02:00
Adrien Grand 7d8708716e QueryBuilder does not need generics. #18133
QueryBuilder has generics, but those are never used: all call sites use
`QueryBuilder<?>`. Only `AbstractQueryBuilder` needs generics so that the base
class can contain a default implementation for setters that returns `this`.
2016-05-06 08:38:20 +02:00
Ryan Ernst e16af604bf Build: Add pom generation to assemble task
In preparation for a unified release process, we need to be able to
generate the pom files independently of trying to actually publish. This
change adds back the maven-publish plugin just for that purpose. The
nexus plugin still exists for now, so that we do not break snapshots,
but that can be removed at a later time once snapshots are happenign
through the unified tools. Note I also changed the dir jars are written
into so that all our artifacts are under build/distributions.
2016-05-05 17:57:44 -07:00
Jason Tedor 1199cd8e2a Mark IHBT#testFromAndToXContent as awaits fix
This commit marks InnerHitsBuilderTests#testFromAndToXContent as
awaiting a fix.
2016-05-05 20:16:57 -04:00
Robert Muir e3ce6c9048 Painless: add fielddata accessors (.value/.values/.distance()/etc)
This gives better coverage and consistency with the scripting APIs, by
whitelisting the primary search scripting API classes and using them instead
of only Map and List methods.

For example, accessing fields can now be done with `.value` instead of `.0`
because `getValue()` is whitelisted. For now, access to a document's fields in
this way (loads) are fast-pathed in the code, to avoid dynamic overhead.

Access to geo fields and geo distance functions is now supported.

TODO: date support (e.g. whitelist ReadableDateTime methods as a start)
TODO: improve docs (like expressions and groovy have for document's fields)
TODO: remove fast-path hack

Closes #18169

Squashed commit of the following:

commit ec9f24b2424891a7429bb4c0a03f9868cba0a213
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 17:59:37 2016 -0400

    cutover to <Def> instead of <Object> here

commit 9edb1550438acd209733bc36f0d2e0aecf190ecb
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 17:03:02 2016 -0400

    add fast-path for docvalues field loads

commit f8e38c0932fccc0cfa217516130ad61522e59fe5
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 16:47:31 2016 -0400

    Painless: add fielddata accessors (.value/.values/.distance()/etc)
2016-05-05 18:38:41 -04:00
Ali Beyad c4090a1841 Remove the Snapshot class in favor of using SnapshotInfo
o/e/snapshots/Snapshot and o/e/snapshots/SnapshotInfo contain the same
fields and represent the same information.  Snapshot was used to
maintain snapshot information to the snapshot repository, while
SnapshotInfo was used to represent the snapshot information as presented
through the REST layer.  This removes the Snapshot class and combines
all uses into the SnapshotInfo class.

Closes #18167
2016-05-05 16:53:13 -04:00
Nik Everett 4b1c116461 Generate and run tests from the docs
Adds infrastructure so `gradle :docs:check` will extract tests from
snippets in the documentation and execute the tests. This is included
in `gradle check` so it should happen on CI and during a normal build.

By default each `// AUTOSENSE` snippet creates a unique REST test. These
tests are executed in a random order and the cluster is wiped between
each one. If multiple snippets chain together into a test you can annotate
all snippets after the first with `// TEST[continued]` to have the
generated tests for both snippets joined.

Snippets marked as `// TESTRESPONSE` are checked against the response
of the last action.

See docs/README.asciidoc for lots more.

Closes #12583. That issue is about catching bugs in the docs during build.
This catches *some* bugs in the docs during build which is a good start.
2016-05-05 13:58:03 -04:00
Jason Tedor e11b96ca9c Default to server VM and add client VM check
Today we softly warn about running with the client VM. However, we
should really refuse to start in production mode if running with the
client VM as the performance of the client VM is too devastating for a
server application. This commit adds an option to jvm.options to ensure
that we are starting with the server VM (on all 32-bit non-Windows
platforms on server-class machines (2+ CPUs, 2+ GB physical RAM) this is
the default and on all 64-bit platforms this is the only option) and
adds a bootstrap check for the client VM.

Relates #18155
2016-05-05 10:36:21 -04:00
Jason Tedor 784c9e5fb9 Introduce node handshake
This commit introduces a handshake when initiating a light
connection. During this handshake, node information, cluster name, and
version are received from the target node of the connection. This
information can be used to immediately validate that the target node is
a member of the same cluster, and used to set the version on the
stream. This will allow us to extend APIs that are used during initial
cluster recovery without a major version change.

Relates #15971
2016-05-04 20:06:47 -04:00
Christoph Büscher 5a0cfdd6af Change scriptFields member in InnerHitBuilder to set
Adding random shuffling of xContent to InnterHitBuilderTests shows
that the scriptFields are stored in order as a list internally although
they are an unordered json objects in the query dsl.

This changes the internal representation to a set and updates
serialization accordingly.
2016-05-04 17:43:15 +02:00
Christoph Büscher 223d67df4a Consolidate query generation in QueryShardContext
Currently we have a lot of methods left in QueryShardContext that
take parsers or BytesReference arguments to do some xContent
parsing on the shard. While this still seems necessary in some cases
(e.g. percolation, phrase suggester), the shard context should only
be concerned with generating lucene queries from QueryBuilders.

This change removes all of the parseX() methods in favour of two
public methods toQuery(QueryBuilder) and toFilter(QueryBuilder) that
either call the query builders toFilter() or toQuery() method and
move all code required for parsing out to the respective callers.
2016-05-04 16:52:30 +02:00
Jim Ferenczi 8d3427b44d Fix build: restore illegalScorer still in use in ExpressionSearchScript (2) 2016-05-04 16:48:08 +02:00
Jim Ferenczi c4554fedad Fix build: restore illegalScorer still in use in ExpressionSearchScript 2016-05-04 16:46:44 +02:00
Jim Ferenczi 52eb6f3830 Merge pull request #18127 from jimferenczi/breadth_first_needs_score
Add the ability to use the breadth_first mode with nested aggregations (such as `top_hits`) which require access to score information.
2016-05-04 15:57:08 +02:00
Jason Tedor 78d615f320 Merge pull request #18110 from jasontedor/strings-split-as-array
Remove Strings#splitStringToArray

Remove arbitrary separator/wildcard from PathTrie
2016-05-04 09:38:47 -04:00
Jim Ferenczi 052191f2a2 Add the ability to use the breadth_first mode with nested aggregations (such as `top_hits`) which require access to score information.
The score is recomputed lazily for each document belonging to a top bucket.
Relates to #9825
2016-05-04 15:35:45 +02:00
Jason Tedor 9fe5ce9342 Remove arbitrary separator/wildcard from PathTrie
PathTrie has a constructor that allows for an arbitrary separtor and
wildcard, but this constructor is unused and internally we always use
'/' as the separator and '*' as the wildcard. There are no tests for the
case where the separator differs from the default separator and
wildcard. This commit removes this constructor and now all instances of
PathTrie have the default separator and wildcard.
2016-05-04 09:18:30 -04:00
Jason Tedor 2dea449949 Remove Strings#splitStringToArray
This commit removes the method Strings#splitStringToArray and replaces
the call sites with invocations to String#split. There are only two
explanations for the existence of this method. The first is that
String#split is slightly tricky in that it accepts a regular expression
rather than a character to split on. This means that if s is a string,
s.split(".")  does not split on the character '.', but rather splits on
the regular expression '.' which splits on every character (of course,
this is easily fixed by invoking s.split("\\.") instead). The second
possible explanation is that (again) String#split accepts a regular
expression. This means that there could be a performance concern
compared to just splitting on a single character. However, it turns out
that String#split has a fast path for the case of splitting on a single
character and microbenchmarks show that String#split has 1.5x--2x the
throughput of Strings#splitStringToArray. There is a slight behavior
difference between Strings#splitStringToArray and String#split: namely,
the former would return an empty array in cases when the input string
was null or empty but String#split will just NPE at the call site on
null and return a one-element array containing the empty string when the
input string is empty. There was only one place relying on this behavior
and the call site has been modified accordingly.
2016-05-04 08:12:41 -04:00
Isabel Drost-Fromm 283a6d27f1 Clean up merge commit changes 2016-05-04 13:34:11 +02:00
Isabel Drost-Fromm 3bc926b7e0 Merge branch 'master' into enhancement/switch_geodistancesortbuilder_to_geovalidationmethod 2016-05-04 13:16:40 +02:00
Isabel Drost-Fromm 6f15f35819 Remove left over references to ESTestCase 2016-05-04 12:14:39 +02:00
Isabel Drost-Fromm 6b9ac46402 Merge branch 'master' into enhancement/switch_geodistancesortbuilder_to_geovalidationmethod 2016-05-04 11:30:15 +02:00
Isabel Drost-Fromm ad8bf53bbd Fold helper class into abstract sort test class.
Folds the helper class for random object generation into the
abstract sort test class. Removes a few references to ESTestCase
that were not needed due to inheriting from it along the way.
2016-05-04 11:25:23 +02:00
Isabel Drost-Fromm a8bf75983f Merge branch 'master' into tests/switch_to_random_value_other_than_for_sort 2016-05-04 10:24:46 +02:00
Christoph Büscher ca21aa0cb5 Make reset() in QueryShardContext private
The query shard reset() method resets some internal state in the
query shard context, like clearing query names, the filter flag
or named queries. The problem with this method being public is
that it currently (miss?) used for modifying an existing context
for recursive invocatiob, but the contexts that have been reseted
that way cannot be properly set back to their previous state.

This PR is a step towards removing reset() entirely by first making
it only be used internally in QueryShardContext. In places where
reset() was used we can either create new QueryShardContexts or
modify the existing context because it is discarded afterwards anyway.
2016-05-03 18:56:16 +02:00
Jason Tedor 85d2fc0e38 Avoid string concatentation in IngestDocument.FieldPath
Today, the constructor for IngestDocument#FieldPath does a string
concatentation and two object allocation on every field path. This
commit removes these unnecessary operations.

Relates #18108
2016-05-03 10:05:44 -04:00
Yannick Welsch 98dba262cf Use object equality to compare versions in IndexSettings (#18103)
Fixes an issue where updating index metadata on a index from a version that it does not have in its static list of known versions fails.
2016-05-03 11:31:56 +02:00
Daniel Mitterdorfer 0a6f40c7f5 Enable HTTP compression by default with compression level 3
With this commit we compress HTTP responses provided the client
supports it (as indicated by the HTTP header 'Accept-Encoding').

We're also able to process compressed HTTP requests if needed.

The default compression level is lowered from 6 to 3 as benchmarks
have indicated that this reduces query latency with a negligible
increase in network traffic.

Closes #7309
2016-05-03 08:53:15 +02:00
Mike McCandless 0c6d6a5495 also compute completion stats for 2.x shards 2016-05-02 16:14:05 -04:00
Michael McCandless 16c4c251f5 Merge pull request #18094 from mikemccand/completion_stats_once
Don't try to compute completion stats on a reader after we already closed it

Conflicts:
	core/src/main/java/org/elasticsearch/index/shard/IndexShard.java
2016-05-02 14:49:15 -04:00
Jason Tedor 15f33e61b7 Kill redundant conditional in BootstrapCheck#check
This commit removes an unnecessary if statement in Bootstrap#check. The
removed if statement was duplicating the conditionals in the nested if
statements and was merely an artifact of an earlier refactoring.
2016-05-02 12:22:40 -04:00
Jason Tedor 226679c606 Mark MaxMapCountCheck#isSystemCheck as final
This commit marks the method MaxMapCountCheck#isSystemCheck as final as
this method should not be modifiable by inheriting classes (used in
tests).
2016-05-02 11:48:35 -04:00
Adrien Grand 8bdda49f90 Fail automatic string upgrade if the value of `index` is not recognized. #18082
Closes #18062
2016-05-02 17:28:43 +02:00
Jason Tedor 8e178c4f8e Add system bootstrap checks escape hatch
Today when running in production mode the bootstrap checks are
completely unforgiving. But there are cases where an end-user might not
have the ability to modify some of the system-level settings that cause
the bootstrap checks to trip (e.g., guest settings that are inherited
from a host and can not be modified). This commit adds a setting that
allows system-level bootstrap checks to be ignored for these
end-users. We classify certain bootstrap checks into system-level checks
and only those bootstrap checks will be ignored if this flag is
enabled. All other bootstrap checks are still subject to being enforced
if the user is in production mode. We will still log warnings for these
bootstrap checks because the end-user does still need to be made aware
that they are running in a configuration that is less-than-ideal from a
resiliency perspective.

Relates #18088
2016-05-02 10:59:50 -04:00
Isabel Drost-Fromm 372eceb854 Switch to using predicate for testing existing value 2016-05-02 15:41:05 +02:00
Isabel Drost-Fromm 4cf5385e4d Use plain random values for generating *SortBuilders 2016-05-02 13:50:00 +02:00
Isabel Drost-Fromm cf1d0d5935 Merge branch 'master' into tests/switch_to_random_value_other_than_for_sort 2016-05-02 11:38:40 +02:00
Jason Tedor 0b41580656 Remove unused import from ScalingThreadPoolTests 2016-04-29 16:21:59 -04:00
Jason Tedor c82e564886 Remove racy but unnecessary assertion
This commit removes a racy but unnecessary assertion in scaling thread
pool idle test. Namely, the main test thread can reach the removed
assertion before the last few threads in the thread pool have completed
their tasks and caused the completed tasks count on the underlying
executor to be updated. But this assertion is unnecessary. The main test
thread already waits on a latch that is only decremented immediately
before a task completes. This ensures that it was in fact the case that
every submitted task was executed.

Closes #18072
2016-04-29 16:21:22 -04:00
Christoph Büscher 262a814c8d Tests: Fix TermsQueryBuilderTests expectations when lookup returns no terms
When the termslookup (mocked in this case) doesn't return any terms, the
query used to rewrite to an empty boolean query. Now it rewrites to a
MatchNoDocsQuery. This changes the test expectation accordingly.

Closes #18071
2016-04-29 19:29:25 +02:00
Ali Beyad a31b94e237 When checking if an index tombstone can be applied, use both the index
name and uuid because the cluster state may contain an active index of
the same name (but different uuid).

Closes #18058
Closes #18054
2016-04-29 11:40:31 -04:00
Martijn van Groningen 7aca1389e2 ingest: Add `date_index_name` processor.
Closes #17814
2016-04-29 17:20:48 +02:00
Jason Tedor f349c4f135 Return null for "same" thread pool info
This commit modifes the EsThreadPoolTestCase#info helper method to
return null when info for the thread pool can not be found. This really
should only happen for the "same" thread pool, and so we also assert
that we only get to a place where there is no info if the thread pool
that info was requested for is in fact the "same" thread pool. Not
returning null here and instead throwing an exception would fail tests
that tried to lookup info on the "same" thread pool.
2016-04-29 10:29:02 -04:00
Martijn van Groningen 8c77399f1c test: changed test's expectation that a BooleanQuery is returned. A NoMatchDocsQuery is returned instead now. 2016-04-29 15:47:03 +02:00
David Pilato 2232a7cdf3 Merge branch 'pr/cat-size-time-units' 2016-04-29 15:09:14 +02:00
Jason Tedor c25f8ad912 Avoid sliced locked contention in internal engine
Today we use a sliced lock strategy for acquiring locks to prevent
concurrent updates to the same document. The number of sliced locks is
computed as a linear function of the number of logical
processors. Unfortunately, the probability of a collision against a
sliced lock is prone to the birthday problem and grows faster than
expected. In fact, the mathematics works out such that for a fixed
target probability of collision, the number of lock slices should grow
like the square of the number of logical processors. This is
less-than-ideal, and we can do better anyway. This commit introduces a
strategy for avoiding lock contention within the internal
engine. Ideally, we would only have lock contention if there were
concurrent updates to the same document. We can get close to this ideal
world by associating a lock with the ID of each document. This
association can be held in a concurrent hash map. Now, the JDK
ConcurrentHashMap also uses a sliced lock internally, but it has several
strategies for avoiding taking the locks and these locks are only held
for a very short period of time. This implementation associates a
reference count with the lock that is associated with a document ID and
automatically removes the document ID from the concurrent hash map when
the reference count reaches zero.

Relates #18060
2016-04-29 08:05:23 -04:00
Martijn van Groningen 6c3beaa2eb Drop top level inner hits in favour of inner hits defined in the query dsl.
Fix a limitation that prevent from hierarchical inner hits be defined in query dsl.

Removed the nested_path, parent_child_type and query options from inner hits dsl. These options are only set by ES
upon parsing the has_child, has_parent and nested queries are using their respective query builders.

These options are still used internally, when these options are set a new private copy is created based on the
provided InnerHitBuilder and configuring either nested_path or parent_child_type and the inner query of the query builder
being used.

Closes #11118
2016-04-29 11:17:24 +02:00
Jason Tedor 0cf1d16187 Fix line-length violation in SetProcessor
This commit addresses a checkstyle line-length violation in
SetProcessor.java.
2016-04-28 23:41:16 -04:00
Tal Levy 6302fb65a3 add ability to disable ability to override values of existing fields in set processor 2016-04-28 13:50:19 -07:00
Lee Hinman a0ba3c9a02 Merge remote-tracking branch 'dakrone/add-store-status-to-explain' 2016-04-28 13:47:55 -06:00
Lee Hinman cb29e041c4 Handle the case where `allocatedPostIndexCreate` is true 2016-04-28 13:46:06 -06:00
Jay Modi 34b5839487 Merge pull request #17949 from jaymode/list_setting_exists
fix exists method for list settings when using numbered setting format
2016-04-28 12:25:48 -04:00
Martijn van Groningen 22dc06fb7d removed duplicated code 2016-04-28 16:42:12 +02:00
Isabel Drost-Fromm a03b131319 Shorten nested test query builder generation. 2016-04-28 15:31:56 +02:00
Isabel Drost-Fromm 3490daa7ea Get rid of no longer needed missing generation method 2016-04-28 15:27:36 +02:00
Isabel Drost-Fromm 0c7e58de93 Get rid of duplicate random sort order generation 2016-04-28 15:14:15 +02:00
Isabel Drost-Fromm 8dc0610eaa Get rid of duplicated random ascii generation 2016-04-28 15:07:55 +02:00
Isabel Drost-Fromm 47fefdd273 Switch from separate sort_mode to more general randomValueOtherThan
... for sort tests only ...
2016-04-28 14:45:56 +02:00
Isabel Drost-Fromm 3ab006e422 Switch to new random value supplier for test. 2016-04-28 14:27:22 +02:00
Isabel Drost-Fromm a19c426e0f Deprecate coerce/ignore_malformed for GeoBoundingBoxQuery 2016-04-28 14:10:59 +02:00
Mathias Fußenegger a5ab648fae Handle indices=["_all"] when restoring a snapshot (#18025)
Setting indices=["_all"] when restoring a snapshot would throw an IndexNotFoundException instead of selecting all indices as in other APIs.
2016-04-28 14:10:01 +02:00
Isabel Drost-Fromm 3f743a30cf Deprecate coerce/ignore_malformed in GeoDistanceQueryBuilder 2016-04-28 14:06:27 +02:00
Isabel Drost-Fromm 3160798084 Deprecate coerce/ignore_malformed for GeoDistanceRangeQuery 2016-04-28 14:01:54 +02:00
Isabel Drost-Fromm 5306de3ce3 Deprecate coerce/ignore_malformed for GeoPolygonQueryBuilder
Includes update to parsing code, tests, migration docs and reference
docs.
2016-04-28 13:56:50 +02:00
Isabel Drost-Fromm 7bcc079d60 Adjust wording according to review. 2016-04-28 12:39:16 +02:00
Isabel Drost-Fromm c34ba6fe1e Merge branch 'master' into enhancement/switch_boolean_to_match_no_docs_query 2016-04-28 12:35:43 +02:00
Isabel Drost-Fromm 78ff4f52d6 Introduces GeoValidationMethod to GeoDistanceSortBuilder
Previously like in other geo related query parsers we were using
a combination of two booleans for coerce and ignore_malformed
which was error prone and not very clear.

Switched to using GeoValidationMethod instead as we already do
e.g. in GeoBoundingBoxQueryBuilder.

Left support for both, coerce and ignore_malformed in the parser
but deprecated the two in favour of validation method.

Introduced the same deprecation in geo bounding box query builder.
2016-04-28 12:10:56 +02:00
Adrien Grand de43bda3f3 iter 2016-04-28 09:39:08 +02:00
Adrien Grand b77893b1d1 Fail queries on not indexed fields.
While returning no hits on fields that are not mapped may be fine, it is not
for fields that are mapped but not indexed (`index:false`). We should fail the
query in that case rather than returning no hits.
2016-04-28 09:37:04 +02:00
Nik Everett 1df4efccd5 Wrap settings at 140 columns
and remove the checkstyle suppressions.
2016-04-27 18:18:20 -04:00
Nik Everett 6cccd4462c Wrap Version at 140 columns
and remove its checkstyle_suppression line
2016-04-27 17:25:37 -04:00
Nik Everett 47577613cb Wrap tasks code at 140 columns
Switch something from an explicit toString to Strings.toString which
is the same thing but with more code reuse.

Also renamed a constant to be CONSTANT_CASE.
2016-04-27 17:05:09 -04:00
Nik Everett 3468be0075 [test] Remove BulkProcessorRetryIT's generic threadpool setting
It wasn't required to excercise the retries and it caused stability issues.
2016-04-27 16:21:39 -04:00
Nik Everett d9afb8bfd6 [test] Wrap long lines in TasksIT 2016-04-27 15:32:56 -04:00
Nik Everett eb88fd1461 [test] TasksIT expected a request per copy
But instead the field stats API makes a request per primary.
2016-04-27 15:28:35 -04:00
Jim Ferenczi 9af2083765 Add XPointValues: forked utility methods from Lucene's PointValues until LUCENE-7257 is released.
Replace PointValues with XPointValues where needed.
Fixes #18010
2016-04-27 20:51:30 +02:00
Nik Everett 1feb9da3f2 ConstructingObjectParser is ObjectParser for ctors
ObjectParser makes parsing XContent 95% easier. No more nested loops.
No more forgetting to use ParseField. Consistent handling for arrays.
Awesome. But ObjectParser doesn't support building things objects whose
constructor arguments are mixed in with the rest of its properties.
Enter ConstructingObjectParser! ConstructingObjectParser queues up
fields until all of the constructor arguments have been parsed and
then sets them on the target object.

Closes #17352
2016-04-27 14:10:01 -04:00
Lee Hinman e636db1d7f Reorganize final decision, explanation, and store copy calculation, add tests 2016-04-27 10:43:08 -06:00
Jim Ferenczi 573c4f3ed1 Extend field stats:
* Add isSearchable and isAggregatable (collapsed to true if any of the instances of that field are searchable or aggregatable).
* Accept wildcards in field names.
* Add a section named conflicts for fields with the same name but with incompatible types (instead of throwing an exception).
2016-04-27 16:51:53 +02:00
Lee Hinman 1c0fc0a914 Use "_in_millis" for the suffix of delay times
Also change the javadoc to say "milliseconds" instead of "nanoseconds"
2016-04-27 08:35:33 -06:00
Nik Everett 4f9929d439 Javadoc: In Stream* scannability is priority 1! 2016-04-27 09:37:07 -04:00
Isabel Drost-Fromm 0e7f8316c0 Merge branch 'master' into enhancement/switch_boolean_to_match_no_docs_query 2016-04-27 15:17:54 +02:00
Isabel Drost-Fromm 6f8932a044 Switch to MatchNoDocsQuery with explicit reason string 2016-04-27 15:08:28 +02:00
Jason Tedor 0e824fab09 Fix test bug in scaling thread pool idle test
This commit fixes a test bug in the scaling thread pool idle
test. Namely, a random thread pool is chosen which could have a min pool
size of one or four but the while loop was acting as if the min pool
size was four (this is due to the test having been initially written for
only the generic thread pool).

Additionally, a latch is added between the test thread and the work
tasks to reduce the chance of a race condition between the test thread
and last few tasks.
2016-04-27 08:27:38 -04:00
Isabel Drost-Fromm 4c9070c52e Removing question concerning default no docs query. 2016-04-27 14:03:29 +02:00
Isabel Drost-Fromm 63eb81e7af Switches from empty boolean query to matchNoDocs
Closes #17981
2016-04-27 13:19:50 +02:00
Alexander Kazakov a8a33a1a94 Row-centric output for _cat/fielddata 2016-04-27 13:29:02 +03:00
Lee Hinman 93afac0776 Remove unneeded imports 2016-04-26 13:12:36 -06:00
Lee Hinman 2e447b960e Get tests working with the new redesign 2016-04-26 13:06:32 -06:00
Clinton Gormley 4a4e2dcace Updated the 2.3.2 bwc indices with those built from master 2016-04-26 21:03:38 +02:00
Lee Hinman 49c310691c Move remaining logic into Transport class
Also split NodeExplanation into its own class
2016-04-26 10:36:48 -06:00
Clinton Gormley 8a7dec2315 Added 2.3.2 and bwc indices for 2.3.2 2016-04-26 15:04:01 +02:00
Jason Tedor efeec4d096 Merge pull request #17017 from jasontedor/generic-thread-pool
Actually bound the generic thread pool
2016-04-26 08:27:48 -04:00
Jason Tedor b89a935be5 Expand scaling thread pool configuration coverage
This commit slightly expands the scaling thread pool configuration test
coverage. In particular, the test testScalingThreadPoolConfiguration is
expanded to include the case when min is equal to size, and the test
testDynamicThreadPoolSize is expanded to include all possible cases when
size is greater than or equal to min.
2016-04-26 08:17:54 -04:00
Jason Tedor e3126dfac0 Fix index name equality check in RoutingNodes
This commit fixes an index name equality check in RoutingNodes. Namely,
the check was comparing an instance of Index to an instance of
String. Instead, the index name should be obtained from the Index
instance to be compared to the instance of String.

Closes #17982
2016-04-26 07:45:30 -04:00
Jim Ferenczi 543b27f4cd Merge pull request #17780 from jimferenczi/match_nodocs_query
Add MatchNoDocsQuery, a query that matches no documents and prints the reason why in the toString method.
2016-04-26 12:20:12 +02:00
Alexander Reelsen f71eb0b888 Version: Set version to 5.0.0-alpha2 2016-04-26 09:30:26 +02:00
Adrien Grand 31a9845bc2 Remove the `SearchType` setter on `SearchContext`. #17955
It was not used.
2016-04-26 09:08:37 +02:00
Areek Zillur 4a1a03428d Add bwc support for pre-5.0 completion index
This commit adds support for reading and querying
completion fields that were indexed in 2.x
2016-04-25 21:21:56 -04:00
Ali Beyad d39eb2d691 Adds tombstones to cluster state for index deletions
Previously, we would determine index deletes in the cluster state by
comparing the index metadatas between the current cluster state and the
previous cluster state and decipher which ones were missing (the missing
ones are deleted indices).  This led to a situation where a node that
went offline and rejoined the cluster could potentially cause dangling
indices to be imported which should have been deleted, because when a node
rejoins, its previous cluster state does not contain reliable state.

This commit introduces the notion of index tombstones in the cluster
state, where we are explicit about which indices have been deleted.
In the case where the previous cluster state is not useful for index
metadata comparisons, a node now determines which indices are to be
deleted based on these tombstones in the cluster state.  There is also
functionality to purge the tombstones after exceeding a certain amount.

Closes #17265
Closes #16358
Closes #17435
2016-04-25 15:43:20 -04:00
Lee Hinman 08ac66f41e Change corruption explanation 2016-04-25 12:41:10 -06:00
Lee Hinman 89c25c5882 Update tests 2016-04-25 12:25:06 -06:00
Lee Hinman f4d688a2c4 Rewrite ClusterAllocationExplanation to be more object-y 2016-04-25 11:44:30 -06:00
Lee Hinman 1987107d75 Add the shard's store status to the explain API
This adds information similar to what is from the [shard stores
API](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/indices-shards-stores.html)
to the cluster allocation explanation API (in fact, internally it uses
that API).

This means when you have a decision that otherwise could indicate that a
shard can go somewhere, you now have more information:

```json
{
  "shard" : {
    "index" : "i",
    "index_uuid" : "QzoKda9aQCG_hCaZQ18GEg",
    "id" : 0,
    "primary" : true
  },
  "assigned" : false,
  "unassigned_info" : {
    "reason" : "CLUSTER_RECOVERED",
    "at" : "2016-04-11T20:58:04.088Z"
  },
  "allocation_delay" : "0s",
  "allocation_delay_ms" : 0,
  "remaining_delay" : "0s",
  "remaining_delay_ms" : 0,
  "nodes" : {
    "24Qmw4tdRTuVOtjAdtmr5Q" : {
      "node_name" : "Vampire by Night",
      "node_attributes" : { },
      "final_decision" : "YES",
      "weight" : 7.0,
      "decisions" : [ ],
      "store" : {
        "allocation_id" : "aC6qVWA7TT2pgsalYxxUJQ",
        "store_exception" : "IndexFormatTooOldException[Format version is not supported (resource BufferedChecksumIndexInput(SimpleFSIndexInput(path=\"/home/hinmanm/scratch/elasticsearch-5.0.0-alpha1-SNAPSHOT/data/elasticsearch/nodes/0/indices/QzoKda9aQCG_hCaZQ18GEg/0/index/segments_1\"))): -1906795950 (needs to be between 1071082519 and 1071082519). This version of Lucene only supports indexes created with release 5.0 and later.]",
        "allocation" : "UNUSED"
      }
    }
  }
}
```

The "store" section is the new section, and will include allocation, id,
and the exception if there is one.

Relates to #17372
2016-04-25 09:36:38 -06:00
jaymode e70420ef2b convert settings for ResourceWatcherService to new infrastructure
This commit converts the settings for the ResourceWatcherService to use the new infrastructure and
registers the settings so that they do not cause errors when used.
2016-04-25 07:07:14 -04:00
Jason Tedor fd679a7021 Test min pool size of zero for scaling executor
This commit expands the configuration test for scaling executors to
include the case where the min pool size is set to zero.
2016-04-25 06:47:27 -04:00
Jason Tedor e6a06b272e Clarify message on resize scaling executor queues
This commit clarifies an error message that is produced when an attempt
is made to resize the backing queue for a scaling executor. As this
queue is unbounded, resizing the backing queue does not make sense. The
clarification here is to specify that this restriction is because the
executor is a scaling executor.
2016-04-25 06:47:27 -04:00
Jason Tedor d032de2df2 Cleanup o/e/c/u/c/EsExecutors.java
This commit removes two unused imports and applies a few other
formatting cleanups to EsExecutors.java.
2016-04-25 06:47:26 -04:00
Jason Tedor 5608fa7ac1 Actually bound the generic thread pool
This commit actually bounds the size of the generic thread pool. The
generic thread pool was of type cached, a thread pool with an unbounded
number of workers and an unbounded work queue. With this commit, the
generic thread pool is now of type scaling. As such, the cached thread
pool type has been removed. By default, the generic thread pool is
constructed with a core pool size of four, a max pool size of 128 and
idle workers can be reaped after a keep-alive time of thirty seconds
expires. The work queue for this thread pool remains unbounded.
2016-04-25 06:47:26 -04:00
jaymode 2606066e30 fix exists method for list settings when using numbered setting format
The list settings parser supports retrieving lists defined in settings that use a key followed by a `.` and a
number (for example `foo.bar.0`). However, the exists method would indicate that the provided settings
do not contain a value for this setting. This change makes it so that the exists method now handles this
format.
2016-04-24 20:36:58 -04:00
Ryan Ernst d56d8b03c8 Merge pull request #17938 from rjernst/plugin_command_help
Cli: Improve output for usage errors
2016-04-22 14:58:13 -07:00
Ryan Ernst 49c47b24c1 Cli: Improve output for usage errors
When a cli throws a USAGE error, it is implied that the user did
something wrong, and probably needs help in understanding the cli
arguments. This change adds help output before the usage error is
printed.
2016-04-22 14:09:35 -07:00
Ryan Ernst d12a4bb51d Merge pull request #17933 from rjernst/camelcase4
Remove camelCase support
2016-04-22 13:46:43 -07:00
Nik Everett cc1a55423c Reindex: properly mark things as child tasks
Do this by creating a Client subclass that automatically assigns the
parentTask to all requests that come through it. Code that doesn't want
to set the parentTask can call `unwrap` on the Client to get the inner
client instance that doesn't set the parentTask. Reindex uses this for
its ClearScrollRequest so that the request will run properly after the
reindex request has been canceled.
2016-04-22 14:00:11 -04:00
Ryan Ernst d4b3ff983f Merge pull request #17662 from rjernst/verbose_install
Plugin cli: Add verbose output with zip url when installing plugin
2016-04-22 09:26:30 -07:00
Ryan Ernst 55388590c1 Remove camelCase support
Now that the current uses of magical camelCase support have been
deprecated, we can remove these in master (sans remaining issues like
BulkRequest). This change removes camel case support from ParseField,
query types, analysis, and settings lookup.

see #8988
2016-04-22 09:18:10 -07:00
Nik Everett 531bdbfdbc createTask can return null
That is totally allowed.
2016-04-22 11:26:19 -04:00
Nik Everett cdf4778483 Fix javadoc 2016-04-22 11:26:19 -04:00
Nik Everett 51621f9d75 Remove ChildTaskRequest and always pass parentTaskId when building a task
Passing parentTaskId forces the caller to handle the parentTaskId.
2016-04-22 11:26:18 -04:00
Nik Everett ffeb5e38fc Remove parent-less task methods
Callers should explicitly handle parents - either using EMPTY_TASK_ID when
a parent isn't possible or piping parents from the TransportRequest when
possible.
2016-04-22 11:26:18 -04:00
Nik Everett 2b56a42b69 Move parentTaskId into TransportRequest
Now everything can have a parent!
2016-04-22 11:26:18 -04:00
Nik Everett 61f0b665b8 Fix fallback setting for two get/2 2016-04-22 11:10:01 -04:00
Nik Everett 1c2e84ba46 Fail request if rescore window > 10,000
The setting is named `index.max_rescore_window` and defaults to
`index.max_result_window` which defaults to 10,000.
2016-04-22 11:10:01 -04:00
Jason Tedor 21b1da1bea Kill thread local leak
This commit modifies InjectorImpl to prevent a thread local leak in
certain web containers. This leak can arise when starting a node client
inside such a web container. The underlying issue is that the
ThreadLocal instance was created via an anonymous class. Such an
anonymous class has an implicit reference back to the InjectorImpl in
which it was created. The solution here is to not use an anonymous class
but instead just create the reference locally and set it on the thread
local.

Relates #17921
2016-04-22 08:05:33 -04:00
Martijn van Groningen c5ad2e2865 Changed indexed scripts to be stored in the cluster state instead of the `.scripts` index.
Also added max script size soft limit for stored scripts.

Closes #16651
2016-04-22 13:42:55 +02:00
Daniel Mitterdorfer ae61fc4780 Increase number of concurrent requests in NettyHttpRequestSizeLimitIT
For some seeds the number of concurrent requests previously defined
in NettyHttpRequestSizeLimitIT was too low to trigger the intended
breaker limit.

With this commit we increase the number of concurrent requests that
are sent to the test cluster thus triggering the limit.
2016-04-22 13:28:12 +02:00
Adrien Grand 370af45c09 Add back range support to `ip` fields. #17777
`ip` fields currently fail range queries when either bound is inclusive. This
commit makes ranges also work in the exclusive case to be consistent with other
data types.
2016-04-22 09:58:01 +02:00
Isabel Drost-Fromm 1939425378 Merge pull request #17908 from MaineC/enhancement/make-script-writeable
Makes Script type writeable
2016-04-22 08:39:01 +02:00
Isabel Drost-Fromm 36a8f7b7bb Address stylistic comments. 2016-04-21 22:02:17 +02:00
Alexander Kazakov 3046055bd0 Fix calculation of time of term vectors request
This commit addresses an issue in the calculation of the time to execute
a term vectors request. The underlying issue was due to measuring the
took time by passing the starting wall clock time along with the request
and calculating the total time using the ending wall clock time on the
responding node. The fix is to use a relative time source on a single
node.


Relates #17817
2016-04-21 15:46:14 -04:00
Isabel Drost-Fromm ee6e53b581 Merge branch 'master' into enhancement/make-script-writeable 2016-04-21 21:28:55 +02:00
Lee Hinman 2539d94bc9 Merge remote-tracking branch 'dakrone/vagrant-umask' 2016-04-21 12:31:07 -06:00
Lee Hinman 4fca5f734a Explicitly set packaging permissions
This changes our packaging to be explicit about the permissions of files
and directories in the tar.gz, rpm, and deb packages. This is to protect
against a user having an incorrectly set umask when installing.

Additionally, plugins that are installed now have their permissions set
by the plugin installation so that plugins that may have been packaged
with incorrect permissions are secured.

Resolves #17634
2016-04-21 12:30:56 -06:00
Nik Everett 9511c269c6 Remove <T> from Writeable
It isn't needed any more! Hurray!

Closes #17085
2016-04-21 11:14:00 -04:00