Commit Graph

4263 Commits

Author SHA1 Message Date
Simon Willnauer 9ce66cb167 [TEST] Testcase for #6639 2014-06-28 09:12:25 +02:00
Simon Willnauer b2685f132a [TEST] Change es.node.mode default for tests to `local`
In order to speed up test execution we should run in local mode by
default. CI builds will still use network builds all the time.

Closes #6624
2014-06-27 11:57:34 +02:00
Simon Willnauer f0cfdc444f [STORE] Wrap RateLimiter rather than copy RateLimitedIndexOutput
We clone RateLimitedIndexOutput from lucene just to collect pausing
statistics we can do this in a more straight forward way in a delegating
RateLimiter.

Closes #6625
2014-06-27 11:35:13 +02:00
Shay Banon 79af3228ad Thread pool rejection status code should be 429
Thread rejection should return too many requests status code, and not 503, which is used to also show that the cluster is not available
 relates to #6627, but only for rejections for now
closes #6629
2014-06-27 11:15:16 +02:00
Shay Banon 4129bb6a4f Make sure we don't reuse arrays when sending and error back
We want to make sure recycling will not fail for any reason while trying to send a response back that is caused by a failure, for example, if we have circuit breaker on it (at one point), sending an error back will not be affected by it.
closes #6631
2014-06-27 11:12:35 +02:00
Shay Banon e559295228 [TEST] when the test fail, have the exception message as the reason
the test failed but couldn't repro (yet), at the very least, make sure we have the exception message as the reason, can help to track down the failure itself when it happens again
2014-06-27 09:16:51 +02:00
Simon Willnauer f7da6da73a [TEST] suppress sysout checks since CI runs with debug enabled 2014-06-26 19:10:20 +02:00
Robert Muir b55ad98d73 Upgrade to Lucene 4.9 (closes #6623) 2014-06-26 08:18:59 -04:00
Lee Hinman b43b56a6a8 Add a transformer to translate constant BigDecimal to double 2014-06-26 10:52:28 +02:00
Lee Hinman 50bb274efa Remove MVEL as a built-in scripting language 2014-06-26 10:33:28 +02:00
Boaz Leskes 2c2783875e Be more diligent about ThreadPools having names
Add a name parameter to what was the empty ThreadPool constructor. Assert if the the ThreadPool's setting doesn't contain a name.
2014-06-26 10:01:22 +02:00
Clinton Gormley 30c80319c0 Match query with operator and, cutoff_frequency and stacked tokens
If the match query with cutoff_frequency encounters stacked tokens,
like synonyms in the same position, it returns a boolean query instead
of a common terms query.  However, if the original operator was set
to "and", it was ignoring that and resetting the operator to "or".

In fact, if operator is "and" then there is little benefit in using
a common terms query as a must query is already
executed efficiently.
2014-06-25 17:53:43 +02:00
Andrew Raines 534b07a3fb [TEST] Add assertion failure messages 2014-06-25 16:22:20 +02:00
Lee Hinman 5c6d28240f Switch to Groovy as the default scripting language
This is a breaking change to move from MVEL -> Groovy
2014-06-25 12:15:12 +02:00
Lee Hinman 47856ec4cd Add sandboxing for GString-based method invocation 2014-06-25 12:09:32 +02:00
Shay Banon 342563a864 [LOG] better log message 2014-06-25 11:01:20 +02:00
Alexander Reelsen fd9744968f Internal: Made base64 decode parsing to detect more errors
The base64 did not completely check, if there were other characters
after the equals `=` sign. This PR adds some small additional checks.

Closes #6334
2014-06-24 13:01:11 +02:00
Martijn van Groningen e12025f749 [TEST] Improved logging for replica operation failures 2014-06-23 09:28:41 +02:00
Boaz Leskes 3d6d2e700a [Test] testGetFields_complexField should wait for a green cluster
Waiting for ongoing recoveries was not good enough as it can run before the  master finishing processing the started events of primary shards, causing the recovery response to be erroneously empty
2014-06-21 20:15:13 +02:00
Shay Banon 0e83615496 [Test] Use no failures, shard might not have been initialize yet 2014-06-21 12:43:14 +02:00
Boaz Leskes 08ca51d7b6 [TEST] fix a NPE in verifyThreadNames which may happen if thread finishes during sampling 2014-06-21 10:31:45 +02:00
Shay Banon 0d66d3779e Fix optional default script loading
Groovy is optional as a dependency in the classpath, make sure we properly detect when its not at the right time to disable it
closes #6582
2014-06-21 00:27:15 +02:00
Martijn van Groningen 812972ab0e [TEST] Move the waiting for pending tasks to helper methods and let the percolator and update mapping test use these helper methods. 2014-06-20 23:44:33 +02:00
Martijn van Groningen 11251bca92 [TEST] Verify that all pending tasks are rely executed on *all* nodes. 2014-06-20 23:12:52 +02:00
Martijn van Groningen 73e4a9b3f7 Fixed NPE in recovery api by serializing the recovery type in StartRecoveryRequest.
Closes #6190
2014-06-20 22:09:46 +02:00
javanna f16451a446 Refactored AckedClusterStateUpdateTask & co. to remove code repetitions in subclasses
Made AckedClusterStateUpdateTask an abstract class instead of an interface, which contains the common methods.
Also introduced the AckedRequest interface to mark both AcknowledgedRequest & ClusterStateUpdateRequest so that the different ways of updating the cluster state (with or without a MetaData*Service) can share the same code.
Removed ClusterStateUpdateListener as we can just use its base class ActionListener instead.

Closes #6559
2014-06-20 20:14:40 +02:00
Lee Hinman 2708e453ac Re-shade MVEL as a dependency 2014-06-20 11:28:50 +02:00
Lee Hinman c70f6d0171 Add Groovy as a scripting language, add sandboxing for Groovy
Sandboxes the groovy scripting language with multiple configurable
whitelists:

`script.groovy.sandbox.receiver_whitelist`: comma-separated list of string
classes for objects that may have methods invoked.
`script.groovy.sandbox.package_whitelist`: comma-separated list of
packages under which new objects may be constructed.
`script.groovy.sandbox.class_whitelist` comma-separated list of classes
that are allowed to be constructed.

As well as a method blacklist:

`script.groovy.sandbox.method_blacklist`: comma-separated list of
methods that are never allowed to be invoked, regardless of target
object.

The sandbox can be entirely disabled by setting:

`script.groovy.sandbox.enabled: false`
2014-06-20 10:20:16 +02:00
javanna 12fd6ce98c REST api: made it possible to copy the REST headers from REST requests to the corresponding TransportRequest(s)
Introduced the use of the FilterClient in all of the REST actions, which delegates all of the operations to the internal Client, but makes sure that the headers are properly copied if needed from REST requests to TransportRequest(s) when it comes to executing them.

Added new abstract handleRequest method to BaseRestHandler with additional Client argument and made private the client instance member (was protected before) to force the use of the client received as argument.

The list of headers to be copied over is by default empty but can be extended via plugins.

Closes #6513
2014-06-19 18:45:21 +02:00
javanna 8f8b2d7979 Client intermediate interface removal follow-up
After #6517 we ended up registering all of the actions (included admin ones) to the NodeClient.
Made sure that only the proper type of Action instances are registered to each client type.
Also fixed some compiler warnings: unused members, imports and non matching generic types.

Closes #6563
2014-06-19 17:55:25 +02:00
Adrien Grand 8ccfca3a2f Fielddata: Remove BytesValues.WithOrdinals.currentOrd and copyShared.
These methods don't exist in Lucene's sorted set doc values.

Relates to #6524
2014-06-19 12:06:40 +02:00
Adrien Grand 9e624942d8 Fielddata: Move `getTermsEnum` from `AtomicFieldData` to `BytesValues.WithOrdinals`.
Similarly to `SortedSetDocValues.termsEnum()`.

Relates to #6524
2014-06-19 12:01:30 +02:00
Adrien Grand 9b02b5061b Fielddata: Merge ordinals APIs into BytesValues.WithOrdinals.
Mid-term we should switch from `BytesValues` to Lucene's doc values APIs, in
particular the `SortedSetDocValues` class. While `BytesValues.WithOrdinals` and
SortedSetDocValues expose the same functionality, `BytesValues.WithOrdinals`
exposes its ordinals via a different `Ordinals.Docs` object while
`SortedSetDocValues` exposes them on the same object as the one that holds the
values. This commit merges ordinals into `BytesValues.WithOrdinals` in order to
make both classes even closer.

Global ordinals were a bit tricky to migrate so I just changed them to use
Lucene's OrdinalMap that will soon (LUCENE-5767, scheduled for 4.9) have the
same optimizations as our global ordinals.

Close #6524
2014-06-19 12:00:51 +02:00
Adrien Grand 703dbff83d Index field names of documents.
The `exists` and `missing` filters need to merge postings lists of all existing
terms, which can be very costly, especially on high-cardinality fields. This
commit indexes the field names of a document under `_field_names` and reuses it
to speed up the `exists` and `missing` filters.

This is only enabled for indices that are created on or after Elasticsearch
1.3.0.

Close #5659
2014-06-19 11:50:06 +02:00
Adrien Grand e2da2114e7 Mappings: Allow _version to use `disk` as a doc values format.
VersionFieldMapper.defaultDocValuesFormat claims that the default is `disk`.
This is not used to choose the DV format in the index but for mappings
serialization in order to know when the _version doc values format is
different from the default format. This made it impossible to use the `disk`
doc values format since mappings would never retain that information at
serialization time.

Close #6523
2014-06-19 11:39:17 +02:00
Boaz Leskes 5b919d4e4f [TEST] Added (trace) logging to testGetFields_complexField 2014-06-19 11:10:13 +02:00
javanna 2024067465 Java API: BulkRequest#add(Iterable) to support UpdateRequests
Closes #6551
2014-06-19 10:43:05 +02:00
Alexander Reelsen 9569166f94 Mapping: Fix possibility of losing meta configuration on field mapping update
The TTL, size, timestamp and index meta properties could be lost on an
update of a single field mapping due to a wrong comparison in the
merge method (which was caused by a wrong initialization, which marked
an update as explicitely disabled instead of unset.

Closes #5053
2014-06-19 08:39:26 +02:00
Fitblip d18fb8bfbd REST API: Allow to configure JSONP/callback support
Added the http.jsonp.enable option to configure disabling of JSONP responses, as those
might pose a security risk, and can be disabled if unused.

This also fixes bugs in NettyHttpChannel
* JSONP responses were never setting application/javascript as the content-type
* The content-type and content-length headers were being overwritten even if they were set before

Closes #6164
2014-06-19 08:34:38 +02:00
Britta Weber 1cbeaf6c45 significant terms: fix json response
Commit fbd7c9aa5d introduced a regression that caused
the min_doc_count to be equal to the number of documents in the
background set. As a result no buckets were built when the
response for significant terms was created.
This only affected the final XContent response.

closes #6535
2014-06-18 18:51:34 +02:00
javanna a499254566 Fixed typo in TransportAction log line 2014-06-18 17:37:45 +02:00
Boaz Leskes 1114835de5 Also send Refresh and Flush actions to relocation targets
Currently we send relocation & flush actions based on all assigned ShardRoutings. During the final stage of relocation, we may miss to refresh/flush a shard if the coordinating node has not yet processed the cluster state update indicating that a relocation is completed *and* the relocation target node has already processed it (i.e., started the shard and has accepted new indexing requests).

Closes #6545
2014-06-18 16:12:42 +02:00
Martijn van Groningen 68046b64c2 [TEST] Only use the clients from the node. 2014-06-18 16:06:09 +02:00
Martijn van Groningen cb97b79396 [TEST] Disable automatic refresh to prevent unintended field data warming of the the _parent field. 2014-06-18 16:06:09 +02:00
Colin Goodheart-Smithe 7423ce0560 Aggregations: Added percentile rank aggregation
Percentile Rank Aggregation is the reverse of the Percetiles aggregation.  It determines the percentile rank (the proportion of values less than a given value) of the provided array of values.

Closes #6386
2014-06-18 12:02:08 +01:00
Britta Weber fe89ea1ecf percolator: fix handling of nested documents
Nested documents were indexed as separate documents, but it was never checked
if the hits represent nested documents or not. Therefore, nested objects could
match not nested queries and nested queries could also match not nested documents.

Examples are in issue #6540 .

closes #6540
closes #6544
2014-06-18 12:24:47 +02:00
Martijn van Groningen cb9548f811 Changed the type of field docCounts to IntArray instead of LongArray, because a shard can't hold more than Integer.MAX_VALUE a LongArray just takes unnecessary space.
Closes #6529
2014-06-17 13:35:05 +02:00
Simon Willnauer adb5c19849 [CLIENT] Remove unnecessary intermediate interfaces
Client, ClusterAdminClient and IndicesAdminClient had corresponding
intermediate `internal` interfaces that are unnecessary and cause
a lot of casting. This commit removes the intermediate interfaces
and uses the super interfaces directly.

This commit also adds Releaseable to `Node` and `Client` in order to
be used with utilities like try / with.

Closes #4355
Closes #6517
2014-06-17 12:18:37 +02:00
Simon Willnauer e198c58a6b [TEST] Use test.bwc.version if compatibility version is not present 2014-06-17 12:16:10 +02:00
Adrien Grand a06fd46a72 [Benchmark] Fix TermsAggregationSearchBenchmark: The `ordinals` execution mode doesn't exist anymore. 2014-06-17 01:46:02 +02:00